Exclusive Insights: How a Small Decentralized Team Responded to a +$50m Hack

By November 22, 2021 No Comments

Introduction

Potentially, one of the most important snippets of information you learn in cryptocurrency is this:

There are two types of exchanges—those that have been hacked and those that will be hacked.

It is synonymous with “Not your keys, not your crypto”, but nobody tells you it often extends to everything to do with cryptocurrency. 

Your best defense is teaming up with the strongest coders. Building powerful communities. And putting in a ton of effort and development time to constantly re-enforce your project.

But sometimes that’s not even enough. This is the true story of how the Haven protocol survived, and thrived from a +$50-million hack this summer. I’m going to share details of what it was like from the inside.

The Haven Protocol

Our story begins with two smart developers recognizing a hole in the cryptocurrency market and deciding to fill it. 

This hole being the inherent clash between price stability for crypto-assets and privacy for crypto-assets. 

As you may have realized, you can have one or the other, but not both at the same time. For example, you can’t track Monero’s transactions along the blockchain, but by the same token, you can’t be sure that the $50 worth you buy today will be worth that much tomorrow. It may be more, or it may be less.

The opposite is true for Stablecoin like USDC—which you are certain (relatively speaking) that it will remain at the $1 price point day-after-day. The challenge here is that anyone from the government, to scammers, to your nosy neighbours, can review your transactions.

How then do you find a combination of both?

This is where the idea of private finance (PriFi) comes in – and is exactly what the Haven Protocol is all about. 

Development on the project, just like any other, hasn’t been a straight line “up and to the right”. We’ve had our fair share of challenges. But growth in the project really started picking up over time, as the community got stronger, and our core development team started to really push forward.

Everything was coming up. Until it wasn’t. And despite the progress that had been made, nobody saw it coming.

June 22, 2021

The attack on the Haven network begins. 

It’s sophisticated, well-planned and aggressive. So much in fact, that our team doesn’t detect it until 5 a.m. the next day. 

A hacker (or hackers) have modified the miner-reward validation code, taking advantage of a previously undetected vulnerability. They’ve gotten away with a much higher mining award than should have been given. 

The problem was easy to spot because miner transactions are public. There were two places in the chain where our core team of developers replicated the exploit and were able to structure a patch.

Working non-stop, they had the patch out to the majority of the nodes on the network by 1:30 p.m. the same day the hack was discovered.

But the hackers weren’t done, and our core team was back in the ‘meeting room’ within a week. A couple of days later our team spotted two additional suspicious transactions in the explorer, wherein they found another vulnerability we had never discovered before. This one was more sinister and impactful than the first.

The hackers minted thousands of counterfeit xBTC coins and xAssets, which correlated with an unusually high selling volume of XHV on KuCoin.

Not only were tens of millions of dollars at stake, but this directly threatened the tokenomics of the protocol. If the team hadn’t seen it when they did, the hackers could have minted enough xAssets and cashed out to BTC or ETH to crater the value of XHV. By flooding the market with assets, the price could have approached zero, and the protocol would have been destroyed.

There would be no coming back from that. 

The Severity Of The Situation

It is sometimes difficult to comprehend the sacrifices of the people behind cryptocurrency projects. Decentralization, sudo-anonymous profiles, and interesting projects often lead us to believe that there is a “magical team” or “big company” with “millions of dollars” behind a project.

But this is not often the case.

The Haven team has grown from a grass-roots movement. Focused around the idea of private finance (PriFi), individuals have gathered from all walks of life to support and contribute to the project.

People have literally given up jobs, executive positions at companies, have put families on hold, and have sacrificed businesses and opportunity to push an idea forward. This idea, the future of private finance (PriFi) is, we believe, bigger than any one individual. Yes, while difficult, the sacrifices are worth it.

But when a hacker (or hackers) threaten to take it all down in the blink of an eye, the impact ripples through the community, and you realize that everything you’ve been fighting for could disappear in an instant. This is an extremely difficult position to be in.

It is made even more difficult when the hacker shows up in your community, taunting and harassing the community while spouting false claims that “the hack wasn’t financially motivated”, which was clearly not the case.

It is only a strong vision, strong community, and dedicated individuals who hold it all together in the face of adversity. 

Looking it dead in the eye, no one on the team flinched. We all knew that the response had to be meticulously calculated and executed. So our team and community banded together, and got to work.

The Response

First our team spread the word about the hack to our exchange partners. This included KuCoin, TradeOgre, and Bittrex. We requested they close all XHV wallets to keep the attacker from depositing and selling their ill-gotten XHV minted from the counterfeit assets. 

Understanding that we might need to take more drastic measures, we shut down the pricing record mechanism of the protocol temporarily, which determines how much xUSD or xAssets are minted in each conversion.

As the hacker taunted the community and the fixes were being implemented, it became evident that one more hard decision needed to be made. For this we approached the community and presented a vote.

A massively difficult decision, the question was proposed to the community – do we roll back the blockchain or keep on progressing with the hacker in our midst, and an existential threat to the idea of private finance.

Rollback

Typically, rolling back a blockchain is a contentious issue. It means invalidating any transactions you may have made. It means coins you’ve uncovered through mining go away. Gains that have been made from transactions, are completely invalidated and wiped out. 

It’s not a decision that’s made lightly. 

In this case, it also meant the Haven Protocol could reverse transactions that happened during the attack, eliminating or at least mitigating the damage that was mounting. 

The community was in favor of the rollback, and voted it in with overwhelming support.

While damage was done, this finalized a fix to a problem that had put us in an existential crisis and allowed us to move forward with lessons learned and a stronger base. Here’s what was done to strengthen the protocol even further.

The Way Forward

The Haven Protocol has transitioned from unhacked to hacked and come out the other end with a far improved protocol based on everything that was learned over the course of the hack.

Our core team improved efficiencies in the Haven codebase by refactoring the code and by strengthening the protocol wherever weakness was found.

Our team wrote the new validation, tested it, and had it analyzed by Cypher Stack, a research firm with experts in cryptography and Monero. If you’re so inclined, you can read Cypher Stack’s analysis here.

The Haven 2.0 codebase has since been subject to penetration testing for several months by an external Monero expert. 

We’ve made moves to leverage our community even more – launching our public testnet to make sure Haven 2.0 had been reviewed by as many developers as possible. 

And finally we’ve properly incentivized a Bug Bounty Program, offering rewards of up to $100,000 for identification of serious issues.

We have learned from our mistakes, and have come out the other side stronger.

What’s next?

The Haven Protocol is now moving to becoming integrated with the cross-chain decentralized exchange THORChain, taking Haven global. 

The community intends to prove to the rest of the world what it already knows: that Haven is the future of PriFi, and Haven 2.0 will be a significant foundation for anyone who wants to maintain both privacy and stability in an uncertain world.

Bio:

AHawk discovered Haven Protocol in 2018 and has been a community leader for the project since 2019. As a crypto investor and enthusiast, he believes the concept of a Monero-based private stablecoin ecosystem will truly revolutionize how people protect their financial privacy and interact with crypto in the years ahead. You can learn more about the Haven Protocol and the community by going here: https://havenprotocol.org/