Chronicle of a Vulnerability Foretold
Hey Snook people (’cause let’s face it, if you are reading this, you’re one of us), it’s Shahar Larry.
This one I decided to write under my own name. It feels more true this way.
I joined Snook in March 2021, just a few weeks after its inception.
Since then, I have had the privilege to be a member of Snook’s core team. One day I will recount this experience, as it was, is, and no doubt continue to be simply exceptional. But first I want to tell you about some of the work done behind the scenes.
Snook was built fast. It took us 6 months to move from an idea to the launch of a fully functioning MVP. I’ve seen rapidly built projects before. They tend to be flimsy. This was not the case. The team I joined, especially the product and development team, would not compromise on anything that had to do with the game and its look and feel. Honestly, at times it was nerve racking. For example, we were all set to launch in the beginning of August, but after the 2nd beta test, the product team insisted we postpone the launch, scrap the old game design, and replace it with a fresh and more exciting GUI and UX. Nerves of steel. Everything we did, had to be the best and match our shared vision. “Good enough” was not an option. It cost us mainly sleeping hours (and a couple of relationships…)
As Snook’s development was maturing and with the launch date just over the horizon, we started looking for a code audit provider. ConsenSys Diligence was an easy choice with their track record and reputation. But we underestimated how busy these guys are, and after a short discussion, they suggested a Spot Check would be enough and allow us to launch in time and them to squeeze us in.
As the title of this piece implies, we missed something. Fortunately, as Billy Shakespeare wrote All’s Well That Ends Well.
Ok, time to dig in. You ready?
A couple of weeks ago, we found a potential vulnerability in our code. It did not pose any risk of misallocation of funds. It only meant that vested $SNKs could get locked, effectively removing them from circulation. (At worst it would have meant greater scarcity :D.)
We fixed it.
All’s well that ends well.
Snook was launched on Polygon’s Mainnet on September 2nd, immediately after ConsenSys completed the Spot Check and cleared the code.
We missed something.
[Note: You may have noticed that I sometimes write “we” and other conjugations of the plural pronoun (our, us, etc.) in bold letters. That’s to discern between the times I mean both the Snook and the ConsenSys Diligence team (we) and other times. K?]
Snook, we have a vulnerability!
On September 14th at 11:52 EDT our Diligence auditor found a vulnerability. It was an unforeseen result of a previous recommendation they gave us. It was missed in the Spot Check. These things happen.
Before I go “under the hood” here’s the gist:
Snook’s Vesting Contract controls ~36.67M $SNKs (out of the 40M minted). It acts like a smart “tap” that once initiated by a beneficiary, releases funds according to preset conditions. The beneficiary can be a team member, like me, or a “cyber-organ” like the Treasury.
The vulnerability was the result of two independent issues, each on its own would not cause any problem. The first issue was with the designation of a certain function in the release of funds process as “public”. That means anyone could call (activate) it. That alone would not be a problem. I mean, if someone wants to pay the transaction gas for sending funds to my wallet, by all means, go ahead. The second issue was with the fact that this function, originally designed to provide the (read-only) amount to be transferred was modified last minute. It now could change the value of a specific storage variable that indicates if funds were already sent or not.
Now, with this modification, the first issue, i.e., the public designation, became a problem. It meant that a malicious user could theoretically call it and change that storage variable, indicating that funds were already transferred, when in fact they weren’t. It would not actually transfer the funds. It would, however, effectively lock them.
Since the Vesting Contract is not upgradable, it looked like we had a real problem.
It wasn’t all bad. There were several “silver linings”:
- Funds were never at any real risk — As mentioned in the TL;DR the vulnerability did not pose any risk of funds’ misallocation. It only meant that vested $SNKs could get locked. That makes it far less attractive for exploitation and basically a potential act of cyber-vandalism with no upside to the perpetrator.
- We caught it before it could be exploited — Thanks to the impressive vigilance of ConsenSys we caught it in time. You see, the first time this vulnerability could theoretically be exploited would be after the first vesting due date, i.e., October 1st. We had two weeks to resolve this issue.
We needed to come up with a quick solution. For those of you who are not interested in the slightly more technical description of the vulnerability, you can jump ahead.
You’re still here? Let’s dive in, try to keep up.
Those of you that follow us know the graph below (you can find it here):
As mentioned above the Vesting Contract controls the allocation of $SNKs to a list of beneficiaries (e.g., the Snook team, supporters and contributors, advisors, the Treasury, an ecosystem fund and Liquidity pool). Once a pre-scheduled timestamp is reached, funds become available for distribution to the predefined list of beneficiary wallets and anyone (most likely the beneficiary) can call the release() function to claim the funds to the beneficiary’s wallet. The release() function calls another function — getRealeaseableAmount(). Its job is to tell the release() function how many $SNKs to release. Here’s how a malicious user could exploit the vulnerability:
- She could call on getRealeaseableAmount() because it is public
- This would then change a storage variable to indicate that the transaction was already completed.
- The beneficiary would then try to claim the funds but would not be able to because the release() function would have an indication that the transfer already happened.
This would not have happened if the getRealeaseableAmount() function would have been private. In such a case only the Vesting Contract could call it via the release() function.
That Ends Well!
So nooo? How did you guys solve this?
Well, funny story. It turns out that we found another issue in the code that saved our ass.
As I mentioned above, the Vesting Contract is not upgradable. That meant we could not change it (by for example making the getRealeaseableAmount() function private and/or preventing it from being able to change storage variables). The only solution was to burn the contract.
That was a depressing thought.
It seemed to imply that we would have to redeploy everything and re-mint and redistribute $SNKs.
A big fat bag of d!*k$!!
We really did not want to do that.
Luckily, we had ConsenSys Diligence on our side and we are grateful for that. We were and are impressed with their commitment and skill. They were the ones that found the vulnerability; they reported it; they stepped up and found a fix. Their unfaltering sense of responsibility is impressive. Still, you might imagine the whole thing was a bit awkward…
Shortly after we discovered the vulnerability, we got hold of Gonçalo Sá, ConsenSys Diligence’s co-founder and Security Researcher (and all in all a great guy). He ‘rallied the troops’ on his end and both teams tried to find a way to avoid what seemed unavoidable.
Replacing everything with exact duplicates
On September 17th at 13:08 EDT, 73 hours and 16 min after the initial report, we found a solution.
It turned out that we inadvertently forgot to revoke a minting role on an L1 account. That role allowed a specific Snook core-team user to mint $SNKs. That meant we could write a new Vesting Contract all shiny and impregnable; burn the old one (with the 36.67M $SNKs — but they were not yet in circulation anyway); mint ~36.67M new $SNKs that would be distributed by the new Vesting Contract. It’s kinda like that one liner by the philosopher and comedian Steven Wright: “Last night somebody broke into my apartment and replaced everything with exact duplicates.”
While writing this the thought had occurred to me that we could have fixed the problem and not tell anyone. No one would know and it would plug the hole.
This would have been a lot easier, less embarrassing for us and for ConsenSys Diligence.
But that’s not us.
This reminds me of something I learned as a child. A Talmudic (old Hebrew Law) legal argument called Miggo. Miggo is Aramaic and means something close to “since”. The idea is that a claim used by a defendant should be accepted as true since if he was a liar, he could have made a different claim that would have been more beneficial for him. Sort of, if I was a liar, why would I use this lie when I could have used a much better lie.
Right from the onset of this rabbit-hole chain of events it was clear to us that we would publish everything.
And so, we have.
Epilogue — It’s the collaboration Stupid!
The past couple of weeks were trying. Not only as a time-sensitive problem-solving challenge, but also for us as a team.
We’ve learned a lot.
The first lesson may sound weird. Our first reaction was to be angry with ConsenSys. In retrospect that was wasteful and a bit unfair.
People make mistakes. We had put pressure on them to meet a tough deadline. However, they showed their true colors when despite our expectation for a push back, they did not abdicate and they took responsibility, working hand in hand with us to find a fix. Which they did. Not Us. They are pros and we are impressed with their command of the technology and diligence.
The second learning can be summed up with the wonderful words of wisdom by Miracle Max: “…you rush a miracle maybe you get a rotten miracle”
Remember — what we do, all of us in the crypto, NFT and Blockchain world is promote collaboration (not wasting time on pointing figures). That is the true meaning of these platforms and that is why they will change the world. Collaboration is not measured when everything is smooth. It is tested when things break down.
Shahar for the Snook Core Team