Introduction
The smart contract auditing game in crypto is broken, and if we want to see a lot less people get rekt, we need to fix it.
Right now, a lot of contracts make it to mainnet without an audit and actually it’s pretty understandable. The price of an audit varies from expensive to insane, and neither of those buckets guarantees quality.
The reality of the game is that there is a damn sight more contracts written than there is bandwidth for auditing, and thus an extreme supply/demand mismatch has driven prices for audits exponential, pushing them out of the reach for most projects.
Even if they do have the budget, there is often several months of wait time, which in this space can mean getting front run by more risk friendly degens who don’t care as much about user safety. And that’s if you even get in contact with them; most of these firms will just ignore you unless you’ve just pulled off the mother of all raises with a16z or something.
Anyway, you get the picture.
But.. broader than any of this, I’m not convinced centralised approaches will generate the safest contracts, which is what this article is about. We’ll talk about the game theory of smart contract auditing and why a collaborative DAO approach has potentially a 10X improvement on the current state of play.
The Background
I’m thinking about this currently, because we’re taking a tranche of our FactoryDAO smart contracts to a code4rena C4 audit contest; and I’m convinced it’s a more effective way to scale auditing in the space than centralised approaches.
The background here is that I wrote a tweet last year that reflected some of the internal conversations I’ve been having with my good friend and CTO at FactoryDAO about the game theory of contract auditing. I laid out a few ideas (which I’ll elaborate on in this article) and pinged it into LobsterDAO. I didn’t immediately get canceled by Ivan for doing so, which prompted me to think it wasn’t an absolutely terrible idea. A couple of months later White Hat superhero samzsun opened his DMs for questions, and I sent him the above tweet thread for his thoughts to which he kindly responded:
“kinda sounds like what code4rena is doing”.
And it was! So here we are.
The Game Theory of Auditing
I’ll explain the code4rena approach and how I think what they’re doing as a DAO is a huge step in the right direction, but I think it’s worth getting into some first principles thinking about smart contract auditing processes first.
Firstly, there is no such thing as a “safe” smart contract. These pieces of code live on the internet holding or routing money and can do so for many years. No code is ever perfect and even if it was at that time, the space changes around it and new composability risks emerge dynamically over time as primitives like flash loans emerge1.
Smart Contracts can sometimes create monumentally large rolling bounties that attract every Black Hat hacker in the world out of their shadowy internet dwellings. If they pull off a successful score, they can make even the biggest bank heists of old seem like dime store smash and grabs. The rapid explosion of the cryptospace has been a hackers wet dream, and consequently, we’ve got so used to eye wateringly large hacks that anything less a 8 digit heist barely makes it to the news and doesn’t get you into the top 50 of the rekt leaderboard.
So, let’s start with a simple “game”.
The Trust Game
Alice writes a smart contract that she thinks will collect a large amount of Total Value Locked (TVL).
Being the diligent sort, she wants this reviewed, so she passes it to Bob.
Bob is a smart contract galaxy brain and Alice is sure that if there’s a critical bug, Bob will find it.
He does.
Here comes the game theory. Bob has found a bug in the contract that can be exploited to drain the funds out of the contract when it hits mainnet.
He has two clear options:
1) Tell Alice about it, who can fix the bug and save the poor degens from getting rekt.
2) Don’t tell Alice about it and rug the contract himself and make away with boat money.
With Option 1, Bob does not don the Black Hat and can go to his death bed with a clear conscience without crossing the moral point of no return. He gets mega kudos from Alice, and… that’s about it. There was no formal payment arrangement, the arrangement is held together by trust.
With Option 2, Bob turns out to not care at all about wearing the Black Hat, crossed the moral point of no return ages ago, believes his Op Sec is on point, knows how to use mixers like Tornado Cash and knows people who can flip crypto for cash where he lives, no questions asked.
In Option 1, Bob gets nothing much more than a pat on the back. In Option 2, Bob could get well… lots of money.
We’ve had silly meme farm tokens contracts in recent memory that have picked up over a billion dollars in TVL in days. Anything is possible. In fact, the temptation could be SO high that there are a lot of people that would flip from Option 1 to 2 based on the size of the pot.
This is the simplest game and as you can see it's flawed. The game is a friendly trust game and that’s about it. Not what this space is about.
The Mainnet Game
In this world, Alice doesn’t have any “trustworthy” galaxy brain frens and is a solo degen with a strong following on Discord and Twitter eagerly awaiting her next TVLtastic smart contract.
So, she uses a bounty system like immunefi, finds a decent amount of money from somewhere (say $100k) to post as a bounty and launches her contract free into the cryptoeconomic battleground.
In this game, there are many Bobs. Let’s say we have Bob1 to Bob100.
This legion of Bobs vary in attitude, skill and moral framework. We can stack them in type from white hat hero to the blackest of black hats.
Now, Alice has built a contract that particularly tickles the ape fancy and it has accrued $100m in the first day.
There’s a bug. It’s a nasty one. It’s possible to drain the entire amount and high tail it outta there with full bags of digital loot, but it is a tricky one to find.
Now it’s just a matter of which Bob finds it first. Double plus good Bob would probably save the day anyway without the bounty, because he’s an all round good guy, but he’s just not that good of a hacker, so we go down the Bob stack.
Next, we have a collection of responsible Bobs, who if they discover it first, will go straight for the “responsible disclosure” process on immunefi and claim the funds. Importantly, we need to recognise this is a race. First to disclose wins. That’s good. It means we’ve created an urgency which attracts the responsible Bobs to the game and it’s a case of most skilful Bob first claims the prize.
Skill races are good.
Then we’ve got the Bobby Black Hats who are willing to dance the dance of trying to exit $100m of crypto through insanely traceable blockchains and spend the rest of their lives looking over their shoulder for a surprise trip in a van. These guys are the main problem with this approach. They don’t really care about the bounty as it’s a rug race between the Bobby Black Hats and the responsible Bobs. Let’s hope the good guys prevail!
In the middle, we’ve got temptation Bobs. Whether they will be swayed is a function of the size of the bounty on either side of the digital tracks. The bigger the bounty the more likely they are to keep the white hat on; the bigger the TVL and smaller the bounty, the greater the chance they’re up for a rug and run.
Bounty systems are essentially a mechanism design. You tune your bounty reflective of your TVL to hopefully flip the goodish guys to good and maybe even some bad. The principle here is everyone has a price, so price accordingly.
The limitations are that Alice needs a fat wedge of cash to be able to post the bounty in the first place, which isn’t always on the cards. Also, Black Hats gonna Black Hat and because we’re live on mainnet there’s a rug race going on alongside the skill race. Not ideal, but way better than nothing.
The Auditing Game
In this world, Alice has found enough cash to be able to pay an auditor to check these contracts before they hit mainnet.
The deal is that the contracts get looked at by skilled professionals who see LOTS of contracts and are therefore well versed in the latest attack vectors, and she gets a shiny PDF proving to prospective apes that the contracts are “safe”.
The game is as follows:
Alice pays $60k to auditing company SuperAudit.
SuperAudit has a stable of contract Clives, from which they assign the job of reviewing the contract.
Clive 2 receives the job, and he gets a flat rate salary whatever happens. The pressure is on though–if the contract gets breached, Clive 2’s job is on the line as is the gleaming PDF driven reputation of SuperAudit.
The problem is that SuperAudit has a queue a mile long for their PDFs, and they could probably sell 10X the number of PDFs if only they had more Clives (the real bottleneck).
So they have an economic problem, how many Clives to assign? If it’s just one, then we’re essentially at something close to The Trust Game. Clive 2 could discover a bug, “miss it” and then go and rug the contract himself. He might lose his job, but who cares, boat money YOLOOOO.
Adding another Clive costs them double the bandwidth but now Clive 2 and Clive 1 are in a prisoners dilemma. If the rug hunting Clive 2 discovers a bug, he’s gotta hope that Clive 1 doesn’t discover it as well, and this also creates that juicy competition dynamic, where in this case the bounty is brownie points with the SuperAudit top brass.
Of course Clive 1 and Clive 2 can collude and rug together, but in any case this is a better scenario than before since as we all know, 2 Clives are better than 1. The issue here is that the more demand there is in the market, the greater the incentive we have to assign a single probably burnt out Clive. This happens a lot.
The difference in this game is that we have a PDF, which handily absolves Alice of her diligence responsibilities and keeps the apes happy, but really that’s all it is, a PDF. There is no assumed responsibility beyond some repetitional stake from the auditing company, and there are seldom deals available that allow minor updates to the contract to be re-audited at a lower cost, leaving teams making minor updates but opening up big vulnerabilities.
Of course there are auditing companies that have legions of Clives and will assign full teams to a job, but they are megabucks. Even if you have the money, it’s like getting an audience with the Pope. This is a major bottleneck in the space and needs to be resolved.
In The Auditing Game, we have a game that improves on the flimsy dynamic of The Trust Game. We have contracts reviewed before they accrue TVL, so we’re better than The Mainnet Game, but it’s far from ideal. And remember, this scenario is only available if you can find a SuperAudit willing to take your money.
The DAO Game
Now imagine we have a DAO full of contract Clives, but also some Alices and Bobs. It’s an informal set-up, even pseudo-anonymous, but there’s money to be made and flexes to be had.
In this scenario, Alice presents her contracts to the DAO and an open number of Solidity degens enter into a skill race. There’s no rug race here because we’re pre-mainnet. Good.
Now we’re getting the dynamic we had in the Mainnet Game, but before we have TVL.
The game theory doesn’t really change. We have a set of n-players, some of whom might be malicious, some of whom might be double plus good.
But… it doesn’t really matter.
Why? Because..
the more eyes the better.
In The Auditing Game, we have one or two Clives rather poorly incentivised to find the bugs. Here we have a diverse range of actors competing in a skill race. Unlike The Bounty Game, there is no TVL at stake to bump temptation Bobs to the dark side2, just the possibility that they could find a juicy bug and wait till mainnet for max rug. The difference here though is that instead of waiting to see if a tired Clive 2 will see it before it gets a pass, we have many eyes all looking at the same time.
With each subsequent pair of eyes, the probability that a discovered bug doesn’t get declared drops (probably exponentially) to zero. The power of high-n peer review.
This again is an incentive game. We put a bounty on the line that is something close to what Alice paid for her audit, but instead of SuperAudit taking the lion's share it goes to the participants in The DAO Game. Decentralisation in action.
Additionally, performance in this skill race is about the best test of knowledge you can imagine in blockchain development. The record you could pick up in an environment like this could make you one of the most desirable devs in the space. If you perform well in a game like this, we at FactoryDAO will hire you and I’m sure every sane project in the space would too. Double the incentive.
If this works though, you don’t even need a normie salaried job; you can just find bugs in smart contracts whenever you can be bothered and live the full DAO life. Dream.
The code4rena C4 Audit Contest
Much to my delight, almost this exact scenario is happening in this space as we speak, and we have jumped at the chance to get our contracts into the arena.
In the code4rena DAO participants in the contest are known as ‘Wardens’, projects like FactoryDAO are ‘Sponsors’ and a third layer are ‘Judges’ who adjudicate the contest and ensure that fair attribution is passed on to the Wardens playing the game and that reported issues are codified correctly against there severity.
Code4rena contests frequently get 20+ Wardens playing and their reports are excellent. That’s a lot of eyes, way more than you would ever get in even the top tier auditing outfits.
The game is incredibly well structured and isn’t a winner take all game like The Bounty Game. In the arena multiple Wardens can find the same bug, submit a report and share the pot between themselves, which is weighted based on severity. If no critical bugs are found, the incentives trickle down to gas optimisation and QA reporting, all graded on a curve.
This is very cool. In my previous life as an academic, I spent a fair amount of time in assessment theory and the code4arena contests are one of the most well designed assessments and collaborative learning scenarios I’ve seen3. It creates competition but incentivises collaboration. It’s a model that could very well be abstracted to a wide range of educational type use cases.
Summary
I’ll say it again, there’s no such thing as a safe smart contract, but I’m convinced that massive, collaborative games like the one curated by the code4rena DAO are the way to level up smart contract auditing in the space.
Of course, there are some trade-offs: we don't get the shiny PDF from a big name (personally this doesn’t impress me much, but it works on the degens) and it's entirely new and people perhaps don’t necessarily trust it yet. Additionally, some combination of all of the above games is optimal, security is an on-going game and should be reflective of the risk in the contracts.
But, I’m convinced this is the way. I wouldn’t be surprised if we saw many DAOs doing this kind of thing in the future or even some of the big auditors integrating DAOs into their practice. This is also one of the best examples of DAOs solving real problems and I'm excited to see how this contest goes.
flash loans are like firing digital cannon balls through your smart contracts. When these first emerged on the scene early 2020 lots of contracts got rekt to bits from oracle attacks.
This game does work for mainnet audits too, in a similar fashion to the mainnet game, potentially more improved due to time bounded coordination and therefore more competition.
Massive kudos to Scott Lewis the mechanism designer responsible for the code4rena design!