The Race Against Time and Strategy: About the AnySwap Rescue and Things We Have Learnt

Discover the critical insights from AnySwap's emergency rescue operation against a smart contract attack and the lessons learned for DeFi security

The Race Against Time and Strategy: About the AnySwap Rescue and Things We Have Learnt

On Jan 18th, our monitoring system detected an attack against the AnySwap project (aka Multichain). The vulnerability is due to the flawed anySwapOutUnderlyingWithPermit() function whose verification mechanism can be bypassed to withdraw the approved tokens.

Although the project has adopted different approaches (e.g., sending transactions to the users, as shown in Figure 1) to notify affected users, a number of them failed to take timely actions (i.e., revoking the approvals) to protect their funds. As a result, attackers could continuously launch the attack to get victim's funds.

Figure 1: The project notified the victim through Ethereum messages

To protect potential victims, after much deliberation, our team decided to perform an emergency rescue for the AnySwap vulnerable contract (0x6b7a87899490EcE95443e979cA9485CBE7E71522) on Ethereum. Specifically, we can transfer the vulnerable account's funds to our white hat wallet, which is a mulsig-wallet (0xd186540FbCc460f6a3A9e705DC6d2406cBcc1C47) on Ethereum.

To make our white-hat rescue transparent, we documented our intention in a pdf file and shared the file hash with the community. This can differentiate our emergency rescue from the attack, while at the same time do not leak the details (since the vulnerability can still be exploited.) We launched the emergency rescue from Jan 21st, 2022 to Mar 11th, 2022, and the corresponding messages had been released to the public, as follows:

Figure 2: Our announcement to perform the rescue
Figure 3: The announcement to stop the rescue

The emergency rescue is not a trivial task. Indeed, there exist several challenges, including both technical and non-technical issues, to make a successful rescue. As the emergency rescue already terminated, we can comprehensively review the whole process and share the lessons we have learnt. We believe this experience can shed some lights on securing the DeFi ecosystem.

Key Takeaways (TL;DR)

  • There exist competitions between different participants, including the whitehats and the attackers. The payment fee for the Flashbots increased rapidly as times went by.
  • Flashbots did not always work. Instead, some attackers turned to use the normal mempool to launch a successful attack by adopting some sophisticated strategies.
  • Some attacker was whitewashed by returning back part of the stolen fund, while the remaining was kept as the bounty award. This phenomenon, though not the first appearance, is controversial in the community because such an incentive may not be fair to the real whitehats.
  • To convince the community, it is a good practice for the whitehat to make the action public known in advance without leaking any detailed sensitive information.
  • The community can work together to perform the resuce more effectively and efficiently. For example, building a coordination mechanism to reduce/avoid whitehat competitions.

In the following, we first give the whole picture of the rescue during this period. Then, we illustrate our way to perform the rescue and the challenges need to be solved. After that, we discuss some lessons we have learnt from the rescue. Finally, we provide some thoughts/suggestions which might be meaningful to secure the ecosystem.

0x1 Rescues and Attacks

0x1.1 Overall Result

The attacks and rescues we observed and investigated in this report have been last for months, ranging from block 14028474 (Jan 18th, 2022) to block 14421215 (Mar 20th, 2022).

The accounts that performed rescues and attacks are summarized in the following table. For the sake of simplicity, only the first four bits of the address are given to represent an EOA. An account is either a rescue account or an attack account. It is worth noting that the type of an account is determined based on either the labels marked by Etherscan.io, or the transferring destination addresses we observed.

In total, there are 9 rescue accounts which rescued 483.027693 ETH (with fee 295.970554 ETH, i.e., 61.27% of the total amount) and 21 attack accounts that harvested 1433.092224 ETH (with fee 148.903707 ETH, i.e., 10.39% of the total amount), respectively. It is worth noting that the loss is a roughly estimate due to some complicated interactions. For instance, an attack account might turn into a rescue account after negotiating with AnySwap, and we will discuss such a phenomenon later. The last column of the table indicates the fee sent to the miner, which were used to win the competition of using the Flashbots.

No. Account Type # of Victims # of Loss (ETH) # of Fee (ETH)
1 0x14ca** Rescue account 50 432.958062 287.849654
2 0x9a65** Rescue account 23 22.569429 0.000000
3 0x9117** Rescue account 14 18.897622 7.213585
4 0x17d2** Rescue account 3 3.552833 0.000000
5 0x6360** Rescue account 21 3.540061 0.907168
6 0x0edd** Rescue account 7 1.498706 0.000000
7 0x281e** Rescue account 1 0.006000 0.000000
8 0xd83b** Rescue account 1 0.004000 0.000000
9 0x8af3** Rescue account 6 0.000980 0.000147
10 0x4986** Attack account 332 456.004547 0.000000
11 0xfa27** Attack account 42 433.438935 46.636389
12 0x48e9** Attack account 66 312.014657 0.000000
13 0x5738** Attack account 67 83.589240 62.587238
14 0x34b2** Attack account 7 63.599821 20.642705
15 0xd374** Attack account 86 45.452703 12.824763
16 0x1fe7** Attack account 9 12.817241 0.000000
17 0x98f5** Attack account 20 8.381273 0.000000
18 0x455d** Attack account 11 5.047377 0.544263
19 0x1b45** Attack account 6 4.942442 3.074813
20 0x3ec7** Attack account 6 3.705686 0.741137
21 0xbca4** Attack account 1 2.784250 1.392125
22 0xb0ab** Attack account 18 0.834068 0.296000
23 0x0a5b** Attack account 1 0.286750 0.143375
24 0x2d3a** Attack account 2 0.080090 0.000000
25 0x835d** Attack account 5 0.063945 0.000000
26 0x1dbd** Attack account 1 0.027431 0.012893
27 0x813d** Attack account 1 0.019528 0.008007
28 0x85dd** Attack account 6 0.002240 0.000000
29 0x2394** Attack account 1 0.000000 0.000000
30 0x6360** Attack account 2 0.000000 0.000000

0x1.2 The Trend of the Fee to Bid the Flashbots

As mentioned earlier, whitehats need to compete with attackers to send the transactions. As a result, the percentage of the fee to the miner (in the Flashbots transactions) may reflect the level of the competition. To quantitatively measure it, we investigate the fee percentage (including attack transactions and rescue transactions respectively) for each block.

Figure 4 gives the trend we observed so far (from block 14028474 to block 14369199). The first several attack transactions do not involve any fee, which means few (even no) competitions were there during that period. This is reasonable because these early attacks might not be well known to others.

As a matter of fact, the first attack with fee (10%) occurred at block 14029765. Since then, the fee percentage increased rapidly as more participants were involved. For example, the percentage reached 80% at block 14072385, and soon reached 91% at block 14129449.

In short, the trend suggests that it is definitely an arms race to win the competition by adjusting more fee to the miner(s).

Figure 4: Fee percentage of attacks and rescues

0x2 Our Rescue and the Challenges

0x2.1 The Way to Perform the Rescue

The basic idea to perform the rescue is quite straightforward. Specifically, we need to monitor the accounts that have approved WETH to the vulnerable contract. When there is any WETH transferred to the account, we can directly transfer it to our multi-sig wallet by exploiting the vulnerable AnySwap contract. The key requirements are:

  • R1: Efficiently locating the transactions that transfer tokens to the victim accounts. We name these transactions as transferred TXs in the following.
  • R2: Correctly crafting the transactions to perform the rescue. We name these transactions as rescue TXs in the following.
  • R3: Successfully frontrunning the transactions sent by the attackers (and any other third-parties). We name these transactions as attack TXs in the following.

R1 and R2 are not obstacles for us. Specifically, we have built an internal system to monitor the mempool, which allows us to timely locate the transferring transactions. Meanwhile, we have developed a tool to build the transactions automatically as well.

However, R3 remains a challenge. One may expect that the Flashbots can be used to win the competition. However, it is not that easy to achieve the goal. Firstly, the attackers may also adopt the Flashbots. As a fee-bidding system, the successful rate may depend on the fee specified to the miner. The strategy to set the fee needs to be determined. Secondly, using Flashbots might not be a good choice cause the intensive competition. Therefore, we also send normal transactions using the mempool. To guarantee the succeed, the strategy to place the transaction in the right position needs to be considered. Finally, we also battle with other whitehats, whose behavior seems to be suspicious in some cases.

0x2.2 Competitions We Involved

In total, we attempted to protect distinct 171 potential victims, while 10 of them protected themselves by revoking the approvals right before we were trying to perform the rescue. For the remaining 161 valid victims, we only succeeded to rescue 14 of them due to the competitions. The failure cases are summarized in the following table, involving 3 rescue accounts and 16 attack accounts.

No. Account Type # of Victims # of Loss (ETH) # of Fee (ETH) Ave % of Fee
1 0x14ca** Rescue account 44 431.651020 286.891724 66.46%
2 0x9a65** Rescue account 7 11.321441 0.000000 0.00%
3 0x6360** Rescue account 3 3.300000 0.891000 27.00%
4 0x48e9** Attack account 35 301.681589 0.000000 0.00%
5 0x5738** Attack account 58 78.482472 58.851862 74.99%
6 0x34b2** Attack account 2 53.591712 17.685265 33.00%
7 0xd374** Attack account 6 23.658698 10.073638 42.58%
8 0x4986** Attack account 16 22.900105 0.000000 0.00%
9 0x1fe7** Attack account 6 12.057241 0.000000 0.00%
10 0x1b45** Attack account 5 4.402442 3.010013 68.37%
11 0xbca4** Attack account 1 2.784250 1.392125 50.00%
12 0x98f5** Attack account 8 2.339543 0.000000 0.00%
13 0x455d** Attack account 3 0.741817 0.175454 23.65%
14 0xfa27** Attack account 3 0.320288 0.032590 10.18%
15 0x0a5b** Attack account 1 0.286750 0.143375 50.00%
16 0x3ec7** Attack account 1 0.245000 0.049000 20.00%
17 0xee7e** Attack account 1 0.190000 0.096900 51.00%
18 0x835d** Attack account 3 0.024533 0.000000 0.00%
19 0xb0ab** Attack account 1 0.000618 0.000000 0.00%

Hence there do exist some lessons we are willing to share with the community.

0x3 Some Lessons We have Learnt

0x3.1 How to Determine the Number of Fee to Flashbots' miner?

In summary, we were beaten by 12 competitors, including 2 rescue accounts and 10 attack accounts, which were using the Flashbots.

Our strategy to set the miner fee was quite conservative. Specifically, we aimed to protect the victims by spending the fee as little as possible. Hence when we did not actively use/increase the fee unless some successful attack TXs had already set the fee. For example, if an attacker sets the fee to 10% of the fund, then we might use 11% for the next rescue to compete with that attacker. However, the result suggests that this strategy did not work as the attackers (or even some whitehats) often (if not always) aggressively increased the fee to beat others, as follows:

  • Figure 5 shows that the attacker 0x5738** set the fee percentage to 70% at block 14071986.
  • Figure 6 shows that the whitehat 0x14ca** set the fee percentage to 79% at block 14072255.
  • Figure 7 shows that the whitehat 0x14ca** set the fee percentage to 80%, at block 14072385.
  • Figure 8 shows that the whitehat 0x9117** set the fee percentage to 81% at block 14072417.
  • Figure 9 shows that the attacker 0x5738** set the fee percentage to 86% at block 14073395.
Figure 5: 70% fee specified by the attacker 0x5738
Figure 6: 79% fee specified by the whitehat 0x14ca**
Figure 7: 80% fee specified by the whitehat 0x14ca**
Figure 8: 81% fee specified by the whitehat 0x9117**
Figure 9: 86% fee specified by the attacker 0x5738**

In brief, it seems to be a zero-sum game which requires modelling the behaviors of different participants to win the competition.

In practice, however, it is challenging to figure out better/optimal strategies, and meanwhile to reduce the cost as much as possible.

0x3.2 How to Place the Right Position in the Mempool?

Now it looks like the rescue would rely on the arms race of fee competition to bid the Flashbots. However, we found that using Flashbots was not panacea due to the intense competitions from other participants which had nothing to do with the rescuing and attacking. In such a case, even the highest fee specified by an attack Tx cannot guarantee to win the chance to use the Flashbots.

Alternatively, a normal transaction with the right position in the mempool may have the opportunity to achieve the goal. The right position here means the rescue/attack TX should be placed at the position behind the transferred TX, and the position should be very close to the transferred TX (the closer, the better). Note that by using the strategy, the attacker 0x48e9** harvested 312.014657 ETH without paying any fee to Flashbots miners.

The following four figures demonstrate the top 2 biggest profits the attacker have made:

  • Figure 10 shows that a victim deposited 50 ETH at position 65 of block 14051020, and figure 11 shows that the attacker harvested the 50 ETH at position 66 of the same block.
  • Figure 12 shows that a victim deposited 200 ETH at position 161 of block 14052155, and figure 13 shows that the attacker harvested the 200 ETH at position 164 of the same block.
Figure 10: Transferred TX at position 65 sent by the victim 0x3acb**
Figure 11: Attack TX at position 66 sent by the attacker 0x48e9**
Figure 12: Transferred TX at position 161 sent by the victim 0xbea9**
Figure 13: Attack TX at position 164 sent by the attacker 0x48e9**

Obviously, this sophisticated strategy is quite useful and enlightening, and deserves more attentions and efforts to learn something from it.

0x4 Some Other Thoughts

0x4.1 Whitehat Hack or Attack?

When it comes to the recognition of the whitehat hacks, they might be not as straightforward as one may expect.

For example, address 0xfa27 was labelled as Multichain Exploiter 4 (Whitehat) by Etherscan.io. Actually, at the very beginning, it was labelled as Multichain Exploiter 4. After several rounds of negotiation between the attacker and the AnySwap project, the attacker was persuaded to return back part of the stolen funds.

  • In TX 0x3c3d**, AnySwap contacted the attacker:

First and foremost thanks for getting the weth. I was not aware of the hack and realized the situation only because the weth never arrived in my wallet after the cowswap transaction. Considering the amount at stake, would you accept 50 eth as a fair tip? Here is mine txn: 0x2db9a6a51604e2be8b2c3469773afb201f0b48a318fb7e5f5e49175e818df5ba 0xe50ed602bd916fc304d53c4fed236698b71691a95774ff0aeeb74b699c6227f7

pls verify by sending me back a tx. I will send you the rest 258 eth. 39 eth went to the miners because there other attackers so I have to pay that tip to save the money.

  • In TX 0x354f**, AnySwap acknowledged with thanks after receiving the funds:

Well received, thank you for your honesty.

Obviously, this attacker was whitewashed and also made some profits from the attack. The similar cases have occurred several times in the past, and they are still controversial in the community because such an incentive may not be fair.

0x4.2 Competition Between Whitehat Hacks?

It is necessary to build a coordination mechanism to reduce/avoid competitions between the whitehats. Such a competition inevitably leads to a waste of the rescue power. In this rescue, there exist 54 victims (with 450 ETH fund) which were protected by other three whitehats, while we also tried to perform the rescue.

The competition between whitehats not only wasted the rescue power, but also raised the cost to perform the rescue. For example, as shown in figure 7 and figure 8, the fee spent by the two rescue TXs with different whitehats are 80% and 81% respectively.

Unfortunately, the whitehats will not step back unless there exists any mechanism to coordinate with each other. Otherwise, it is impossible to make the competition disappear.

0x4.3 How to Make a Better Rescue

On the one side, to convince the community, it is a good practice for the whitehat to make the action public known in advance without leaking any detailed sensitive information. There is enough time to do it as the rescue is usually a seesaw battle with multiple trials, which is different from some one-time effort like blocking a particular attack. Of course, the detailed information regarding the vulnerabilities should not be leaked.

To achieve this, the detailed information will not be disclosed at the very beginning and will be disclosed to the community after completing the rescue, just as what we have done for the AnySwap rescue. However, the hash of the document with the whitehat intention can be shared with the community.

On the other side, the community can do more things to perform the resuce more effectively and efficiently, including, but not limited to:

  • Flashbots/Miner may provide some the green channel for the certified whitehats. The green channel can provide a high priority to front-run the attackers' TXs, and avoid competition between whitehats.
  • The projects being attacked cover the cost of Flashbots/miners.
  • The projects may apply convenient and fast notification mechanisms to users.
  • The project may apply the emergency mechanism in the code.

About BlockSec

BlockSec is a pioneering blockchain security company established in 2021 by a group of globally distinguished security experts. The company is committed to enhancing security and usability for the emerging Web3 world in order to facilitate its mass adoption. To this end, BlockSec provides smart contract and EVM chain security auditing services, the Phalcon platform for security development and blocking threats proactively, the MetaSleuth platform for fund tracking and investigation, and MetaSuites extension for web3 builders surfing efficiently in the crypto world.

To date, the company has served over 300 esteemed clients such as MetaMask, Uniswap Foundation, Compound, Forta, and PancakeSwap, and received tens of millions of US dollars in two rounds of financing from preeminent investors, including Matrix Partners, Vitalbridge Capital, and Fenbushi Capital.

Official website: https://blocksec.com/

Official Twitter account: https://twitter.com/BlockSecTeam

Sign up for the latest updates