Telcoin Security Incident Post-mortem and In-Depth Analysis

A comprehensive post-mortem and analysis of the security breach Telcoin experienced on Christmas Day 2023.

Telcoin Security Incident Post-mortem and In-Depth Analysis

December 25, 2023, our monitoring system detected a series of malicious activities targeting Telcoin. We assisted the Telcoin team in identifying the root cause as the improper initialization of wallet contracts, which arose from inconsistencies between the wallet’s actual implementation and its corresponding proxy. This report aims to provide a thorough analysis to fully comprehend the incident.

0x0: Basic Design

Before examining the vulnerability, it is important to first understand the relationships between the involved smart contracts. Essentially, these can be abstracted into a combination of three design patterns: CloneFactory, Cloneable Proxy and Beacon Proxy patterns, which are depicted in the following diagram.

0x1: Vulnerability Analysis

The vulnerability stems from improper initialization of wallet contracts, which is due to a mismatch between the wallet’s actual implementation and its corresponding proxy. Specifically, during the initialization process, the proxy initialized storage slot 0 to a non-zero state by writing to the least significant bits of the storage location. Subsequently, the wallet code also wrote to storage slot 0, thereby overwriting the proxy’s initial value in the least significant bits. This issue is not the result of an inherent vulnerability in either of the smart contracts but rather the interaction between the two.

In the following, we will illustrate the details based on the transaction trace provided below:

Specifically, within the CloneableProxy:Proxy.initialize() function, there is a delegatecall that invokes the Wallet.initialize() function. This invocation occurs through a delegatecall to the CloneableProxy:Implementation.initialize() function. As a result, any modifications to the storage made by the Wallet.initialize() function will be reflected in the storage of the CloneableProxy:Proxy contract.

To understand the implications fully, it is necessary to examine the storage layout of the CloneableProxy:Proxy contract. The definition of this contract is outlined as follows:

Given that neither the Proxy nor the ERC1967Upgrade contracts have storage variables, slot 0 is instead utilized by two storage variables — _initialized and _initializing — both inherited from the Initializable contract.

Now, let us examine the Wallet contract. Within the Wallet.initialize() function, it is evident that slot 0xaa serves as the initialization flag. This is underscored by the following code snippet in lines 3–4 and 11–12:

Note that slot 0 is allocated for _state, which stores the next 32 bytes from the calldata following the function selector, as indicated in line 21. For additional detailed information, refer to the commentary at the beginning of the Wallet contract:

There is a discernible mismatch regarding the use of slot 0: the CloneableProxy:Proxy contract interprets it as the initialization flag, whereas the Wallet:initialize() function treats it as the state of the wallet.

Consequently, after the initialization process, the least significant two bytes of slot 0 will be reset to zero. This effectively sets both _initialized and _initializing to zero. As a result, the CloneableProxy:Proxy contract becomes vulnerable to re-initialization through the initialize() function, as the initializer modifier’s protection can be circumvented.

Obviously, the potential for exploitation hinges on the state of the wallet. Once updated to a non-zero value, the wallet’s state prevents any further re-initialization of this contract. After initialization, as the wallet state is updated with each transaction, the likelihood that the least significant two bytes of slot 0 will be non-zero increases, which effectively immunizes the wallet from being re-initialized. This explains why the majority of the vulnerable wallets had little or no transaction history, leaving them exposed to the attack.

0x2: Attack Analysis

The attacker began by reinitializing the vulnerable CloneableProxy:Proxy contract to alter the address of the Beacon contract. Subsequently, the attacker proceeded to transfer the assets contained within the CloneableProxy:Proxy contract, as detailed below:

It is noteworthy that several vulnerable contracts were compromised in a single transaction, with the attacker executing this strategy repeatedly.

0x3: Summary of Attacks

We observed 4,958 total attacks, executed by six distinct accounts, as follows:

0x4: Fix and Recommendations

Our investigation has revealed that the core issue stems from the inconsistent use of storage slot 0, leading to the possibility of reinitializing vulnerable contracts. Obviously, the fix involves careful management of storage allocations.

From this incident, we have gleaned several critical insights:

  • Exercise extreme caution when manipulating storage slots with inline assembly, as errors can lead to significant vulnerabilities.

  • Maintain vigilant monitoring of contract statuses. The ability to respond quickly hinges on receiving timely alerts.

  • Implement a pause mechanism within contracts to enable immediate suspension of activities if a compromise is detected.

Furthermore, the fact that this incident involved multiple attack transactions targeting different wallet contracts highlights the pressing need for threat monitoring and attack blocking solutions, such as Phalcon Block. Such tools are essential in mitigating future risks and safeguarding against prospective losses.

0x5: Timeline of Events

09:23 AM PST, December 25: Our system detected the first malicious transaction on the Polygon network:

10:28 AM PST, December 25 : Telcoin support team reported the incident in the internal communication channel.

10:32 AM — 10:37 AM PST, December 25 : Telcoin team members notified the Telcoin Discord community and created an emergency call with all relevant key team members.

10:45 AM PST, December 25: A Web Application Firewall rule was implemented to restrict all access to Telcoin Infrastructure.

11:02 AM PST, December 25: The Telcoin team initiated talks with Seal 911

11:11 AM PST, December 25: A war room was established by the Telcoin team and other security members to discover the root cause of the issue and discuss potential solutions to block the attack.

01:14 PM PST, December 25: The Telcoin team publicly issued an alert to users via X (Twitter):

03:39 PM PST, December 25: Telcoin contacted Chainalysis & Slowmist to help investigate the stolen funds by undergoing an investigation in which they labeled the stolen wallets and addresses and shared this intel with exchanges.

10:35 PM PST, December 25: Upon invitation from the Telcoin team, we joined the war room and shared our analysis to determine the root cause.

11:00 PM PST December 25 to 02:06 AM PST December 26: After identifying the root cause, the Telcoin team successfully devised a mitigation strategy by replicating the exploit in a secure and controlled manner. This process involved reinitializing the wallet proxies to align with a new, securely implemented beacon. Given that this exploit was a one-time opportunity, Telcoin can preemptively update the wallet configurations, thereby disabling the attacker’s capability to further exploit these vulnerabilities.

02:07 AM PST December 26 to 02:14 AM PST December 26: The Telcoin team implemented the mitigation process across all wallets not previously compromised to ensure comprehensive coverage. For rapid and efficient deployment, the process was executed in batches within a strictly defined time window. Telcoin then began the process of preparing then internally testing the remediation plan for a complete restoration of all impacted wallets and permanent fix for all wallets.

04:52 PM PST December 26: Telcoin and our team initiated discussions on the following topics:

  • Incident Overview

  • Timeline of Events: A chronological account of how the incident unfolded.

  • Root Cause Analysis: An in-depth analysis of the underlying causes of the incident.

  • Recommendations for Improvement

  • Code Audit

12:27 PM PST December 29: After several rounds of discussion, we began collaborating with the Telcoin team to draft the post-mortem report and audit the remediations.

Jan 03, 2024: The draft of the post-mortem was provided.

Jan 04, 2024: Our audit findings, including identified issues and recommendations, were presented.

Jan 04 to Jan 10, 2024: We collaborated with the Telcoin team to finalize the post-mortem and review the fixes.

Also, it is worth noting that from December 25 to the present date, Telcoin has been actively working in close collaboration with blockchain investigative firms and law enforcement regarding the incident.

About BlockSec

BlockSec is a pioneering blockchain security company established in 2021 by a group of globally distinguished security experts. The company is committed to enhancing security and usability for the emerging Web3 world in order to facilitate its mass adoption. To this end, BlockSec provides smart contract and EVM chain security auditing services, the Phalcon platform for security development and blocking threats proactively, the MetaSleuth platform for fund tracking and investigation, and MetaDock extension for web3 builders surfing efficiently in the crypto world.

To date, the company has served over 300 esteemed clients such as MetaMask, Uniswap Foundation, Compound, Forta, and PancakeSwap, and received tens of millions of US dollars in two rounds of financing from preeminent investors, including Matrix Partners, Vitalbridge Capital, and Fenbushi Capital.


X(Twitter): @BlockSec @Phalcon

Sign up for the latest updates