‍

Summary

Several weeks ago, an accounting error on Stride related to a tombstoned validator on Evmos affected the  stEVMOS redemption rate, artificially inflating it by roughly 0.1. Related to that issue, stEVMOS unbondings failed for around week, causing a delay in triggering any stEVMOS redemptions. No other stTokens were affected.

This bug caused stEVMOS minters during that team to receive slightly less stEVMOS than they should have during the days the redemption rate was increased. Stride contributors have  worked with Numia to identify the relevant minters, and will compensate them for any EVMOS that they should have received. In total, this amounts to roughly 30,000 stEVMOS (approximately $2,700). 

This incident report explains what happened, and how contributors have worked to prevent this issue in the future. 

‍

Incident and Impact

On September 10th, 2023 and September 13th, 2023, the stEVMOS redemption rate increased artificially by 5% each time. For context, the stEVMOS redemption rate represents the rate at which the protocol will redeem EVMOS for stEVMOS. For example, a redemption rate of 1.3 indicates that 10 stEVMOS can be redeemed for 13 EVMOS, and, correspondingly, 13 EVMOS can be liquid staked for 10 stEVMOS.

The redemption rate increased due to a race condition in an edge case of the Stride unbondings. When Stride issues unbonding messages, it almost always does them in one transaction (containing many sub-messages), and then processes the unbondings upon success. However, the protocol happened to enter into a state where it split the Evmos unbonding messages into two batches, in order to ensure each batch would have enough gas to fit into one transaction. This typically would not result in an issue, but the Stride delegation set on Evmos contained one tombstoned validator, which prevented some unbondings from processing correctly. However, the other batch of unbondings processed correctly, which caused the protocol to incorrectly assume all unbondings processed correctly. 

As a result, the protocol assumed that all unbondings finished successfully, and burned the corresponding stEVMOS. This caused an artificial increase in the stEVMOS redemption rate. 

The Stride v16 software upgrade fixed this issue, and triggered the remaining unbondings as a one-off message in the software upgrade. This also moved the stEVMOS redemption rate to its correct value. All unbondings are now currently processing, and should finish on October 5th, and be claimable on October 6th. 

The issue is now resolved, and stEVMOS functionality is resumed as normal.

‍

The Future 

This highlighted an important race condition on Stride that has now been fixed. Unbondings now will not occur in multiple batches, and there should be no issue if a particular batch of unbondings fails (it will get retried in the next epoch upon failure). 

Stride’s most recent upgrade also added much tighter checks on the redemption rate, to prevent artificial changes like this from happening again. If the Stride protocol detects any small anomalies in the redemption rate, the associated host zone operations will be halted . That means no more liquid staking or redemptions on that host zone until Stride governance can restart the zone, after determining everything is functioning as expected.

To reiterate, all stEVMOS minters that were affected will be fully compensated, and the full scope of impact is roughly $2,700. The Stride Foundation is working with core Evmos groups to make sure that no one else was adversely affected. 

This incident is a reminder that DeFi, while the new frontier of finance, is still experimental and contains risk. Stride contributors have worked to implement as many security features as possible on the Stride blockchain. The blockchain is minimal, has undergone many audits, has rate limits, and various custom security features. Despite this, risk remains in using any DeFi product. Luckily, the existing safeguards on the Stride chain limited the magnitude of losses. 

Stride core contributors are exploring additional features that can mitigate any future incidents. Stride’s design and process goals are to minimize the chance of bugs existing, and ensure that the impact of any  bugs that do exist are minimized as much as possible. Security was, is, and always will be the #1 priority of Stride core contributors.

---

Note: We want to recognize 0xNick for their analysis. They were incredibly helpful in both diagnosing the issue and calculating the best way to compensate users.