Agoric Mainnet Update: Cosmos Barberry Patch

Attention Validators:

This morning, core Cosmos SDK developers released a security patch for the high-severity Cosmos Barberry issue. The Agoric team independently identified a medium severity issue in its fork of the cosmos-sdk, and the patch for this was distributed along with the Cosmos Barberry security patch release. Per the runbook, an advisory was sent to Agoric validators to give them ample notice to prepare for security coordination.

A “soft-patch” release for these issues is now available, and it can be applied as a soft-patch without having to go through a full consensus upgrade or on-chain governance. Within approximately 45 minutes of the release, Agoric validators exceeded the 33.3% patching threshold per the recommendation that validators should update their nodes as quickly as possible in order to protect mainnet from exploitation. A PismoD branch will be available soon, and the patches for both issues will be added to the next release candidate (rc3) for mainnet1b and Vaults to ensure the patches are included in any consensus upgrade to promote the release candidate to mainnet.

We anticipate that the full details of this issue will be published at a later time by core Cosmos developers, and the Agoric team will be publishing a retrospective of the issue surfaced in its fork of cosmos-sdk once all impacted parties we are in contact with have patched.

If you have questions about this incident, the patching instructions, or any emergency security coordination in the future, you can reach our team directly at security@agoric.com.

HUGE thanks to Agoric validators for your fast response, prompt communication and professionalism through this patch process! We are well aware how busy this period of time is with Barberry patches on many chains and we greatly appreciate your efforts.
:pray:

We know many validators/node operators have run into apphash errors on mainnet since the soft-patch for Barberry went out last week.

If your node is experiencing issues and you have not applied the Barberry patch, we recommend installing PismoD as soon as you can.

From there, if you still run into issues and you are on PismoD or you’re using the Barberry patch instructions that bump the agoric-labs/cosmos-sdk version number, please share a copy of your [~/.agoric/data] state directory so that we can help troubleshoot and resolve whatever issues you are experiencing.

If you’re in the Agoric Opco Delegation program and you’ve experienced jailing since last Thursday, please note that your delegation is safe. The issues impacting mainnet are the result of a software bug caused by an error message in the go module package/tab that differs between patched and unpatched validators, causing apphash errors . :beetle:

1 Like