Upstream sync - v2.44.0 #78
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There are currently a number of bugs in the flat DB trie hash computation. These bugs are improbable in 'real' scenarios (hence the ability to sync the assorted large chains), and, the repercussions of encountering them are low (generally re-computing the hash at a later block will succeed). Still, they can cause the process to crash, or deadlock, and a clever adversary could feasibly construct a block or storage update designed to trigger these bugs. (Or, an unlucky block may trigger them inadvertently). Based on the tracing in the code, it seems that some of these bugs may have been witnessed in the wild, but not reliably reproduced (for instance when `maxlen >= len(curr)`). 1. There is an infinite loop that can occur in _nextSiblingOfParentInMem. This occurs in the account path when the c.k[1] entry is 'nil' and the code will continuously retry resolving this entry indefinitely. 2. When the next trie node is deeper in the trie than the previous, but is not a descendent of the previous trie node, then the old trie node is inadvertently left in memory instead of nil-ed. This results in an incorrect hash being computed. 3. When the last trie node being processed is comprised entirely of 'f' nibbles, the 'FirstNotCoveredPrefix' returns an empty byte array, because there is no next nibble sub-tree. This causes keys to inappropriately be reprocessed triggering either an index out of bounds panic or incorrect hash results. 4. When the _nextSiblingInDB path is triggered, if the next nibble subtree contains no trie table entries, then any keys with a prefix in that subtree will be skipped resulting in incorrect hash results. The fuzzing seeds included cover all four of these cases (for both accounts and storage where appropriate). As fuzzing is baked into the native go 1.18 toolchain, running 'go test' (as the Makefile currently does) is adequate to cover these failures. To run additional fuzzing the `-fuzz` flag may be provided as documented in the go test doc. Co-authored-by: Jason Yellick <[email protected]>
This PR extends the function merged in erigontech#7337 to cover the proof generation paths as well. The first commit simply migrates the internal proof checking code from the rpc `eth_getProof` test to be available externally exported under `turbo/trie`. In the process, it translates the in place testing assertions to return more normally as errors and adapts to use the pre-existing trie node types which are defined slightly differently (although this code is still intended only for testing purposes). The second commit refactors the existing trie fuzzing tests from the previous PR and adds new fuzzing tests for the proof generation. In order to validate the proofs, these fuzzing tests fundamentally do two things. Firstly, after bootstrapping the test (by seeding, and modifying the db), for each key we compute the 'naive' proof utilizing the existing code in `proof.go` and the in memory `trie.Trie` structure. The `trie.Trie` code actually had a couple small bugs which are fixed in this PR (not handling value nodes, and not honoring the `NewTestRLPTrie` contract of pre-RLP encoded nodes in proof generation). Secondly, we re-compute the same proof for the flatDB production variant, and verify that it is exactly the same proof as computed by the naive implementation. This fuzzing has been run for ~72 hours locally with no errors. Although this code hasn't identified any new bugs in the proof generation path, it improves coverage and should help to prevent regressions. Additional extensions will be provided in a subsequent PR. --------- Co-authored-by: Jason Yellick <[email protected]>
This addresses the last known deficiency of the eth_getProof implementation. The previous code would return an error in the event that the element was not found in the trie. EIP-1186 allows for 'negative' proofs where a proof demonstrates that an element cannot be in the trie, so this commit updates the logic to support that case. Co-authored-by: Jason Yellick <[email protected]>
Co-authored-by: Alex Sharp <[email protected]>
…h#7385) Small additions to block receipts. Now it includes `dataGasPrice` and `dataGasUsed`. `dataGas` is a new type of a gas and blobs are priced in it.
Improved CPU usage by 10x, we just really avoid re-compressing and re-validating public keys
This PR adds the new p2p containers from Deneb into Caplin erigontech#7389
Transaction extension to support BlobTxWrapper (or network representation of BlobTx). BlobTxWrapper wraps BlobTx with additional data: blobs, commitments and proofs. Unwrap() returns unwrapped tx if its network tx, otherwise returns itself.
Small additions to support eip-4844 transaction marshalling
…l Erigon's databases (erigontech#7415)
To expose context flags through diagnostic endpoint `/debug/metrics/flags`. The current `/debug/metrics/cmdline` only provides the command run by user. In the case when user runs Erigon using config file, `cmdline`’s info does not truly reflect the ‘launch setting' and flags set that Erigon is running on. ## Example ### Command ``` erigon --datadir /tmp/data --config test.yaml ``` ### Pseudo config file (in yaml) ``` datadir : '/tmp/data' chain : "sepolia" http : true metrics: true private.api.addr : "localhost:9090" http.api : ["eth","debug","net"] ``` ### Output ``` SUCCESS chain=sepolia config=test.yaml datadir=/tmp/data http=true http.api=eth,debug,net metrics=true private.api.addr=localhost:9090 ```
…db (erigontech#7607) it's step towards saving canonical and non-canonical bodies in same table (and txs also in same own table). to reduce write amplification (cheaper re-orgs) PR change: reading BaseTxNum from existing snapshots instead of DB DB will store in field body.BaseTxNum - non-canonical TxnID Snapshots will store only canonical TxNum in field body.BaseTxNum
Co-authored-by: Alex Sharp <[email protected]>
pcw109550
approved these changes
Jul 17, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Upstream sync PR
erigon
ledgerwatch/erigonv2.44.0 ->testinprod-io/op-erigon0fb48c7erigon-lib
ledgerwatch/erigon-libf166e7d ->testinprod-io/erigon-lib5340344erigon-interfaces
ledgerwatch/interfaces4862127 ->testinprod-io/erigon-interfaces9393a05