Skip to content

Commit

Permalink
Merge branch 'main' of github.com:hyperledger/besu into EIP4788
Browse files Browse the repository at this point in the history
  • Loading branch information
pinges committed Aug 22, 2023
2 parents 51d1179 + 0e55a9f commit eb9c9c4
Show file tree
Hide file tree
Showing 8 changed files with 168 additions and 32 deletions.
23 changes: 15 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Changelog

## 23.7.1
## 23.7.2

### Breaking Changes

Expand All @@ -14,25 +14,25 @@
- Make smart contract permissioning features work with london fork [#5727](https://github.com/hyperledger/besu/pull/5727)
- Add type to PendingTransactionDetail, fix eth_subscribe [#5729](https://github.com/hyperledger/besu/pull/5729)
- EvmTool "run" mode did not reflect contracts created within the transaction. [#5755](https://github.com/hyperledger/besu/pull/5755)
- Update native libraries that have JPMS friendly module names [#5749](https://github.com/hyperledger/besu/pull/5749)
- Fixing snapsync issue with forest during the heal step [#5776](https://github.com/hyperledger/besu/pull/5776)

### Download Links


## 23.7.0
## 23.7.1

### Breaking Changes
- Removed deprecated GoQuorum permissioning interop [#5607](https://github.com/hyperledger/besu/pull/5607)
- Removed support for version 0 of the database as it is no longer used by any active node. [#5698](https://github.com/hyperledger/besu/pull/5698)

### Additions and Improvements
- `evmtool` launcher binaries now ship as part of the standard distribution. [#5701](https://github.com/hyperledger/besu/pull/5701)
- `evmtool` launcher binaries now ship as part of the standard distribution. [#5701](https://github.com/hyperledger/besu/pull/5701)
- EvmTool now executes the `execution-spec-tests` via the `t8n` and `b11r`. See the [README](ethereum/evmtool/README.md) in EvmTool for more instructions.
- Improve lifecycle management of the transaction pool [#5634](https://github.com/hyperledger/besu/pull/5634)
- Add extension points in AbstractCreateOperation for EVM libraries to react to contract creations [#5656](https://github.com/hyperledger/besu/pull/5656)
- Update to Tuweni 2.4.2. [#5684](https://github.com/hyperledger/besu/pull/5684)
- Decouple data field from Enum JsonRpcError by creating new enum holder RpcErrorType[#5629](https://github.com/hyperledger/besu/pull/5629)
- Update to bouncycastle 1.75 [#5675](https://github.com/hyperledger/besu/pull/5675)
- Update to bouncycastle 1.75 [#5675](https://github.com/hyperledger/besu/pull/5675)
- Extend OperationTracer with new methods [#5662](https://github.com/hyperledger/besu/pull/5662)
- Eip 6780 selfdestruct [#5430](https://github.com/hyperledger/besu/pull/5430)
- Add new debug_getRawTransaction to the DEBUG engine [#5635](https://github.com/hyperledger/besu/pull/5635)
Expand All @@ -42,11 +42,18 @@
- Align the implementation of Eth/68 `NewPooledTransactionHashes` to other clients, using unsigned int for encoding size. [#5640](https://github.com/hyperledger/besu/pull/5640)
- Failure at startup when enabling layered txpool before initial sync done [#5636](https://github.com/hyperledger/besu/issues/5636)
- Remove miner-related option warnings if the change isn't using Ethash consensus algorithm [#5669](https://github.com/hyperledger/besu/pull/5669)
- Fix for pending transactions reference leak [#5693](https://github.com/hyperledger/besu/pull/5693)
- Fix for pending transactions reference leak [#5693](https://github.com/hyperledger/besu/pull/5693)
- Address a performance regression observed in import testing [#5734](https://github.com/hyperledger/besu/pull/5734)
- Update native libraries that have JPMS friendly module names [#5749](https://github.com/hyperledger/besu/pull/5749)

### Download Links
https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.7.0/besu-23.7.0.tar.gz / sha256: 083efc26e22fa20bd04c9a6311e50dd93092f001e5d639d023fd7a61173616dd
https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.7.0/besu-23.7.0.zip / sha256: 019a5ce3b7b94e76a6bac08bc23e3fec9880e235928b3c5378541927690046d7
https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.7.1/besu-23.7.1.tar.gz / sha256: 85dce66c2dbd21b4e5d3310770434dd373018a046b78d5037f6d4955256793cd
https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/23.7.1/besu-23.7.1.zip / sha256: dfac11b2d6d9e8076ab2f86324d48d563badf76fd2a4aadc4469a97aef374ef5


## 23.7.0

- Was not released (failed burn-in test)


## 23.4.4
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,11 @@ public Optional<Bytes> getAccountStateTrieNode(final Bytes location, final Bytes
return isClosedGet() ? Optional.empty() : super.getAccountStateTrieNode(location, nodeHash);
}

@Override
public Optional<Bytes> getTrieNodeUnsafe(final Bytes key) {
return isClosedGet() ? Optional.empty() : super.getTrieNodeUnsafe(key);
}

@Override
public Optional<Bytes> getAccountStorageTrieNode(
final Hash accountHash, final Bytes location, final Bytes32 nodeHash) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -166,32 +166,24 @@ public Optional<Bytes> getAccountStateTrieNode(final Bytes location, final Bytes
}
}

/**
* Retrieves the storage trie node associated with the specified account and location, if
* available.
*
* @param accountHash The hash of the account.
* @param location The location within the storage trie.
* @param maybeNodeHash The optional hash of the storage trie node to validate the retrieved data
* against.
* @return The optional bytes of the storage trie node.
*/
@Override
public Optional<Bytes> getAccountStorageTrieNode(
final Hash accountHash, final Bytes location, final Optional<Bytes32> maybeNodeHash) {
if (maybeNodeHash.filter(hash -> hash.equals(MerkleTrie.EMPTY_TRIE_NODE_HASH)).isPresent()) {
final Hash accountHash, final Bytes location, final Bytes32 nodeHash) {
if (nodeHash.equals(MerkleTrie.EMPTY_TRIE_NODE_HASH)) {
return Optional.of(MerkleTrie.EMPTY_TRIE_NODE);
} else {
return composedWorldStateStorage
.get(TRIE_BRANCH_STORAGE, Bytes.concatenate(accountHash, location).toArrayUnsafe())
.map(Bytes::wrap)
.filter(data -> maybeNodeHash.map(hash -> Hash.hash(data).equals(hash)).orElse(true));
.filter(b -> Hash.hash(b).equals(nodeHash));
}
}

@Override
public Optional<Bytes> getAccountStorageTrieNode(
final Hash accountHash, final Bytes location, final Bytes32 nodeHash) {
return getAccountStorageTrieNode(accountHash, location, Optional.ofNullable(nodeHash));
public Optional<Bytes> getTrieNodeUnsafe(final Bytes key) {
return composedWorldStateStorage
.get(TRIE_BRANCH_STORAGE, Bytes.concatenate(key).toArrayUnsafe())
.map(Bytes::wrap);
}

public Optional<byte[]> getTrieLog(final Hash blockHash) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ private Optional<Bytes> getTrieNode(final Bytes32 nodeHash) {
}
}

@Override
public Optional<Bytes> getTrieNodeUnsafe(final Bytes key) {
return keyValueStorage.get(key.toArrayUnsafe()).map(Bytes::wrap);
}

@Override
public FlatDbMode getFlatDbMode() {
return FlatDbMode.NO_FLATTENED;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,15 @@ public interface WorldStateStorage {

Optional<Bytes> getAccountStorageTrieNode(Hash accountHash, Bytes location, Bytes32 nodeHash);

/**
* This method allows obtaining a TrieNode in an unsafe manner, without verifying the consistency
* of the obtained node. Checks such as node hash verification are not performed here.
*
* @param key of the trie node
* @return value of the trie node
*/
Optional<Bytes> getTrieNodeUnsafe(Bytes key);

Optional<Bytes> getNodeData(Bytes location, Bytes32 hash);

FlatDbMode getFlatDbMode();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import org.hyperledger.besu.ethereum.eth.sync.snapsync.request.SnapDataRequest;
import org.hyperledger.besu.ethereum.trie.CompactEncoding;
import org.hyperledger.besu.ethereum.trie.MerkleTrie;
import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
import org.hyperledger.besu.ethereum.worldstate.FlatDbMode;
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage.Updater;
Expand Down Expand Up @@ -59,13 +60,17 @@ protected int doPersist(
@Override
public Optional<Bytes> getExistingData(
final SnapWorldDownloadState downloadState, final WorldStateStorage worldStateStorage) {
Optional<Bytes> accountStorageTrieNode =
worldStateStorage.getAccountStorageTrieNode(
getAccountHash(),
getLocation(),
null); // push null to not check the hash in the getAccountStorageTrieNode method
if (accountStorageTrieNode.isPresent()) {
return accountStorageTrieNode

final Optional<Bytes> storageTrieNode;
if (worldStateStorage.getDataStorageFormat().equals(DataStorageFormat.FOREST)) {
storageTrieNode = worldStateStorage.getTrieNodeUnsafe(getNodeHash());
} else {
storageTrieNode =
worldStateStorage.getTrieNodeUnsafe(Bytes.concatenate(getAccountHash(), getLocation()));
}

if (storageTrieNode.isPresent()) {
return storageTrieNode
.filter(node -> Hash.hash(node).equals(getNodeHash()))
.or(
() -> { // if we have a storage in database but not the good one we will need to fix
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
/*
* Copyright Hyperledger Besu Contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.ethereum.eth.sync.snapsync.request.heal;

import org.hyperledger.besu.datatypes.Address;
import org.hyperledger.besu.datatypes.Hash;
import org.hyperledger.besu.ethereum.bonsai.storage.BonsaiWorldStateKeyValueStorage;
import org.hyperledger.besu.ethereum.core.InMemoryKeyValueStorageProvider;
import org.hyperledger.besu.ethereum.core.TrieGenerator;
import org.hyperledger.besu.ethereum.eth.sync.snapsync.SnapWorldDownloadState;
import org.hyperledger.besu.ethereum.rlp.RLP;
import org.hyperledger.besu.ethereum.storage.StorageProvider;
import org.hyperledger.besu.ethereum.storage.keyvalue.WorldStateKeyValueStorage;
import org.hyperledger.besu.ethereum.trie.MerkleTrie;
import org.hyperledger.besu.ethereum.worldstate.DataStorageFormat;
import org.hyperledger.besu.ethereum.worldstate.StateTrieAccountValue;
import org.hyperledger.besu.ethereum.worldstate.WorldStateStorage;
import org.hyperledger.besu.metrics.noop.NoOpMetricsSystem;
import org.hyperledger.besu.services.kvstore.InMemoryKeyValueStorage;

import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;

import org.apache.tuweni.bytes.Bytes;
import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.ArgumentsProvider;
import org.junit.jupiter.params.provider.ArgumentsSource;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;

@ExtendWith(MockitoExtension.class)
class StorageTrieNodeHealingRequestTest {

@Mock private SnapWorldDownloadState downloadState;
final List<Address> accounts =
List.of(
Address.fromHexString("0xdeadbeef"),
Address.fromHexString("0xdeadbeee"),
Address.fromHexString("0xdeadbeea"),
Address.fromHexString("0xdeadbeeb"));

private WorldStateStorage worldStateStorage;

private Hash account0Hash;
private Hash account0StorageRoot;

static class StorageFormatArguments implements ArgumentsProvider {
@Override
public Stream<? extends Arguments> provideArguments(final ExtensionContext context) {
return Stream.of(
Arguments.of(DataStorageFormat.BONSAI), Arguments.of(DataStorageFormat.FOREST));
}
}

public void setup(final DataStorageFormat storageFormat) {
if (storageFormat.equals(DataStorageFormat.FOREST)) {
worldStateStorage = new WorldStateKeyValueStorage(new InMemoryKeyValueStorage());
} else {
final StorageProvider storageProvider = new InMemoryKeyValueStorageProvider();
worldStateStorage =
new BonsaiWorldStateKeyValueStorage(storageProvider, new NoOpMetricsSystem());
}
final MerkleTrie<Bytes, Bytes> trie =
TrieGenerator.generateTrie(
worldStateStorage,
accounts.stream().map(Address::addressHash).collect(Collectors.toList()));
account0Hash = accounts.get(0).addressHash();
account0StorageRoot =
trie.get(account0Hash)
.map(RLP::input)
.map(StateTrieAccountValue::readFrom)
.map(StateTrieAccountValue::getStorageRoot)
.orElseThrow();
}

@ParameterizedTest
@ArgumentsSource(StorageFormatArguments.class)
void shouldDetectExistingData(final DataStorageFormat storageFormat) {
setup(storageFormat);
final StorageTrieNodeHealingRequest request =
new StorageTrieNodeHealingRequest(
account0StorageRoot, account0Hash, Hash.EMPTY, Bytes.EMPTY);

Assertions.assertThat(request.getExistingData(downloadState, worldStateStorage)).isPresent();
}

@ParameterizedTest
@ArgumentsSource(StorageFormatArguments.class)
void shouldDetectMissingData(final DataStorageFormat storageFormat) {
setup(storageFormat);
final StorageTrieNodeHealingRequest request =
new StorageTrieNodeHealingRequest(Hash.EMPTY, account0Hash, Hash.EMPTY, Bytes.EMPTY);

Assertions.assertThat(request.getExistingData(downloadState, worldStateStorage)).isEmpty();
}
}
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version=23.7.1-SNAPSHOT
version=23.7.2-SNAPSHOT

org.gradle.welcome=never
# Set exports/opens flags required by Google Java Format and ErrorProne plugins. (JEP-396)
Expand Down

0 comments on commit eb9c9c4

Please sign in to comment.