Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statetrie: nibbles #5719

Closed
wants to merge 14 commits into from
Closed

statetrie: nibbles #5719

wants to merge 14 commits into from

Conversation

bbroder-algo
Copy link
Contributor

@bbroder-algo bbroder-algo commented Aug 30, 2023

This PR adds types and utility functions for manipulating nibbles (4-bit value arrays). It is specifically to support a new package, statetrie, which will use Nibbles and their associated utility functions.

The statetrie is designed to have keys with sequences of 4-bit bytes. A statetrie with 16-ary branch nodes has smaller proofs than a full byte (256-ary).

@codecov
Copy link

codecov bot commented Aug 30, 2023

Codecov Report

Merging #5719 (ff75a3e) into master (2670e94) will increase coverage by 0.06%.
Report is 12 commits behind head on master.
The diff coverage is 100.00%.

❗ Current head ff75a3e differs from pull request most recent head 4ad682d. Consider uploading reports for the commit 4ad682d to get more accurate results

@@            Coverage Diff             @@
##           master    #5719      +/-   ##
==========================================
+ Coverage   55.51%   55.57%   +0.06%     
==========================================
  Files         473      474       +1     
  Lines       66316    66378      +62     
==========================================
+ Hits        36814    36891      +77     
+ Misses      27005    26996       -9     
+ Partials     2497     2491       -6     
Files Changed Coverage Δ
crypto/statetrie/nibbles/nibbles.go 100.00% <100.00%> (ø)

... and 13 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@bbroder-algo bbroder-algo marked this pull request as ready for review August 30, 2023 23:42
Copy link
Contributor

@ahangsu ahangsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally lgtm

crypto/statetrie/nibbles.go Outdated Show resolved Hide resolved
crypto/statetrie/nibbles.go Outdated Show resolved Hide resolved
crypto/statetrie/nibbles.go Outdated Show resolved Hide resolved
crypto/statetrie/nibbles_test.go Outdated Show resolved Hide resolved
crypto/statetrie/nibbles.go Outdated Show resolved Hide resolved
crypto/statetrie/nibbles.go Outdated Show resolved Hide resolved
Copy link
Contributor

@jasonpaulos jasonpaulos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good start to the statetrie code.

However, I wonder if putting the nibble code in its own package (e.g. crypto/statetrie/nibbles or crypto/nibbles) might be beneficial. The advantages would be:

  • Since the nibble code doesn't import any other go-algorand packages (and assuming the broader crypto/statetrie package would certainly do so), it would make a nice standalone package that could be used anywhere without risk of circular imports.
  • Other packages would see its exported symbols as nibbles.*. This can simplify naming of your functions. For example, instead of equalNibbles, you could just export Equals, and consumers would see it as nibbles.Equals.

j := 0
for i := 0; i < length; i++ {
halfWidth = !halfWidth
if halfWidth {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I'd personally find i % 2 == 0 to be a clearer condition than halfWidth (and I see you use it in pack)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// [0x1, 0x2, 0x3, 0x4] -> [0x12, 0x34], false
// [0x1] -> [0x10], true
// [] -> [], false
func (ns *Nibbles) pack() ([]byte, bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use a pointer receiver?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duh
#93d95

crypto/statetrie/nibbles.go Outdated Show resolved Hide resolved
buf.WriteByte(3)
}

return buf.Bytes()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: since you know the output length, it's likely more efficient to make a byte array than use bytes.Buffer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
for i, n := range sampleNibbles {
require.True(t, bytes.Equal(shiftNibbles(n, 1), sampleNibblesShifted1[i]))
require.True(t, bytes.Equal(shiftNibbles(n, 2), sampleNibblesShifted2[i]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you test 0 shift as well? And might as well do a negative number since you built in support for that case

edit: I see you do that a few lines down. Could you relocate that closer to here? And might as well put it in the loop instead of just testing with sampleNibbles[0]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{{}, {0x1}},
}
for _, n := range sampleEqualTrue {
require.True(t, equalNibbles(n[0], n[1]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no need for the sampleEqualTrue array, you could just iterate over sampleEqualsFalse and test equalNibbles(n[0], n[0])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

require.True(t, bytes.Equal(b, sampleNibblesPacked[i]))

unp := unpack(b, half)
require.True(t, bytes.Equal(unp, n))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see more usage of require.Equals and related purpose-driven functions over require.True. There are two benefits to this:

  1. (subjective) Assertion code is simpler and easier to read
  2. (objective) When an error occurs, more context is printed in the error message, which makes it way easier to debug. With require.True, all we'd get is a message that says something like "got false when expected true"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ahangsu ahangsu self-requested a review August 31, 2023 19:32

func TestNibbles(t *testing.T) { // nolint:paralleltest // Serial tests for trie for the moment
partitiontest.PartitionTest(t)
// t.Parallel()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not parallel? there is not external dependencies

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, #a33795

crypto/statetrie/nibbles/nibbles_test.go Show resolved Hide resolved
Copy link
Contributor

@ahangsu ahangsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

traverse back for another round of review

crypto/statetrie/nibbles/nibbles.go Outdated Show resolved Hide resolved
crypto/statetrie/nibbles/nibbles.go Outdated Show resolved Hide resolved
crypto/statetrie/nibbles/nibbles_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@algorandskiy algorandskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for the test update, one more suggestion of printing the seed in case of failure.

crypto/statetrie/nibbles/nibbles_test.go Outdated Show resolved Hide resolved
@algorandskiy algorandskiy mentioned this pull request Sep 28, 2023
@algorandskiy
Copy link
Contributor

superseeded by #5759

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants