Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated KeyBuilder Fast Access #3768

Merged
merged 10 commits into from
Feb 20, 2025
Merged

Updated KeyBuilder Fast Access #3768

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
af73cb5
Updated `KeyBuilder` Fast Access
cschuchardt88 Feb 15, 2025
5b3a394
Update
cschuchardt88 Feb 15, 2025
bfe7c1a
Merge branch 'master' into fix/keybuilder-fast
cschuchardt88 Feb 15, 2025
c8eb900
Update src/Neo/SmartContract/KeyBuilder.cs
shargon Feb 15, 2025
c56c005
fixed bug
cschuchardt88 Feb 15, 2025
49e113b
Merge changes
cschuchardt88 Feb 15, 2025
cb63144
Merge branch 'master' into fix/keybuilder-fast
cschuchardt88 Feb 15, 2025
002ca6a
Merge branch 'master' into fix/keybuilder-fast
cschuchardt88 Feb 18, 2025
2f82d0b
Update src/Neo/SmartContract/KeyBuilder.cs
cschuchardt88 Feb 19, 2025
422ec20
Merge branch 'master' into fix/keybuilder-fast
NGDAdmin Feb 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 0 additions & 13 deletions benchmarks/Neo.Benchmarks/SmartContract/Benchmarks.StorageKey.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,19 +34,6 @@ public void KeyBuilder_AddInt()
throw new InvalidOperationException();
}

[Benchmark]
public void KeyBuilder_AddIntWithoutPrealloc()
{
var key = new KeyBuilder(1, 0, 0)
.AddBigEndian(1)
.AddBigEndian(2)
.AddBigEndian(3);

var bytes = key.ToArray();
if (bytes.Length != prefixSize + 3 * sizeof(int))
throw new InvalidOperationException();
}

[Benchmark]
public void KeyBuilder_AddBytes()
{
Expand Down
51 changes: 29 additions & 22 deletions src/Neo/SmartContract/KeyBuilder.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
using Neo.IO;
using System;
using System.Buffers.Binary;
using System.IO;
using System.Runtime.CompilerServices;

namespace Neo.SmartContract
Expand All @@ -24,7 +23,9 @@ namespace Neo.SmartContract
/// </summary>
public class KeyBuilder
{
private readonly MemoryStream _stream;
private readonly Memory<byte> _cacheData;
private int _keyLength = 0;


/// <summary>
/// Initializes a new instance of the <see cref="KeyBuilder"/> class.
Expand All @@ -34,12 +35,20 @@ public class KeyBuilder
/// <param name="keySizeHint">The hint of the storage key size(including the id and prefix).</param>
public KeyBuilder(int id, byte prefix, int keySizeHint = ApplicationEngine.MaxStorageKeySize)
{
Span<byte> data = stackalloc byte[sizeof(int)];
BinaryPrimitives.WriteInt32LittleEndian(data, id);
if (keySizeHint < 0)
throw new ArgumentOutOfRangeException(nameof(keySizeHint));

_cacheData = new byte[keySizeHint];
BinaryPrimitives.WriteInt32LittleEndian(_cacheData.Span, id);

_keyLength = sizeof(int);
_cacheData.Span[_keyLength++] = prefix;
}

_stream = new(keySizeHint);
_stream.Write(data);
_stream.WriteByte(prefix);
private void CheckLength(int length)
{
if ((length + _keyLength) > _cacheData.Length)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Increase it?

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? Is not >= or what do you mean

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, increase the array instead of throw an error

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the Length is passed in, we need to enforce that length. 99% of the time we know the length up front?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise is not a hint, and is not working as before, it should not happen so it won't have any impact in the performance

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had @superboyiii test this on mainnet and testnet, no key is bigger than 64 bytes. So we need to enforce. If you want bigger key you must know the size before hand.

#3705 (review)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It haven't got any sense to me, throw an exception when is possible to increase the buffer, usability vs optimization? Could you clarify why we should not increase the buffer and preserve the previous behavior?

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already did state that. You pass in a LENGTH. What's the point of doing that if it not enforced? Im sick of everything being programmed around these bugs. It needs to stop. Have some discipline.

throw new OverflowException("Input data too Large!");
}

/// <summary>
Expand All @@ -50,7 +59,8 @@ public KeyBuilder(int id, byte prefix, int keySizeHint = ApplicationEngine.MaxSt
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public KeyBuilder Add(byte key)
{
_stream.WriteByte(key);
CheckLength(1);
_cacheData.Span[_keyLength++] = key;
return this;
}

Expand All @@ -62,7 +72,9 @@ public KeyBuilder Add(byte key)
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public KeyBuilder Add(ReadOnlySpan<byte> key)
{
_stream.Write(key);
CheckLength(key.Length);
key.CopyTo(_cacheData.Span[_keyLength..]);
_keyLength += key.Length;
return this;
}

Expand All @@ -72,14 +84,16 @@ public KeyBuilder Add(ReadOnlySpan<byte> key)
/// <param name="key">Part of the key represented by a byte array.</param>
/// <returns>A reference to this instance after the add operation has completed.</returns>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public KeyBuilder Add(byte[] key) => Add(key.AsSpan());
public KeyBuilder Add(byte[] key) =>
Add(key.AsSpan());

/// <summary>
/// Adds part of the key to the builder.
/// </summary>
/// <param name="key">Part of the key.</param>
/// <returns>A reference to this instance after the add operation has completed.</returns>
public KeyBuilder Add(ISerializableSpan key) => Add(key.GetSpan());
public KeyBuilder Add(ISerializableSpan key) =>
Add(key.GetSpan());

/// <summary>
/// Adds part of the key to the builder in BigEndian.
Expand Down Expand Up @@ -141,18 +155,11 @@ public KeyBuilder AddBigEndian(ulong key)
/// Gets the storage key generated by the builder.
/// </summary>
/// <returns>The storage key.</returns>
public byte[] ToArray()
{
using (_stream)
{
return _stream.ToArray();
}
}
public byte[] ToArray() =>
_cacheData[.._keyLength].ToArray();

public static implicit operator StorageKey(KeyBuilder builder)
{
return new StorageKey(builder.ToArray());
}
public static implicit operator StorageKey(KeyBuilder builder) =>
new(builder._cacheData[..builder._keyLength].ToArray());
}
}

Expand Down