Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated KeyBuilder Fast Access #3768

Merged
merged 10 commits into from
Feb 20, 2025
13 changes: 0 additions & 13 deletions benchmarks/Neo.Benchmarks/SmartContract/Benchmarks.StorageKey.cs
Original file line number Diff line number Diff line change
Expand Up @@ -34,19 +34,6 @@ public void KeyBuilder_AddInt()
throw new InvalidOperationException();
}

[Benchmark]
public void KeyBuilder_AddIntWithoutPrealloc()
{
var key = new KeyBuilder(1, 0, 0)
.AddBigEndian(1)
.AddBigEndian(2)
.AddBigEndian(3);

var bytes = key.ToArray();
if (bytes.Length != prefixSize + 3 * sizeof(int))
throw new InvalidOperationException();
}

[Benchmark]
public void KeyBuilder_AddBytes()
{
Expand Down
51 changes: 29 additions & 22 deletions src/Neo/SmartContract/KeyBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
using Neo.IO;
using System;
using System.Buffers.Binary;
using System.IO;
using System.Runtime.CompilerServices;

namespace Neo.SmartContract
Expand All @@ -24,7 +23,9 @@ namespace Neo.SmartContract
/// </summary>
public class KeyBuilder
{
private readonly MemoryStream _stream;
private readonly Memory<byte> _cacheData;
private int _keyLength = 0;


/// <summary>
/// Initializes a new instance of the <see cref="KeyBuilder"/> class.
Expand All @@ -34,12 +35,20 @@ public class KeyBuilder
/// <param name="keySizeHint">The hint of the storage key size(including the id and prefix).</param>
public KeyBuilder(int id, byte prefix, int keySizeHint = ApplicationEngine.MaxStorageKeySize)
{
Span<byte> data = stackalloc byte[sizeof(int)];
BinaryPrimitives.WriteInt32LittleEndian(data, id);
if (keySizeHint < 0)
throw new ArgumentOutOfRangeException(nameof(keySizeHint));

_cacheData = new byte[keySizeHint];
BinaryPrimitives.WriteInt32LittleEndian(_cacheData.Span, id);

_keyLength = sizeof(int);
_cacheData.Span[_keyLength++] = prefix;
}

_stream = new(keySizeHint);
_stream.Write(data);
_stream.WriteByte(prefix);
private void CheckLength(int length)
{
if ((length + _keyLength) > _cacheData.Length)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Increase it?

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? Is not >= or what do you mean

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, increase the array instead of throw an error

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the Length is passed in, we need to enforce that length. 99% of the time we know the length up front?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise is not a hint, and is not working as before, it should not happen so it won't have any impact in the performance

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had @superboyiii test this on mainnet and testnet, no key is bigger than 64 bytes. So we need to enforce. If you want bigger key you must know the size before hand.

#3705 (review)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It haven't got any sense to me, throw an exception when is possible to increase the buffer, usability vs optimization? Could you clarify why we should not increase the buffer and preserve the previous behavior?

Copy link
Member Author

@cschuchardt88 cschuchardt88 Feb 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already did state that. You pass in a LENGTH. What's the point of doing that if it not enforced? Im sick of everything being programmed around these bugs. It needs to stop. Have some discipline.

throw new OverflowException("Input data too Large!");
}

/// <summary>
Expand All @@ -50,7 +59,8 @@ public KeyBuilder(int id, byte prefix, int keySizeHint = ApplicationEngine.MaxSt
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public KeyBuilder Add(byte key)
{
_stream.WriteByte(key);
CheckLength(1);
_cacheData.Span[_keyLength++] = key;
return this;
}

Expand All @@ -62,7 +72,9 @@ public KeyBuilder Add(byte key)
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public KeyBuilder Add(ReadOnlySpan<byte> key)
{
_stream.Write(key);
CheckLength(key.Length);
key.CopyTo(_cacheData.Span[_keyLength..]);
_keyLength += key.Length;
return this;
}

Expand All @@ -72,14 +84,16 @@ public KeyBuilder Add(ReadOnlySpan<byte> key)
/// <param name="key">Part of the key represented by a byte array.</param>
/// <returns>A reference to this instance after the add operation has completed.</returns>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public KeyBuilder Add(byte[] key) => Add(key.AsSpan());
public KeyBuilder Add(byte[] key) =>
Add(key.AsSpan());

/// <summary>
/// Adds part of the key to the builder.
/// </summary>
/// <param name="key">Part of the key.</param>
/// <returns>A reference to this instance after the add operation has completed.</returns>
public KeyBuilder Add(ISerializableSpan key) => Add(key.GetSpan());
public KeyBuilder Add(ISerializableSpan key) =>
Add(key.GetSpan());

/// <summary>
/// Adds part of the key to the builder in BigEndian.
Expand Down Expand Up @@ -141,18 +155,11 @@ public KeyBuilder AddBigEndian(ulong key)
/// Gets the storage key generated by the builder.
/// </summary>
/// <returns>The storage key.</returns>
public byte[] ToArray()
{
using (_stream)
{
return _stream.ToArray();
}
}
public byte[] ToArray() =>
_cacheData[.._keyLength].ToArray();

public static implicit operator StorageKey(KeyBuilder builder)
{
return new StorageKey(builder.ToArray());
}
public static implicit operator StorageKey(KeyBuilder builder) =>
new(builder._cacheData[..builder._keyLength].ToArray());
}
}

Expand Down