fix: Handle unmaterialize set endings in UTF8FaunaReader

[pnwpedro]

Description

• Handle unmaterialized set endings by tracking an internal token type on the stack and buffering the value
• Handle unmaterialized set tokens by buffering the value. This is the first case where we need to buffer a token that has an associated value. This implementation is naive, but that's intentional due to time constraints.

Motivation and context

BT-5260

Materialized and unmaterialized sets look slightly different on the wire. In the case of materialized sets, we encounter two end object json tokens that represent the end of the set. In the unmaterialized case, we only encounter one. When the reader encounters the unmaterialized set, it will cause the token stream to be shifted by one.

How was the change tested?

Added unit test covering the scenario.

Screenshots (if appropriate):

Change types

• • Bug fix (non-breaking change that fixes an issue)
• • New feature (non-breaking change that adds functionality)
• • Breaking change (backwards-incompatible fix or feature)

Checklist:

• • My code follows the code style of this project.
• • My change requires a change to Fauna documentation.
• • My change requires a change to the README, and I have updated it accordingly.

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

missed part of the fix

[github-actions]

Summary

Summary

Generated on:	10/22/2024 - 23:42:40
Parser:	OpenCover
Assemblies:	1
Classes:	121
Files:	95
Line coverage:	80.3% (3243 of 4035)
Covered lines:	3243
Uncovered lines:	792
Coverable lines:	4035
Total lines:	8980
Branch coverage:	70.4% (1052 of 1494)
Covered branches:	1052
Total branches:	1494
Method coverage:	Feature is only available for sponsors
Tag:	74ae1b1

Coverage

Fauna - 80.3%

Name	Line	Branch
Fauna	80.3%	70.4%
Fauna.BaseClient	83.3%	72.2%
Fauna.Client	73.4%	52.9%
Fauna.Configuration	100%	100%
Fauna.Core.Connection	95.3%	72.9%
Fauna.Core.Endpoints	100%	100%
Fauna.Core.ErrorInfo	100%
Fauna.Core.QueryFailure	100%	66.6%
Fauna.Core.QueryOptions	82.1%	81.8%
Fauna.Core.QueryResponse	70%	77.2%
Fauna.Core.QueryStats	57.1%
Fauna.Core.QuerySuccess`1	69.2%	50%
Fauna.Core.RetryConfiguration	100%	100%
Fauna.Core.Stats	100%
Fauna.Core.StatsCollector	86.2%	100%
Fauna.Core.StreamEnumerable`1	100%	66.6%
Fauna.Exceptions.AbortException	100%	83.3%
Fauna.Exceptions.AuthenticationException	100%
Fauna.Exceptions.AuthorizationException	100%
Fauna.Exceptions.BadGatewayException	0%
Fauna.Exceptions.ConstraintFailure	100%
Fauna.Exceptions.ConstraintFailureException	100%
Fauna.Exceptions.ContendedTransactionException	100%
Fauna.Exceptions.ExceptionHandler	84.3%	89.3%
Fauna.Exceptions.FaunaException	25%
Fauna.Exceptions.InvalidRequestException	100%
Fauna.Exceptions.NetworkException	100%
Fauna.Exceptions.NullDocumentException	100%	100%
Fauna.Exceptions.ProtocolException	0%
Fauna.Exceptions.QueryCheckException	100%
Fauna.Exceptions.QueryRuntimeException	100%
Fauna.Exceptions.QueryTimeoutException	100%
Fauna.Exceptions.SerializationException	100%
Fauna.Exceptions.ServiceException	80.9%
Fauna.Exceptions.ThrottlingException	42.8%
Fauna.Exceptions.TimeoutException	100%
Fauna.IQueryFragmentExtensions	0%
Fauna.Linq.DataContext	84.1%	87.5%
Fauna.Linq.DataContextBuilder`1	92.1%	86.6%
Fauna.Linq.IntermediateQueryHelpers	80%	83.3%
Fauna.Linq.IPipelineExecutor	55.2%	28.2%
Fauna.Linq.IQuerySource	0%	0%
Fauna.Linq.LookupTable	40%	16.6%
Fauna.Linq.MappedDeserializer`2	66.6%
Fauna.Linq.Pipeline	100%	100%
Fauna.Linq.ProjectionAnalysisVisitor	96.1%	87.5%
Fauna.Linq.ProjectionDeserializer	77.2%	58.3%
Fauna.Linq.ProjectionRewriteVisitor	100%	100%
Fauna.Linq.QuerySource	100%
Fauna.Linq.QuerySource`1	93.7%	90.7%
Fauna.Linq.QuerySourceExtensions	25%	50%
Fauna.Linq.SubQuerySwitch	56.7%	19.4%
Fauna.Mapping.BaseFieldAttribute	100%
Fauna.Mapping.CollectionAttribute	100%
Fauna.Mapping.FieldAttribute	100%
Fauna.Mapping.FieldInfo	100%	91.6%
Fauna.Mapping.FieldName	100%	100%
Fauna.Mapping.IdAttribute	100%	50%
Fauna.Mapping.IgnoreAttribute	100%
Fauna.Mapping.MappingContext	100%	100%
Fauna.Mapping.MappingInfo	96.5%	100%
Fauna.Mapping.TsAttribute	100%
Fauna.Query	100%
Fauna.QueryArr	58.6%	50%
Fauna.QueryExpr	80%	50%
Fauna.QueryLiteral	86.1%	62.5%
Fauna.QueryObj	32.1%	0%
Fauna.QueryStringHandler	100%	100%
Fauna.QueryVal	79.5%	55.5%
Fauna.Serialization.BaseRefSerializer`1	81.8%	78.5%
Fauna.Serialization.BaseSerializer`1	21%	16.6%
Fauna.Serialization.BooleanSerializer	92.8%	83.3%
Fauna.Serialization.ByteSerializer	92.8%	83.3%
Fauna.Serialization.BytesSerializer	92.8%	83.3%
Fauna.Serialization.ClassSerializer`1	83.1%	80.9%
Fauna.Serialization.DateOnlySerializer	92.8%	83.3%
Fauna.Serialization.DateTimeOffsetSerializer	92.8%	83.3%
Fauna.Serialization.DateTimeSerializer	92.8%	83.3%
Fauna.Serialization.DictionarySerializer`1	82.3%	75%
Fauna.Serialization.DoubleSerializer	92.8%	83.3%
Fauna.Serialization.DynamicSerializer	72.9%	76.7%
Fauna.Serialization.FloatSerializer	92.8%	83.3%
Fauna.Serialization.IntSerializer	92.8%	83.3%
Fauna.Serialization.ListSerializer`1	82.9%	66.6%
Fauna.Serialization.LongSerializer	92.8%	83.3%
Fauna.Serialization.ModuleSerializer	28.5%	16.6%
Fauna.Serialization.NamedRefSerializer`1	90.9%
Fauna.Serialization.NullableSerializer`1	92.8%	100%
Fauna.Serialization.NullableStructSerializer`1	90.9%	100%
Fauna.Serialization.PageSerializer`1	89.7%	87.5%
Fauna.Serialization.QueryArrSerializer	0%	0%
Fauna.Serialization.QueryExprSerializer	77.7%	66.6%
Fauna.Serialization.QueryLiteralSerializer	55.5%	50%
Fauna.Serialization.QueryObjSerializer	69.2%	50%
Fauna.Serialization.QuerySerializer	60%	50%
Fauna.Serialization.QueryValSerializer	69.2%	50%
Fauna.Serialization.RefSerializer`1	90.9%
Fauna.Serialization.SByteSerializer	92.8%	83.3%
Fauna.Serialization.Serializer	90.4%	82.6%
Fauna.Serialization.ShortSerializer	92.8%	83.3%
Fauna.Serialization.StreamSerializer	50%	50%
Fauna.Serialization.StringSerializer	93.3%	87.5%
Fauna.Serialization.TokenTypeExtensions	35%	20.8%
Fauna.Serialization.UIntSerializer	92.8%	83.3%
Fauna.Serialization.UShortSerializer	92.8%	83.3%
Fauna.Serialization.Utf8FaunaReader	73.3%	78.8%
Fauna.Serialization.Utf8FaunaWriter	90.6%
Fauna.StreamOptions	61.5%
Fauna.Types.BaseRef`1	55%
Fauna.Types.BaseRefBuilder`1	95%	81.8%
Fauna.Types.Event`1	85.4%	66.6%
Fauna.Types.Module	52.6%	20%
Fauna.Types.NamedRef`1	60%	62.5%
Fauna.Types.Page`1	100%
Fauna.Types.Ref`1	60%	62.5%
Fauna.Types.Stream	66.6%	35.7%
Fauna.Util.DefaultExpressionSwitch`1	0%
Fauna.Util.Expressions	100%	100%
Fauna.Util.ExpressionSwitch`1	84.8%	75.5%
Fauna.Util.Extensions.PaginationExtensions	100%	100%
Fauna.Util.Extensions.TypeExtensions	93.3%	81.2%
Fauna.Util.Logger	78.7%	40%

Delta Summary

Generated on:	10/22/2024 - 23:42
Description	Previous	Current	Delta
Coverage date:	10/22/2024 - 23:41	10/22/2024 - 23:42
Tag:	main	74ae1b1
Line coverage:	80.3%	80.3%	0.0%
Covered lines:	3230	3243	13
Coverable lines:	4022	4035	13
Total lines:	16427	16445	18
Branch coverage:	70.2%	70.4%	0.2%
Covered branches:	1046	1052	6
Total branches:	1488	1494	6
Method coverage:	Feature is only available for sponsors