Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: client merging concurrent calls that shouldn't be merged #1013

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

Farber98
Copy link
Contributor

@Farber98 Farber98 commented Jan 14, 2025

Description

This PR fixes an issue in our Solana client where certain RPC calls could be incorrectly merged by the singleflight.Group mechanism. Previously, if different calls had identical method names but varied in arguments (e.g., different commitments or slot numbers), singleflight would treat them as the same request and return the same response to all callers.

To address this

We are now incorporating request-specific parameters into the deduplication keys. This ensures that truly distinct calls (e.g., different commitments or different slot numbers) are not mistakenly merged into a single request.

https://smartcontract-it.atlassian.net/browse/NONEVM-1161

@Farber98 Farber98 changed the title fix client concurrent calls fix Solana client potentially merging concurrent calls that shouldn't be merged Jan 14, 2025
@Farber98 Farber98 changed the title fix Solana client potentially merging concurrent calls that shouldn't be merged fix: client merging concurrent calls that shouldn't be merged Jan 14, 2025
@Farber98 Farber98 marked this pull request as ready for review January 14, 2025 17:22
@Farber98 Farber98 requested a review from a team as a code owner January 14, 2025 17:22
@Farber98 Farber98 self-assigned this Jan 14, 2025
DylanTinianov
DylanTinianov previously approved these changes Jan 15, 2025
Comment on lines 190 to 196
var endSlotStr string
if endSlot == nil {
endSlotStr = "nil"
} else {
endSlotStr = fmt.Sprintf("%d", *endSlot)
}
key := fmt.Sprintf("GetBlocks(%d,%s)", startSlot, endSlotStr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT:

Suggested change
var endSlotStr string
if endSlot == nil {
endSlotStr = "nil"
} else {
endSlotStr = fmt.Sprintf("%d", *endSlot)
}
key := fmt.Sprintf("GetBlocks(%d,%s)", startSlot, endSlotStr)
endSlotStr := "nil"
if endSlot != nil {
endSlotStr = fmt.Sprint(*endSlot)
}
key := fmt.Sprintf("GetBlocks(%d,%s)", startSlot, endSlotStr)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

much better

DylanTinianov
DylanTinianov previously approved these changes Jan 15, 2025
@@ -352,7 +364,11 @@ func (c *Client) GetBlock(ctx context.Context, slot uint64) (*rpc.GetBlockResult
defer done()
ctx, cancel := context.WithTimeout(ctx, c.txTimeout)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the rest of the call can be replaced with GetBlock(ctx context.Context, slot uint64)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mean replacing:

// Adding slot to the key so concurrent calls to GetBlock for different slots are not merged. Without including the slot,
	// it would treat all GetBlock calls as identical and merge them, returning whichever block it fetched first to all callers.
	key := fmt.Sprintf("GetBlockWithOpts(%d)", slot)
	v, err, _ := c.requestGroup.Do(key, func() (interface{}, error) {
		version := uint64(0) // pull all tx types (legacy + v0)
		return c.rpc.GetBlockWithOpts(ctx, slot, &rpc.GetBlockOpts{
			Commitment:                     c.commitment,
			MaxSupportedTransactionVersion: &version,
		})
	})
	return v.(*rpc.GetBlockResult), err

with

return c.rpc.GetBlock(ctx, slot)

If that's the case I think this is not ideal as we are not providing the optional parameters. Under the hood we would be calling GetBlockWithOpts but not providing the following opts:

&rpc.GetBlockOpts{
	Commitment: c.commitment,
	MaxSupportedTransactionVersion: &version,
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry. I meant to leave this comment for the GetLatestBlock. It can look as follows:

func (c *Client) GetLatestBlock(ctx context.Context) (*rpc.GetBlockResult, error) {
	// get latest confirmed slot
	slot, err := c.SlotHeightWithCommitment(ctx, c.commitment)
	if err != nil {
		return nil, fmt.Errorf("GetLatestBlock.SlotHeight: %w", err)
	}

	// get block based on slot
	done := c.latency("latest_block")
	defer done()
	return c.GetBlock(ctx, slot)
}

Instead of

func (c *Client) GetLatestBlock(ctx context.Context) (*rpc.GetBlockResult, error) {
	// get latest confirmed slot
	slot, err := c.SlotHeightWithCommitment(ctx, c.commitment)
	if err != nil {
		return nil, fmt.Errorf("GetLatestBlock.SlotHeight: %w", err)
	}

	// get block based on slot
	done := c.latency("latest_block")
	defer done()
	ctx, cancel := context.WithTimeout(ctx, c.txTimeout)
	defer cancel()
	v, err, _ := c.requestGroup.Do("GetBlockWithOpts", func() (interface{}, error) {
		version := uint64(0) // pull all tx types (legacy + v0)
		return c.rpc.GetBlockWithOpts(ctx, slot, &rpc.GetBlockOpts{
			Commitment:                     c.commitment,
			MaxSupportedTransactionVersion: &version,
		})
	})
	return v.(*rpc.GetBlockResult), err
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch

@@ -157,7 +161,9 @@ func (c *Client) GetTransaction(ctx context.Context, txHash solana.Signature, op
opts.Commitment = c.commitment
}

v, err, _ := c.requestGroup.Do("GetTransaction", func() (interface{}, error) {
// Use txHash in the key so different signatures won't be merged on concurrent calls.
key := fmt.Sprintf("GetTransaction(%s)", txHash.String())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible to pass custom opts. We should include them into the key or remove it from the function arguments.
I'm leaning towards second option as the method is only used in one test

Copy link
Contributor Author

@Farber98 Farber98 Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Removing this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to make some changes in the new multi_client to remove the extra param

@cl-sonarqube-production
Copy link

Quality Gate failed Quality Gate failed

Failed conditions
67.7% Coverage on New Code (required ≥ 75%)

See analysis details on SonarQube

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants