-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Batch Amendment #5060
base: develop
Are you sure you want to change the base?
Batch Amendment #5060
Conversation
This is not finished. Needs to only revert the transactions submitted not the entire open ledger.
Also, since it seems this PR is still under active development, could you convert it to a draft? |
|
||
env(noop(bob), ter(tesSUCCESS)); | ||
auto const carol = Account("carol"); | ||
env.fund(XRP(220), alice, bob, carol); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would prefer not hardcoding the values since they are subjected to change, and it's hard for the reviewer to know the exact numbers. Should use following variables for account funding:
auto const acctReserve = env.current()->fees().accountReserve(0);
auto const incReserve = env.current()->fees().increment;
auto const fee = env.current()->fees().base;
env.fund(acctReserve + ..., alice, bob, carol);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(feel free to leave these test-related comments for later, since they are less urgent)
batch::sig(bob), | ||
ter(telINSUF_FEE_P)); | ||
// Using 1 XRP to create insufficient reserve result | ||
auto const batchFee = XRP(1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto. we should specify the batchFee
relative to the fee variables, so it's easier to understand, something like
auto const acctReserve = env.current()->fees().accountReserve(0);
auto const incReserve = env.current()->fees().increment;
auto const fee = env.current()->fees().base;
...
...
...
auto const batchFee = fee * 2; // Just making this up, should change the value accordingly
env(batch::batch(alice, seq, batchFee, tfOnlyOne), | ||
batch::add(pay(alice, bob, XRP(999)), seq + 1), | ||
batch::add(pay(alice, bob, XRP(1)), seq + 2), | ||
batch::add(pay(alice, bob, XRP(1)), seq + 3), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: doesn't change the test behavior, prefer to change the amount of the unprocessed txn to be different from the previous txn
batch::add(pay(alice, bob, XRP(1)), seq + 3), | |
batch::add(pay(alice, bob, XRP(2)), seq + 3), |
// BEAST_EXPECT(jrr[jss::meta][sfParentBatchID.jsonName] == | ||
// batchId); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this commented out?
jtx::Account account; | ||
}; | ||
|
||
Json::Value |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Json::Value | |
std::optional<Json::Value> |
BEAST_EXPECT(transactions.size() == batchResults.size() + 1); | ||
|
||
// Validate ttBatch is correct index | ||
auto const txn = getTxByIndex(jrr, 0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this assuming that Batch transactions are the only ones in the ledger?
Json::Value jsonTx; | ||
jsonTx[jss::binary] = false; | ||
jsonTx[jss::transaction] = batchResult.txHash; | ||
jsonTx[jss::id] = 1; | ||
Json::Value const jrr = | ||
env.rpc("json", "tx", to_string(jsonTx))[jss::result]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Json::Value jsonTx; | |
jsonTx[jss::binary] = false; | |
jsonTx[jss::transaction] = batchResult.txHash; | |
jsonTx[jss::id] = 1; | |
Json::Value const jrr = | |
env.rpc("json", "tx", to_string(jsonTx))[jss::result]; | |
Json::Value const jrr = | |
env.rpc("tx", batchResult.txHash)[jss::result]; |
std::uint32_t sequence, | ||
std::optional<std::uint32_t> ticket = std::nullopt) | ||
{ | ||
std::uint32_t const index = jv[jss::RawTransactions].size(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why doesn't this use batch
in jtx/batch.h
?
XRPAmount | ||
calcBatchFee( | ||
test::jtx::Env const& env, | ||
uint32_t const& signers, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
uint32_t const& signers, | |
uint32_t const& numSigners, |
batch( | ||
jtx::Account const& account, | ||
uint32_t seq, | ||
STAmount const& fee, | ||
std::uint32_t flags); | ||
|
||
/** Adds a new Batch Txn on a JTx. */ | ||
class add |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thoughts on calling these outer
and inner
instead?
env(batch::batch(alice, seq, batchFee, tfAllOrNothing), | ||
txflags(tfDisallowXRP), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this not be simplified like this?
env(batch::batch(alice, seq, batchFee, tfAllOrNothing), | |
txflags(tfDisallowXRP), | |
env(batch::batch(alice, seq, batchFee, tfDisallowXRP), |
{ | ||
JLOG(ctx.j.trace()) | ||
<< "BatchTrace[" << batchId << "]:" << "too many flags."; | ||
return temMALFORMED; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should probably also return temINVALID_FLAG
singleSignHelper( | ||
STObject const& signer, | ||
Slice const& data, | ||
STTx::RequireFullyCanonicalSig requireCanonicalSig, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this param can be removed, since rules
is a global variable now.
That looks like it might get complicated, though, so I'm fine if you don't want to touch that in this PR.
// We don't allow both a non-empty sfSigningPubKey and an sfSigners. | ||
// That would allow the transaction to be signed two ways. So if both | ||
// fields are present the signature is invalid. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment should stay.
multiSignHelper( | ||
STArray const& signers, | ||
AccountID const& txnAccountID, | ||
bool const fullyCanonical, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO this should be aligned with singleSignHelper
and either both should have the fullyCanonical
bool or both should use the STTx::RequireFullyCanonicalSig
value.
// Make sure the MultiSigners are present. Otherwise they are not | ||
// attempting multi-signing and we just have a bad SigningPubKey. | ||
if (!batchSigner.isFieldPresent(sfSigners)) | ||
return Unexpected("Empty SigningPubKey."); | ||
|
||
// We don't allow both an sfSigners and an sfTxnSignature. Both fields | ||
// being present would indicate that the transaction is signed both ways. | ||
if (batchSigner.isFieldPresent(sfTxnSignature)) | ||
return Unexpected("Cannot both single- and multi-sign."); | ||
|
||
STArray const& signers{batchSigner.getFieldArray(sfSigners)}; | ||
|
||
// There are well known bounds that the number of signers must be within. | ||
if (signers.size() < minMultiSigners || | ||
signers.size() > maxMultiSigners(&rules)) | ||
return Unexpected("Invalid Signers array size."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a lot of these checks are identical between checkBatchMultiSign
and checkMultiSign
- can they be incorporated into the multiSignHelper
function? Should also have the side benefit of making the diff smaller.
@@ -36,13 +36,35 @@ class ApplyContext | |||
{ | |||
public: | |||
explicit ApplyContext( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should these constructors also have assert checks?
std::optional<uint256 const> const& batchId, | ||
STTx const& tx_, | ||
TER preclaimResult_, | ||
XRPAmount baseFee_, | ||
ApplyFlags flags, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One more underscore missed
STTx const& tx_, | ||
TER preclaimResult_, | ||
XRPAmount baseFee_, | ||
ApplyFlags flags, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One more underscore missed
src/xrpld/app/tx/detail/Batch.cpp
Outdated
if (stx.isFieldPresent(sfTxnSignature)) | ||
{ | ||
JLOG(ctx.j.trace()) | ||
<< "BatchTrace[" << batchId | ||
<< "]:" << "inner txn cannot include TxnSignature." | ||
<< "index: " << i; | ||
return temINVALID_BATCH; | ||
} | ||
|
||
if (!stx.getSigningPubKey().empty()) | ||
{ | ||
JLOG(ctx.j.trace()) | ||
<< "BatchTrace[" << batchId | ||
<< "]:" << "inner txn must include empty SigningPubKey." | ||
<< "index: " << i; | ||
return temINVALID_BATCH; | ||
} | ||
|
||
if (stx.isFieldPresent(sfSigners)) | ||
{ | ||
JLOG(ctx.j.trace()) << "BatchTrace[" << batchId | ||
<< "]:" << "inner txn cannot include Signers." | ||
<< "index: " << i; | ||
return temINVALID_BATCH; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These can all be combined into one if
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed: accc8e0
} | ||
|
||
Json::Value | ||
addBatchTx( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why isn't this function (and the below ones) part of the jtx
helper file? Seems like there's some overlap there.
|
||
// temINVALID_BATCH: Batch: TransactionType missing in array entry. | ||
{ | ||
auto const txBlob = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test is hard to audit. Can it be written to use the existing JSON structures, but instead of using batch::add
, the JSON is constructed by hand?
env(batch::batch(alice, seq, batchFee, tfAllOrNothing), | ||
batch::add(pay(alice, bob, XRP(10)), seq + 1), | ||
batch::add(tx, seq + 2), | ||
ter(tesSUCCESS)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this test fail with temINVALID_FEE
or something?
// This function is called by several different parts of the codebase | ||
// under no circumstances will we ever accept an inner txn within a batch | ||
// txn from the network. | ||
auto const tx = *transaction->getSTransaction(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
auto const tx = *transaction->getSTransaction(); | |
auto const sttx = *transaction->getSTransaction(); |
consider renaming this since tx
conflicts with transaction
@@ -1475,12 +1495,13 @@ NetworkOPsImp::apply(std::unique_lock<std::mutex>& batchLock) | |||
auto const toSkip = | |||
app_.getHashRouter().shouldRelay(e.transaction->getID()); | |||
|
|||
if (toSkip) | |||
if (auto const txn = *(e.transaction->getSTransaction()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (auto const txn = *(e.transaction->getSTransaction()); | |
if (auto const sttx = *(e.transaction->getSTransaction()); |
consider renaming txn
since it conflicts with tx
{ | ||
protocol::TMTransaction tx; | ||
Serializer s; | ||
|
||
e.transaction->getSTransaction()->add(s); | ||
txn.add(s); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto
std::vector<uint256> | ||
getBatchTransactionIDs() const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this live on STTx
, not STObject
?
@@ -465,6 +465,12 @@ TRANSACTION(ttPERMISSIONED_DOMAIN_DELETE, 63, PermissionedDomainDelete, ({ | |||
{sfDomainID, soeREQUIRED}, | |||
})) | |||
|
|||
/** This transaction type wraps inner transactions for batch. */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
/** This transaction type wraps inner transactions for batch. */ | |
/** This transaction type batches together transactions. */ |
std::vector<uint256> | ||
STObject::getBatchTransactionIDs() const | ||
{ | ||
assert(getFieldArray(sfRawTransactions).size() != 0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert(getFieldArray(sfRawTransactions).size() != 0); | |
XRPL_ASSERT(isFieldPresent(sfRawTransactions) && getFieldArray(sfRawTransactions).size() != 0); |
for (STObject const& rb : getFieldArray(sfRawTransactions)) | ||
batch_txn_ids_.push_back(rb.getHash(HashPrefix::transactionID)); | ||
|
||
assert(batch_txn_ids_.size() == getFieldArray(sfRawTransactions).size()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert(batch_txn_ids_.size() == getFieldArray(sfRawTransactions).size()); | |
XRPL_ASSERT(batch_txn_ids_.size() == getFieldArray(sfRawTransactions).size()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same with all the other asserts in this PR
catch (std::exception const&) | ||
{ | ||
} | ||
return Unexpected("Internal signature check failure."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this line be in the catch
block?
// skip batch txns | ||
if (tx->isFlag(tfInnerBatchTxn)) | ||
{ | ||
assert( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assert( | |
XRPL_ASSERT( |
assert( | ||
txpair.second && | ||
txpair.second->isFieldPresent(sfParentBatchID)); | ||
continue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to write a unit test to cover this line?
XRPAmount baseFee, | ||
Application& app_, | ||
OpenView& base_, | ||
std::optional<uint256 const> const& batchId, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
std::optional<uint256 const> const& batchId, | |
std::optional<uint256 const> const& batchId_, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just want to point out that the reason there is no underscore is because the .cpp file has no underscore for the arg. The reason for this is so that you don't end up with `batchId_(batchId_)
So imo all of these _ changes are probably incorrect. I will circle back to them though as I dont think its that important.
src/xrpld/app/tx/detail/Batch.cpp
Outdated
} | ||
|
||
// Add the inner account to the unique signers set. | ||
uniqueSigners.emplace(innerAccount); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some rough pseudo-code that might help with rearranging the code (I know it's a weird mishmash of syntaxes, it's pseudo-code for a reason):
set<Account> uniqueAccounts;
for innerTx in rawTransactions:
// existing processing for each inner tx
uniqueAccounts.insert(innerTx[sfAccount]);
set<Account> uniqueSigners;
uniqueSigners.insert(tx[sfAccount]);
for signer in signers:
Account signerAccount = signer[sfAccount];
if (!uniqueSigners.insert(signerAccount).second)
// duplicate signer
return temBAD_SIGNER;
if (!uniqueAccounts.pop(signerAccount))
// signerAccount isn't included in the raw transactions
return temBAD_SIGNER;
// remove the outer account, if included
if (uniqueAccounts.contains(tx[sfAccount])
uniqueAccounts.pop(tx[sfAccount]);
if (!uniqueAccounts.empty())
// all BatchSigners and the outer account have been removed
// any account left doesn't have a signature included
// (I think this is the part that isn't currently being checked)
return temBAD_SIGNER;
} | ||
|
||
auto rawTxns = ctx.tx.getFieldArray(sfRawTransactions); | ||
if (rawTxns.size() == 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this check size >= 1
? No reason to allow length-1 Batches, right?
} | ||
return tesSUCCESS; | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should there be a preclaim
that makes sure all the Account
s in the inner Batches exist? Or is it fine to just trust that to the inner Batch validation?
{ | ||
JLOG(ctx.j.trace()) | ||
<< "BatchTrace[" << batchId << "]:" << "invalid batch signers."; | ||
return temBAD_SIGNER; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a test that covers this line
JLOG(ctx.j.trace()) | ||
<< "BatchTrace[" << batchId | ||
<< "]:" << "duplicate signer found: " << signerAccount; | ||
return temBAD_SIGNER; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a test that covers this line
JLOG(ctx.j.trace()) << "BatchTrace[" << batchId | ||
<< "]:" << "hashes array size does not match txns."; | ||
return temMALFORMED; // LCOV_EXCL_LINE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this throw something more serious now, since this shouldn't be possible anymore? Maybe an assert instead?
<< "BatchTrace[" << batchId | ||
<< "]:" << "TransactionType missing in inner txn." | ||
<< "txID: " << hash; | ||
return temINVALID_BATCH; // LCOV_EXCL_LINE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this line excluded from code coverage?
@@ -140,7 +140,7 @@ XRPNotCreated::visitEntry( | |||
bool | |||
XRPNotCreated::finalize( | |||
STTx const& tx, | |||
TER const, | |||
TER const res, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unnecessary change now
!ctx.tx.getSigningPubKey().empty() || | ||
ctx.tx.isFieldPresent(sfSigners)) | ||
{ | ||
return temINVALID_FLAG; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Judging by the comment, this line isn't possible to hit - in which case this should be excluded from code coverage.
Unsure whether that's the right error code though - perhaps it should be temBAD_SIGNATURE
instead?
<< "BatchTrace[" << batchId | ||
<< "]:" << "batch cannot have an inner batch txn." | ||
<< "txID: " << hash; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit:
<< "BatchTrace[" << batchId | |
<< "]:" << "batch cannot have an inner batch txn." | |
<< "txID: " << hash; | |
<< "BatchTrace[" << batchId << "]: " | |
<< "batch cannot have an inner batch txn. " | |
<< "txID: " << hash; |
ditto with all the other comments
@@ -208,6 +208,7 @@ transResults() | |||
MAKE_ERROR(temARRAY_EMPTY, "Malformed: Array is empty."), | |||
MAKE_ERROR(temARRAY_TOO_LARGE, "Malformed: Array is too large."), | |||
MAKE_ERROR(temBAD_TRANSFER_FEE, "Malformed: Transfer fee is outside valid range."), | |||
MAKE_ERROR(temINVALID_BATCH, "Malformed: Invalid inner batch transaction."), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thoughts on just calling the error code temINVALID_INNER_BATCH
? Feels a bit clearer.
@@ -2777,6 +2798,10 @@ NetworkOPsImp::pubProposedTransaction( | |||
std::shared_ptr<STTx const> const& transaction, | |||
TER result) | |||
{ | |||
// never publish an inner txn inside a batch txn | |||
if (transaction->isFlag(tfInnerBatchTxn)) | |||
return; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way to cover this line with a test?
High Level Overview of Change
Context of Change
Type of Change
.gitignore
, formatting, dropping support for older tooling)API Impact
libxrpl
change (any change that may affectlibxrpl
or dependents oflibxrpl
)Test Plan
https://gist.github.com/dangell7/3afbe8c4597a58dc2b02be598f9b7d54