Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Move commit logic and add tests that prepare for transaction function changes #1202

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
242d8f0
Add a unit test for commit as a non-tx
danieljbruce Sep 13, 2023
8a7b0ea
use linter
danieljbruce Nov 1, 2023
d0cd928
Write tests to capture current behavior of error
danieljbruce Nov 1, 2023
ed5a301
Add tests for passing a response
danieljbruce Nov 1, 2023
b3b13b4
run async close to working
danieljbruce Nov 1, 2023
85ec536
Add runAsync to promise excludes
danieljbruce Nov 1, 2023
d26ebd3
Remove space
danieljbruce Nov 1, 2023
2d13f56
Change to use this instead of self
danieljbruce Nov 1, 2023
3f66406
Eliminate unused comments
danieljbruce Nov 1, 2023
ff76a6d
Add two comments
danieljbruce Nov 1, 2023
d7806bd
Remove the commit test for this PR
danieljbruce Nov 1, 2023
01d7837
Clarify types throughout the function
danieljbruce Nov 1, 2023
dff3183
Add a bit more typing for clarity
danieljbruce Nov 1, 2023
f4e05bd
Change types used in the data client callback
danieljbruce Nov 2, 2023
6d50e94
run the linter
danieljbruce Nov 2, 2023
875bf4b
Add comments to clarify PR
danieljbruce Nov 2, 2023
ea80c65
Refactor the parsing logic out of run
danieljbruce Nov 2, 2023
14af87e
Change interface of request promise callback
danieljbruce Nov 2, 2023
3a28892
Move commit functionality to a new function
danieljbruce Nov 2, 2023
3293a9c
Add the commit tests
danieljbruce Nov 3, 2023
f4a41d6
Fix the tests so that they pass on commit
danieljbruce Nov 3, 2023
1d8b374
refactor one of the mocks
danieljbruce Nov 3, 2023
41beeca
Merge branch 'nodejs-transaction-redesign-feature-branch-1a-add-commi…
danieljbruce Nov 3, 2023
ba363e0
Hide data completely
danieljbruce Nov 6, 2023
9f50cba
PR use if/else block
danieljbruce Nov 6, 2023
f96471d
Add comments to document the new functions
danieljbruce Nov 6, 2023
59ee72a
Update return type in docs
danieljbruce Nov 6, 2023
34e961e
Update the tests to include runAsync
danieljbruce Nov 6, 2023
38cd2ef
refactor: Break transaction.run into smaller pieces for use with asyn…
danieljbruce Nov 6, 2023
08fe164
Merge branch 'nodejs-transaction-change-run' of https://github.com/da…
danieljbruce Nov 6, 2023
ad39a90
Rename a function to be more descriptive
danieljbruce Nov 6, 2023
e56a475
chore(deps): update dependency @types/sinon to v17 (#1197)
renovate-bot Nov 6, 2023
5a896bd
chore(deps): update dependency @types/is to v0.0.25 (#1198)
renovate-bot Nov 7, 2023
c129bd2
Modify comment
danieljbruce Nov 7, 2023
c31193e
Create a transaction wrapper class for testing
danieljbruce Nov 8, 2023
a608589
Clean up mocked transaction wrapper
danieljbruce Nov 8, 2023
2d23d07
Move test information for commit into commit block
danieljbruce Nov 8, 2023
e09ab36
Add gapic mocked tests for aggregation query
danieljbruce Nov 8, 2023
fbbb0d8
Fixing up the runQuery test
danieljbruce Nov 8, 2023
cf85fd0
Finished the runQuery tests
danieljbruce Nov 8, 2023
39b9f24
Finished the get tests
danieljbruce Nov 8, 2023
dce0836
remove only
danieljbruce Nov 8, 2023
5074de0
Add try catch blocks to handle errors in the tests
danieljbruce Nov 8, 2023
abf58ed
Merge branch 'nodejs-transaction-change-run' of https://github.com/da…
danieljbruce Nov 9, 2023
0d05f4c
Merge branch 'nodejs-transaction-redesign-feature-branch-1a-add-commi…
danieljbruce Nov 9, 2023
b86c79b
chore: update cloud-rad version to ^0.4.0 (#1199)
gcf-owl-bot[bot] Nov 13, 2023
1cfaa6c
Add comments and general cleanup
danieljbruce Nov 14, 2023
2ff43b8
Remove some redundant tests
danieljbruce Nov 14, 2023
351840c
Correct the comment to be more accurate
danieljbruce Nov 14, 2023
e797c34
General improvements to code quality
danieljbruce Nov 15, 2023
4eb7a5e
Add data client check
danieljbruce Nov 15, 2023
355c0a5
Eliminate the mocked function variable
danieljbruce Nov 15, 2023
310b320
Move begin transaction setup code
danieljbruce Nov 15, 2023
b151740
Replace the TODO for the key
danieljbruce Nov 15, 2023
a369202
Update description
danieljbruce Nov 15, 2023
99530ea
Add comments to describe the purpose of signaller
danieljbruce Nov 15, 2023
a42c7fe
Add both dones back in
danieljbruce Nov 15, 2023
ca6f63b
mockedBeginTransaction should be Function
danieljbruce Nov 15, 2023
3a76ee0
Use ECMA script modifier
danieljbruce Nov 15, 2023
e250bad
Remove TODOs that no longer apply
danieljbruce Nov 15, 2023
a146626
beginTransaction type definition
danieljbruce Nov 15, 2023
7430ede
Merge branch 'main' of https://github.com/googleapis/nodejs-datastore…
danieljbruce Nov 15, 2023
f312716
Merge branch 'nodejs-transaction-redesign-feature-branch' of https://…
danieljbruce Nov 17, 2023
cc4bbb8
Add the setupBeginTransaction method
danieljbruce Nov 17, 2023
cb3ec1b
Add a comment for private runCommit method
danieljbruce Nov 17, 2023
7b0a98a
Modify comment to be more clear
danieljbruce Nov 17, 2023
8b0d3c7
Update comments
danieljbruce Nov 17, 2023
b7ebc51
Eliminate redundant test code
danieljbruce Nov 17, 2023
c050d50
revert comment
danieljbruce Nov 17, 2023
a5efe6e
refactor: Move commit logic and add tests that prepare for transactio…
danieljbruce Nov 17, 2023
2544ed9
Merge branch 'nodejs-transaction-redesign-feature-branch-2aa-adding-t…
danieljbruce Nov 17, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/.OwlBot.lock.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,5 @@
# limitations under the License.
docker:
image: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest
digest: sha256:abc68a9bbf4fa808b25fa16d3b11141059dc757dbc34f024744bba36c200b40f
# created: 2023-10-04T20:56:40.710775365Z
digest: sha256:e92044720ab3cb6984a70b0c6001081204375959ba3599ef6c42dd99a7783a67
# created: 2023-11-10T00:24:05.581078808Z
2 changes: 1 addition & 1 deletion .kokoro/release/docs-devsite.sh

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,12 @@
"devDependencies": {
"@google-cloud/storage": "^7.0.1",
"@types/extend": "^3.0.1",
"@types/is": "0.0.24",
"@types/is": "0.0.25",
"@types/js-yaml": "^4.0.0",
"@types/mocha": "^9.0.0",
"@types/node": "^20.4.9",
"@types/proxyquire": "^1.3.28",
"@types/sinon": "^10.0.0",
"@types/sinon": "^17.0.0",
"async": "^3.2.4",
"c8": "^8.0.1",
"gapic-tools": "^0.2.0",
Expand Down
237 changes: 126 additions & 111 deletions src/transaction.ts
Original file line number Diff line number Diff line change
Expand Up @@ -161,117 +161,7 @@ class Transaction extends DatastoreRequest {
: () => {};
const gaxOptions =
typeof gaxOptionsOrCallback === 'object' ? gaxOptionsOrCallback : {};

if (this.skipCommit) {
setImmediate(callback);
return;
}

const keys: Entities = {};

this.modifiedEntities_
// Reverse the order of the queue to respect the "last queued request
// wins" behavior.
.reverse()
// Limit the operations we're going to send through to only the most
// recently queued operations. E.g., if a user tries to save with the
// same key they just asked to be deleted, the delete request will be
// ignored, giving preference to the save operation.
.filter((modifiedEntity: Entity) => {
const key = modifiedEntity.entity.key;

if (!entity.isKeyComplete(key)) return true;

const stringifiedKey = JSON.stringify(modifiedEntity.entity.key);

if (!keys[stringifiedKey]) {
keys[stringifiedKey] = true;
return true;
}

return false;
})
// Group entities together by method: `save` mutations, then `delete`.
// Note: `save` mutations being first is required to maintain order when
// assigning IDs to incomplete keys.
.sort((a, b) => {
return a.method < b.method ? 1 : a.method > b.method ? -1 : 0;
})
// Group arguments together so that we only make one call to each
// method. This is important for `DatastoreRequest.save`, especially, as
// that method handles assigning auto-generated IDs to the original keys
// passed in. When we eventually execute the `save` method's API
// callback, having all the keys together is necessary to maintain
// order.
.reduce((acc: Entities, entityObject: Entity) => {
const lastEntityObject = acc[acc.length - 1];
const sameMethod =
lastEntityObject && entityObject.method === lastEntityObject.method;

if (!lastEntityObject || !sameMethod) {
acc.push(entityObject);
} else {
lastEntityObject.args = lastEntityObject.args.concat(
entityObject.args
);
}

return acc;
}, [])
// Call each of the mutational methods (DatastoreRequest[save,delete])
// to build up a `req` array on this instance. This will also build up a
// `callbacks` array, that is the same callback that would run if we
// were using `save` and `delete` outside of a transaction, to process
// the response from the API.
.forEach(
(modifiedEntity: {method: string; args: {reverse: () => void}}) => {
const method = modifiedEntity.method;
const args = modifiedEntity.args.reverse();
Datastore.prototype[method].call(this, args, () => {});
}
);

// Take the `req` array built previously, and merge them into one request to
// send as the final transactional commit.
const reqOpts = {
mutations: this.requests_
.map((x: {mutations: google.datastore.v1.Mutation}) => x.mutations)
.reduce(
(a: {concat: (arg0: Entity) => void}, b: Entity) => a.concat(b),
[]
),
};

this.request_(
{
client: 'DatastoreClient',
method: 'commit',
reqOpts,
gaxOpts: gaxOptions || {},
},
(err, resp) => {
if (err) {
// Rollback automatically for the user.
this.rollback(() => {
// Provide the error & API response from the failed commit to the
// user. Even a failed rollback should be transparent. RE:
// https://github.com/GoogleCloudPlatform/google-cloud-node/pull/1369#discussion_r66833976
callback(err, resp);
});
return;
}

// The `callbacks` array was built previously. These are the callbacks
// that handle the API response normally when using the
// DatastoreRequest.save and .delete methods.
this.requestCallbacks_.forEach(
(cb: (arg0: null, arg1: Entity) => void) => {
cb(null, resp);
}
);
callback(null, resp);
}
);
this.#runCommit(gaxOptions, callback);
}

/**
Expand Down Expand Up @@ -561,6 +451,131 @@ class Transaction extends DatastoreRequest {
});
}

/**
* This function is a pass-through for the transaction.commit method
* It contains the business logic used for committing a transaction
*
* @param {object} [gaxOptions] Request configuration options, outlined here:
* https://googleapis.github.io/gax-nodejs/global.html#CallOptions.
* @param {function} callback The callback function.
* @private
*/
#runCommit(
gaxOptions: CallOptions,
callback: CommitCallback
): void | Promise<CommitResponse> {
if (this.skipCommit) {
setImmediate(callback);
return;
}

const keys: Entities = {};

this.modifiedEntities_
// Reverse the order of the queue to respect the "last queued request
// wins" behavior.
.reverse()
// Limit the operations we're going to send through to only the most
// recently queued operations. E.g., if a user tries to save with the
// same key they just asked to be deleted, the delete request will be
// ignored, giving preference to the save operation.
.filter((modifiedEntity: Entity) => {
const key = modifiedEntity.entity.key;

if (!entity.isKeyComplete(key)) return true;

const stringifiedKey = JSON.stringify(modifiedEntity.entity.key);

if (!keys[stringifiedKey]) {
keys[stringifiedKey] = true;
return true;
}

return false;
})
// Group entities together by method: `save` mutations, then `delete`.
// Note: `save` mutations being first is required to maintain order when
// assigning IDs to incomplete keys.
.sort((a, b) => {
return a.method < b.method ? 1 : a.method > b.method ? -1 : 0;
})
// Group arguments together so that we only make one call to each
// method. This is important for `DatastoreRequest.save`, especially, as
// that method handles assigning auto-generated IDs to the original keys
// passed in. When we eventually execute the `save` method's API
// callback, having all the keys together is necessary to maintain
// order.
.reduce((acc: Entities, entityObject: Entity) => {
const lastEntityObject = acc[acc.length - 1];
const sameMethod =
lastEntityObject && entityObject.method === lastEntityObject.method;

if (!lastEntityObject || !sameMethod) {
acc.push(entityObject);
} else {
lastEntityObject.args = lastEntityObject.args.concat(
entityObject.args
);
}

return acc;
}, [])
// Call each of the mutational methods (DatastoreRequest[save,delete])
// to build up a `req` array on this instance. This will also build up a
// `callbacks` array, that is the same callback that would run if we
// were using `save` and `delete` outside of a transaction, to process
// the response from the API.
.forEach(
(modifiedEntity: {method: string; args: {reverse: () => void}}) => {
const method = modifiedEntity.method;
const args = modifiedEntity.args.reverse();
Datastore.prototype[method].call(this, args, () => {});
}
);

// Take the `req` array built previously, and merge them into one request to
// send as the final transactional commit.
const reqOpts = {
mutations: this.requests_
.map((x: {mutations: google.datastore.v1.Mutation}) => x.mutations)
.reduce(
(a: {concat: (arg0: Entity) => void}, b: Entity) => a.concat(b),
[]
),
};

this.request_(
{
client: 'DatastoreClient',
method: 'commit',
reqOpts,
gaxOpts: gaxOptions || {},
},
(err, resp) => {
if (err) {
// Rollback automatically for the user.
this.rollback(() => {
// Provide the error & API response from the failed commit to the
// user. Even a failed rollback should be transparent. RE:
// https://github.com/GoogleCloudPlatform/google-cloud-node/pull/1369#discussion_r66833976
callback(err, resp);
});
return;
}

// The `callbacks` array was built previously. These are the callbacks
// that handle the API response normally when using the
// DatastoreRequest.save and .delete methods.
this.requestCallbacks_.forEach(
(cb: (arg0: null, arg1: Entity) => void) => {
cb(null, resp);
}
);
callback(null, resp);
}
);
}

/**
* This function parses results from a beginTransaction call
*
Expand Down
Loading
Loading