Skip to content
This repository has been archived by the owner on Jan 9, 2024. It is now read-only.

change insert_one() to insert_many() in mongo_db save function. #455

Merged
merged 15 commits into from
Jan 6, 2024

Conversation

bwook00
Copy link
Contributor

@bwook00 bwook00 commented Jan 5, 2024

close #438

(Plus, if there is duplicate "_id" key in mongo db, it occurs error and stop everything)
-> This feature is already well supported by mongodb itself (�BulkwriteError)

@bwook00 bwook00 requested a review from vkehfdl1 January 5, 2024 05:11
Copy link
Contributor

@vkehfdl1 vkehfdl1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Plus, if there is duplicate "_id" key in mongo db, it occurs error and stop everything)

You need to add test code for this kind of situation.

@bwook00 bwook00 requested a review from vkehfdl1 January 5, 2024 05:34
Copy link
Contributor

@vkehfdl1 vkehfdl1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to give option to user about upsert.
If upsert is True, duplicate id does not occur error and just overwrite existing id.
If upsert is False, duplicate id occurs error.
Sometimes upsert option is super convenient, so we might support upsert feature. Find way to implement this with pymongo.

@bwook00 bwook00 requested a review from vkehfdl1 January 5, 2024 07:44
@bwook00
Copy link
Contributor Author

bwook00 commented Jan 5, 2024

If I use pymongo's update_many() with upsert=True, it would update all existing passages into a "single passage".

So we take the upsert parameter directly, and if true, we put the existing ids in a list and only update the duplicates (and even then, we use bulk_wirte to minimize the iteration as much as possible).

Instead of updating all of them, we just proceed with insert_many() for non-existing ids.

(I've been googling hard for a function to update all at once in pymongo, but I haven't found one...)

@bwook00 bwook00 requested a review from vkehfdl1 January 6, 2024 06:17
@bwook00 bwook00 requested a review from vkehfdl1 January 6, 2024 13:32
@bwook00 bwook00 requested a review from vkehfdl1 January 6, 2024 14:40
Copy link
Contributor

@vkehfdl1 vkehfdl1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finally..

@vkehfdl1 vkehfdl1 merged commit b0fe755 into main Jan 6, 2024
2 checks passed
@vkehfdl1 vkehfdl1 deleted the Hotfix/#438 branch January 6, 2024 14:53
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Refactor] MongoDB use insert_one at save()
2 participants