pymongo.errors.CursorNotFound: Cursor not found, cursor id: 22364095374

[SeekPoint]

New link [Set cover problem] HAS => [25th]
['Graz', 'Austria']
Adding : Set cover problem ===> ['Graz', 'Austria']
New link [Set cover problem] HAS => [Graz]
New link [Set cover problem] HAS => [Austria]
['doi']
Adding : Set cover problem ===> ['doi']
New link [Set cover problem] HAS => [doi]
['related', 'media', 'Set', 'Setcover', 'links', 'cover', 'problem', 'Commons', 'coverproblem']
Adding : Set cover problem ===> ['related', 'media', 'Set', 'Setcover', 'links', 'cover', 'problem', 'Commons', 'coverproblem']
New link [Set cover problem] HAS => [related]
New link [Set cover problem] HAS => [media]
New link [Set cover problem] HAS => [Set]
New link [Set cover problem] HAS => [Setcover]
New link [Set cover problem] HAS => [links]
New link [Set cover problem] HAS => [cover]
New link [Set cover problem] HAS => [problem]
New link [Set cover problem] HAS => [Commons]
New link [Set cover problem] HAS => [coverproblem]
['Solutions', 'Set', 'Benchmarks', 'WinnerDetermination', 'NPoptimization', 'Cover', 'Covering', 'HiddenOptimum', 'optimization', 'SetCover', 'CoveringSet', 'Hidden', 'Determinationcompendium', 'Winner', 'Minimum', 'MinimumSet', 'Optimum', 'Determination', 'compendium', 'problems']
Adding : Set cover problem ===> ['Solutions', 'Set', 'Benchmarks', 'WinnerDetermination', 'NPoptimization', 'Cover', 'Covering', 'HiddenOptimum', 'optimization', 'SetCover', 'CoveringSet', 'Hidden', 'Determinationcompendium', 'Winner', 'Minimum', 'MinimumSet', 'Optimum', 'Determination', 'compendium', 'problems']
New link [Set cover problem] HAS => [Solutions]
New link [Set cover problem] HAS => [Set]
New link [Set cover problem] HAS => [Benchmarks]
New link [Set cover problem] HAS => [WinnerDetermination]
New link [Set cover problem] HAS => [NPoptimization]
New link [Set cover problem] HAS => [Cover]
New link [Set cover problem] HAS => [Covering]
New link [Set cover problem] HAS => [HiddenOptimum]
New link [Set cover problem] HAS => [optimization]
New link [Set cover problem] HAS => [SetCover]
New link [Set cover problem] HAS => [CoveringSet]
New link [Set cover problem] HAS => [Hidden]
New link [Set cover problem] HAS => [Determinationcompendium]
New link [Set cover problem] HAS => [Winner]
New link [Set cover problem] HAS => [Minimum]
New link [Set cover problem] HAS => [MinimumSet]
New link [Set cover problem] HAS => [Optimum]
New link [Set cover problem] HAS => [Determination]
New link [Set cover problem] HAS => [compendium]
New link [Set cover problem] HAS => [problems]
Set cover problem processed with 534 nodes.
34 wiki documents processed so far...
Traceback (most recent call last):
File "build_knowledge.py", line 113, in
for topic,sentence in iter_topic(crawl_collection,args['start']):
File "build_knowledge.py", line 41, in iter_topic
for wiki in crawl_collection.query({'downloaded': True},field=None,skip=start):
File "/home/mldl/ub16_prj/vor-knowledge-graph/pylib/knowledge/datasource.py", line 21, in query
for n in query:
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 1189, in next
if len(self.__data) or self._refresh():
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 1126, in _refresh
self.__send_message(g)
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 978, in __send_message
codec_options=self.__codec_options)
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 1067, in _unpack_response
return response.unpack_response(cursor_id, codec_options)
File "/usr/local/lib/python3.5/dist-packages/pymongo/message.py", line 1418, in unpack_response
self.raw_response(cursor_id)
File "/usr/local/lib/python3.5/dist-packages/pymongo/message.py", line 1384, in raw_response
raise CursorNotFound(msg, 43, errobj)
pymongo.errors.CursorNotFound: Cursor not found, cursor id: 22364095374
mldl@ub1604:/ub16_prj/vor-knowledge-graph$
mldl@ub1604:/ub16_prj/vor-knowledge-graph$

[tao-pr]

Hello @lovejasmine
Two fun facts might surprise you.

1. The issue is caused by a too-heavy concurrent data loading from MongoDB. So the cursor is busy when it iterates to the next task and it ends up having an error message saying CursorNotFound.

2. The workaround to the issue has already been addressed in another outstanding branch I'm working on but it's on halt. https://github.com/starcolon/vor-knowledge-graph/blob/feature/content-segmentation/pylib/knowledge/datasource.py#L20

So I would recommend you to take the extra options you see on the line as linked above to your code. Run it again it will go fine.

[SeekPoint]

mldl@ub1604:/ub16_prj/vor-knowledge-graph$ python3 build_knowledge.py --verbose --root mypass --limit 10
Initialising knowledge graph database...
Connecting OrientDB: vor
[WARNING] Preparing graph schema
com.orientechnologies.orient.core.exception.OSchemaException - Class 'TOPIC' already exists in current database
DB name="vor"
[Graph clearance] done
Loading POS patterns...
Loading stopwords...
Initialising wikipedia crawling collection...
Iterating over crawling database...
mldl@ub1604:/ub16_prj/vor-knowledge-graph$

Does it looks right?

and more:
mldl@ub1604:/ub16_prj/vor-knowledge-graph$ python3 build_wordvec.py --limit 10 --out out_model
Exporting crawled data to text file...
[Done]
Training word2vec...
b'Starting training using file /home/mldl/ub16_prj/vor-knowledge-graph/mine.txt\n'b'Vocab size: 1\n'b'Words in train file: 0\n'[Done]
Word2Vec model is saved at : /home/mldl/ub16_prj/vor-knowledge-graph/out_model
[Word2Vec model spec]
... Model shape : (1, 10)
[Word2Vec REPL]:
Enter a word to try: hello
Traceback (most recent call last):
File "build_wordvec.py", line 102, in
repl(model)
File "build_wordvec.py", line 87, in repl
indexes, metrics = model.cosine(w)
File "/usr/local/lib/python3.5/dist-packages/word2vec/wordvectors.py", line 76, in cosine
metrics = np.dot(self.vectors, self[word].T)
File "/usr/local/lib/python3.5/dist-packages/word2vec/wordvectors.py", line 45, in getitem
return self.get_vector(word)
File "/usr/local/lib/python3.5/dist-packages/word2vec/wordvectors.py", line 54, in get_vector
idx = self.ix(word)
File "/usr/local/lib/python3.5/dist-packages/word2vec/wordvectors.py", line 42, in ix
return self.vocab_hash[word]
KeyError: 'hello'
mldl@ub1604:/ub16_prj/vor-knowledge-graph$

[tao-pr]

Warnings are fine and ignorable. The error on the bottom part means you are querying for the word which doesn't exist in the vector space. I would suggest you to try inputting a word picked from the content you fetched.

[SeekPoint]

the error raised yet:

Set cover problem processed with 534 nodes.
35 wiki documents processed so far...
Traceback (most recent call last):
File "build_knowledge.py", line 113, in
for topic,sentence in iter_topic(crawl_collection,args['start']):
File "build_knowledge.py", line 41, in iter_topic
for wiki in crawl_collection.query({'downloaded': True},field=None,skip=start):
File "/home/mldl/ub16_prj/vor-knowledge-graph/pylib/knowledge/datasource.py", line 21, in query
for n in query:
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 1189, in next
if len(self.__data) or self._refresh():
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 1126, in _refresh
self.__send_message(g)
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 978, in __send_message
codec_options=self.__codec_options)
File "/usr/local/lib/python3.5/dist-packages/pymongo/cursor.py", line 1067, in _unpack_response
return response.unpack_response(cursor_id, codec_options)
File "/usr/local/lib/python3.5/dist-packages/pymongo/message.py", line 1418, in unpack_response
self.raw_response(cursor_id)
File "/usr/local/lib/python3.5/dist-packages/pymongo/message.py", line 1384, in raw_response
raise CursorNotFound(msg, 43, errobj)
pymongo.errors.CursorNotFound: Cursor not found, cursor id: 30134005583
mldl@ub1604:~/ub16_prj/vor-knowledge-graph$

[tao-pr]

as suggested, have you tried updating the query function in pylib/knowledge/datasource.py#L20
so it becomes?:

query = self.src.find(conditions) if skip==0 else self.src.find(filter=conditions,skip=skip,no_cursor_timeout=True)

With the cursor timeout option added in you should no longer see such error message.

[SeekPoint]

sure , I did it

[tao-pr]

interesting. can you please let me know your mongodb version?