positive item sampling and fix infinite loop #7
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
isipalma
commented
Jun 16, 2021
isipalma
commented
- Put a limit in sampling loop to prevent infinite loop
- Change method for sampling. Always the positive item is in the profile
aaossa
requested changes
Aug 5, 2021
Comment on lines
+137
to
+144
| "# Mark interactions used for evaluation procedure if needed\n", | ||
| "if \"evaluation\" not in interactions_df:\n", | ||
| " print(\"\\nApply evaluation split...\")\n", | ||
| " interactions_df = mark_evaluation_rows(interactions_df)\n", | ||
| " # Check if new column exists and has boolean dtype\n", | ||
| " assert interactions_df[\"evaluation\"].dtype.name == \"bool\"\n", | ||
| " print(f\">> Interactions: {interactions_df.shape}\")\n", | ||
| "\n", |
There was a problem hiding this comment.
I forgot, why was this needed here?
There was a problem hiding this comment.
I just noticed, the code was not present in this repository but it was in mine, right?
| @@ -202,22 +210,27 @@ | |||
| "metadata": {}, | |||
| "outputs": [], | |||
| "source": [ | |||
| "def random_triplet_sampling(samples_per_user, hashes_container, desc=None):\n", | |||
| "def random_triplet_sampling(samples_per_user, hashes_container, desc=None, limit_iteration=10000):\n", | |||
There was a problem hiding this comment.
Why 10000? Maybe we could use the number of interaction as limit, or a proportion of said number. If I have a million records, and need to sample an important number of it, a proportion of len(interactions_df) (or interactions_df.size, not sure which one is better) would be more appropriate than a fixed number
Comment on lines
+221
to
+224
| " aux_limit = limit_iteration\n", | ||
| " while n > 0:\n", | ||
| " if aux_limit == 0:\n", | ||
| " break\n", |
There was a problem hiding this comment.
aux_limit does not change its value, in line 247 we should use aux_limit instead of limit_iteration and that may be a fix
Comment on lines
-258
to
-260
| "assert len(samples_training) >= TOTAL_SAMPLES_TRAIN\n", | ||
| "assert len(samples_testing) >= TOTAL_SAMPLES_VALID\n", | ||
| "\n", |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.