Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Massive increase in the number of API queries after updating to 0.2 #17

Open
RundownRhino opened this issue Jul 13, 2019 · 4 comments
Open

Comments

@RundownRhino
Copy link

Soon after updating to 0.2, the script started asking me for a new key every time I loaded a youtube video. I quickly found out that the cause of that was that the API requests weren't working, and the cause of that was that the Google API daily quota of 10000 queries was exhausted. Before, the script rarely used more than 1000 requests a day, meaning that the number of queries was increased more than tenfold by the update.

@RundownRhino RundownRhino changed the title Massive increase in the number of API queries after update to 0.2 Massive increase in the number of API queries after updating to 0.2 Jul 13, 2019
@pcouy
Copy link
Owner

pcouy commented Jul 13, 2019

Thank you for the feedback. I ran into this issue as well, but I was working on https://github.com/pcouy/mps-youtube/tree/origin/master and did not suspect I reached API quotas because of this script.

@Seneral , I reverted your pull request for now. I will keep you updated if I find why this happened. Since you do not seem to have encountered the issue, can you describe your typical use of Youtube ?

pcouy added a commit that referenced this issue Jul 13, 2019
Change version number to fix issues for people who upgraded to 0.2 (#17 #18 #13)
@Seneral
Copy link
Contributor

Seneral commented Jul 14, 2019

That's weird, I see strong spikes in my graph as well (two in the last month) together with a big spike of Errors. I don't know if that was still with the other version though.
Only thing that changed in regards to this is that it slices them into 50 elements per request (otherwise it would return an error) and that it recognizes when info has been denied, in which case an video is marked as deleted (previously it would send those requests over and over again).

Without those errors I have about 1800 requests per week, with my current setup I would expect no more than 500 per week. So I'll check, but only after the next two exams are over.

@Seneral
Copy link
Contributor

Seneral commented Aug 7, 2019

Ok I found a problem in my code where when the main description was delayed long enough (e.g. tag couldn't be found in page, response was not as expected) it would merge it with the normal requests later on. If NOW you had a playlist open with more than 30 videos (so that the first batch has already more than 50 videos waiting to be replaced) it would take the first 50 of the batch and then add the main video ID ontop, for a total of 51 IDs, which is obviously invalid.

Now the problem is: The original code before didn't check for the ID limit at all, which means this problem shouldn't be new.

I also did some other changes to optimize the time flow and remove overlapping requests. Originally a check would execute every second, now it will only execute once all videos on the page have been replaced, before that it will simply start the next check once the API response has been received.

Implementation here: Seneral@73b9ffe
Will test it over the next few days

@Seneral
Copy link
Contributor

Seneral commented Aug 12, 2019

Ok so ne errors at all the past few days, so that is definitely fixed.

Another idea that I have that might cause higher requests than usual: In my version, I've changed to selected elements that are being reverted to include ALL videos seen on the page. Previously, there were way less. I don't know what content you usually watch but if you have, say, a large playlist open, it would require a request for each bunch of 50 videos. It's only cached through page load, so autoplay or similar won't break caching, but other stuff might and I haven't tested that.
To counter that I could try to only regard visible videos. This would mean that scrolling down or scrolling in said playlist would have the names pop up delayed, but it's better than nothing. I could increase the interval timing to around 200ms to mitigate that issue (1sec currently).
So are the plans, once I have time again.

Seneral pushed a commit to Seneral/YoutubeAutotranslateCanceler that referenced this issue Sep 21, 2021
Change version number to fix issues for people who upgraded to 0.2 (pcouy#17 pcouy#18 pcouy#13)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants