fix: tiktok returned invalid reponse caused by bot detection

[arhen]

This PR to solve the issues #1187 #1193 #1179 by simulating user movement using scrolling after open the page.

[anarchopythonista]

Some thoughts:

Could we get away from time.sleep with something like this:

await page.wait_for_load_state("networkidle")

I don't know playwright well enough to know if this would work. Currently, I'm working on getting an environment up to reproduce.

I also think

page.mouse.move(0, 0)
page.mouse.move(0, 100)

would make it fairly easy for TikTok to distinguish scrapers from legitimate desktop users. What are your thoughts on importing random and doing something like this:

x, y = 0, random.randint(0, 50)
a, b = random.randint(1, 50), random.randint(100, 200)

page.mouse.move(x, y)
page.mouse.move(a, b)

[arhen]

Some thoughts:

Could we get away from time.sleep with something like this:

await page.wait_for_load_state("networkidle")

I don't know playwright well enough to know if this would work. Currently, I'm working on getting an environment up to reproduce.

I also think

page.mouse.move(0, 0)
page.mouse.move(0, 100)

would make it fairly easy for TikTok to distinguish scrapers from legitimate desktop users. What are your thoughts on importing random and doing something like this:

x, y = 0, random.randint(0, 50)
a, b = random.randint(1, 50), random.randint(100, 200)

page.mouse.move(x, y)
page.mouse.move(a, b)

Yeah, sure. I'll add later. Thanks

[davidteather]

Thank you! I'll add the updated code by @anarchopythonista and I found an alternative way to get around it but this will be more robust