Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pirate Bay Not Finding Results #3305

Open
wymangr opened this issue May 26, 2022 · 8 comments
Open

Pirate Bay Not Finding Results #3305

wymangr opened this issue May 26, 2022 · 8 comments

Comments

@wymangr
Copy link

wymangr commented May 26, 2022

I have been using headphones for a while now and it has worked in the past and I've always used pirate bay as my search provider. However, it hasn't found any new music in quite a long time.

Headphones Version: 83398cb
Pirate Bay Proxy URL: Tried blank (Default) and https://thepiratebay.org

Logs (Using RHCP as example):
26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Marking album: 0580e402-e6e8-4b5b-894a-399a10c63adb as wanted...
26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Searching for wanted albums
26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Searching for "Red Hot Chili Peppers - Unlimited Love" since it was marked as wanted
26-May-2022 09:29:43 - DEBUG :: ('CP Server Thread-11',) : Using search term: Red Hot Chili Peppers Unlimited Love
26-May-2022 09:29:43 - INFO :: ('CP Server Thread-11',) : Searching The Pirate Bay using term: Red Hot Chili Peppers Unlimited Love
26-May-2022 09:29:43 - DEBUG :: ('CP Server Thread-11',) : Requesting URL via GET method: https://thepiratebay.org/search/Red Hot Chili Peppers Unlimited Love/0/7/101
26-May-2022 09:29:45 - INFO :: ('CP Server Thread-11',) : No results found from The Pirate Bay using term: Red Hot Chili Peppers Unlimited Love
26-May-2022 09:29:45 - INFO :: ('CP Server Thread-11',) : No more results found for: Red Hot Chili Peppers - Unlimited Love

Using the requesting URL, I see multiple results in the browser:

image
@wymangr
Copy link
Author

wymangr commented May 26, 2022 •
edited
Loading

From what I can tell, this is the HTML that is stored in the data variable when is searches:
data = request.request_soup(url=providerurl + category, headers=headers)

And looking at that, it appears that maybe Pirate Bay has switched to rendering it's result table to javascript and the requests library is unable to get the HTML of the table?

<!DOCTYPE html>
<html lang="en">
   <head>
      <meta charset="utf-8"/>
      <meta content="width=device-width, initial-scale=1, shrink-to-fit=no" name="viewport"/>
      <title>The Pirate Bay - The galaxy's most resilient bittorrent site</title>
      <link href="https://torrindex.net/static/normalize.css" rel="stylesheet" type="text/css"/>
      <link href="https://torrindex.net/static/tpb.css" rel="stylesheet" type="text/css"/>
      <link href="//apibay.org/" rel="dns-prefetch"/>
      <link href="//torrindex.net/" rel="dns-prefetch"/>
      <link href="//ilyf4amifh.com/" rel="dns-prefetch"/>
      <link href="//abservinean.com/" rel="dns-prefetch"/>
      <link href="//a.exdynsrv.com/" rel="dns-prefetch"/>
      <link href="//a.exosrv.com/" rel="dns-prefetch"/>
      <link href="//cdn.engine.spotscenered.info/" rel="dns-prefetch"/>
      <link href="//syndication.exdynsrv.com/" rel="dns-prefetch"/>
      <link href="//d1f05vr3sjsuy7.cloudfront.net/" rel="dns-prefetch"/>
      <script>var country='CH';</script>
      <script src="https://thepiratebay.org/static/main.js"></script>
      <script src="https://torrindex.net/static/tinysort.min.js"></script>
      <script async="" data-cfasync="false" src="//ilyf4amifh.com/pn21ywqw/z/sc/scssx/1846387/lib.js" type="text/javascript"></script>
      <script data-cfasync="false" src="//d1f05vr3sjsuy7.cloudfront.net/?srvfd=908284"></script>
   </head>
   <body id="browse" onload="jswarnclear()">
      <main>
         <b><font color="RED"><label id="jscrwarn">Enable JS in your browser!</label></font></b>
         <script>document.getElementById("jscrwarn").innerHTML='';</script>
         <b><font color="RED"><label id="jscrwarn2">You may be blocking important javascript components, check that main.js is loaded or the webpage won't work.</label></font></b>
         <h1><label id="tlt"></label></h1>
         <div class="adblock" id="ad-bottom">
            <div class="ad728 align-center">
               <a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&amp;aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/728t.png"/></a> 
            </div>
            <div class="ad468 align-center">
               <a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&amp;aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/468.png" style="text-decoration:none; border-bottom:none; color:none;"/></a> 
            </div>
            <div class="ad234 align-center">
               <script type="application/javascript">
                  var ad_idzone = "3804619",
                  ad_width = "300",
                  ad_height = "100"
               </script>
               <script src="https://a.exdynsrv.com/ads.js" type="application/javascript"></script>
            </div>
         </div>
         <div class="browse">
            <section class="col-left ad120">
               <a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&amp;aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/120.png"/></a> 
            </section>
            <script>print_search()</script>
            <section class="col-right ad120">
               <a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&amp;aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/160.png"/></a> 
            </section>
         </div>
         <div class="adblock" id="ad-bottom">
            <div class="ad728 align-center">
               <a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&amp;aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/728f.png"/></a> 
            </div>
            <div class="ad468 align-center">
               <a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&amp;aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/468.png" style="text-decoration:none; border-bottom:none; color:none;"/></a> 
            </div>
            <div class="ad234 align-center">
               <script type="application/javascript">
                  var ad_idzone = "3804621",
                  ad_width = "300",
                  ad_height = "250"
               </script>
               <script src="https://a.exdynsrv.com/ads.js" type="application/javascript"></script>
            </div>
         </div>
      </main>
      <header class="row">
         <script>print_header1()</script>
         <input id="search" name="q" placeholder="Search here..." title="Pirate Search" type="search"/>
         <script>document.getElementById("search").value = getParameterByName('q')</script>
         <script>print_header2()</script>
         <section class="col-right ad468" id="had468">
            <a href="https://ttf.trmobc.com/aff_ad?campaign_id=6&amp;aff_id=460" style="text-decoration:none; border-bottom:none; color:none;"><img src="https://torrindex.net/images/epv/468.png" style="text-decoration:none; border-bottom:none; color:none;"/></a> 
         </section>
         <section class="col-right ad234" id="had234"></section>
      </header>
      <script>
         print_footer();
         mark_selected();
      </script>
      <script>do_pop()</script>
      <script>do_interstitial()</script>
   </body>
</html>

@Mikaciu Mikaciu mentioned this issue Feb 5, 2023
Merged
@AdeHub
Copy link
Collaborator

AdeHub commented Aug 9, 2024

have switched the default to use apibay.org, this is in develop if you want to test

@Mikaciu
Copy link
Contributor

Mikaciu commented Aug 10, 2024

Hello @AdeHub, I have a cloudflare error for apibay.org, do you have it too ?

@AdeHub
Copy link
Collaborator

AdeHub commented Aug 10, 2024

Hello @AdeHub, I have a cloudflare error for apibay.org, do you have it too ?

No, got logs for that?

Could also try proxy https://prbay.top/

@AdeHub AdeHub mentioned this issue Aug 11, 2024
Open
@Mikaciu
Copy link
Contributor

Mikaciu commented Aug 15, 2024

Hello @AdeHub, I have a cloudflare error for apibay.org, do you have it too ?

No, got logs for that?

Could also try proxy https://prbay.top/

Yep!

headphones-1           | 15-Aug-2024 14:09:08 - INFO :: ('CP Server Thread-4',) : Searching for "Amaranthe - The Catalyst" since it was marked as wanted
headphones-1           | 15-Aug-2024 14:09:08 - DEBUG :: ('CP Server Thread-4',) : Using search term: Amaranthe The Catalyst
headphones-1           | 15-Aug-2024 14:09:08 - INFO :: ('CP Server Thread-4',) : Searching The Pirate Bay using term: Amaranthe The Catalyst
headphones-1           | 15-Aug-2024 14:09:08 - DEBUG :: ('CP Server Thread-4',) : Requesting URL via GET method: http://apibay.org/q.php?q=Amaranthe The Catalyst&cat=101
headphones-1           | 15-Aug-2024 14:09:08 - ERROR :: ('CP Server Thread-4',) : Unable to connect to remote host. Check if the remote host is up and running.

For prbay, same result unfortunately!

headphones-1           | 15-Aug-2024 14:10:48 - INFO :: ('CP Server Thread-10',) : Searching for "Amaranthe - The Catalyst" since it was marked as wanted
headphones-1           | 15-Aug-2024 14:10:48 - DEBUG :: ('CP Server Thread-10',) : Using search term: Amaranthe The Catalyst
headphones-1           | 15-Aug-2024 14:10:48 - INFO :: ('CP Server Thread-10',) : Searching The Pirate Bay using term: Amaranthe%20The%20Catalyst
headphones-1           | 15-Aug-2024 14:10:48 - DEBUG :: ('CP Server Thread-10',) : Requesting URL via GET method: https://prbay.top/search/Amaranthe%20The%20Catalyst/0/7/101
headphones-1           | 15-Aug-2024 14:10:48 - ERROR :: ('CP Server Thread-10',) : Unable to connect to remote host. Check if the remote host is up and running.
headphones-1           | 15-Aug-2024 14:10:48 - INFO :: ('CP Server Thread-10',) : Search for wanted albums complete

@Mikaciu
Copy link
Contributor

Mikaciu commented Aug 15, 2024

I tried another proxy and I have the same error:

headphones-1           | 15-Aug-2024 14:14:13 - INFO :: ('CP Server Thread-2',) : Searching for "Amaranthe - Manifest" since it was marked as wanted
headphones-1           | 15-Aug-2024 14:14:13 - DEBUG :: ('CP Server Thread-2',) : Using search term: Amaranthe Manifest
headphones-1           | 15-Aug-2024 14:14:13 - INFO :: ('CP Server Thread-2',) : Searching The Pirate Bay using term: Amaranthe%20Manifest
headphones-1           | 15-Aug-2024 14:14:13 - DEBUG :: ('CP Server Thread-2',) : Requesting URL via GET method: https://tprbay.xyz/search/Amaranthe%20Manifest/0/7/101
headphones-1           | 15-Aug-2024 14:14:13 - ERROR :: ('CP Server Thread-2',) : Unable to connect to remote host. Check if the remote host is up and running.
headphones-1           | 15-Aug-2024 14:14:13 - INFO :: ('CP Server Thread-2',) : Search for wanted albums complete

Although if you check the URL, you can see that there is one result for this search :(

@AdeHub
Copy link
Collaborator

AdeHub commented Aug 16, 2024

I am not getting that error, apibay, prbay,tprbay all working for me

TPB seems to be pretty flaky, have you looked at using Torznab with Jackett/Prowlarr

@Mikaciu
Copy link
Contributor

Mikaciu commented Aug 16, 2024 •
edited
Loading

Hello @AdeHub,

I reproduced locally, apibay is working fine when running Headphones using the python script.
What I can't understand is why it does not want to connect when using a docker image :(

EDIT: got it! DNS issue.

WOuld you be interested in adding a dockerfile to the repo ? The one from linuxserver is outdated :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AdeHub @Mikaciu @wymangr