clear

.codeclimate.yml

@@ -0,0 +1,2 @@
+exclude_patterns:
+  - "yggtorrentscraper/categories.py"

.gitignore

@@ -0,0 +1,13 @@
+torrents/
+__pycache__/
+.tox
+yggtorrentdownloader.egg-info/
+.coverage
+.pytest_cache
+coverage.xml
+build/
+dist/
+*.pyc
+*.torrent
+torrents/
+yggtorrentscraper.egg-info

.travis.yml

@@ -0,0 +1,17 @@
+language:
+  python
+
+os:
+  - linux
+
+python:
+  - "3.6"
+  - "3.7"
+  - "3.8"
+
+install:
+  - pip install -r requirements.txt
+  - pip install tox-travis
+
+script:
+  - tox

README.md

@@ -0,0 +1,287 @@
+# YggTorrentScraper
+
+[![PyPI version](https://badge.fury.io/py/yggtorrentscraper.svg)](https://badge.fury.io/py/yggtorrentscraper)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/2d860dc88dfa467eb07105f559ba352a)](https://www.codacy.com/app/Harkame/YggTorrentScraper?utm_source=github.com&utm_medium=referral&utm_content=Harkame/YggTorrentScraper&utm_campaign=Badge_Grade)
+[![Maintainability](https://api.codeclimate.com/v1/badges/341218ef79de77052e43/maintainability)](https://codeclimate.com/github/Harkame/YggTorrentScraper/maintainability)
+[![Build Status](https://travis-ci.org/Harkame/YggTorrentScraper.svg?branch=master)](https://travis-ci.org/Harkame/YggTorrentScraper)
+[![codecov](https://codecov.io/gh/Harkame/YggTorrentScraper/branch/master/graph/badge.svg)](https://codecov.io/gh/Harkame/YggTorrentScraper)
+
+## Installation
+
+``` bash
+
+pip install yggtorrentscraper
+
+```
+
+OR
+
+clone this repository and
+
+``` bash
+
+pip install -r requirements.txt
+
+python setup.py install
+
+```
+
+### Dependencies
+
+-   [BeautifulSoup 4](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)
+-   [lxml](https://github.com/lxml/lxml.git)
+-   [requests](https://github.com/psf/requests.git)
+
+## Usage
+
+### Initialization
+
+Actual cloudflare bypassers like https://github.com/VeNoMouS/cloudscraper seem to have some difficulties for now.
+
+The easiest solution is to set « cf_clearance » cookie manually, you can find it in your browser.
+
+
+``` python
+
+import requests
+from yggtorrentscraper import YggTorrentScraper
+
+cookie = requests.cookies.create_cookie(
+    name="cf_clearance",
+    value="1b23f59d619777532aea667960d00269fc49517e-1589279876-0-150",
+)
+session = requests.session()
+
+session.cookies.set_cookie(cookie)
+
+scraper = YggTorrentScraper(session)
+
+```
+
+#### Change TLD
+
+YggTorrent TLD is change regularly, you can specify it at YggTorrentScraper construction with optionnal parameters yggtorrent_tld
+
+``` python
+
+from yggtorrentscraper import set_yggtorrent_tld
+
+set_yggtorrent_tld("new_tld")
+
+```
+
+Session is requiered to download torrent
+
+### Login (optionnal)
+
+**I highly recommend you to not use your main account, YggTorrent ban bots**
+
+Requiered only for download torrent's file
+
+``` python
+
+if(scraper.login("myidentifiant", "mypassword")):
+    print("Login success")
+else:
+    print("Login failed")
+
+```
+
+### Search torrents
+
+Return url's results torrent for specified search
+
+``` python
+
+torrents_url = scraper.search({name : "walking dead s08"})
+
+"""
+
+https://www2.yggtorrent.ch/torrent/filmvid▒o/s▒rie-tv/227730-the+walking+dead+s08+complete+vostfr+proper+720p+hdtv+x264-expm5
+https://www2.yggtorrent.ch/torrent/filmvid▒o/s▒rie-tv/227752-the+walking+dead+s08+complete+vostfr+proper+hdtv+xvid-expm5
+https://www2.yggtorrent.ch/torrent/filmvid▒o/s▒rie-tv/227763-the+walking+dead+s08+vostfr+web-dl+x264-ark01
+https://www2.yggtorrent.ch/torrent/filmvid▒o/s▒rie-tv/227764-the+walking+dead+s08+vostfr+720p+amzn+web-dl+dd5+1+h264-ark01
+https://www2.yggtorrent.ch/torrent/filmvid▒o/s▒rie-tv/227765-the+walking+dead+s08+vostfr+1080p+amzn+web-dl+ddp5+1+h264-ark01
+
+...
+
+"""
+
+```
+
+#### Search an torrents by uploader
+
+Return url's results torrent for specified search
+
+``` python
+
+torrents_url = scraper.search({name : "walking dead s09", "uploader" : 'brandit'})
+
+```
+
+#### Search torrents with sorted results
+
+Return url's results torrent for specified search
+
+**YggTorrent's sorting is bugged, in general the results are sorted but sometimes you can find some torrents at a wrong position**
+
+-   sort : name/publish_date/size/completed/seed/leech
+-   order : asc/desc
+
+``` python
+
+torrents_url = scraper.search({"name": "blue oyster cult", "sort": "completed", "order": "desc"})
+
+```
+
+#### Search by category, subcategory and options_index
+
+**Complete categories tree is available in file [categories.py](https://github.com/Harkame/YggTorrentScraper/blob/master/yggtorrentscraper/categories.py)**
+
+``` python
+
+parameters = {
+    "name": "walking dead",
+    "category": "films_&_videos",
+    "subcategory": "serie_tv",
+    "options": {
+        "langue": {"francais_(vff/truefrench)"},
+        "episode": {"saison_complete"},
+        "qualite": {"bluray_[full]"},
+    },
+}
+
+
+research = scraper.search(parameters)
+
+```
+
+### Most completed
+
+Return url's of most completed (top 100) downloaded torrents
+
+``` python
+
+most_completed = scraper.most_completed()
+
+"""
+
+https://www2.yggtorrent.ch/torrent/filmvideo/serie-tv/440445-game-of-thrones-s08e02-multi-1080p-amzn-web-dl-dd5-1-x264-ark01
+https://www2.yggtorrent.ch/torrent/application/windows/316475-microsoft-toolkit-v2-6-4-activateur-office-2016---2019-windows-10
+https://www2.yggtorrent.ch/torrent/filmvideo/animation/431851-asterix-le-secret-de-la-potion-magique-2018-french-1080p-hdlight-x264-ac3-toxic
+https://www2.yggtorrent.ch/torrent/application/windows/330032-windows-microsoft-office-2019-build-10730-20102-activation-francais
+https://www2.yggtorrent.ch/torrent/filmvideo/serie-tv/436928-game-of-thrones-s08e01-multi-1080p-amzn-web-dl-dd5-1-h264-ark01
+
+...
+
+"""
+
+```
+
+### Details
+
+Get torrent's details
+
+``` python
+torrent = scraper.extract_details('https://www2.yggtorrent.ch/torrent/filmvideo/serie-tv/440445-game-of-thrones-s08e02-multi-1080p-amzn-web-dl-dd5-1-x264-ark01')
+
+print(torrent)
+
+"""
+
+Name      : Game.of.Thrones.S08E02.MULTi.1080p.AMZN.WEB-DL.DD5.1.x264-ARK01
+Url       : N/A (Not authentified)
+Uploaded  : 2019-04-22 06:10:00
+Size      : 1.40Go
+Uploader  : Anonyme
+Completed : 37157
+Seeders   : 2254
+Leechers  : 2
+
+Files (1) :
+
+Comments (15) :
+
+"""
+
+print(torrent.__str__(files=True, comments=True))
+
+"""
+Name      : Game.of.Thrones.S08E02.MULTi.1080p.AMZN.WEB-DL.DD5.1.x264-ARK01
+Url       : N/A
+Uploaded  : 2019-04-22 06:10:00
+Size      : 1.40Go
+Uploader  : Anonyme
+Completed : 37157
+Seeders   : 2254
+Leechers  : 2
+
+Files (1) :
+size      :
+file_name :Game.of.Thrones.S08E02.MULTi.1080p.AMZN.WEB-DL.DD5.1.x264-ARK01.mkv"}
+
+
+Comments (15) :
+Author : Beleg_5
+Posted : 10 jours
+Text   : Merci beaucoup.
+
+Author : StephZher
+Posted : 20 jours
+Text   : Merci c'est cool !
+
+...
+
+"""
+
+```
+
+### Download
+
+Download torrent's file (.torrent), requiered to be logged
+
+``` python
+
+if(scraper.login("myidentifiant", "mypassword")):
+    print("Login success")
+
+    torrent = scraper.extract_details('https://www2.yggtorrent.ch/torrent/filmvideo/serie-tv/440445-game-of-thrones-s08e02-multi-1080p-amzn-web-dl-dd5-1-x264-ark01')
+
+    scraper.download_from_torrent(torrent)
+
+    """
+    OR
+    """
+
+    scraper.download_from_torrent_url('https://www2.yggtorrent.ch/torrent/filmvideo/serie-tv/440445-game-of-thrones-s08e02-multi-1080p-amzn-web-dl-dd5-1-x264-ark01')
+else:
+    print("Login failed")
+
+```
+
+### Logout (optionnal)
+
+``` python
+
+scraper.logout()
+
+```
+
+## TODO
+
+-   More tests, find non bugged search (especially for sort tests)
+
+## Test
+
+Declare environment variables (requiered for login, download tests)
+
+-   YGGTORRENT_IDENTIFIANT
+-   YGGTORRENT_PASSWORD
+
+``` bash
+
+pip install tox
+
+tox
+
+```

config.yml

@@ -0,0 +1,35 @@
+list_research:
+  - research:
+    name:
+      The.Walking.Dead.S09
+    limit:
+      0
+    sort: #name|publish_date|size|completed|seed|leech
+      size
+    order: #asc|desc
+      desc
+
+  # - research:
+  #   name:
+  #     test
+  #   files:
+  #     - avi
+  #     - mkv
+  #   uploaders:
+  #     - uploader1
+  #     - uploader2
+  #   descriptions:
+  #       - description1
+  #       - description2
+  #   sort: #name|publish_date|size|completed|seed|leech
+  #     size
+  #   order: #asc|desc
+  #     desc
+  #   limit:
+  #     1
+
+destination_path:
+  ./torrents
+
+yggtorrent_tld:
+  ch

help.txt

@@ -0,0 +1,33 @@
+git update-index --assume-unchanged config.yml
+
+git update-index --no-assume-unchanged config.yml
+
+https://packaging.python.org/tutorials/packaging-projects/
+
+pip install setuptools wheel twine
+
+python setup.py sdist bdist_wheel
+
+twine upload dist/*.tar.gz
+
+codecov --token=$CODECOV_TOKEN -f=./coverage.xml
+
+https://coverage.readthedocs.io/en/coverage-4.2/excluding.html
+
+def pretty_print_POST(request):
+    """
+    At this point it is completely built and ready
+    to be fired; it is "prepared".
+
+    However pay attention at the formatting used in
+    this function because it is programmed to be pretty
+    printed and may differ from the actual request.
+    """
+    print(
+        "{}\n{}\r\n{}\r\n\r\nBody : {}".format(
+            "-----------START-----------",
+            request.method + " " + request.url,
+            "\r\n".join("{}: {}".format(k, v) for k, v in request.headers.items()),
+            request.body,
+        )
+    )

requirements.txt

@@ -0,0 +1,4 @@
+BeautifulSoup4
+lxml
+requests
+requests_toolbelt