Exeption: 403

[daveramseymusic]

Thanks for making this and sharing it, but I haven't been able to use it yet.

I tried and found this error:

`user$ rcp https://www.realclearpolitics.com/epolls/2021/governor/va/virginia_governor_youngkin_vs_mcauliffe-7373.html --output general.csv

Traceback (most recent call last):
File "/opt/anaconda3/bin/rcp", line 8, in
sys.exit(main())
File "/opt/anaconda3/lib/python3.7/site-packages/rcp/main.py", line 38, in main
poll_data = get_poll_data(url, csv_output=True)
File "/opt/anaconda3/lib/python3.7/site-packages/rcp/rcp.py", line 73, in get_poll_data
soup = _html(poll)
File "/opt/anaconda3/lib/python3.7/site-packages/rcp/rcp.py", line 15, in _html
raise Exception(res.status)
Exception: 403
`

[daveramseymusic]

I found a workaround as I did my own scraping HERE.

but this is a snippet of the code I used with BS4 to not get the 403 error:

# create a header so the website thinks this is a web-browser
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.76 Safari/537.36'} # This is chrome, you can set whatever browser you like
# save your response
r = requests.get(url, headers=headers)

print(r.status_code)
print(r.url)