humble

HTTP Headers Analyzer

"A journey of a thousand miles begins with a single step. - Lao Tzu"

"And if you don't keep your feet, there's no knowing where you might be swept off to. - Bilbo Baggins"

Table of contents

Features
Screenshots
Installation & Update
Usage
Advanced Usage
Checks: Missing Headers
Checks: Fingerprint Headers
Checks: Deprecated Headers and Insecure Values
Checks: Empty Values
Guidelines included
To-Do
Further Reading
Contribute
Acknowledgements
License

Features

✔️ 14 checks of missing HTTP response headers.
✔️ 997 checks of fingerprinting through HTTP response headers.
✔️ 86 checks of deprecated HTTP response headers/protocols or with insecure/wrong values.
✔️ TLS/SSL checks: requires https://testssl.sh/ and Unix machine!.
✔️ Browser compatibility check for enabled security headers.
✔️ Two types of analysis: brief and detailed, along with HTTP response headers.
✔️ Export of analysis to HTML5, JSON, PDF 1.4 and TXT.
✔️ The analysis includes dozens of references, official documentation and technical articles.
✔️ i10n: analysis results in English or Spanish.
✔️ Saves each analysis, showing (at the end) the improvements or deficiencies in relation to the last one.
✔️ Shows analysis statistics: either against a specific URL or all of them.
✔️ Shows fingerprint statistics: either against a specific term or the Top 20.
✔️ Code reviewed via Flake8, SonarLint and Sourcery.
✔️ Tested, one by one, on thousands of URLs.
✔️ Fully tested and working on Windows (10 20H2 - 19042.985) and Linux (Kali 2021.1).
✔️ Almost all the code under one of the most permissive licenses: MIT.
✔️ Regularly updated.
✔️ Technical resource in OWASP Secure Headers Project and Kali Linux (this should be the latest Release).
✔️ Developed entirely in my spare time, no strings attached: feel free to try it out and integrate it into your projects!.

Screenshots

.: (Windows) - Brief analysis.

.: (Linux) - Brief analysis and retrieved HTTP headers.

.: (Linux) - Detailed analysis in Spanish.

.: (Linux) - TLS/SSL checks (requires https://testssl.sh/ and Unix machine, for now!).

.: (Linux) - List of HTTP fingerprint headers based on a specific term.

.: (Windows) - Detailed analysis saved as PDF. Example.

.: (Linux) - Detailed analysis saved as HTML. Example.

.: (Linux) - Brief analysis saved as JSON. Example.

.: (Linux) - Analysis history file: Date, URL, Missing, Fingerprint, Deprecated/Insecure, Empty headers & Total warnings (the four previous totals).

.: (Linux) - Statistics of the analysis performed against a specific URL.

.: (Linux) - Statistics of the analysis performed against all URLs.

Installation & Update

NOTE: Python 3.9 or higher is required.

# install python3 and python3-pip if not exist
(Windows) https://www.python.org/downloads/windows/
(Linux) if not installed by default, install them via, e.g. Synaptic, apt, dnf, yum ...

# install git
(Windows) https://git-scm.com/download/win
(Linux) https://git-scm.com/download/linux

# clone the repository
$ git clone https://github.com/rfc-st/humble.git

# change the working directory to humble
$ cd humble

# install the requirements
$ pip3 install -r requirements.txt

# update humble (every week, inside humble's working directory)
$ git pull

# or download the latest release (every four to five weeks)
https://github.com/rfc-st/humble/releases

Usage

(Windows) $ py humble.py
(Linux)   $ python3 humble.py

usage: humble.py [-h] [-a] [-b] [-e [PATH]] [-f [TERM]] [-g] [-l {es}] [-o {html,json,pdf,txt}] [-r] [-u URL] [-v]

humble (HTTP Headers Analyzer) - https://github.com/rfc-st/humble

options:
  -h, --help              show this help message and exit
  -a                      show statistics of the performed analysis (will be global if '-u URL' is omitted)
  -b                      show a brief analysis (if omitted, a detailed one will be shown)
  -e [PATH]               show TLS/SSL checks; requires the PATH of testssl.sh (https://testssl.sh/); Unix is required (for now)!.
  -f [TERM]               show fingerprint statistics (will be the Top 20 if "TERM", e.g. "Google", is omitted)
  -g                      show guidelines for securing popular web servers/services
  -l {es}                 show the analysis in the indicated language (if omitted, English will be used)
  -o {html,json,pdf,txt}  save analysis to 'URL_headers_yyyymmdd.ext' file (.json files will contain a brief analysis)
  -r                      show full HTTP response headers and a detailed analysis
  -u URL                  schema and URL to analyze. E.g. https://google.com
  -v, --version           show the version of this tool and check for updates

Advanced Usage

.: (Linux) - Show only the analysis summary.

$ python3 humble.py -u https://www.spacex.com | grep -A 8 "\!." | sed $'1i \n'

.: (Windows, in Spanish) - show only the analysis summary (PowerShell >= 7 required).

$ py humble.py -u https://www.spacex.com -l es | Select-String -Pattern '!.' -Context 1,8 -NoEmphasis

.: (Linux) - Show only the URL, date and analysis summary.

$ python3 humble.py -u https://www.spacex.com | grep -A7 -E "0. Info|\!." | grep -v "^\[1\." | sed 's/[--]//g' | sed -e '/./b' -e :n -e 'N;s/\n$//;tn' | sed $'1i \n'

.: (Linux) - Show only the deprecated headers/protocols and insecure values.

$ python3 humble.py -u https://www.spacex.com | sed '/3. /,/4. /!d' | sed '$d' | sed $'1i \n'

.: (Linux) - Check for HTTP client errors (4XX).

$ python3 humble.py -u https://block.fiverr.com | grep -A1 -B5 'Note : \|Nota : ' --color=never

.: (Linux) - Analyze multiple URLs and save the results as PDFs.

$ datasets=('https://facebook.com' 'https://www.microsoft.com' 'https://www.spacex.com'); for dataset in "${datasets[@]}"; do python3 humble.py -u "$dataset" -o pdf; done

Checks: Missing Headers

Show / Hide

Cache-Control	Clear-Site-Data	Content-Type
Content-Security-Policy	Cross-Origin-Embedder-Policy	Cross-Origin-Opener-Policy
Cross-Origin-Resource-Policy	NEL	Permissions-Policy
Referrer-Policy	Strict-Transport-Security	X-Content-Type-Options
X-Frame-Options	X-Permitted-Cross-Domain-Policies

Checks: Fingerprint headers

Check this file.

Checks: Deprecated headers/protocols and insecure values

Check this file.

Checks: Empty values

Any HTTP response header.

Guidelines included to enable security HTTP headers

• Amazon AWS
• Apache HTTP Server
• Cloudflare
• MaxCDN
• Microsoft Internet Information Services
• Nginx

To-do

• Add more header/value checks (only security-oriented)
• Google Style Python Docstrings and maybe documentation via Sphinx.

Further reading

https://caniuse.com/
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://github.com/search?q=http+headers+analyze
https://github.com/search?q=http+headers+secure
https://github.com/search?q=http+headers+security
https://owasp.org/www-project-secure-headers/
https://securityheaders.com/
https://scotthelme.co.uk/
https://webtechsurvey.com/common-response-headers
https://www.w3.org

Contribute

• Report a Bug.
• Create a Feature request.
• Report a Security Vulnerability.
• Send me an email with your suggestions!: [email protected]

Thanks for your time!! :).

Acknowledgements

• Azathothas for reporting this bug.
• bulaktm for this suggestion.
• David, for believing in the usefulness of this tool.
• Eduardo, for making possible the first Demo and for the example "(Linux) - Analyze multiple URLs and save the results as PDFs".
• gl4nce for this suggestion.
• İDRİS BUDAK for reporting the need to this check.
• manuel-sommer for this and this suggestions.

License

MIT © 2020-2023 Rafa 'Bluesman' Faura ([email protected])
Original Creator - Rafa 'Bluesman' Faura ([email protected])