Security Headers Checker

TL; DR

The script (and burp plugin) validates whether the headers pertaining to security are present and if present, whether they have been configured securely. It implements checks identified by

Table of Contents

Introduction

Applications can set secure HTTP response headers as an additional layer of defense that prevents browsers from running into easy preventable vulnerabilities. The script in this repository validates whether the headers pertaining to security are present and if present, whether they have been configured securely. In summary, the script implements the checks identified by

Installation and Execution

Make sure you have Python 2 or Python 3 installed. The current limitation of Python 3 is that the 'ScriptSrc'-style tags of the Content-Security-Policy are not analyzed.

Install the dependencies:

pip install -r requirements.txt

Start the script as follows.

python securityheaders.py URI

Use the -h flag for all options.

python securityheaders.py -h

In the example below, we execute the script to validate the headers of google.com.

python securityheaders.py google.com

Checking the response headers of google.

By default the script will display all the response headers it received from the server. To not display those, execute the --skipheaders flag with the InfoCollector value.

python securityheaders.py google.com --skipcheckers InfoCollector

Do not display header info.

The script also shows if security headers are missing. To disable those checks, add HeaderMissingChecker to the --skipheaders flag.

python securityheaders.py google.com --skipcheckers InfoCollector HeaderMissingChecker

Do not display missing headers.

The script can show a list of supported checkers via the --listcheckers flag.

python securityheaders.py --listcheckers

Built-in checkers.

By default the script executes all of the listed checkers. Disable checkers with the --skipcheckers flag or execute specific checkers with the --checkers flag. If a checker has children, then the script skips or executes all the children checkers. In the example below, the script executes all checkers that find Content Security Policy issues, but skips the checkers that fire when the CSP header is missing.

python securityheaders.py https://scotthelme.co.uk --checkers CSPChecker --skipcheckers HeaderMissingChecker

CSP Checkers.

By default the script shows the output in a tabular format. To create output in a different format, use the --formatter flag. In the example below, the script outputs the findings as CSV. Note that formatters base64-encode the fields that may contain control characters for that format. For instance, for CSV the description is base 64 encoded as it may contain commas.

python securityheaders.py https://scotthelme.co.uk --formatter csv

Output to CSV.

python securityheaders.py --listformatters

List of supported output formats.

To write output to a file rather displaying it on the screen, use the --file flag.

python securityheaders.py https://scotthelme.co.uk --file ./tmp

To check multiple websites, separate them with a comma.

python securityheaders.py https://scotthelme.co.uk,https://google.com --skipcheckers InfoCollector HeaderMissingChecker

Multiple sites

To merge output into one table, use the --flatten flag. Avoid this if you are checking many (500k+) websites, as you may run out of memory.

python securityheaders.py https://scotthelme.co.uk,https://google.com --flatten --skipcheckers InfoCollector HeaderMissingChecker

Multiple sites

To load URLs from a file, use a filename rather than a URL.

python securityheaders.py top10.txt --flatten

If the file is a CSV file, tell the column with URLs with the --urlcolumn flag (zero-indexed). To skip the CSV header row, use the --startrow flag.

python securityheaders.py majestic_10.csv --startrow 2 --urlcolumn 2

To analyze a saved response, use the --response rather than a URL.

python securityheaders.py --response google.txt

By passing a single dash (-) to --response the file will be read from sys.stdin

python securityheaders.py --response -
HTTP/1.1 200 OK
Date: Sun, 14 Oct 2018 12:59:11 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en-GB"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop
[TRUNCATED]

To supply headers directly, use the --headers and specify the headers separated with a newline.

python securityheaders.py --headers $'foo: bar\nbar: baz'

Using Docker

For easy portablity, we added support for Docker for the cli tool.

Build the container:

docker build -t securityheaders:latest .

Run the container:

docker run -it --rm securityheaders:latest {URI} [options]

Note: if you wish to add your own bypass files, use Docker volumes:

docker run -it --rm -v ~/whitelists:/securityheaders/conf securityheaders:latest {URI} [options]

Using Burp

The checkers are also available as a BurpSuite plugin. The plugin does not display missing security headers or information about headers; i.e. it uses the --checker Checker --skipcheckers InfoCollector HeaderMissingChecker flags. Feel free to modify the code if you want to display those; I may or may not implement a configuration screen.

Install it as follows.

  1. Go to Extender, Extensions, and click on Add Extension. Select python and load the burpecheaders.py file. Load the burpsecheaders.py file
  2. Once BurpSuite loads the plugin successfully, visit a website and observe that the plugin reports issues under the scanner tab. Scanner shows issues of the plugin

Observe that the plugin highlights the offending header/directives/keywords in the response headers. BurpSuite highlights the insecure headers

Security Headers

This section lists background information that help you understand the issues the tool reports. The reference section at the end of this README points you to more detailed information.

Security headers are HTTP headers that enable the server to influence the behavior of the browser from a security perspective. The remainder of this section describes security headers and their common insecure configurations.

Content Security Policy

A Content Security Policy (CSP) consists of a set of directives that restrict how a webpage loads resources, such as scripts and media files. The CSP protects a web page from various attacks, such as cross-site scripting (XSS) and clickjacking.

The directives that a CSP support include the following:

The tool validates the following best practices for CSPs (CSPChecker).

Specify default-src

Always include the default-src directive as the CSP allows loading a resource if there is not a specific directive and default-src was not specified as a fall back mechanism.

The tool will thus flag the following policy as insecure (CSPMissingDirectiveChecker).

Content-Security-Policy: img-src 'none';

The policy does not specify default-src and thus allows loading of scripts of questionable origins.

Specify object-src, script-src, and base-uri

Explicitly specify object-src, script-src, and base-uri, as without those directives and a restrictive default-src directive, execution of malicious JavaScript is still possible:

The tool will thus flag the following policy as insecure (CSPMissingDirectiveChecker).

Content-Security-Policy: img-src 'none';

Avoid using unsafe-inline

Using 'unsafe-inline' allows execution of third-party JavaScript inline. It is better to use hashes or nonces for the inline scripts.

The tool will thus flag the following policy as insecure (CSPUnsafeInlineChecker).

Content-Security-Policy: script-src 'unsafe-inline';

Avoid using unsafe-eval

Using 'unsafe-eval' allows execution of untrusted JavaScript at runtime with eval.

The tool will thus flag the following policy as insecure (CSPUnsafeEvalChecker).

Content-Security-Policy: script-src 'unsafe-eval';

Avoid using the wild-card

Avoid setting object-src to the wildcard character *, as it allows loading of arbitrary plugins that can execute JavaScript (e.g. Flash).

The tool will thus flag the following policy as insecure (CSPWildCardChecker).

Content-Security-Policy: object-src '*';

Avoid specifying an IP source

Avoid using localhost as a source as it allows accessing a server with resources running on localhost. For production environments, it is recommended to remove this.

The tool will thus flag the following policy as insecure (CSPIPSourceChecker).

Content-Security-Policy: script-src https://127.0.0.1;

Avoid using deprecated directives

Avoid using deprecated directives. For instance, report-uri is deprecated in CSP3.

The tool will thus flag the following policy as insecure (CSPDeprecatedDirectiveChecker).

Content-Security-Policy: report-uri https://report-uri.com/report

Avoid using sources that start with http:

Avoid allowing resources to be loaded over HTTP, as an attacker may be able to modify those resources via e.g. a Man-in-the-middle attack. The impact depends on the type of resource.

The tool will thus flag the following policy as insecure (CSPPlainUrlSchemesChecker and CSPSCRHTTPChecker).

Content-Security-Policy: script-src http:;

Avoid using sources that are untrusted

Fetch directives define from where content may be loaded. These directives should be set to 'self', 'none' or a whitelist of trusted domains to prevent injection of third-party content. If the whitelist of trusted domains contains compromised hosts, then malicious media may be loaded into the application.

The tool validates whether the policy allows loading of resources from known untrusted/compromised hosts. The list of known untrusted domains is taken from https://github.com/google/csp-evaluator/tree/master/whitelist_bypasses.

The tool will thus flag the following policy as insecure as gstatic is know to host Angular libraries that can bypass this CSP (CSPFlashObjectWhitelistBypassChecker and CSPScriptWhitelistBypassChecker).

Content-Security-Policy: default-src 'none'; script-src https://gstatic.com

Moreover, the tool validates whether the policy trusts whole content delivery networks. The tool will thus flag the following policy as insecure as cloudflare.net hosts many libraries that can bypass this CSP (CSPScriptWhitelistCDNBypassChecker).

Content-Security-Policy: default-src 'none'; script-src *.cloudflare.net

HTTP Strict Transport Security

The HTTP Strict Transport Security (HSTS) header ensures that all communication (to a website) is being protected via SSL/TLS. If the browser encounters this header, it will use HTTPS for all subsequent communications with that server.

The header has the following directives:

An example policy as as follows:

 Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

The tool performs the following checks (HSTSChecker).

Include the includeSubdomains directive

From OWASP:

Cookies can be manipulated from sub-domains, so omitting the includeSubDomains option permits a broad range of cookie-related attacks that HSTS would otherwise prevent by requiring a valid certificate for a subdomain. Ensuring the "Secure Flag" is set on all cookies will also prevent, some, but not all, of the same attacks.

The tool will thus flag the following HSTS header as insecure as the includeSubDomains option is not set (HSTSSubdomainsChecker).

 Strict-Transport-Security: max-age=31536000

Avoid setting the max-age directive to 0

The browser will no longer regard the host as an HSTS host and therefore the HSTS header is not enforced.

The tool will flag the following as insecure (HSTSMaxAgeZeroChecker).

 Strict-Transport-Security: max-age=0; includeSubDomains; preload

X-Frame-Options

The X-Frame-Options HTTP header mitigates click-jacking attacks by limiting what can be rendered in a frame. The header can specify one of the following options:

The tool validates the following (XFrameOptionsChecker).

Avoid using allow-from

Allow-from allows rendering from other domains. Note that this might be OK and thus need to be manually verified.

The tool will thus flag the following as insecure (XFrameOptionsNotAllowFromChecker).

 X-Frame-Options: ALLOW-FROM https://example.com/

X-Content-Type-Options

The X-Content-Type-Options header stops a browser from trying to MIME-sniff the content type and forces it to stick with the declared content-type.

The tool performs the following checks (XContentTypeOptionsChecker).

Avoid setting the header to anything other than nosniff

Setting the value to nosniff blocks MIME-sniffing (i.e. guessing the content-type based on the contents) for script and style types and thus prevents transforming non-executable MIME types into executable MIME types.

The tool will NOT flag the following header (XContentTypeOptionsNoSniffChecker).

X-Content-Type-Options: nosniff

X-XSS-Protection

The X-XSS-Protection header sets the configuration for the cross-site scripting filter built into most browsers. If it is enabled, the browser will stop pages from loading when it detects reflected cross-site scripting (XSS) attacks.

The value can be any of the following:

The tool validates the following (XXSSProtectionChecker).

Avoid disabling the filter

The tool detects whether the filter has been disabled.

X-XSS-Protection: 0

Avoid using a HTTP report URI

The tool detects whether reports are send to a HTTP endpoint. The tool flags the following as insecure (XXSSProtectionHTTPSReportChecker).

 X-XSS-Protection: 1; report=http://example.com/report

Use block

Blocking the attack is preferred (XXSSProtectionBlockChecker). The tool will thus report the following as an informational error.

 X-XSS-Protection: 1

Cross-Origin Resource Sharing

A Cross-Origin Resource Sharing (CORS) policy controls whether and how content running on other origins can interact with the origin that publishes the policy.

The policy is defined by multiple headers:

The tool performs the following checks.

Avoid allowing access from all origins

Allowing access from all origins (*) allows the requested resource to be shared with any origin; i.e. the standard same origin policy enforced by the browser is discarded. The tool will thus report the following as insecure (AccessControlAllowOriginStarChecker).

Access-Control-Allow-Origin: *

Avoid using the null header with credentialed requests

Allowing the null origin for requests with credentials enables an attacker to hijack the victim's session (redirects and local files have a null origin). The tool will thus report the following as insecure (AccessControlAllowOriginNullChecker).

Access-Control-Allow-Origin: null
Access-Control-Allow-Credentials: true

Only allow HTTPS origins for requests with credentials

Allowing non-HTTPS origins in a CORS policy enables an attacker who is able to perform a man-in-the-middle (MiTM) between the victim and the trusted partner to inject malicious code which can be used to attack the server sending the policy. The tool will thus report the following as insecure (AccessControlAllowOriginHTTPCredsChecker).

Access-Control-Allow-Origin: http://google.com, https://facebook.com, http://synopsys.com
Access-Control-Allow-Credentials: true

Avoid setting the preflight time for longer than 30 minutes

The Access-Control-Max-Age header instructs the browser to cache responses to preflight requests. The time is how long the browser can cache the request. If this is too long, a browser may use outdated information. The tool will thus report the following as insecure (AccessControlMaxAgeTooLongChecker).

Access-Control-Max-Age: 20000

Avoid exposing sensitive headers

TODO

(e.g. do not expose auth headers)

Referrer-Policy

The Referer HTTP header is set by browsers to tell the server the page that brought it there. The Referrer-Policy header specifies whether the browser is allowed to send the Referer header.

The Referrer-Policy supports the following directives:

The tool identifies the following issues (ReferrerPolicyChecker).

Avoid using unsafe-url and origin-when-cross-origin

These values allow transfer of sensitive information (via the Referer header) from HTTPS environments to HTTP environments.

The tool flags the following as insecure (ReferrerPolicyInsecureChecker).

 Referrer-Policy: unsafe-url

Feature-Policy

The Feature-Policy header allows a site to control which features and APIs can be used in the browser (location services, etc.).

This policy is still under development: each browser can implement the directives it desires. The directives supported by Google Chrome are listed below.

The values for each of these features can be:

The tool identifies the following issues (FeaturePolicyChecker).

Avoid using the wild-card character

The wild-card character allows any embedee to access the feature in the given browsing context. The tool flags thus the following as insecure (FeaturePolicyWildCardChecker).

camera *; microphone *

Other

Syntax Errors

The tool also identifies the following syntactical errors (SyntaxChecker) for all headers.

References

The following pages were helpful in understanding the different security headers.

Clear-Site-Data References

Content-Security-Policy References

CORS References

Feature-Policy References

HPKP References

HTTP-Strict-Transport-Security (HSTS) References

Referrer-Policy References

X-Frame-Options (XFO) References

X-Content-Type-Options References

X-Download-Options References

X-Permitted-Cross-Domain-Policies References

X-XSS-Protection References

TODOs