CRS-support/ftw

Project: ftw (GitHub Link)

ftw-master
- CONTRIBUTORS.md
- ftw
  - testrunner.py
  - pytest_plugin.py
  - util.py
  - ruleset.py
  - http.py
  - util
    - output
      - path-10-valid-invalid-urle-preference.yaml
      - b08-02-request-body-urlencoded-param-name.yaml
      - m08-07-partial-quote.yaml
      - m03-03-multiple-boundaries.yaml
      - m09-data-after-last-boundary.yaml
      - path-35-utf8-encoded-fullwidth-mapping.yaml
      - m08-03-whitespace-in-quoted-boundary.yaml
      - m01-02-invalid-separator.yaml
      - m07-02-special-chars-in-boundary.yaml
      - m15-01-invalid-part.yaml
      - m14-01-disposition-php-quoting.yaml
      - path-34-utf8-invalid-encoding.yaml
      - b01-01-query-string.yaml
      - path-05-u-bestfit.yaml
      - m08-06-partial-quote.yaml
      - path-19-control-chars-encoded.yaml
      - m13-04-disposition-folding.yaml
      - m11-01-invalid-multipart-type.yaml
      - path-12-NUL-encoded.yaml
      - m03-02-multiple-boundaries.yaml
      - b03-03-header-referer.yaml
      - path-00-baseline.yaml
      - path-18-backslash-separator-u-encoded.yaml
      - m12-01-disposition-multiple-param-names.yaml
      - m13-06-disposition-folding.yaml
      - m11-03-invalid-multipart-type.yaml
      - m04-whitespace-after-parameter-name.yaml
      - path-29-utf8-separators-overlong-encoded-4.yaml
      - m18-cr-line.yaml
      - m17-02-first-boundary-crlf.yaml
      - m07-01-special-chars-in-boundary.yaml
      - m02-02-invalid-parameter-name.yaml
      - b03-01-header.yaml
      - m03-01-multiple-boundaries.yaml
      - path-06-utf8-bare-bestfit.yaml
      - path-08-invalid-url-encoding.yaml
      - path-03-utf8-encoded.yaml
      - m08-04-quote-in-quoted-boundary.yaml
      - path-37-unicode-normalization.yaml
      - m13-05-disposition-folding.yaml
      - m11-04-invalid-multipart-type.yaml
      - b10-04-multipart-file-contents.yaml
      - m06-whitespace-after-parameter-value.yaml
      - path-26-utf8-overlong-bare-4.yaml
      - path-20-control-chars-bare.yaml
      - path-14-backslash-separator.yaml
      - path-38-utf8-bare-fullwidth-mapping.yaml
      - m11-07-invalid-multipart-type.yaml
      - m13-01-disposition-folding.yaml
      - b06-01-request-protocol.yaml
      - path-04-utf8-bare.yaml
      - path-36-double-url-decoding.yaml
      - b03-02-header-user-agent.yaml
      - b03-05-header-authorization-username.yaml
      - path-32-utf8-separators-overlong-bare-4.yaml
      - b00-01-normal.yaml
      - b10-03-multipart-param-filename.yaml
      - m12-04-disposition-name-partial-quote.yaml
      - path-28-utf8-separators-overlong-encoded-3.yaml
      - m12-02-disposition-name-no-quotes.yaml
      - m16-lf-line-endings.yaml
      - path-23-utf8-overlong-encoded-4.yaml
      - path-22-utf8-overlong-encoded-3.yaml
      - path-33-u-fullwidth-mapping.yaml
      - m17-01-first-boundary-lf.yaml
      - path-11-valid-invalid-u-preference.yaml
      - b02-02-request-hostname-header.yaml
      - b03-06-header-authorization-password.yaml
      - b07-01-trailing-header-cookie.yaml
      - b03-04-header-cookie.yaml
      - path-16-forward-slash-separator-u-encoded.yaml
      - m12-05-disposition-name-partial-quote.yaml
      - m00-01-normal.yaml
      - path-02-u-encoding.yaml
      - b02-01-request-hostname-uri.yaml
      - path-15-forward-slash-separator-url-encoded.yaml
      - b09-01-request-body-json.yaml
      - m11-02-invalid-multipart-type.yaml
      - path-24-utf8-overlong-bare-2.yaml
      - b08-01-request-body-urlencoded-param-value.yaml
      - m12-03-disposition-name-single-quotes.yaml
      - m10-boundary-case-sensitivity.yaml
      - path-31-utf8-separators-overlong-bare-3.yaml
      - b10-02-multipart-param-name.yaml
      - b05-01-request-method.yaml
      - m11-05-invalid-multipart-type.yaml
      - m13-02-disposition-folding-isspace.yaml
      - m08-02-whitespace-in-quoted-boundary.yaml
      - m08-05-quote-in-quoted-boundary.yaml
      - m08-01-quoted-boundary.yaml
      - path-21-utf8-overlong-encoded-2.yaml
      - m11-06-invalid-multipart-type.yaml
      - path-01-url-encoding.yaml
      - path-17-backslash-separator-url-encoded.yaml
      - m13-03-disposition-folding.yaml
      - m01-01-invalid-separator.yaml
      - path-09-invalid-u-encoding.yaml
      - m19-multiple-ct-headers.yaml
      - path-25-utf8-overlong-bare-3.yaml
      - path-27-utf8-separators-overlong-encoded-2.yaml
      - b04-01-request-filename.yaml
      - m05-whitespace-before-parameter-value.yaml
      - m03-04-multiple-boundaries.yaml
      - b10-05-multipart-epilogue.yaml
      - path-13-NUL-bare.yaml
      - path-30-utf8-separators-overlong-bare-2.yaml
      - m08-08-whitespace-after-boundary.yaml
      - m02-01-invalid-parameter-name.yaml
      - path-07-utf8-encoded-bestfit.yaml
      - b10-01-multipart-preamble.yaml
    - ironbee.py
    - request_to_yaml.py
  - logchecker.py
  - __init__.py
  - errors.py
- MANIFEST
- LICENSE
- test
  - test_default.py
  - yaml
    - EXAMPLE.yaml
  - test_modsecurityv2.py
  - integration
    - test_logcontains.py
    - test_cookie.py
    - test_http.py
    - test_runner.py
    - LOGCONTAINSFIXTURE.yaml
    - test_nologcontains.py
    - BASICFIXTURE.yaml
    - COOKIEFIXTURE.yaml
    - NOLOGCONTAINSFIXTURE.yaml
  - __init__.py
  - unit
    - test_ruleset.py
    - test_response.py
    - test_logchecker.py
- setup.py
- setup.cfg
- .travis.yml
- README.md
- tools
  - build_journal.py
- requirements.txt
- .gitignore
- docs
  - YAMLFormat.md
  - ExtendingFTW.md
  - Journaling.md

Framework for Testing WAFs (FTW)

Purpose

This project was created by researchers from ModSecurity and Fastly to help provide rigorous tests for WAF rules. It uses the OWASP Core Ruleset V3 as a baseline to test rules on a WAF. Each rule from the ruleset is loaded into a YAML file that issues HTTP requests that will trigger these rules. Users can verify the execution of the rule after the tests are issued to make sure the expected response is received from an attack

Goals / Use cases include:

Find regressions in WAF deployments by using continuous integration and issuing repeatable attacks to a WAF
Provide a testing framework for new rules into ModSecurity, if a rule is submitted it MUST have corresponding positive & negative tests
Evaluate WAFs against a common, agreeable baseline ruleset (OWASP)
Test and verify custom rules for WAFs that are not part of the core rule set

For our 1.0 release announcement, check out the OWASP CRS Blog

Installation

git clone https://github.com/CRS-support/ftw.git
cd ftw
virtualenv env && source ./env/bin/activate
pip install -r requirements.txt
py.test -s -v test/test_default.py --ruledir=test/yaml

Writing your first tests

The core of FTW is it's extensible yaml based tests. This section lists a few resources on how they are formatted, how to write them and how you can use them.

OWASP CRS wrote a great blog post describing how FTW tests are written and executed.

YAMLFormat.md is ground truth of all yaml fields that are currently understood by FTW.

After reading these two resources, you should be able to get started in writing tests. You will most likely be checking against status code responses, or web request responses using the log_contains directive. For integrating FTW to test regexes within your WAF logs, refer to ExtendingFTW.md

Provisioning Apache+Modsecurity+OWASP CRS

If you require an environment for testing WAF rules, there has been one created with Apache, Modsecurity and version 3.0.0 of the OWASP core ruleset. This can be deployed by:

Checking out the repository: git clone https://github.com/fastly/waf_testbed.git
Typing vagrant up