avast/retdec-regression-tests-framework

RetDec Regression Tests Framework

A framework for writing and running regression tests for RetDec and related tools.

Requirements

To run regression tests, you must have:

Python >= 3.4
Clang 3.9.1 (exactly this version; download a pre-built package from here and extract it somewhere)
Cloned our retdec repository, built and installed RetDec.
Cloned our retdec-regression-tests repository that contains test cases.

Additionally, on Microsoft Windows, you need a 32-bit and 64-bit GCC compiler from MSYS2:

Download and run an MSYS2 installer:
- 64-bit Windows: http://sourceforge.net/projects/msys2/files/Base/x86_64/msys2-x86_64-20150916.exe/download
Open an MSYS2 shell.
Install a 32-bit GCC compiler via pacman -S mingw-w64-i686-gcc.
Install a 64-bit GCC compiler via pacman -S mingw-w64-x86_64-gcc.

Additionally, to run tests for our IDA plugin, you must have:

IDA 7.x
Cloned our retdec-idaplugin repository, built and installed our IDA plugin.

Installation

Clone the repository.
Step inside it:
```
$ cd retdec-regression-tests-framework
```

Install all the needed third-party Python packages, either into a virtual environment:

$ python -m venv virtualenv
$ source virtualenv/bin/activate
$ pip install -r requirements.txt

or into your home directory:

$ pip install --user -r requirements.txt

Verify that all dependencies have been installed by running
```
python check_dependencies.py
```

Create a new file config_local.ini with the following content (you will need to adjust the paths; use Windows style paths on Windows, e.g. C:\path\to\clang):

[runner]
; Path to the extracted Clang package containing subdirectories such as bin, include, lib, share.
clang_dir = /path/to/clang
; Path to the cloned repository containing regression tests.
tests_root_dir = /path/to/retdec-regression-tests
; Path to the RetDec's installation directory.
retdec_install_dir = /path/to/retdec/installed

Additionally, if you plan to run tests for our IDA plugin, you have to also include the following settings into the [runner] section of the config_local.ini file:

idaplugin_tests_enabled = 1
; Path to the IDA 7.x directory.
idaplugin_ida_dir = /path/to/ida
; Path to our script run-ida-decompilation from the retdec-idaplugin repository.
idaplugin_script = /path/to/retdec-idaplugin/scripts/run-ida-decompilation.py

Use

Use MSYS2 shell on Microsoft Windows.

To run all tests, execute

$ python runner.py

To run only tests in the given directory, execute

$ python runner.py path/to/directory

For more information, execute

$ python runner.py --help

Documentation

The documentation is available online. To generate it locally, execute

$ make docs

Then, open docs/_build/html/index.html in your favorite web browser.

License

The framework includes several third-party libraries, whose code and licensing information is provided in the deps subdirectory.

Contributing

See RetDec contribution guidelines.