Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective, ICCV'17

Seong Joon Oh, Mario Fritz, Bernt Schiele.

Max-Planck Institute for Informatics.

Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective, ICCV'17

TEASER

Users like sharing personal photos with others through social media. At the same time, they might want to make automatic identification in such photos difficult or even impossible. Classic obfuscation methods such as blurring are not only unpleasant but also not as effective as one would expect. Recent studies on adversarial image perturbations (AIP) suggest that it is possible to confuse recognition systems effectively without unpleasant artifacts. However, in the presence of counter measures against AIPs, it is unclear how effective AIP would be in particular when the choice of counter measure is unknown. Game theory provides tools for studying the interaction between agents with uncertainties in the strategies. We introduce a general game theoretical framework for the user-recogniser dynamics, and present a case study that involves current state of the art AIP and person recognition techniques. We derive the optimal strategy for the user that assures an upper bound on the recognition rate independent of the recogniser's counter measure.

Installation

Clone this repository recursively.

$ git clone https://github.com/coallaoh/AIP.git --recursive

Install Caffe

$ cd caffe

Follow caffe installation to configure Makefile.config, and run

$ make -j50 && make pycaffe

Install Liblinear

Install Liblinear for crawling pretrained svm models.

Python requirements

Install Python requirements.

$ pip install numpy scipy && pip install -r ./pip-requirements

Install OpenCV for python, following the instructions in: http://opencv.org.

Data downloading

Download the pretrained person recognition models (convnet + svm), head crops, and PIPA annotations. 

$ ./downloads.sh

Running

The script for computing different variants of adversarial perturbations in the paper and evaluating them against recogniser's defense measures on the PIPA vallidation set is

$ ./src/aip/test.py

Before running it, please change the following variables according to your local machine.

The script can then run on the command line. After computing the game table for every strategy pair (e.g. table 4 in the paper), the equilibrium (optimal) strategies and the value of the game is computed via Nashpy (https://github.com/drvinceknight/Nashpy). Run the script 

$ ./src/game/eq.py

to obtain the game theoretical results in the paper.

Contact

For any problem with implementation or bug, please contact Seong Joon Oh (coallaoh at gmail).

Citation

    @inproceedings{joon17iccv,
        title = {Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective},
        author = {Oh, Seong Joon and Fritz, Mario and Schiele, Bernt},
        year = {2017},
        booktitle = {International Conference on Computer Vision (ICCV)},
        note = {to appear},
        pubstate = {published},
        tppubtype = {inproceedings}
    }