video2frame

Video2frame is also an easy-to-use tool to extract frames from video.

Why this tool

Forwchen's vid2frame tool is great, but I am always confused by their parameters. At the same time, I also want to add something I need to the tool.

So I re-wrote the code. And now, it is a new wheel. It is hard to make a PR since I changed the code style.

How to use

  1. Establish the environment

    We recommend using conda to establish the environment. Just using 

    conda env create -f install/conda-environment.yml

    You can also do it manually. This project relays on the following packages:

    • Python
    • FFmpeg
    • Python packages (can be installed using pip install -r install/pip-requirements.txt)
      • h5py
      • lmdb
      • numpy
      • easydict
      • tqdm

  2. Make the annotation json file

    The json file should like

    {
    "meta": {
        "class_num": 2,
        "class_name": [
            "class_1",
            "class_2"
        ]
    },
    "annotation": {
        "label1_abcdefg": {
            "path": "path/to/the/video/file_1.mp4",
            "class": 1
        },
        "label2_asdfghj": {
            "path": "path/to/the/video/file_2.mp4",
            "class": 2
        }
    }
}

  3. Extract frames using video2frame.py

    Examples

    • Using the default options:

      python video2frame.py dataset.json

    • Specify the output file name:

      python video2frame.py dataset.json --db_name my_dataset

    • Using lmdb rather than hdf5:

      python video2frame.py dataset.json --db_type LMDB

      or 

      python video2frame.py dataset.json --db_name my_dataset.lmdb

    • Random clip 5 seconds:

      python video2frame.py dataset.json --duration 5.0

    • Get 3 video clips with a length of 5 seconds:

      python video2frame.py dataset.json --clips 3 --duration 5.0

    • Resize the frames to 320x240:

      python video2frame.py dataset.json --resize_mode 1 --resize 320x240

    • Keep the aspect ration, and resize the shorter side to 320:

      python video2frame.py dataset.json --resize_mode 2 --resize S320

    • Keep the aspect ration, and resize the longer side to 240:

      python video2frame.py dataset.json --resize_mode 2 --resize L240

    • Extract 5 frames per second:

      python video2frame.py dataset.json --fps 5

    • Uniformly sample 16 frames per video:

      python video2frame.py dataset.json --sample_mode 1 --sample 16

    • Randomly sample 16 frames per video:

      python video2frame.py dataset.json --sample_mode 2 --sample 16

    • Use 16 threads to speed-up:

      python video2frame.py dataset.json --threads 16

    • Resize the frames to 320x240, extract one frame every two seconds, uniformly sample 32 frames per video, and using 20 threads:

      python video2frame.py dataset.json \
    --resize_mode 1 \ 
    --resize 320x240 \
    --fps 0.5 \
    --sample_mode 1 \
    --sample 32 \
    --threads 20

    All parameters

    usage: video2frame.py [-h] [--db_name DB_NAME]
                      [--db_type {LMDB,HDF5,FILE,PKL}] [--tmp_dir TMP_DIR]
                      [--clips CLIPS] [--duration DURATION]
                      [--resize_mode {0,1,2}] [--resize RESIZE] [--fps FPS]
                      [--sample_mode {0,1,2,3}] [--sample SAMPLE]
                      [--threads THREADS] [--keep]
                      annotation_file

positional arguments:
  annotation_file       The annotation file, in json format

optional arguments:
  -h, --help            show this help message and exit
  --db_name DB_NAME     The database to store extracted frames (default: None)
  --db_type {LMDB,HDF5,FILE,PKL}
                        Type of the database (default: HDF5)
  --tmp_dir TMP_DIR     Temporary folder (default: /tmp)
  --clips CLIPS         Num of clips per video (default: 1)
  --duration DURATION   Length of each clip (default: -1)
  --resize_mode {0,1,2}
                        Resize mode
                          0: Do not resize
                          1: 800x600: Resize to WxH
                          2: L600 or S600: keep the aspect ration and scale the longer/shorter side to s (default: 0)
  --resize RESIZE       Parameter of resize mode (default: None)
  --fps FPS             Sample the video at X fps (default: -1)
  --sample_mode {0,1,2,3}
                        Frame sampling options
                          0: Keep all frames
                          1: Uniformly sample n frames
                          2: Randomly sample n continuous frames
                          3: Randomly sample n frames
                          4: Sample 1 frame every n frames (default: 0)
  --sample SAMPLE       How many frames (default: None)
  --threads THREADS     Number of threads (default: 0)
  --keep                Do not delete temporary files at last (default: False)

Tools

  1. video_folder_to_json.py

    A json generator where the videos are arranged in this way:

    root/swimming/xxx.mp4
root/swimming/xxy.avi
root/swimming/xxz.flv

root/dancing/123.mkv
root/dancing/nsdf3.webm
root/dancing/asd932_.mov

  2. something_to_json.py

    A json generator that converts the Something-Something dataset.

  3. ucf101_to_json.py

    A json generator that converts the UCF101 dataset.

Examples

  1. pytorch_skvideo_dataset.py

    Get frames using skvideo package, when training and evaluating. It is OKay when your batch size is small, and your CPUs are powerful enough.

  2. pytorch_lmdb_video_dataset.py

    A PyTorch Dataset example to read LMDB dataset.

  3. pytorch_hdf5_video_dataset.py

    A PyTorch Dataset example to read HDF5 dataset.

    ALWAYS ENSURE num_workers=0 OR num_workers=1 OF YOUR DATA LOADER.

  4. pytorch_pkl_video_dataset.py

    A PyTorch Dataset example to read pickle dataset.

  5. pytorch_file_video_dataset.py

    A PyTorch Dataset example to read image files dataset.