VALAN, short for Vision and Language Agent Navigation is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation, in photo-realistic environments, such as Matterport3D and StreetLearn. Such tasks require agents to interpret natural language instructions/dialog to navigate in photo-realistic environments in order to achieve prescribed navigation goals. We have added a minimal set of abstractions on top of SEED RL allowing us to generalize the architecture to solve a variety of other RL problems.
This package contains the implementations of the following problems:
See Mehta et al. for details about our implementation for Touchdown and the data supporting it.
For a detailed description of the architecture please read Lansing et al. Please cite the paper if you use the code from this repository in your work.
@article{lansing2019valan,
title={VALAN: Vision and Language Agent Navigation},
author={Larry Lansing and Vihan Jain and Harsh Mehta and Haoshuo Huang and Eugene Ie},
year={2019},
eprint={1912.03241},
archivePrefix={arXiv},
primaryClass={cs.LG}
}TODO
TODO
TODO
This is not an official Google product.