This project is a prototype for experimental purposes only and production grade code is not released here.

Deep LSTM siamese network for text similarity

It is a tensorflow based implementation of deep siamese LSTM network to capture phrase/sentence similarity using character embeddings.

This code provides architecture for learning two kinds of tasks:

For both the tasks mentioned above it uses a multilayer siamese LSTM network and euclidian distance based contrastive loss to learn input pair similairty.

Capabilities

Given adequate training pairs, this model can learn Semantic as well as structural similarity. For eg:

Phrases :

For phrases, the model learns character based embeddings to identify structural/syntactic similarities.

Sentences :

(More examples Ref: semEval dataset)

For Sentences, the model uses pre-trained word embeddings to identify semantic similarities.

Categories of pairs, it can learn as similar:

Training Data

Environment

How to run

Training

$ python train.py [options/defaults]

options:
  -h, --help            show this help message and exit
  --is_char_based IS_CHAR_BASED
            is character based syntactic similarity to be used for phrases.
            if false then word embedding based semantic similarity is used.
            (default: True)
  --word2vec_model WORD2VEC_MODEL
                this flag will be used only if IS_CHAR_BASED is False
            word2vec pre-trained embeddings file (default: wiki.simple.vec)
  --word2vec_format WORD2VEC_FORMAT
            this flag will be used only if IS_CHAR_BASED is False
            word2vec pre-trained embeddings file format (bin/text/textgz)(default: text)
  --embedding_dim EMBEDDING_DIM
                        Dimensionality of character embedding (default: 100)
  --dropout_keep_prob DROPOUT_KEEP_PROB
                        Dropout keep probability (default: 0.5)
  --l2_reg_lambda L2_REG_LAMBDA
                        L2 regularizaion lambda (default: 0.0)
  --max_document_words MAX_DOCUMENT_WORDS
                        Max length (left to right max words to consider) in
                        every doc, else pad 0 (default: 100)
  --training_files TRAINING_FILES
                        Comma-separated list of training files (each file is
                        tab separated format) (default: None)
  --hidden_units HIDDEN_UNITS
                        Number of hidden units(default:50)
  --batch_size BATCH_SIZE
                        Batch Size (default: 128)
  --num_epochs NUM_EPOCHS
                        Number of training epochs (default: 200)
  --evaluate_every EVALUATE_EVERY
                        Evaluate model on dev set after this many steps
                        (default: 2000)
  --checkpoint_every CHECKPOINT_EVERY
                        Save model after this many steps (default: 2000)
  --allow_soft_placement [ALLOW_SOFT_PLACEMENT]
                        Allow device soft device placement
  --noallow_soft_placement
  --log_device_placement [LOG_DEVICE_PLACEMENT]
                        Log placement of ops on devices
  --nolog_device_placement

Evaluation

$ python eval.py --model graph#.pb

Performance

Phrases:

Sentences:

References

  1. Learning Text Similarity with Siamese Recurrent Networks
  2. Siamese Recurrent Architectures for Learning Sentence Similarity