I’m finally able to share some results from my PhD research. I’ve been working on a natural language inference system and the results we have so far from the experiments are very promising.
We achieve the state of the art result for sentence encoding approaches in Stanford NLI (SNLI) test set, achieving 86.6% accuracy. This is on par with the previous state of the art, but requires less trainable parameters to achieve the result.
We achieve the new state of the art in SciTail NLI task by AllenAI. Our score 86.0% is +2.7% absolute improvement on the previous state of the art.
We also tested our model using the Facebook AI’s SentEval sentence embedding evaluation library and our model outperforms Facebook’s own Infersent model in 7 out of 10 SentEval tasks and 8 out of 10 probing tasks designed to evaluate the models’ ability to capture some important linguistic properties.
Link to the paper: https://arxiv.org/abs/1808.08762
Link to the PyTorch code: https://github.com/Helsinki-NLP/HBMP