We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. We focus on improving deployed QA systems that do not allow re-training or when re-training comes at a high cost. Our re-ranking approach learns a similarity function using n-gram based features using the query, the answer and the initial system confidence as input. We implement a state-of-the-art QA pipeline using neural sentence embeddings that encode queries in the same space than the answer index and evaluate the QA our re-ranking approach with simulated user feedback.

qa system
Question answering system

re-ranking procedure
We train a re-ranking model for improving the top-10 answer candidates of a pre-trained QA system.