Introduction to Question Answering.
1. Paradigms for QA
Information Retrieval based approaches - workflow:
- Question processing
- detect question type, answer type (taxonomy), focus, relations, named entites
- formulate queries to send to search engine
- Passage retrieval
- retrieve ranked documents
- break into suitable passages and rerank
- Answer processing
- extract candidate answers
- rank candidates
There are detailed discussions about answer type detection, passage retrieval, answer extraction (processing, features, machine learning opportunities) in the original slides.
Knowledge based approaches - workflow:
- Build semantic representation of the queries
- datetime, location, entities, numerical quantities
- Map from semantics to query structure
- geospatial databases
- ontologies (wikipedia infoboxes, etc)
- restaurant review sources
- scientific databases
2. Evaluation Metrics
In addition to commonly used classification metrics in machine learning, mean reciprocal rank also can be used
\[
MRR = \frac{1}{N}\sum^{N}_{i=1}\frac{1}{rank_{i}},
\]
where \(rank_{i}\) is the rank of correct answer for query \(i\).