))
Task-aware Retrieval with Instructions
Manage episode 355037182 series 3446693
Konten disediakan oleh Zeta Alpha. Semua konten podcast termasuk episode, grafik, dan deskripsi podcast diunggah dan disediakan langsung oleh Zeta Alpha atau mitra platform podcast mereka. Jika Anda yakin seseorang menggunakan karya berhak cipta Anda tanpa izin, Anda dapat mengikuti proses yang diuraikan di sini https://id.player.fm/legal.
Andrew Yates (Assistant Prof at University of Amsterdam) and Sergi Castella (Analyst at Zeta Alpha) discuss the paper "Task-aware Retrieval with Instructions" by Akari Asai et al. This paper proposes to augment a conglomerate of existing retrieval and NLP datasets with natural language instructions (BERRI, Bank of Explicit RetRieval Instructions) and use it to train TART (Multi-task Instructed Retriever).
📄 Paper: https://arxiv.org/abs/2211.09260
🍻 BEIR benchmark: https://arxiv.org/abs/2104.08663
📈 LOTTE (Long-Tail Topic-stratified Evaluation, introduced in ColBERT v2): https://arxiv.org/abs/2112.01488
Timestamps:
00:00 Intro: "Task-aware Retrieval with Instructions"
02:20 BERRI, TART, X^2 evaluation
04:00 Background: recent works in domain adaptation
06:50 Instruction Tuning 08:50 Retrieval with descriptions
11:30 Retrieval with instructions
17:28 BERRI, Bank of Explicit RetRieval Instructions
21:48 Repurposing NLP tasks as retrieval tasks
23:53 Negative document selection
27:47 TART, Multi-task Instructed Retriever
31:50 Evaluation: Zero-shot and X^2 evaluation
39:20 Results on Table 3 (BEIR, LOTTE)
50:30 Results on Table 4 (X^2-Retrieval)
55:50 Ablations
57:17 Discussion: user modeling, future work, scale
13 episode
Bagikan
