Github: allenai/ir_datasets

`ir_datasets`: Catalog

ir_datasets provides a common interface to many IR ranking datasets.

Getting Started

Install with pip:

pip install ir_datasets==0.3.2

Guides:

Dataset Index

✅: Data available as automatic download

⚠️: Data available from a third party

Dataset	docs	queries	qrels	scoreddocs	docpairs
`antique`	✅
`antique/test`	✅	✅	✅
`antique/test/non-offensive`	✅	✅	✅
`antique/train`	✅	✅	✅
`antique/train/split200-train`	✅	✅	✅
`antique/train/split200-valid`	✅	✅	✅
`aquaint`	⚠️
`aquaint/trec-robust-2005`	⚠️	✅	✅
`car`
`car/v1.5`	✅
`car/v1.5/test200`	✅	✅	✅
`car/v1.5/train/fold0`	✅	✅	✅
`car/v1.5/train/fold1`	✅	✅	✅
`car/v1.5/train/fold2`	✅	✅	✅
`car/v1.5/train/fold3`	✅	✅	✅
`car/v1.5/train/fold4`	✅	✅	✅
`car/v1.5/trec-y1`	✅	✅
`car/v1.5/trec-y1/auto`	✅	✅	✅
`car/v1.5/trec-y1/manual`	✅	✅	✅
`clinicaltrials`
`clinicaltrials/2017`	✅
`clinicaltrials/2017/trec-pm-2017`	✅	✅	✅
`clinicaltrials/2017/trec-pm-2018`	✅	✅	✅
`clinicaltrials/2019`	✅
`clinicaltrials/2019/trec-pm-2019`	✅	✅	✅
`clueweb09`	⚠️
`clueweb09/ar`	⚠️
`clueweb09/catb`	⚠️
`clueweb09/catb/trec-web-2009`	⚠️	✅	✅
`clueweb09/catb/trec-web-2010`	⚠️	✅	✅
`clueweb09/catb/trec-web-2011`	⚠️	✅	✅
`clueweb09/catb/trec-web-2012`	⚠️	✅	✅
`clueweb09/de`	⚠️
`clueweb09/en`	⚠️
`clueweb09/en/trec-web-2009`	⚠️	✅	✅
`clueweb09/en/trec-web-2010`	⚠️	✅	✅
`clueweb09/en/trec-web-2011`	⚠️	✅	✅
`clueweb09/en/trec-web-2012`	⚠️	✅	✅
`clueweb09/es`	⚠️
`clueweb09/fr`	⚠️
`clueweb09/it`	⚠️
`clueweb09/ja`	⚠️
`clueweb09/ko`	⚠️
`clueweb09/pt`	⚠️
`clueweb09/trec-mq-2009`	⚠️	✅	✅
`clueweb09/zh`	⚠️
`clueweb12`	⚠️
`clueweb12/b13`	⚠️
`clueweb12/b13/clef-ehealth`	⚠️	✅	✅
`clueweb12/b13/clef-ehealth/cs`	⚠️	✅	✅
`clueweb12/b13/clef-ehealth/de`	⚠️	✅	✅
`clueweb12/b13/clef-ehealth/fr`	⚠️	✅	✅
`clueweb12/b13/clef-ehealth/hu`	⚠️	✅	✅
`clueweb12/b13/clef-ehealth/pl`	⚠️	✅	✅
`clueweb12/b13/clef-ehealth/sv`	⚠️	✅	✅
`clueweb12/b13/ntcir-www-1`	⚠️	✅	✅
`clueweb12/b13/ntcir-www-2`	⚠️	✅	✅
`clueweb12/b13/ntcir-www-3`	⚠️	✅
`clueweb12/b13/trec-misinfo-2019`	⚠️	✅	✅
`clueweb12/trec-web-2013`	⚠️	✅	✅
`clueweb12/trec-web-2014`	⚠️	✅	✅
`codesearchnet`	✅
`codesearchnet/challenge`	✅	✅	✅
`codesearchnet/test`	✅	✅	✅
`codesearchnet/train`	✅	✅	✅
`codesearchnet/valid`	✅	✅	✅
`cord19`	✅
`cord19/fulltext`	✅
`cord19/fulltext/trec-covid`	✅	✅	✅
`cord19/trec-covid`	✅	✅	✅
`cord19/trec-covid/round1`	✅	✅	✅
`cord19/trec-covid/round2`	✅	✅	✅
`cord19/trec-covid/round3`	✅	✅	✅
`cord19/trec-covid/round4`	✅	✅	✅
`cord19/trec-covid/round5`	✅	✅	✅
`gov`	⚠️
`gov/trec-web-2002`	⚠️	✅	✅
`gov/trec-web-2002/named-page`	⚠️	✅	✅
`gov/trec-web-2003`	⚠️	✅	✅
`gov/trec-web-2003/named-page`	⚠️	✅	✅
`gov/trec-web-2004`	⚠️	✅	✅
`gov2`	⚠️
`gov2/trec-mq-2007`	⚠️	✅	✅
`gov2/trec-mq-2008`	⚠️	✅	✅
`gov2/trec-tb-2004`	⚠️	✅	✅
`gov2/trec-tb-2005`	⚠️	✅	✅
`gov2/trec-tb-2005/efficiency`	⚠️	✅	✅
`gov2/trec-tb-2005/named-page`	⚠️	✅	✅
`gov2/trec-tb-2006`	⚠️	✅	✅
`gov2/trec-tb-2006/efficiency`	⚠️	✅	✅
`gov2/trec-tb-2006/efficiency/10k`	⚠️	✅
`gov2/trec-tb-2006/efficiency/stream1`	⚠️	✅
`gov2/trec-tb-2006/efficiency/stream2`	⚠️	✅
`gov2/trec-tb-2006/efficiency/stream3`	⚠️	✅	✅
`gov2/trec-tb-2006/efficiency/stream4`	⚠️	✅
`gov2/trec-tb-2006/named-page`	⚠️	✅	✅
`highwire`	✅
`highwire/trec-genomics-2006`	✅	✅	✅
`highwire/trec-genomics-2007`	✅	✅	✅
`medline`
`medline/2004`	✅
`medline/2004/trec-genomics-2004`	✅	✅	✅
`medline/2004/trec-genomics-2005`	✅	✅	✅
`medline/2017`	✅
`medline/2017/trec-pm-2017`	✅	✅	✅
`medline/2017/trec-pm-2018`	✅	✅	✅
`msmarco-document`	✅
`msmarco-document/dev`	✅	✅	✅	✅
`msmarco-document/eval`	✅	✅		✅
`msmarco-document/orcas`	✅	✅	✅	✅
`msmarco-document/train`	✅	✅	✅	✅
`msmarco-document/trec-dl-2019`	✅	✅	✅	✅
`msmarco-document/trec-dl-2019/judged`	✅	✅	✅	✅
`msmarco-document/trec-dl-2020`	✅	✅	✅	✅
`msmarco-document/trec-dl-2020/judged`	✅	✅	✅	✅
`msmarco-passage`	✅
`msmarco-passage/dev`	✅	✅	✅	✅
`msmarco-passage/dev/judged`	✅	✅	✅	✅
`msmarco-passage/dev/small`	✅	✅	✅
`msmarco-passage/eval`	✅	✅		✅
`msmarco-passage/eval/small`	✅	✅
`msmarco-passage/train`	✅	✅	✅	✅	✅
`msmarco-passage/train/judged`	✅	✅	✅	✅	✅
`msmarco-passage/train/medical`	✅	✅	✅	✅	✅
`msmarco-passage/train/split200-train`	✅	✅	✅	✅	✅
`msmarco-passage/train/split200-valid`	✅	✅	✅	✅	✅
`msmarco-passage/trec-dl-2019`	✅	✅	✅	✅
`msmarco-passage/trec-dl-2019/judged`	✅	✅	✅	✅
`msmarco-passage/trec-dl-2020`	✅	✅	✅	✅
`msmarco-passage/trec-dl-2020/judged`	✅	✅	✅	✅
`msmarco-qna`	✅
`msmarco-qna/dev`	✅	✅	✅	✅
`msmarco-qna/eval`	✅	✅		✅
`msmarco-qna/train`	✅	✅	✅	✅
`nfcorpus`	✅
`nfcorpus/dev`	✅	✅	✅
`nfcorpus/dev/nontopic`	✅	✅	✅
`nfcorpus/dev/video`	✅	✅	✅
`nfcorpus/test`	✅	✅	✅
`nfcorpus/test/nontopic`	✅	✅	✅
`nfcorpus/test/video`	✅	✅	✅
`nfcorpus/train`	✅	✅	✅
`nfcorpus/train/nontopic`	✅	✅	✅
`nfcorpus/train/video`	✅	✅	✅
`nyt`	⚠️
`nyt/train`	⚠️	✅	✅
`nyt/valid`	⚠️	✅	✅
`pmc`
`pmc/v1`	✅
`pmc/v1/trec-cds-2014`	✅	✅	✅
`pmc/v1/trec-cds-2015`	✅	✅	✅
`pmc/v2`	✅
`pmc/v2/trec-cds-2016`	✅	✅	✅
`trec-arabic`	⚠️
`trec-arabic/ar2001`	⚠️	✅	✅
`trec-arabic/ar2002`	⚠️	✅	✅
`trec-mandarin`	⚠️
`trec-mandarin/trec5`	⚠️	✅	✅
`trec-mandarin/trec6`	⚠️	✅	✅
`trec-robust04`	⚠️	✅	✅
`trec-robust04/fold1`	⚠️	✅	✅
`trec-robust04/fold2`	⚠️	✅	✅
`trec-robust04/fold3`	⚠️	✅	✅
`trec-robust04/fold4`	⚠️	✅	✅
`trec-robust04/fold5`	⚠️	✅	✅
`trec-spanish`	⚠️
`trec-spanish/trec3`	⚠️	✅	✅
`trec-spanish/trec4`	⚠️	✅	✅
`tweets2013-ia`	✅
`tweets2013-ia/trec-mb-2013`	✅	✅	✅
`tweets2013-ia/trec-mb-2014`	✅	✅	✅
`vaswani`	✅	✅	✅
`wikir`
`wikir/en1k`	✅
`wikir/en1k/test`	✅	✅	✅	✅
`wikir/en1k/training`	✅	✅	✅	✅
`wikir/en1k/validation`	✅	✅	✅	✅
`wikir/en59k`	✅
`wikir/en59k/test`	✅	✅	✅	✅
`wikir/en59k/training`	✅	✅	✅	✅
`wikir/en59k/validation`	✅	✅	✅	✅
`wikir/es13k`	✅
`wikir/es13k/test`	✅	✅	✅	✅
`wikir/es13k/training`	✅	✅	✅	✅
`wikir/es13k/validation`	✅	✅	✅	✅
`wikir/fr14k`	✅
`wikir/fr14k/test`	✅	✅	✅	✅
`wikir/fr14k/training`	✅	✅	✅	✅
`wikir/fr14k/validation`	✅	✅	✅	✅
`wikir/it16k`	✅
`wikir/it16k/test`	✅	✅	✅	✅
`wikir/it16k/training`	✅	✅	✅	✅
`wikir/it16k/validation`	✅	✅	✅	✅

ir_datasets: Catalog

Getting Started

Dataset Index

Other Versions

`ir_datasets`: Catalog