Github: allenai/ir_datasets

ir_datasets: Catalog

ir_datasets provides a common interface to many IR ranking datasets.

Getting Started

Install with pip:

pip install ir_datasets==0.3.1

Guides:

Dataset Index

✅: Data available as automatic download

⚠️: Data available from a third party

Dataset docs queries qrels scoreddocs docpairs
antique
antique/test
antique/test/non-offensive
antique/train
antique/train/split200-train
antique/train/split200-valid
aquaint⚠️
aquaint/trec-robust-2005⚠️
car
car/v1.5
car/v1.5/test200
car/v1.5/train/fold0
car/v1.5/train/fold1
car/v1.5/train/fold2
car/v1.5/train/fold3
car/v1.5/train/fold4
car/v1.5/trec-y1
car/v1.5/trec-y1/auto
car/v1.5/trec-y1/manual
clinicaltrials
clinicaltrials/2017
clinicaltrials/2017/trec-pm-2017
clinicaltrials/2017/trec-pm-2018
clinicaltrials/2019
clinicaltrials/2019/trec-pm-2019
clueweb09⚠️
clueweb09/ar⚠️
clueweb09/catb⚠️
clueweb09/catb/trec-web-2009⚠️
clueweb09/catb/trec-web-2010⚠️
clueweb09/catb/trec-web-2011⚠️
clueweb09/catb/trec-web-2012⚠️
clueweb09/de⚠️
clueweb09/en⚠️
clueweb09/en/trec-web-2009⚠️
clueweb09/en/trec-web-2010⚠️
clueweb09/en/trec-web-2011⚠️
clueweb09/en/trec-web-2012⚠️
clueweb09/es⚠️
clueweb09/fr⚠️
clueweb09/it⚠️
clueweb09/ja⚠️
clueweb09/ko⚠️
clueweb09/pt⚠️
clueweb09/trec-mq-2009⚠️
clueweb09/zh⚠️
clueweb12⚠️
clueweb12/b13⚠️
clueweb12/b13/clef-ehealth⚠️
clueweb12/b13/clef-ehealth/cs⚠️
clueweb12/b13/clef-ehealth/de⚠️
clueweb12/b13/clef-ehealth/fr⚠️
clueweb12/b13/clef-ehealth/hu⚠️
clueweb12/b13/clef-ehealth/pl⚠️
clueweb12/b13/clef-ehealth/sv⚠️
clueweb12/b13/ntcir-www-1⚠️
clueweb12/b13/ntcir-www-2⚠️
clueweb12/b13/ntcir-www-3⚠️
clueweb12/b13/trec-misinfo-2019⚠️
clueweb12/trec-web-2013⚠️
clueweb12/trec-web-2014⚠️
codesearchnet
codesearchnet/challenge
codesearchnet/test
codesearchnet/train
codesearchnet/valid
cord19
cord19/fulltext
cord19/fulltext/trec-covid
cord19/trec-covid
gov⚠️
gov/trec-web-2002⚠️
gov/trec-web-2002/named-page⚠️
gov/trec-web-2003⚠️
gov/trec-web-2003/named-page⚠️
gov/trec-web-2004⚠️
gov2⚠️
gov2/trec-mq-2007⚠️
gov2/trec-mq-2008⚠️
gov2/trec-tb-2004⚠️
gov2/trec-tb-2005⚠️
gov2/trec-tb-2005/efficiency⚠️
gov2/trec-tb-2005/named-page⚠️
gov2/trec-tb-2006⚠️
gov2/trec-tb-2006/efficiency⚠️
gov2/trec-tb-2006/efficiency/10k⚠️
gov2/trec-tb-2006/efficiency/stream1⚠️
gov2/trec-tb-2006/efficiency/stream2⚠️
gov2/trec-tb-2006/efficiency/stream3⚠️
gov2/trec-tb-2006/efficiency/stream4⚠️
gov2/trec-tb-2006/named-page⚠️
highwire
highwire/trec-genomics-2006
highwire/trec-genomics-2007
medline
medline/2004
medline/2004/trec-genomics-2004
medline/2004/trec-genomics-2005
medline/2017
medline/2017/trec-pm-2017
medline/2017/trec-pm-2018
msmarco-document
msmarco-document/dev
msmarco-document/eval
msmarco-document/orcas
msmarco-document/train
msmarco-document/trec-dl-2019
msmarco-document/trec-dl-2019/judged
msmarco-document/trec-dl-2020
msmarco-document/trec-dl-2020/judged
msmarco-passage
msmarco-passage/dev
msmarco-passage/dev/judged
msmarco-passage/dev/small
msmarco-passage/eval
msmarco-passage/eval/small
msmarco-passage/train
msmarco-passage/train/judged
msmarco-passage/train/medical
msmarco-passage/train/split200-train
msmarco-passage/train/split200-valid
msmarco-passage/trec-dl-2019
msmarco-passage/trec-dl-2019/judged
msmarco-passage/trec-dl-2020
msmarco-passage/trec-dl-2020/judged
msmarco-qna
msmarco-qna/dev
msmarco-qna/eval
msmarco-qna/train
nfcorpus
nfcorpus/dev
nfcorpus/dev/nontopic
nfcorpus/dev/video
nfcorpus/test
nfcorpus/test/nontopic
nfcorpus/test/video
nfcorpus/train
nfcorpus/train/nontopic
nfcorpus/train/video
nyt⚠️
nyt/train⚠️
nyt/valid⚠️
pmc
pmc/v1
pmc/v1/trec-cds-2014
pmc/v1/trec-cds-2015
pmc/v2
pmc/v2/trec-cds-2016
trec-arabic⚠️
trec-arabic/ar2001⚠️
trec-arabic/ar2002⚠️
trec-mandarin⚠️
trec-mandarin/trec5⚠️
trec-mandarin/trec6⚠️
trec-robust04⚠️
trec-robust04/fold1⚠️
trec-robust04/fold2⚠️
trec-robust04/fold3⚠️
trec-robust04/fold4⚠️
trec-robust04/fold5⚠️
trec-spanish⚠️
trec-spanish/trec3⚠️
trec-spanish/trec4⚠️
tweets2013-ia
tweets2013-ia/trec-mb-2013
tweets2013-ia/trec-mb-2014
vaswani
wikir
wikir/en1k
wikir/en1k/test
wikir/en1k/training
wikir/en1k/validation
wikir/en59k
wikir/en59k/test
wikir/en59k/training
wikir/en59k/validation
wikir/es13k
wikir/es13k/test
wikir/es13k/training
wikir/es13k/validation
wikir/fr14k
wikir/fr14k/test
wikir/fr14k/training
wikir/fr14k/validation
wikir/it16k
wikir/it16k/test
wikir/it16k/training
wikir/it16k/validation

Other Versions