← home
Github: datasets/nfcorpus.py

ir_datasets: NFCorpus (NutritionFacts)

Index
  1. nfcorpus
  2. nfcorpus/dev
  3. nfcorpus/dev/nontopic
  4. nfcorpus/dev/video
  5. nfcorpus/test
  6. nfcorpus/test/nontopic
  7. nfcorpus/test/video
  8. nfcorpus/train
  9. nfcorpus/train/nontopic
  10. nfcorpus/train/video

"nfcorpus"

"NFCorpus is a full-text English retrieval data set for Medical Information Retrieval. It contains a total of 3,244 natural language queries (written in non-technical English, harvested from the NutritionFacts.org site) with 169,756 automatically extracted relevance judgments for 9,964 medical documents (written in a complex terminology-heavy language), mostly from PubMed."

docs
5.4K docs

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  }
}

"nfcorpus/dev"

Official dev set. Queries include both title and combinted "all" text field (titles, descriptions, topics, transcripts and comments)

queries
325 queries

Language: en

Query type:
NfCorpusQuery: (namedtuple)
  1. query_id: str
  2. title: str
  3. all: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, all>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev queries
[query_id]    [title]    [all]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('title'))

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.dev.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.dev')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
15K qrels
Query relevance judgment type:
TrecQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int
  4. iteration: str

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.3.2K22.0%
2A direct link from the query to the document the cited sources section of a page.11K74.5%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev qrels --format tsv
[query_id]    [doc_id]    [relevance]    [iteration]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics('title'),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.dev.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 325
  },
  "qrels": {
    "count": 14589,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 521,
          "2": 10864,
          "1": 3204
        }
      }
    }
  }
}

"nfcorpus/dev/nontopic"

Official dev set, filtered to exclude queries from topic pages.

queries
144 queries

Language: en

Query type:
GenericQuery: (namedtuple)
  1. query_id: str
  2. text: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev/nontopic")
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev/nontopic queries
[query_id]    [text]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev/nontopic')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics())

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.dev.nontopic.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev/nontopic")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev/nontopic docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev/nontopic')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.dev.nontopic')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
4.4K qrels
Query relevance judgment type:
GenericQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.699 16.1%
2A direct link from the query to the document the cited sources section of a page.3.1K72.0%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev/nontopic")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev/nontopic qrels --format tsv
[query_id]    [doc_id]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev/nontopic')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics(),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.dev.nontopic.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 144
  },
  "qrels": {
    "count": 4353,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 521,
          "2": 3133,
          "1": 699
        }
      }
    }
  }
}

"nfcorpus/dev/video"

Official dev set, filtered to only include queries from video pages.

queries
102 queries

Language: en

Query type:
NfCorpusVideoQuery: (namedtuple)
  1. query_id: str
  2. title: str
  3. desc: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev/video")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, desc>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev/video queries
[query_id]    [title]    [desc]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev/video')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('title'))

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.dev.video.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev/video")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev/video docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev/video')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.dev.video')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
3.1K qrels
Query relevance judgment type:
GenericQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.678 22.1%
2A direct link from the query to the document the cited sources section of a page.2.0K64.5%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/dev/video")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/dev/video qrels --format tsv
[query_id]    [doc_id]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/dev/video')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics('title'),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.dev.video.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 102
  },
  "qrels": {
    "count": 3068,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 411,
          "2": 1979,
          "1": 678
        }
      }
    }
  }
}

"nfcorpus/test"

Official test set. Queries include both title and combinted "all" text field (titles, descriptions, topics, transcripts and comments)

queries
325 queries

Language: en

Query type:
NfCorpusQuery: (namedtuple)
  1. query_id: str
  2. title: str
  3. all: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, all>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test queries
[query_id]    [title]    [all]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('title'))

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.test.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.test')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
16K qrels
Query relevance judgment type:
TrecQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int
  4. iteration: str

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.3.5K22.0%
2A direct link from the query to the document the cited sources section of a page.12K74.3%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test qrels --format tsv
[query_id]    [doc_id]    [relevance]    [iteration]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics('title'),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.test.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 325
  },
  "qrels": {
    "count": 15820,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 576,
          "1": 3486,
          "2": 11758
        }
      }
    }
  }
}

"nfcorpus/test/nontopic"

Official test set, filtered to exclude queries from topic pages.

queries
144 queries

Language: en

Query type:
GenericQuery: (namedtuple)
  1. query_id: str
  2. text: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test/nontopic")
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test/nontopic queries
[query_id]    [text]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test/nontopic')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics())

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.test.nontopic.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test/nontopic")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test/nontopic docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test/nontopic')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.test.nontopic')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
4.5K qrels
Query relevance judgment type:
GenericQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.676 14.9%
2A direct link from the query to the document the cited sources section of a page.3.3K72.4%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test/nontopic")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test/nontopic qrels --format tsv
[query_id]    [doc_id]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test/nontopic')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics(),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.test.nontopic.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 144
  },
  "qrels": {
    "count": 4540,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 576,
          "1": 676,
          "2": 3288
        }
      }
    }
  }
}

"nfcorpus/test/video"

Official test set, filtered to only include queries from video pages.

queries
102 queries

Language: en

Query type:
NfCorpusVideoQuery: (namedtuple)
  1. query_id: str
  2. title: str
  3. desc: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test/video")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, desc>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test/video queries
[query_id]    [title]    [desc]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test/video')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('title'))

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.test.video.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test/video")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test/video docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test/video')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.test.video')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
3.1K qrels
Query relevance judgment type:
GenericQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.610 19.6%
2A direct link from the query to the document the cited sources section of a page.2.0K65.4%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/test/video")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/test/video qrels --format tsv
[query_id]    [doc_id]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/test/video')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics('title'),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.test.video.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 102
  },
  "qrels": {
    "count": 3108,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 464,
          "2": 2034,
          "1": 610
        }
      }
    }
  }
}

"nfcorpus/train"

Official train set. Queries include both title and combinted "all" text field (titles, descriptions, topics, transcripts and comments)

queries
2.6K queries

Language: en

Query type:
NfCorpusQuery: (namedtuple)
  1. query_id: str
  2. title: str
  3. all: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, all>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train queries
[query_id]    [title]    [all]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('title'))

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.train.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.train')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
139K qrels
Query relevance judgment type:
TrecQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int
  4. iteration: str

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.29K20.6%
2A direct link from the query to the document the cited sources section of a page.106K76.3%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train qrels --format tsv
[query_id]    [doc_id]    [relevance]    [iteration]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics('title'),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.train.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 2594
  },
  "qrels": {
    "count": 139350,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 4279,
          "2": 106296,
          "1": 28775
        }
      }
    }
  }
}

"nfcorpus/train/nontopic"

Official train set, filtered to exclude queries from topic pages.

queries
1.1K queries

Language: en

Query type:
GenericQuery: (namedtuple)
  1. query_id: str
  2. text: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train/nontopic")
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train/nontopic queries
[query_id]    [text]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train/nontopic')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics())

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.train.nontopic.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train/nontopic")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train/nontopic docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train/nontopic')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.train.nontopic')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
37K qrels
Query relevance judgment type:
GenericQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.6.7K18.0%
2A direct link from the query to the document the cited sources section of a page.26K70.6%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train/nontopic")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train/nontopic qrels --format tsv
[query_id]    [doc_id]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train/nontopic')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics(),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.train.nontopic.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 1141
  },
  "qrels": {
    "count": 37383,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 4279,
          "2": 26384,
          "1": 6720
        }
      }
    }
  }
}

"nfcorpus/train/video"

Official train set, filtered to only include queries from video pages.

queries
812 queries

Language: en

Query type:
NfCorpusVideoQuery: (namedtuple)
  1. query_id: str
  2. title: str
  3. desc: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train/video")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, desc>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train/video queries
[query_id]    [title]    [desc]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train/video')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('title'))

You can find more details about PyTerrier retrieval here.

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.nfcorpus.train.video.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
5.4K docs

Inherits docs from nfcorpus

Language: en

Document type:
NfCorpusDoc: (namedtuple)
  1. doc_id: str
  2. url: str
  3. title: str
  4. abstract: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train/video")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, title, abstract>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train/video docs
[doc_id]    [url]    [title]    [abstract]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train/video')
# Index nfcorpus
indexer = pt.IterDictIndexer('./indices/nfcorpus')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'abstract'])

You can find more details about PyTerrier indexing here.

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.nfcorpus.train.video')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
27K qrels
Query relevance judgment type:
GenericQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int

Relevance levels

Rel.DefinitionCount%
0Marginally relevant, based on topic containment.0 0.0%
1A link exists from the query to another query that directly links to the document.6.3K22.8%
2A direct link from the query to the document the cited sources section of a page.18K64.3%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("nfcorpus/train/video")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance>

You can find more details about the Python API here.

CLI 
ir_datasets export nfcorpus/train/video qrels --format tsv
[query_id]    [doc_id]    [relevance]
...

You can find more details about the CLI here.

PyTerrier 
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:nfcorpus/train/video')
index_ref = pt.IndexRef.of('./indices/nfcorpus') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
    [pipeline],
    dataset.get_topics('title'),
    dataset.get_qrels(),
    [MAP, nDCG@20]
)

You can find more details about PyTerrier experiments here.

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.nfcorpus.train.video.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Boteva2016Nfcorpus}

Bibtex:

@inproceedings{Boteva2016Nfcorpus, title="A Full-Text Learning to Rank Dataset for Medical Information Retrieval", author = "Vera Boteva and Demian Gholipour and Artem Sokolov and Stefan Riezler", booktitle = "Proceedings of the European Conference on Information Retrieval ({ECIR})", location = "Padova, Italy", publisher = "Springer", year = 2016 }
Metadata 
{
  "docs": {
    "count": 5371,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": "MED-"
      }
    }
  },
  "queries": {
    "count": 812
  },
  "qrels": {
    "count": 27465,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "3": 3536,
          "2": 17669,
          "1": 6260
        }
      }
    }
  }
}