ir_datasets: Washington PostTo use this dataset, you need a copy of Washington Post Collection, provided by NIST.
Your organization may already have a copy. If this is the case, you may only need to complete a new "Individual Argeement". Otherwise, your organization will need to file the "Organizational agreement" with NIST. It can take some time to process, but you will end up with a password-protected download link.
The source file required is WashingtonPost.v2.tar.gz.
ir_datasets expects the above file to be copied/linked under ~/.ir_datasets/wapo/WashingtonPost.v2.tar.gz.
The Washington Post collection.
Version 2 of the Washington Post collection, consisting of articles published between 2012-2017.
The collection is obtained from NIST by requesting it from NIST here.
body contains all body text in plain text format, including paragrphs and multi-media captions. body_paras_html contains only source paragraphs and contains HTML markup. body_media contains images, videos, tweets, and galeries, along with a link to the content and a textual caption.
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2")
for doc in dataset.docs_iter():
doc # namedtuple<doc_id, url, title, author, published_date, kicker, body, body_paras_html, body_media>
You can find more details about the Python API here.
ir_datasets export wapo/v2 docs
[doc_id] [url] [title] [author] [published_date] [kicker] [body] [body_paras_html] [body_media]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v2')
# Index wapo/v2
indexer = pt.IterDictIndexer('./indices/wapo_v2', meta={"docno": 36})
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'author', 'kicker', 'body'])
You can find more details about PyTerrier indexing here.
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.wapo.v2')
for doc in dataset.iter_documents():
print(doc) # an AdhocDocumentStore
break
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore
{
"docs": {
"count": 595037,
"fields": {
"doc_id": {
"max_len": 36,
"common_prefix": ""
}
}
}
}
The TREC Common Core 2018 benchmark.
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-core-2018")
for query in dataset.queries_iter():
query # namedtuple<query_id, title, description, narrative>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-core-2018 queries
[query_id] [title] [description] [narrative]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-core-2018')
index_ref = pt.IndexRef.of('./indices/wapo_v2') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('title'))
You can find more details about PyTerrier retrieval here.
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.wapo.v2.trec-core-2018.queries') # AdhocTopics
for topic in topics.iter():
print(topic) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.
Inherits docs from wapo/v2
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-core-2018")
for doc in dataset.docs_iter():
doc # namedtuple<doc_id, url, title, author, published_date, kicker, body, body_paras_html, body_media>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-core-2018 docs
[doc_id] [url] [title] [author] [published_date] [kicker] [body] [body_paras_html] [body_media]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-core-2018')
# Index wapo/v2
indexer = pt.IterDictIndexer('./indices/wapo_v2', meta={"docno": 36})
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'author', 'kicker', 'body'])
You can find more details about PyTerrier indexing here.
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.wapo.v2.trec-core-2018')
for doc in dataset.iter_documents():
print(doc) # an AdhocDocumentStore
break
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore
Relevance levels
| Rel. | Definition | Count | % |
|---|---|---|---|
| 0 | not relevant | 22K | 85.0% |
| 1 | relevant | 2.1K | 7.9% |
| 2 | highly relevant | 1.9K | 7.1% |
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-core-2018")
for qrel in dataset.qrels_iter():
qrel # namedtuple<query_id, doc_id, relevance, iteration>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-core-2018 qrels --format tsv
[query_id] [doc_id] [relevance] [iteration]
...
You can find more details about the CLI here.
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-core-2018')
index_ref = pt.IndexRef.of('./indices/wapo_v2') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
[pipeline],
dataset.get_topics('title'),
dataset.get_qrels(),
[MAP, nDCG@20]
)
You can find more details about PyTerrier experiments here.
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.wapo.v2.trec-core-2018.qrels') # AdhocAssessments
for topic_qrels in qrels.iter():
print(topic_qrels) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.
{
"docs": {
"count": 595037,
"fields": {
"doc_id": {
"max_len": 36,
"common_prefix": ""
}
}
},
"queries": {
"count": 50
},
"qrels": {
"count": 26233,
"fields": {
"relevance": {
"counts_by_value": {
"0": 22285,
"2": 1865,
"1": 2083
}
}
}
}
}
The TREC News 2018 Background Linking task. The task is to find relevant background information for the provided articles.
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-news-2018")
for query in dataset.queries_iter():
query # namedtuple<query_id, doc_id, url>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-news-2018 queries
[query_id] [doc_id] [url]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-news-2018')
index_ref = pt.IndexRef.of('./indices/wapo_v2') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('doc_id'))
You can find more details about PyTerrier retrieval here.
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.wapo.v2.trec-news-2018.queries') # AdhocTopics
for topic in topics.iter():
print(topic) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.
Inherits docs from wapo/v2
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-news-2018")
for doc in dataset.docs_iter():
doc # namedtuple<doc_id, url, title, author, published_date, kicker, body, body_paras_html, body_media>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-news-2018 docs
[doc_id] [url] [title] [author] [published_date] [kicker] [body] [body_paras_html] [body_media]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-news-2018')
# Index wapo/v2
indexer = pt.IterDictIndexer('./indices/wapo_v2', meta={"docno": 36})
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'author', 'kicker', 'body'])
You can find more details about PyTerrier indexing here.
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.wapo.v2.trec-news-2018')
for doc in dataset.iter_documents():
print(doc) # an AdhocDocumentStore
break
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore
Relevance levels
| Rel. | Definition | Count | % |
|---|---|---|---|
| 0 | The document provides little or no useful background information. | 6.5K | 76.0% |
| 2 | The document provides some useful background or contextual information that would help the user understand the broader story context of the target article. | 1.2K | 14.0% |
| 4 | The document provides significantly useful background ... | 584 | 6.9% |
| 8 | The document provides essential useful background ... | 164 | 1.9% |
| 16 | The document _must_ appear in the sidebar otherwise critical context is missing. | 106 | 1.2% |
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-news-2018")
for qrel in dataset.qrels_iter():
qrel # namedtuple<query_id, doc_id, relevance, iteration>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-news-2018 qrels --format tsv
[query_id] [doc_id] [relevance] [iteration]
...
You can find more details about the CLI here.
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-news-2018')
index_ref = pt.IndexRef.of('./indices/wapo_v2') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
[pipeline],
dataset.get_topics('doc_id'),
dataset.get_qrels(),
[MAP, nDCG@20]
)
You can find more details about PyTerrier experiments here.
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.wapo.v2.trec-news-2018.qrels') # AdhocAssessments
for topic_qrels in qrels.iter():
print(topic_qrels) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.
Bibtex:
@inproceedings{Soboroff2018News, title={TREC 2018 News Track Overview}, author={Ian Soboroff and Shudong Huang and Donna Harman}, booktitle={TREC}, year={2018} }{
"docs": {
"count": 595037,
"fields": {
"doc_id": {
"max_len": 36,
"common_prefix": ""
}
}
},
"queries": {
"count": 50
},
"qrels": {
"count": 8508,
"fields": {
"relevance": {
"counts_by_value": {
"16": 106,
"2": 1189,
"0": 6465,
"4": 584,
"8": 164
}
}
}
}
}
The TREC News 2019 Background Linking task. The task is to find relevant background information for the provided articles.
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-news-2019")
for query in dataset.queries_iter():
query # namedtuple<query_id, doc_id, url>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-news-2019 queries
[query_id] [doc_id] [url]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-news-2019')
index_ref = pt.IndexRef.of('./indices/wapo_v2') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pipeline(dataset.get_topics('doc_id'))
You can find more details about PyTerrier retrieval here.
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.wapo.v2.trec-news-2019.queries') # AdhocTopics
for topic in topics.iter():
print(topic) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.
Inherits docs from wapo/v2
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-news-2019")
for doc in dataset.docs_iter():
doc # namedtuple<doc_id, url, title, author, published_date, kicker, body, body_paras_html, body_media>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-news-2019 docs
[doc_id] [url] [title] [author] [published_date] [kicker] [body] [body_paras_html] [body_media]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-news-2019')
# Index wapo/v2
indexer = pt.IterDictIndexer('./indices/wapo_v2', meta={"docno": 36})
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'author', 'kicker', 'body'])
You can find more details about PyTerrier indexing here.
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.wapo.v2.trec-news-2019')
for doc in dataset.iter_documents():
print(doc) # an AdhocDocumentStore
break
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore
Relevance levels
| Rel. | Definition | Count | % |
|---|---|---|---|
| 0 | The document provides little or no useful background information. | 13K | 80.6% |
| 2 | The document provides some useful background or contextual information that would help the user understand the broader story context of the target article. | 1.7K | 10.7% |
| 4 | The document provides significantly useful background ... | 660 | 4.2% |
| 8 | The document provides essential useful background ... | 431 | 2.8% |
| 16 | The document _must_ appear in the sidebar otherwise critical context is missing. | 273 | 1.7% |
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v2/trec-news-2019")
for qrel in dataset.qrels_iter():
qrel # namedtuple<query_id, doc_id, relevance, iteration>
You can find more details about the Python API here.
ir_datasets export wapo/v2/trec-news-2019 qrels --format tsv
[query_id] [doc_id] [relevance] [iteration]
...
You can find more details about the CLI here.
import pyterrier as pt
from pyterrier.measures import *
pt.init()
dataset = pt.get_dataset('irds:wapo/v2/trec-news-2019')
index_ref = pt.IndexRef.of('./indices/wapo_v2') # assumes you have already built an index
pipeline = pt.BatchRetrieve(index_ref, wmodel='BM25')
# (optionally other pipeline components)
pt.Experiment(
[pipeline],
dataset.get_topics('doc_id'),
dataset.get_qrels(),
[MAP, nDCG@20]
)
You can find more details about PyTerrier experiments here.
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.wapo.v2.trec-news-2019.qrels') # AdhocAssessments
for topic_qrels in qrels.iter():
print(topic_qrels) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.
Bibtex:
@inproceedings{Soboroff2019News, title={TREC 2019 News Track Overview}, author={Ian Soboroff and Shudong Huang and Donna Harman}, booktitle={TREC}, year={2019} }{
"docs": {
"count": 595037,
"fields": {
"doc_id": {
"max_len": 36,
"common_prefix": ""
}
}
},
"queries": {
"count": 60
},
"qrels": {
"count": 15655,
"fields": {
"relevance": {
"counts_by_value": {
"2": 1677,
"0": 12614,
"8": 431,
"16": 273,
"4": 660
}
}
}
}
}
The TREC News 2020 Background Linking task. The task is to find relevant background information for the provided articles.
If you have a copy of the v3 dataset, we would appreciate a pull request to add support!
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v3/trec-news-2020")
for query in dataset.queries_iter():
query # namedtuple<query_id, doc_id, url>
You can find more details about the Python API here.
ir_datasets export wapo/v3/trec-news-2020 queries
[query_id] [doc_id] [url]
...
You can find more details about the CLI here.
No example available for PyTerrier
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.wapo.v3.trec-news-2020.queries') # AdhocTopics
for topic in topics.iter():
print(topic) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.
Relevance levels
| Rel. | Definition | Count | % |
|---|---|---|---|
| 0 | The document provides little or no useful background information. | 15K | 86.4% |
| 2 | The document provides some useful background or contextual information that would help the user understand the broader story context of the target article. | 1.6K | 9.0% |
| 4 | The document provides significantly useful background ... | 631 | 3.6% |
| 8 | The document provides essential useful background ... | 132 | 0.7% |
| 16 | The document _must_ appear in the sidebar otherwise critical context is missing. | 50 | 0.3% |
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v3/trec-news-2020")
for qrel in dataset.qrels_iter():
qrel # namedtuple<query_id, doc_id, relevance, iteration>
You can find more details about the Python API here.
ir_datasets export wapo/v3/trec-news-2020 qrels --format tsv
[query_id] [doc_id] [relevance] [iteration]
...
You can find more details about the CLI here.
No example available for PyTerrier
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.wapo.v3.trec-news-2020.qrels') # AdhocAssessments
for topic_qrels in qrels.iter():
print(topic_qrels) # An AdhocTopic
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.
{
"queries": {
"count": 50
},
"qrels": {
"count": 17764,
"fields": {
"relevance": {
"counts_by_value": {
"0": 15348,
"2": 1603,
"4": 631,
"8": 132,
"16": 50
}
}
}
}
}
(no description provided)
Language: en
Examples:
import ir_datasets
dataset = ir_datasets.load("wapo/v4")
for doc in dataset.docs_iter():
doc # namedtuple<doc_id, url, title, author, published_date, kicker, body, body_paras_html, body_media>
You can find more details about the Python API here.
ir_datasets export wapo/v4 docs
[doc_id] [url] [title] [author] [published_date] [kicker] [body] [body_paras_html] [body_media]
...
You can find more details about the CLI here.
import pyterrier as pt
pt.init()
dataset = pt.get_dataset('irds:wapo/v4')
# Index wapo/v4
indexer = pt.IterDictIndexer('./indices/wapo_v4')
index_ref = indexer.index(dataset.get_corpus_iter(), fields=['url', 'title', 'author', 'kicker', 'body'])
You can find more details about PyTerrier indexing here.
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.wapo.v4')
for doc in dataset.iter_documents():
print(doc) # an AdhocDocumentStore
break
This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore
{
"docs": {}
}