GOV2 - ir_datasets

`"gov2"`

GOV2 web document collection. Used for the TREC Terabyte Track.

The dataset is obtained for a fee from UoG, and is shipped as a hard drive. More information is provided here.

Document collection site

docs

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

`"gov2/trec-mq-2007"`

TREC 2007 Million Query track.

queries docs qrels Citation

Language: multiple/other/unknown

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-mq-2007')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-mq-2007')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecPrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
method: int
iprob: float

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-mq-2007')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, method, iprob>

`"gov2/trec-mq-2008"`

TREC 2008 Million Query track.

queries docs qrels Citation

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-mq-2008')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-mq-2008')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecPrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
method: int
iprob: float

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-mq-2008')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, method, iprob>

`"gov2/trec-tb-2004"`

The TREC Terabyte Track 2004 ad-hoc ranking benchmark. Contains 50 queries with deep relevance judgments.

queries docs qrels Citation

Language: en

Query type:

TrecQuery: (namedtuple)

query_id: str
title: str
description: str
narrative: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2004')
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, description, narrative>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2004')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2004')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

`"gov2/trec-tb-2005"`

The TREC Terabyte Track 2005 ad-hoc ranking benchmark. Contains 50 queries with deep relevance judgments.

queries docs qrels Citation

Language: en

Query type:

TrecQuery: (namedtuple)

query_id: str
title: str
description: str
narrative: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005')
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, description, narrative>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

`"gov2/trec-tb-2005/efficiency"`

The TREC Terabyte Track 2005 efficiency ranking benchmark. Contains 50,000 queries from a search engine, including the 50 topics from gov2/trec-tb-2005. Only the 50 topics have judgments.

queries docs qrels Citation

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005/efficiency')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005/efficiency')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005/efficiency')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

`"gov2/trec-tb-2005/named-page"`

The TREC Terabyte Track 2005 named page ranking benchmark. Contains 252 queries with titles that resemble bookmark labels. Relevance judgments include near-duplicate pages and other pages that may satisfy the bookmark label.

queries docs qrels Citation

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005/named-page')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005/named-page')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2005/named-page')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

`"gov2/trec-tb-2006"`

The TREC Terabyte Track 2006 ad-hoc ranking benchmark. Contains 50 queries with deep relevance judgments.

queries docs qrels Citation

Language: en

Query type:

TrecQuery: (namedtuple)

query_id: str
title: str
description: str
narrative: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006')
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, description, narrative>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

`"gov2/trec-tb-2006/efficiency"`

The TREC Terabyte Track 2006 efficiency ranking benchmark. Contains 100,000 queries from a search engine, including the 50 topics from gov2/trec-tb-2006. Only the 50 topics have judgments.

queries docs qrels Citation

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

`"gov2/trec-tb-2006/efficiency/10k"`

Small stream from gov2/trec-tb-2006/efficiency, with 10,000 queries.

queries docs

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/10k')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/10k')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

`"gov2/trec-tb-2006/efficiency/stream1"`

Stream 1 of gov2/trec-tb-2006/efficiency (25,000 queries).

queries docs

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream1')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream1')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

`"gov2/trec-tb-2006/efficiency/stream2"`

Stream 2 of gov2/trec-tb-2006/efficiency (25,000 queries).

queries docs

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream2')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream2')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

`"gov2/trec-tb-2006/efficiency/stream3"`

Stream 3 of gov2/trec-tb-2006/efficiency (25,000 queries).

queries docs qrels

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream3')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream3')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant
2	Highly Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream3')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

`"gov2/trec-tb-2006/efficiency/stream4"`

Stream 4 of gov2/trec-tb-2006/efficiency (25,000 queries).

queries docs

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream4')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/efficiency/stream4')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

`"gov2/trec-tb-2006/named-page"`

The TREC Terabyte Track 2006 named page ranking benchmark. Contains 181 queries with titles that resemble bookmark labels. Relevance judgments include near-duplicate pages and other pages that may satisfy the bookmark label.

queries docs qrels Citation

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/named-page')
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

Language: en

Document type:

Gov2Doc: (namedtuple)

doc_id: str
url: str
http_headers: str
body: bytes
body_content_type: str

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/named-page')
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, url, http_headers, body, body_content_type>

Query relevance judgment type:

TrecQrel: (namedtuple)

query_id: str
doc_id: str
relevance: int
iteration: str

Relevance levels

Rel.	Definition
0	Not Relevant
1	Relevant

Example


import ir_datasets
dataset = ir_datasets.load('gov2/trec-tb-2006/named-page')
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

ir_datasets: GOV2

"gov2"

"gov2/trec-mq-2007"

"gov2/trec-mq-2008"

"gov2/trec-tb-2004"

"gov2/trec-tb-2005"

"gov2/trec-tb-2005/efficiency"

"gov2/trec-tb-2005/named-page"

"gov2/trec-tb-2006"

"gov2/trec-tb-2006/efficiency"

"gov2/trec-tb-2006/efficiency/10k"

"gov2/trec-tb-2006/efficiency/stream1"

"gov2/trec-tb-2006/efficiency/stream2"

"gov2/trec-tb-2006/efficiency/stream3"

"gov2/trec-tb-2006/efficiency/stream4"

"gov2/trec-tb-2006/named-page"

`ir_datasets`: GOV2

`"gov2"`

`"gov2/trec-mq-2007"`

`"gov2/trec-mq-2008"`

`"gov2/trec-tb-2004"`

`"gov2/trec-tb-2005"`

`"gov2/trec-tb-2005/efficiency"`

`"gov2/trec-tb-2005/named-page"`

`"gov2/trec-tb-2006"`

`"gov2/trec-tb-2006/efficiency"`

`"gov2/trec-tb-2006/efficiency/10k"`

`"gov2/trec-tb-2006/efficiency/stream1"`

`"gov2/trec-tb-2006/efficiency/stream2"`

`"gov2/trec-tb-2006/efficiency/stream3"`

`"gov2/trec-tb-2006/efficiency/stream4"`

`"gov2/trec-tb-2006/named-page"`