← home
Github: datasets/trec_mandarin.py

ir_datasets: TREC Mandarin

Index
  1. trec-mandarin
  2. trec-mandarin/trec5
  3. trec-mandarin/trec6

Data Access Information

To use this dataset, you need a copy of the source corpus, provided by the the Linguistic Data Consortium. The specific resource needed is LDC2000T52.

Many organizations already have a subscription to the LDC, so access to the collection can be as easy as confirming the data usage agreement and downloading the corpus. Check with your library for access details.

The source file is: LDC2000T52.tgz.

ir_datasets expects this file to be copied/linked as ~/.ir_datasets/trec-mandarin/corpus.tgz.

"trec-mandarin"

A collection of news articles in Mandarin in Simplified Chinese, used for multi-lingual evaluation in TREC 5 and TREC 6.

Document collection from LDC2000T52.

docs
165K docs

Language: zh

Document type:
TrecDoc: (namedtuple)
  1. doc_id: str
  2. text: str
  3. marked_up_doc: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("trec-mandarin")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, text, marked_up_doc>

You can find more details about the Python API here.

CLI 
ir_datasets export trec-mandarin docs
[doc_id]    [text]    [marked_up_doc]
...

You can find more details about the CLI here.

PyTerrier

No example available for PyTerrier

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.trec-mandarin')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

Citation

ir_datasets.bib:

\cite{Rogers2000Mandarin}

Bibtex:

@misc{Rogers2000Mandarin, title={TREC Mandarin LDC2000T52}, author={Rogers, Willie}, year={2000}, url={https://catalog.ldc.upenn.edu/LDC2000T52}, publisher={Linguistic Data Consortium} }
Metadata 
{
  "docs": {
    "count": 164789,
    "fields": {
      "doc_id": {
        "max_len": 22,
        "common_prefix": ""
      }
    }
  }
}

"trec-mandarin/trec5"

Mandarin Chinese benchmark from TREC 5.

queries
28 queries

Language: multiple/other/unknown

Query type:
TrecMandarinQuery: (namedtuple)
  1. query_id: str
  2. title_en: str
  3. title_zh: str
  4. description_en: str
  5. description_zh: str
  6. narrative_en: str
  7. narrative_zh: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("trec-mandarin/trec5")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title_en, title_zh, description_en, description_zh, narrative_en, narrative_zh>

You can find more details about the Python API here.

CLI 
ir_datasets export trec-mandarin/trec5 queries
[query_id]    [title_en]    [title_zh]    [description_en]    [description_zh]    [narrative_en]    [narrative_zh]
...

You can find more details about the CLI here.

PyTerrier

No example available for PyTerrier

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.trec-mandarin.trec5.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
165K docs

Inherits docs from trec-mandarin

Language: zh

Document type:
TrecDoc: (namedtuple)
  1. doc_id: str
  2. text: str
  3. marked_up_doc: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("trec-mandarin/trec5")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, text, marked_up_doc>

You can find more details about the Python API here.

CLI 
ir_datasets export trec-mandarin/trec5 docs
[doc_id]    [text]    [marked_up_doc]
...

You can find more details about the CLI here.

PyTerrier

No example available for PyTerrier

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.trec-mandarin.trec5')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
16K qrels
Query relevance judgment type:
TrecQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int
  4. iteration: str

Relevance levels

Rel.DefinitionCount%
0not relevant13K86.0%
1relevant2.2K14.0%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("trec-mandarin/trec5")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

You can find more details about the Python API here.

CLI 
ir_datasets export trec-mandarin/trec5 qrels --format tsv
[query_id]    [doc_id]    [relevance]    [iteration]
...

You can find more details about the CLI here.

PyTerrier

No example available for PyTerrier

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.trec-mandarin.trec5.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Harman1997Chinese,Rogers2000Mandarin}

Bibtex:

@inproceedings{Harman1997Chinese, title={Spanish and Chinese Document Retrieval in TREC-5}, author={Alan Smeaton and Ross Wilkinson}, booktitle={TREC}, year={1996} } @misc{Rogers2000Mandarin, title={TREC Mandarin LDC2000T52}, author={Rogers, Willie}, year={2000}, url={https://catalog.ldc.upenn.edu/LDC2000T52}, publisher={Linguistic Data Consortium} }
Metadata 
{
  "docs": {
    "count": 164789,
    "fields": {
      "doc_id": {
        "max_len": 22,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 28
  },
  "qrels": {
    "count": 15588,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "0": 13406,
          "1": 2182
        }
      }
    }
  }
}

"trec-mandarin/trec6"

Mandarin Chinese benchmark from TREC 6.

queries
26 queries

Language: multiple/other/unknown

Query type:
TrecMandarinQuery: (namedtuple)
  1. query_id: str
  2. title_en: str
  3. title_zh: str
  4. description_en: str
  5. description_zh: str
  6. narrative_en: str
  7. narrative_zh: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("trec-mandarin/trec6")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title_en, title_zh, description_en, description_zh, narrative_en, narrative_zh>

You can find more details about the Python API here.

CLI 
ir_datasets export trec-mandarin/trec6 queries
[query_id]    [title_en]    [title_zh]    [description_en]    [description_zh]    [narrative_en]    [narrative_zh]
...

You can find more details about the CLI here.

PyTerrier

No example available for PyTerrier

XPM-IR 
from datamaestro import prepare_dataset
topics = prepare_dataset('irds.trec-mandarin.trec6.queries')  # AdhocTopics
for topic in topics.iter():
    print(topic)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocTopics.

docs
165K docs

Inherits docs from trec-mandarin

Language: zh

Document type:
TrecDoc: (namedtuple)
  1. doc_id: str
  2. text: str
  3. marked_up_doc: str

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("trec-mandarin/trec6")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, text, marked_up_doc>

You can find more details about the Python API here.

CLI 
ir_datasets export trec-mandarin/trec6 docs
[doc_id]    [text]    [marked_up_doc]
...

You can find more details about the CLI here.

PyTerrier

No example available for PyTerrier

XPM-IR 
from datamaestro import prepare_dataset
dataset = prepare_dataset('irds.trec-mandarin.trec6')
for doc in dataset.iter_documents():
    print(doc)  # an AdhocDocumentStore
    break

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocDocumentStore

qrels
9.2K qrels
Query relevance judgment type:
TrecQrel: (namedtuple)
  1. query_id: str
  2. doc_id: str
  3. relevance: int
  4. iteration: str

Relevance levels

Rel.DefinitionCount%
0not relevant6.3K68.0%
1relevant3.0K32.0%

Examples:

Python API 
import ir_datasets
dataset = ir_datasets.load("trec-mandarin/trec6")
for qrel in dataset.qrels_iter():
    qrel # namedtuple<query_id, doc_id, relevance, iteration>

You can find more details about the Python API here.

CLI 
ir_datasets export trec-mandarin/trec6 qrels --format tsv
[query_id]    [doc_id]    [relevance]    [iteration]
...

You can find more details about the CLI here.

PyTerrier

No example available for PyTerrier

XPM-IR 
from datamaestro import prepare_dataset
qrels = prepare_dataset('irds.trec-mandarin.trec6.qrels')  # AdhocAssessments
for topic_qrels in qrels.iter():
    print(topic_qrels)  # An AdhocTopic

This examples requires that experimaestro-ir be installed. For more information about the returned object, see the documentation about AdhocAssessments.

Citation

ir_datasets.bib:

\cite{Wilkinson1998Chinese,Rogers2000Mandarin}

Bibtex:

@inproceedings{Wilkinson1998Chinese, title={Chinese Document Retrieval at TREC-6}, author={Ross Wilkinson}, booktitle={TREC}, year={1997} } @misc{Rogers2000Mandarin, title={TREC Mandarin LDC2000T52}, author={Rogers, Willie}, year={2000}, url={https://catalog.ldc.upenn.edu/LDC2000T52}, publisher={Linguistic Data Consortium} }
Metadata 
{
  "docs": {
    "count": 164789,
    "fields": {
      "doc_id": {
        "max_len": 22,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 26
  },
  "qrels": {
    "count": 9236,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "1": 2958,
          "0": 6278
        }
      }
    }
  }
}