Medline - ir_datasets

`"medline"`

Medical articles from Medline. This collection was used by TREC Genomics 2004-05 (2004 version of dataset) and by TREC Precision Medicine 2017-18 (2017 version).

`"medline/2004"`

3M Medline articles including titles and abstracts, used for the TREC 2004-05 Genomics track.

Documents: Biomedical article titles and abstracts
Information about document collection

docs Metadata

3.7M docs

Language: en

Document type:

MedlineDoc: (namedtuple)

doc_id: str
title: str
abstract: str

Examples:

Python API CLI PyTerrier XPM-IR

import ir_datasets
dataset = ir_datasets.load("medline/2004")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, title, abstract>

You can find more details about the Python API here.

ir_datasets export medline/2004 docs



[doc_id]    [title]    [abstract]
...

You can find more details about the CLI here.

{
  "docs": {
    "count": 3672808,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  }
}

`"medline/2004/trec-genomics-2004"`

The TREC Genomics Track 2004 benchmark. Contains 50 queries with article-level relevance judgments.

Documents: Biomedical article titles and abstracts
Queries: Natural language questions
Qrels: deep, graded
Shared task data site
Shared task paper

queries docs qrels Citation Metadata

50 queries

Language: en

Query type:

TrecGenomicsQuery: (namedtuple)

query_id: str
title: str
need: str
context: str

Examples:

Python API CLI PyTerrier XPM-IR

import ir_datasets
dataset = ir_datasets.load("medline/2004/trec-genomics-2004")
for query in dataset.queries_iter():
    query # namedtuple<query_id, title, need, context>

You can find more details about the Python API here.

ir_datasets export medline/2004/trec-genomics-2004 queries



[query_id]    [title]    [need]    [context]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	not relevant	`0`	0.0%
1	possibly relevant	`4.6K`	56.0%
2	definitely relevant	`3.6K`	44.0%

{
  "docs": {
    "count": 3672808,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 50
  },
  "qrels": {
    "count": 8268,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "2": 3639,
          "1": 4629
        }
      }
    }
  }
}

`"medline/2004/trec-genomics-2005"`

The TREC Genomics Track 2005 benchmark. Contains 50 queries with article-level relevance judgments.

Documents: Biomedical article titles and abstracts
Queries: Natural language questions
Qrels: deep, graded
Shared task data site
Shared task paper

queries docs qrels Citation Metadata

50 queries

Language: en

Query type:

GenericQuery: (namedtuple)

query_id: str
text: str

Examples:

Python API CLI PyTerrier XPM-IR

import ir_datasets
dataset = ir_datasets.load("medline/2004/trec-genomics-2005")
for query in dataset.queries_iter():
    query # namedtuple<query_id, text>

You can find more details about the Python API here.

ir_datasets export medline/2004/trec-genomics-2005 queries



[query_id]    [text]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	not relevant	`35K`	88.5%
1	possibly relevant	`2.1K`	5.2%
2	definitely relevant	`2.5K`	6.3%

{
  "docs": {
    "count": 3672808,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 50
  },
  "qrels": {
    "count": 39958,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "0": 35374,
          "2": 2525,
          "1": 2059
        }
      }
    }
  }
}

`"medline/2017"`

26M Medline and AACR/ASCO Proceedings articles including titles and abstracts. This collection is used for the TREC 2017-18 TREC Precision Medicine track.

Documents: Biomedical article titles and abstracts
Information about document collection

docs Metadata

27M docs

Language: en

Document type:

MedlineDoc: (namedtuple)

doc_id: str
title: str
abstract: str

Examples:

Python API CLI PyTerrier XPM-IR

import ir_datasets
dataset = ir_datasets.load("medline/2017")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, title, abstract>

You can find more details about the Python API here.

ir_datasets export medline/2017 docs



[doc_id]    [title]    [abstract]
...

You can find more details about the CLI here.

{
  "docs": {
    "count": 26740025,
    "fields": {
      "doc_id": {
        "max_len": 15,
        "common_prefix": ""
      }
    }
  }
}

`"medline/2017/trec-pm-2017"`

The TREC Precision Medicine (PM) Track 2017 benchmark. Contains 30 queries containing disease, gene, and target demographic information.

Documents: Biomedical article titles and abstracts
Queries: Specific to TREC PM information need
Qrels: deep, graded
Shared task data site
Shared task paper

queries docs qrels Citation Metadata

30 queries

Language: en

Query type:

TrecPm2017Query: (namedtuple)

query_id: str
disease: str
gene: str
demographic: str
other: str

Examples:

Python API CLI PyTerrier XPM-IR

import ir_datasets
dataset = ir_datasets.load("medline/2017/trec-pm-2017")
for query in dataset.queries_iter():
    query # namedtuple<query_id, disease, gene, demographic, other>

You can find more details about the Python API here.

ir_datasets export medline/2017/trec-pm-2017 queries



[query_id]    [disease]    [gene]    [demographic]    [other]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	not relevant	`19K`	82.9%
1	possibly relevant	`1.9K`	8.2%
2	definitely relevant	`2.0K`	8.9%

{
  "docs": {
    "count": 26740025,
    "fields": {
      "doc_id": {
        "max_len": 15,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 30
  },
  "qrels": {
    "count": 22642,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "0": 18767,
          "1": 1853,
          "2": 2022
        }
      }
    }
  }
}

`"medline/2017/trec-pm-2018"`

The TREC Precision Medicine (PM) Track 2018 benchmark. Contains 50 queries containing disease, gene, and target demographic information.

Documents: Biomedical article titles and abstracts
Queries: Specific to TREC PM information need
Qrels: deep, graded
Shared task data site
Shared task paper

queries docs qrels Citation Metadata

50 queries

Language: en

Query type:

TrecPmQuery: (namedtuple)

query_id: str
disease: str
gene: str
demographic: str

Examples:

Python API CLI PyTerrier XPM-IR

import ir_datasets
dataset = ir_datasets.load("medline/2017/trec-pm-2018")
for query in dataset.queries_iter():
    query # namedtuple<query_id, disease, gene, demographic>

You can find more details about the Python API here.

ir_datasets export medline/2017/trec-pm-2018 queries



[query_id]    [disease]    [gene]    [demographic]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	not relevant	`17K`	75.1%
1	possibly relevant	`2.1K`	9.6%
2	definitely relevant	`3.4K`	15.3%

{
  "docs": {
    "count": 26740025,
    "fields": {
      "doc_id": {
        "max_len": 15,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 50
  },
  "qrels": {
    "count": 22429,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "0": 16841,
          "2": 3442,
          "1": 2146
        }
      }
    }
  }
}

ir_datasets: Medline

"medline"

"medline/2004"

"medline/2004/trec-genomics-2004"

"medline/2004/trec-genomics-2005"

"medline/2017"

"medline/2017/trec-pm-2017"

"medline/2017/trec-pm-2018"

`ir_datasets`: Medline

`"medline"`

`"medline/2004"`

`"medline/2004/trec-genomics-2004"`

`"medline/2004/trec-genomics-2005"`

`"medline/2017"`

`"medline/2017/trec-pm-2017"`

`"medline/2017/trec-pm-2018"`