KILT - ir_datasets

`"kilt"`

KILT is a corpus used for various "knowledge intensive language tasks".

Documents: Wikipedia articles
Repository
Paper
Leaderboard

docs Citation Metadata

5.9M docs

Language: en

Document type:

KiltDoc: (namedtuple)

doc_id: str
title: str
text: str
text_pieces: Tuple[str, ...]
anchors: Tuple[
KiltDocAnchor: (namedtuple)
1. text: str
2. href: str
3. paragraph_id: int
4. start: int
5. end: int
, ...]
categories: Tuple[str, ...]
wikidata_id: str
history_revid: str
history_timestamp: str
history_parentid: str
history_pageid: str
history_url: str

Examples:

Python API CLI PyTerrier

import ir_datasets
dataset = ir_datasets.load("kilt")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, title, text, text_pieces, anchors, categories, wikidata_id, history_revid, history_timestamp, history_parentid, history_pageid, history_url>

You can find more details about the Python API here.

ir_datasets export kilt docs



[doc_id]    [title]    [text]    [text_pieces]    [anchors]    [categories]    [wikidata_id]    [history_revid]    [history_timestamp]    [history_parentid]    [history_pageid]    [history_url]
...

You can find more details about the CLI here.

{
  "docs": {
    "count": 5903530,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  }
}

`"kilt/codec"`

CODEC Entity Ranking sub-task.

Task Repository
See also: codec, the document ranking subtask

queries docs qrels Metadata

36 queries

Language: en

Query type:

CodecQuery: (namedtuple)

query_id: str
query: str
narrative: str

Examples:

Python API CLI PyTerrier

import ir_datasets
dataset = ir_datasets.load("kilt/codec")
for query in dataset.queries_iter():
    query # namedtuple<query_id, query, narrative>

You can find more details about the Python API here.

ir_datasets export kilt/codec queries



[query_id]    [query]    [narrative]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	Not Relevant. This entity is not useful or on topic.	`6.7K`	65.1%
1	Not Valuable. It is useful to understand what this entity is for understanding this topic.	`1.9K`	18.7%
2	Somewhat valuable. It is important to understand what this entity is for understanding this topic.	`1.0K`	10.1%
3	Very Valuable. It is absolutely critical to understand what this entity is for understanding this topic.	`626`	6.1%

{
  "docs": {
    "count": 5903530,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 36
  },
  "qrels": {
    "count": 10239,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "0": 6665,
          "1": 1910,
          "2": 1038,
          "3": 626
        }
      }
    }
  }
}

`"kilt/codec/economics"`

Subset of codec that only contains topics about economics.

queries docs qrels Metadata

12 queries

Language: en

Query type:

CodecQuery: (namedtuple)

query_id: str
query: str
narrative: str

Examples:

Python API CLI PyTerrier

import ir_datasets
dataset = ir_datasets.load("kilt/codec/economics")
for query in dataset.queries_iter():
    query # namedtuple<query_id, query, narrative>

You can find more details about the Python API here.

ir_datasets export kilt/codec/economics queries



[query_id]    [query]    [narrative]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	Not Relevant. Not useful or on topic.	`596`	37.4%
1	Not Valuable. Consists of definitions or background.	`545`	34.2%
2	Somewhat Valuable. Includes valuable topic-specific arguments, evidence, or knowledge.	`330`	20.7%
3	Very Valuable. Includes central topic-specific arguments, evidence, or knowledge. This does not include general definitions or background.	`121`	7.6%

{
  "docs": {
    "count": 5903530,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 12
  },
  "qrels": {
    "count": 1592,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "0": 596,
          "2": 330,
          "1": 545,
          "3": 121
        }
      }
    }
  }
}

`"kilt/codec/history"`

Subset of codec that only contains topics about history.

queries docs qrels Metadata

12 queries

Language: en

Query type:

CodecQuery: (namedtuple)

query_id: str
query: str
narrative: str

Examples:

Python API CLI PyTerrier

import ir_datasets
dataset = ir_datasets.load("kilt/codec/history")
for query in dataset.queries_iter():
    query # namedtuple<query_id, query, narrative>

You can find more details about the Python API here.

ir_datasets export kilt/codec/history queries



[query_id]    [query]    [narrative]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	Not Relevant. Not useful or on topic.	`870`	51.3%
1	Not Valuable. Consists of definitions or background.	`509`	30.0%
2	Somewhat Valuable. Includes valuable topic-specific arguments, evidence, or knowledge.	`235`	13.9%
3	Very Valuable. Includes central topic-specific arguments, evidence, or knowledge. This does not include general definitions or background.	`81`	4.8%

{
  "docs": {
    "count": 5903530,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 12
  },
  "qrels": {
    "count": 1695,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "0": 870,
          "1": 509,
          "2": 235,
          "3": 81
        }
      }
    }
  }
}

`"kilt/codec/politics"`

Subset of codec that only contains topics about politics.

queries docs qrels Metadata

12 queries

Language: en

Query type:

CodecQuery: (namedtuple)

query_id: str
query: str
narrative: str

Examples:

Python API CLI PyTerrier

import ir_datasets
dataset = ir_datasets.load("kilt/codec/politics")
for query in dataset.queries_iter():
    query # namedtuple<query_id, query, narrative>

You can find more details about the Python API here.

ir_datasets export kilt/codec/politics queries



[query_id]    [query]    [narrative]
...

You can find more details about the CLI here.

Rel.	Definition	Count	%
0	Not Relevant. Not useful or on topic.	`609`	33.0%
1	Not Valuable. Consists of definitions or background.	`765`	41.5%
2	Somewhat Valuable. Includes valuable topic-specific arguments, evidence, or knowledge.	`359`	19.5%
3	Very Valuable. Includes central topic-specific arguments, evidence, or knowledge. This does not include general definitions or background.	`110`	6.0%

{
  "docs": {
    "count": 5903530,
    "fields": {
      "doc_id": {
        "max_len": 8,
        "common_prefix": ""
      }
    }
  },
  "queries": {
    "count": 12
  },
  "qrels": {
    "count": 1843,
    "fields": {
      "relevance": {
        "counts_by_value": {
          "1": 765,
          "2": 359,
          "0": 609,
          "3": 110
        }
      }
    }
  }
}

ir_datasets: KILT

"kilt"

"kilt/codec"

"kilt/codec/economics"

"kilt/codec/history"

"kilt/codec/politics"

`ir_datasets`: KILT

`"kilt"`

`"kilt/codec"`

`"kilt/codec/economics"`

`"kilt/codec/history"`

`"kilt/codec/politics"`