← home
Github: datasets/trec_fair.py

ir_datasets: TREC Fair Ranking

Index
  1. trec-fair
  2. trec-fair/2021
  3. trec-fair/2021/eval
  4. trec-fair/2021/train
  5. trec-fair/2022
  6. trec-fair/2022/train

"trec-fair"

The TREC Fair Ranking track evaluates systems according to how well they fairly rank documents.


"trec-fair/2021"

The TREC Fair Ranking track evaluates systems according to how well they fairly rank documents.

docsMetadata
6.3M docs

Language: en

Document type:
FairTrecDoc: (namedtuple)
  1. doc_id: str
  2. title: str
  3. text: str
  4. marked_up_text: str
  5. url: str
  6. quality_score: Optional[float]
  7. geographic_locations: Optional[List[str]]
  8. quality_score_disk: Optional[str]

Examples:

Python APICLIPyTerrierXPM-IR
import ir_datasets
dataset = ir_datasets.load("trec-fair/2021")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, title, text, marked_up_text, url, quality_score, geographic_locations, quality_score_disk>

You can find more details about the Python API here.


"trec-fair/2021/eval"

Official TREC Fair Ranking 2021 evaluation set.

queriesdocsqrelsMetadata
49 queries

Language: en

Query type:
FairTrecEvalQuery: (namedtuple)
  1. query_id: str
  2. text: str
  3. keywords: List[str]
  4. scope: str

Examples:

Python APICLIPyTerrierXPM-IR
import ir_datasets
dataset = ir_datasets.load("trec-fair/2021/eval")
for query in dataset.queries_iter():
    query # namedtuple<query_id, text, keywords, scope>

You can find more details about the Python API here.


"trec-fair/2021/train"

Official TREC Fair Ranking 2021 train set.

queriesdocsqrelsMetadata
57 queries

Language: en

Query type:
FairTrecQuery: (namedtuple)
  1. query_id: str
  2. text: str
  3. keywords: List[str]
  4. scope: str
  5. homepage: str

Examples:

Python APICLIPyTerrierXPM-IR
import ir_datasets
dataset = ir_datasets.load("trec-fair/2021/train")
for query in dataset.queries_iter():
    query # namedtuple<query_id, text, keywords, scope, homepage>

You can find more details about the Python API here.


"trec-fair/2022"

The TREC Fair Ranking 2022 track focuses on fairly prioritising Wikimedia articles for editing to provide a fair exposure to articles from different groups.

docsMetadata
6.5M docs

Language: en

Document type:
FairTrec2022Doc: (namedtuple)
  1. doc_id: str
  2. title: str
  3. text: str
  4. url: str
  5. pred_qual: Optional[float]
  6. qual_cat: Optional[str]
  7. page_countries: Optional[List[str]]
  8. page_subcont_regions: Optional[List[str]]
  9. source_countries: Optional[Dict[str,int]]
  10. source_subcont_regions: Optional[Dict[str,int]]
  11. gender: Optional[List[str]]
  12. occupations: Optional[List[str]]
  13. years: Optional[List[int]]
  14. num_sitelinks: Optional[int]
  15. relative_pageviews: Optional[float]
  16. first_letter: Optional[str]
  17. creation_date: Optional[str]
  18. first_letter_category: Optional[str]
  19. gender_category: Optional[str]
  20. creation_date_category: Optional[str]
  21. years_category: Optional[str]
  22. relative_pageviews_category: Optional[str]
  23. num_sitelinks_category: Optional[str]

Examples:

Python APICLIPyTerrierXPM-IR
import ir_datasets
dataset = ir_datasets.load("trec-fair/2022")
for doc in dataset.docs_iter():
    doc # namedtuple<doc_id, title, text, url, pred_qual, qual_cat, page_countries, page_subcont_regions, source_countries, source_subcont_regions, gender, occupations, years, num_sitelinks, relative_pageviews, first_letter, creation_date, first_letter_category, gender_category, creation_date_category, years_category, relative_pageviews_category, num_sitelinks_category>

You can find more details about the Python API here.


"trec-fair/2022/train"

Official TREC Fair Ranking 2022 train set.

queriesdocsqrelsMetadata
50 queries

Language: en

Query type:
FairTrec2022TrainQuery: (namedtuple)
  1. query_id: str
  2. text: str
  3. url: str

Examples:

Python APICLIPyTerrierXPM-IR
import ir_datasets
dataset = ir_datasets.load("trec-fair/2022/train")
for query in dataset.queries_iter():
    query # namedtuple<query_id, text, url>

You can find more details about the Python API here.