Python CSV KeyError: 'id' — Diagnose and Fix a UTF-8 BOM Header

If Python raises KeyError: 'id' even though the CSV appears to start with id, inspect the header before changing your data. A UTF-8 byte order mark (BOM) can turn that first key into '\ufeffid'. For that specific case, read the file with encoding="utf-8-sig".

The same bytes produce different first-header keys. Diagram based on the verified Python example.

This walkthrough reproduces the failure and the fix using Python 3.12.13 on macOS, tested on September 6, 2026. The example uses synthetic data and only the standard library.

Quick diagnosis

Check the first header with repr(). If it is \ufeffid, use utf-8-sig. If it is id;name or id, follow the delimiter or whitespace checks below.

1. See the actual key, including invisible characters

Run this against your CSV. The repr() output makes the invisible prefix visible. Reading the first three bytes is a separate check: ef bb bf identifies the UTF-8 signature at the start of this file.

from pathlib import Path
import csv

path = Path("input.csv")
with path.open("rb") as f:
    print("First bytes:", f.read(3).hex(" "))
with path.open(encoding="utf-8", newline="") as f:
    reader = csv.DictReader(f)
    print("Headers:", [repr(x) for x in (reader.fieldnames or [])])

2. Reproduce the error without downloading a file

import csv
import io

raw = "id,name\r\n00123,서울\r\n".encode("utf-8-sig")
with io.TextIOWrapper(io.BytesIO(raw), encoding="utf-8", newline="") as f:
    row = next(csv.DictReader(f))
    print(repr(next(iter(row))))
    try:
        print(row["id"])
    except KeyError as error:
        print(type(error).__name__, str(error))

Observed output:

'\ufeffid'
KeyError 'id'

The row exists. The lookup fails because id and \ufeffid are different strings. In this reproduction, neither a missing row nor damaged Korean text causes the error.

3. Read the BOM-aware way and validate the header

import csv

with open("input.csv", encoding="utf-8-sig", newline="") as f:
    reader = csv.DictReader(f)
    headers = reader.fieldnames or []
    if "id" not in headers:
        raise ValueError(f"Expected an id column; got {headers!r}")
    for row in reader:
        print(row["id"])

For the sample above, the value is 00123. Our test also confirmed that the same reader handles the UTF-8 sample without a BOM. It does not turn the identifier into a number.

Python documents how utf-8-sig skips an initial UTF-8 signature. Its CSV documentation explains header-based dictionaries and the recommended newline="" file setting.

4. If it still fails, follow the header evidence

• ['id;name']: your comma reader sees one column. If the file uses semicolons, pass delimiter=";".
• [' id', 'name']: the first header has a space. Confirm the intended schema before trimming or renaming columns.
• ['ID', 'name']: the spelling differs by case. Use the actual key or an explicit, checked mapping.
• An empty list: there is no header to map. Check whether the file is empty or whether the export failed.

The wrong-delimiter and leading-space cases were included in our tests: switching to utf-8-sig did not fix either one. Avoid silently falling back to row.get("id", "") before diagnosing the schema; that would hide the failed lookup.

What this fix does not do

It does not detect every encoding, repair invalid UTF-8 bytes, or remove a U+FEFF character embedded in a cell. We tested invalid UTF-8 and an interior U+FEFF separately: decoding failed for the former and preserved the latter. Keep the original export while diagnosing the problem.

For choosing the encoding of a new export, see UTF-8 vs UTF-8 BOM for CSV: a tested Python read/write matrix.