Splitting a CSV row with a plain string split on commas can produce more fields than the row actually contains, because commas inside quoted values are part of the data rather than delimiters between fields. The csv module in Python's standard library provides csv.reader, a callable that returns a reader object; iterating over that object parses rows while tracking whether it is inside or outside a quoted field. Testing a row containing a quoted name with an internal comma, alongside a phrase with escaped quotation marks, showed the two approaches producing different field counts from the same input.
The naive split problem
A simple way to parse CSV-like text is to call split on the comma character. This works when every field is free of commas, but breaks as soon as a field legitimately contains one. In the tested example, a row held three logical fields: a numeric identifier, a name that itself contained a comma, and a phrase containing escaped quotation marks. Splitting that row on commas produced four fields instead of three, because the comma inside the name was treated as if it separated two fields.
The underlying issue is that a plain split has no awareness of quoting conventions. It treats every comma as a delimiter regardless of whether that comma sits inside quotation marks meant to protect a value. Nothing in the split operation itself accounts for quoted regions of text.
How csv.reader parses differently
csv.reader, a callable provided by the csv module, returns a reader object; iterating over that object tracks state as it reads a row, distinguishing between characters that appear inside a quoted field and characters that appear outside one. A comma encountered while inside quotes is treated as literal data; a comma encountered outside quotes is treated as a field separator.
In the tested example, csv.reader parsed the same row into three fields, matching the intended structure of the data, while the naive split produced four. This difference came entirely from how each approach treated the comma inside the quoted name.
Quoted field boundaries
Wrapping a field in double quotation marks signals that the field may contain a comma without that comma being treated as a delimiter. The quotation marks mark the start and end of the field's content; they are not part of the value once parsed. A quoted field containing a comma is understood by csv.reader as a single field, even though it visually contains the delimiter character.
This is precisely the behavior a naive split lacks: it has no concept of a quoted region, so it splits on every comma it encounters, whether or not that comma is meant to be protected by surrounding quotes.
Escaped quotes inside quoted fields
In the dialect used by the csv module's default settings, a literal quotation mark that needs to appear inside a quoted field is represented by writing it twice in a row. csv.reader recognizes this doubled representation and converts it back into a single literal quotation mark in the parsed field value. The tested phrase included an escaped quotation mark, and csv.reader correctly reduced it to a single quote character in the output.
A plain split cannot perform this conversion. It would leave any doubled quotation marks in the output exactly as written, rather than interpreting them as an escape sequence for a single quote.
Comparing results between methods
Comparing the two approaches on the same input showed different field counts when a comma was protected by quotes: three fields from csv.reader against four from split. However, matching field counts between two parsing approaches does not by itself confirm that either approach parsed the row correctly. Counts can match by coincidence, or a method could produce the right number of fields with wrong content.
Verifying correctness requires checking the actual content of each field, not only how many fields resulted. Confirm that a comma expected inside one field stayed inside that field, and that any escaped quotation marks were converted to single quotes as expected.
Scope and limitations
These observations come from testing the csv module's reader callable in a single Python 3.12 run against a single synthetic example row using the module's default dialect. No testing was done against Excel-generated files, pandas, third-party CSV libraries, or non-default dialect settings.
These results should not be read as a description of the CSV format in general or as confirmation that any parser complies with a formal specification. They describe the behavior observed for the specific callable and inputs that were tested.
Run the verified example
Python
import csv
import io
line = '42,"Smith, Ada","said ""hello"""'
naive = line.split(",")
parsed = next(csv.reader(io.StringIO(line)))
assert len(naive) == 4
assert parsed == ["42", "Smith, Ada", 'said "hello"']
print("split fields:", len(naive))
print("csv fields:", len(parsed))
print(parsed)Observed output
split fields: 4
csv fields: 3
['42', 'Smith, Ada', 'said "hello"']Test scope and sources
Executed 2026-09-06 with Python 3.12.13 on Darwin. Tests use standard-library csv and synthetic strings. No Excel, pandas or third-party parser was executed. Do not generalize to every CSV dialect.
댓글 없음:
댓글 쓰기