Original portfolio tutorial · 15 October 2026 · Synthetic data

Unterminated CSV quotes are a parse error, not a repaired row

A CSV importer can produce a row even though its last quoted field never closes. If the application then announces a successful import, a damaged file becomes an apparently valid business record. This example makes that distinction visible: valid quoted commas, line breaks and doubled quotes survive, while an unterminated quote prevents the batch from entering a review workflow.

Define the accepted shape before parsing

The example accepts UTF-8 text with exactly two columns: code and description. It preserves codes as strings, including leading zeros. An empty description is permitted by this small example; deciding whether a product description is mandatory belongs to a separate business rule. The accepted header order is intentional. A larger importer might allow extra columns, but that would require an explicit policy rather than an accidental consequence of the parser.

Reproduce the failure rather than repairing the fixture

An opening quote changes how subsequent line breaks are interpreted. Appending a closing quote automatically would assert that every remaining line belongs to the same description. The importer cannot infer that intent. Keep the malformed fixture malformed and reject it. The support response can request a new export, accompanied by the source filename and a parser location, without placing the original customer fields into a monitoring label.

Runnable example

import csv
import io
import json

def records(text):
    reader = csv.DictReader(io.StringIO(text, newline=""), strict=True)
    headers = reader.fieldnames or []
    if headers != ["code", "description"]:
        raise ValueError("Expected unique code,description headers in that order")
    result = []
    for row in reader:
        if None in row or any(value is None for value in row.values()):
            raise ValueError("Record width differs from the header")
        result.append(row)
    if not result:
        raise ValueError("No data records")
    return result

good = 'code,description\n0012,"bolt, large"\n0013,"first\nsecond"\n0014,"say ""ready"""\n'
rows = records(good)
assert rows[0]["code"] == "0012"
assert rows[0]["description"] == "bolt, large"
assert rows[1]["description"] == "first\nsecond"
assert rows[2]["description"] == 'say "ready"'
bad = 'code,description\n0012,"unfinished\n'
try:
    records(bad)
except csv.Error:
    pass
else:
    raise AssertionError("Unterminated quote was accepted")
for text in ("code,code\n0012,bolt\n",
             "code,description\n0012,bolt,extra\n",
             "code,description\n0012\n",
             "code,description\n"):
    try:
        records(text)
    except (csv.Error, ValueError):
        pass
    else:
        raise AssertionError("Invalid shape was accepted")
print(json.dumps({"valid_records": len(rows),
                  "leading_zero_preserved": True,
                  "malformed_quote_rejected": True,
                  "invalid_shapes_rejected": 4}, indent=2))

Run the complete example

Save the downloadable file as strict_csv_example.py and run python strict_csv_example.py. It uses the Python standard library and creates no output files. The assertions cover three accepted records and five rejection paths: one malformed quote and four invalid shapes. The successful output states valid_records: 3, leading_zero_preserved: true, malformed_quote_rejected: true and invalid_shapes_rejected: 4. These are executed checks of the bundled example, not a claim that every dialect or production importer has been validated.

Why strict mode is only one layer

Python's CSV documentation describes strict mode as raising csv.Error for bad CSV input. It does not replace a schema or catalogue lookup. A syntactically valid record can still have an extra field, omit a required field, repeat a header or reference an unknown product code. Here the header comparison catches duplicate names before the first data row is mapped; width checks catch the extra and missing fields. A review system should distinguish these reasons so its next action is clear.

Keep records separate from physical lines

The accepted second record contains a quoted line break. Treating each physical line as one business record would split it incorrectly. For diagnostics, label the position accurately: reader.line_num reports consumed source lines, while an application record index counts parsed records. Neither number should be described as the other. For actual files, open with newline='' and the documented encoding. The in-memory fixture uses StringIO with the same newline intent to avoid platform newline rewriting in the test itself.

Do not publish partial output on a parse failure

A parser may yield several rows before it discovers a damaged tail. That is useful for diagnosis, but it is not permission to upload those earlier rows into a catalogue. This example collects all records before returning. For large files, use a staged output and publish it only after the entire parse and schema pass complete. Specify size limits, field limits and cancellation separately. This short example keeps everything in memory and has not established a safe maximum file size.

Preserve the next decision

After parsing, a matching layer can classify records as matched or requiring review. A parse error is earlier in the pipeline: there may not be a trustworthy record boundary to attach a correction to. Report the batch as invalid instead of calling it a successful batch with zero useful rows. Keep the original input unchanged, and make the user's recovery action explicit. Do not silently trim identifiers, substitute decimal separators or remove line breaks to obtain a green status.

Verification and limits

The downloadable example was run on the workspace's bundled Python. It passed the stated assertions. It is a portfolio tutorial with synthetic records, not a commissioned customer implementation. It does not validate arbitrary separators, legacy encodings, spreadsheet execution safety, catalogue semantics or large-file performance. Code and explanatory text were produced with AI assistance and checked against execution results. The public Python CSV reference is linked below for parser behavior; the rejection policy and fixture design are specific to this example.

Recorded execution output

{
  "valid_records": 3,
  "leading_zero_preserved": true,
  "malformed_quote_rejected": true,
  "invalid_shapes_rejected": 4
}

Download the complete Python example

Reference: Official documentation. Runtime used: Python 3.12.14.