Original portfolio tutorial · 15 October 2026 · Synthetic data

A completed batch can still be on hold

A batch job can exit normally and still produce records that must not be uploaded. A green process result answers whether the program completed. It does not answer whether the business output is ready. This example derives three bounded outcomes from a review result: ready_for_review, hold and invalid. It deliberately excludes product identifiers and raw descriptions from the signal object.

Start with the downstream action

In the illustrated workflow, a matched row has resolved the checks required by its matching layer. A review row still has an unresolved issue. Even an all-matched batch is ready for review, rather than automatically approved for publication. That wording avoids promoting a technical match into commercial approval. A parse failure is invalid: the number of complete business records is unknown, and the code does not turn that unknown into a fabricated zero.

Define a small outcome vocabulary

Use a vocabulary with an explicit owner and a finite set of values. This example has three batch outcomes and two row statuses. An unexpected row status raises an error. Silently mapping a future state to success is attractive for compatibility, but would make a new rejection path disappear from the dashboard. If the upstream matcher adds another status, change its aggregation rule deliberately and update the fixture before deploying the adapter.

Runnable example

import json

OUTCOMES = ("ready_for_review", "hold", "invalid")

def signals(batch):
    if batch.get("parse_error"):
        return {"batch_outcome": "invalid", "rows_matched": 0,
                "rows_for_review": 0, "rows_observed": None}
    rows = batch.get("rows")
    if not isinstance(rows, list) or not rows:
        raise ValueError("An empty/missing batch is not a ready batch")
    counts = {"matched": 0, "review": 0}
    for row in rows:
        status = row.get("status")
        if status not in counts:
            raise ValueError("Unknown row status")
        counts[status] += 1
    outcome = "hold" if counts["review"] else "ready_for_review"
    return {"batch_outcome": outcome, "rows_matched": counts["matched"],
            "rows_for_review": counts["review"], "rows_observed": len(rows)}

fixtures = [
    {"rows": [{"status": "matched", "code": "0012"},
              {"status": "matched", "code": "SECRET-CUSTOMER-SKU"}]},
    {"rows": [{"status": "matched"}, {"status": "review"},
              {"status": "review"}]},
    {"parse_error": True},
]
results = [signals(fixture) for fixture in fixtures]
assert [r["batch_outcome"] for r in results] == list(OUTCOMES)
assert results[1]["rows_matched"] + results[1]["rows_for_review"] == 3
assert results[2]["rows_observed"] is None
assert "SECRET-CUSTOMER-SKU" not in json.dumps(results)
for invalid in ({"rows": []}, {"rows": [{"status": "new_state"}]}):
    try:
        signals(invalid)
    except ValueError:
        pass
    else:
        raise AssertionError("Invalid state silently classified")
print(json.dumps(results, indent=2))

Run the complete example

Save the downloadable file as batch_outcomes_example.py and run python batch_outcomes_example.py. It prints one result for each outcome. The first contains two matched records and ready_for_review. The second contains one matched record, two review records and hold. The third contains invalid and rows_observed: null. The assertions also reject an empty batch and an unknown status. No telemetry SDK, collector, database or network endpoint is used by this script.

A useful accounting invariant

For a successfully parsed batch, rows_matched plus rows_for_review must equal rows_observed. That is a coverage check: every known record reaches exactly one classification. It is not proof that the classification is correct. The second fixture enforces that sum. For an invalid parse, the adapter returns no observed count because a damaged quoted field can absorb multiple physical lines. Recording zero there would mean no records existed, which has not been established.

Keep identity out of aggregate labels

A product code helps investigate a specific exception, but putting each code into a metric label creates a new series for each product. It may also disclose values that were never needed for operational monitoring. The example includes a conspicuous synthetic identifier in its input and asserts that it is absent from the aggregated JSON. That narrow check shows the adapter excludes raw row values. It is not a general privacy audit of an entire importer, its logs or its tracing system.

Map the adapter to metrics carefully

An eventual exporter could increment a completed-batches counter with a bounded outcome attribute and record row counts using a bounded classification attribute. Those are proposed integration choices. Repeatedly exporting the same saved result must not repeatedly increment completion as though a new batch occurred. Emit completion once at the job boundary, and keep retry attempts separate from business records. If using a cumulative counter, let the telemetry implementation manage its lifetime rather than reconstructing cumulative totals from arbitrary JSON snapshots.

Retries need an explicit counting boundary

A network retry of an exporter is different from a retry of the import job. The former should not create another business batch. The latter may represent a second attempt at the same input and needs an attempt policy. A stable correlation identifier can help trace that relationship, but should not become an unbounded aggregate metric label. Define where the job is considered terminal and test repeated callbacks at that boundary before claiming exactly-once reporting.

Connect operational success to business review

A useful dashboard can show the count of completed jobs alongside the share on hold. A rising hold count suggests a matching, catalogue or input issue even when process failures remain zero. Keep the denominator explicit. Failed parses are a batch outcome, not matched records, and an all-matched result still has a review step. Alerts need a volume threshold and a time window; this tutorial does not recommend an untested universal percentage.

Verification and limits

The runnable adapter passed its three outcome fixtures, accounting assertion, raw-value exclusion assertion, and two invalid-input checks. It operates on synthetic review objects and does not implement the matching algorithm. It does not export OpenTelemetry or show data inside SigNoz. Those integrations require a separate executed test with current SDK and collector versions. This distinction is the basis of a proposed observability article, not a claim of production deployment or an accepted paid assignment.

Recorded execution output

[
  {
    "batch_outcome": "ready_for_review",
    "rows_matched": 2,
    "rows_for_review": 0,
    "rows_observed": 2
  },
  {
    "batch_outcome": "hold",
    "rows_matched": 1,
    "rows_for_review": 2,
    "rows_observed": 3
  },
  {
    "batch_outcome": "invalid",
    "rows_matched": 0,
    "rows_for_review": 0,
    "rows_observed": null
  }
]

Download the complete Python example

Reference: Official documentation. Runtime used: Python 3.12.14.