Skip to main content

Learn · structured logging

Redact one structured log record

Produce real JSON with mgf-common, keep useful fields queryable, and verify that two made-up sensitive values are gone.

What you will prove

A formatter can preserve context while removing known sensitive shapes.

The exercise records an order ID, a nested profile, and an upstream note. It then checks the parsed JSON before printing a stable subset. The training values are fake; do not put a real secret into a test just to see whether it disappears.

Before you start

Use Python 3.11 or newer in a fresh directory.

  • Confirm your interpreter with python --version.
  • Make sure your machine can reach Magogi's public package index.
  • On Windows, replace python3 with py if that is how Python is installed.

Step 1

Create an environment and install the tested version.

Use one index. It serves the Magogi package and proxies the dependencies needed by this install.

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --index-url https://devpi.magogi.org/magogi/prod/+simple/ "mgf-common==0.52.0"

Windows PowerShell activation: .venv\Scripts\Activate.ps1. Windows Command Prompt activation: .venv\Scripts\activate.bat.

Step 2

Download and run the example.

Save the script beside your virtual environment. The core install is sufficient; this exercise does not require the optional observability integrations.

Download the Python example Read the lesson-file license 
python structured_logging_redaction.py

Step 3

Compare the stable result.

{
  "app": "package-lesson",
  "app_version": "1.0",
  "level": "INFO",
  "logger": "lesson.checkout",
  "msg": "checkout prepared",
  "order_id": "order-demo-104",
  "profile": {
    "password": "[REDACTED]",
    "tier": "training"
  },
  "upstream_note": "using [REDACTED] now"
}

The formatter's full record also contains a timestamp, host, process ID, and source line. Those values change between runs, so the example parses the record, asserts the meaningful fields, and prints only the subset above.

bootstrap(..., setup_logging=False) registers the app identity while the exercise owns one in-memory handler. extra adds the domain fields. JsonFormatter applies Redactor.default() before serialization. The password is caught by its key, and the bearer token is caught by its value pattern.

Boundary

Opaque objects need special care.

The default redactor walks mappings, lists, tuples, sets, bytes, dataclasses, and named tuples. It does not inspect arbitrary private object state. An opaque object's representation may be serialized after the redaction walk. Keep sensitive data in a supported structured container and never place it in an object's representation.

This exercise proves two fake values with version 0.52.0. It does not prove that every possible secret shape is recognized, that every handler in an application is configured correctly, or that a deployment's log transport is secure.

Troubleshooting

Three common failure modes

  • No matching distribution: check Python 3.11+, the version, and the index URL exactly as printed above.
  • A field remains visible: put it in a supported container. Add a project-specific name with Redactor(extra_keys=frozenset({"field_name"})).
  • KeyError from extra: names such as name, msg, and levelname belong to Python's LogRecord. Rename the domain field, for example to customer_name.

Next step

Check the exact bytes of a file

Use SHA-256 to compare two downloadable fixtures and catch one intentional mismatch.