Python

lumber-cloud is Lumber's Python client. It encrypts every log with AES-256-GCM before it leaves your process, and it plugs into Python's logging, so the loggers your code and its libraries already use can send to Lumber too.

Install

pip install lumber-cloud

Python 3.9 or later. Its one dependency is cryptography. The package is lumber-cloud on PyPI, and you import it as lumber_cloud.

Quick start

import os
import lumber_cloud as lumber

lumber.config(api_key=os.environ["LUMBER_API_KEY"])

lumber.info("Server started", port=8000)
lumber.warn("Rate limit approaching", request_id="req-123", user_id=456)
lumber.error("Payment failed", trace_id="trace-abc", order_id=789)

Call config() once at startup; every module that imports lumber_cloud then logs with the same client. Logs are sent from a background thread, so logging never waits on the network, and logs still queued when your program exits get up to 5 seconds to send.

The API key comes from the Lumber app (see Getting Started). Keep it in an environment variable or your secrets manager, never in source control, and never in an app or game you ship to people (why).

With logging

LumberHandler sends records from Python's logging to Lumber:

import logging
import os
from lumber_cloud import LumberHandler

logging.basicConfig(level=logging.INFO)  # keep printing to the console
logging.getLogger().addHandler(LumberHandler(os.environ["LUMBER_API_KEY"]))

log = logging.getLogger(__name__)
log.info("Report ready in %.1fs", 2.4, extra={"request_id": "req-123", "rows": 1500})

try:
    charge(order)
except PaymentError:
    log.exception("Payment failed")  # sent at ERROR with the traceback

Each record becomes a log at the matching level (CRITICAL is FATAL), with its arguments filled into its message. extra= keys go into the log's metadata, except request_id and trace_id, which are the log's own fields, and context, which goes into its context. The logger's name, file, line and function go into the context too.

Django

Add the handler to LOGGING in your settings:

LOGGING = {
    "version": 1,
    "handlers": {
        "lumber": {
            "class": "lumber_cloud.LumberHandler",
            "api_key": os.environ["LUMBER_API_KEY"],
            "level": "INFO",
        },
    },
    "root": {"handlers": ["lumber"], "level": "INFO"},
}

To share one client between a handler and your own calls, pass it in: LumberHandler(client=lumber.config(api_key=...)).

Log methods

The module's functions log with the client config() made. LumberClient has the same methods, for when you need more than one.

Data

Keyword arguments become the log's fields:

lumber.info(
    "User signed in",
    request_id="req-abc",           # the log's request ID
    trace_id="trace-xyz",           # the log's trace ID
    context={"region": "us-east"},  # merged into its context
    metadata={"plan": "pro"},       # merged into its metadata
    user_id=123,                    # any other keyword goes into metadata
    exc_info=err,                   # an exception (or True for the one being handled)
)

An explicit metadata key wins over a colliding keyword. An exception goes into metadata as exception (ValueError: bad amount) and traceback. Values JSON can't hold are sent as text (a datetime as ISO 8601, a set as a list), and a NaN as null. Everything, the message included, is encrypted.

Options

lumber.config(
    api_key=os.environ["LUMBER_API_KEY"],
    environment="production",  # sent as `env` in every log's context
    auto_context=True,         # send the process id and hostname with every log
    caller=False,              # also send the file, line and function that logged
    caller_depth=0,            # frames to skip for `caller`, one per wrapper around Lumber
    timeout=10.0,              # seconds to wait for Lumber to answer one log
    max_queue=10_000,          # logs waiting to send before new ones are dropped
    shutdown_timeout=5.0,      # seconds queued logs get when the program exits
)

Delivery

Logs are sent in order, one POST /logs per log, from a daemon thread over a connection kept open between logs.

• A network error or a 5xx counts as a failed attempt. Each log gets 5 attempts, waiting 1, 2, 4, then 8 seconds between them, and is then dropped with a warning on stderr.
• A log Lumber refuses (a 4xx, such as an invalid API key) is dropped at once with a warning, since sending it again won't help.
• When your team has sent its plan's logs for the month, Lumber answers 429: the client warns once and drops logs until the month turns.
• While Lumber can't be reached, up to max_queue logs wait; past that, new ones are dropped.

Forking servers (gunicorn, uWSGI) are handled: a worker process starts its own sending thread and connection with its first log.

Source

The client is open source: Lumber-Python on GitHub, published as lumber-cloud on PyPI. It's tested against the same known answers for encryption and the wire format as Lumber's other clients.