Logs are for searching, not for reading

Most logging code is written for an imagined reader scrolling through a terminal, watching events go by. That reader does not exist in production. What exists is someone with a customer complaint, a rough time window, and a search box.

Everything that makes a log line good follows from that.

String interpolation destroys the thing you need

Here is the line almost everyone writes first:

console.log(`Sync failed for location ${locationId} on ${platform}: ${err.message}`);

It reads beautifully. It is also nearly useless at 3am, because the only thing you can do with it is substring search. You cannot ask how many locations failed on this platform in the last hour, or show me every failure for this account, because locationId is not a field — it is some characters in the middle of a sentence.

The structured version reads worse and works:

logger.error({
  event: "sync.failed",
  platform,          // "google" | "yelp" | "facebook"
  locationId,
  accountId,
  attempt,
  reason: err.code ?? "unknown",
  err,
}, "review sync failed");

Now the same data answers questions you didn't anticipate. event:"sync.failed" grouped by platform tells you whether one provider is down or everything is. Filtering by accountId answers the support ticket directly. None of that required predicting the question in advance, which is the point — at 3am you do not know what you are looking for yet.

Give events stable names

event: "sync.failed" matters more than it looks. A stable, low-cardinality event name is what makes a log searchable by kind rather than by wording.

Log messages get edited. Someone improves the phrasing, and every saved search and alert built on that phrasing silently stops matching. A named event survives rewording, because the human-readable message becomes decoration rather than the identifier.

I use noun.verb in the past tense — sync.failed, email.sent, agent.tool_called. The convention matters less than having one.

Log the identifiers you will search by, on every line

The single most common gap: the line that records the failure has the error but not the account. So you find the failure and still cannot tell whose it is.

The fix is a request-scoped logger that carries the identifiers automatically, rather than relying on remembering to pass them:

// One child logger per request; every line it emits carries these fields.
app.use((req, _res, next) => {
  req.log = logger.child({
    requestId: req.id,
    accountId: req.auth?.accountId,
    route: req.route?.path,
  });
  next();
});

If a field has to be added by hand at each call site, it will be missing from exactly the line you needed it on.

Levels are a routing decision

Level inflation is what makes logs unusable. Everything becomes info, so info means nothing, so nobody filters, so volume wins.

The rule I hold to is that a level answers who is woken by this:

• error — something failed that a person must act on. If nobody would act, it is not an error.
• warn — degraded but handled. A retry succeeded on the third attempt; a provider is stale but the others are current.
• info — the state changes you would want in an audit trail. A message sent, a job completed, a sync run. One or two per request, not twenty.
• debug — off in production, on when you are actually investigating.

The consequence worth stating plainly: an error that fires routinely and needs no action trains everyone to ignore errors. That is a worse outcome than not logging it at all.

Never log the payload

It is tempting, during an incident, to log the whole request body. It is also how credentials, tokens and personal data end up in a third-party log service with a long retention window and a much wider access list than your database.

Log identifiers and shapes. messageLength: 412 is nearly always as useful as the message, and it is not a disclosure.

Traces answer the question logs cannot

Logs tell you what happened. They do not tell you where the time went. When someone asks why a request took eight seconds, structured logs give you a scattering of timestamps to reassemble by hand.

Adding OpenTelemetry spans around the boundaries — the database call, the provider request, the model call — turns that into a picture. The reason to do it before you need it is that the interesting comparison is against last week, and you cannot instrument the past.

The test

Before shipping a log line, ask: three months from now, with a customer complaint and a fifteen-minute window, would this line be findable by someone who does not know it exists?

If finding it requires already knowing the wording, it is decoration. If it can be found by account, by event, by outcome, it is observability.