Filter JSON logs and count errors with jq

Use jq to select errors from JSON Lines logs, keep useful fields, and count failures by service.

how-to3 steps

Updated

in short

Use select to filter one log event at a time. For a small file, slurp the records and group them to count errors by service.

Use the small fixture below to check the workflow before applying it to your own project.

Tools used

Before you start

  • A Bash or Zsh terminal on Linux or macOS
  • The listed tools installed and available on PATH

Steps

  1. Step 1: Create three log events

    Each line is a complete JSON object.

    sh
    trmn_guide_dir="$(mktemp -d)"
cd "$trmn_guide_dir"
cat > events.jsonl <<'EOF'
{"service":"api","status":200,"message":"ok"}
{"service":"api","status":500,"message":"timeout"}
{"service":"worker","status":503,"message":"unavailable"}
EOF

  2. Step 2: Select error events

    The compact output keeps one matching event per line.

    sh
    jq -c 'select(.status >= 500) | {service, status, message}' events.jsonl
    output
    {"service":"api","status":500,"message":"timeout"}
{"service":"worker","status":503,"message":"unavailable"}

  3. Step 3: Count errors by service

    Slurp loads the whole file into memory. Use this grouping example on bounded files, not an endless log stream.

    sh
    jq -sc 'map(select(.status >= 500)) | group_by(.service) | map({service: .[0].service, errors: length})' events.jsonl
    output
    [{"service":"api","errors":1},{"service":"worker","errors":1}]

Further reading

[jq manual](https://jqlang.org/manual/)

FAQ

Why does malformed input fail?
jq expects valid JSON values. Remove non-JSON prefixes or parse them before filtering.
Does this change the file?
No. Both commands read events.jsonl and write their results to standard output.

related

jq details