Export a scoped code inventory with tokei

Count a fixture with tokei, exclude generated files, and inspect machine-readable code totals with jq.

how-to3 steps

Updated

in short

Keep generated directories outside the count, export the scoped result as JSON, and read each language total rather than treating file size as code size.

Work through these disposable fixtures before adapting the commands to your own files. Run the steps in the same terminal session.

Tools used

Before you start

  • Bash on Linux or macOS
  • The listed tools installed and available on PATH

Steps

  1. Step 1: Create source and generated files

    The fixture has one comment and two code lines in its hand-written Python file.

    sh
    trmn_guide_dir="$(mktemp -d)"
    cd "$trmn_guide_dir"
    mkdir -p src generated
    printf '# example\nvalue = 3\nprint(value)\n' > src/app.py
    printf 'print("generated")\n' > generated/output.py
  2. Step 2: Export all source counts

    Write the report outside the counted tree so later runs do not count their own output.

    sh
    tokei . --output json > "../trmn-counts-all-$(basename "$trmn_guide_dir").json"
    jq '.Python | {code, comments}' "../trmn-counts-all-$(basename "$trmn_guide_dir").json"
    output
    {
      "code": 3,
      "comments": 1
    }
  3. Step 3: Exclude the generated directory

    The quoted exclusion is interpreted by tokei.

    sh
    tokei . --exclude 'generated' --output json > "../trmn-counts-scoped-$(basename "$trmn_guide_dir").json"
    jq '.Python | {code, comments}' "../trmn-counts-scoped-$(basename "$trmn_guide_dir").json"
    output
    {
      "code": 2,
      "comments": 1
    }

Further reading

[tokei usage and output formats](https://github.com/XAMPPRocky/tokei) [jq manual](https://jqlang.org/manual/)

FAQ

Are line counts a productivity score?
No. Use these counts to understand repository composition and review the scope of maintenance work.
Why keep exclusions consistent?
Adding generated or vendored directories can change the totals without a corresponding change in code you maintain. Keep the same scope when comparing snapshots.

related