Creating snip Filters

You are an expert at writing declarative YAML filters for snip, a CLI proxy that reduces LLM token consumption by filtering shell output.

Filter File Location

• Built-in filters: filters/*.yaml (embedded in the binary at build time)
• User filters: ~/.config/snip/filters/*.yaml (override built-in filters by name)
• Per-project filters: configure additional directories via filters.dir array in ~/.config/snip/config.toml (e.g. dir = ["~/.config/snip/filters", "${env.PWD}/.snip"]). Later directories take priority.

Filter Structure

Every filter is a YAML file with this structure:

name: "tool-subcommand"          # Required. Unique identifier, used for registry lookup.
version: 1                       # Schema version (always 1 for now).
description: "What this filter does"  # Human-readable purpose.

match:                           # Required. When to apply this filter.
  command: "tool"                # Required. The CLI tool name (e.g., "git", "go", "npm").
  subcommand: "sub"             # Optional. First non-flag argument (e.g., "test", "log").
  exclude_flags: ["-v", "--json"]  # Optional. Skip filter if user passes any of these.
  require_flags: ["--all"]      # Optional. Only apply if user passes ALL of these.

inject:                          # Optional. Modify command args before execution.
  args: ["--json"]              # Arguments to append to the command.
  defaults:                     # Flag defaults, only added if flag not already present.
    "-n": "10"
  skip_if_present: ["--json"]   # Don't inject anything if any of these flags are present.

streams: ["stdout", "stderr"]    # Optional. Which streams to filter. Default: ["stdout"].
                                 # Use ["stderr"] for tools that output to stderr (e.g., bun test).
                                 # Use ["stdout", "stderr"] to filter both streams merged together.

pipeline:                        # Required. Ordered list of transformation actions.
  - action: "keep_lines"
    pattern: "\\S"
  - action: "head"
    n: 20

on_error: "passthrough"          # What to do if the pipeline fails: "passthrough" or "empty".

Match Rules

• command is matched exactly against the first token of the shell command.
• subcommand is matched against the first non-flag argument.
• Flag matching uses prefix matching: "-v" matches both -v and -verbose.
• Registry lookup is O(1) by key "command" or "command:subcommand".

Inject Behavior

• Injected args are inserted before any -- separator, otherwise appended.
• defaults only apply if their flag key is not already present in the user's args.
• If any flag in skip_if_present is found, the entire inject block is skipped.

The 16 Pipeline Actions

Line Filtering

Action	Params	Description
keep_lines	pattern (regex)	Keep only lines matching the pattern
remove_lines	pattern (regex)	Remove lines matching the pattern
head	n (int, default 10), overflow_msg (string, default "+{remaining} more lines")	Keep first N lines
tail	n (int, default 10)	Keep last N lines
dedup	normalize ([]string of regexes to strip before comparing), top (int, 0=all)	Deduplicate lines, output "text (xN)" for repeats

Line Transformation

Action	Params	Description
truncate_lines	max (int, default 80), ellipsis (string, default "...")	Truncate long lines
strip_ansi	(none)	Remove ANSI escape codes
compact_path	(none)	Remove directory prefixes from file paths

Extraction & Grouping

Action	Params	Description
regex_extract	pattern (regex with capture groups), format (string using $0, $1, $2...)	Extract data via regex capture groups
group_by	pattern (regex with capture group), format (template, default "{{.Key}}: {{.Count}}"), top (int)	Group lines by capture group, count occurrences
aggregate	patterns (map of name->regex), format (Go template)	Count matches for named patterns across all input
state_machine	states (map of state definitions with keep, until, next)	Stateful line filtering with transitions

JSON Processing

Action	Params	Description
json_extract	fields ([]string), format (template, optional)	Extract fields from JSON input
json_schema	max_depth (int, default 3)	Output JSON type schema
ndjson_stream	group_by (string field name), format (template with .Key, .Count, .Events)	Process newline-delimited JSON

Formatting

Action	Params	Description
format_template	template (Go text/template, required)	Format output using Go template

Template Data for format_template

The template receives:

• {{.lines}} - all current lines joined with newlines
• {{.count}} - number of lines
• {{.groups}} - map from group_by action (if used earlier in pipeline)
• {{.stats}} - map from aggregate action (if used earlier in pipeline)

Metadata Flow Between Actions

• group_by sets metadata "groups" (map[string]int)
• aggregate sets metadata "stats" (map[string]int)
• format_template can access both via {{.groups}} and {{.stats}}
• All other actions pass metadata through unchanged

Design Principles

1. Start with keep_lines pattern "\\S" to strip blank lines early.
2. Use inject to request machine-readable output (e.g., --json, --porcelain) then filter that structured data.
3. Respect user intent: use exclude_flags to skip filtering when the user explicitly requests a different format.
4. Always set on_error: "passthrough" so raw output is returned if filtering fails.
5. Chain actions from broad to specific: filter noise first, then extract, then format.
6. Keep output minimal but useful: the goal is 60-90% token reduction while preserving actionable information.

Examples

Simple: remove noise lines

name: "npm-install"
version: 1
description: "Condensed npm install output"
match:
  command: "npm"
  subcommand: "install"
pipeline:
  - action: "remove_lines"
    pattern: "^(npm warn|npm notice)"
  - action: "keep_lines"
    pattern: "\\S"
  - action: "aggregate"
    patterns:
      added: "^added "
      removed: "^removed "
      up_to_date: "up to date"
    format: "{{if gt .up_to_date 0}}up to date{{else}}{{.added}} added, {{.removed}} removed{{end}}"
on_error: "passthrough"

Intermediate: inject flags + extract structured data

name: "go-test"
version: 1
description: "Condensed go test output with pass/fail summary"
match:
  command: "go"
  subcommand: "test"
  exclude_flags: ["-json", "-v", "-bench", "-run"]
inject:
  args: ["-json"]
  skip_if_present: ["-json", "-v", "-bench"]
pipeline:
  - action: "keep_lines"
    pattern: "\\S"
  - action: "keep_lines"
    pattern: "\"Test\":\""
  - action: "keep_lines"
    pattern: "\"Action\":\"(pass|fail)\""
  - action: "aggregate"
    patterns:
      passed: '"Action":"pass"'
      failed: '"Action":"fail"'
    format: "{{if and (eq .passed 0) (eq .failed 0)}}No tests found{{else}}{{.passed}} passed, {{.failed}} failed{{end}}"
on_error: "passthrough"

Advanced: state machine for multi-section output

name: "cargo-test"
version: 1
description: "Condensed cargo test output"
match:
  command: "cargo"
  subcommand: "test"
pipeline:
  - action: "remove_lines"
    pattern: "^\\s*(Compiling|Downloading|Downloaded|Updating|Running|Executable)"
  - action: "keep_lines"
    pattern: "\\S"
  - action: "state_machine"
    states:
      start:
        keep: "^(test |running |test result)"
        until: "^failures"
        next: "failures"
      failures:
        keep: "."
        until: "^$"
        next: "done"
  - action: "aggregate"
    patterns:
      pass: "\\.\\.\\. ok$"
      fail: "\\.\\.\\. FAILED$"
      ignored: "\\.\\.\\. ignored$"
  - action: "format_template"
    template: "{{.lines}}"
on_error: "passthrough"

Workflow to Create a New Filter

1. Identify the command and its typical verbose output.
2. Run the command and capture raw output to understand the structure.
3. Decide what to keep: what information does the LLM actually need?
4. Check if the tool has a machine-readable flag (--json, --porcelain, etc.) that would make filtering easier -- use inject if so.
5. Write the pipeline: strip blanks, filter/extract, aggregate, format.
6. Test the filter by placing it in ~/.config/snip/filters/ and running the command through snip.
7. To contribute: add the YAML to filters/ in the repo and submit a PR.