rewrite call-classify skill for gemma
save good data to call types
NEED calltype in TUI, put them in config.json, if keybinding has calltypes and does not have a direct keybinding, popup a selection box
delete birdnet labels, too noisy

ollama run gemma4:e4b
ollama launch pi --model gemma4:e4b
ollama run qwen3.5:9b
ollama launch pi --model qwen3.5:9b
tell me what you know about fiordland tokoeka

## Names
- Roroa
- Great Spotted Kiwi
- grskiw1
**Nocturnal** but may be active during the dawn chorus
## Tips
- **Male**: most common, bold vertical stripes ~1/3 to 1/2 spectrogram height
- **Female**: higher frequency, thinner, often fainter - don't mistake faintness for absence
- **Duet**: if ANY female component visible with male, it's a Duet
- **Distant calls**: fainter but still valid - only skip if pattern unclear

## Names
- Long-tailed Cuckoo
- Long-tailed Koel
- LTC
- lotkoe1 
**Active during the day and the night**
## Tips
**True LTC pattern** (from verified examples):
- **Ascending whistle**: rising frequency sweep from ~1-2 kHz up to ~4-6 kHz
- **Shape**: clear diagonal upward line on spectrogram, NOT vertical bars
- **Duration**: typically 1-3 seconds per note
- **Quality**: smooth, flute-like whistle, not harsh or buzzy
- **Harmonics**: may show faint harmonics above main note
**Common confusion**:
- Kiwi calls (vertical bars, repetitive)
- Bellbird/Tui (more complex, variable)
- Other cuckoos (different pitch patterns)
**When uncertain**: Compare directly against the 10 verified R620 reference examples in `Long-tailed Cuckoo/` folder.

# Chaffinch Calltypes and ID
## ALARM_Huit
huit' - thin notes that slope up slightly from left to right, evenly spaced a few seconds apart across the clip
## CONTACT_Chink
'chink' - vertical (no slope) ticks, typically 2 notes close together (sometimes 1 or 3), often followed by another similar burst a few seconds later
## SONG_Chip-chip-chip-tell-tell-tell-cherry-erry-erry-tissi-cheweeo
complex multi-syllable phrase (~2-3s), a dense cluster of broader vertical brushes with wider frequency spread; frequency contour starts mid, dips in the middle, and ends HIGH (terminal flourish), often with a curved/descending sweep in between, if compressed in time shows a steep extended tick shape, up and to the right
# Names
Chaffinch
comcha

#!/usr/bin/env python3
"""Classify bird-call spectrograms with a local Ollama vision model and apply
certainty upgrades via `skraak calls modify`.
Species-agnostic: reads calltype definitions and eBird code from the matching
call-library entry at <call-library>/<Species>/SKILL.md.
Autonomous orchestrator: classify -> modify -> log -> checkpoint -> next.
Resumable. Safe to Ctrl-C and relaunch.
"""
import argparse
import base64
import json
import re
import subprocess
import sys
import time
import urllib.request
from pathlib import Path
SKILLS_ROOT = Path(__file__).resolve().parent.parent
DEFAULT_CALL_LIBRARY = SKILLS_ROOT / "call-library"
DEFAULT_SKRAAK_BIN = (SKILLS_ROOT.parent.parent / "skraak").resolve()
UNKNOWN_HINT = "the spectrogram doesn't match any above (unclear, faint, fragmented, or not this species)"
CLIP_RE = re.compile(r"^(?P<prefix>.+?)_(?P<basename>.+)_(?P<start>\d+)_(?P<end>\d+)\.png$")
def parse_species_skill(skill_md: Path):
    """Parse a call-library species SKILL.md.
    Expected structure:
        # <anything> Calltypes and ID
        ## <CALLTYPE_FOLDER_NAME>
        <description text, one or more paragraphs>
        ## <...>
        ...
        # Names
        <Common Name>
        <ebird code>
    Returns: (calltypes, common_name, species_code) where calltypes is a dict
    ordered by token alphabetically: {token: {"folder": str, "hint": str}}.
    """
    text = skill_md.read_text()
    calltypes = {}
    names = []
    section = None
    current_folder = None
    current_lines = []
    def flush_calltype():
        nonlocal current_folder, current_lines
        if current_folder is None:
            return
        hint = " ".join(current_lines).strip()
        token = current_folder.split("_", 1)[0].lower()
        calltypes[token] = {"folder": current_folder, "hint": hint}
        current_folder = None
        current_lines = []
    for line in text.splitlines():
        if line.startswith("## "):
            flush_calltype()
            current_folder = line[3:].strip()
            section = "calltype"
        elif line.startswith("# "):
            flush_calltype()
            header = line[2:].strip().lower()
            section = "names" if "name" in header else "other"
        else:
            if section == "calltype" and line.strip():
                current_lines.append(line.strip())
            elif section == "names" and line.strip():
                names.append(line.strip())
    flush_calltype()
    common_name = names[0] if names else None
    species_code = names[1] if len(names) >= 2 else None
    if not calltypes:
        sys.exit(f"no calltypes parsed from {skill_md}")
    if not species_code:
        sys.exit(f"no species code (second line under # Names) in {skill_md}")
    # Stable alphabetical order by token (avoids primacy/recency bias toward any class).
    calltypes = dict(sorted(calltypes.items()))
    return calltypes, common_name, species_code
def b64(path: Path) -> str:
    return base64.b64encode(path.read_bytes()).decode()
def load_refs(species_dir: Path, calltypes: dict, species_code: str,
              refs_per_type: int) -> dict:
    """Prefer curated real-data refs (<code>+*.png) over Internet_*."""
    out = {}
    for token, meta in calltypes.items():
        folder = species_dir / meta["folder"]
        all_pngs = sorted(folder.glob("*.png"))
        real = [p for p in all_pngs if p.name.startswith(f"{species_code}+")]
        internet = [p for p in all_pngs if not p.name.startswith(f"{species_code}+")]
        pngs = (real + internet)[:refs_per_type]
        if not pngs:
            sys.exit(f"no reference PNGs in {folder}")
        out[token] = [(p.name, b64(p)) for p in pngs]
    return out
def build_prompt(calltypes: dict, common_name: str, refs: dict,
                 query_png: Path, no_think: bool) -> list:
    tokens = list(calltypes.keys()) + ["unknown"]
    options = "|".join(tokens)
    hints_block = "".join(
        f"  {t:<8} - {calltypes[t]['hint']}\n" for t in calltypes
    ) + f"  {'unknown':<8} - {UNKNOWN_HINT}\n"
    prefix = "/no_think " if no_think else ""
    content = [{"type": "text", "text": prefix +
        f"You are classifying bird-call spectrograms. All are {common_name}.\n"
        f"Choose one of these calltypes:\n{hints_block}"
        "\nLabeled REFERENCE examples follow, then ONE query spectrogram.\n"
        "Respond with STRICT JSON only: "
        f'{{"calltype":"{options}","confidence":"low|medium|high","reason":"one short sentence"}}'
    }]
    for token, pairs in refs.items():
        content.append({"type": "text", "text": f"\n=== REFERENCE: {token} ==="})
        for _, data in pairs:
            content.append({"type": "image_url",
                            "image_url": {"url": f"data:image/png;base64,{data}"}})
    content.append({"type": "text", "text": "\n=== QUERY (classify this) ==="})
    content.append({"type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{b64(query_png)}"}})
    return [{"role": "user", "content": content}]
def classify(ollama_url: str, model: str, messages: list, timeout: int) -> dict:
    payload = {
        "model": model,
        "messages": messages,
        "temperature": 0,
        "response_format": {"type": "json_object"},
    }
    req = urllib.request.Request(
        ollama_url,
        data=json.dumps(payload).encode(),
        headers={"Content-Type": "application/json"},
    )
    with urllib.request.urlopen(req, timeout=timeout) as resp:
        body = json.loads(resp.read())
    raw = body["choices"][0]["message"]["content"]
    usage = body.get("usage", {})
    try:
        parsed = json.loads(raw)
    except json.JSONDecodeError:
        parsed = None
    return {"raw": raw, "parsed": parsed, "usage": usage}
def run_modify(skraak: str, data_file: Path, reviewer: str, filt: str,
               segment: str, species_label: str, certainty: int) -> dict:
    cmd = [
        skraak, "calls", "modify",
        "--file", str(data_file),
        "--reviewer", reviewer,
        "--filter", filt,
        "--segment", segment,
        "--species", species_label,
        "--certainty", str(certainty),
    ]
    proc = subprocess.run(cmd, capture_output=True, text=True, timeout=60)
    return {
        "cmd": cmd,
        "returncode": proc.returncode,
        "stdout": proc.stdout[-500:],
        "stderr": proc.stderr[-500:],
    }
def load_checkpoint(path: Path) -> set:
    if not path.exists():
        return set()
    return {l.strip() for l in path.read_text().splitlines() if l.strip()}
def append_line(path: Path, text: str) -> None:
    with path.open("a") as f:
        f.write(text + "\n")
def parse_args():
    p = argparse.ArgumentParser()
    p.add_argument("--clips-dir", required=True,
                   help="Folder of spectrogram PNGs (from `skraak calls clip`).")
    p.add_argument("--source-folder", required=True,
                   help="Folder containing the original .data files.")
    p.add_argument("--prefix", required=True,
                   help="Same --prefix used when clips were generated.")
    p.add_argument("--species-common-name", required=True,
                   help="Common name matching a subfolder in --call-library (e.g. Chaffinch).")
    p.add_argument("--filter", dest="filt", required=True,
                   help="ML filter name matching labels (e.g. opensoundscape-multi-1.0).")
    p.add_argument("--log-path", required=True)
    p.add_argument("--checkpoint-path", required=True)
    p.add_argument("--call-library", default=str(DEFAULT_CALL_LIBRARY),
                   help="Path to the call-library skill root.")
    p.add_argument("--reviewer", default="gemma4-e4b")
    p.add_argument("--model", default="gemma4:e4b")
    p.add_argument("--ollama-url", default="http://localhost:11434/v1/chat/completions")
    p.add_argument("--skraak-bin", default=str(DEFAULT_SKRAAK_BIN))
    p.add_argument("--data-ext", default=".WAV.data",
                   help="Extension of .data files (AudioMoth default .WAV.data).")
    p.add_argument("--refs-per-type", type=int, default=3)
    p.add_argument("--batch-size", type=int, default=10)
    p.add_argument("--limit", type=int, default=0, help="0 = no limit")
    p.add_argument("--request-timeout", type=int, default=300)
    p.add_argument("--high-certainty", type=int, default=90,
                   help="Certainty to set for 'high' picks. Use 91 to mark a re-pass.")
    p.add_argument("--medium-certainty", type=int, default=80)
    p.add_argument("--no-think", action="store_true",
                   help="Prepend /no_think (for qwen3/thinking models).")
    return p.parse_args()
def main():
    args = parse_args()
    clips_dir = Path(args.clips_dir)
    source = Path(args.source_folder)
    log_path = Path(args.log_path)
    ckpt_path = Path(args.checkpoint_path)
    species_dir = Path(args.call_library) / args.species_common_name
    skill_md = species_dir / "SKILL.md"
    if not skill_md.exists():
        sys.exit(f"missing: {skill_md}")
    calltypes, common_name, species_code = parse_species_skill(skill_md)
    common_name = common_name or args.species_common_name
    refs = load_refs(species_dir, calltypes, species_code, args.refs_per_type)
    conf_map = {"high": args.high_certainty, "medium": args.medium_certainty}
    done = load_checkpoint(ckpt_path)
    all_pngs = sorted(clips_dir.glob("*.png"))
    todo = [p for p in all_pngs if p.name not in done]
    if args.limit > 0:
        todo = todo[:args.limit]
    print(f"species: {common_name} ({species_code})  "
          f"calltypes: {list(calltypes)}  refs/type: {args.refs_per_type}", flush=True)
    print(f"total pngs: {len(all_pngs)}, already done: {len(done)}, "
          f"to process this run: {len(todo)}", flush=True)
    stats = {"modify": 0, "skip": 0, "error": 0,
             "by_calltype": {t: 0 for t in list(calltypes) + ["unknown", "other"]}}
    t_batch_start = time.time()
    for i, png in enumerate(todo, 1):
        t0 = time.time()
        m = CLIP_RE.match(png.name)
        if not m:
            append_line(log_path, json.dumps({"clip": png.name, "action": "skip",
                                              "error": "filename_parse_failed"}))
            append_line(ckpt_path, png.name)
            stats["error"] += 1
            continue
        basename = m.group("basename")
        start, end = m.group("start"), m.group("end")
        segment = f"{start}-{end}"
        data_file = source / f"{basename}{args.data_ext}"
        record = {"clip": png.name, "basename": basename, "segment": segment,
                  "data_file": str(data_file)}
        try:
            result = classify(args.ollama_url, args.model,
                              build_prompt(calltypes, common_name, refs, png, args.no_think),
                              args.request_timeout)
        except Exception as e:
            record.update(action="skip", error=f"classify_failed: {e}",
                          elapsed_s=round(time.time() - t0, 2))
            append_line(log_path, json.dumps(record))
            append_line(ckpt_path, png.name)
            stats["error"] += 1
            continue
        record["llm_raw"] = result["raw"]
        record["llm_parsed"] = result["parsed"]
        record["llm_usage"] = result["usage"]
        parsed = result["parsed"] or {}
        calltype = str(parsed.get("calltype", "")).strip().lower()
        confidence = str(parsed.get("confidence", "")).strip().lower()
        if calltype in stats["by_calltype"]:
            stats["by_calltype"][calltype] += 1
        else:
            stats["by_calltype"]["other"] += 1
        certainty = conf_map.get(confidence)
        if calltype in calltypes and certainty:
            species_label = f"{species_code}+{calltype}"
            if not data_file.exists():
                record.update(action="skip", error="data_file_missing")
                stats["error"] += 1
            else:
                mod = run_modify(args.skraak_bin, data_file, args.reviewer,
                                 args.filt, segment, species_label, certainty)
                record.update(action="modify", modify=mod,
                              species_label=species_label, certainty=certainty)
                if mod["returncode"] == 0:
                    stats["modify"] += 1
                else:
                    stats["error"] += 1
        else:
            record.update(action="skip",
                          skip_reason=f"calltype={calltype!r} confidence={confidence!r}")
            stats["skip"] += 1
        record["elapsed_s"] = round(time.time() - t0, 2)
        append_line(log_path, json.dumps(record))
        append_line(ckpt_path, png.name)
        if i % args.batch_size == 0 or i == len(todo):
            dt = time.time() - t_batch_start
            remaining = len(todo) - i
            eta_min = (dt / args.batch_size) * remaining / 60 if args.batch_size else 0
            bc = stats["by_calltype"]
            bc_str = " ".join(f"{k}={v}" for k, v in bc.items() if v or k != "other")
            print(f"[{i}/{len(todo)}] "
                  f"modify={stats['modify']} skip={stats['skip']} err={stats['error']} | "
                  f"{bc_str} | batch {dt:.1f}s, eta {eta_min:.1f}m",
                  flush=True)
            t_batch_start = time.time()
    print("done.", flush=True)
    print(json.dumps(stats, indent=2), flush=True)
if __name__ == "__main__":
    main()

---
name: call-classification-ollama
description: Bulk-classify ML-generated bird call segments autonomously using a local Ollama vision model (gemma4:e4b). Classify-and-apply loop over hundreds or thousands of clips, with checkpointing and JSONL audit log. Use when the manual /call-classification flow would be too slow.
---
# Autonomous Call Classification with Local Ollama
Batch alternative to `/call-classification`. Runs a local vision LLM over a
folder of spectrogram PNGs, parses a strict-JSON reply, and applies
`skraak calls modify` for every confident pick. Runs unattended for hours;
user reviews results live (or after the fact) in the `skraak classify` TUI.
**This skill does not inspect spectrograms itself — it orchestrates the
Python script at `classify.py` in this folder.** That script is the
orchestrator and runs autonomously once launched.
For per-species calltype descriptions and reference spectrograms, see
`/call-library/<Species>/`. `classify.py` reads those at runtime.
## When to use
- Hundreds or thousands of segments to classify for a single species.
- You have a validated set of reference PNGs under `/call-library/<Species>/`.
- You're OK with the quality tradeoff: gemma4:e4b is useful but imperfect.
  It reliably spots the dominant calltype and leaves ambiguous clips at their
  original certainty. Always review results in the TUI afterwards.
Prefer `/call-classification` (the interactive skill) when: few clips, a new
species you're still building intuition for, or when spectrogram features are
subtle and you want Claude to reason per-clip.
## Prerequisites
1. Ollama running locally with the chosen model pulled:
   ```bash
   ollama list                          # verify
   ollama pull gemma4:e4b               # if missing
   curl -s http://localhost:11434/v1/models   # sanity check endpoint
   ```
2. `skraak` binary built at the repo root (`cd skraak && go build -o skraak`).
3. Species exists under `/call-library/<Species>/` with:
   - `SKILL.md` listing calltypes (as `## FOLDER_NAME` headings, e.g.
     `## ALARM_Huit`) plus a `# Names` section whose second non-empty line is
     the eBird code (e.g. `comcha`).
   - One subfolder per calltype, containing reference PNGs. Curated
     real-data refs named `<ebird_code>+<calltype>_*.png` are preferred over
     legacy `Internet_*.png` and are picked first.
## Workflow
### Step 1 — Scope
Get from user: **folder path**, **filter** (e.g. `opensoundscape-multi-1.0`),
**species** (eBird code, e.g. `comcha`), **certainty band** to process
(typically `70` for first pass, or `90` for a re-classification pass).
Optionally ask whether the run should stamp `--high-certainty 91` (or any
value distinct from `90`) so the results are filterable in the TUI as a
distinct pass.
### Step 2 — Generate clips
```bash
./skraak calls clip \
  --folder <folder_path> \
  --output /tmp/<species>_<pass>_clips \
  --prefix <pass_tag> \
  --filter <filter> \
  --species <ebird_code>[+<calltype>] \
  --certainty <n> \
  --color
```
- `--prefix` becomes part of each PNG filename: `<prefix>_<basename>_<start>_<end>.png`.
  Use distinct prefixes per pass (e.g. `batch1`, `batch2`) for clean separate runs.
- `--color` matches what the model was tuned against. Keep it on.
- Default `--size 224` is fine. Bump to 448 only if you need more detail
  (costs more tokens per request).
Confirm the clip count matches the expected segment count before launching
the long run.
### Step 3 — Launch the classifier
```bash
SKILL_DIR=/home/david/go/src/skraak/.claude/skills/call-classification-ollama
python3 "$SKILL_DIR/classify.py" \
  --clips-dir /tmp/<species>_<pass>_clips \
  --source-folder <folder_path> \
  --prefix <pass_tag> \
  --species-common-name <CommonName> \
  --filter <filter> \
  --log-path /tmp/<species>_<pass>_log.jsonl \
  --checkpoint-path /tmp/<species>_<pass>_checkpoint.txt \
  --high-certainty 90 \
  > /tmp/<species>_<pass>_run.log 2>&1 &
```
Launch in the background for long runs. Progress lines print every 10 clips
(batch size `--batch-size`). Ctrl-C is safe; rerun the same command and the
checkpoint resumes.
**Common overrides:**
- `--high-certainty 91` — mark this as a second pass; find+review via
  `skraak calls classify --certainty 91`.
- `--limit 100` — smoke-test before the full run.
- `--refs-per-type 3` (default). Bump to 4–5 only if the model is getting
  stuck in a single-class bias; more refs is not automatically better.
- `--no-think` — for qwen3 / thinking models. With gemma4 leave off.
### Step 4 — Review
While the script runs (or after), open `skraak classify` filtered on
`--reviewer gemma4-e4b` (or the certainty you set) to spot-check. The JSONL
log at `--log-path` records every decision — parseable with `jq`:
```bash
jq -c '{clip, ct: .llm_parsed.calltype, conf: .llm_parsed.confidence, act: .action}' \
  /tmp/<species>_<pass>_log.jsonl | head
```
## Tuning notes from prior runs
- **Reference-image quality dominates everything.** A first pass that
  produced zero `song` or `alarm` picks (only `contact`/`unknown`) was fixed
  by swapping Internet-sourced refs for curated real-data refs.
- **Write concrete, observable visual descriptions in the species SKILL.md**
  — "slopes up slightly from left to right, evenly spaced" works much better
  than "alarm call". The model's reasoning text will echo these descriptions
  back at you when it's working well.
- **Reference order is alphabetical by calltype token.** The parser sorts to
  avoid primacy/recency bias toward any class.
- **Confidence calibration is poor.** Gemma4 tends to stamp `high` on every
  confident pick and `medium`/`low` only on `unknown`. Don't over-read the
  confidence — treat it as a coarse signal.
- **Latency** ~6s/clip at 2 refs/class, ~8s at 3 refs/class. Plan
  accordingly: 1000 clips ≈ 2 hours.
- **Qwen3 thinking models** dump output into reasoning by default and
  return empty `content`. `/no_think` prefix works in simple prompts but
  can be ignored on multi-part prompts — gemma4 is the reliable pick.
## Rollback
Every modify is stamped with `--reviewer gemma4-e4b` (or whatever was
passed). To reverse a pass:
```bash
# Via SQL on the live DB
skraak sql --db ./db/skraak.duckdb \
  "UPDATE label SET certainty = 70 WHERE reviewer = 'gemma4-e4b' AND certainty = 91"
```
or re-run with the correct values to overwrite.
## Related skills
- `/call-classification` — interactive human/Claude review (fewer clips,
  higher scrutiny). Use first on a new species until you trust the refs.
- `/call-library` — the per-species reference store this skill reads from.

## Names
- Roroa
- Great Spotted Kiwi
- grskiw1
**Nocturnal** but may be active during the dawn chorus
## Tips
- **Male**: most common, bold vertical stripes ~1/3 to 1/2 spectrogram height
- **Female**: higher frequency, thinner, often fainter - don't mistake faintness for absence
- **Duet**: if ANY female component visible with male, it's a Duet
- **Distant calls**: fainter but still valid - only skip if pattern unclear

## Names
- Long-tailed Cuckoo
- Long-tailed Koel
- LTC
- lotkoe1 
**Active during the day and the night**
## Tips
**True LTC pattern** (from verified examples):
- **Ascending whistle**: rising frequency sweep from ~1-2 kHz up to ~4-6 kHz
- **Shape**: clear diagonal upward line on spectrogram, NOT vertical bars
- **Duration**: typically 1-3 seconds per note
- **Quality**: smooth, flute-like whistle, not harsh or buzzy
- **Harmonics**: may show faint harmonics above main note
**Common confusion**:
- Kiwi calls (vertical bars, repetitive)
- Bellbird/Tui (more complex, variable)
- Other cuckoos (different pitch patterns)
**When uncertain**: Compare directly against the 10 verified R620 reference examples in `Long-tailed Cuckoo/` folder.

Note skill: call-classification

Note use skill: call-classification

**Base folder:** [path]
**Folders:**
[list]
**Reviewer:**
[name]
**Filter**
[name]
**Species**
[species+calltype]
**Input Certainty**
[number]
**Commands:**
```bash
./skraak calls clip --folder "<folder_path>" --prefix <prefix> --output /tmp/<prefix>/ \
  --filter <filter> --species <species+calltype> --certainty <number> --size 448 --color
./skraak calls modify --file <.data file> --reviewer <you> --filter opensoundscape-XXX-1.X --segment 12-15 --species <species+calltype> --certainty 90 [--bookmark --comment "Clear example of male call"]
```
**Prompt:**
1. Make a todo list for the folders before starting.
2. Run run the command to extract the low-certainty unclassified segments.
3. Read the species reference spectrograms from the skill subfolder to calibrate your eye for calls before examining images.
4. Read every PNG. 
5. Execute changes.  
6. Mark the folder done in your todo list, then move to the next.
**Read every png image**
**Goal is to recover missed calls.**
Identify any species+calltype calls hidden in the pool.
**Goal is to increase certainty for species and calltype**
Increase certainty on species+calltype

Take a look at all comcha (Chaffinch) calls in folder /media/david/SSD4/Twenty_Four_Seven/R620/2024-05-06 for the opensoundscape-multi-1.0 filter and certainty 70.
sort calls into:
- comcha+song (set certainty 90, reviewer )
- comcha+contact (set certainty 90, reviewer )
- comcha+alarm (set certainty 90, reviewer )
- **If you don't know, leave unchanged**
Work in batches of 10, run the modify commands as you complete each batch.
**The goal is to increase certainty for species and annotate calltype**

Reference examples live in this skill's folder: `<common_name>/<calltype>/` with paired `.png`/`.wav`. 

Reference examples live in skill folder: `/call-library/<common_name>/<calltype>/` with paired `.png`/`.wav`.