# Changelog
All notable changes to the Skraak MCP Server are documented here.
## [2026-02-11] CLI Refactoring — Two-Layer Architecture
**Major refactoring: Separated core logic from MCP types, added CLI commands**
**Problem:** All tool functions were tightly coupled to MCP SDK types (`*mcp.CallToolRequest`, `*mcp.CallToolResult`). This meant functionality could only be invoked via MCP protocol — no CLI access for power users.
**Solution:** Two-layer architecture separating core logic from MCP adapters.
**Created:**
- `cmd/mcp.go` — MCP server setup + 10 thin adapter wrappers (~3 lines each)
- `cmd/import.go` — `skraak import bulk` CLI command with flag parsing
- `cmd/sql.go` — `skraak sql` CLI command for ad-hoc queries
**Modified (mechanical, all tools/):**
- Removed `*mcp.CallToolRequest` parameter (was never used — `req` always ignored)
- Removed `*mcp.CallToolResult` from returns (was always empty `&mcp.CallToolResult{}`)
- Removed `import "github.com/modelcontextprotocol/go-sdk/mcp"` from all tool files
- Updated test files (`integration_test.go`, `pattern_test.go`) to match new signatures
- Updated `main.go` to pure dispatcher: `mcp | import | sql`
**Architecture:**
```
main.go              → pure dispatcher
cmd/mcp.go           → MCP server + adapter wrappers (ONLY file importing mcp SDK)
cmd/import.go        → CLI: skraak import bulk --db ... --dataset ... --csv ... --log ...
cmd/sql.go           → CLI: skraak sql --db ... "SELECT ..."
tools/*.go           → core logic, NO mcp dependency (plain Go structs in/out)
utils/, db/, etc.    → unchanged
```
**Benefits:**
- CLI access for power users without MCP
- Token savings (CLI avoids MCP protocol overhead)
- Code sharing between CLI and MCP
- MCP SDK contained to one file
- All tests pass
---
## [2026-02-10] Bulk File Import Cluster Assignment Bug Fix
**Critical Bug Fix: Files now correctly distributed across multiple clusters for same location**
**Problem:** When the same location appeared multiple times in the CSV with different date ranges, all files ended up in the last cluster created instead of being distributed across their respective clusters.
**Root Cause:** The `clusterIDMap` used only `LocationID` as the key, causing each new cluster for the same location to overwrite the previous one in the map.
**Solution:** Changed map key from `LocationID` to composite key `LocationID|DateRange`.
**Modified:**
- `tools/bulk_file_import.go` (lines 125, 171-172, 183-184)
**Impact:**
- Data integrity restored
- Multiple date ranges per location now works correctly
- Simple 3-line fix, backwards compatible
---
## [2026-02-07] File Modification Time Fallback
**Enhancement: Added file modification time as third timestamp fallback**
**Problem:** Small clusters (1-2 files) failed variance-based filename disambiguation because the algorithm needs multiple samples to determine date format (YYYYMMDD vs YYMMDD vs DDMMYY).
**Timestamp Resolution Order:**
```
1. AudioMoth comment → timestamp
2. Filename parsing → timestamp
3. File modification time → timestamp (NEW!)
4. FAIL (skip file with error)
```
**Modified:**
- `utils/cluster_import.go` - Added FileModTime fallback in `batchProcessFiles()`
**Benefits:**
- Fewer failures in small clusters
- No performance impact
- Backwards compatible
- Simple 10-line change
---
## [2026-02-07] Cluster Import Logic Extraction
**Major refactoring: Extracted shared cluster import logic into utils module**
**Key Insight:** A cluster is the atomic unit of import (one SD card / one recording session / one folder).
**Created:**
- `utils/cluster_import.go` (553 lines) - Single source of truth for cluster imports
  - `ImportCluster()` - Main entry point
  - `scanClusterFiles()` - Recursive WAV file scanning
  - `batchProcessFiles()` - Batch processing with variance-based parsing
  - `insertClusterFiles()` - Transactional insertion
**Modified:**
- `tools/import_files.go` - 75% code reduction (650 lines → 161 lines)
- `tools/bulk_file_import.go` - Bug fixes:
  - **CRITICAL BUG FIXED:** Now inserts into `file_dataset` table (was missing!)
  - **CRITICAL BUG FIXED:** Now inserts into `moth_metadata` table (was missing!)
**Benefits:**
- Bug fixed: 68,043 orphaned files found in test database
- ~500 lines of duplicated code eliminated
- Single source of truth for all import logic
---
## [2026-02-06] Tool Consolidation
**Consolidated 8 write/update tools → 4 create_or_update tools**
**Deleted:**
- 8 separate create/update tool files
**Added:**
- `tools/dataset.go` - `create_or_update_dataset`
- `tools/location.go` - `create_or_update_location`
- `tools/cluster.go` - `create_or_update_cluster`
- `tools/pattern.go` - `create_or_update_pattern`
**Design:**
- Omit `id` field → CREATE mode (generates nanoid)
- Provide `id` field → UPDATE mode (verifies exists)
**Benefits:**
- Tool count: 14 → 10
- ~31% less code (~320 lines removed)
- Shared validation logic
---
## [2026-02-06] Test Script Consolidation
**Rationalized and consolidated shell test scripts**
**Removed redundant scripts:**
- 6 incomplete/redundant test scripts
**Current test suite (8 scripts):**
1. `get_time.sh` - Time tool
2. `test_sql.sh` - SQL query tool
3. `test_tools.sh` - All create_or_update tools
4. `test_import_file.sh` - Single file import
5. `test_import_selections.sh` - ML selection import
6. `test_bulk_import.sh` - Bulk CSV import
7. `test_resources_prompts.sh` - Resources/prompts
8. `test_all_prompts.sh` - All 6 prompts
---
## [2026-02-06] Bulk File Import Tool
**New Feature: CSV-based bulk import across multiple locations and clusters**
**Added:**
- `tools/bulk_file_import.go` - CSV-based bulk import (~500 lines)
**Features:**
- CSV-driven import for multiple locations
- Auto-cluster creation
- Progress logging to file
- Summary statistics
**CSV Format:**
```csv
location_name,location_id,directory_path,date_range,sample_rate,file_count
Site A,loc123456789,/path/to/recordings,2024-01,48000,150
```
---
## [2026-02-02] Single File Import Tool
**New Feature: Import individual WAV files**
**Added:**
- `tools/import_file.go` - Single file import implementation (~300 lines)
**Features:**
- Import one WAV file at a time with detailed feedback
- Same processing pipeline as batch import
- Duplicate detection with `is_duplicate` flag
- Atomic operation (succeeds completely or fails)
---
## [2026-01-29] ML Selection Import Tool
**New Feature: Import ML-detected kiwi call selections from folder structure**
**Added:**
- `utils/selection_parser.go` - Selection parsing utilities
- `utils/selection_parser_test.go` - 34 test cases
- `tools/import_ml_selections.go` - MCP tool (~1050 lines)
**Features:**
- Folder structure: `Clips_{filter_name}_{date}/Species/CallType/*.wav+.png`
- Two-pass file matching (exact, then fuzzy)
- Comprehensive validation
- Transactional import
---
## [2026-01-28] Comprehensive Go Unit Testing
**Added comprehensive unit test suite**
**Added:**
- `utils/astronomical_test.go` - 11 test cases
- `utils/audiomoth_parser_test.go` - 36 test cases
- `utils/filename_parser_test.go` - 60 test cases
- `utils/wav_metadata_test.go` - 22 test cases
- `utils/xxh64_test.go` - 6 test cases
**Coverage:**
- 170+ tests total
- 91.5% code coverage
---
## [2026-01-26] Generic SQL Tool + Codebase Rationalization
**Major architectural change: Replaced 6 specialized tools with generic SQL**
**Deleted:**
- 6 specialized query tools (datasets, locations, clusters, files)
- 2 obsolete test scripts
**Added:**
- `tools/sql.go` - Generic `execute_sql` tool (~200 lines)
- `shell_scripts/test_sql.sh` - Comprehensive SQL test suite
**Modified:**
- `prompts/examples.go` - Rewritten to teach SQL patterns
**Benefits:**
- Full SQL expressiveness (JOINs, aggregates, CTEs)
- Infinite query possibilities vs 6 fixed queries
- More aligned with MCP philosophy
- Smaller codebase (2 tools instead of 8)
**Security:**
- Database read-only
- Validation blocks write operations
- Parameterized queries prevent SQL injection
- Row limits prevent overwhelming responses
---
## [2026-01-26] Shell Scripts Organization
**Reorganized all shell scripts into `shell_scripts/` directory**
- Keeps project root clean
- All scripts updated with correct relative paths