refactor to get db omports out of utils, but still have failing tests, may need updating

quietlight

May 6, 2026, 11:00 PM

JZRF7OBJNERB4NIB37RSAF3ZK2A4RBWSCFV5OCRXZYVGPSNOWKTAC

Dependencies

• [2] AVQ66WO4 tools/ refactor
• [3] JAT3DXOL cyclo over 15
• [4] QVIGQOQZ more work on utils/ with glm
• [5] LQLC7S3A trying gemini: Inconsistent Standards in @utils/ refactoring
• [6] LBWQJEDH minor refactor and more tests for utils/
• [7] VYNOHQJW tidied up CLAUDE.md
• [8] RUVJ3V4N cyclo to 14 now
• [9] KZKLAINJ run out of space on nest, cleaned out
• [10] KLUEQ6X5 cyclo 21+
• [11] WKQ7LFTP refactor of utils/
• [12] 2P27XV3D fixed cyclo over 30

Change contents

• file addition: placeholders.go (----------)
  [3.1]
  + package utils

  +

  + import "strings"

  +

  + // Placeholders generates SQL placeholder string for IN clauses (e.g. "?, ?, ?")

  + func Placeholders(n int) string {

  + if n == 0 {

  + return ""

  + }

  + ph := make([]string, n)

  + for i := range ph {

  + ph[i] = "?"

  + }

  + return strings.Join(ph, ", ")

  + }
• edit in utils/mapping.go at line 9
  [3.89645]→[3.4802:4816](∅→∅)
  −

  − "skraak/db"
• replacement in utils/mapping.go at line 190
  [3.94145]→[3.910:1027](∅→∅)
  − query := `SELECT label FROM species WHERE label IN (` + db.Placeholders(len(speciesLabels)) + `) AND active = true`
  [3.94145]

  [3.1027]
  + query := `SELECT label FROM species WHERE label IN (` + Placeholders(len(speciesLabels)) + `) AND active = true`
• replacement in utils/mapping.go at line 235
  [3.95199]→[3.5149:5249](∅→∅)
  − WHERE s.label = ? AND ct.label IN (` + db.Placeholders(len(ctLabels)) + `) AND ct.active = true`
  [3.95199]

  [3.95296]
  + WHERE s.label = ? AND ct.label IN (` + Placeholders(len(ctLabels)) + `) AND ct.active = true`
• edit in utils/file_import.go at line 142
  [3.138642]

  [3.138642]
  + Exec(query string, args ...any) (sql.Result, error)
• file addition: deps_test.go (----------)
  [3.1]
  + package utils

  +

  + import (

  + "go/parser"

  + "go/token"

  + "os/exec"

  + "strings"

  + "testing"

  + )

  +

  + // TestPackageDependencies enforces the project's package dependency rules

  + // defined in CLAUDE.md. Packages may only import packages below them:

  + //

  + // cmd → tools, tui, utils, db

  + // tools → utils, db

  + // tui → tools, utils

  + // utils → (nothing — leaf package)

  + // db → utils

  + //

  + // Note: This test lives in utils/ only because it needs to be in some

  + // package. It validates rules for ALL project packages.

  + func TestPackageDependencies(t *testing.T) {

  + rules := map[string]map[string]bool{

  + // pkg → set of allowed skraak-internal imports

  + "skraak/cmd": {"skraak/tools": true, "skraak/tui": true, "skraak/utils": true, "skraak/db": true},

  + "skraak/tools": {"skraak/utils": true, "skraak/db": true},

  + "skraak/tui": {"skraak/tools": true, "skraak/utils": true},

  + "skraak/utils": {}, // leaf package — no skraak imports allowed

  + "skraak/db": {"skraak/utils": true},

  + }

  +

  + for pkg, allowed := range rules {

  + imports := getInternalImports(t, pkg)

  + for _, imp := range imports {

  + if !allowed[imp] {

  + t.Errorf("%s imports %s — not allowed by dependency rules", pkg, imp)

  + }

  + }

  + }

  + }

  +

  + // TestNoDirectDBImportInUtils checks that no file in utils/ imports skraak/db.

  + // This is a fast source-level check that doesn't require `go list`.

  + func TestNoDirectDBImportInUtils(t *testing.T) {

  + fset := token.NewFileSet()

  + pkgs, err := parser.ParseDir(fset, ".", nil, parser.ImportsOnly)

  + if err != nil {

  + t.Fatalf("parse utils/: %v", err)

  + }

  +

  + for _, pkg := range pkgs {

  + for filename, file := range pkg.Files {

  + for _, imp := range file.Imports {

  + path := strings.Trim(imp.Path.Value, `"`)

  + if path == "skraak/db" {

  + t.Errorf("%s: forbidden import of skraak/db (utils is the leaf package)", filename)

  + }

  + }

  + }

  + }

  + }

  +

  + // getInternalImports returns the skraak-internal imports for a package using `go list`.

  + func getInternalImports(t *testing.T, pkg string) []string {

  + t.Helper()

  + out, err := exec.Command("go", "list", "-f", "{{range .Imports}}{{.}}\n{{end}}", pkg).Output()

  + if err != nil {

  + t.Fatalf("go list %s: %v", pkg, err)

  + }

  +

  + var internal []string

  + for line := range strings.SplitSeq(string(out), "\n") {

  + line = strings.TrimSpace(line)

  + if strings.HasPrefix(line, "skraak/") || line == "skraak" {

  + internal = append(internal, line)

  + }

  + }

  + return internal

  + }
• edit in utils/cluster_import.go at line 10
  [3.171622]→[3.171622:171636](∅→∅)
  −

  − "skraak/db"
• replacement in utils/cluster_import.go at line 42
  [3.172945]→[3.172945:173013](∅→∅)
  − // ImportCluster imports all WAV files from a folder into a cluster
  [3.172945]

  [3.173013]
  + // ImportCluster imports all WAV files from a folder into a cluster.

  + // The caller must provide an open transaction via tx; this function does NOT

  + // commit or rollback — the caller owns the transaction lifecycle.
• replacement in utils/cluster_import.go at line 60
  [3.173596]→[3.173596:173639](∅→∅)
  − // 5. Batch insert in single transaction:
  [3.173596]

  [3.173639]
  + // 5. Batch insert using the provided transaction:
• edit in utils/cluster_import.go at line 65
  [3.173784]→[3.173784:173815](∅→∅)
  − // - All-or-nothing commit
• edit in utils/cluster_import.go at line 66
  [3.173848]→[3.173848:173985](∅→∅)
  − //

  − // Transaction behavior: ALL files succeed or ALL rollback

  − // This preserves cluster integrity (cluster = complete recording session)
• replacement in utils/cluster_import.go at line 67
  [3.174005]→[3.174005:174024](∅→∅)
  − database *sql.DB,
  [3.174005]

  [3.174024]
  + database DB,

  + tx *sql.Tx,
• replacement in utils/cluster_import.go at line 112
  [3.175051]→[3.175051:175082](∅→∅)
  − // Batch insert into database
  [3.175051]

  [3.175082]
  + // Batch insert into database using the provided transaction
• replacement in utils/cluster_import.go at line 114
  [3.175143]→[3.175143:175155](∅→∅)
  − database,
  [3.175143]

  [3.175155]
  + tx,
• replacement in utils/cluster_import.go at line 150
  [3.175951]→[3.175951:176034](∅→∅)
  − func GetLocationData(database *sql.DB, locationID string) (*LocationData, error) {
  [3.175951]

  [3.176034]
  + func GetLocationData(database DB, locationID string) (*LocationData, error) {
• replacement in utils/cluster_import.go at line 165
  [3.176410]→[3.176410:176489](∅→∅)
  − func EnsureClusterPath(database *sql.DB, clusterID, folderPath string) error {
  [3.176410]

  [3.176489]
  + func EnsureClusterPath(database DB, clusterID, folderPath string) error {
• replacement in utils/cluster_import.go at line 339
  [3.7188]→[3.7188:7206](∅→∅)
  − tx *db.LoggedTx,
  [3.7188]

  [3.2355]
  + tx *sql.Tx,
• replacement in utils/cluster_import.go at line 341
  [3.2382]→[3.7221:7270](∅→∅),[3.7221]→[3.7221:7270](∅→∅)
  − fileStmt, datasetStmt, mothStmt *db.LoggedStmt,
  [3.2382]

  [3.183040]
  + fileStmt, datasetStmt, mothStmt *sql.Stmt,
• replacement in utils/cluster_import.go at line 395
  [3.8605]→[3.8605:8689](∅→∅)
  − fileStmt *db.LoggedStmt

  − datasetStmt *db.LoggedStmt

  − mothStmt *db.LoggedStmt
  [3.8605]

  [3.8689]
  + fileStmt *sql.Stmt

  + datasetStmt *sql.Stmt

  + mothStmt *sql.Stmt
• replacement in utils/cluster_import.go at line 401
  [3.8771]→[3.8771:8859](∅→∅)
  − func prepareClusterStmts(ctx context.Context, tx *db.LoggedTx) (*clusterStmts, error) {
  [3.8771]

  [3.183436]
  + func prepareClusterStmts(ctx context.Context, tx *sql.Tx) (*clusterStmts, error) {
• replacement in utils/cluster_import.go at line 444
  [3.184971]→[3.9372:9454](∅→∅)
  − // insertClusterFiles inserts all file data into database in a single transaction
  [3.184971]

  [3.9454]
  + // insertClusterFiles inserts all file data into database using the provided transaction.

  + // The caller is responsible for committing or rolling back the transaction.
• replacement in utils/cluster_import.go at line 447
  [3.9479]→[3.9479:9498](∅→∅)
  − database *sql.DB,
  [3.9479]

  [3.2702]
  + tx *sql.Tx,
• edit in utils/cluster_import.go at line 452
  [3.9658]→[3.9658:9836](∅→∅)
  − tx, err := db.BeginLoggedTx(ctx, database, "import_audio_files")

  − if err != nil {

  − return 0, 0, nil, fmt.Errorf("failed to begin transaction: %w", err)

  − }

  − defer tx.Rollback()
• edit in utils/cluster_import.go at line 474
  [3.186466]→[3.186466:186470](∅→∅),[3.186470]→[3.10239:10276](∅→∅),[3.10276]→[3.186529:186601](∅→∅),[3.186529]→[3.186529:186601](∅→∅)
  − }

  −

  − if err := tx.Commit(); err != nil {

  − return 0, 0, errors, fmt.Errorf("transaction commit failed: %w", err)
• edit in utils/audiomoth_parser_test.go at line 4
  [3.194811]→[3.194811:194824](∅→∅)
  − "skraak/db"
• replacement in utils/audiomoth_parser_test.go at line 78
  [3.1341]→[3.1341:1450](∅→∅)
  − if result.Gain != db.GainMedium {

  − t.Errorf("Gain incorrect: got %s, want %s", result.Gain, db.GainMedium)
  [3.1341]

  [3.1450]
  + if result.Gain != GainMedium {

  + t.Errorf("Gain incorrect: got %s, want %s", result.Gain, GainMedium)
• replacement in utils/audiomoth_parser_test.go at line 118
  [3.2564]→[3.2564:2669](∅→∅)
  − if result.Gain != db.GainHigh {

  − t.Errorf("Gain incorrect: got %s, want %s", result.Gain, db.GainHigh)
  [3.2564]

  [3.2669]
  + if result.Gain != GainHigh {

  + t.Errorf("Gain incorrect: got %s, want %s", result.Gain, GainHigh)
• replacement in utils/audiomoth_parser_test.go at line 132
  [3.2955]→[3.2955:2979](∅→∅)
  − expected db.GainLevel
  [3.2955]

  [3.2979]
  + expected GainLevel
• replacement in utils/audiomoth_parser_test.go at line 134
  [3.2983]→[3.2983:3134](∅→∅)
  − {"low", db.GainLow},

  − {"low-medium", db.GainLowMedium},

  − {"medium", db.GainMedium},

  − {"medium-high", db.GainMediumHigh},

  − {"high", db.GainHigh},
  [3.2983]

  [3.3134]
  + {"low", GainLow},

  + {"low-medium", GainLowMedium},

  + {"medium", GainMedium},

  + {"medium-high", GainMediumHigh},

  + {"high", GainHigh},
• replacement in utils/audiomoth_parser_test.go at line 180
  [3.200732]→[3.200732:200756](∅→∅)
  − expected db.GainLevel
  [3.200732]

  [3.200756]
  + expected GainLevel
• replacement in utils/audiomoth_parser_test.go at line 183
  [3.200776]→[3.200776:201026](∅→∅)
  − {"low", db.GainLow, false},

  − {"LOW", db.GainLow, false},

  − {" low ", db.GainLow, false},

  − {"low-medium", db.GainLowMedium, false},

  − {"medium", db.GainMedium, false},

  − {"medium-high", db.GainMediumHigh, false},

  − {"high", db.GainHigh, false},
  [3.200776]

  [3.201026]
  + {"low", GainLow, false},

  + {"LOW", GainLow, false},

  + {" low ", GainLow, false},

  + {"low-medium", GainLowMedium, false},

  + {"medium", GainMedium, false},

  + {"medium-high", GainMediumHigh, false},

  + {"high", GainHigh, false},
• replacement in utils/audiomoth_parser_test.go at line 316
  [3.205416]→[3.205416:205540](∅→∅)
  − if result.Gain != db.GainMedium {

  − t.Errorf("Gain should be normalized: got %s, want %s", result.Gain, db.GainMedium)
  [3.205416]

  [3.205540]
  + if result.Gain != GainMedium {

  + t.Errorf("Gain should be normalized: got %s, want %s", result.Gain, GainMedium)
• edit in utils/audiomoth_parser.go at line 9
  [3.206959]

  [3.206959]
  + )

  +

  + // GainLevel represents the gain_level enum for AudioMoth recordings

  + type GainLevel string
• replacement in utils/audiomoth_parser.go at line 14
  [3.206960]→[3.206960:206973](∅→∅)
  − "skraak/db"
  [3.206960]

  [3.206973]
  + // AudioMoth gain level enum constants

  + const (

  + GainLow GainLevel = "low"

  + GainLowMedium GainLevel = "low-medium"

  + GainMedium GainLevel = "medium"

  + GainMediumHigh GainLevel = "medium-high"

  + GainHigh GainLevel = "high"
• replacement in utils/audiomoth_parser.go at line 27
  [3.207112]→[3.207112:207137](∅→∅)
  − Gain db.GainLevel
  [3.207112]

  [3.207137]
  + Gain GainLevel
• replacement in utils/audiomoth_parser.go at line 214
  [3.213405]→[3.213405:213465](∅→∅)
  − func parseGainLevel(gainStr string) (db.GainLevel, error) {
  [3.213405]

  [3.213465]
  + func parseGainLevel(gainStr string) (GainLevel, error) {
• replacement in utils/audiomoth_parser.go at line 219
  [3.213552]→[3.213552:213577](∅→∅)
  − return db.GainLow, nil
  [3.213552]

  [3.213577]
  + return GainLow, nil
• replacement in utils/audiomoth_parser.go at line 221
  [3.213597]→[3.213597:213628](∅→∅)
  − return db.GainLowMedium, nil
  [3.213597]

  [3.213628]
  + return GainLowMedium, nil
• replacement in utils/audiomoth_parser.go at line 223
  [3.213644]→[3.213644:213672](∅→∅)
  − return db.GainMedium, nil
  [3.213644]

  [3.213672]
  + return GainMedium, nil
• replacement in utils/audiomoth_parser.go at line 225
  [3.213693]→[3.213693:213725](∅→∅)
  − return db.GainMediumHigh, nil
  [3.213693]

  [3.213725]
  + return GainMediumHigh, nil
• replacement in utils/audiomoth_parser.go at line 227
  [3.213739]→[3.213739:213765](∅→∅)
  − return db.GainHigh, nil
  [3.213739]

  [3.213765]
  + return GainHigh, nil
• replacement in tools/import_files.go at line 74
  [3.347513]→[3.347513:347592](∅→∅)
  − clusterOutput, err := utils.ImportCluster(database, utils.ClusterImportInput{
  [3.347513]

  [3.347592]
  + tx, err := db.BeginLoggedTx(ctx, database, "import_audio_files")

  + if err != nil {

  + return output, fmt.Errorf("failed to begin transaction: %w", err)

  + }

  +

  + clusterOutput, err := utils.ImportCluster(database, tx.UnderlyingTx(), utils.ClusterImportInput{
• edit in tools/import_files.go at line 87
  [3.347764]

  [3.347764]
  + tx.Rollback()
• edit in tools/import_files.go at line 89
  [3.347826]

  [3.347826]
  + }

  +

  + if err := tx.Commit(); err != nil {

  + return output, fmt.Errorf("transaction commit failed: %w", err)
• replacement in tools/bulk_file_import.go at line 466
  [3.637481]→[3.637481:637560](∅→∅)
  − clusterOutput, err := utils.ImportCluster(database, utils.ClusterImportInput{
  [3.637481]

  [3.637560]
  +

  + ctx := context.Background()

  + tx, err := db.BeginLoggedTx(ctx, database, "import_audio_files")

  + if err != nil {

  + return nil, fmt.Errorf("failed to begin transaction: %w", err)

  + }

  +

  + clusterOutput, err := utils.ImportCluster(database, tx.UnderlyingTx(), utils.ClusterImportInput{
• edit in tools/bulk_file_import.go at line 481
  [3.637703]

  [3.637703]
  + tx.Rollback()
• edit in tools/bulk_file_import.go at line 485
  [3.637725]

  [3.637725]
  + if err := tx.Commit(); err != nil {

  + return nil, fmt.Errorf("transaction commit failed: %w", err)

  + }

  +
• replacement in db/utils.go at line 3
  [3.2511]→[3.2511:2528](∅→∅)
  − import "strings"
  [3.2511]

  [3.2528]
  + import (

  + "skraak/utils"

  + )
• replacement in db/utils.go at line 7
  [3.2529]→[3.2529:2593](∅→∅)
  − // Placeholders generates SQL placeholder string for IN clauses
  [3.2529]

  [3.2593]
  + // Placeholders generates SQL placeholder string for IN clauses.

  + // Delegates to utils.Placeholders.
• replacement in db/utils.go at line 10
  [3.2627]→[3.2627:2749](∅→∅)
  − if n == 0 {

  − return ""

  − }

  − ph := make([]string, n)

  − for i := range ph {

  − ph[i] = "?"

  − }

  − return strings.Join(ph, ", ")
  [3.2627]

  [3.2749]
  + return utils.Placeholders(n)
• edit in db/types.go at line 6
  [3.790984]

  [3.790984]
  +

  + "skraak/utils"
• replacement in db/types.go at line 188
  [3.797689]→[3.797689:797780](∅→∅)
  − // GainLevel represents the gain_level enum for AudioMoth recordings

  − type GainLevel string
  [3.797689]

  [3.797780]
  + // GainLevel is re-exported from utils for backward compatibility.

  + type GainLevel = utils.GainLevel
• replacement in db/types.go at line 191
  [3.797781]→[3.797781:797820](∅→∅)
  − // AudioMoth gain level enum constants
  [3.797781]

  [3.797820]
  + // Gain level constants re-exported from utils.
• replacement in db/types.go at line 193
  [3.797828]→[3.797828:798017](∅→∅)
  − GainLow GainLevel = "low"

  − GainLowMedium GainLevel = "low-medium"

  − GainMedium GainLevel = "medium"

  − GainMediumHigh GainLevel = "medium-high"

  − GainHigh GainLevel = "high"
  [3.797828]

  [3.798017]
  + GainLow = utils.GainLow

  + GainLowMedium = utils.GainLowMedium

  + GainMedium = utils.GainMedium

  + GainMediumHigh = utils.GainMediumHigh

  + GainHigh = utils.GainHigh
• edit in db/tx_logger.go at line 133
  [3.845566]

  [3.845566]
  + // UnderlyingTx returns the underlying *sql.Tx for use by packages that

  + // should not import db (e.g. utils). Prefer using LoggedTx methods directly

  + // when event logging is desired.

  + func (l *LoggedTx) UnderlyingTx() *sql.Tx {

  + return l.tx

  + }

  +
• replacement in CLAUDE.md at line 13
  [3.1195619]→[3.1195619:1195722](∅→∅)
  − **Simple rule:** If called by `cmd/`, it goes in `tools/`. If called by `tools/`, it goes in `utils/`.
  [3.1195619]

  [3.1195722]
  + **Dependency rule:** packages may only import packages below them in this list:
• replacement in CLAUDE.md at line 15
  [3.1195723]→[3.1195723:1195803](∅→∅)
  − - **`utils/`** - Reusable helpers (no MCP types, no `*Input`/`*Output` structs)
  [3.1195723]

  [3.165]
  + - **`cmd/*.go`** - CLI commands (parse flags, call tools, print JSON)
• replacement in CLAUDE.md at line 17
  [3.240]→[3.1195946:1196016](∅→∅),[3.1195946]→[3.1195946:1196016](∅→∅)
  − - **`cmd/*.go`** - CLI commands (parse flags, call tools, print JSON)
  [3.240]

  [3.1196016]
  + - **`tui/`** - TUI (interactive classify UI)

  + - **`utils/`** - Reusable helpers (no MCP types, no `*Input`/`*Output` structs, no `db` import)

  + - **`db/`** - Database connection, types, transactions (may import `utils/`)

  +

  + `utils/` is the leaf package — it must not import `cmd/`, `tools/`, `tui/`, or `db/`.
• replacement in CLAUDE.md at line 33
  [3.360]→[3.1196402:1196489](∅→∅),[3.1196402]→[3.1196402:1196489](∅→∅)
  − utils/*.go → Reusable helpers

  − db/ → Database connection + types
  [3.360]

  [3.1196489]
  + tui/ → Interactive classify UI

  + db/ → Database connection + types + transactions

  + utils/*.go → Reusable helpers (leaf package, no db import)
• edit in CHANGELOG.md at line 4
  [3.1198010]

  [2.30288]
  +

  + ## [2026-05-07] Remove utils → db package dependency

  +

  + Three changes to eliminate the `utils → db` import edge, restoring clean layering

  + (`cmd → tools → utils` and `cmd → db`, with no upward dependency from utils):

  +

  + - **Moved `Placeholders()` from `db/` to `utils/`**: It's a pure string function

  + (generates `?, ?, ?` for SQL IN clauses) with no database dependency. `db.Placeholders`

  + now delegates to `utils.Placeholders` for backward compatibility.
• edit in CHANGELOG.md at line 14
  [2.30289]

  [2.30289]
  + - **Moved `GainLevel` type and constants from `db/` to `utils/`**: It's a string enum

  + that naturally belongs next to `AudioMothData` in the audiomoth parser. `db.GainLevel`

  + is now a type alias (`type GainLevel = utils.GainLevel`) and the constants are

  + re-exported for backward compatibility.

  +

  + - **Refactored `ImportCluster` to accept `*sql.Tx` instead of managing its own

  + transaction**: The caller (tools/) now opens the `LoggedTx`, passes the underlying

  + `*sql.Tx` via `UnderlyingTx()`, and manages commit/rollback. This removes

  + `db.BeginLoggedTx`, `db.LoggedTx`, and `db.LoggedStmt` from utils entirely.

  + Added `UnderlyingTx()` method to `db.LoggedTx` to expose the raw `*sql.Tx`.

  +

  + - **Extended `DB` interface** with `Exec()`: `EnsureClusterPath` needed `Exec` which

  + was missing from the original `DB` interface. Both `*sql.DB` and `*sql.Tx` satisfy

  + the updated interface.

  +

  + Result: `utils/` no longer imports `skraak/db`. The dependency now flows correctly:

  + `db → utils` (for Placeholders and GainLevel re-exports).

  +