release 1.0.0: cost control, 41 tools, 29 skills, custom commands, auto-load

src/skills-md/data.md

@@ -0,0 +1,40 @@
+---
+name: data
+description: Process, validate, or transform data. Use when parsing files, cleaning datasets, designing a data pipeline, or debugging a transform that produces wrong output.
+---
+
+# Data
+
+Bad data fails silently and far away from where it entered. Validate at the boundary, keep the
+raw, and make every transform checkable.
+
+## Validate at the boundary
+
+Parse and validate when data enters the system, not when it is used. A schema check at the edge
+turns a corrupt record into a clear rejection; skipping it turns the same record into a wrong
+answer three layers later. Reject loudly, with the record and the reason — never coerce and
+carry on.
+
+## Keep the raw
+
+Store the untransformed input alongside the derived. When a transform turns out to be wrong,
+the raw lets you recompute; without it, the information is gone. Derived data is rebuildable;
+source data is not.
+
+## Transformations are pure and tested
+
+A transform takes input and returns output with no hidden state, so it can be tested on a
+fixture and re-run safely. Test the edge cases that actually occur in data: the empty field,
+the wrong type, the unexpected null, the duplicate, the encoding that is not UTF-8.
+
+## Duplicates, nulls, and ranges are the usual corruption
+
+Check for: unexpected duplicates on a key, nulls where a value is required, values outside a
+sane range (a negative age, a date in the future), and referential breaks (an id pointing at
+nothing). These four catch most real-world data problems before they reach a report.
+
+## Idempotent pipelines
+
+A step that can be re-run without duplicating or corrupting its output is a step you can retry
+after a failure. Key on a stable id and upsert rather than blind-insert. A pipeline you cannot
+safely re-run is a pipeline you will one day have to fix by hand at 2am.