How I Built a File-Timestamp-Based Feedback Loop to Enforce AI Output Quality

The problem: AI outputs are probabilistic, and prompts have a ceiling

LLMs produce probabilistic outputs. No matter how good your prompt is, edge cases will fail — hallucinations, omissions, format drift, and confident-sounding rationalizations that don't hold up.

I noticed this while using Claude Code daily: the AI would say "done" but the file wasn't written. It would claim "logs updated" but the timestamps were three days old. The AI wasn't lying — probabilistic output is inherently unstable.

Pure prompt engineering is fighting probability with probability. The ultimate defense must be deterministic, mechanical checks.

The solution: 4 of 5 steps are scripts. Only 1 requires AI.

I built an agent configuration system with a closed-loop feedback mechanism:

self-model.md (current self-cognition)
    ↓
Session executes (AI works based on config)
    ↓
Growth data accumulates (what worked, what failed)
    ↓
quality-gate.py detects staleness (file timestamps + exit codes, pure Python)
    ↓
Writes .self-model-stale flag to disk
    ↓
Next startup: health-check.py detects flag → triggers AI to regenerate self-model
    ↓ (loop closes)

4 steps are mechanical scripts: file timestamp checks, exit code gates, JSONL audit trails, flag file I/O.
1 step requires AI: content regeneration — synthesizing accumulated growth data into updated self-cognition.

Machines do the checking. Humans and AI do the judging. This isn't philosophy — it's engineering.

Key design decisions

1. Zero dependencies, stdlib only

Every script uses only Python's standard library. A quality check tool can't introduce new dependency risks.

2. Dual-layer gate: soft reminder + hard block

• Process layer (soft): Rule execution rate low? Remind, but don't block.
• Output layer (hard): Learning logs not updated? Exit 2, hard block. Delivery must be complete.

The boundary isn't importance — it's "can this be fixed later?"

3. Filesystem as database

No vector databases. No cloud services. All identity data, growth logs, and audit records are local Markdown + JSON files. Git-auditable, offline-capable, fully self-sovereign.

External validation: submitting to a 100K-star project

I extracted one module (delivery-gate) from my personal system and submitted it to ECC (100K+ stars).

Result: maintainer daltino reviewed and approved it with praise. Maintainer affaan-m personally merged two follow-up PRs. A 200-line Python script went through 4 rounds of community bot review + human maintainer review, catching 9 issues I hadn't found in self-testing.

Open-source community review is the best free QA you'll ever get. This became my "open-source flywheel" methodology: build for yourself → extract module → find a community gap → submit PR → merge back into your own system.

If you want to do something similar

1. Dogfood it first. My system ran through 50+ real sessions before I submitted anything.
2. Scripts, not prompts. If you can check it with an if/else in Python, don't describe it in natural language.
3. Small PRs win. For large projects, 100-300 lines is the sweet spot for maintainer review.
4. Use the gap-filling template. "This repo has X and Y. But there is no Z. This PR fills that gap."

The real takeaway

I'm a junior-year undergrad. My raw coding speed probably doesn't beat CS majors who eat LeetCode for breakfast. But I've learned one thing that matters more:

The core competency of the AI era isn't typing speed — it's knowing what to let AI do, and what must be enforced with deterministic rules.