🧠 CrumbVector

Semantic Code Search with Trust Layer

Certified Code in Vector Space - nur vertrauenswürdiger Code wird indexiert!

🎯 Was ist das?

┌─────────────────────────────────────────────────────────────────────────────┐
│                                                                             │
│   CERTIFIED REPO  →  CHUNKING  →  EMBEDDING  →  VECTOR DB  →  SEARCH      │
│                                                                             │
│   ┌─────────┐       ┌───────┐     ┌───────┐     ┌───────┐     ┌────────┐  │
│   │ CKL ✓   │  ──►  │ Code  │ ──► │ Vector│ ──► │ChromaDB│ ◄──│ Query  │  │
│   │ Trust:4 │       │ Chunks│     │ [768] │     │ Local  │    │ "beat" │  │
│   └─────────┘       └───────┘     └───────┘     └───────┘     └────────┘  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

CrumbVector ermöglicht:
- 🔍 Semantische Suche in Code-Repositories
- 🔐 Trust-Layer: Nur zertifizierte Repos werden indexiert
- 🏠 Komplett lokal: Ollama + ChromaDB (keine Cloud!)
- 🎵 Optimiert für Strudel/Music Patterns

🚀 Quickstart

# 1. Setup
cd crumbvector
./setup.sh

# 2. Ollama starten (falls nicht läuft)
ollama serve &
ollama pull nomic-embed-text

# 3. CrumbMidi indexieren
./crumbvector.py ingest ../crumbmidi --local

# 4. Suchen!
./crumbvector.py search "one drop reggae beat"
./crumbvector.py search "filter house bass" --genre house
./crumbvector.py search "polyrhythm african" --lang strudel

📦 Dependencies

# Python Packages
pip install chromadb requests pyyaml --break-system-packages

# Ollama (für lokale Embeddings)
# macOS:
brew install ollama

# Linux:
curl -fsSL https://ollama.com/install.sh | sh

# Embedding Model
ollama pull nomic-embed-text

🔐 Trust System

Nur Repos mit gültiger .crumbvector/trust.yaml werden indexiert:

# .crumbvector/trust.yaml
certification:
  status: verified
  license: CKL-1.0
  verified_by: crumbforest
  verified_at: 2026-01-13

trust_level: 4  # 1-5 Scale

allowed_for:
  - teaching
  - code_completion
  - pattern_search
  - remix

Trust Levels

💻 CLI Commands

Ingest

# Lokales Verzeichnis
./crumbvector.py ingest ./mein-repo --local

# Git Repository
./crumbvector.py ingest https://github.com/user/repo

# Mit Mindest-Trust
./crumbvector.py ingest ./repo --local --min-trust 3

# In andere Collection
./crumbvector.py ingest ./repo --local --collection patterns

Search

# Einfache Suche
./crumbvector.py search "wie mache ich einen beat"

# Mit Filtern
./crumbvector.py search "bass line" --genre reggae
./crumbvector.py search "drum pattern" --lang strudel
./crumbvector.py search "euclid rhythm" --genre baobab

# Mehr Ergebnisse
./crumbvector.py search "filter sweep" -n 10

🎵 Beispiel: CrumbMidi Suche

$ ./crumbvector.py search "one drop beat" --genre reggae

🔍 CrumbVector - Searching
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
   Query: one drop beat
   Genre: reggae

━━━ Result 1 (distance: 0.234) ━━━
📁 patterns/REGGAE_DUB_75BPM.js
🏷️  strudel | pattern | reggae | ONE DROP
🔐 Trust: 4 | CKL-1.0

let oneDrop = stack(
  sound("~ ~ bd ~").gain(1.1).lpf(150).room(0.2),
  sound("~ ~ rim ~").gain(0.7).hpf(800).room(0.15)
)
...

📁 Architektur

crumbvector/
├── crumbvector.py      # Main script
├── setup.sh            # Installation helper
├── CONCEPT.md          # Detailed architecture docs
├── README.md           # This file
└── crumbvector_db/     # ChromaDB storage (auto-created)

🔄 Pipeline

1. INGEST
   ├── Clone/Read Repository
   ├── Verify Trust (.crumbvector/trust.yaml)
   ├── Check License
   └── Extract Metadata

2. CHUNK
   ├── Strudel: By sections & let/const patterns
   ├── Python: By functions & classes
   ├── Bash: By functions
   └── Markdown: By headers

3. EMBED (Ollama)
   ├── Prepare text with metadata context
   ├── Generate vector (nomic-embed-text)
   └── 768-dimensional embeddings

4. STORE (ChromaDB)
   ├── Vectors + Documents + Metadata
   ├── Persistent local storage
   └── Fast similarity search

5. QUERY
   ├── Natural language input
   ├── Convert to embedding
   ├── Similarity search with filters
   └── Return ranked results

🎛️ Chunking Strategien

Strudel Patterns (.js)

// Erkennt Sections:
// ═══════════════════════════════════════════════════════════════
// SECTION TITLE
// ═══════════════════════════════════════════════════════════════

// Und Pattern-Definitionen:
let patternName = stack(...)
const anotherPattern = note(...)

Python

# Chunked by:
def function_name():    # → function chunk
    ...

class ClassName:        # → class chunk
    ...

Markdown

# Header 1        → Section chunk
## Header 2       → Section chunk
### Header 3      → Section chunk

🦊 Integration mit Crumbforest

┌─────────────────────────────────────────────────────────────────────────────┐
│                                                                             │
│   CrumbMidi (Patterns)                                                      │
│        │                                                                    │
│        ▼                                                                    │
│   CrumbVector (Index)  ◄──── Query: "wie mache ich house drums?"           │
│        │                                                                    │
│        ▼                                                                    │
│   ┌─────────────────────────────────────────────────────────────┐          │
│   │ Result: OLDSCHOOL_HOUSE_128BPM.js                           │          │
│   │ "sound('bd bd bd bd')" + "sound('~ hh ~ hh')"               │          │
│   │ Trust: 4 | License: CKL                                     │          │
│   └─────────────────────────────────────────────────────────────┘          │
│        │                                                                    │
│        ▼                                                                    │
│   Strudel (Play!)                                                          │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

🔮 Roadmap

• [ ] FastAPI Server (crumbvector serve)
• [ ] Web UI für Suche
• [ ] VSCode Extension
• [ ] Multi-Repo Support
• [ ] Trust Verification via Signatures
• [ ] Code Completion Integration
• [ ] Strudel REPL Integration

🦊 FunkFox says

"Nicht jeder Code verdient Vertrauen.
 Nicht jede Antwort ist richtig.

 CrumbVector gibt dir:
 ✓ Verified Code
 ✓ Licensed Content
 ✓ No Hallucination

 Code finden. Code verstehen. Code vertrauen."

#crumbvector #rag #semantic-search #chromadb #ollama #trust

🧠🔐📚