Spike S4: JSONL discovery + sessions registry — GO
99.8% of 1318 real transcripts (449 MB) yield sessionId+cwd in 1.37s (head+tail reads only). Munge cwd→dir validated on 100% of files. Registry pid/procStart liveness check validated 3/3. 12 line types inventoried, ai-title is the best title source.
This commit is contained in:
21
spikes/s4-discovery/VERDICT.md
Normal file
21
spikes/s4-discovery/VERDICT.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Spike S4 — Découverte JSONL + registre — VERDICT : ✅ GO
|
||||
|
||||
Exécuté le 2026-06-11 sur la machine de Johan (CLI 2.1.170/2.1.173 actifs, transcripts 2.1.139 → 2.1.173). Script : `scan.mjs`, résultats bruts : `result.json`.
|
||||
|
||||
## Critères Go
|
||||
|
||||
| Critère | Cible | Mesuré | Verdict |
|
||||
|---|---|---|---|
|
||||
| JSONL exploitables (sessionId + cwd) | ≥ 95 % | **99,8 %** (1316/1318) | ✅ |
|
||||
| Temps de scan (tête 256 Ko + queue 64 Ko) | < 3 s | **1,37 s** (1318 fichiers, 449 Mo) | ✅ |
|
||||
| Crash sur fichier malformé | 0 | **0** (parseur tolérant ligne à ligne) | ✅ |
|
||||
| Validation pid/procStart du registre | fiable | **3/3** entrées : pid vivant + procStart == champ 22 de /proc/pid/stat | ✅ |
|
||||
|
||||
## Enseignements pour le claude-adapter
|
||||
|
||||
1. **Les 2 seuls échecs** sont des fichiers ne contenant que des lignes `queue-operation` (pas de vraies sessions) → à classer « non-session », pas en erreur.
|
||||
2. **Munge cwd → dossier validé à 100 %** (1316/1316) : `cwd.replace(/[^A-Za-z0-9]/g, '-')` reproduit exactement le nom du dossier projet. Le lookup direct worktree → dossier est fiable en O(1) ; l'inversion reste interdite (ambiguë), le `cwd` se lit dans le contenu.
|
||||
3. **12 types de lignes observés** : `user`, `last-prompt`, `attachment`, `assistant`, `queue-operation`, `ai-title`, `file-history-snapshot`, `system`, `permission-mode`, `agent-name`, `mode`, `bridge-session`. `ai-title` (470 fichiers) est une source de titre supérieure au `slug` → priorité titre : `ai-title` > `summary` > `last-prompt` > 1er prompt user.
|
||||
4. **22 versions du CLI** dans les transcripts → base de fixtures multi-versions toute prête pour les tests du parseur.
|
||||
5. **Registre `~/.claude/sessions`** : statuts observés sur 2 jours : `busy`, `idle`, `waiting` (+ `waitingFor: "permission prompt"`). `kind: interactive`, `peerProtocol: 1`. Les 3 entrées actuelles sont toutes valides (pas de fichier stale en ce moment) — le comportement post-kill -9 sera couvert par S1.
|
||||
6. **Rattachement** : 573 sessions sous `~/WebstormProjects` (dont worktrees `*-wt-*` avec leur propre dossier projet, confirmé), 743 hors périmètre (dont 323 sur la racine WebstormProjects elle-même) → la vue « sessions hors worktree connu » est nécessaire dès le MVP.
|
||||
141
spikes/s4-discovery/result.json
Normal file
141
spikes/s4-discovery/result.json
Normal file
@@ -0,0 +1,141 @@
|
||||
{
|
||||
"scan": {
|
||||
"files": 1318,
|
||||
"bytes": 470368918,
|
||||
"ok": 1316,
|
||||
"noCwd": 2,
|
||||
"noSessionId": 0,
|
||||
"empty": 0,
|
||||
"errors": 0,
|
||||
"mb": "449",
|
||||
"scanMs": "1368",
|
||||
"okPct": "99.8 %"
|
||||
},
|
||||
"mungeValidation": {
|
||||
"mungeMatch": 1316,
|
||||
"mungeMismatch": 0,
|
||||
"mismatchSamples": []
|
||||
},
|
||||
"lineTypes": {
|
||||
"user": 1315,
|
||||
"last-prompt": 1313,
|
||||
"attachment": 1308,
|
||||
"assistant": 1204,
|
||||
"queue-operation": 1001,
|
||||
"ai-title": 470,
|
||||
"file-history-snapshot": 431,
|
||||
"system": 398,
|
||||
"permission-mode": 387,
|
||||
"agent-name": 228,
|
||||
"mode": 219,
|
||||
"bridge-session": 6
|
||||
},
|
||||
"versions": {
|
||||
"2.1.139": 16,
|
||||
"2.1.140": 86,
|
||||
"2.1.142": 19,
|
||||
"2.1.143": 113,
|
||||
"2.1.144": 58,
|
||||
"2.1.145": 81,
|
||||
"2.1.146": 43,
|
||||
"2.1.150": 137,
|
||||
"2.1.152": 59,
|
||||
"2.1.153": 18,
|
||||
"2.1.156": 80,
|
||||
"2.1.159": 77,
|
||||
"2.1.160": 58,
|
||||
"2.1.161": 102,
|
||||
"2.1.162": 60,
|
||||
"2.1.163": 3,
|
||||
"2.1.165": 20,
|
||||
"2.1.166": 6,
|
||||
"2.1.168": 83,
|
||||
"2.1.169": 74,
|
||||
"2.1.170": 90,
|
||||
"2.1.173": 33
|
||||
},
|
||||
"registry": {
|
||||
"count": 3,
|
||||
"entries": [
|
||||
{
|
||||
"file": "1061541.json",
|
||||
"pid": 1061541,
|
||||
"sessionId": "026cc229-c1cd-4327-9c53-68406794e870",
|
||||
"cwd": "/home/johan/WebstormProjects",
|
||||
"status": "busy",
|
||||
"waitingFor": null,
|
||||
"kind": "interactive",
|
||||
"version": "2.1.170",
|
||||
"peerProtocol": 1,
|
||||
"updatedAt": 1781192047718,
|
||||
"alive": true,
|
||||
"procStartOk": true
|
||||
},
|
||||
{
|
||||
"file": "1701999.json",
|
||||
"pid": 1701999,
|
||||
"sessionId": "8733906c-832a-4e74-aab7-7d8b2f5eb9d2",
|
||||
"cwd": "/home/johan/WebstormProjects",
|
||||
"status": "idle",
|
||||
"waitingFor": null,
|
||||
"kind": "interactive",
|
||||
"version": "2.1.173",
|
||||
"peerProtocol": 1,
|
||||
"updatedAt": 1781191339305,
|
||||
"alive": true,
|
||||
"procStartOk": true
|
||||
},
|
||||
{
|
||||
"file": "597768.json",
|
||||
"pid": 597768,
|
||||
"sessionId": "95f8eaf3-feeb-4fbe-8848-fa7a994c8d7f",
|
||||
"cwd": "/home/johan/WebstormProjects",
|
||||
"status": "busy",
|
||||
"waitingFor": null,
|
||||
"kind": "interactive",
|
||||
"version": "2.1.170",
|
||||
"peerProtocol": 1,
|
||||
"updatedAt": 1781192263383,
|
||||
"alive": true,
|
||||
"procStartOk": true
|
||||
}
|
||||
]
|
||||
},
|
||||
"attachment": {
|
||||
"inWebstormProjects": 573,
|
||||
"outside": 743,
|
||||
"topDirs": {
|
||||
"(racine)": 323,
|
||||
"beehelp_prospect": 133,
|
||||
"techos_web": 31,
|
||||
"beehelp_api": 27,
|
||||
"techos_api": 23,
|
||||
"beehelp_web": 15,
|
||||
"teknoroot-simulator-web": 5,
|
||||
"beehelp_claude_marketplace": 3,
|
||||
"beehelp_lambdas": 2,
|
||||
"teknoroot-simulator-api": 2,
|
||||
"Beehelp_iac": 1,
|
||||
"Techos_V1_outdated": 1,
|
||||
"bee-tiptap-resolver": 1,
|
||||
"beehelp_api-wt-qualiobee-docs": 1,
|
||||
"beehelp_api-wt": 1
|
||||
}
|
||||
},
|
||||
"failuresSample": [
|
||||
{
|
||||
"fp": "/home/johan/.claude/projects/-home-johan-WebstormProjects-beehelp-api/0811ca37-c488-45aa-8a9e-5ecd6c87dfeb.jsonl",
|
||||
"why": "pas de cwd",
|
||||
"types": [
|
||||
"queue-operation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"fp": "/home/johan/.claude/projects/-tmp/59e9a9f8-a5f2-4773-b1f6-72fa8b9514a9.jsonl",
|
||||
"why": "pas de cwd",
|
||||
"types": [
|
||||
"queue-operation"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
145
spikes/s4-discovery/scan.mjs
Normal file
145
spikes/s4-discovery/scan.mjs
Normal file
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env node
|
||||
// Spike S4 — découverte des sessions JSONL (~/.claude/projects) + registre (~/.claude/sessions)
|
||||
// Critères Go : ≥95 % des JSONL exploitables (sessionId+cwd), scan < 3 s (tête+queue),
|
||||
// validation pid/procStart du registre, parseur strictement tolérant.
|
||||
import { readdirSync, statSync, openSync, readSync, closeSync, readFileSync, existsSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { homedir } from 'node:os';
|
||||
|
||||
const PROJECTS = join(homedir(), '.claude', 'projects');
|
||||
const SESSIONS = join(homedir(), '.claude', 'sessions');
|
||||
const HEAD_BYTES = 256 * 1024;
|
||||
const TAIL_BYTES = 64 * 1024;
|
||||
|
||||
function readChunk(path, position, length) {
|
||||
const fd = openSync(path, 'r');
|
||||
try {
|
||||
const buf = Buffer.alloc(length);
|
||||
const n = readSync(fd, buf, 0, length, position);
|
||||
return buf.subarray(0, n).toString('utf8');
|
||||
} finally { closeSync(fd); }
|
||||
}
|
||||
|
||||
// Parse tolérant : chaque ligne en try/catch, types inconnus ignorés.
|
||||
function parseLines(text, partialFirst = false, partialLast = false) {
|
||||
const lines = text.split('\n');
|
||||
if (partialFirst) lines.shift(); // 1re ligne potentiellement coupée (lecture de queue)
|
||||
if (partialLast) lines.pop(); // dernière ligne potentiellement coupée (lecture de tête)
|
||||
const out = [];
|
||||
for (const l of lines) {
|
||||
if (!l.trim()) continue;
|
||||
try { out.push(JSON.parse(l)); } catch { /* ligne coupée ou malformée : ignorée */ }
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function extractMeta(objs, meta) {
|
||||
for (const o of objs) {
|
||||
if (o.sessionId && !meta.sessionId) meta.sessionId = o.sessionId;
|
||||
if (o.cwd && !meta.cwd) meta.cwd = o.cwd;
|
||||
if (o.gitBranch && !meta.gitBranch) meta.gitBranch = o.gitBranch;
|
||||
if (o.version && !meta.version) meta.version = o.version;
|
||||
if (o.slug && !meta.slug) meta.slug = o.slug;
|
||||
if (o.type && meta.types) meta.types.add(o.type);
|
||||
if (o.timestamp) meta.lastTs = o.timestamp;
|
||||
if (o.isSidechain === true) meta.hasSidechain = true;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- 1. Scan ~/.claude/projects ----------
|
||||
const t0 = process.hrtime.bigint();
|
||||
const report = { files: 0, bytes: 0, ok: 0, noCwd: 0, noSessionId: 0, empty: 0, errors: 0 };
|
||||
const versions = new Map();
|
||||
const allTypes = new Map(); // type de ligne → occurrences (sur les chunks lus)
|
||||
const failures = [];
|
||||
const sessions = [];
|
||||
|
||||
for (const dir of readdirSync(PROJECTS)) {
|
||||
const dirPath = join(PROJECTS, dir);
|
||||
let entries;
|
||||
try { entries = readdirSync(dirPath, { withFileTypes: true }); } catch { continue; }
|
||||
for (const e of entries) {
|
||||
if (!e.isFile() || !e.name.endsWith('.jsonl')) continue;
|
||||
const fp = join(dirPath, e.name);
|
||||
report.files++;
|
||||
try {
|
||||
const st = statSync(fp);
|
||||
report.bytes += st.size;
|
||||
if (st.size === 0) { report.empty++; failures.push({ fp, why: 'vide' }); continue; }
|
||||
const meta = { types: new Set() };
|
||||
const head = readChunk(fp, 0, Math.min(HEAD_BYTES, st.size));
|
||||
extractMeta(parseLines(head, false, st.size > HEAD_BYTES), meta);
|
||||
if (st.size > HEAD_BYTES + TAIL_BYTES) {
|
||||
const tail = readChunk(fp, st.size - TAIL_BYTES, TAIL_BYTES);
|
||||
extractMeta(parseLines(tail, true, false), meta);
|
||||
}
|
||||
for (const t of meta.types) allTypes.set(t, (allTypes.get(t) ?? 0) + 1);
|
||||
if (!meta.sessionId) { report.noSessionId++; failures.push({ fp, why: 'pas de sessionId', types: [...meta.types] }); continue; }
|
||||
if (!meta.cwd) { report.noCwd++; failures.push({ fp, why: 'pas de cwd', types: [...meta.types] }); continue; }
|
||||
report.ok++;
|
||||
if (meta.version) versions.set(meta.version, (versions.get(meta.version) ?? 0) + 1);
|
||||
sessions.push({ dir, file: e.name, sessionId: meta.sessionId, cwd: meta.cwd, branch: meta.gitBranch, mtimeMs: st.mtimeMs });
|
||||
} catch (err) {
|
||||
report.errors++;
|
||||
failures.push({ fp, why: `exception: ${err.message}` });
|
||||
}
|
||||
}
|
||||
}
|
||||
const scanMs = Number(process.hrtime.bigint() - t0) / 1e6;
|
||||
|
||||
// ---------- 2. Validation du mapping nom-de-dossier vs cwd ----------
|
||||
const munge = (p) => p.replace(/[^A-Za-z0-9]/g, '-');
|
||||
let mungeMatch = 0, mungeMismatch = 0;
|
||||
const mismatchSamples = [];
|
||||
for (const s of sessions) {
|
||||
if (munge(s.cwd) === s.dir) mungeMatch++;
|
||||
else { mungeMismatch++; if (mismatchSamples.length < 5) mismatchSamples.push({ dir: s.dir, cwd: s.cwd }); }
|
||||
}
|
||||
|
||||
// ---------- 3. Registre ~/.claude/sessions ----------
|
||||
const registry = [];
|
||||
if (existsSync(SESSIONS)) {
|
||||
for (const f of readdirSync(SESSIONS)) {
|
||||
if (!f.endsWith('.json')) continue;
|
||||
try {
|
||||
const r = JSON.parse(readFileSync(join(SESSIONS, f), 'utf8'));
|
||||
let alive = false, procStartOk = null;
|
||||
try {
|
||||
const stat = readFileSync(`/proc/${r.pid}/stat`, 'utf8');
|
||||
// champ 22 = starttime ; le nom du process (champ 2) peut contenir espaces/parenthèses → parser après ')'
|
||||
const after = stat.slice(stat.lastIndexOf(')') + 2).split(' ');
|
||||
const starttime = after[19]; // champ 22 global = index 19 après le champ 3
|
||||
alive = true;
|
||||
procStartOk = String(r.procStart) === String(starttime);
|
||||
} catch { alive = false; }
|
||||
registry.push({ file: f, pid: r.pid, sessionId: r.sessionId, cwd: r.cwd, status: r.status,
|
||||
waitingFor: r.waitingFor ?? null, kind: r.kind, version: r.version, peerProtocol: r.peerProtocol,
|
||||
updatedAt: r.updatedAt, alive, procStartOk });
|
||||
} catch (err) { registry.push({ file: f, error: err.message }); }
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- 4. Rattachement cwd → repos réels ----------
|
||||
const ROOT = join(homedir(), 'WebstormProjects');
|
||||
let inRoot = 0, outRoot = 0;
|
||||
const byTopDir = new Map();
|
||||
for (const s of sessions) {
|
||||
if (s.cwd.startsWith(ROOT + '/') || s.cwd === ROOT) {
|
||||
inRoot++;
|
||||
const top = s.cwd === ROOT ? '(racine)' : s.cwd.slice(ROOT.length + 1).split('/')[0];
|
||||
byTopDir.set(top, (byTopDir.get(top) ?? 0) + 1);
|
||||
} else outRoot++;
|
||||
}
|
||||
|
||||
// ---------- Rapport ----------
|
||||
const pct = (n) => report.files ? ((n / report.files) * 100).toFixed(1) + ' %' : 'n/a';
|
||||
console.log(JSON.stringify({
|
||||
scan: { ...report, mb: (report.bytes / 1048576).toFixed(0), scanMs: scanMs.toFixed(0), okPct: pct(report.ok) },
|
||||
mungeValidation: { mungeMatch, mungeMismatch, mismatchSamples },
|
||||
lineTypes: Object.fromEntries([...allTypes.entries()].sort((a, b) => b[1] - a[1])),
|
||||
versions: Object.fromEntries([...versions.entries()].sort()),
|
||||
registry: { count: registry.length, entries: registry },
|
||||
attachment: { inWebstormProjects: inRoot, outside: outRoot,
|
||||
topDirs: Object.fromEntries([...byTopDir.entries()].sort((a, b) => b[1] - a[1]).slice(0, 15)) },
|
||||
failuresSample: failures.slice(0, 10),
|
||||
}, null, 1));
|
||||
Reference in New Issue
Block a user