KEd language assist

Why we split KEd Studio (internal) from a future KEd Editor (public), where Kifuliiru word data lives, and how autocomplete leads to autocompose.

Working checklist: NEXT_STEPS.md

Why two editors?

| | KEd Studio (this repo) | KEd Editor (future, public) | |---|---|---| | Audience | Wekify / Kifuliiru Lab — publishing team | Community, learners, contributors | | Job | Import generated data, design books, export PDF/DOCX | Write and edit Kifuliiru text with language help | | Data | .ked.json packages from generators | Same lexicon + user-authored content | | Network | Works offline in desktop shell | Online-first; sync contributions back |

We are not building the public editor in this repo yet. We are adding autocomplete to Studio now because:

1. Editors already type Kifuliiru in cover text and page bodies.
2. Studio is where we dogfood the lexicon before shipping it publicly.
3. The same KEd Core lexicon + suggest logic will power both apps later.

Roadmap: autocomplete → autocompose

| Stage | What the user gets | Data needed | |--------|-------------------|-------------| | 1. Autocomplete (now) | Type a prefix → pick a Kifuliiru word with glosses | Lexicon: ~6k dictionary entries + verbs | | 2. Rich suggest | Filter by word type (noun, verb), noun class, semantics | semantics_dictionary, relationship graph | | 3. Autocompose (later) | Pick a sentence pattern → valid word combinations suggested | Semantics matrix + grammar templates (generators already prototype this) | | 4. Public editor | Community writes with assist; approved words flow back to lab DB | Sync pipeline to generators / Supabase |

Autocompose needs enough structured data (lexicon + compatibility rules). Autocomplete is the first step — it forces us to bundle and maintain that data in Studio.

Where Kifuliiru data lives

Primary source (use this)

kifuliiru-data-generators Postgres dictionary — source of truth for the lab.

| Tables | Contents | |--------|----------| | dictionary_nouns, dictionary_adjectives, dictionary_adverbs, … | Multilingual entries: kifuliiru, english, french, swahili, noun class, semantics | | verbs | Verb lemmas (~598+ documented) | | semantics_dictionary | Noun-centric compatibility (for future autocompose) | | sentences, generated_sentences, longer_texts | Sentence corpora (separate from word autocomplete) |

Documented scale: ~6,237 dictionary entries (generators docs/DICTIONARY_GROWTH_STRATEGY.md).

APIs already built for autocomplete:

| Endpoint | Use | |----------|-----| | GET /api/text-builder/vocabulary?isActive=true | Full lexicon in TextBuilderVocabulary shape (best for export) | | GET /api/text-builder/vocabulary?search=… | Prefix search server-side | | GET /api/dictionary?search=… | Cross-table dictionary search | | GET /api/stats | Row counts per table |

Reuse in generators (do not rewrite):

• src/services/text-to-blocks-converter.ts → TextToBlocksConverter.suggest()
• src/components/text-builder/TextInputWithAutocomplete.tsx → UI pattern

Studio ports the suggest algorithm into lib/lexicon/suggest.ts (no cross-repo import).

Static fallbacks (offline / export)

| Path (generators repo) | Records | Notes | |------------------------|---------|--------| | data/translations/dictionary-verbs.json | 1,237 verbs | Good dev seed; verbs only | | data/Data/Data Sentences/*.csv | ~1,320 sentences | Not for word autocomplete | | docs/kifuliiru_unique_words.txt | 638 tokens | No glosses — avoid |

Secondary sources (enrichment later)

| Source | Path / system | Notes | |--------|---------------|--------| | Tabula / Lola magambo | Supabase public.magambo | Community dictionary; audio, examples; different schema | | Legacy Python | ked-kifuliiru-editor/legacy/python/... | Archived generators; reference only |

Studio data pipeline

Desktop Studio must work offline. Lexicon ships as a bundled JSON file, refreshed from generators on a schedule or before release.

generators Postgres
 → GET /api/text-builder/vocabulary?isActive=true
 → npm run lexicon:export (this repo)
 → public/data/kifuliiru-lexicon.json
 → Studio loads at startup (useKifuliiruLexicon)
 → suggestLexicon() on contenteditable input (UI next)
```text

### Export lexicon

```bash
# Generators must be running with DB connected
cd ../kifuliiru-data-generators && npm run dev

# In another terminal (default: http://localhost:3000)
cd ked-kifuliiru-editor
npm run lexicon:export

# Custom generators URL (e.g. port 3001)
GENERATORS_URL=http://localhost:3001 npm run lexicon:export
```text

If the API is unreachable, the script falls back to `../kifuliiru-data-generators/data/translations/dictionary-verbs.json` (verbs only).

Commit `public/data/kifuliiru-lexicon.json` before desktop releases, or run export in CI.

---

## Implementation in this repo

| File | Role | |------|------| | `lib/lexicon/types.ts` | `LexiconEntry`, `LexiconBundle` | | `lib/lexicon/suggest.ts` | Prefix/contains scoring (ported from generators) | | `lib/lexicon/load.ts` | Fetch bundled JSON | | `hooks/useKifuliiruLexicon.ts` | Load lexicon on app start | | `scripts/export-lexicon.mjs` | Pull from generators API → JSON | | `components/KifuliiruAutocomplete.tsx` | *(next)* Dropdown on page/cover editing |

### Autocomplete UX (next session)

- Trigger on typing in `contenteditable` page bodies and cover fields.
- Default language: **Kifuliiru**; toggle EN / FR / SW for gloss-first typing.
- Show: Kifuliiru form + English gloss in dropdown.
- Reuse scoring from `suggestLexicon()`; port keyboard nav from `TextInputWithAutocomplete.tsx`.

### Autocompose (later — generators)

- `semantics_dictionary` + sentence templates in generators Text Builder.
- Studio calls generators API when online, or bundles a trimmed semantics snapshot for desktop.
- Not started in Studio until autocomplete is stable on the full lexicon.

---

## Product names (recap)

| Name | Role | |------|------| | **KEd Studio** | Internal publishing workstation — **autocomplete first** | | **KEd Editor** | Future public writing surface — same lexicon + autocompose | | **KEd Core** | Shared `lib/lexicon`, suggest, normalize (extracted when Editor ships) |

---

## Success criteria

**Autocomplete v1**

- [ ] `npm run lexicon:export` produces full lexicon JSON from generators DB.
- [ ] Studio loads lexicon; status shows entry count.
- [ ] Typing in a page body suggests Kifuliiru words with glosses.
- [ ] Works offline in installed desktop app (bundled JSON).

**Autocompose (future)**

- [ ] Sentence pattern + noun picks valid verbs/adjectives from semantics graph.
- [ ] Enough coverage in `semantics_dictionary` to demo one book type (e.g. simple sentences).