L
LocalMode-AI/LocalMode
Data Extractor Blockui/blocks/agents/data-extractorSchema-validated structured-data extraction from free text over its OWN on-device WebGPU-only WebLLM model (default Qwen3-1.7B-q4f16_1-MLC): five zod templates (contact, event, review, recipe, job), useGenerateObject json mode with maxRetries 3 retry/self-correction and a visible attempts counter, and an artifacts canvas that docks the validated JSON into a sortable data table + a chart built from the extraction's REAL numeric data (documented empty-chart state otherwise). Ships a justified block-local WebLLM catalog slice (model-catalog.ts) + the five templates + deriveArtifacts (templates.ts). Models download only behind an explicit Load action. Split from ui/blocks/agent-structured-data (Data Extractor mode).
Research Agent Blockui/blocks/agents/research-agentA tool-using ReAct research agent over its OWN on-device WebGPU-only WebLLM model (default Qwen3-1.7B-q4f16_1-MLC): three block-local tools (search over a bundled 8-article knowledge base, note, calculate), a step-by-step reasoning timeline, and a human-in-the-loop tool-approval gate (default on) with pause/approve/deny-feedback and immutable decision receipts. Ships a justified block-local WebLLM catalog slice (model-catalog.ts) and toolset + fixture corpus (tools.ts). Models download only behind an explicit Load action. Split from ui/blocks/agent-structured-data (Research Agent mode).
Audio Classifier Blockui/blocks/audio/audio-classifierMediaPipe YAMNet (521 environmental sound categories) top-8 classification of a microphone recording (MediaRecorder; stop -> automatic classification; recording indicator) or an uploaded audio file, rendered as a ranked list with the top prediction highlighted. Ships a justified block-local record->classify hook (use-audio-classifier.ts — imports @localmode/core classifyAudio + @localmode/mediapipe audioClassifier under the blocks carve-out; not promotable as a zero-@localmode registry hook). Fully on-device. Regrouped from ui/blocks/vision-lab (Audio tab) into the audio category.
Audiobook Reader Blockui/blocks/audio/audiobook-readerLong-text streaming synthesis (useStreamSpeech) with early playback that begins before synthesis completes, a live clause counter + current-clause display, speed control (0.5x–2.0x), pause/resume/stop transport, a WAV download (audio-{voiceId}-{timestamp}.wav), a one-shot single-take synthesis path, a 10,000-character limit with pre-flight validation, and one-click sample texts. Kokoro downloads only on the first synthesize/stream action. Fully on-device. Split from ui/blocks/audio-studio (Audiobook tab).
Live Transcription Blockui/blocks/audio/live-transcriptionOpen-mic streaming speech-to-text (useLiveTranscribe) with VAD-segmented utterances, plus a listen→plan→speak turn-taking assistant (useTurnTaker: Granite 4.0 350M plans, Kokoro speaks) selectable via a two-mode picker. VAD choice: the zero-download energy VAD (default) or a gated ~2MB Silero ONNX VAD. Stop calls dispose() so the microphone MediaStream is released; barge-in + interrupt supported. Owns its own STT selector, its own adapter-backed inference-device probe for the Granite planner, and holds the Silero provider block-local (the module-scoped Silero singleton in audio-studio/models.ts is eliminated). Fully on-device. Split from ui/blocks/audio-studio (Live tab).
Meeting Assistant Blockui/blocks/audio/meeting-assistantAudio upload OR pasted transcript → Transcribe → Summarize (DistilBART, maxLength 200 / minLength 60) → Extract action items (useGenerateObject on Granite 4.0 350M) with a visible three-step indicator, a cancel that preserves completed steps, action-item priority badges + completion toggles with a live X/Y count, a [No speech detected] short-circuit, and a structured dated .txt export. Owns its own STT selector (Whisper Tiny EN default + Moonshine-Base recommendation hint) and its own adapter-backed inference-device probe for the Granite LM. Fully on-device. Split from ui/blocks/audio-studio (Meeting tab).
Voice Explorer Blockui/blocks/audio/voice-explorerBrowse, search, preview, and A/B-compare all 29 Kokoro voices (grouped by language label; play/stop preview where starting one preview stops the previous; sequential A→B synthesis over shared text with a cancel). Kokoro (~86MB) downloads only on the first preview or comparison. Fully on-device. Split from ui/blocks/audio-studio (Voices tab).
Voice Notes Blockui/blocks/audio/voice-notesRecord (push-to-talk from a selectable mic) or upload audio → Whisper/Moonshine transcription → saved notes with synced word-by-word replay (real Whisper segment timestamps; disabled with a hint for Moonshine), block-local semantic search over the notes (bge-small), a [No speech detected] fallback, cancel-in-flight, and delete-leaves-the-index. Owns its own speech-to-text model selector; every model downloads only behind an explicit action. Ships a justified block-local note index hook (use-note-index.ts — the VectorDB + indexed-id set that WAS a module-scoped singleton in audio-studio/models.ts, now block-local under the blocks carve-out). Fully on-device. Split from ui/blocks/audio-studio (Notes tab).
Chat Blockui/blocks/chatThe flagship on-device chat: a 76-model multi-provider selector (transformers, WebLLM, wllama GGUF, LiteRT) with capability gating, vision attachments, DeepSeek-R1 reasoning display, semantic response caching, agent mode with tools, custom GGUF URLs, system prompt + parameter controls, and slash commands — everything runs in the browser.
Device Report Blockui/blocks/device/device-reportA zero-download device capability report rendered entirely from browser APIs (useCapabilities / useStorageQuota / useAdaptiveBatchSize): hardware cores/memory/GPU name, six feature flags (WebGPU/WASM/SIMD/Threads/IndexedDB/Workers), storage used/quota/available, browser + OS line, a reference 1B-Q4_K_M readiness verdict, and adaptive batch size + reasoning. Composes device-badge, device-capability-grid, browser-compat-card, storage-meter, and adaptive-batch-card. Zero network and zero model bytes on mount. Split from ui/blocks/device-model-lab (device report section).
GGUF Explorer Blockui/blocks/device/gguf-explorerBrowse 160,000+ HuggingFace GGUF models and the curated wllama catalog, inspect any file's metadata via a ~4KB HTTP Range read (checkGGUFBrowserCompatFromURL — never a model download), get a browser-compat verdict, and hand a model off to the chat block. Consumes the PROMOTED @localmode/wllama model-discovery utils (searchGGUFModels / listGGUFFiles / HFApiError) instead of a block-local hf-api.ts; keeps a justified block-local chat-handoff emitter (handoff.ts). Cursor pagination, typed search errors + retry, and an explicit Download-to-cache prefetch. No model bytes on load. Split from ui/blocks/device-model-lab (GGUF explorer section).
Model Advisor Blockui/blocks/device/model-advisorRanked on-device model recommendations that compute their OWN device context via useCapabilities and rank useModelRecommendations / getModelRegistry over the in-memory registry with ZERO network: a 21-task grouped selector, ranked model-recommendation-card results with a device-context line + result-count + empty states, two-model comparison via model-comparison-panel (max 2, third replaces the older), and a collapsible custom-model registration form (11 ModelRegistryEntry fields, per-field validation, in-memory notice, registerModel() + refresh). No model download. Split from ui/blocks/device-model-lab (model advisor section).
DevTools Drawer Blockui/blocks/devtools-drawerA global on-device observability drawer: six tabbed surfaces (inference queues, event log, pipeline runs, model cache, device capabilities, per-collection VectorDB stats) over the @localmode/devtools bridge, plus a lazy framework-agnostic host — devtools code loads only on first open, closing keeps collectors running, power-off fully disables.
Background Remover Blockui/blocks/image-studio/background-removerRemove an image's background entirely on-device: SegFormer (Xenova/segformer-b0-finetuned-ade-512-512, ~15MB) segments the subject, the highest-scoring mask is composited as an alpha channel (promoted applyMaskToImage) to a transparent-background PNG shown on a checkerboard Original/Result comparison, with a processing overlay + cancel, best-mask confidence, Download PNG (promoted downloadDataUrl), clear/reset, and dismissible errors with a retry path. The model loads lazily on the first dropped image; nothing downloads on page load. Split out of the tabbed image-studio Remove BG tool (split-image-privacy).
Image Captioner Blockui/blocks/image-studio/image-captionerGenerate alt-text captions for images with ViT-GPT2 (Xenova/vit-gpt2-image-captioning, ~230MB) entirely on-device, accumulating results in a gallery (caption as the card label, native per-card delete) with copy-caption-to-clipboard + transient copied feedback, remove-single, Clear All, a captioned-count indicator, and a compact add-another dropzone once results exist. Validates JPEG/PNG/WebP/GIF up to 10MB; the model loads lazily on the first dropped image. Consumes none of the promoted image ops. Split out of the tabbed image-studio Caption tool (split-image-privacy).
Image Enhancer Blockui/blocks/image-studio/image-enhancerUpscale and restore photos with Swin2SR super-resolution running fully in the browser — a 2x lightweight default (Xenova/swin2SR-lightweight-x2-64), plus 4x classical (Xenova/swin2SR-classical-sr-x4-64) and a real-world Restore variant (Xenova/swin2SR-realworld-sr-x4-64-bsrgan-psnr), each behind its own lazily-loaded model with persistent per-mode results. Presents the result in an Original/Enhanced toggle with a model/scale/status info panel and an Enhanced Nx badge; Download Enhanced PNG (promoted imageResultToDataUrl/getImageDimensions/downloadDataUrl), upload-new, cancel with a first-run hint, and dismissible validation + inference errors. Nothing downloads until you drop an image. Split out of the tabbed image-studio Enhance tool (split-image-privacy).
Document QA Blockui/blocks/knowledge/document-qaExtractive question answering (Xenova/distilbert-base-cased-distilled-squad, grounding corpus context via engine.search topK 1, High/Medium/Low confidence tiers) over its OWN text/PDF corpus, plus Donut (Xenova/donut-base-finetuned-docvqa) document/invoice QA on an uploaded image (~800 MB, gated). CORE engine only — declares no @localmode/langchain or @localmode/wllama. Split from ui/blocks/knowledge-base (Ask tab QA lanes; absorbs qa-bot + invoice-qa).
RAG Chat Blockui/blocks/knowledge/rag-chatStreaming grounded RAG over its OWN text/PDF corpus — token-by-token answers through the engine-owned generator (granite-4.0-350M) with [n] markers upgraded to hover-card inline citations resolved to retrieved sources (PDF page attribution + similarity), wall-clock duration, and cancel — carrying the core-pipeline ⇄ LangChain engine toggle. Split from ui/blocks/knowledge-base (Ask tab RAG lane; absorbs langchain-rag).
Semantic Search Blockui/blocks/knowledge/semantic-searchFull text/PDF/OCR ingest into its OWN corpus with off/recursive/semantic chunking and a live chunk-boundary preview, then reranked vector search (over-fetch topK×3 → a real cross-encoder from the transformers ONNX group or the wllama GGUF group; raw + rerank scores preserved, never fabricated) with category facets and preset or corpus-calibrated similarity thresholds — carrying the core-pipeline ⇄ LangChain engine toggle. Over the promoted useKnowledgeBase engine layer; ships justified block-local ingest-panel + search-panel siblings. Nothing downloads until an explicit action. Split from ui/blocks/knowledge-base (Ingest + Search tabs; absorbs pdf-search + ocr-scanner).
Vector Data Manager Blockui/blocks/knowledge/vector-data-managerFour-format vector import (Pinecone/ChromaDB/CSV/JSONL + native JSON) with auto format-detection, preview, batched cancellable import and a re-embed toggle for text-only rows; native JSON/CSV/JSONL export via vector-export-panel (native export round-trips); embedding-drift banner + cancellable reindex; and storage-quota + vector-storage observability + the device-adaptive batch size — all over its OWN corpus. CORE engine only. Split from ui/blocks/knowledge-base (Data tab; absorbs data-migrator).
Duplicate Finder Blockui/blocks/photo/duplicate-finderUnion-find near-duplicate detection over cached CLIP image embeddings (no re-embedding) — a tunable threshold slider + Strict/Balanced/Relaxed presets, per-group average similarity via a cosine-similarity-meter, keep-first selection + bulk delete, and a cancellable scan that re-groups (never re-embeds) on threshold change. Own usePhotoLibrary instance + its own media-dropzone ingest. Split from ui/blocks/photo-search (Duplicates tab).
Image Search Blockui/blocks/photo/image-searchText→image AND image→image search over one shared CLIP vector space — embed a free-text query or a reference image and rank the ingested photo library with a top-K slider (default 20), a visible min-similarity threshold (getDefaultThreshold ?? 0.2), and CLIP-scale score bars. Own usePhotoLibrary instance + its own media-dropzone ingest. Split from ui/blocks/photo-search (Search tab; absorbs cross-modal-search + product-search).
Photo Categorizer Blockui/blocks/photo/photo-categorizerEditable-label-set categorization with Photo/Product presets, on-device library re-categorization with progress + cancel (no re-embedding), and a counted, single-select category-facet-list filter over the ingested photos. Own usePhotoLibrary instance + its own media-dropzone ingest (photos are zero-shot categorized as they embed). Split from ui/blocks/photo-search (Categories tab).
Smart Gallery Blockui/blocks/photo/smart-galleryAdaptive, device-aware CLIP photo ingest (streamEmbedManyImages at a computeOptimalBatchSize batch size shown on an adaptive-batch-card) with per-item progress, per-batch zero-shot categorization (other/0 fallback), and a grid/list gallery with filename + category + confidence + similar-count — per-photo delete, clear-all, and a cancellable ingest. Own usePhotoLibrary instance; CLIP downloads only behind an explicit Load. Split from ui/blocks/photo-search (Gallery tab).
Encrypted Vault Blockui/blocks/privacy/encrypted-vaultA passphrase-locked, end-to-end encrypted item store driven by useEncryptedVault — AES-GCM ciphertext at rest in IndexedDB, the key derived from the passphrase in memory only — with encrypted notes and imported text documents rendered as lock-aware vault-item-cards (decrypt-on-view + delete), a lock-status badge, a manual Lock that clears the in-memory key, a locked state that survives reload, and a tamper-evident hash-chained audit log (createAuditLog) with in-UI chain verification and JSONL export. Web Crypto + IndexedDB only — no models download. The IndexedDB vault name is kept at 'privacy-vault' for storage-key stability across the category rename. Split out of the tabbed privacy-vault Vault tab (split-image-privacy).
PII Redactor Blockui/blocks/privacy/pii-redactorDetect PII with on-device NER (Xenova/bert-base-NER, BIO prefixes stripped, low-confidence filtered), redact it with reverse-offset [<TYPE>] placeholders honoring per-type toggles plus a core redactPII() regex pass (emails/phones/SSNs/cards), render entity insights (stats bar + relationship graph), copy/export the redacted text, and demonstrate differential privacy over the redacted text's embedding (Xenova/all-MiniLM-L6-v2, ~23MB) via dpEmbeddingMiddleware + a createPrivacyBudget tracker with a DP-applied provenance badge. Models download only behind the explicit Scan action. Ships a block-local redaction.ts (pure entity-mapping/redaction/graph helpers) alongside the component — justified as >130 lines of block-specific logic, too large to inline. Split out of the tabbed privacy-vault Redact tab (split-image-privacy).
Model Evaluator Blockui/blocks/text-insights/model-evaluatorEvaluate a selected classifier over a selected labeled dataset via useEvaluateModel, reporting accuracy plus macro precision/recall/F1, run duration, and a color-coded confusion matrix, with completed/total progress and cancel, JSON export, and clear-results. Keeps one justified local file (datasets.ts) holding the labeled evaluation datasets and the radio-card selector, too large to inline without harming readability (design D5). No model bytes download until an explicit in-block action.
Sentiment Analyzer Blockui/blocks/text-insights/sentiment-analyzerClassify one text or thousands with DistilBERT SST-2 (one item per line, or a .txt/.csv first column) via useSequentialBatch — streaming results in input order with per-item error isolation, a cancel that keeps partial results, determinate progress plus throughput (items/sec, elapsed, ETA), aggregate positive/negative stats over every completed item, and a 100-row windowed result list. The model downloads only behind Run. No model bytes download until an explicit in-block action.
Text Classifier Blockui/blocks/text-insights/text-classifierCustom-label zero-shot classification with MobileBERT MNLI over a user-editable label set (add trimmed + case-insensitively deduped, remove, live count) with an email/message textarea and sample loader. Results render a top-result hero (winning label + confidence) above a ranked score list of all candidate labels. Guards empty input and an empty label set; the model downloads only behind Run. No model bytes download until an explicit in-block action.
Threshold Calibrator Blockui/blocks/text-insights/threshold-calibratorCalibrate a similarity threshold from corpus embeddings via useCalibrateThreshold at percentile 90, over a selected embedding model and corpus, rendering the calibrated threshold versus the model's getDefaultThreshold preset, metadata, distribution statistics, and the MODEL_THRESHOLD_PRESETS reference with the selected model highlighted. The provider-prefixed model id is normalized so the panel, preset highlight, and presets reference align. The model downloads only behind Calibrate. No model bytes download until an explicit in-block action.
Language Detector Blockui/blocks/text/language-detectorDebounced (400 ms) automatic language detection across 110 languages (up to five candidates with name + ISO code + confidence) that runs once typed text reaches 10 characters with no button press, plus a two-text cosine-similarity comparison via the on-device MediaPipe text embedder. Fully on-device; models download only when typing crosses the auto-detect threshold or Compare is pressed. Regrouped from ui/blocks/vision-lab (Language tab) into a new standalone text category (roadmap D2 — NOT merged into text-insights).
Live Tracker Blockui/blocks/vision/live-trackerReal-time MediaPipe streaming trackers on live webcam video — one 4-mode block: Hands (21-point skeleton + handedness), Pose (33-point skeleton + person count), Face (478-point mesh + top-8 expression blendshapes behind a default-on toggle), and Gestures (recognized category with a human-readable label). Live FPS; exactly one tracker/WASM task runs at a time. Ships a justified block-local hot-path canvas util (draw-landmarks.ts — single consumer, imports only @localmode/core topology constants under the blocks carve-out). Fully on-device. Split from ui/blocks/vision-lab (Track tab).
Object Detector Blockui/blocks/vision/object-detectorOne-shot DETR (Xenova/detr-resnet-50) object detection on an uploaded image, the bundled sample, or a webcam-captured still — color-coded bounding boxes, a unique-label legend, an N-found count, a ranked scored list, in-flight download progress with cancel, clear-to-reset, and dismiss+retry — plus a live MediaPipe BlazeFace webcam face loop with a numeric face count and captured-still overlay. Fully on-device; models download only behind explicit actions. Split from ui/blocks/vision-lab (Detect tab).
Complete Blockui/blocks/writing-tools/completeFill-mask word prediction via ModernBERT (top-5) with [MASK] authoring, ranked predictions (token, confidence score, confidence label, score bar) and a substituted full-sentence preview, and click-to-apply-and-iterate. This capability is Transformers.js-only — there is no Chrome AI fill-mask — and the badge says so. No model bytes download until an explicit in-block action.
Summarize Blockui/blocks/writing-tools/summarizeSummarize a document with extractive and abstractive modes and short/medium/long length presets, rendered in a split pane with input/summary word counts, a compression-ratio badge and bar, and a reading-time-saved estimate. On the Transformers.js path abstractive is served by DistilBART and extractive by a real on-device sentence-extraction algorithm drawing sentences verbatim from the source; Chrome summary styles are offered only when the Chrome Summarizer API actually serves the block. Resolves Chrome Built-in AI when available and falls back automatically behind an explicit download gate, with a truthful provider badge. No model bytes download until an explicit in-block action.
Translate Blockui/blocks/writing-tools/translateTranslate across 24 directed offline Opus-MT language pairs (English-centric, every model id verified) with swap-direction that carries the output into the input, per-panel character counts, copy-with-confirmation, and cancel. Resolves the Chrome Translator API when available (zero download) and automatically falls back to the pair's Opus-MT Transformers.js model behind an explicit download gate, with a truthful provider badge. No model bytes download until an explicit in-block action.
Write Blockui/blocks/writing-tools/writeA draft editor with live length feedback, preset and free-text edit instructions with prompt improvement, and an AI edit reviewed as a before/after diff with explicit Accept/Reject (Accept applies exactly the after text). Resolves the Chrome Prompt API when available (zero download) and automatically falls back to a Llama-3.2-1B Transformers.js instruct model behind an explicit download gate, with a truthful provider badge. No model bytes download until an explicit in-block action.