ChrisJohnson

Problem

Local research without inspectable structure

I needed a way to explore internal documents as a graph without uploading them to a cloud corpus. Notes, papers, and source files sit in local folders; the links between them are not visible, and most tools that surface those links require the files to leave the machine.

The requirement was a 3D graph I could ingest a folder into, where selecting a node opens the source document. Rendering was secondary to whether an edge could be explained.

Approach

Client-side graph

The application is a client-side SPA. Parsing, embeddings, similarity, and clustering run in the browser in web workers so ingest does not block the scene. Corpora are stored in IndexedDB. Documents do not leave the tab.

Optional cloud AI is off by default. Ollama is available when a local model is required. Production CSP enforces this constraint, and an air-gapped build is provided for the same reason.

Relationship model

3D rendering is not the difficult part. Assigning structure that can be inspected is.

Extraction is layered. Deterministic signals run first: markdown links, filename mentions, and TF-IDF keywords. Path and folder affinity is used as a layout hint, not as an edge.

Local embeddings run second: transformers.js and BGE-small in a web worker, with cosine similarity and a top-k cap so a corpus of a few hundred documents does not produce an unreadable graph. Optional LLM enrichment runs third, for summaries and canonical topic labels. The graph is functional without the LLM.

Every edge carries evidence so the interface can show why two documents are related.

Ingest

A selected folder is walked recursively. Source-code repositories respect gitignore. Scanned PDFs are processed with bundled Tesseract. Folders can be watched through the File System Access API; on the desktop application the native watch interval is about one second.

The demo corpus ingests at about 11 documents per second. For larger corpora, semantic similarity is computed in chunks so the tab remains responsive. The working loop is ingest, explore, and read source.

Interchange

OpenUSD export is an interop sidecar rather than the primary interface. A single .usda file represents documents as prims and edges as curves that retain evidence, with a detailed/summary variantSet.

The stage opens in usdview, Omniverse, and Reality Composer Pro. A Python CLI validates the schema and packages .usdz. usd-agent can query the stage. The in-app loop is unchanged by this export path.

Collaboration

Collaboration shipped after the v1 spec listed it as a non-goal. Rooms use Yjs and WebRTC, with sanitized invite links (#collab=). Air-gapped builds omit this path.

The session synchronizes annotations, peer presence, and view state: selection, camera, follow-presenter, and filters. Documents remain local to each peer. The room is not a shared cloud corpus.

Presence is shown as rings in the scene, with follow-presenter to align cameras. Signaling is opt-in. CSP permits it for a session; the air-gapped build does not.

Outcome

Local-first workspace

Shareable graph URLs omit full text and file bytes. Multiple named corpora are supported. There is a macOS application and a Linux AppImage. The entry bundle is about 80 kB; heavier modules lazy-load.

The result is a local-first workspace: documents stay on the machine, relationships are inspectable, scenes can be exported through OpenUSD, and rooms synchronize view state without sharing the corpus.