Semantic search for your notes, offline and without the cloud

You can search your notes by meaning with nothing leaving your computer. It takes two things on your own disk: a small embedding model that turns text into numbers, and an index of those numbers for every passage you have written. With both local, a query like “why did I switch hosting” finds a note that says “moved the blog to a cheaper VPS after the March outage”. That works with the Wi-Fi off and no API key.

In October 2026 the realistic local routes inside a notes app are Jotura’s Local provider, Smart Lookup and Smart Connections for Obsidian, and the semantic search built into Joplin 3.7. Outside a notes app you have AnythingLLM, a self-hosted Khoj server, and command-line tools such as Simon Willison’s llm, Ollama and qmd. Each runs the embedding model on your machine after a one-time download, and the last section gives you a test that proves it.

What an embedding is

An embedding model is a small neural network that reads a passage of text and returns a list of a few hundred numbers, called a vector. Passages that mean similar things get vectors that sit close together, even when they share no words. “Renew the TLS certificate” and “the HTTPS cert expires in May” land near each other, and a note about your sister’s birthday lands far from both.

Semantic search runs your query through the same model and returns the passages whose vectors are nearest. Most tools first split long notes into chunks of a heading section or a few hundred words, so a hit points at the passage that matched. Joplin’s documentation names the weak spot. Semantic search is poor at “exact-token lookups such as names, IDs or error codes”, and a single word “carries too little meaning to match reliably”, so keep keyword search for those.

For the search to stay private, your machine has to embed your notes, store the vectors and embed each query. Some tools build the index locally and still send each query to a hosted API.

Which one fits

Your situation	Start with	Why
A folder of Markdown notes, an Obsidian vault included, that you want to search in a notes app	Jotura with the Local provider	34 MB English model on desktop, beside keyword search. Settings preselects OpenAI, so pick Local before you enable it
You stay in Obsidian	Smart Lookup, plus Smart Connections	Local model built in, free Core tier, works on mobile
You stay in Joplin	Joplin 3.7’s own semantic search	Built in, about 140 MB model, desktop only
A plain folder of Markdown and a terminal, any editor	qmd or llm with a local model	Reads the folder in place and writes its index outside your notes
Chat-style answers from your documents	AnythingLLM Desktop	Local embedder and vector database by default

Every local option at a glance

Option	What it is	Embeds on your machine	What touches the network	Status, October 2026
Jotura	Notes app	With the Local provider; Settings preselects OpenAI	34 MB model download; encrypted vectors in a synced vault	0.11.1
Smart Connections and Smart Lookup	Obsidian plugins	Yes, built-in local model	Model download; a cloud provider only when you configure one	Smart Connections 4.7.2, August 2026
Copilot for Obsidian with Miyo	Plugin plus a local service	Through Miyo, model not documented	Miyo Relay for remote access, paid after a trial	Semantic search moved to Miyo
Joplin	Notes app	Yes, about 140 MB model	Model download	3.7 stable since September 6, 2026; 3.7.21 latest
AnythingLLM Desktop	Chat with your documents	Yes, all-MiniLM-L6-v2 (25 MB)	Model download; anonymous usage telemetry until you turn it off	Maintained
Khoj	Self-hosted search and chat server	Yes, a local search model by default	Only the server you run	2.0.0-beta.28, March 2026; hosted service shut April 15, 2026
Reor	Notes app	Yes, through Transformers.js	Model download	Archived March 7, 2026
llm with sentence-transformers or Ollama	Command line	Yes	Model download	Maintained
qmd	Command line	Yes, three GGUF models	About 2 GB of models on first run	Maintained

Jotura’s local model

Jotura is a Markdown notes app for desktop and Android that opens a folder of .md files as it stands, with search by meaning in the desktop app. An Obsidian vault opens as-is, as the Coming from Obsidian page explains. Semantic search is off until you turn it on, and Settings preselects OpenAI’s text-embedding-3-small as the recommended provider. OpenAI ranks better, especially on non-English notes, and it sends your note text and every query to OpenAI under your own key. The Local provider keeps everything on your machine.

1. Open Settings, go to the Semantic search section, and set Provider to Local before you click Enable. The Enable button on the search page banner reuses the last recorded provider, which is OpenAI until you change it.
2. Jotura downloads bge-small-en-v1.5-int8, about 34 MB, from releases.jotura.io, checks its SHA-256, and embeds every note on your CPU or GPU. The index takes about 1.5 KB per chunk, and the status bar shows progress and time remaining.
3. Pick Semantic or Smart on the search page. Smart merges the keyword and meaning rankings.

The docs table of what leaves your machine lists one exception. In a vault that uses Jotura Sync, each device uploads its vectors, encrypted with your sync key, to an embedding cache so your other computers skip re-embedding. An unsynced vault never contacts the cache.

The model is English-only, and meaning search does not run on the Android app. Jotura has no plugins, so you cannot swap the model. It is also closed source, so the network test below is how you confirm the local claim. The bundled CLI queries the same index and exits with code 13 when semantic search is not set up in the app; the CLI search reference has the details.

jotura search "why did I switch hosting" --mode semantic --json

Obsidian plugins

Obsidian’s built-in search matches words, so meaning search comes from community plugins. Smart Lookup takes a typed question and returns the notes that match it. Smart Connections shows a side panel of notes related to the one you have open.

Both ship with a local embedding model and need no API key on the free Core tier. After a one-time model download, embedding and lookup run inside Obsidian. A cloud provider “receives eligible source text only when you configure one”, according to the Smart Connections site. The 4.7 releases added more built-in local models, multilingual ones among them, and paid Pro tiers add ranking controls and inline connections.

The plugins keep their index in a .smart-env/ folder inside your vault. Obsidian Sync carries it to your other devices, and the plugin’s README asks anyone using a third-party sync tool to add .smart-env/ to its ignore patterns. Install both plugins from Settings, Community plugins, let the first index finish, then run the network test below.

Copilot for Obsidian moved its semantic search out of the plugin into Miyo, a separate local-first service from the same team. Its docs say the old in-plugin index is gone and local use is free. They do not name the embedding model, so the network test is your check.

Joplin 3.7

Joplin added on-device semantic search in version 3.7, first released as stable on September 6, 2026. Open the Configuration screen, go to the AI section and tick Enable AI features. Joplin downloads a model of about 140 MB and indexes your notes in the background. From then on, its documentation says, “no note content is sent to a cloud service.”

Semantic results appear in the normal search box, except for queries that use filters such as title:. The feature runs on Apple Silicon Macs, Windows and Linux, and Intel Macs and the mobile apps do not get it.

The index is not synced, so each computer builds its own. Joplin keeps notes in its own database and exports Markdown, which matters to anyone who wants a plain folder of .md files.

Standalone apps

AnythingLLM

AnythingLLM Desktop embeds documents with all-MiniLM-L6-v2, a 25 MB English model that runs on your CPU, and stores the vectors in a local LanceDB database. You ask a question and get an answer drawn from your notes, so pick a local chat model in its settings to keep the answer offline too. The app sends anonymous usage telemetry by default. Its privacy page says this never includes document content and that you can opt out in the app settings.

Khoj

Khoj is a search and chat server with clients for the browser, Obsidian, Emacs and desktop. Its hosted service shut down on April 15, 2026, so running your own server is now the only way to use it. Install it with python -m pip install 'khoj[local]' or Docker. The search docs say it “sets up decent default local search model config”.

Point the Obsidian plugin’s Khoj URL at your server, by default http://127.0.0.1:42110. Offline chat needs a local model server such as Ollama, with at least 8 GB of RAM.

Reor

Reor was a local-first notes app that embedded every note with Transformers.js and LanceDB. It still runs offline, though its repository was archived on March 7, 2026 and will get no fixes.

From the command line

Simon Willison’s llm with a local model

The llm tool stores embeddings in SQLite and finds the nearest ones. The llm-sentence-transformers plugin runs models on your machine and registers all-MiniLM-L6-v2 as mini-l6:

pipx install llm
llm install llm-sentence-transformers
llm embed-multi notes -m mini-l6 --files ~/notes '**/*.md' --store
llm similar notes -c 'why did I switch hosting'

The first embed-multi downloads the model from Hugging Face, and HF_HUB_OFFLINE=1 stops later contact. llm collections path prints the location of the embeddings.db file. Results come back as JSON with each file’s path as its id.

--files embeds each file as one passage and does no chunking. MiniLM reads only about the first 200 words, so split long notes first or pick a model with a longer input window. For other languages, register a multilingual model:

llm sentence-transformers register paraphrase-multilingual-MiniLM-L12-v2 --alias multi

Ollama embeddings

Ollama serves embedding models from localhost:11434. Google’s embeddinggemma has 300 million parameters, downloads at 622 MB and covers more than 100 languages. The llm-ollama plugin plugs it into the same workflow:

ollama pull embeddinggemma
llm install llm-ollama
llm embed-multi notes-gemma -m embeddinggemma --files ~/notes '**/*.md' --store
llm similar notes-gemma -c 'why did I switch hosting'

qmd

qmd is a small search engine for Markdown folders that combines keyword search, vector search and a local reranking model. It needs Node.js 22 or later, plus Homebrew’s SQLite on macOS:

npm install -g @tobilu/qmd
qmd collection add ~/notes --name notes
qmd embed
qmd vsearch "why did I switch hosting"
qmd query "why did I switch hosting"

query adds query expansion and reranking on top of the vector search that vsearch runs. On first run qmd pulls three GGUF models from Hugging Face into ~/.cache/qmd/models/, about 2 GB in total, then runs offline through node-llama-cpp.

Building your own with sqlite-vec

sqlite-vec is a SQLite extension for storing vectors and running nearest-neighbor queries, installed with pip install sqlite-vec or npm install sqlite-vec. It is pre-v1, so expect breaking changes. With any local embedding model, the core is two statements:

create virtual table chunks using vec0(embedding float[384]);
select rowid, distance from chunks
where embedding match :query_vector
order by distance limit 10;

The time goes into the code around it, such as splitting notes at headings and re-embedding only what changed. Among the command-line tools, qmd already does the splitting, cutting documents into chunks of about 900 tokens at headings and paragraph breaks.

Prove it is local: the network test

1. Install the tool while online, and wait for the model download and the first index to finish.
2. Quit the app, turn off Wi-Fi and unplug Ethernet.
3. Reopen it and search for a note using words that do not appear in that note.
4. A correct result means embedding and search run on your machine. With the first index finished, an error or an empty list means some part runs remotely.

Step 3 matters because some tools fall back to keyword search when embedding fails. Jotura does this with an OpenAI index and says “Showing keyword results.” A query that shares words with the target note can pass on keyword matching alone.

The test shows a tool can work offline. To see what it sends once you reconnect, watch its connections during a search. On macOS the free LuLu or the paid Little Snitch shows each connection as it opens, and OpenSnitch does the same on Linux. On Windows, run this in PowerShell while you search, with the app’s process name in place of Joplin:

Get-NetTCPConnection -State Established -OwningProcess (Get-Process Joplin).Id |
  Select-Object RemoteAddress, RemotePort

On macOS or Linux, lsof -i -P -n | grep -i joplin gives the same view. A local tool shows no new connections while you type queries.

Frequently asked questions

Can I search my notes by meaning without an internet connection?

Yes. Any tool that runs its embedding model on your computer works offline once the model has downloaded. Jotura’s Local provider, Smart Lookup in Obsidian, Joplin 3.7 and the command-line tools above all do this. A tool set to a cloud provider such as OpenAI needs the network for every query.

Do local embedding models work for notes in other languages?

English-only models such as all-MiniLM-L6-v2 and bge-small-en-v1.5 index other languages and match them poorly. Use a multilingual model instead. embeddinggemma through Ollama covers more than 100 languages, paraphrase-multilingual-MiniLM-L12-v2 covers more than 50, and recent Smart Connections releases include multilingual local models.

Can I use offline semantic search on my phone?

Smart Lookup runs in Obsidian’s mobile app and answers typed questions there, and Smart Connections shows related notes on mobile too. Joplin’s semantic search is desktop only, and Jotura’s search by meaning does not run on its Android app.

Is semantic search better than keyword search?

Semantic search finds half-remembered ideas phrased in other words, and it ranks exact names and error messages poorly, so the two work best together. A merged mode, such as Jotura’s Smart mode or qmd query, puts both rankings in one list. For a wider look at notes apps and their search, see the best Markdown note apps.