Configuring Embeddings

Embeddings are the backbone of Witsy’s Retrieval-Augmented Generation (RAG) capabilities. They allow you to index local documents—like PDFs, text files, and markdown—so your AI assistant can "search" through your knowledge base to provide context-aware answers.

Witsy currently supports embedding models from OpenAI and Ollama.

How to Set Up OpenAI Embeddings

OpenAI provides high-performance, cloud-based embedding models. This is the fastest way to get started if you already have an OpenAI API key.

1. Open Settings in Witsy.
2. Navigate to Engines > OpenAI.
3. Ensure your API Key is entered and valid.
4. In the Models section, locate the Embedding Model dropdown.
5. Select your preferred model:
   • text-embedding-3-small: Optimized for speed and efficiency (Recommended).
   • text-embedding-3-large: Higher accuracy for complex datasets.
   • text-embedding-ada-002: The legacy standard.

Note: Using OpenAI embeddings involves a small cost per token processed. For large document repositories, consider using Ollama for a free, local alternative.

How to Set Up Local Embeddings with Ollama

If you prefer to keep your data entirely private on your machine, you can use Ollama to generate embeddings locally.

1. Download an Embedding Model

First, you must pull an embedding-specific model via your terminal:

ollama pull nomic-embed-text

(Witsy also works well with all-minilm or mxbai-embed-large).

2. Configure Witsy

1. Ensure Ollama is running on your machine.
2. In Witsy, go to Settings > Engines > Ollama.
3. Scroll to the Embedding Model setting.
4. Select nomic-embed-text (or the model you downloaded) from the list.
5. Click Save.

How to Use Embeddings with Document Repositories

Once you have configured an embedding engine, you can apply it to a knowledge base.

Create a New Repository

1. Go to the Library or Document Repositories section in Witsy.
2. Click New Repository.
3. Give your repository a name and choose a local folder to index.
4. Under Embedding Configuration, select the engine you configured (OpenAI or Ollama).
5. Click Create and Index.

Switching Embedding Models

If you decide to switch from OpenAI to Ollama (or vice versa), you must re-index your documents:

1. Open the Repository Settings.
2. Change the Embedding Engine/Model.
3. Trigger a Re-index.
   • Why? Vector data generated by one model (e.g., OpenAI) is not compatible with another (e.g., Ollama). The "coordinates" of your data will change, and the search will fail if the indices don't match.

Recipe: Optimizing for Performance vs. Privacy

| Goal | Recommended Engine | Recommended Model | | :--- | :--- | :--- | | Highest Accuracy | OpenAI | text-embedding-3-large | | Best Balance | OpenAI | text-embedding-3-small | | 100% Privacy/Free | Ollama | nomic-embed-text | | Low Resource Usage | Ollama | all-minilm |

Troubleshooting Common Issues

• "No Embedding Engine Configured": Ensure that at least one provider (OpenAI or Ollama) is fully set up in the Engines tab and that a model is selected in the "Embedding" dropdown.
• Indexing is slow: Local embeddings (Ollama) depend on your CPU/GPU. If you are indexing thousands of files, OpenAI will generally be faster.
• Search results are irrelevant: If your search results seem off, try re-indexing the repository. If you are using a very small model (like all-minilm), consider upgrading to nomic-embed-text or OpenAI’s v3 models for better semantic understanding.