Local Embeddings with OpenAI

Text embeddings are the foundation of semantic search and retrieval augmented generation (RAG) pipelines. And unlike text generation models, there are many open-source state-of-the-art (SOTA) embedding models that are small enough to run on-device, saving significant amounts on AI spend:

​

Installing Function LLM

We have created a tiny utility library, Function LLM that patches the OpenAI client to generate embeddings on-device. First, install the library:

# Run this in Terminal
$ npm install fxn-llm

​

Generate Embeddings Locally

We will be using Nomic’s @nomic/nomic-embed-text-v1.5-quant embedding model which boasts better performance than OpenAI’s text-embedding-3-small:

import { locally } from "fxn-llm"
import { OpenAI } from "openai"

// 💥 Create your OpenAI client
let openai = new OpenAI();

// 🔥 Make it local
openai = locally(openai);

// 🚀 Generate embeddings
const embeddings = openai.embeddings.create({
  model: "@nomic/nomic-embed-text-v1.5-quant",
  input: "search_query: Hello world!"
});