AnythingLLM API Key: Connect a Custom OpenAI Endpoint

Which API key do you mean

People who search anything llm api key usually mean one of two things, and the two are easy to mix up. The first is the key AnythingLLM sends to a model provider so it can generate answers. The second is a developer API key that AnythingLLM itself creates, so that other programs can call AnythingLLM's own API. They are different keys with different jobs.

This article covers the first one: giving AnythingLLM a key and endpoint for an OpenAI-compatible chat API. You will take the key from the API key page, paste it into AnythingLLM along with the base URL https://api.llmapisource.com/v1, and use the model ID uncensored. If you instead need to call AnythingLLM from your own code, generate that key inside the AnythingLLM settings; it has nothing to do with this provider key.

Before you start: test the key outside AnythingLLM

Half of the support questions about custom endpoints turn out to be a wrong key or a typo in the URL. A thirty-second test with curl separates "the endpoint is wrong" from "the app is configured wrong". List the models first:

Copycurl https://api.llmapisource.com/v1/models \
  -H "Authorization: Bearer YOUR_KEY" | grep uncensored

You should see a line that includes uncensored. Then send one chat request:

Copycurl https://api.llmapisource.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "uncensored",
    "messages": [{"role": "user", "content": "Reply with the word ready."}],
    "max_tokens": 20
  }'

If both calls work, the key and URL are good and anything that fails later is a setting inside the app. If you do not have a key yet, sign up with Google or with email and password. New users can claim $0.50 of trial credit with one click on the key page, which needs no card and is valid for 7 days. That is enough to finish this whole setup and test it.

Setting it up in AnythingLLM

AnythingLLM supports several LLM providers, and one of them is a generic OpenAI-compatible option. The wording of menus changes between versions, so treat the names below as a guide rather than an exact script.

1. Open the AnythingLLM settings and find the LLM preference (sometimes labeled LLM provider).
2. Choose Generic OpenAI from the provider list.
3. Set the Base URL to https://api.llmapisource.com/v1. Include the /v1 and leave off a trailing /chat/completions; the app adds the path itself.
4. Paste your key into the API Key field. The app sends it as Authorization: Bearer, which is what this API expects.
5. Set the Chat Model Name to uncensored.
6. Set the Token context window to 100000. The model's window is 100,000 tokens, prompt plus completion, and a low value here makes AnythingLLM trim your history earlier than it needs to.
7. Leave max tokens empty, or set a ceiling you are comfortable with. It is optional on the API side.
8. Save, open a workspace and send a message.

If the workspace has its own chat settings, check that it is using the system default provider rather than an older override. A workspace pinned to a previous provider is a common reason a change appears to do nothing.

Embeddings and documents need a separate piece

AnythingLLM does two jobs. It chats with a model, and it indexes your documents so the model can answer from them. The second job uses an embedding model, and this API does not offer embeddings. It exposes POST /v1/chat/completions and GET /v1/models, with text in and text out.

That does not block you. AnythingLLM lets you choose the embedder separately from the chat model, and it can use a built-in local embedder or another provider you configure. Set the embedder to something that is not this endpoint, and use this endpoint only for the chat model. The same applies to image or audio features: the model is text only, with no image, audio or video generation and no fine-tuning.

When you chat with documents, retrieved passages are pasted into the prompt, so input tokens grow. At $0.15 per 1M input tokens, a 6,000-token prompt with 500 tokens of output costs 6,000 x $0.15 / 1,000,000 = $0.0009 for input plus 500 x $0.50 / 1,000,000 = $0.00025 for output, or $0.00115 per question. Document-heavy workspaces cost more than plain chat, but still a fraction of a cent per question.

Checking streaming, tools and JSON mode

AnythingLLM streams replies, and the API supports Server-Sent Events with stream: true, ending with a usage chunk. If your replies appear all at once instead of token by token, the problem is usually a proxy in front of the app that buffers responses, not the endpoint. You can check streaming from a script with the OpenAI Python SDK:

Copyfrom openai import OpenAI

client = OpenAI(
    base_url="https://api.llmapisource.com/v1",
    api_key="YOUR_KEY",
)

stream = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "List three uses of a local knowledge base."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\ntokens:", chunk.usage.prompt_tokens, chunk.usage.completion_tokens)

Tool calling and JSON mode work through the same chat endpoint, which matters if you build agents or flows around the app. JSON mode is a single field:

Copycurl https://api.llmapisource.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "uncensored",
    "response_format": {"type": "json_object"},
    "messages": [
      {"role": "system", "content": "Return a JSON object with keys title and summary."},
      {"role": "user", "content": "Summarize: a notes app that works offline."}
    ]
  }'

Tools use the usual tools and tool_choice fields, with results returned as role: "tool" messages. Whether AnythingLLM exposes a given agent feature depends on the app version, so test the call in a script first and then try the feature in the app. The docs show a full tool-calling example.

Fixing common errors

Symptom	Likely cause	Fix
401 or an authentication error	Key pasted with a space, or an old key	Copy the key again from the key page and paste it without spaces
404 or "not found"	Base URL missing /v1, or has /chat/completions on the end	Use exactly https://api.llmapisource.com/v1
Model not found	Model name typed differently	Use uncensored, lowercase, nothing added
429 with rate_limited	More than 300 requests a minute (60 on a trial key)	Slow down or batch less aggressively
429 with concurrency	More than 8 requests at once (2 on a trial key)	Reduce parallel workspaces or agents; a stream holds its slot until it ends
429 with a Retry-After: 5 header	Temporary overload	Wait 5 seconds and retry
Replies cut off early	Low max tokens in the app	Raise it or clear it

The X-Request-Id header on every response helps if you contact support about a specific call. The balance in X-Balance-USD also tells you quickly whether a failure is simply an empty account.

Getting better answers from a workspace

Once the connection works, a few workspace habits make a visible difference. Write a system prompt for the workspace that states the role, the tone and what to do when the documents do not contain the answer. Without that last instruction a model will often guess, and a guess that sounds confident is worse than a plain "I could not find that in the documents".

Temperature is the other lever. For question answering over your own files, a low value such as 0.2 to 0.4 keeps the answers close to the retrieved text. For brainstorming or drafting, raise it toward 0.8. The API passes temperature, top_p, stop, seed and the penalties through, so whatever the app sends reaches the model unchanged.

Keep an eye on how many snippets the workspace attaches to each question. Every extra snippet is more input tokens, and with a 100,000-token window you have room, but you also pay for each one on every turn. Start with a small number of snippets and raise it only when answers miss relevant material.

Privacy and cost notes

Prompts sent through the API are not used for training. Your documents still travel to the endpoint inside prompts when AnythingLLM retrieves passages, so decide which workspaces you are comfortable sending out. For material that must never leave your machine, use a local model instead.

Billing is pay as you go from prepaid credit: $0.15 per 1M input and $0.50 per 1M output tokens, top-ups from $20 to $500, with bonuses of 5% from $50, 10% from $100 and 15% from $200. Credit never expires, and failed or refused requests are not charged. The pricing page lists the payment methods.

More guides

• Are There Any LLM With a Free API? What Free Costs
• Best LLM for Roleplay 2025: What to Look For
• LiteLLM vs Direct Uncensored LLM API