Skip to content

feat(embedding): add a provider-neutral LiteLLM embedding node #2065

Open

Description

@krishhgg

Problem

embedding_openai declares six profiles with modelSource: "litellm", but
OpenAIEmbeddingWrapper never reads modelSource. It passes every selected
model directly to LangChain's OpenAIEmbeddings using the default OpenAI
endpoint and probes the model during startup.

The affected profiles are:

  • text-embedding-004
  • text-embedding-005
  • text-embedding-3-small-inference
  • text-embedding-ada-002-v2
  • text-embedding-large-exp-03-07
  • text-embedding-preview-0409

Selecting one sends a non-OpenAI model identifier to OpenAI, so startup fails
before the node processes documents or questions.

CodeRabbit found this while reviewing the regenerated README in #1975:
#1975 (comment)

The short-term documentation clarification is committed in #1975. It identifies
the three profiles that currently work and explains that the other six are not
routed.

Proposed direction

Create a provider-neutral embedding_litellm node instead of adding multiple
providers to embedding_openai.

Keep embedding_openai limited to direct OpenAI models. A separate LiteLLM
node makes the provider boundary, credentials, dependencies, model naming, and
failure behavior explicit.

Use LiteLLM's embedding API or a deliberately configured LiteLLM proxy. Do not
send LiteLLM profiles through OpenAIEmbeddings without an explicit compatible
endpoint.

Before moving any of the six profiles, verify that LiteLLM currently supports
the model and determine its canonical provider-prefixed identifier and required
credentials. The existing raw identifiers should not be assumed to be routable.

Acceptance criteria

  • Add an embedding_litellm node with the same documents and questions
    lane contract as the existing text embedding nodes.
  • Route document batches and single questions through LiteLLM's embedding
    surface or an explicitly configured LiteLLM proxy.
  • Require provider-qualified model identifiers where LiteLLM requires them.
  • Define secure, provider-appropriate credential and endpoint configuration;
    do not assume one OpenAI API key works for every provider.
  • Probe vector size through the selected LiteLLM backend, not the OpenAI
    backend.
  • Preserve the requirement that ingestion and querying use the same embedding
    model and vector dimensions.
  • Audit all six affected profiles. Add only profiles proven to work through
    the selected route; remove or leave unsupported any model that cannot be
    verified.
  • Keep embedding_openai with only direct OpenAI profiles after the migration.
  • Add tests for profile routing, authentication selection, document batching,
    question embedding, startup probing, and clear provider errors.
  • Add a README following docs/development/nodes/readme-schema.md and regenerate
    machine-owned parameter sections.
  • Validate both nodes with scripts/validate-node-readme.py and relevant node
    tests.

Why a separate node

embedding_openai is provider-specific in its name, dependency, credential
model, and implementation. Teaching it to switch among unrelated providers
would make its contract misleading and create conditional authentication and
endpoint behavior inside an OpenAI node. embedding_litellm gives users an
explicit choice between direct OpenAI access and provider-neutral routing.

Activity

  1. github-actions commented on Aug 21, 2026

    @github-actions
    Contributor
    🤖 Internal: Discord sync marker

    Auto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete.

  2. airouter-dev commented on Aug 27, 2026

    @airouter-dev

    The separate embedding_litellm node is a good boundary. I would make the acceptance contract explicit around the vector shape and provider identity:

    • Keep embedding_openai limited to direct OpenAI credentials and models. The LiteLLM node should own its endpoint, credential reference, provider-qualified model ID, timeout, and retry policy.
    • Treat the configured value as an API root and normalize /v1 once. Test origin URLs and URLs already ending in /v1, and assert that ingestion and query requests both reach /v1/embeddings without path duplication.
    • Probe the selected model through the same backend used for real batches. Record the returned dimension and require ingestion/query configuration to use the same model and dimension; never infer a dimension from a model name.
    • Preserve batch order, reject empty data, validate finite numeric vectors, and surface 401/403, wrong-root 404, 429, timeout, and dimension mismatch as distinct actionable errors.
    • Keep provider-qualified IDs intact and do not assume that an identifier accepted by the catalog is available to every credential. Credentials and input text must stay out of logs and generated README/config output.
    • Add a fake HTTP server for valid batches, empty data, malformed vectors, model-not-found, and changing dimensions. Keep external service checks manual and opt-in.

    If an account-visible embedding model is available, I operate AI Router, whose OpenAI-compatible API root is https://api.ai-router.dev/v1. It can be an optional manual fixture for the embeddings path only; it should not be treated as evidence that every account or model supports embeddings.

    Disclosure: I represent AI Router. This is a provider-neutral RAG/embedding test suggestion, not a request for a named provider integration or a pricing/free-credit promotion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions