Spring AI Provider

Spring AI is an open-source framework that provides a unified interface for interacting with various language models via OpenAI-compatible endpoints. Rensei integrates with Spring AI to support on-prem and custom LLM deployments.

Provider Summary

Spring AI supports only byok. Profiles must include byok in auth_modes and supply a credential in the credentials.byok field. See Auth Modes and Model Profiles.

When to Use Spring AI

Use Spring AI when you want:

• On-prem deployments - Route to your own managed LLM service.
• OpenAI-compatible APIs - Any endpoint that mimics OpenAI's API (e.g., LM Studio, vLLM, text-generation-webui).
• Custom model hosting - You manage the LLM and endpoint; Spring AI bridges the connection.

Prerequisites

Before creating a Spring AI profile, you must have:

1. A running Spring AI endpoint (or any OpenAI-compatible endpoint).
2. The endpoint URL (e.g., https://lm.company.internal:8000).
3. An API key (if your endpoint requires authentication).

Auth Mode: BYOK Only

Spring AI only supports byok (bring your own key). The key is the API token you use to authenticate to your endpoint.

Setup:

1. In Settings → Integrations, click Add Provider and select Spring AI.
2. Paste your endpoint URL and API key.
3. Click Test Connection - Rensei makes a test call to validate the endpoint.
4. In Settings → Model Profiles, create a profile with:
   • Provider: spring-ai
   • Auth mode: byok (only option)
   • Model ID: the model name your endpoint recognizes (e.g., mistral-7b, llama-2-70b)

Configuration

All configuration flows through the openai namespace (since Spring AI mimics OpenAI's API):

{
  "providerConfig": {
    "openai": {
      "endpoint": "https://lm.company.internal:8000",
      "contextWindow": 4096,
      "temperature": 0.7
    }
  }
}

Supported fields:

Example Profile: On-Prem Mistral

rensei profile create \
  --name "on-prem-mistral" \
  --provider spring-ai \
  --model-id "mistral-7b-instruct" \
  --auth-mode byok \
  --scope project \
  --project-id my-project

Then in a workflow LLM node:

nodeId: llm_call
nodeType: action/llm.inference
config:
  profileId: prof_on_prem_mistral
  systemPrompt: "You are a helpful assistant."

Endpoint Validation

At dispatch time, Rensei validates the endpoint is reachable and responds to API calls. If validation fails, dispatch returns an error with debugging hints:

Spring AI endpoint validation failed:
- URL: https://lm.company.internal:8000
- Status: HTTP 403 Forbidden
- Suggestion: Check API key and network access

Common Issues

"Connection refused"

The endpoint is not running or is unreachable. Verify:

curl -H "Authorization: Bearer $API_KEY" \
  https://lm.company.internal:8000/v1/models

"Invalid API key"

Your BYOK credential is expired or malformed. Update it in Settings → Integrations.

"Timeout"

The endpoint is overloaded or slow to respond. Check endpoint health and increase timeout if needed (operator-configurable via platform settings).

Pricing & Cost

Spring AI incurs no platform usage fees - you pay only for your endpoint infrastructure (compute, storage, bandwidth). Cost events still emit for audit purposes, but no metering charges apply.

Limitations

• BYOK only - No metered or shared auth modes.
• No hosted SLA - Endpoint uptime depends on your infrastructure.
• Manual scaling - You manage capacity; Rensei doesn't auto-scale your endpoint.
• Model selection - Limited to models your endpoint supports.

For production use, ensure your endpoint has:

• Redundancy (load-balanced endpoints or failover).
• Monitoring and alerting.
• Rate limiting to prevent runaway requests.
• Timeout configuration appropriate for your models.

Example: Integration with LM Studio

If you're running LM Studio locally on your development machine:

1. Start LM Studio server (default: http://localhost:1234).
2. In Rensei Settings → Integrations, add a Spring AI provider:
   • Endpoint: http://localhost:1234
   • API key: (LM Studio doesn't require auth; use a dummy value or leave blank if Rensei allows it)
3. Create a profile pointing to your loaded model (e.g., mistral-7b).
4. Dispatch workflows from your development machine to your local LM Studio.

Further Reading

• Auth Modes - Full auth mode reference
• Model Catalog & Routing - Catalog management
• Spring Framework Documentation - Official Spring AI reference