Skip to content

Add Friendli as inference provider #924

Description

@Lee-Si-Yoon

Summary

FriendliAI provides an OpenAI-compatible Chat Completions API at https://api.friendli.ai/serverless/v1/chat/completions. The model list is available at https://api.friendli.ai/serverless/v1/models and includes models already in the Codebuff catalog (GLM 5.2, MiniMax M2.5) under org/model IDs (zai-org/GLM-5.2, MiniMaxAI/MiniMax-M2.5).

Why

Adding Friendli as a provider would let Codebuff route these models through Friendli's serverless endpoint directly, which may offer different pricing/latency characteristics compared to existing routes (OpenRouter, CrofAI, Fireworks).

Friendli-specific considerations

  • Model IDs: org/model format (e.g. zai-org/GLM-5.2). Neither ALLOWED_MODEL_PREFIXES (common/src/constants/model-config.ts) nor providerModelNames currently covers a friendli prefix.
  • Reasoning: Friendli handles reasoning differently from OpenRouter/DeepSeek. Instead of reasoning field or <think> tags, it uses:
    • parse_reasoning (boolean) — enables deterministic reasoning parsing
    • include_reasoning (boolean) — controls whether parsed reasoning appears in response
    • chat_template_kwargs.enable_thinking (boolean) — toggles reasoning for controllable models (e.g. GLM-5.2)
    • Response: message.reasoning_content (non-streaming) / delta.reasoning_content (streaming)
    • Docs: https://friendli.ai/docs/guides/reasoning
  • API reference: https://friendli.ai/docs/openapi/model-apis/chat-completions

What's needed

Since provider routing lives in web/src/llm-api/ (private), I can contribute the public-side changes if a maintainer is open to adding the route:

  • friendli to ALLOWED_MODEL_PREFIXES
  • Friendli model IDs to model-config.ts (or use the existing zai-org/ prefix for GLM)

Is there interest in adding this route?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions