Forge LLMs API reference (Preview)

The @forge/llm SDK gives you a lightweight, purpose-built client for invoking Atlassian-hosted LLMs directly from Forge runtime functions.

Use chat() for structured multi-turn exchanges. Use stream() to incrementally receive LLM responses as smaller chunks. Provide 'tool' definitions so the model can call typed functions, and inspect returned usage to guide adaptive behaviour.

For runnable examples (tool wiring, retries, error handling), see the Forge LLMs tutorials and example apps section.

Module

The SDK requires the llm module to be defined in your manifest.yml. If the SDK is used without declaring this module, linting will fail with an error like:

1
2
Error: LLM package is used but 'llm' module is not defined in the manifest

The SDK can automatically fix your manifest. After linting, the manifest will include:

Example of corrected manifest.yml:

1
2
modules:
  llm:
    - key: llm-app
      model:
        - claude

Versioning

The llm module is required to enable Forge LLMs. When you add the llm module to an app's manifest.yml, it triggers a major version upgrade and requires administrators of existing installations to review and approve the update.

Method signature

The SDK supports the following methods:

1
2
list() => Promise<ModelListResponse>
chat(Prompt) => Promise<LlmResponse>
stream(Prompt) => Promise<StreamResponse>

Both methods support a chat and a stream interface available for all supported models. See the @forge/llm package for the most up-to-date request and response schema definitions.

Request schema

1
2
```javascript
interface Prompt {
  model: string;
  messages: {
    role: "system" | "user" | "assistant" | "tool";
    content: string | { type: 'text'; text: string; }[];
  }[];
  max_completion_tokens?: number;
  temperature?: number;
  top_p?: number;
  tools?: {
    type: "function";
    function: {
      name: string;
      description: string;
      parameters: object;
    };
  }[];
  tool_choice?: "auto" | "none" | "required" | { type: "function"; function: { name: string } };
}

Response schema

1
2
interface LlmResponse {
  choices: {
    finish_reason: string;
    index?: number;
    message: {
      content: string | { type: "text"; text: string; }[];
      role: "assistant";
      tool_calls?: {
        id: string;
        type: "function";
        index: number;
        function: { name: string; arguments: object; };
      }[];
    };
  }[];
  usage?: { input_token?: number; output_token?: number; total_token?: number; };
}

interface StreamResponse extends AsyncIterable<LlmResponse> {
  close(): Promise<void> | undefined;
}

interface ModelListResponse {
  models: {
    model: string;
    status: "active" | "deprecated";
  }[];
}

Example: Using chat

1
2
import { chat } from '@forge/llm';
try {
  const response = await chat({
    model: 'claude-3-7-sonnet-20250219',
    messages: [
      {
        role: 'user', content: 'Write a short poem about Forge LLMs.'
      }
    ],
  });

  console.log("#### LLM response:", JSON.stringify(response));
} catch (err) {
  console.error('#### LLM request failed:', { error:  err.context?.responseText });
  throw err;
}

Example: Using stream

1
2
import { stream } from '@forge/llm';
try {
  const response = await stream({
    model: 'claude-3-7-sonnet-20250219',
    messages: [
      {
        role: 'user', content: 'Write a short poem about Forge LLMs.'
      }
    ],
  });

  for await (const chunk of response) {
    console.log("#### LLM response:", JSON.stringify(chunk));
  }

  response.close();

} catch (err) {
  console.error('#### LLM request failed:', { error:  err.context?.responseText });
  throw err;
}

Validation rules

The following request validation rules apply to specific models: