Preview features are deemed stable; however, they remain under active development and may be subject to shorter deprecation windows. Preview features are suitable for early adopters in production environments.

We release preview features so partners and developers can study, test, and integrate them prior to General Availability (GA). For more information, see Forge release phases: EAP, Preview, and GA.

The @forge/llm SDK gives you a lightweight, purpose-built client for invoking Atlassian-hosted LLMs directly from Forge runtime functions.

Use chat() for structured multi-turn exchanges. Use stream() to incrementally receive LLM responses as smaller chunks. Provide tool definitions so the model can call typed functions, and inspect returned usage to guide adaptive behaviour.

For runnable examples (tool wiring, retries, error handling), see the Forge LLMs tutorials and example apps section.

Module

The SDK requires the llm module to be defined in your manifest.yml. If the SDK is used without declaring this module, linting will fail with an error like:

1
2
Error: LLM package is used but 'llm' module is not defined in the manifest

The SDK can automatically fix your manifest. After linting, the manifest will include:

Example of corrected manifest.yml:

1
2
modules:
  llm:
    - key: llm-app
      model:
        - claude

The app retains its Runs on Atlassian eligibility after the module is added.

Versioning

The llm module is required to enable Forge LLMs. When you add the llm module to an app's manifest.yml, it triggers a major version upgrade and requires administrators of existing installations to review and approve the update.

Method signature

The SDK supports the following methods:

1
2
list() => Promise<ModelListResponse>
chat(Prompt) => Promise<LlmResponse>
stream(Prompt) => Promise<StreamResponse>

Both methods support a chat and a stream interface available for all supported models. See the @forge/llm package for the most up-to-date request and response schema definitions.

Request schema

1
2
type Prompt = LlmRequest & {
  model: string;
};

interface LlmRequest {
  messages: Message[];
  temperature?: number;
  max_completion_tokens?: number;
  top_p?: number;
  tools?: Tool[];
  tool_choice?: ToolChoice;
}
type Message = SystemMessage | UserMessage | AssistantMessage | ToolMessage;

interface SystemMessage {
  content: Content;
  role: "system";
}
interface UserMessage {
  content: Content;
  role: "user";
}
interface ToolMessage {
  content: Content;
  role: "tool";
  tool_call_id?: string;
  name?: string;
}
interface AssistantMessage {
  content: Content;
  role: "assistant";
  tool_calls?: ToolCall[];
}

interface ToolCall {
  id: string;
  type: "function";
  index: number;
  function: {
    name: string;
    arguments: object;
  };
}

type ToolChoice =
  | "auto"
  | "none"
  | "required"
  | {
      type: "function";
      function: {
        name: string;
      };
    };

interface Tool {
  type: "function";
  function: {
    name: string;
    description: string;
    parameters: Record<string, unknown>;
  };
}

type Content = string | ContentPart[];

type ContentPart = TextPart;

interface TextPart {
  type: "text";
  text: string;
}

The temperature and top_p fields are not supported by all models. Omit both fields when using a model that doesn't support them. For the list of affected models, see the validation rules.

Response schema

1
2
interface LlmResponse {
  choices: Choice[];
  usage?: Usage;
}

interface Choice {
  finish_reason: string;
  index?: number;
  message: AssistantMessage;
}

interface Usage {
  input_tokens?: number;
  output_tokens?: number;
  total_tokens?: number;
}

interface AssistantMessage {
  content: Content;
  role: "assistant";
  tool_calls?: ToolCall[];
}

interface StreamResponse extends AsyncIterable<LlmResponse> {
  close(): Promise<void> | undefined;
}

interface ModelListResponse {
  models: {
    model: string,
    status: "active" | "deprecated",
  }[];
}

Example: Using chat

1
2
import { chat } from "@forge/llm";
try {
  const response = await chat({
    model: "claude-opus-4-6",
    messages: [
      {
        role: "user",
        content: "Write a short poem about Forge LLMs.",
      },
    ],
  });

  console.log("#### LLM response:", JSON.stringify(response));
} catch (err) {
  console.error("#### LLM request failed:", {
    error: err.context?.responseText,
  });
  throw err;
}

Example: Using stream

1
2
import { stream } from "@forge/llm";
try {
  const response = await stream({
    model: "claude-opus-4-6",
    messages: [
      {
        role: "user",
        content: "Write a short poem about Forge LLMs.",
      },
    ],
  });

  for await (const chunk of response) {
    console.log("#### LLM response:", JSON.stringify(chunk));
  }

  response.close();
} catch (err) {
  console.error("#### LLM request failed:", {
    error: err.context?.responseText,
  });
  throw err;
}

Validation rules

The following request validation rules apply:

Parameters	Applies to	Rule
`temperature`, `top_p`	All models	`temperature` and `top_p` cannot be specified together. Provide only one, not both.
`temperature`, `top_p`	Selected models	Some models do not support these parameters. Omit both from requests to any model listed below.

Models that do not support `temperature` and `top_p`

The following models do not support the temperature and top_p sampling parameters. Omit both parameters from requests to these models:

claude-opus-4-7
claude-sonnet-5