mirror of
https://github.com/anomalyco/opencode.git
synced 2026-11-16 13:05:14 +03:00
refactor(ai): move speech, transcription, video and realtime to experimental (#60484)
This commit is contained in:
committed by
GitHub
Unverified
parent
e505852682
commit
ec64c1872e
+87
-71
@@ -1,6 +1,7 @@
|
||||
# @opencode/ai
|
||||
|
||||
Schema-first APIs for text, images, video, speech, transcription, and decisions, built with Effect.
|
||||
Schema-first APIs for text, images, and decisions, built with Effect. Video, speech, transcription, extraction, and
|
||||
realtime voice are [experimental](#experimental).
|
||||
|
||||
`@opencode/ai` is the Promise API: one managed runtime behind the LLM and media methods, plus asset helpers. `ai.file`
|
||||
and `ai.write` load `node:fs/promises` on first use, so no Effect `FileSystem` is needed:
|
||||
@@ -165,8 +166,8 @@ Raw usage is preserved in `usage.providerMetadata.google`. `inputTokens` follows
|
||||
`contextTokens` uses its full `raw_prompt_token` count when supplied. These can differ substantially with server-side
|
||||
continuation. Explicit caches, hosted tools, and generated media are not supported by this initial protocol.
|
||||
|
||||
The same configured facade names image, video, speech, and transcription models. `Image.generate` resolves the
|
||||
provider's image route from the model and returns `Media.Asset`s with lazily decoded bytes:
|
||||
The same configured facade names image models (and the experimental video, speech, and transcription models).
|
||||
`Image.generate` resolves the provider's image route from the model and returns `Media.Asset`s with lazily decoded bytes:
|
||||
|
||||
```ts
|
||||
import { NodeFileSystem } from "@effect/platform-node"
|
||||
@@ -309,38 +310,6 @@ CloudflareAIGateway.configure({ accountId }).decision("typesafe/jev")
|
||||
OpenAI and Vercel AI Gateway use OpenAI's `/v1/decisions` API. TypeSafe, OpenCode, OpenRouter, Baseten and
|
||||
DigitalOcean use System One (`/systemone`), and Cloudflare sends the System One body through `/ai/run`.
|
||||
|
||||
### Extraction (experimental)
|
||||
|
||||
Extraction models read typed values out of text instead of answering questions with probabilities. A field of type
|
||||
`text`, `number`, `date`, `time`, `datetime`, `duration`, `list`, `estimate` or `forecast` gets an answer typed to
|
||||
match; other types are sent and answered as-is. Only Cloudflare's Clairic serves it today, so it lives under
|
||||
`@opencode/ai/experimental`.
|
||||
|
||||
```ts
|
||||
import { ai } from "@opencode/ai"
|
||||
import { Extraction } from "@opencode/ai/experimental"
|
||||
import { CloudflareWorkersAI } from "@opencode/ai/providers"
|
||||
|
||||
const response = await ai.run(
|
||||
Extraction.generate({
|
||||
model: CloudflareWorkersAI.configure({ accountId }).experimental.extraction("@cf/cloudflare/clairic"),
|
||||
input: "Invoice INV-2291, issued 2026-09-02, net 30. Total $1,080.00. Contact: Dana Ruiz, Priya Shah.",
|
||||
fields: {
|
||||
invoiceNumber: { type: "text" },
|
||||
dueDate: { type: "date", instructions: "Issue date plus the net payment days" },
|
||||
total: { type: "number", decimals: 2, unit: "USD" },
|
||||
contacts: { type: "list", items: "text" },
|
||||
},
|
||||
}),
|
||||
)
|
||||
response.answers.total.value // 1080, or null when the input does not say
|
||||
```
|
||||
|
||||
Value answers carry `guess`, `nullProbability` and `confidence`; text answers carry a `span` into `input`. Estimates and
|
||||
forecasts return `median`, `p10` and `p90`. Field options other than `type`, `instructions` and `nullThreshold` go to
|
||||
Clairic as written. Effect users provide `RequestExecutor.fetchLayer` (or `AIClient.layer`). Images are not supported
|
||||
yet.
|
||||
|
||||
## Alibaba Cloud Model Studio
|
||||
|
||||
`Alibaba` provides standard Model Studio inference. Configure a region explicitly, then select
|
||||
@@ -910,7 +879,42 @@ const program = Effect.gen(function* () {
|
||||
|
||||
The hosted result is represented as a provider-executed tool call and a tool result whose content carries the generated image as a file. Gemini image-capable models instead emit a first-class `media` `LLMEvent` for inline image output (`response.message` then carries a `media` part). Retaining `response.message` preserves the generated image for continuation on both routes.
|
||||
|
||||
## Video generation
|
||||
## Public API
|
||||
|
||||
Everything below is exported from `@opencode/ai/effect` unless noted.
|
||||
|
||||
- **`LLM.request({...})`** — build a provider-neutral `LLMRequest`. Accepts ergonomic inputs (`system: string`, `prompt: string`) that normalize into the canonical Schema classes.
|
||||
- **`LLM.generate` / `LLM.stream`** — run direct input or an `LLMRequest` through `LLMClient` for one-import use.
|
||||
- **`LLM.loop`** — stream model calls, running `tool(...)` tools between them until the model stops calling tools.
|
||||
- **`Message.user(...)` / `Message.assistant(...)` / `Message.tool(...)`** — message constructors from the canonical schema model.
|
||||
- **`LanguageModel.make(...)` / `ToolCallPart.make(...)` / `ToolResultPart.make(...)` / `ToolDefinition.make(...)`** — model and tool-related constructors from the canonical schema model.
|
||||
- **`LLMEvent.is.*`** — typed guards (`is.textDelta`, `is.toolCall`, `is.finish`, …) for filtering streams.
|
||||
- **`Image.request` / `generate` / `stream` / `start` / `resume`** — images over inline, streaming (partial previews), and queued routes through a provider-neutral request and response model.
|
||||
- **`ImageClient`** — Effect service and layer for image execution, parallel to `LLMClient`.
|
||||
- **`Media`** — the shared asset type (`Media.Asset`, `Media.Source`) and constructors used by messages, tool results, and media requests.
|
||||
- **`Generation`** — provider-neutral handle for an in-flight media generation (`await`, `refresh`, `cancel`, `events`) used by queued media routes.
|
||||
- **`Decision.request` / `Decision.generate`** — typed predicate, choice, and score questions over text, JSON, and media input; `DecisionClient` is its Effect service and layer.
|
||||
- **`AIClient.layer` / `AIClient.layerWith(executor)`** — every modality client plus the request executor in one layer.
|
||||
- **`@opencode/ai`** (Promise root) — `AI.make({ layer?, clients? })` and a default `ai` client exposing `llm`, `image`, and `decision` (plus `experimental.video`, `experimental.speech`, and `experimental.transcription`) as Promise / `AsyncIterable` APIs, `run(effect)` for any package Effect, plus `file`, `write`, `bytes`, `base64`, and `materialize` for assets.
|
||||
- **`@opencode/ai/experimental`** — APIs that may change without compatibility guarantees: `Video`, `Speech`, `Transcription`, `Extraction`, and `Realtime`. See [Experimental](#experimental).
|
||||
|
||||
## Experimental
|
||||
|
||||
These APIs may change without compatibility guarantees. The Effect API is exported from `@opencode/ai/experimental`,
|
||||
the Promise client exposes it under `ai.experimental`, and test layers come from `@opencode/ai/experimental/testing`.
|
||||
|
||||
- **`Video`**, **`Speech`**, and **`Transcription`** — media generation and speech-to-text, with `VideoClient`,
|
||||
`SpeechClient`, and `TranscriptionClient` as their Effect services. Promise: `ai.experimental.video`, `.speech`, and
|
||||
`.transcription`. Testing: `TestVideo`, `TestSpeech`, and `TestTranscription`.
|
||||
- **`Extraction`** — typed values read out of text.
|
||||
- **`Realtime`** — `Realtime.connect` starts an OpenAI, xAI, or Gemini voice session and returns a WebRTC answer SDP or
|
||||
a short-lived WebSocket URL and token.
|
||||
|
||||
The provider selectors for these modalities (`.video(...)`, `.speech(...)`, `.transcription(...)`,
|
||||
`.experimental.extraction(...)`) and their protocol entrypoints (such as `@opencode/ai/protocols/openai-speech`) are
|
||||
part of the same experimental surface.
|
||||
|
||||
### Video generation
|
||||
|
||||
Video mirrors `Image` with one difference: every provider is asynchronous, so the route is a submit-then-poll
|
||||
`Generation`. Models come from `.video(...)` selectors on the `Google` (Veo), `XAI`, and `Runway` facades.
|
||||
@@ -919,7 +923,7 @@ Common fields (`frames`, `references`, `video`, `durationSeconds`, `aspect`, `re
|
||||
under `providerOptions`, inferred from the selected model.
|
||||
|
||||
```ts
|
||||
import { Video } from "@opencode/ai/effect"
|
||||
import { Video } from "@opencode/ai/experimental"
|
||||
import { Google, Runway } from "@opencode/ai/providers"
|
||||
|
||||
const google = Google.configure({ apiKey: process.env.GOOGLE_GENERATIVE_AI_API_KEY })
|
||||
@@ -981,9 +985,9 @@ Provider notes:
|
||||
- **Runway** expects pixel ratios in `aspect` for most models (`"1280:720"`), pins `X-Runway-Version`, reports
|
||||
`usage: { type: "credits" }`, and its output URLs expire after 24–48 hours.
|
||||
|
||||
The promise client exposes the same surface: `ai.video.start(...)` resolves to a handle with `await`, `events`,
|
||||
`result`, `refresh`, `cancel`, and `token`; `ai.video.generate`, `ai.video.resume(model, token)`, and
|
||||
`ai.video.stream` mirror the Effect API. The handle's `status` and `progress` are a snapshot from when it was
|
||||
The promise client exposes the same surface: `ai.experimental.video.start(...)` resolves to a handle with `await`, `events`,
|
||||
`result`, `refresh`, `cancel`, and `token`; `ai.experimental.video.generate`, `ai.experimental.video.resume(model, token)`, and
|
||||
`ai.experimental.video.stream` mirror the Effect API. The handle's `status` and `progress` are a snapshot from when it was
|
||||
created; `refresh()` resolves to a new handle. Every promise method and stream accepts `{ signal }`: like `fetch`,
|
||||
aborting rejects the Promise or throws from the `for await` loop with `signal.reason` (an `AbortError` `DOMException`
|
||||
unless `abort(reason)` passed one), while `break` stops a stream without throwing.
|
||||
@@ -991,13 +995,13 @@ unless `abort(reason)` passed one), while `break` stops a stream without throwin
|
||||
```ts
|
||||
import { ai } from "@opencode/ai"
|
||||
|
||||
const generation = await ai.video.start({ model, prompt })
|
||||
const generation = await ai.experimental.video.start({ model, prompt })
|
||||
for await (const event of generation.events({ poll: { interval: 10_000 } })) console.log(event.type)
|
||||
const video = await generation.result({ signal })
|
||||
await ai.write(video.video, "./kite.mp4")
|
||||
```
|
||||
|
||||
## Speech generation
|
||||
### Speech generation
|
||||
|
||||
Speech (text-to-speech) is one request whose response is parsed incrementally, so every route supports both
|
||||
`Speech.generate` (the whole file) and `Speech.stream` (audio chunks as they arrive). Models come from `.speech(...)`
|
||||
@@ -1006,7 +1010,8 @@ selectors on the `OpenAI`, `Google` (Gemini TTS), `ElevenLabs`, `Cartesia`, and
|
||||
before any network call; provider-native controls live under `providerOptions`, inferred from the selected model.
|
||||
|
||||
```ts
|
||||
import { Media, Speech, SpeechClient, SpeechEvent } from "@opencode/ai/effect"
|
||||
import { Media } from "@opencode/ai/effect"
|
||||
import { Speech, SpeechEvent } from "@opencode/ai/experimental"
|
||||
import { ElevenLabs, OpenAI } from "@opencode/ai/providers"
|
||||
|
||||
const openai = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY })
|
||||
@@ -1068,20 +1073,20 @@ Provider notes:
|
||||
and `providerOptions` lower to query parameters (`encoding`, `container`, `sample_rate`, `bit_rate`); `pcm` is
|
||||
`linear16` without a container. Auth is `Authorization: Token <DEEPGRAM_API_KEY>`.
|
||||
|
||||
The promise client mirrors the Effect API; `ai.speech.stream` is an `AsyncIterable`.
|
||||
The promise client mirrors the Effect API; `ai.experimental.speech.stream` is an `AsyncIterable`.
|
||||
|
||||
```ts
|
||||
import { ai } from "@opencode/ai"
|
||||
|
||||
const response = await ai.speech.generate({ model, text: "Hello from OpenCode.", voice: "coral" })
|
||||
const response = await ai.experimental.speech.generate({ model, text: "Hello from OpenCode.", voice: "coral" })
|
||||
await ai.write(response.audio, "hello.mp3")
|
||||
|
||||
for await (const event of ai.speech.stream({ model, text: "Hello from OpenCode.", voice: "coral" })) {
|
||||
for await (const event of ai.experimental.speech.stream({ model, text: "Hello from OpenCode.", voice: "coral" })) {
|
||||
if (event.type === "audio-delta") player.write(event.chunk)
|
||||
}
|
||||
```
|
||||
|
||||
## Transcription
|
||||
### Transcription
|
||||
|
||||
Transcription (speech-to-text) is the one modality whose providers use every route kind: OpenAI and Gemini stream,
|
||||
Deepgram and ElevenLabs answer inline, and AssemblyAI is queued. `Transcription.generate` and `Transcription.stream`
|
||||
@@ -1093,7 +1098,8 @@ any network call; a route may return more than asked.
|
||||
|
||||
```ts
|
||||
import { Console, Effect, Stream } from "effect"
|
||||
import { Media, Transcription, TranscriptionEvent } from "@opencode/ai/effect"
|
||||
import { Media } from "@opencode/ai/effect"
|
||||
import { Transcription, TranscriptionEvent } from "@opencode/ai/experimental"
|
||||
import { AssemblyAI, Deepgram, OpenAI } from "@opencode/ai/providers"
|
||||
|
||||
const openai = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY })
|
||||
@@ -1146,33 +1152,43 @@ The promise client mirrors the Effect API:
|
||||
|
||||
```ts
|
||||
const audio = await ai.file("./call.mp3")
|
||||
const text = (await ai.transcription.generate({ model, audio })).text
|
||||
for await (const event of ai.transcription.stream({ model, audio })) if (event.type === "text-delta") write(event.delta)
|
||||
const generation = await ai.transcription.start({ model: assemblyai, audio })
|
||||
const text = (await ai.experimental.transcription.generate({ model, audio })).text
|
||||
for await (const event of ai.experimental.transcription.stream({ model, audio }))
|
||||
if (event.type === "text-delta") write(event.delta)
|
||||
const generation = await ai.experimental.transcription.start({ model: assemblyai, audio })
|
||||
const transcript = await generation.await({ poll: { interval: 3_000 } })
|
||||
```
|
||||
|
||||
## Public API
|
||||
### Extraction
|
||||
|
||||
Everything below is exported from `@opencode/ai/effect` unless noted.
|
||||
Extraction models read typed values out of text instead of answering questions with probabilities. A field of type
|
||||
`text`, `number`, `date`, `time`, `datetime`, `duration`, `list`, `estimate` or `forecast` gets an answer typed to
|
||||
match; other types are sent and answered as-is. Only Cloudflare's Clairic serves it today.
|
||||
|
||||
- **`LLM.request({...})`** — build a provider-neutral `LLMRequest`. Accepts ergonomic inputs (`system: string`, `prompt: string`) that normalize into the canonical Schema classes.
|
||||
- **`LLM.generate` / `LLM.stream`** — run direct input or an `LLMRequest` through `LLMClient` for one-import use.
|
||||
- **`LLM.loop`** — stream model calls, running `tool(...)` tools between them until the model stops calling tools.
|
||||
- **`Message.user(...)` / `Message.assistant(...)` / `Message.tool(...)`** — message constructors from the canonical schema model.
|
||||
- **`LanguageModel.make(...)` / `ToolCallPart.make(...)` / `ToolResultPart.make(...)` / `ToolDefinition.make(...)`** — model and tool-related constructors from the canonical schema model.
|
||||
- **`LLMEvent.is.*`** — typed guards (`is.textDelta`, `is.toolCall`, `is.finish`, …) for filtering streams.
|
||||
- **`Image.request` / `generate` / `stream` / `start` / `resume`** — images over inline, streaming (partial previews), and queued routes through a provider-neutral request and response model.
|
||||
- **`ImageClient`** — Effect service and layer for image execution, parallel to `LLMClient`.
|
||||
- **`Media`** — the shared asset type (`Media.Asset`, `Media.Source`) and constructors used by messages, tool results, and media requests.
|
||||
- **`Generation`** — provider-neutral handle for an in-flight media generation (`await`, `refresh`, `cancel`, `events`) used by queued media routes.
|
||||
- **`Video.request` / `generate` / `stream` / `start` / `resume`** — queued video generation through a provider-neutral request; `VideoClient` is its Effect service and layer.
|
||||
- **`Speech.request` / `Speech.generate` / `Speech.stream`** — text-to-speech through a provider-neutral request; `SpeechClient` is its Effect service and layer.
|
||||
- **`Transcription.request` / `generate` / `stream` / `start` / `resume`** — speech-to-text over inline, streaming, and queued routes; `TranscriptionClient` is its Effect service and layer.
|
||||
- **`Decision.request` / `Decision.generate`** — typed predicate, choice, and score questions over text, JSON, and media input; `DecisionClient` is its Effect service and layer.
|
||||
- **`AIClient.layer` / `AIClient.layerWith(executor)`** — every modality client plus the request executor in one layer.
|
||||
- **`@opencode/ai`** (Promise root) — `AI.make({ layer?, clients? })` and a default `ai` client exposing `llm`, `image`, `video`, `speech`, `transcription`, and `decision` as Promise / `AsyncIterable` APIs, `run(effect)` for any package Effect, plus `file`, `write`, `bytes`, `base64`, and `materialize` for assets.
|
||||
- **`@opencode/ai/experimental`** — APIs that may still change: `Extraction` (typed values read out of text) and `Realtime` (`Realtime.connect` starts an OpenAI, xAI, or Gemini voice session and returns a WebRTC answer SDP or a short-lived WebSocket URL and token).
|
||||
```ts
|
||||
import { ai } from "@opencode/ai"
|
||||
import { Extraction } from "@opencode/ai/experimental"
|
||||
import { CloudflareWorkersAI } from "@opencode/ai/providers"
|
||||
|
||||
const response = await ai.run(
|
||||
Extraction.generate({
|
||||
model: CloudflareWorkersAI.configure({ accountId }).experimental.extraction("@cf/cloudflare/clairic"),
|
||||
input: "Invoice INV-2291, issued 2026-09-02, net 30. Total $1,080.00. Contact: Dana Ruiz, Priya Shah.",
|
||||
fields: {
|
||||
invoiceNumber: { type: "text" },
|
||||
dueDate: { type: "date", instructions: "Issue date plus the net payment days" },
|
||||
total: { type: "number", decimals: 2, unit: "USD" },
|
||||
contacts: { type: "list", items: "text" },
|
||||
},
|
||||
}),
|
||||
)
|
||||
response.answers.total.value // 1080, or null when the input does not say
|
||||
```
|
||||
|
||||
Value answers carry `guess`, `nullProbability` and `confidence`; text answers carry a `span` into `input`. Estimates and
|
||||
forecasts return `median`, `p10` and `p90`. Field options other than `type`, `instructions` and `nullThreshold` go to
|
||||
Clairic as written. Effect users provide `RequestExecutor.fetchLayer` (or `AIClient.layer`). Images are not supported
|
||||
yet.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -1218,7 +1234,7 @@ For explicit compaction, script a `CompactionResponse` through `push`, `always`,
|
||||
|
||||
For `compact(request, { mechanism: "trigger" })`, script a `CompactionCheckpointResponse` instead. It carries `checkpoint`, `responseID`, and optional `usage`. Endpoint and trigger calls reject each other's fixtures; both share the same queue, gates, lazy recording, and fallback controls.
|
||||
|
||||
`TestImage`, `TestVideo`, `TestSpeech`, `TestTranscription`, and `TestDecision` follow the same pattern: `testLayer()` provides the modality's client and `Test`, with the same `push`, `always`, `serve`, `requests`, `wait`, and `gate` controls. Script an `ImageResponse` (or the modality's response) or an `AIError`. `stream` expands the response into its events, and `start` returns an already finished generation that `resume` accepts by token. Decision responses are returned as scripted, without checking the answers against the questions.
|
||||
`TestImage` and `TestDecision` (and `TestVideo`, `TestSpeech`, and `TestTranscription` from `@opencode/ai/experimental/testing`) follow the same pattern: `testLayer()` provides the modality's client and `Test`, with the same `push`, `always`, `serve`, `requests`, `wait`, and `gate` controls. Script an `ImageResponse` (or the modality's response) or an `AIError`. `stream` expands the response into its events, and `start` returns an already finished generation that `resume` accepts by token. Decision responses are returned as scripted, without checking the answers against the questions.
|
||||
|
||||
The Promise client accepts the same layers as `AI.make({ clients: TestImage.testLayer() })`; `ai.run(TestImage.Test.use(...))` reaches the controls.
|
||||
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
"./effect": "./src/effect.ts",
|
||||
"./testing": "./src/testing.ts",
|
||||
"./experimental": "./src/experimental.ts",
|
||||
"./experimental/testing": "./src/experimental/testing.ts",
|
||||
"./route": "./src/route.ts",
|
||||
"./providers": "./src/providers.ts",
|
||||
"./providers/alibaba": "./src/providers/alibaba.ts",
|
||||
|
||||
@@ -116,63 +116,6 @@ export type {
|
||||
ImageRoute,
|
||||
} from "./image.js"
|
||||
export { Image } from "./image.js"
|
||||
export { VideoClient } from "./video-client.js"
|
||||
export {
|
||||
VideoAspectRatio,
|
||||
VideoEvent,
|
||||
VideoFrames,
|
||||
VideoModel,
|
||||
VideoModelSchema,
|
||||
VideoRequest,
|
||||
VideoResponse,
|
||||
} from "./video.js"
|
||||
export type {
|
||||
VideoModelOptions,
|
||||
VideoOptions,
|
||||
VideoRequestFor,
|
||||
VideoRequestInput,
|
||||
VideoResolution,
|
||||
VideoRoute,
|
||||
} from "./video.js"
|
||||
export { Video } from "./video.js"
|
||||
export { SpeechClient } from "./speech-client.js"
|
||||
export {
|
||||
SpeechEvent,
|
||||
SpeechModel,
|
||||
SpeechModelSchema,
|
||||
SpeechRequest,
|
||||
SpeechResponse,
|
||||
SpeechTimestamp,
|
||||
SpeechVoice,
|
||||
} from "./speech.js"
|
||||
export type {
|
||||
SpeechFormat,
|
||||
SpeechModelOptions,
|
||||
SpeechOptions,
|
||||
SpeechRequestFor,
|
||||
SpeechRequestInput,
|
||||
SpeechRoute,
|
||||
} from "./speech.js"
|
||||
export { Speech } from "./speech.js"
|
||||
export { TranscriptionClient } from "./transcription-client.js"
|
||||
export {
|
||||
TranscriptionEvent,
|
||||
TranscriptionModel,
|
||||
TranscriptionModelSchema,
|
||||
TranscriptionRequest,
|
||||
TranscriptionResponse,
|
||||
TranscriptionSegment,
|
||||
TranscriptionTimestamps,
|
||||
TranscriptionWord,
|
||||
} from "./transcription.js"
|
||||
export type {
|
||||
TranscriptionModelOptions,
|
||||
TranscriptionOptions,
|
||||
TranscriptionRequestFor,
|
||||
TranscriptionRequestInput,
|
||||
TranscriptionRoute,
|
||||
} from "./transcription.js"
|
||||
export { Transcription } from "./transcription.js"
|
||||
export { DecisionClient } from "./decision-client.js"
|
||||
export {
|
||||
ChoiceAnswer,
|
||||
|
||||
@@ -1,3 +1,60 @@
|
||||
export { VideoClient } from "./video-client.js"
|
||||
export {
|
||||
VideoAspectRatio,
|
||||
VideoEvent,
|
||||
VideoFrames,
|
||||
VideoModel,
|
||||
VideoModelSchema,
|
||||
VideoRequest,
|
||||
VideoResponse,
|
||||
} from "./video.js"
|
||||
export type {
|
||||
VideoModelOptions,
|
||||
VideoOptions,
|
||||
VideoRequestFor,
|
||||
VideoRequestInput,
|
||||
VideoResolution,
|
||||
VideoRoute,
|
||||
} from "./video.js"
|
||||
export { Video } from "./video.js"
|
||||
export { SpeechClient } from "./speech-client.js"
|
||||
export {
|
||||
SpeechEvent,
|
||||
SpeechModel,
|
||||
SpeechModelSchema,
|
||||
SpeechRequest,
|
||||
SpeechResponse,
|
||||
SpeechTimestamp,
|
||||
SpeechVoice,
|
||||
} from "./speech.js"
|
||||
export type {
|
||||
SpeechFormat,
|
||||
SpeechModelOptions,
|
||||
SpeechOptions,
|
||||
SpeechRequestFor,
|
||||
SpeechRequestInput,
|
||||
SpeechRoute,
|
||||
} from "./speech.js"
|
||||
export { Speech } from "./speech.js"
|
||||
export { TranscriptionClient } from "./transcription-client.js"
|
||||
export {
|
||||
TranscriptionEvent,
|
||||
TranscriptionModel,
|
||||
TranscriptionModelSchema,
|
||||
TranscriptionRequest,
|
||||
TranscriptionResponse,
|
||||
TranscriptionSegment,
|
||||
TranscriptionTimestamps,
|
||||
TranscriptionWord,
|
||||
} from "./transcription.js"
|
||||
export type {
|
||||
TranscriptionModelOptions,
|
||||
TranscriptionOptions,
|
||||
TranscriptionRequestFor,
|
||||
TranscriptionRequestInput,
|
||||
TranscriptionRoute,
|
||||
} from "./transcription.js"
|
||||
export { Transcription } from "./transcription.js"
|
||||
export { Extraction, ExtractionModel, ExtractionRequest, ExtractionResponse } from "./experimental/extraction.js"
|
||||
export type {
|
||||
ExtractionAnswerFor,
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
export { TestSpeech } from "../testing/speech.js"
|
||||
export { TestTranscription } from "../testing/transcription.js"
|
||||
export { TestVideo } from "../testing/video.js"
|
||||
+46
-43
@@ -230,49 +230,6 @@ export const make = <R = never>(options: Options<R> = {}) => {
|
||||
resume: (model: ImageModel, token: unknown, options?: RunOptions) =>
|
||||
run(Image.resume(model, token), options).then(handle),
|
||||
},
|
||||
video: {
|
||||
request: Video.request,
|
||||
start: <const Model extends VideoModel>(input: VideoRequestInput<Model> | VideoRequest, options?: RunOptions) =>
|
||||
run(Video.start(input), options).then(handle),
|
||||
generate: <const Model extends VideoModel>(
|
||||
input: VideoRequestInput<Model> | VideoRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => run(Video.generate(input, { poll: options?.poll }), options),
|
||||
resume: (model: VideoModel, token: unknown, options?: RunOptions) =>
|
||||
run(Video.resume(model, token), options).then(handle),
|
||||
stream: <const Model extends VideoModel>(
|
||||
input: VideoRequestInput<Model> | VideoRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => iterate(Video.stream(input, { poll: options?.poll }), options),
|
||||
},
|
||||
speech: {
|
||||
request: Speech.request,
|
||||
generate: <const Model extends SpeechModel>(
|
||||
input: SpeechRequestInput<Model> | SpeechRequest,
|
||||
options?: RunOptions,
|
||||
) => run(Speech.generate(input), options),
|
||||
stream: <const Model extends SpeechModel>(
|
||||
input: SpeechRequestInput<Model> | SpeechRequest,
|
||||
options?: RunOptions,
|
||||
) => iterate(Speech.stream(input), options),
|
||||
},
|
||||
transcription: {
|
||||
request: Transcription.request,
|
||||
generate: <const Model extends TranscriptionModel>(
|
||||
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => run(Transcription.generate(input, { poll: options?.poll }), options),
|
||||
stream: <const Model extends TranscriptionModel>(
|
||||
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => iterate(Transcription.stream(input, { poll: options?.poll }), options),
|
||||
start: <const Model extends TranscriptionModel>(
|
||||
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
|
||||
options?: RunOptions,
|
||||
) => run(Transcription.start(input), options).then(handle),
|
||||
resume: (model: TranscriptionModel, token: unknown, options?: RunOptions) =>
|
||||
run(Transcription.resume(model, token), options).then(handle),
|
||||
},
|
||||
decision: {
|
||||
request: Decision.request,
|
||||
generate: <const Model extends DecisionModel, const Questions extends DecisionQuestions>(
|
||||
@@ -280,6 +237,52 @@ export const make = <R = never>(options: Options<R> = {}) => {
|
||||
options?: RunOptions,
|
||||
) => run(Decision.generate(input), options),
|
||||
},
|
||||
/** Experimental: may change without compatibility guarantees. */
|
||||
experimental: {
|
||||
speech: {
|
||||
request: Speech.request,
|
||||
generate: <const Model extends SpeechModel>(
|
||||
input: SpeechRequestInput<Model> | SpeechRequest,
|
||||
options?: RunOptions,
|
||||
) => run(Speech.generate(input), options),
|
||||
stream: <const Model extends SpeechModel>(
|
||||
input: SpeechRequestInput<Model> | SpeechRequest,
|
||||
options?: RunOptions,
|
||||
) => iterate(Speech.stream(input), options),
|
||||
},
|
||||
transcription: {
|
||||
request: Transcription.request,
|
||||
generate: <const Model extends TranscriptionModel>(
|
||||
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => run(Transcription.generate(input, { poll: options?.poll }), options),
|
||||
stream: <const Model extends TranscriptionModel>(
|
||||
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => iterate(Transcription.stream(input, { poll: options?.poll }), options),
|
||||
start: <const Model extends TranscriptionModel>(
|
||||
input: TranscriptionRequestInput<Model> | TranscriptionRequest,
|
||||
options?: RunOptions,
|
||||
) => run(Transcription.start(input), options).then(handle),
|
||||
resume: (model: TranscriptionModel, token: unknown, options?: RunOptions) =>
|
||||
run(Transcription.resume(model, token), options).then(handle),
|
||||
},
|
||||
video: {
|
||||
request: Video.request,
|
||||
start: <const Model extends VideoModel>(input: VideoRequestInput<Model> | VideoRequest, options?: RunOptions) =>
|
||||
run(Video.start(input), options).then(handle),
|
||||
generate: <const Model extends VideoModel>(
|
||||
input: VideoRequestInput<Model> | VideoRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => run(Video.generate(input, { poll: options?.poll }), options),
|
||||
resume: (model: VideoModel, token: unknown, options?: RunOptions) =>
|
||||
run(Video.resume(model, token), options).then(handle),
|
||||
stream: <const Model extends VideoModel>(
|
||||
input: VideoRequestInput<Model> | VideoRequest,
|
||||
options?: AwaitOptions & RunOptions,
|
||||
) => iterate(Video.stream(input, { poll: options?.poll }), options),
|
||||
},
|
||||
},
|
||||
dispose: () => runtime.dispose(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,9 +2,6 @@ export * as TestLLM from "./testing.js"
|
||||
export { compileRequest } from "./route/client.js"
|
||||
export { TestDecision } from "./testing/decision.js"
|
||||
export { TestImage } from "./testing/image.js"
|
||||
export { TestSpeech } from "./testing/speech.js"
|
||||
export { TestTranscription } from "./testing/transcription.js"
|
||||
export { TestVideo } from "./testing/video.js"
|
||||
|
||||
import {
|
||||
LLMClient,
|
||||
|
||||
@@ -12,13 +12,8 @@ import {
|
||||
LLM,
|
||||
LLMClient,
|
||||
Media,
|
||||
Speech,
|
||||
SpeechClient,
|
||||
SpeechEvent,
|
||||
TranscriptionClient,
|
||||
Video,
|
||||
VideoClient,
|
||||
} from "@opencode/ai/effect"
|
||||
import { Speech, SpeechClient, SpeechEvent, TranscriptionClient, Video, VideoClient } from "@opencode/ai/experimental"
|
||||
import { Route, Protocol, RequestExecutor, WebSocketTransport } from "@opencode/ai/route"
|
||||
import {
|
||||
Alibaba,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { Effect, Stream } from "effect"
|
||||
import type { Client, GenerationHandle } from "../src/index.js"
|
||||
import type { Decision, Generation, Image, LLM, LLMClient } from "../src/effect.js"
|
||||
import type { Speech, Transcription, Video } from "../src/experimental.js"
|
||||
|
||||
// The Promise client mirrors the Effect API: same methods, same arguments, same results. The only allowed differences
|
||||
// are `signal` on the options bag, Promise for Effect, AsyncIterable for Stream and GenerationHandle for Generation.
|
||||
@@ -13,6 +14,9 @@ export type Surface = {
|
||||
readonly llm: Operations<typeof LLM> & Pick<typeof LLMClient, "compact" | "canCompact">
|
||||
readonly image: typeof Image
|
||||
readonly decision: typeof Decision
|
||||
readonly speech: typeof Speech
|
||||
readonly transcription: typeof Transcription
|
||||
readonly video: typeof Video
|
||||
}
|
||||
|
||||
type Operations<M> = { [K in keyof M as M[K] extends (...args: never) => unknown ? K : never]: M[K] }
|
||||
@@ -68,4 +72,7 @@ export type Checks = [
|
||||
Assert<Drift<Client["llm"], Surface["llm"]>>,
|
||||
Assert<Drift<Client["image"], Surface["image"]>>,
|
||||
Assert<Drift<Client["decision"], Surface["decision"]>>,
|
||||
Assert<Drift<Client["experimental"]["speech"], Surface["speech"]>>,
|
||||
Assert<Drift<Client["experimental"]["transcription"], Surface["transcription"]>>,
|
||||
Assert<Drift<Client["experimental"]["video"], Surface["video"]>>,
|
||||
]
|
||||
|
||||
@@ -4,7 +4,8 @@ import { tmpdir } from "node:os"
|
||||
import { join } from "node:path"
|
||||
import { Effect, Layer, Schema } from "effect"
|
||||
import { HttpClientRequest } from "effect/http"
|
||||
import { AIError, LLMEvent, Media, SpeechEvent, TranscriptionEvent } from "../src/effect.js"
|
||||
import { AIError, LLMEvent, Media } from "../src/effect.js"
|
||||
import { SpeechEvent, TranscriptionEvent } from "../src/experimental.js"
|
||||
import { RequestExecutor, type StreamOptions } from "../src/route.js"
|
||||
import { AI } from "../src/index.js"
|
||||
import { AssemblyAI, OpenAI, Runway } from "../src/providers.js"
|
||||
@@ -174,7 +175,7 @@ describe("AI promise client", () => {
|
||||
const ai = AI.make({ layer: executor(seen) })
|
||||
const model = Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5")
|
||||
|
||||
const generation = await ai.video.start({ model, prompt: "A kite" })
|
||||
const generation = await ai.experimental.video.start({ model, prompt: "A kite" })
|
||||
expect(generation.id).toBe("task_1")
|
||||
expect(generation.status).toBe("queued")
|
||||
expect(generation.token).toEqual({ taskID: "task_1" })
|
||||
@@ -188,12 +189,12 @@ describe("AI promise client", () => {
|
||||
expect(response.video.source).toMatchObject({ type: "url", url: "https://runway.test/out.mp4" })
|
||||
expect(response.usage).toEqual({ type: "credits", credits: 5 })
|
||||
|
||||
const resumed = await ai.video.resume(model, JSON.parse(JSON.stringify(generation.token)))
|
||||
const resumed = await ai.experimental.video.resume(model, JSON.parse(JSON.stringify(generation.token)))
|
||||
expect(resumed.status).toBe("completed")
|
||||
expect((await resumed.await()).videos).toHaveLength(1)
|
||||
|
||||
const events: Array<string> = []
|
||||
for await (const event of ai.video.stream({ model, prompt: "A kite" }, { poll: { interval: 10 } })) {
|
||||
for await (const event of ai.experimental.video.stream({ model, prompt: "A kite" }, { poll: { interval: 10 } })) {
|
||||
events.push(event.type)
|
||||
}
|
||||
expect(events).toEqual(["video", "finish"])
|
||||
@@ -229,7 +230,7 @@ describe("AI promise client", () => {
|
||||
test("exposes a generation's snapshot and terminal state", async () => {
|
||||
const ai = AI.make({ layer: executor([]) })
|
||||
const model = Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5")
|
||||
const generation = await ai.video.start({ model, prompt: "A kite" })
|
||||
const generation = await ai.experimental.video.start({ model, prompt: "A kite" })
|
||||
expect(generation.snapshot).toEqual({ id: "task_1", status: "queued", progress: undefined, position: undefined })
|
||||
expect(generation.terminal).toBe(false)
|
||||
const refreshed = await generation.refresh()
|
||||
@@ -241,7 +242,7 @@ describe("AI promise client", () => {
|
||||
test("observes a started generation's events and fetches its result", async () => {
|
||||
const ai = AI.make({ layer: executor([]) })
|
||||
const model = Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5")
|
||||
const generation = await ai.video.start({ model, prompt: "A kite" })
|
||||
const generation = await ai.experimental.video.start({ model, prompt: "A kite" })
|
||||
|
||||
const events: Array<string> = []
|
||||
for await (const event of generation.events({ poll: { interval: 10 } })) events.push(event.type)
|
||||
@@ -274,11 +275,16 @@ describe("AI promise client", () => {
|
||||
const bytes = Media.bytes(Uint8Array.from([0x49, 0x44, 0x33]), "audio/mpeg")
|
||||
|
||||
expect(
|
||||
(await ai.transcription.generate({ model: openai.transcription("gpt-4o-mini-transcribe"), audio: bytes })).text,
|
||||
(
|
||||
await ai.experimental.transcription.generate({
|
||||
model: openai.transcription("gpt-4o-mini-transcribe"),
|
||||
audio: bytes,
|
||||
})
|
||||
).text,
|
||||
).toBe("Hello there.")
|
||||
|
||||
const deltas: Array<string> = []
|
||||
for await (const event of ai.transcription.stream({
|
||||
for await (const event of ai.experimental.transcription.stream({
|
||||
model: openai.transcription("gpt-4o-mini-transcribe"),
|
||||
audio: bytes,
|
||||
}))
|
||||
@@ -288,7 +294,7 @@ describe("AI promise client", () => {
|
||||
const model = AssemblyAI.configure({ apiKey: "test", baseURL: "https://assemblyai.test" }).transcription(
|
||||
"universal-2",
|
||||
)
|
||||
const generation = await ai.transcription.start({ model, audio })
|
||||
const generation = await ai.experimental.transcription.start({ model, audio })
|
||||
expect(generation.token).toEqual({ transcriptID: "tr_1" })
|
||||
expect((await generation.await({ poll: { interval: 10 } })).text).toBe("Hello there.")
|
||||
await ai.dispose()
|
||||
@@ -298,11 +304,12 @@ describe("AI promise client", () => {
|
||||
const ai = AI.make({ layer: executor([]) })
|
||||
const model = openai.speech("gpt-4o-mini-tts")
|
||||
|
||||
const response = await ai.speech.generate({ model, text: "Hello", voice: "coral", format: "pcm" })
|
||||
const response = await ai.experimental.speech.generate({ model, text: "Hello", voice: "coral", format: "pcm" })
|
||||
expect(await ai.run(response.audio.bytes())).toEqual(Uint8Array.from([1, 2, 3]))
|
||||
|
||||
const events: Array<SpeechEvent> = []
|
||||
for await (const event of ai.speech.stream({ model, text: "Hello", voice: "coral" })) events.push(event)
|
||||
for await (const event of ai.experimental.speech.stream({ model, text: "Hello", voice: "coral" }))
|
||||
events.push(event)
|
||||
expect(events.map((event) => event.type)).toEqual(["audio-delta", "audio-delta", "finish"])
|
||||
const finish = events.find(SpeechEvent.is.finish)
|
||||
expect(await ai.run(finish!.audio.bytes())).toEqual(Uint8Array.from([1, 2, 3]))
|
||||
@@ -361,7 +368,10 @@ describe("AI promise client", () => {
|
||||
).toBe(reason)
|
||||
|
||||
const preAborted = await Array.fromAsync(
|
||||
ai.speech.stream({ model: openai.speech("gpt-4o-mini-tts"), text: "Hello" }, { signal: aborted.signal }),
|
||||
ai.experimental.speech.stream(
|
||||
{ model: openai.speech("gpt-4o-mini-tts"), text: "Hello" },
|
||||
{ signal: aborted.signal },
|
||||
),
|
||||
).catch((error: unknown) => error)
|
||||
expect(preAborted).toBe(aborted.signal.reason)
|
||||
expect(preAborted).toMatchObject({ name: "AbortError" })
|
||||
@@ -381,7 +391,7 @@ describe("AI promise client", () => {
|
||||
expect(midStreamFailure).toMatchObject({ name: "AbortError" })
|
||||
|
||||
const model = Runway.configure({ apiKey: "test", baseURL: "https://runway.test/v1" }).video("gen4.5")
|
||||
const generation = await ai.video.start({ model, prompt: "A kite" })
|
||||
const generation = await ai.experimental.video.start({ model, prompt: "A kite" })
|
||||
const polling = new AbortController()
|
||||
const events: Array<string> = []
|
||||
const eventsFailure = await Array.fromAsync(
|
||||
@@ -544,8 +554,8 @@ describe("AI promise client", () => {
|
||||
test("the default client is created lazily and can be disposed", async () => {
|
||||
expect(typeof AI.ai.llm.generate).toBe("function")
|
||||
expect(typeof AI.ai.image.generate).toBe("function")
|
||||
expect(typeof AI.ai.video.start).toBe("function")
|
||||
expect(typeof AI.ai.speech.generate).toBe("function")
|
||||
expect(typeof AI.ai.experimental.video.start).toBe("function")
|
||||
expect(typeof AI.ai.experimental.speech.generate).toBe("function")
|
||||
await AI.ai.dispose()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Transcription } from "../../src/effect.js"
|
||||
import { Transcription } from "../../src/experimental.js"
|
||||
import { AssemblyAI } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TRANSCRIPT, audio, audioRecording } from "./transcription-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Speech, SpeechEvent } from "../../src/effect.js"
|
||||
import { Speech, SpeechEvent } from "../../src/experimental.js"
|
||||
import { Cartesia } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TEXT, collectSpeech } from "./speech-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Speech } from "../../src/effect.js"
|
||||
import { Speech } from "../../src/experimental.js"
|
||||
import { Deepgram } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TEXT, collectSpeech } from "./speech-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Stream } from "effect"
|
||||
import { Transcription } from "../../src/effect.js"
|
||||
import { Transcription } from "../../src/experimental.js"
|
||||
import { Deepgram } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TRANSCRIPT, audio, audioRecording, dialog } from "./transcription-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Speech, SpeechEvent } from "../../src/effect.js"
|
||||
import { Speech, SpeechEvent } from "../../src/experimental.js"
|
||||
import { ElevenLabs } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TEXT, collectSpeech } from "./speech-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Stream } from "effect"
|
||||
import { Transcription } from "../../src/effect.js"
|
||||
import { Transcription } from "../../src/experimental.js"
|
||||
import { ElevenLabs } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TRANSCRIPT, audio, audioRecording, dialog } from "./transcription-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Speech } from "../../src/effect.js"
|
||||
import { Speech } from "../../src/experimental.js"
|
||||
import { Google } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TEXT, collectSpeech } from "./speech-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Stream } from "effect"
|
||||
import { Transcription, TranscriptionEvent } from "../../src/effect.js"
|
||||
import { Transcription, TranscriptionEvent } from "../../src/experimental.js"
|
||||
import { Google } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TRANSCRIPT, audio, audioRecording, dialog } from "./transcription-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Video } from "../../src/effect.js"
|
||||
import { Video } from "../../src/experimental.js"
|
||||
import { Google } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { queuedPoll } from "./queued-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Speech } from "../../src/effect.js"
|
||||
import { Speech } from "../../src/experimental.js"
|
||||
import { OpenAI } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TEXT, collectSpeech } from "./speech-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Stream } from "effect"
|
||||
import { Transcription, TranscriptionEvent } from "../../src/effect.js"
|
||||
import { Transcription, TranscriptionEvent } from "../../src/experimental.js"
|
||||
import { OpenAI } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { TRANSCRIPT, audio, audioRecording } from "./transcription-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect } from "effect"
|
||||
import { Video } from "../../src/effect.js"
|
||||
import { Video } from "../../src/experimental.js"
|
||||
import { Runway } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { queuedPoll } from "./queued-recording.js"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { expect } from "bun:test"
|
||||
import { Effect, Stream } from "effect"
|
||||
import { SpeechEvent } from "../../src/effect.js"
|
||||
import { SpeechEvent } from "../../src/experimental.js"
|
||||
|
||||
export const TEXT = "Hello from OpenCode."
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Stream } from "effect"
|
||||
import { Video } from "../../src/effect.js"
|
||||
import { Video } from "../../src/experimental.js"
|
||||
import { XAI } from "../../src/providers.js"
|
||||
import { recordedTests } from "../recorded-test.js"
|
||||
import { queuedPoll } from "./queued-recording.js"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Layer, Stream } from "effect"
|
||||
import { Base64 } from "effect/encoding"
|
||||
import { Speech, SpeechClient, SpeechEvent } from "../src/effect.js"
|
||||
import { Speech, SpeechClient, SpeechEvent } from "../src/experimental.js"
|
||||
import { Cartesia, Deepgram, ElevenLabs, Google, OpenAI } from "../src/providers.js"
|
||||
import { it } from "./lib/effect.js"
|
||||
import { dynamicResponse } from "./lib/http.js"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { Stream } from "effect"
|
||||
import { Speech, SpeechModel, type SpeechEvent, type SpeechOptions } from "../src/effect.js"
|
||||
import { Speech, SpeechModel, type SpeechEvent, type SpeechOptions } from "../src/experimental.js"
|
||||
import { ElevenLabs, OpenAI, Runway } from "../src/providers.js"
|
||||
|
||||
type StreamItem<T> = T extends Stream.Stream<infer A, infer _E, infer _R> ? A : never
|
||||
|
||||
@@ -1,23 +1,18 @@
|
||||
import { describe, expect, test } from "bun:test"
|
||||
import { Effect, Layer, Stream } from "effect"
|
||||
import { AIError, Decision, DecisionResponse, Image, ImageResponse, Media, RateLimitError } from "../src/effect.js"
|
||||
import {
|
||||
AIError,
|
||||
Decision,
|
||||
DecisionResponse,
|
||||
Image,
|
||||
ImageResponse,
|
||||
Media,
|
||||
RateLimitError,
|
||||
Speech,
|
||||
SpeechResponse,
|
||||
Transcription,
|
||||
TranscriptionResponse,
|
||||
Video,
|
||||
VideoResponse,
|
||||
} from "../src/effect.js"
|
||||
} from "../src/experimental.js"
|
||||
import { AI } from "../src/index.js"
|
||||
import { OpenAI, Runway } from "../src/providers.js"
|
||||
import { TestDecision, TestImage, TestSpeech, TestTranscription, TestVideo } from "../src/testing.js"
|
||||
import { TestDecision, TestImage } from "../src/testing.js"
|
||||
import { TestSpeech, TestTranscription, TestVideo } from "../src/experimental/testing.js"
|
||||
import { testEffect } from "./lib/effect.js"
|
||||
|
||||
const openai = OpenAI.configure({ apiKey: "test" })
|
||||
@@ -113,13 +108,17 @@ describe("media test clients", () => {
|
||||
)
|
||||
|
||||
expect((await ai.image.generate({ model: openai.image("gpt-image-2"), prompt: "A lighthouse" })).image).toBe(png)
|
||||
const generation = await ai.video.start({ model: runway, prompt: "A kite" })
|
||||
expect((await ai.video.resume(runway, generation.token).then((resumed) => resumed.await())).video).toBe(mp4)
|
||||
const generation = await ai.experimental.video.start({ model: runway, prompt: "A kite" })
|
||||
expect(
|
||||
await ai.bytes((await ai.speech.generate({ model: openai.speech("gpt-4o-mini-tts"), text: "Hi" })).audio),
|
||||
(await ai.experimental.video.resume(runway, generation.token).then((resumed) => resumed.await())).video,
|
||||
).toBe(mp4)
|
||||
expect(
|
||||
await ai.bytes(
|
||||
(await ai.experimental.speech.generate({ model: openai.speech("gpt-4o-mini-tts"), text: "Hi" })).audio,
|
||||
),
|
||||
).toEqual(Uint8Array.from([4, 5, 6]))
|
||||
const deltas = await Array.fromAsync(
|
||||
ai.transcription.stream({ model: openai.transcription("gpt-4o-mini-transcribe"), audio }),
|
||||
ai.experimental.transcription.stream({ model: openai.transcription("gpt-4o-mini-transcribe"), audio }),
|
||||
)
|
||||
expect(deltas.map((event) => event.type)).toEqual(["finish"])
|
||||
const decided = await ai.decision.generate({
|
||||
|
||||
@@ -2,7 +2,8 @@ import { describe, expect } from "bun:test"
|
||||
import { Effect, Fiber, Layer, Stream } from "effect"
|
||||
import * as TestClock from "effect/testing/TestClock"
|
||||
import { HttpClientRequest } from "effect/http"
|
||||
import { Media, Transcription, TranscriptionClient, type TranscriptionEvent } from "../src/effect.js"
|
||||
import { Media } from "../src/effect.js"
|
||||
import { Transcription, TranscriptionClient, type TranscriptionEvent } from "../src/experimental.js"
|
||||
import { AssemblyAI, Deepgram, ElevenLabs, Google, OpenAI } from "../src/providers.js"
|
||||
import { it } from "./lib/effect.js"
|
||||
import { dynamicResponse, json, observe, type Call } from "./lib/http.js"
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
import type { Stream } from "effect"
|
||||
import { Media } from "../src/effect.js"
|
||||
import {
|
||||
Media,
|
||||
Transcription,
|
||||
TranscriptionModel,
|
||||
type TranscriptionEvent,
|
||||
type TranscriptionOptions,
|
||||
} from "../src/effect.js"
|
||||
} from "../src/experimental.js"
|
||||
import { AssemblyAI, Deepgram, OpenAI } from "../src/providers.js"
|
||||
|
||||
type StreamItem<T> = T extends Stream.Stream<infer A, infer _E, infer _R> ? A : never
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import { describe, expect } from "bun:test"
|
||||
import { Effect, Fiber, Layer, Stream } from "effect"
|
||||
import * as TestClock from "effect/testing/TestClock"
|
||||
import { Media, Video, VideoClient, type GenerationEvent, type VideoEvent } from "../src/effect.js"
|
||||
import { Media, type GenerationEvent } from "../src/effect.js"
|
||||
import { Video, VideoClient, type VideoEvent } from "../src/experimental.js"
|
||||
import { Google, Runway, XAI } from "../src/providers.js"
|
||||
import { it } from "./lib/effect.js"
|
||||
import { dynamicResponse, json, observe, settle, type Call, type HandlerInput } from "./lib/http.js"
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { Effect } from "effect"
|
||||
import { Generation, Media } from "../src/effect.js"
|
||||
import {
|
||||
Generation,
|
||||
Media,
|
||||
Video,
|
||||
VideoClient,
|
||||
VideoModel,
|
||||
@@ -9,7 +8,7 @@ import {
|
||||
type VideoModelOptions,
|
||||
type VideoOptions,
|
||||
type VideoRequestFor,
|
||||
} from "../src/effect.js"
|
||||
} from "../src/experimental.js"
|
||||
import type { Service } from "../src/video-client.js"
|
||||
import { Anthropic, Google, OpenAI, Runway, XAI } from "../src/providers.js"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user