Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c93f393072 |
No files matched your search
@@ -1,79 +1,41 @@
|
|||||||
# 1-h Tools (`1h-tools`)
|
# OpenClaw Tools (`openclaw-plugin-tools`)
|
||||||
|
|
||||||
OpenClaw 统一私有扩展插件集合,聚合了自定义 Agent 工具、媒体生成驱动(视频/图像,支持 OpenAI 与 Gemini 双协议)以及内置技能库。
|
Collection of custom tools and bundled skills for OpenClaw.
|
||||||
|
|
||||||
各模块采用**就近同构原则(Co-location)**维护其独立代码、测试与文档,避免全局单页膨胀与文档漂移。
|
## Step-by-Step Construction History
|
||||||
|
|
||||||
---
|
1. **Step 1 — Initialize Scaffold (`chore: openclaw plugins init tools`)**
|
||||||
|
Generated the standard plugin skeleton via `openclaw plugins init tools`.
|
||||||
|
2. **Step 2 — Compile & Build Metadata (`build: compile TypeScript to dist and add .gitignore`)**
|
||||||
|
Compiled `src/index.ts` into `dist/index.js` and ran `openclaw plugins build --entry ./dist/index.js` so downstream installs do not require a local build step.
|
||||||
|
3. **Step 3 — Enable Multi-Skill Bundling (`feat: enable bundled skills directory in openclaw.plugin.json`)**
|
||||||
|
Added `"skills": ["./skills"]` to `openclaw.plugin.json` and `"skills"` to `package.json#files` so OpenClaw discovers all skills placed under `skills/<name>/SKILL.md`.
|
||||||
|
4. **Step 4 — Bundle First Skill (`feat(skills): bundle bitwarden agent-access skill`)**
|
||||||
|
Added `skills/agent-access/SKILL.md` for Bitwarden Agent Access (`aac`).
|
||||||
|
5. **Step 5 — Documentation (`docs: document step-by-step plugin construction and usage`)**
|
||||||
|
Documented project layout and installation workflow.
|
||||||
|
|
||||||
## 包含模块清单
|
## Project Structure
|
||||||
|
|
||||||
| 模块类别 | 标识符 (ID) | 契约类型 | 协议支持 | 说明 | 详细文档 |
|
|
||||||
| :--- | :--- | :--- | :--- | :--- | :--- |
|
|
||||||
| **视频生成** | `1h-openai-video` | `videoGenerationProviders` | OpenAI `/v1/videos` | OpenAI 格式视频生成驱动(支持自建网关 / gemini-web2api / Sora 协议) | [查看文档 →](src/video/README.md) |
|
|
||||||
| **图像生成 (OpenAI)** | `1h-openai-image` | `imageGenerationProviders` | OpenAI `/v1/images` | OpenAI 格式图像生成驱动(支持 gpt-image-2 等,修复 16:9 尺寸 Bug 与 URL 模式) | [查看文档 →](src/image/README.md) |
|
|
||||||
| **图像生成 (Gemini)** | `1h-gemini-image` | `imageGenerationProviders` | Gemini `:generateContent` | Gemini 原生图像生成驱动(专职适配 gemini-3.1-flash-image,秒级出图) | [查看文档 →](src/image/README.md) |
|
|
||||||
| **基础工具** | `echo` | `tools` | Tool Plugin | 基础测试与连通性回显工具 | [查看文档 →](src/tools/README.md) |
|
|
||||||
| **内置技能** | `agent-access` | `skills` | OpenClaw Skill | Bitwarden Agent Access 自动化技能 | [查看技能 →](skills/agent-access/SKILL.md) |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 安装与更新
|
|
||||||
|
|
||||||
### 安装插件
|
|
||||||
在 OpenClaw Gateway 主机上执行:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
openclaw plugins install git:https://git.1-h.cc/OpenClaw/openclaw-plugin-tools.git --force --acknowledge-install-policy-warning --accept-capabilities
|
|
||||||
```
|
|
||||||
|
|
||||||
### 插件更新
|
|
||||||
当 Gitea 仓库有新提交时,在终端执行:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
openclaw plugins update 1h-tools
|
|
||||||
```
|
|
||||||
|
|
||||||
### 开发与验证
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 运行单元测试
|
|
||||||
pnpm test
|
|
||||||
|
|
||||||
# 编译 TypeScript 产物
|
|
||||||
pnpm run build
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 项目结构
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
openclaw-plugin-tools/
|
openclaw-plugin-tools/
|
||||||
├── openclaw.plugin.json # 插件统一清单与契约配置
|
├── openclaw.plugin.json # Plugin manifest (declares tools & skills dir)
|
||||||
├── package.json # 包元数据与入口定义
|
├── package.json # Package metadata & entrypoints
|
||||||
├── README.md # 根目录索引导航
|
|
||||||
├── src/
|
├── src/
|
||||||
│ ├── index.ts # 插件注册统一入口
|
│ ├── index.ts # Tool plugin TypeScript source
|
||||||
│ ├── index.test.ts # 统一注册测试
|
│ └── index.test.ts # Unit tests
|
||||||
│ ├── types.d.ts # Plugin SDK 补充类型定义
|
├── dist/ # Pre-compiled JS artifacts
|
||||||
│ ├── tools/ # 工具模块集合
|
└── skills/ # Bundled skills directory
|
||||||
│ │ ├── echo.ts
|
|
||||||
│ │ └── README.md # 📍 工具专属说明
|
|
||||||
│ ├── video/ # 视频生成驱动模块
|
|
||||||
│ │ ├── openai.ts # 1h-openai-video 驱动实现
|
|
||||||
│ │ └── README.md # 📍 视频驱动专属说明
|
|
||||||
│ └── image/ # 图像生成驱动模块集合
|
|
||||||
│ ├── openai.ts # 1h-openai-image 驱动实现
|
|
||||||
│ ├── gemini.ts # 1h-gemini-image 驱动实现
|
|
||||||
│ └── README.md # 📍 图像驱动专属说明
|
|
||||||
└── skills/ # 官方规范就近内置技能
|
|
||||||
└── agent-access/
|
└── agent-access/
|
||||||
└── SKILL.md # 📍 技能自描述文件
|
└── SKILL.md # Bitwarden Agent Access skill
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
## Installation & Update
|
||||||
|
|
||||||
## License
|
```bash
|
||||||
|
# Install from Gitea
|
||||||
|
openclaw plugins install git:git.1-h.cc/OpenClaw/openclaw-plugin-tools --force
|
||||||
|
|
||||||
[MIT](LICENSE)
|
# Validate plugin
|
||||||
|
npm run plugin:validate
|
||||||
|
```
|
||||||
Vendored
-21
@@ -1,21 +0,0 @@
|
|||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import type { ImageGenerationProvider, ImageGenerationRequest } from "openclaw/plugin-sdk/image-generation";
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
export interface GeminiImagePluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
}
|
|
||||||
export declare function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined;
|
|
||||||
export declare function resolveGeminiImageConfig(cfg: ImageGenerationRequest["cfg"] | undefined, apiPluginConfig?: Record<string, unknown>): GeminiImagePluginConfig | undefined;
|
|
||||||
export declare function resolveConfiguredGeminiImageBaseUrl(pluginCfg?: GeminiImagePluginConfig): string;
|
|
||||||
export declare function buildGeminiImageGenerationProvider({ isProviderApiKeyConfiguredAsync, getPluginConfig, }: {
|
|
||||||
isProviderApiKeyConfiguredAsync: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfiguredAsync"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): ImageGenerationProvider;
|
|
||||||
Vendored
-197
@@ -1,197 +0,0 @@
|
|||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
const DEFAULT_GEMINI_IMAGE_BASE_URL = "http://host.incus.internal:8080/v1beta";
|
|
||||||
const DEFAULT_GEMINI_IMAGE_MODEL = "gemini-3.1-flash-image";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
export function resolvePluginApiKey(rawKey) {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
export function resolveGeminiImageConfig(cfg, apiPluginConfig) {
|
|
||||||
const toolsConfig = (apiPluginConfig ?? cfg?.plugins?.entries?.["1h-tools"]?.config);
|
|
||||||
return toolsConfig?.geminiImage ?? toolsConfig?.image;
|
|
||||||
}
|
|
||||||
export function resolveConfiguredGeminiImageBaseUrl(pluginCfg) {
|
|
||||||
return (normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.GEMINI_IMAGE_BASE_URL) ??
|
|
||||||
DEFAULT_GEMINI_IMAGE_BASE_URL);
|
|
||||||
}
|
|
||||||
export function buildGeminiImageGenerationProvider({ isProviderApiKeyConfiguredAsync, getPluginConfig, }) {
|
|
||||||
return {
|
|
||||||
id: "1h-gemini-image",
|
|
||||||
label: "1-h Gemini Image",
|
|
||||||
defaultModel: DEFAULT_GEMINI_IMAGE_MODEL,
|
|
||||||
models: [DEFAULT_GEMINI_IMAGE_MODEL],
|
|
||||||
isConfiguredAsync: async (ctx) => {
|
|
||||||
const pluginCfg = resolveGeminiImageConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.GEMINI_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY)) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return await isProviderApiKeyConfiguredAsync({
|
|
||||||
provider: "google",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxCount: 1,
|
|
||||||
supportsSize: false,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: false,
|
|
||||||
},
|
|
||||||
edit: {
|
|
||||||
enabled: true,
|
|
||||||
maxCount: 1,
|
|
||||||
maxInputImages: 3,
|
|
||||||
supportsSize: false,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: false,
|
|
||||||
},
|
|
||||||
geometry: {
|
|
||||||
aspectRatios: ["1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3", "21:9"],
|
|
||||||
},
|
|
||||||
output: {
|
|
||||||
formats: ["jpeg", "png"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateImage(req) {
|
|
||||||
const pluginCfg = resolveGeminiImageConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey = resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.GEMINI_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY);
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } = await import("openclaw/plugin-sdk/provider-auth-runtime");
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "google",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
});
|
|
||||||
if (!auth.apiKey) {
|
|
||||||
throw new Error("Gemini Image API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
const [{ assertOkOrThrowHttpError, createProviderOperationDeadline, postJsonRequest, readProviderJsonResponse, resolveProviderHttpRequestConfig, resolveProviderOperationTimeoutMs, sanitizeConfiguredModelProviderRequest, },] = await Promise.all([import("openclaw/plugin-sdk/provider-http")]);
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "Gemini Image Generation",
|
|
||||||
});
|
|
||||||
const rawBaseUrl = resolveConfiguredGeminiImageBaseUrl(pluginCfg);
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } = resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: rawBaseUrl,
|
|
||||||
defaultBaseUrl: DEFAULT_GEMINI_IMAGE_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
},
|
|
||||||
provider: "google",
|
|
||||||
capability: "image",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
const model = normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_GEMINI_IMAGE_MODEL;
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
const parts = [];
|
|
||||||
const inputImages = req.inputImages ?? [];
|
|
||||||
for (const img of inputImages) {
|
|
||||||
parts.push({
|
|
||||||
inlineData: {
|
|
||||||
mimeType: img.mimeType || "image/jpeg",
|
|
||||||
data: img.buffer.toString("base64"),
|
|
||||||
},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
parts.push({
|
|
||||||
text: req.prompt,
|
|
||||||
});
|
|
||||||
const jsonBody = {
|
|
||||||
contents: [
|
|
||||||
{
|
|
||||||
parts,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
generationConfig: {
|
|
||||||
responseMimeType: "image/jpeg",
|
|
||||||
},
|
|
||||||
};
|
|
||||||
const normalizedBaseUrl = baseUrl.endsWith("/") ? baseUrl.slice(0, -1) : baseUrl;
|
|
||||||
const url = `${normalizedBaseUrl}/models/${model}:generateContent`;
|
|
||||||
const { response, release } = await postJsonRequest({
|
|
||||||
url,
|
|
||||||
headers,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
});
|
|
||||||
let payload;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "Gemini image generation request failed");
|
|
||||||
payload = await readProviderJsonResponse(response, "Gemini image generation failed");
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
if (payload.error?.message) {
|
|
||||||
throw new Error(`Gemini image generation error: ${payload.error.message}`);
|
|
||||||
}
|
|
||||||
const candidateParts = payload.candidates?.[0]?.content?.parts ?? [];
|
|
||||||
const generatedImages = [];
|
|
||||||
for (let i = 0; i < candidateParts.length; i++) {
|
|
||||||
const part = candidateParts[i];
|
|
||||||
if (part.inlineData?.data) {
|
|
||||||
const buffer = Buffer.from(part.inlineData.data, "base64");
|
|
||||||
const mimeType = part.inlineData.mimeType || "image/jpeg";
|
|
||||||
const ext = mimeType.includes("png") ? "png" : "jpg";
|
|
||||||
generatedImages.push({
|
|
||||||
buffer,
|
|
||||||
mimeType,
|
|
||||||
fileName: `image-${i + 1}.${ext}`,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (generatedImages.length === 0) {
|
|
||||||
const textMessage = candidateParts.find((p) => p.text)?.text;
|
|
||||||
throw new Error(textMessage
|
|
||||||
? `Gemini returned text instead of an image: ${textMessage}`
|
|
||||||
: "Gemini image generation returned no image data");
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
images: generatedImages,
|
|
||||||
model,
|
|
||||||
metadata: {
|
|
||||||
finishReason: payload.candidates?.[0]?.finishReason,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
Vendored
-27
@@ -1,27 +0,0 @@
|
|||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import type { ImageGenerationProvider, ImageGenerationRequest } from "openclaw/plugin-sdk/image-generation";
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
export interface ImagePluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
responseFormat?: "url" | "b64_json";
|
|
||||||
}
|
|
||||||
export declare function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined;
|
|
||||||
export declare function resolveImageConfig(cfg: ImageGenerationRequest["cfg"] | undefined, apiPluginConfig?: Record<string, unknown>): ImagePluginConfig | undefined;
|
|
||||||
export declare function resolveConfiguredImageBaseUrl(cfg: ImageGenerationRequest["cfg"], pluginCfg?: ImagePluginConfig): string;
|
|
||||||
export declare function resolveImageSize(params: {
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
model?: string;
|
|
||||||
}): string;
|
|
||||||
export declare function buildOpenAIImageGenerationProvider({ isProviderApiKeyConfigured, getPluginConfig, }: {
|
|
||||||
isProviderApiKeyConfigured: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfigured"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): ImageGenerationProvider;
|
|
||||||
Vendored
-306
@@ -1,306 +0,0 @@
|
|||||||
import { bufferToBlobPart } from "openclaw/plugin-sdk/blob-runtime";
|
|
||||||
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
const DEFAULT_OPENAI_IMAGE_BASE_URL = "https://api.openai.com/v1";
|
|
||||||
const DEFAULT_OPENAI_IMAGE_MODEL = "gpt-image-2";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
const DEFAULT_SIZE = "1024x1024";
|
|
||||||
const SUPPORTED_SIZES = [
|
|
||||||
"1024x1024",
|
|
||||||
"1536x1024",
|
|
||||||
"1024x1536",
|
|
||||||
"1792x1024",
|
|
||||||
"1024x1792",
|
|
||||||
"2048x2048",
|
|
||||||
"2048x1152",
|
|
||||||
"3840x2160",
|
|
||||||
];
|
|
||||||
const SUPPORTED_ASPECT_RATIOS = [
|
|
||||||
"1:1",
|
|
||||||
"16:9",
|
|
||||||
"9:16",
|
|
||||||
"4:3",
|
|
||||||
"3:4",
|
|
||||||
"3:2",
|
|
||||||
"2:3",
|
|
||||||
"21:9",
|
|
||||||
];
|
|
||||||
export function resolvePluginApiKey(rawKey) {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
export function resolveImageConfig(cfg, apiPluginConfig) {
|
|
||||||
const direct = apiPluginConfig?.image;
|
|
||||||
const fromCfg = cfg?.plugins?.entries?.["1h-tools"]?.config?.image;
|
|
||||||
return direct ?? fromCfg;
|
|
||||||
}
|
|
||||||
export function resolveConfiguredImageBaseUrl(cfg, pluginCfg) {
|
|
||||||
return (normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_BASE_URL) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_BASE_URL) ??
|
|
||||||
normalizeOptionalString(cfg?.models?.providers?.openai?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_BASE_URL) ??
|
|
||||||
DEFAULT_OPENAI_IMAGE_BASE_URL);
|
|
||||||
}
|
|
||||||
export function resolveImageSize(params) {
|
|
||||||
const explicitSize = normalizeOptionalString(params.size);
|
|
||||||
if (explicitSize) {
|
|
||||||
return explicitSize;
|
|
||||||
}
|
|
||||||
// Safe aspect ratio normalization preventing issue #168665 (proxy rejecting 2048x1152)
|
|
||||||
switch (normalizeOptionalString(params.aspectRatio)) {
|
|
||||||
case "1:1":
|
|
||||||
return "1024x1024";
|
|
||||||
case "16:9":
|
|
||||||
return "1792x1024";
|
|
||||||
case "9:16":
|
|
||||||
return "1024x1792";
|
|
||||||
case "4:3":
|
|
||||||
case "3:2":
|
|
||||||
return "1536x1024";
|
|
||||||
case "3:4":
|
|
||||||
case "2:3":
|
|
||||||
return "1024x1536";
|
|
||||||
case "21:9":
|
|
||||||
return "1792x1024";
|
|
||||||
default:
|
|
||||||
return DEFAULT_SIZE;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
export function buildOpenAIImageGenerationProvider({ isProviderApiKeyConfigured, getPluginConfig, }) {
|
|
||||||
return {
|
|
||||||
id: "1h-image",
|
|
||||||
label: "1-h Image",
|
|
||||||
defaultModel: DEFAULT_OPENAI_IMAGE_MODEL,
|
|
||||||
models: [DEFAULT_OPENAI_IMAGE_MODEL, "gpt-image-1.5", "gemini-3.1-flash-image"],
|
|
||||||
isConfigured: (ctx) => {
|
|
||||||
const pluginCfg = resolveImageConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_API_KEY)) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return isProviderApiKeyConfigured({
|
|
||||||
provider: "openai",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxCount: 4,
|
|
||||||
supportsSize: true,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: true,
|
|
||||||
},
|
|
||||||
edit: {
|
|
||||||
enabled: true,
|
|
||||||
maxCount: 4,
|
|
||||||
maxInputImages: 5,
|
|
||||||
supportsSize: true,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: true,
|
|
||||||
},
|
|
||||||
geometry: {
|
|
||||||
sizes: [...SUPPORTED_SIZES],
|
|
||||||
aspectRatios: [...SUPPORTED_ASPECT_RATIOS],
|
|
||||||
resolutions: ["1K", "2K", "4K"],
|
|
||||||
},
|
|
||||||
output: {
|
|
||||||
formats: ["png", "jpeg", "webp"],
|
|
||||||
qualities: ["low", "medium", "high", "auto"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateImage(req) {
|
|
||||||
const pluginCfg = resolveImageConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey = resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_API_KEY);
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } = await import("openclaw/plugin-sdk/provider-auth-runtime");
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "openai",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
modelApi: "openai-responses",
|
|
||||||
});
|
|
||||||
if (!auth.apiKey) {
|
|
||||||
throw new Error("OpenAI / Bifrost Image API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
const [{ assertOkOrThrowHttpError, createProviderOperationDeadline, createProviderOperationTimeoutResolver, fetchWithTimeoutGuarded, postJsonRequest, postMultipartRequest, readProviderJsonResponse, resolveProviderHttpRequestConfig, resolveProviderOperationTimeoutMs, sanitizeConfiguredModelProviderRequest, },] = await Promise.all([import("openclaw/plugin-sdk/provider-http")]);
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "OpenAI Image Generation",
|
|
||||||
});
|
|
||||||
const providerConfig = req.cfg?.models?.providers?.openai;
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
...providerConfig?.request,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
const rawBaseUrl = resolveConfiguredImageBaseUrl(req.cfg, pluginCfg);
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } = resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: rawBaseUrl,
|
|
||||||
defaultBaseUrl: DEFAULT_OPENAI_IMAGE_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
},
|
|
||||||
provider: "openai",
|
|
||||||
capability: "image",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
const model = normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_OPENAI_IMAGE_MODEL;
|
|
||||||
const size = resolveImageSize({
|
|
||||||
size: req.size,
|
|
||||||
aspectRatio: req.aspectRatio,
|
|
||||||
model,
|
|
||||||
});
|
|
||||||
const count = Math.max(1, Math.min(req.count ?? 1, 4));
|
|
||||||
const inputImages = req.inputImages ?? [];
|
|
||||||
const isEdit = inputImages.length > 0;
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
let response;
|
|
||||||
let release;
|
|
||||||
const responseFormat = pluginCfg?.responseFormat ?? "url";
|
|
||||||
if (isEdit) {
|
|
||||||
const form = new FormData();
|
|
||||||
form.set("model", model);
|
|
||||||
form.set("prompt", req.prompt);
|
|
||||||
form.set("n", String(count));
|
|
||||||
form.set("size", size);
|
|
||||||
form.set("response_format", responseFormat);
|
|
||||||
inputImages.forEach((img, idx) => {
|
|
||||||
const mime = img.mimeType || "image/png";
|
|
||||||
const ext = extensionForMime(mime)?.slice(1) || "png";
|
|
||||||
const name = img.fileName || `input-${idx + 1}.${ext}`;
|
|
||||||
form.append(idx === 0 ? "image" : "image[]", new File([bufferToBlobPart(img.buffer)], name, { type: mime }));
|
|
||||||
});
|
|
||||||
const multipartHeaders = new Headers(headers);
|
|
||||||
multipartHeaders.delete("Content-Type");
|
|
||||||
({ response, release } = await postMultipartRequest({
|
|
||||||
url: `${baseUrl}/images/edits`,
|
|
||||||
headers: multipartHeaders,
|
|
||||||
body: form,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
const jsonHeaders = new Headers(headers);
|
|
||||||
jsonHeaders.set("Content-Type", "application/json");
|
|
||||||
const jsonBody = {
|
|
||||||
model,
|
|
||||||
prompt: req.prompt,
|
|
||||||
n: count,
|
|
||||||
size,
|
|
||||||
response_format: responseFormat,
|
|
||||||
};
|
|
||||||
if (req.quality) {
|
|
||||||
jsonBody.quality = req.quality;
|
|
||||||
}
|
|
||||||
({ response, release } = await postJsonRequest({
|
|
||||||
url: `${baseUrl}/images/generations`,
|
|
||||||
headers: jsonHeaders,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
let payload;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "Image generation request failed");
|
|
||||||
payload = await readProviderJsonResponse(response, "OpenAI image generation failed");
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
if (payload.error?.message) {
|
|
||||||
throw new Error(`Image generation error: ${payload.error.message}`);
|
|
||||||
}
|
|
||||||
const dataItems = payload.data ?? [];
|
|
||||||
if (dataItems.length === 0) {
|
|
||||||
throw new Error("Image generation returned no image data");
|
|
||||||
}
|
|
||||||
const generatedImages = [];
|
|
||||||
const downloadTimeout = createProviderOperationTimeoutResolver({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
for (let i = 0; i < dataItems.length; i++) {
|
|
||||||
const item = dataItems[i];
|
|
||||||
if (item.b64_json) {
|
|
||||||
const buffer = Buffer.from(item.b64_json, "base64");
|
|
||||||
generatedImages.push({
|
|
||||||
buffer,
|
|
||||||
mimeType: "image/png",
|
|
||||||
fileName: `image-${i + 1}.png`,
|
|
||||||
revisedPrompt: item.revised_prompt,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
else if (item.url) {
|
|
||||||
const dlResult = await fetchWithTimeoutGuarded(item.url, { method: "GET" }, downloadTimeout(), fetchFn, {
|
|
||||||
allowPrivateNetwork,
|
|
||||||
auditContext: "openai-image-download",
|
|
||||||
});
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(dlResult.response, "Failed to download generated image");
|
|
||||||
const arrayBuf = await dlResult.response.arrayBuffer();
|
|
||||||
const mime = dlResult.response.headers.get("content-type") || "image/png";
|
|
||||||
const ext = extensionForMime(mime)?.slice(1) || "png";
|
|
||||||
generatedImages.push({
|
|
||||||
buffer: Buffer.from(arrayBuf),
|
|
||||||
mimeType: mime,
|
|
||||||
fileName: `image-${i + 1}.${ext}`,
|
|
||||||
revisedPrompt: item.revised_prompt,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
await dlResult.release();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (generatedImages.length === 0) {
|
|
||||||
throw new Error("Failed to extract any valid image from provider response");
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
images: generatedImages,
|
|
||||||
model,
|
|
||||||
metadata: {
|
|
||||||
created: payload.created,
|
|
||||||
size,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
Vendored
-27
@@ -1,27 +0,0 @@
|
|||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import type { ImageGenerationProvider, ImageGenerationRequest } from "openclaw/plugin-sdk/image-generation";
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
export interface ImagePluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
responseFormat?: "url" | "b64_json";
|
|
||||||
}
|
|
||||||
export declare function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined;
|
|
||||||
export declare function resolveImageConfig(cfg: ImageGenerationRequest["cfg"] | undefined, apiPluginConfig?: Record<string, unknown>): ImagePluginConfig | undefined;
|
|
||||||
export declare function resolveConfiguredImageBaseUrl(cfg: ImageGenerationRequest["cfg"], pluginCfg?: ImagePluginConfig): string;
|
|
||||||
export declare function resolveImageSize(params: {
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
model?: string;
|
|
||||||
}): string;
|
|
||||||
export declare function buildOpenAIImageGenerationProvider({ isProviderApiKeyConfiguredAsync, getPluginConfig, }: {
|
|
||||||
isProviderApiKeyConfiguredAsync: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfiguredAsync"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): ImageGenerationProvider;
|
|
||||||
Vendored
-305
@@ -1,305 +0,0 @@
|
|||||||
import { bufferToBlobPart } from "openclaw/plugin-sdk/blob-runtime";
|
|
||||||
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
const DEFAULT_OPENAI_IMAGE_BASE_URL = "https://api.openai.com/v1";
|
|
||||||
const DEFAULT_OPENAI_IMAGE_MODEL = "gpt-image-2";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
const DEFAULT_SIZE = "1024x1024";
|
|
||||||
const SUPPORTED_SIZES = [
|
|
||||||
"1024x1024",
|
|
||||||
"1536x1024",
|
|
||||||
"1024x1536",
|
|
||||||
"1792x1024",
|
|
||||||
"1024x1792",
|
|
||||||
"2048x2048",
|
|
||||||
"2048x1152",
|
|
||||||
"3840x2160",
|
|
||||||
];
|
|
||||||
const SUPPORTED_ASPECT_RATIOS = [
|
|
||||||
"1:1",
|
|
||||||
"16:9",
|
|
||||||
"9:16",
|
|
||||||
"4:3",
|
|
||||||
"3:4",
|
|
||||||
"3:2",
|
|
||||||
"2:3",
|
|
||||||
"21:9",
|
|
||||||
];
|
|
||||||
export function resolvePluginApiKey(rawKey) {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
export function resolveImageConfig(cfg, apiPluginConfig) {
|
|
||||||
const toolsConfig = (apiPluginConfig ?? cfg?.plugins?.entries?.["1h-tools"]?.config);
|
|
||||||
return toolsConfig?.openaiImage ?? toolsConfig?.image;
|
|
||||||
}
|
|
||||||
export function resolveConfiguredImageBaseUrl(cfg, pluginCfg) {
|
|
||||||
return (normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_BASE_URL) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_BASE_URL) ??
|
|
||||||
normalizeOptionalString(cfg?.models?.providers?.openai?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_BASE_URL) ??
|
|
||||||
DEFAULT_OPENAI_IMAGE_BASE_URL);
|
|
||||||
}
|
|
||||||
export function resolveImageSize(params) {
|
|
||||||
const explicitSize = normalizeOptionalString(params.size);
|
|
||||||
if (explicitSize) {
|
|
||||||
return explicitSize;
|
|
||||||
}
|
|
||||||
// Safe aspect ratio normalization preventing issue #168665 (proxy rejecting 2048x1152)
|
|
||||||
switch (normalizeOptionalString(params.aspectRatio)) {
|
|
||||||
case "1:1":
|
|
||||||
return "1024x1024";
|
|
||||||
case "16:9":
|
|
||||||
return "1792x1024";
|
|
||||||
case "9:16":
|
|
||||||
return "1024x1792";
|
|
||||||
case "4:3":
|
|
||||||
case "3:2":
|
|
||||||
return "1536x1024";
|
|
||||||
case "3:4":
|
|
||||||
case "2:3":
|
|
||||||
return "1024x1536";
|
|
||||||
case "21:9":
|
|
||||||
return "1792x1024";
|
|
||||||
default:
|
|
||||||
return DEFAULT_SIZE;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
export function buildOpenAIImageGenerationProvider({ isProviderApiKeyConfiguredAsync, getPluginConfig, }) {
|
|
||||||
return {
|
|
||||||
id: "1h-openai-image",
|
|
||||||
label: "1-h OpenAI Image",
|
|
||||||
defaultModel: DEFAULT_OPENAI_IMAGE_MODEL,
|
|
||||||
models: [DEFAULT_OPENAI_IMAGE_MODEL, "gpt-image-2.5", "gpt-image-2.5-flare", "gpt-image-2.5-sunburst"],
|
|
||||||
isConfiguredAsync: async (ctx) => {
|
|
||||||
const pluginCfg = resolveImageConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_API_KEY)) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return await isProviderApiKeyConfiguredAsync({
|
|
||||||
provider: "openai",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxCount: 4,
|
|
||||||
supportsSize: true,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: true,
|
|
||||||
},
|
|
||||||
edit: {
|
|
||||||
enabled: true,
|
|
||||||
maxCount: 4,
|
|
||||||
maxInputImages: 5,
|
|
||||||
supportsSize: true,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: true,
|
|
||||||
},
|
|
||||||
geometry: {
|
|
||||||
sizes: [...SUPPORTED_SIZES],
|
|
||||||
aspectRatios: [...SUPPORTED_ASPECT_RATIOS],
|
|
||||||
resolutions: ["1K", "2K", "4K"],
|
|
||||||
},
|
|
||||||
output: {
|
|
||||||
formats: ["png", "jpeg", "webp"],
|
|
||||||
qualities: ["low", "medium", "high", "auto"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateImage(req) {
|
|
||||||
const pluginCfg = resolveImageConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey = resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_API_KEY);
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } = await import("openclaw/plugin-sdk/provider-auth-runtime");
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "openai",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
modelApi: "openai-responses",
|
|
||||||
});
|
|
||||||
if (!auth.apiKey) {
|
|
||||||
throw new Error("OpenAI / Bifrost Image API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
const [{ assertOkOrThrowHttpError, createProviderOperationDeadline, createProviderOperationTimeoutResolver, fetchWithTimeoutGuarded, postJsonRequest, postMultipartRequest, readProviderJsonResponse, resolveProviderHttpRequestConfig, resolveProviderOperationTimeoutMs, sanitizeConfiguredModelProviderRequest, },] = await Promise.all([import("openclaw/plugin-sdk/provider-http")]);
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "OpenAI Image Generation",
|
|
||||||
});
|
|
||||||
const providerConfig = req.cfg?.models?.providers?.openai;
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
...providerConfig?.request,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
const rawBaseUrl = resolveConfiguredImageBaseUrl(req.cfg, pluginCfg);
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } = resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: rawBaseUrl,
|
|
||||||
defaultBaseUrl: DEFAULT_OPENAI_IMAGE_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
},
|
|
||||||
provider: "openai",
|
|
||||||
capability: "image",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
const model = normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_OPENAI_IMAGE_MODEL;
|
|
||||||
const size = resolveImageSize({
|
|
||||||
size: req.size,
|
|
||||||
aspectRatio: req.aspectRatio,
|
|
||||||
model,
|
|
||||||
});
|
|
||||||
const count = Math.max(1, Math.min(req.count ?? 1, 4));
|
|
||||||
const inputImages = req.inputImages ?? [];
|
|
||||||
const isEdit = inputImages.length > 0;
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
let response;
|
|
||||||
let release;
|
|
||||||
const responseFormat = pluginCfg?.responseFormat ?? "url";
|
|
||||||
if (isEdit) {
|
|
||||||
const form = new FormData();
|
|
||||||
form.set("model", model);
|
|
||||||
form.set("prompt", req.prompt);
|
|
||||||
form.set("n", String(count));
|
|
||||||
form.set("size", size);
|
|
||||||
form.set("response_format", responseFormat);
|
|
||||||
inputImages.forEach((img, idx) => {
|
|
||||||
const mime = img.mimeType || "image/png";
|
|
||||||
const ext = extensionForMime(mime)?.slice(1) || "png";
|
|
||||||
const name = img.fileName || `input-${idx + 1}.${ext}`;
|
|
||||||
form.append(idx === 0 ? "image" : "image[]", new File([bufferToBlobPart(img.buffer)], name, { type: mime }));
|
|
||||||
});
|
|
||||||
const multipartHeaders = new Headers(headers);
|
|
||||||
multipartHeaders.delete("Content-Type");
|
|
||||||
({ response, release } = await postMultipartRequest({
|
|
||||||
url: `${baseUrl}/images/edits`,
|
|
||||||
headers: multipartHeaders,
|
|
||||||
body: form,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
const jsonHeaders = new Headers(headers);
|
|
||||||
jsonHeaders.set("Content-Type", "application/json");
|
|
||||||
const jsonBody = {
|
|
||||||
model,
|
|
||||||
prompt: req.prompt,
|
|
||||||
n: count,
|
|
||||||
size,
|
|
||||||
response_format: responseFormat,
|
|
||||||
};
|
|
||||||
if (req.quality) {
|
|
||||||
jsonBody.quality = req.quality;
|
|
||||||
}
|
|
||||||
({ response, release } = await postJsonRequest({
|
|
||||||
url: `${baseUrl}/images/generations`,
|
|
||||||
headers: jsonHeaders,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
let payload;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "Image generation request failed");
|
|
||||||
payload = await readProviderJsonResponse(response, "OpenAI image generation failed");
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
if (payload.error?.message) {
|
|
||||||
throw new Error(`Image generation error: ${payload.error.message}`);
|
|
||||||
}
|
|
||||||
const dataItems = payload.data ?? [];
|
|
||||||
if (dataItems.length === 0) {
|
|
||||||
throw new Error("Image generation returned no image data");
|
|
||||||
}
|
|
||||||
const generatedImages = [];
|
|
||||||
const downloadTimeout = createProviderOperationTimeoutResolver({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
for (let i = 0; i < dataItems.length; i++) {
|
|
||||||
const item = dataItems[i];
|
|
||||||
if (item.b64_json) {
|
|
||||||
const buffer = Buffer.from(item.b64_json, "base64");
|
|
||||||
generatedImages.push({
|
|
||||||
buffer,
|
|
||||||
mimeType: "image/png",
|
|
||||||
fileName: `image-${i + 1}.png`,
|
|
||||||
revisedPrompt: item.revised_prompt,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
else if (item.url) {
|
|
||||||
const dlResult = await fetchWithTimeoutGuarded(item.url, { method: "GET" }, downloadTimeout(), fetchFn, {
|
|
||||||
allowPrivateNetwork,
|
|
||||||
auditContext: "openai-image-download",
|
|
||||||
});
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(dlResult.response, "Failed to download generated image");
|
|
||||||
const arrayBuf = await dlResult.response.arrayBuffer();
|
|
||||||
const mime = dlResult.response.headers.get("content-type") || "image/png";
|
|
||||||
const ext = extensionForMime(mime)?.slice(1) || "png";
|
|
||||||
generatedImages.push({
|
|
||||||
buffer: Buffer.from(arrayBuf),
|
|
||||||
mimeType: mime,
|
|
||||||
fileName: `image-${i + 1}.${ext}`,
|
|
||||||
revisedPrompt: item.revised_prompt,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
await dlResult.release();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (generatedImages.length === 0) {
|
|
||||||
throw new Error("Failed to extract any valid image from provider response");
|
|
||||||
}
|
|
||||||
return {
|
|
||||||
images: generatedImages,
|
|
||||||
model,
|
|
||||||
metadata: {
|
|
||||||
created: payload.created,
|
|
||||||
size,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
Vendored
+2
-14
@@ -1,14 +1,2 @@
|
|||||||
declare const entry: Omit<{
|
declare const _default: import("openclaw/plugin-sdk/tool-plugin").DefinedToolPluginEntry;
|
||||||
id: string;
|
export default _default;
|
||||||
name: string;
|
|
||||||
description: string;
|
|
||||||
kind?: import("openclaw/plugin-sdk/core").OpenClawPluginDefinition["kind"];
|
|
||||||
configSchema?: import("openclaw/plugin-sdk/core").OpenClawPluginConfigSchema | (() => import("openclaw/plugin-sdk/core").OpenClawPluginConfigSchema);
|
|
||||||
reload?: import("openclaw/plugin-sdk/core").OpenClawPluginDefinition["reload"];
|
|
||||||
nodeHostCommands?: import("openclaw/plugin-sdk/core").OpenClawPluginDefinition["nodeHostCommands"];
|
|
||||||
securityAuditCollectors?: import("openclaw/plugin-sdk/core").OpenClawPluginDefinition["securityAuditCollectors"];
|
|
||||||
register: NonNullable<import("openclaw/plugin-sdk/core").OpenClawPluginDefinition["register"]>;
|
|
||||||
}, "configSchema"> & {
|
|
||||||
configSchema: import("openclaw/plugin-sdk/core").OpenClawPluginConfigSchema;
|
|
||||||
};
|
|
||||||
export default entry;
|
|
||||||
Vendored
+16
-39
@@ -1,40 +1,17 @@
|
|||||||
import { jsonResult } from "openclaw/plugin-sdk/core";
|
import { Type } from "typebox";
|
||||||
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
|
import { defineToolPlugin } from "openclaw/plugin-sdk/tool-plugin";
|
||||||
import { echoTool } from "./tools/echo.js";
|
export default defineToolPlugin({
|
||||||
import { buildOpenAIVideoGenerationProvider } from "./video/openai.js";
|
id: "tools",
|
||||||
import { buildOpenAIImageGenerationProvider } from "./image/openai.js";
|
name: "Tools",
|
||||||
import { buildGeminiImageGenerationProvider } from "./image/gemini.js";
|
description: "Add Tools tools to OpenClaw.",
|
||||||
const entry = definePluginEntry({
|
tools: (tool) => [
|
||||||
id: "1h-tools",
|
tool({
|
||||||
name: "1-h Tools",
|
name: "echo",
|
||||||
description: "Custom Tools and Media Providers (OpenAI/Gemini) for OpenClaw.",
|
description: "Echo input text.",
|
||||||
register(api) {
|
parameters: Type.Object({
|
||||||
// 1. Register tools
|
input: Type.String({ description: "Text to echo." }),
|
||||||
api.registerTool({
|
}),
|
||||||
name: echoTool.name,
|
execute: async ({ input }) => ({ input }),
|
||||||
label: echoTool.label,
|
}),
|
||||||
description: echoTool.description,
|
],
|
||||||
parameters: echoTool.parameters,
|
|
||||||
execute: async (_toolCallId, params) => {
|
|
||||||
const result = await echoTool.execute(params);
|
|
||||||
return jsonResult(result);
|
|
||||||
},
|
|
||||||
});
|
|
||||||
// 2. Register OpenAI-compatible video generation provider
|
|
||||||
api.registerVideoGenerationProvider(buildOpenAIVideoGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync: api.runtime.modelAuth.isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig: () => api.pluginConfig,
|
|
||||||
}));
|
|
||||||
// 3. Register OpenAI-compatible image generation provider
|
|
||||||
api.registerImageGenerationProvider(buildOpenAIImageGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync: api.runtime.modelAuth.isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig: () => api.pluginConfig,
|
|
||||||
}));
|
|
||||||
// 4. Register Gemini-native image generation provider
|
|
||||||
api.registerImageGenerationProvider(buildGeminiImageGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync: api.runtime.modelAuth.isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig: () => api.pluginConfig,
|
|
||||||
}));
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
export default entry;
|
|
||||||
Vendored
-16
@@ -1,16 +0,0 @@
|
|||||||
import { Type, type Static } from "typebox";
|
|
||||||
export declare const EchoInputSchema: Type.TObject<{
|
|
||||||
input: Type.TString;
|
|
||||||
}>;
|
|
||||||
export type EchoInput = Static<typeof EchoInputSchema>;
|
|
||||||
export declare const echoTool: {
|
|
||||||
name: string;
|
|
||||||
label: string;
|
|
||||||
description: string;
|
|
||||||
parameters: Type.TObject<{
|
|
||||||
input: Type.TString;
|
|
||||||
}>;
|
|
||||||
execute: ({ input }: EchoInput) => Promise<{
|
|
||||||
input: string;
|
|
||||||
}>;
|
|
||||||
};
|
|
||||||
Vendored
-11
@@ -1,11 +0,0 @@
|
|||||||
import { Type } from "typebox";
|
|
||||||
export const EchoInputSchema = Type.Object({
|
|
||||||
input: Type.String({ description: "Text to echo." }),
|
|
||||||
});
|
|
||||||
export const echoTool = {
|
|
||||||
name: "echo",
|
|
||||||
label: "Echo",
|
|
||||||
description: "Echo input text.",
|
|
||||||
parameters: EchoInputSchema,
|
|
||||||
execute: async ({ input }) => ({ input }),
|
|
||||||
};
|
|
||||||
Vendored
-60
@@ -1,60 +0,0 @@
|
|||||||
import { type MediaKind } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import type { createProviderOperationDeadline, postMultipartRequest } from "openclaw/plugin-sdk/provider-http";
|
|
||||||
import type { VideoGenerationProvider, VideoGenerationRequest } from "openclaw/plugin-sdk/video-generation";
|
|
||||||
declare const OPENAI_VIDEO_SIZES: readonly ["720x1280", "1280x720", "1024x1792", "1792x1024"];
|
|
||||||
export type OpenAIVideoRequestPolicy = {
|
|
||||||
allowPrivateNetwork: boolean;
|
|
||||||
dispatcherPolicy?: Parameters<typeof postMultipartRequest>[0]["dispatcherPolicy"];
|
|
||||||
};
|
|
||||||
export type OpenAIVideoStatus = "queued" | "in_progress" | "completed" | "failed";
|
|
||||||
export type OpenAIReferenceAsset = {
|
|
||||||
kind: Extract<MediaKind, "image" | "video">;
|
|
||||||
file: File;
|
|
||||||
};
|
|
||||||
export type OpenAIVideoResponse = {
|
|
||||||
id?: string;
|
|
||||||
model?: string;
|
|
||||||
status?: OpenAIVideoStatus;
|
|
||||||
prompt?: string | null;
|
|
||||||
seconds?: string;
|
|
||||||
size?: string;
|
|
||||||
error?: string | {
|
|
||||||
code?: string;
|
|
||||||
message?: string;
|
|
||||||
} | null;
|
|
||||||
};
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
export interface VideoPluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
}
|
|
||||||
export declare function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined;
|
|
||||||
export declare function resolveVideoConfig(cfg: VideoGenerationRequest["cfg"] | undefined, apiPluginConfig?: Record<string, unknown>): VideoPluginConfig | undefined;
|
|
||||||
export declare function resolveConfiguredOpenAIBaseUrl(cfg: VideoGenerationRequest["cfg"], pluginCfg?: VideoPluginConfig): string;
|
|
||||||
export declare function readOpenAIVideoFailureMessage(payload: OpenAIVideoResponse): string | undefined;
|
|
||||||
export declare function resolveDurationSeconds(durationSeconds: number | undefined): "4" | "8" | "12" | undefined;
|
|
||||||
export declare function resolveSize(params: {
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
resolution?: string;
|
|
||||||
}): (typeof OPENAI_VIDEO_SIZES)[number] | undefined;
|
|
||||||
export declare function resolveReferenceAsset(req: VideoGenerationRequest): OpenAIReferenceAsset | null;
|
|
||||||
export declare function fetchOpenAIVideoDownload(params: {
|
|
||||||
url: string;
|
|
||||||
init: RequestInit;
|
|
||||||
deadline: ReturnType<typeof createProviderOperationDeadline>;
|
|
||||||
fetchFn: typeof fetch;
|
|
||||||
} & OpenAIVideoRequestPolicy): Promise<any>;
|
|
||||||
export declare function buildOpenAIVideoGenerationProvider({ isProviderApiKeyConfigured, getPluginConfig, }: {
|
|
||||||
isProviderApiKeyConfigured: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfigured"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): VideoGenerationProvider;
|
|
||||||
export {};
|
|
||||||
Vendored
-372
@@ -1,372 +0,0 @@
|
|||||||
import { bufferToBlobPart } from "openclaw/plugin-sdk/blob-runtime";
|
|
||||||
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
const DEFAULT_OPENAI_VIDEO_BASE_URL = "https://api.openai.com/v1";
|
|
||||||
const DEFAULT_OPENAI_VIDEO_MODEL = "sora-2";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
const POLL_INTERVAL_MS = 2_500;
|
|
||||||
const MAX_POLL_ATTEMPTS = 120;
|
|
||||||
const OPENAI_VIDEO_SECONDS = [4, 8, 12];
|
|
||||||
const OPENAI_VIDEO_SIZES = ["720x1280", "1280x720", "1024x1792", "1792x1024"];
|
|
||||||
export function resolvePluginApiKey(rawKey) {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
export function resolveVideoConfig(cfg, apiPluginConfig) {
|
|
||||||
const direct = apiPluginConfig?.video;
|
|
||||||
const fromCfg = cfg?.plugins?.entries?.["1h-tools"]?.config?.video;
|
|
||||||
return direct ?? fromCfg;
|
|
||||||
}
|
|
||||||
export function resolveConfiguredOpenAIBaseUrl(cfg, pluginCfg) {
|
|
||||||
return (normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_BASE_URL) ??
|
|
||||||
normalizeOptionalString(cfg?.models?.providers?.openai?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_BASE_URL) ??
|
|
||||||
DEFAULT_OPENAI_VIDEO_BASE_URL);
|
|
||||||
}
|
|
||||||
export function readOpenAIVideoFailureMessage(payload) {
|
|
||||||
if (payload.status !== "failed") {
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
if (typeof payload.error === "string") {
|
|
||||||
return normalizeOptionalString(payload.error) ?? "OpenAI video generation failed";
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(payload.error?.message) ?? "OpenAI video generation failed";
|
|
||||||
}
|
|
||||||
export function resolveDurationSeconds(durationSeconds) {
|
|
||||||
if (typeof durationSeconds !== "number" || !Number.isFinite(durationSeconds)) {
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
const rounded = Math.max(OPENAI_VIDEO_SECONDS[0], Math.round(durationSeconds));
|
|
||||||
const nearest = OPENAI_VIDEO_SECONDS.reduce((best, current) => Math.abs(current - rounded) < Math.abs(best - rounded) ? current : best);
|
|
||||||
return String(nearest);
|
|
||||||
}
|
|
||||||
export function resolveSize(params) {
|
|
||||||
const explicitSize = normalizeOptionalString(params.size);
|
|
||||||
const supportedSize = OPENAI_VIDEO_SIZES.find((size) => size === explicitSize);
|
|
||||||
if (supportedSize) {
|
|
||||||
return supportedSize;
|
|
||||||
}
|
|
||||||
switch (normalizeOptionalString(params.aspectRatio)) {
|
|
||||||
case "9:16":
|
|
||||||
return "720x1280";
|
|
||||||
case "16:9":
|
|
||||||
return "1280x720";
|
|
||||||
case "4:7":
|
|
||||||
return "1024x1792";
|
|
||||||
case "7:4":
|
|
||||||
return "1792x1024";
|
|
||||||
default:
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
if (params.resolution === "1080P") {
|
|
||||||
return "1792x1024";
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
export function resolveReferenceAsset(req) {
|
|
||||||
const allAssets = [...(req.inputImages ?? []), ...(req.inputVideos ?? [])];
|
|
||||||
if (allAssets.length === 0) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
if (allAssets.length > 1) {
|
|
||||||
throw new Error("OpenAI video generation supports at most one reference image or video.");
|
|
||||||
}
|
|
||||||
const [asset] = allAssets;
|
|
||||||
if (!asset?.buffer) {
|
|
||||||
throw new Error("OpenAI video generation currently requires local image/video uploads for reference assets.");
|
|
||||||
}
|
|
||||||
const kind = (req.inputVideos?.length ?? 0) > 0 ? "video" : "image";
|
|
||||||
const mimeType = normalizeOptionalString(asset.mimeType) || (kind === "video" ? "video/mp4" : "image/png");
|
|
||||||
const extension = extensionForMime(mimeType)?.slice(1) ?? (mimeType.startsWith("video/") ? "mp4" : "png");
|
|
||||||
const fileName = normalizeOptionalString(asset.fileName) || `reference-${kind}.${extension}`;
|
|
||||||
return {
|
|
||||||
kind,
|
|
||||||
file: new File([bufferToBlobPart(asset.buffer)], fileName, { type: mimeType }),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
export async function fetchOpenAIVideoDownload(params) {
|
|
||||||
const { assertOkOrThrowHttpError, createProviderOperationTimeoutResolver, executeProviderOperationWithRetry, fetchProviderDownloadResponse, fetchWithTimeoutGuarded, } = await import("openclaw/plugin-sdk/provider-http");
|
|
||||||
const timeoutMs = createProviderOperationTimeoutResolver({
|
|
||||||
deadline: params.deadline,
|
|
||||||
defaultTimeoutMs: params.deadline.timeoutMs ?? DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
if (!params.allowPrivateNetwork && !params.dispatcherPolicy) {
|
|
||||||
const response = await fetchProviderDownloadResponse({
|
|
||||||
url: params.url,
|
|
||||||
init: params.init,
|
|
||||||
deadline: params.deadline,
|
|
||||||
fetchFn: params.fetchFn,
|
|
||||||
provider: "openai",
|
|
||||||
requestFailedMessage: "OpenAI video download failed",
|
|
||||||
});
|
|
||||||
return {
|
|
||||||
response,
|
|
||||||
release: async () => { },
|
|
||||||
};
|
|
||||||
}
|
|
||||||
return await executeProviderOperationWithRetry({
|
|
||||||
provider: "openai",
|
|
||||||
stage: "download",
|
|
||||||
operation: async () => {
|
|
||||||
const result = await fetchWithTimeoutGuarded(params.url, params.init, timeoutMs(), params.fetchFn, {
|
|
||||||
...(params.allowPrivateNetwork ? { ssrfPolicy: { allowPrivateNetwork: true } } : {}),
|
|
||||||
...(params.dispatcherPolicy ? { dispatcherPolicy: params.dispatcherPolicy } : {}),
|
|
||||||
auditContext: "openai-video-download",
|
|
||||||
});
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(result.response, "OpenAI video download failed");
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
await result.release();
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
export function buildOpenAIVideoGenerationProvider({ isProviderApiKeyConfigured, getPluginConfig, }) {
|
|
||||||
return {
|
|
||||||
id: "1h-video",
|
|
||||||
label: "1-h Video",
|
|
||||||
defaultModel: DEFAULT_OPENAI_VIDEO_MODEL,
|
|
||||||
models: [DEFAULT_OPENAI_VIDEO_MODEL, "sora-2-pro", "gemini-video"],
|
|
||||||
isConfigured: (ctx) => {
|
|
||||||
const pluginCfg = resolveVideoConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.GEMINI_WEB2API_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY)) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return isProviderApiKeyConfigured({
|
|
||||||
provider: "openai",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxVideos: 1,
|
|
||||||
maxDurationSeconds: 12,
|
|
||||||
supportedDurationSeconds: OPENAI_VIDEO_SECONDS,
|
|
||||||
supportsSize: true,
|
|
||||||
sizes: OPENAI_VIDEO_SIZES,
|
|
||||||
},
|
|
||||||
imageToVideo: {
|
|
||||||
enabled: true,
|
|
||||||
maxVideos: 1,
|
|
||||||
maxInputImages: 1,
|
|
||||||
maxDurationSeconds: 12,
|
|
||||||
supportedDurationSeconds: OPENAI_VIDEO_SECONDS,
|
|
||||||
supportsSize: true,
|
|
||||||
sizes: OPENAI_VIDEO_SIZES,
|
|
||||||
},
|
|
||||||
videoToVideo: {
|
|
||||||
enabled: true,
|
|
||||||
maxVideos: 1,
|
|
||||||
maxInputVideos: 1,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateVideo(req) {
|
|
||||||
const pluginCfg = resolveVideoConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey = resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.GEMINI_WEB2API_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY);
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } = await import("openclaw/plugin-sdk/provider-auth-runtime");
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "openai",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
modelApi: "openai-responses",
|
|
||||||
});
|
|
||||||
if (!auth.apiKey || (auth.mode !== undefined && auth.mode !== "api-key")) {
|
|
||||||
throw new Error("OpenAI API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
const [{ assertOkOrThrowHttpError, createProviderOperationDeadline, createProviderOperationTimeoutResolver, pollProviderOperationJson, postJsonRequest, postMultipartRequest, readProviderJsonResponse, resolveProviderOperationTimeoutMs, resolveProviderHttpRequestConfig, sanitizeConfiguredModelProviderRequest, }, { downloadGeneratedVideoAsset, resolveGeneratedMediaMaxBytes },] = await Promise.all([
|
|
||||||
import("openclaw/plugin-sdk/provider-http"),
|
|
||||||
import("openclaw/plugin-sdk/media-generation-runtime"),
|
|
||||||
]);
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "OpenAI video generation",
|
|
||||||
});
|
|
||||||
const providerConfig = req.cfg.models?.providers?.openai;
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
...providerConfig?.request,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } = resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: resolveConfiguredOpenAIBaseUrl(req.cfg, pluginCfg),
|
|
||||||
defaultBaseUrl: DEFAULT_OPENAI_VIDEO_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
},
|
|
||||||
provider: "openai",
|
|
||||||
capability: "video",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
const model = normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_OPENAI_VIDEO_MODEL;
|
|
||||||
const seconds = resolveDurationSeconds(req.durationSeconds);
|
|
||||||
const size = resolveSize(req);
|
|
||||||
const referenceAsset = resolveReferenceAsset(req);
|
|
||||||
const isVideoEdit = referenceAsset?.kind === "video";
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
let response;
|
|
||||||
let release;
|
|
||||||
if (referenceAsset) {
|
|
||||||
const form = new FormData();
|
|
||||||
form.set("prompt", req.prompt);
|
|
||||||
if (isVideoEdit) {
|
|
||||||
form.set("video", referenceAsset.file);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
form.set("model", model);
|
|
||||||
if (seconds) {
|
|
||||||
form.set("seconds", seconds);
|
|
||||||
}
|
|
||||||
if (size) {
|
|
||||||
form.set("size", size);
|
|
||||||
}
|
|
||||||
form.set("input_reference", referenceAsset.file);
|
|
||||||
}
|
|
||||||
const multipartHeaders = new Headers(headers);
|
|
||||||
multipartHeaders.delete("Content-Type");
|
|
||||||
({ response, release } = await postMultipartRequest({
|
|
||||||
url: `${baseUrl}/videos${isVideoEdit ? "/edits" : ""}`,
|
|
||||||
headers: multipartHeaders,
|
|
||||||
body: form,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
const jsonHeaders = new Headers(headers);
|
|
||||||
jsonHeaders.set("Content-Type", "application/json");
|
|
||||||
const jsonBody = {
|
|
||||||
model,
|
|
||||||
prompt: req.prompt,
|
|
||||||
...(seconds ? { seconds } : {}),
|
|
||||||
...(size ? { size } : {}),
|
|
||||||
};
|
|
||||||
({ response, release } = await postJsonRequest({
|
|
||||||
url: `${baseUrl}/videos`,
|
|
||||||
headers: jsonHeaders,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
let submitted;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "OpenAI video generation failed");
|
|
||||||
submitted = await readProviderJsonResponse(response, "OpenAI video generation failed");
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
const failureMessage = readOpenAIVideoFailureMessage(submitted);
|
|
||||||
if (failureMessage) {
|
|
||||||
throw new Error(failureMessage);
|
|
||||||
}
|
|
||||||
const videoId = normalizeOptionalString(submitted.id);
|
|
||||||
if (!videoId) {
|
|
||||||
throw new Error("OpenAI video generation response missing video id");
|
|
||||||
}
|
|
||||||
const completed = submitted.status === "completed"
|
|
||||||
? submitted
|
|
||||||
: await pollProviderOperationJson({
|
|
||||||
url: `${baseUrl}/videos/${videoId}`,
|
|
||||||
headers,
|
|
||||||
deadline: createProviderOperationDeadline({
|
|
||||||
timeoutMs: resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
}),
|
|
||||||
label: `OpenAI video generation task ${videoId}`,
|
|
||||||
}),
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
fetchFn,
|
|
||||||
maxAttempts: MAX_POLL_ATTEMPTS,
|
|
||||||
pollIntervalMs: POLL_INTERVAL_MS,
|
|
||||||
requestFailedMessage: "OpenAI video status request failed",
|
|
||||||
timeoutMessage: `OpenAI video generation task ${videoId} did not finish in time`,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
auditContext: "openai-video-status",
|
|
||||||
isComplete: (payload) => payload.status === "completed",
|
|
||||||
getFailureMessage: readOpenAIVideoFailureMessage,
|
|
||||||
});
|
|
||||||
const url = new URL(`${baseUrl}/videos/${videoId}/content`);
|
|
||||||
url.searchParams.set("variant", "video");
|
|
||||||
const video = await downloadGeneratedVideoAsset({
|
|
||||||
url: url.toString(),
|
|
||||||
timeoutMs: createProviderOperationTimeoutResolver({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
}),
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
fetchFn,
|
|
||||||
provider: "openai",
|
|
||||||
label: "OpenAI generated video download",
|
|
||||||
requestFailedMessage: "OpenAI video download failed",
|
|
||||||
maxBytes: resolveGeneratedMediaMaxBytes(req.cfg, "video"),
|
|
||||||
validateBinaryResponse: true,
|
|
||||||
fetchResponse: async ({ deadline: downloadDeadline }) => await fetchOpenAIVideoDownload({
|
|
||||||
url: url.toString(),
|
|
||||||
init: {
|
|
||||||
method: "GET",
|
|
||||||
headers: new Headers({
|
|
||||||
...Object.fromEntries(headers.entries()),
|
|
||||||
Accept: "application/binary",
|
|
||||||
}),
|
|
||||||
},
|
|
||||||
deadline: downloadDeadline,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
return {
|
|
||||||
videos: [video],
|
|
||||||
model: completed.model ?? submitted.model ?? model,
|
|
||||||
metadata: {
|
|
||||||
videoId,
|
|
||||||
status: completed.status,
|
|
||||||
seconds: completed.seconds ?? submitted.seconds,
|
|
||||||
size: completed.size ?? submitted.size,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
Vendored
-60
@@ -1,60 +0,0 @@
|
|||||||
import { type MediaKind } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import type { createProviderOperationDeadline, postMultipartRequest } from "openclaw/plugin-sdk/provider-http";
|
|
||||||
import type { VideoGenerationProvider, VideoGenerationRequest } from "openclaw/plugin-sdk/video-generation";
|
|
||||||
declare const OPENAI_VIDEO_SIZES: readonly ["720x1280", "1280x720", "1024x1792", "1792x1024"];
|
|
||||||
export type OpenAIVideoRequestPolicy = {
|
|
||||||
allowPrivateNetwork: boolean;
|
|
||||||
dispatcherPolicy?: Parameters<typeof postMultipartRequest>[0]["dispatcherPolicy"];
|
|
||||||
};
|
|
||||||
export type OpenAIVideoStatus = "queued" | "in_progress" | "completed" | "failed";
|
|
||||||
export type OpenAIReferenceAsset = {
|
|
||||||
kind: Extract<MediaKind, "image" | "video">;
|
|
||||||
file: File;
|
|
||||||
};
|
|
||||||
export type OpenAIVideoResponse = {
|
|
||||||
id?: string;
|
|
||||||
model?: string;
|
|
||||||
status?: OpenAIVideoStatus;
|
|
||||||
prompt?: string | null;
|
|
||||||
seconds?: string;
|
|
||||||
size?: string;
|
|
||||||
error?: string | {
|
|
||||||
code?: string;
|
|
||||||
message?: string;
|
|
||||||
} | null;
|
|
||||||
};
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
export interface VideoPluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
}
|
|
||||||
export declare function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined;
|
|
||||||
export declare function resolveVideoConfig(cfg: VideoGenerationRequest["cfg"] | undefined, apiPluginConfig?: Record<string, unknown>): VideoPluginConfig | undefined;
|
|
||||||
export declare function resolveConfiguredOpenAIBaseUrl(cfg: VideoGenerationRequest["cfg"], pluginCfg?: VideoPluginConfig): string;
|
|
||||||
export declare function readOpenAIVideoFailureMessage(payload: OpenAIVideoResponse): string | undefined;
|
|
||||||
export declare function resolveDurationSeconds(durationSeconds: number | undefined): "4" | "8" | "12" | undefined;
|
|
||||||
export declare function resolveSize(params: {
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
resolution?: string;
|
|
||||||
}): (typeof OPENAI_VIDEO_SIZES)[number] | undefined;
|
|
||||||
export declare function resolveReferenceAsset(req: VideoGenerationRequest): OpenAIReferenceAsset | null;
|
|
||||||
export declare function fetchOpenAIVideoDownload(params: {
|
|
||||||
url: string;
|
|
||||||
init: RequestInit;
|
|
||||||
deadline: ReturnType<typeof createProviderOperationDeadline>;
|
|
||||||
fetchFn: typeof fetch;
|
|
||||||
} & OpenAIVideoRequestPolicy): Promise<any>;
|
|
||||||
export declare function buildOpenAIVideoGenerationProvider({ isProviderApiKeyConfiguredAsync, getPluginConfig, }: {
|
|
||||||
isProviderApiKeyConfiguredAsync: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfiguredAsync"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): VideoGenerationProvider;
|
|
||||||
export {};
|
|
||||||
Vendored
-371
@@ -1,371 +0,0 @@
|
|||||||
import { bufferToBlobPart } from "openclaw/plugin-sdk/blob-runtime";
|
|
||||||
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
const DEFAULT_OPENAI_VIDEO_BASE_URL = "https://api.openai.com/v1";
|
|
||||||
const DEFAULT_OPENAI_VIDEO_MODEL = "sora-2";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
const POLL_INTERVAL_MS = 2_500;
|
|
||||||
const MAX_POLL_ATTEMPTS = 120;
|
|
||||||
const OPENAI_VIDEO_SECONDS = [4, 8, 12];
|
|
||||||
const OPENAI_VIDEO_SIZES = ["720x1280", "1280x720", "1024x1792", "1792x1024"];
|
|
||||||
export function resolvePluginApiKey(rawKey) {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
export function resolveVideoConfig(cfg, apiPluginConfig) {
|
|
||||||
const toolsConfig = (apiPluginConfig ?? cfg?.plugins?.entries?.["1h-tools"]?.config);
|
|
||||||
return toolsConfig?.openaiVideo ?? toolsConfig?.video;
|
|
||||||
}
|
|
||||||
export function resolveConfiguredOpenAIBaseUrl(cfg, pluginCfg) {
|
|
||||||
return (normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_BASE_URL) ??
|
|
||||||
normalizeOptionalString(cfg?.models?.providers?.openai?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_BASE_URL) ??
|
|
||||||
DEFAULT_OPENAI_VIDEO_BASE_URL);
|
|
||||||
}
|
|
||||||
export function readOpenAIVideoFailureMessage(payload) {
|
|
||||||
if (payload.status !== "failed") {
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
if (typeof payload.error === "string") {
|
|
||||||
return normalizeOptionalString(payload.error) ?? "OpenAI video generation failed";
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(payload.error?.message) ?? "OpenAI video generation failed";
|
|
||||||
}
|
|
||||||
export function resolveDurationSeconds(durationSeconds) {
|
|
||||||
if (typeof durationSeconds !== "number" || !Number.isFinite(durationSeconds)) {
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
const rounded = Math.max(OPENAI_VIDEO_SECONDS[0], Math.round(durationSeconds));
|
|
||||||
const nearest = OPENAI_VIDEO_SECONDS.reduce((best, current) => Math.abs(current - rounded) < Math.abs(best - rounded) ? current : best);
|
|
||||||
return String(nearest);
|
|
||||||
}
|
|
||||||
export function resolveSize(params) {
|
|
||||||
const explicitSize = normalizeOptionalString(params.size);
|
|
||||||
const supportedSize = OPENAI_VIDEO_SIZES.find((size) => size === explicitSize);
|
|
||||||
if (supportedSize) {
|
|
||||||
return supportedSize;
|
|
||||||
}
|
|
||||||
switch (normalizeOptionalString(params.aspectRatio)) {
|
|
||||||
case "9:16":
|
|
||||||
return "720x1280";
|
|
||||||
case "16:9":
|
|
||||||
return "1280x720";
|
|
||||||
case "4:7":
|
|
||||||
return "1024x1792";
|
|
||||||
case "7:4":
|
|
||||||
return "1792x1024";
|
|
||||||
default:
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
if (params.resolution === "1080P") {
|
|
||||||
return "1792x1024";
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
export function resolveReferenceAsset(req) {
|
|
||||||
const allAssets = [...(req.inputImages ?? []), ...(req.inputVideos ?? [])];
|
|
||||||
if (allAssets.length === 0) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
if (allAssets.length > 1) {
|
|
||||||
throw new Error("OpenAI video generation supports at most one reference image or video.");
|
|
||||||
}
|
|
||||||
const [asset] = allAssets;
|
|
||||||
if (!asset?.buffer) {
|
|
||||||
throw new Error("OpenAI video generation currently requires local image/video uploads for reference assets.");
|
|
||||||
}
|
|
||||||
const kind = (req.inputVideos?.length ?? 0) > 0 ? "video" : "image";
|
|
||||||
const mimeType = normalizeOptionalString(asset.mimeType) || (kind === "video" ? "video/mp4" : "image/png");
|
|
||||||
const extension = extensionForMime(mimeType)?.slice(1) ?? (mimeType.startsWith("video/") ? "mp4" : "png");
|
|
||||||
const fileName = normalizeOptionalString(asset.fileName) || `reference-${kind}.${extension}`;
|
|
||||||
return {
|
|
||||||
kind,
|
|
||||||
file: new File([bufferToBlobPart(asset.buffer)], fileName, { type: mimeType }),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
export async function fetchOpenAIVideoDownload(params) {
|
|
||||||
const { assertOkOrThrowHttpError, createProviderOperationTimeoutResolver, executeProviderOperationWithRetry, fetchProviderDownloadResponse, fetchWithTimeoutGuarded, } = await import("openclaw/plugin-sdk/provider-http");
|
|
||||||
const timeoutMs = createProviderOperationTimeoutResolver({
|
|
||||||
deadline: params.deadline,
|
|
||||||
defaultTimeoutMs: params.deadline.timeoutMs ?? DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
if (!params.allowPrivateNetwork && !params.dispatcherPolicy) {
|
|
||||||
const response = await fetchProviderDownloadResponse({
|
|
||||||
url: params.url,
|
|
||||||
init: params.init,
|
|
||||||
deadline: params.deadline,
|
|
||||||
fetchFn: params.fetchFn,
|
|
||||||
provider: "openai",
|
|
||||||
requestFailedMessage: "OpenAI video download failed",
|
|
||||||
});
|
|
||||||
return {
|
|
||||||
response,
|
|
||||||
release: async () => { },
|
|
||||||
};
|
|
||||||
}
|
|
||||||
return await executeProviderOperationWithRetry({
|
|
||||||
provider: "openai",
|
|
||||||
stage: "download",
|
|
||||||
operation: async () => {
|
|
||||||
const result = await fetchWithTimeoutGuarded(params.url, params.init, timeoutMs(), params.fetchFn, {
|
|
||||||
...(params.allowPrivateNetwork ? { ssrfPolicy: { allowPrivateNetwork: true } } : {}),
|
|
||||||
...(params.dispatcherPolicy ? { dispatcherPolicy: params.dispatcherPolicy } : {}),
|
|
||||||
auditContext: "openai-video-download",
|
|
||||||
});
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(result.response, "OpenAI video download failed");
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
catch (error) {
|
|
||||||
await result.release();
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
export function buildOpenAIVideoGenerationProvider({ isProviderApiKeyConfiguredAsync, getPluginConfig, }) {
|
|
||||||
return {
|
|
||||||
id: "1h-openai-video",
|
|
||||||
label: "1-h OpenAI Video",
|
|
||||||
defaultModel: DEFAULT_OPENAI_VIDEO_MODEL,
|
|
||||||
models: [DEFAULT_OPENAI_VIDEO_MODEL, "sora-2-pro", "gemini-video"],
|
|
||||||
isConfiguredAsync: async (ctx) => {
|
|
||||||
const pluginCfg = resolveVideoConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.GEMINI_WEB2API_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY)) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return await isProviderApiKeyConfiguredAsync({
|
|
||||||
provider: "openai",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxVideos: 1,
|
|
||||||
maxDurationSeconds: 12,
|
|
||||||
supportedDurationSeconds: OPENAI_VIDEO_SECONDS,
|
|
||||||
supportsSize: true,
|
|
||||||
sizes: OPENAI_VIDEO_SIZES,
|
|
||||||
},
|
|
||||||
imageToVideo: {
|
|
||||||
enabled: true,
|
|
||||||
maxVideos: 1,
|
|
||||||
maxInputImages: 1,
|
|
||||||
maxDurationSeconds: 12,
|
|
||||||
supportedDurationSeconds: OPENAI_VIDEO_SECONDS,
|
|
||||||
supportsSize: true,
|
|
||||||
sizes: OPENAI_VIDEO_SIZES,
|
|
||||||
},
|
|
||||||
videoToVideo: {
|
|
||||||
enabled: true,
|
|
||||||
maxVideos: 1,
|
|
||||||
maxInputVideos: 1,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateVideo(req) {
|
|
||||||
const pluginCfg = resolveVideoConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey = resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.GEMINI_WEB2API_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY);
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } = await import("openclaw/plugin-sdk/provider-auth-runtime");
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "openai",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
modelApi: "openai-responses",
|
|
||||||
});
|
|
||||||
if (!auth.apiKey || (auth.mode !== undefined && auth.mode !== "api-key")) {
|
|
||||||
throw new Error("OpenAI API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
const [{ assertOkOrThrowHttpError, createProviderOperationDeadline, createProviderOperationTimeoutResolver, pollProviderOperationJson, postJsonRequest, postMultipartRequest, readProviderJsonResponse, resolveProviderOperationTimeoutMs, resolveProviderHttpRequestConfig, sanitizeConfiguredModelProviderRequest, }, { downloadGeneratedVideoAsset, resolveGeneratedMediaMaxBytes },] = await Promise.all([
|
|
||||||
import("openclaw/plugin-sdk/provider-http"),
|
|
||||||
import("openclaw/plugin-sdk/media-generation-runtime"),
|
|
||||||
]);
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "OpenAI video generation",
|
|
||||||
});
|
|
||||||
const providerConfig = req.cfg.models?.providers?.openai;
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
...providerConfig?.request,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } = resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: resolveConfiguredOpenAIBaseUrl(req.cfg, pluginCfg),
|
|
||||||
defaultBaseUrl: DEFAULT_OPENAI_VIDEO_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
},
|
|
||||||
provider: "openai",
|
|
||||||
capability: "video",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
const model = normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_OPENAI_VIDEO_MODEL;
|
|
||||||
const seconds = resolveDurationSeconds(req.durationSeconds);
|
|
||||||
const size = resolveSize(req);
|
|
||||||
const referenceAsset = resolveReferenceAsset(req);
|
|
||||||
const isVideoEdit = referenceAsset?.kind === "video";
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
let response;
|
|
||||||
let release;
|
|
||||||
if (referenceAsset) {
|
|
||||||
const form = new FormData();
|
|
||||||
form.set("prompt", req.prompt);
|
|
||||||
if (isVideoEdit) {
|
|
||||||
form.set("video", referenceAsset.file);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
form.set("model", model);
|
|
||||||
if (seconds) {
|
|
||||||
form.set("seconds", seconds);
|
|
||||||
}
|
|
||||||
if (size) {
|
|
||||||
form.set("size", size);
|
|
||||||
}
|
|
||||||
form.set("input_reference", referenceAsset.file);
|
|
||||||
}
|
|
||||||
const multipartHeaders = new Headers(headers);
|
|
||||||
multipartHeaders.delete("Content-Type");
|
|
||||||
({ response, release } = await postMultipartRequest({
|
|
||||||
url: `${baseUrl}/videos${isVideoEdit ? "/edits" : ""}`,
|
|
||||||
headers: multipartHeaders,
|
|
||||||
body: form,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
const jsonHeaders = new Headers(headers);
|
|
||||||
jsonHeaders.set("Content-Type", "application/json");
|
|
||||||
const jsonBody = {
|
|
||||||
model,
|
|
||||||
prompt: req.prompt,
|
|
||||||
...(seconds ? { seconds } : {}),
|
|
||||||
...(size ? { size } : {}),
|
|
||||||
};
|
|
||||||
({ response, release } = await postJsonRequest({
|
|
||||||
url: `${baseUrl}/videos`,
|
|
||||||
headers: jsonHeaders,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
let submitted;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "OpenAI video generation failed");
|
|
||||||
submitted = await readProviderJsonResponse(response, "OpenAI video generation failed");
|
|
||||||
}
|
|
||||||
finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
const failureMessage = readOpenAIVideoFailureMessage(submitted);
|
|
||||||
if (failureMessage) {
|
|
||||||
throw new Error(failureMessage);
|
|
||||||
}
|
|
||||||
const videoId = normalizeOptionalString(submitted.id);
|
|
||||||
if (!videoId) {
|
|
||||||
throw new Error("OpenAI video generation response missing video id");
|
|
||||||
}
|
|
||||||
const completed = submitted.status === "completed"
|
|
||||||
? submitted
|
|
||||||
: await pollProviderOperationJson({
|
|
||||||
url: `${baseUrl}/videos/${videoId}`,
|
|
||||||
headers,
|
|
||||||
deadline: createProviderOperationDeadline({
|
|
||||||
timeoutMs: resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
}),
|
|
||||||
label: `OpenAI video generation task ${videoId}`,
|
|
||||||
}),
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
fetchFn,
|
|
||||||
maxAttempts: MAX_POLL_ATTEMPTS,
|
|
||||||
pollIntervalMs: POLL_INTERVAL_MS,
|
|
||||||
requestFailedMessage: "OpenAI video status request failed",
|
|
||||||
timeoutMessage: `OpenAI video generation task ${videoId} did not finish in time`,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
auditContext: "openai-video-status",
|
|
||||||
isComplete: (payload) => payload.status === "completed",
|
|
||||||
getFailureMessage: readOpenAIVideoFailureMessage,
|
|
||||||
});
|
|
||||||
const url = new URL(`${baseUrl}/videos/${videoId}/content`);
|
|
||||||
url.searchParams.set("variant", "video");
|
|
||||||
const video = await downloadGeneratedVideoAsset({
|
|
||||||
url: url.toString(),
|
|
||||||
timeoutMs: createProviderOperationTimeoutResolver({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
}),
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
fetchFn,
|
|
||||||
provider: "openai",
|
|
||||||
label: "OpenAI generated video download",
|
|
||||||
requestFailedMessage: "OpenAI video download failed",
|
|
||||||
maxBytes: resolveGeneratedMediaMaxBytes(req.cfg, "video"),
|
|
||||||
validateBinaryResponse: true,
|
|
||||||
fetchResponse: async ({ deadline: downloadDeadline }) => await fetchOpenAIVideoDownload({
|
|
||||||
url: url.toString(),
|
|
||||||
init: {
|
|
||||||
method: "GET",
|
|
||||||
headers: new Headers({
|
|
||||||
...Object.fromEntries(headers.entries()),
|
|
||||||
Accept: "application/binary",
|
|
||||||
}),
|
|
||||||
},
|
|
||||||
deadline: downloadDeadline,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
return {
|
|
||||||
videos: [video],
|
|
||||||
model: completed.model ?? submitted.model ?? model,
|
|
||||||
metadata: {
|
|
||||||
videoId,
|
|
||||||
status: completed.status,
|
|
||||||
seconds: completed.seconds ?? submitted.seconds,
|
|
||||||
size: completed.size ?? submitted.size,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
+9
-287
@@ -1,300 +1,22 @@
|
|||||||
{
|
{
|
||||||
"id": "1h-tools",
|
"id": "tools",
|
||||||
"name": "1-h Tools",
|
"name": "Tools",
|
||||||
"description": "Custom Tools and Media Providers (OpenAI & Gemini) for OpenClaw.",
|
"description": "Add Tools tools to OpenClaw.",
|
||||||
"version": "0.4.0",
|
"version": "0.1.0",
|
||||||
"categories": [
|
"configSchema": {
|
||||||
"media",
|
"type": "object",
|
||||||
"developer-tools"
|
"properties": {},
|
||||||
],
|
"additionalProperties": false
|
||||||
|
},
|
||||||
"activation": {
|
"activation": {
|
||||||
"onStartup": true
|
"onStartup": true
|
||||||
},
|
},
|
||||||
"skills": [
|
"skills": [
|
||||||
"./skills"
|
"./skills"
|
||||||
],
|
],
|
||||||
"setup": {
|
|
||||||
"providers": [
|
|
||||||
{
|
|
||||||
"id": "1h-openai-video",
|
|
||||||
"envVars": [
|
|
||||||
"OPENAI_VIDEO_API_KEY",
|
|
||||||
"CCLOAD_API_KEY",
|
|
||||||
"BIFROST_API_KEY",
|
|
||||||
"OPENAI_API_KEY"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "1h-openai-image",
|
|
||||||
"envVars": [
|
|
||||||
"OPENAI_IMAGE_API_KEY",
|
|
||||||
"CCLOAD_API_KEY",
|
|
||||||
"BIFROST_API_KEY",
|
|
||||||
"OPENAI_API_KEY"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "1h-gemini-image",
|
|
||||||
"envVars": [
|
|
||||||
"CCLOAD_API_KEY",
|
|
||||||
"GEMINI_API_KEY",
|
|
||||||
"BIFROST_API_KEY"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"contracts": {
|
"contracts": {
|
||||||
"tools": [
|
"tools": [
|
||||||
"echo"
|
"echo"
|
||||||
],
|
|
||||||
"videoGenerationProviders": [
|
|
||||||
"1h-openai-video"
|
|
||||||
],
|
|
||||||
"imageGenerationProviders": [
|
|
||||||
"1h-openai-image",
|
|
||||||
"1h-gemini-image"
|
|
||||||
]
|
]
|
||||||
},
|
|
||||||
"videoGenerationProviderMetadata": {
|
|
||||||
"1h-openai-video": {
|
|
||||||
"authSignals": [
|
|
||||||
{
|
|
||||||
"provider": "openai"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"imageGenerationProviderMetadata": {
|
|
||||||
"1h-openai-image": {
|
|
||||||
"authSignals": [
|
|
||||||
{
|
|
||||||
"provider": "openai"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"1h-gemini-image": {
|
|
||||||
"authSignals": [
|
|
||||||
{
|
|
||||||
"provider": "google"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"configContracts": {
|
|
||||||
"secretInputs": {
|
|
||||||
"paths": [
|
|
||||||
{
|
|
||||||
"path": "video.apiKey",
|
|
||||||
"expected": "string",
|
|
||||||
"ownerKind": "capability"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"path": "openaiVideo.apiKey",
|
|
||||||
"expected": "string",
|
|
||||||
"ownerKind": "capability"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"path": "image.apiKey",
|
|
||||||
"expected": "string",
|
|
||||||
"ownerKind": "capability"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"path": "openaiImage.apiKey",
|
|
||||||
"expected": "string",
|
|
||||||
"ownerKind": "capability"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"path": "geminiImage.apiKey",
|
|
||||||
"expected": "string",
|
|
||||||
"ownerKind": "capability"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"uiHints": {
|
|
||||||
"openaiVideo.apiKey": {
|
|
||||||
"label": "OpenAI Video API Key",
|
|
||||||
"sensitive": true
|
|
||||||
},
|
|
||||||
"openaiImage.apiKey": {
|
|
||||||
"label": "OpenAI Image API Key",
|
|
||||||
"sensitive": true
|
|
||||||
},
|
|
||||||
"geminiImage.apiKey": {
|
|
||||||
"label": "Gemini Image API Key",
|
|
||||||
"sensitive": true
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"configSchema": {
|
|
||||||
"type": "object",
|
|
||||||
"additionalProperties": false,
|
|
||||||
"properties": {
|
|
||||||
"video": {
|
|
||||||
"type": "object",
|
|
||||||
"title": "Video Generation Settings (Alias for openaiVideo)",
|
|
||||||
"additionalProperties": false,
|
|
||||||
"properties": {
|
|
||||||
"baseUrl": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"apiKey": {
|
|
||||||
"type": [
|
|
||||||
"string",
|
|
||||||
"object"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"allowPrivateNetwork": {
|
|
||||||
"type": "boolean",
|
|
||||||
"default": true
|
|
||||||
},
|
|
||||||
"defaultModel": {
|
|
||||||
"type": "string",
|
|
||||||
"default": "gemini-video"
|
|
||||||
},
|
|
||||||
"models": {
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"openaiVideo": {
|
|
||||||
"type": "object",
|
|
||||||
"title": "OpenAI Video Generation Settings (1h-openai-video)",
|
|
||||||
"additionalProperties": false,
|
|
||||||
"properties": {
|
|
||||||
"baseUrl": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"apiKey": {
|
|
||||||
"type": [
|
|
||||||
"string",
|
|
||||||
"object"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"allowPrivateNetwork": {
|
|
||||||
"type": "boolean",
|
|
||||||
"default": true
|
|
||||||
},
|
|
||||||
"defaultModel": {
|
|
||||||
"type": "string",
|
|
||||||
"default": "gemini-video"
|
|
||||||
},
|
|
||||||
"models": {
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"image": {
|
|
||||||
"type": "object",
|
|
||||||
"title": "Image Generation Settings (Alias for openaiImage)",
|
|
||||||
"additionalProperties": false,
|
|
||||||
"properties": {
|
|
||||||
"baseUrl": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"apiKey": {
|
|
||||||
"type": [
|
|
||||||
"string",
|
|
||||||
"object"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"allowPrivateNetwork": {
|
|
||||||
"type": "boolean",
|
|
||||||
"default": true
|
|
||||||
},
|
|
||||||
"defaultModel": {
|
|
||||||
"type": "string",
|
|
||||||
"default": "gpt-image-2"
|
|
||||||
},
|
|
||||||
"responseFormat": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"url",
|
|
||||||
"b64_json"
|
|
||||||
],
|
|
||||||
"default": "url"
|
|
||||||
},
|
|
||||||
"models": {
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"openaiImage": {
|
|
||||||
"type": "object",
|
|
||||||
"title": "OpenAI Image Generation Settings (1h-openai-image)",
|
|
||||||
"additionalProperties": false,
|
|
||||||
"properties": {
|
|
||||||
"baseUrl": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"apiKey": {
|
|
||||||
"type": [
|
|
||||||
"string",
|
|
||||||
"object"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"allowPrivateNetwork": {
|
|
||||||
"type": "boolean",
|
|
||||||
"default": true
|
|
||||||
},
|
|
||||||
"defaultModel": {
|
|
||||||
"type": "string",
|
|
||||||
"default": "gpt-image-2"
|
|
||||||
},
|
|
||||||
"responseFormat": {
|
|
||||||
"type": "string",
|
|
||||||
"enum": [
|
|
||||||
"url",
|
|
||||||
"b64_json"
|
|
||||||
],
|
|
||||||
"default": "url"
|
|
||||||
},
|
|
||||||
"models": {
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"geminiImage": {
|
|
||||||
"type": "object",
|
|
||||||
"title": "Gemini Image Generation Settings (1h-gemini-image)",
|
|
||||||
"additionalProperties": false,
|
|
||||||
"properties": {
|
|
||||||
"baseUrl": {
|
|
||||||
"type": "string"
|
|
||||||
},
|
|
||||||
"apiKey": {
|
|
||||||
"type": [
|
|
||||||
"string",
|
|
||||||
"object"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"allowPrivateNetwork": {
|
|
||||||
"type": "boolean",
|
|
||||||
"default": true
|
|
||||||
},
|
|
||||||
"defaultModel": {
|
|
||||||
"type": "string",
|
|
||||||
"default": "gemini-3.1-flash-image"
|
|
||||||
},
|
|
||||||
"models": {
|
|
||||||
"type": "array",
|
|
||||||
"items": {
|
|
||||||
"type": "string"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
+1
-1
@@ -27,7 +27,7 @@
|
|||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"openclaw": "latest",
|
"openclaw": "latest",
|
||||||
"typescript": "7.0.2",
|
"typescript": "7.0.2",
|
||||||
"vitest": "^3.2.0"
|
"vitest": "^5.0.0"
|
||||||
},
|
},
|
||||||
"openclaw": {
|
"openclaw": {
|
||||||
"extensions": [
|
"extensions": [
|
||||||
|
|||||||
@@ -1,78 +0,0 @@
|
|||||||
# 1-h Image Providers (`1h-openai-image` & `1h-gemini-image`)
|
|
||||||
|
|
||||||
OpenClaw 图像生成驱动模块集合,根据上游协议严格分离为两个专职驱动,彻底杜绝跨协议调用引发的 400 错配。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 包含驱动一览
|
|
||||||
|
|
||||||
| Provider ID | 对应协议 | 默认模型 | 适用场景 / 支持模型 |
|
|
||||||
| :--- | :--- | :--- | :--- |
|
|
||||||
| **`1h-openai-image`** | OpenAI 标准图像协议<br>(`POST /v1/images/generations`) | `gpt-image-2` | 专为号池/公益站的 OpenAI 协议图像生成打造:<br>• `gpt-image-2`<br>• `gpt-image-2.5`<br>• `gpt-image-2.5-flare`<br>• `gpt-image-2.5-sunburst` |
|
|
||||||
| **`1h-gemini-image`** | Google Gemini 原生内容协议<br>(`POST /v1beta/models/...:generateContent`) | `gemini-3.1-flash-image` | 专为渠道 87(CLI-Proxy)的原生 Gemini 图像生成打造:<br>• `gemini-3.1-flash-image` |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 核心特性
|
|
||||||
|
|
||||||
1. **`1h-openai-image`(OpenAI 协议)**:
|
|
||||||
* **尺寸安全映射**:自动将 16:9 / 9:16 安全归一化为通用尺寸(`1792x1024` / `1024x1792` / `1536x1024`),修复 Issue #168665;
|
|
||||||
* **URL 优先与双模自适应**:默认使用 `response_format=url`,完美契合公益站防 400 校验,并自动将链接异步下载为本地图片文件;
|
|
||||||
* **图生图编辑**:传入 `inputImages` 时自动切至 Multipart 表单走 `/v1/images/edits`。
|
|
||||||
|
|
||||||
2. **`1h-gemini-image`(Gemini 协议)**:
|
|
||||||
* **原生 JSON 结构**:以标准的 `contents[].parts` 构造请求,并在返回时原生提取 `inlineData` Base64 流;
|
|
||||||
* **无损回退链**:作为主模型的首选备用渠道,当 OpenAI 渠道拥堵超时时,以纯正 Gemini 协议秒级接管。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 配置说明
|
|
||||||
|
|
||||||
在 `~/.openclaw/openclaw.json` 中配置:
|
|
||||||
|
|
||||||
```json5
|
|
||||||
{
|
|
||||||
"plugins": {
|
|
||||||
"entries": {
|
|
||||||
"1h-tools": {
|
|
||||||
"enabled": true,
|
|
||||||
"config": {
|
|
||||||
"openaiImage": {
|
|
||||||
"baseUrl": "http://host.incus.internal:8080/v1",
|
|
||||||
"defaultModel": "gpt-image-2",
|
|
||||||
"apiKey": "1842894d437ee54443a99390f79811ba9ebb785175af340b492e3398dc317921",
|
|
||||||
"responseFormat": "url",
|
|
||||||
"allowPrivateNetwork": true
|
|
||||||
},
|
|
||||||
"geminiImage": {
|
|
||||||
"baseUrl": "http://host.incus.internal:8080/v1beta",
|
|
||||||
"defaultModel": "gemini-3.1-flash-image",
|
|
||||||
"apiKey": "1842894d437ee54443a99390f79811ba9ebb785175af340b492e3398dc317921",
|
|
||||||
"allowPrivateNetwork": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"agents": {
|
|
||||||
"defaults": {
|
|
||||||
"mediaModels": {
|
|
||||||
"image": {
|
|
||||||
"primary": "1h-openai-image/gpt-image-2",
|
|
||||||
"fallbacks": [
|
|
||||||
"1h-gemini-image/gemini-3.1-flash-image"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 使用方法
|
|
||||||
|
|
||||||
配置完成后,OpenClaw 内置的 `image_generate` 工具即可自动调度:
|
|
||||||
* **正常请求**:默认走 `1h-openai-image/gpt-image-2`;
|
|
||||||
* **容灾回退**:若主模型超时(如公益站排队超时 499),系统自动使用纯正的 `1h-gemini-image/gemini-3.1-flash-image` 协议回退,绝不报 400。
|
|
||||||
@@ -1,302 +0,0 @@
|
|||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
import type {
|
|
||||||
GeneratedImageAsset,
|
|
||||||
ImageGenerationProvider,
|
|
||||||
ImageGenerationRequest,
|
|
||||||
ImageGenerationResult,
|
|
||||||
} from "openclaw/plugin-sdk/image-generation";
|
|
||||||
|
|
||||||
const DEFAULT_GEMINI_IMAGE_BASE_URL = "http://host.incus.internal:8080/v1beta";
|
|
||||||
const DEFAULT_GEMINI_IMAGE_MODEL = "gemini-3.1-flash-image";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface GeminiImagePluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveGeminiImageConfig(
|
|
||||||
cfg: ImageGenerationRequest["cfg"] | undefined,
|
|
||||||
apiPluginConfig?: Record<string, unknown>,
|
|
||||||
): GeminiImagePluginConfig | undefined {
|
|
||||||
const toolsConfig = (apiPluginConfig ?? cfg?.plugins?.entries?.["1h-tools"]?.config) as
|
|
||||||
| {
|
|
||||||
geminiImage?: GeminiImagePluginConfig;
|
|
||||||
image?: GeminiImagePluginConfig;
|
|
||||||
}
|
|
||||||
| undefined;
|
|
||||||
|
|
||||||
return toolsConfig?.geminiImage ?? toolsConfig?.image;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveConfiguredGeminiImageBaseUrl(
|
|
||||||
pluginCfg?: GeminiImagePluginConfig,
|
|
||||||
): string {
|
|
||||||
return (
|
|
||||||
normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.GEMINI_IMAGE_BASE_URL) ??
|
|
||||||
DEFAULT_GEMINI_IMAGE_BASE_URL
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function buildGeminiImageGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig,
|
|
||||||
}: {
|
|
||||||
isProviderApiKeyConfiguredAsync: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfiguredAsync"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): ImageGenerationProvider {
|
|
||||||
return {
|
|
||||||
id: "1h-gemini-image",
|
|
||||||
label: "1-h Gemini Image",
|
|
||||||
defaultModel: DEFAULT_GEMINI_IMAGE_MODEL,
|
|
||||||
models: [DEFAULT_GEMINI_IMAGE_MODEL],
|
|
||||||
isConfiguredAsync: async (ctx) => {
|
|
||||||
const pluginCfg = resolveGeminiImageConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (
|
|
||||||
resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.GEMINI_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY)
|
|
||||||
) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return await isProviderApiKeyConfiguredAsync({
|
|
||||||
provider: "google",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxCount: 1,
|
|
||||||
supportsSize: false,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: false,
|
|
||||||
},
|
|
||||||
edit: {
|
|
||||||
enabled: true,
|
|
||||||
maxCount: 1,
|
|
||||||
maxInputImages: 3,
|
|
||||||
supportsSize: false,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: false,
|
|
||||||
},
|
|
||||||
geometry: {
|
|
||||||
aspectRatios: ["1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3", "21:9"],
|
|
||||||
},
|
|
||||||
output: {
|
|
||||||
formats: ["jpeg", "png"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateImage(req: ImageGenerationRequest): Promise<ImageGenerationResult> {
|
|
||||||
const pluginCfg = resolveGeminiImageConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey =
|
|
||||||
resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.GEMINI_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY);
|
|
||||||
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } = await import(
|
|
||||||
"openclaw/plugin-sdk/provider-auth-runtime"
|
|
||||||
);
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "google",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
});
|
|
||||||
if (!auth.apiKey) {
|
|
||||||
throw new Error("Gemini Image API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
|
|
||||||
const [
|
|
||||||
{
|
|
||||||
assertOkOrThrowHttpError,
|
|
||||||
createProviderOperationDeadline,
|
|
||||||
postJsonRequest,
|
|
||||||
readProviderJsonResponse,
|
|
||||||
resolveProviderHttpRequestConfig,
|
|
||||||
resolveProviderOperationTimeoutMs,
|
|
||||||
sanitizeConfiguredModelProviderRequest,
|
|
||||||
},
|
|
||||||
] = await Promise.all([import("openclaw/plugin-sdk/provider-http")]);
|
|
||||||
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "Gemini Image Generation",
|
|
||||||
});
|
|
||||||
|
|
||||||
const rawBaseUrl = resolveConfiguredGeminiImageBaseUrl(pluginCfg);
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
|
|
||||||
resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: rawBaseUrl,
|
|
||||||
defaultBaseUrl: DEFAULT_GEMINI_IMAGE_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
},
|
|
||||||
provider: "google",
|
|
||||||
capability: "image",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
|
|
||||||
const model =
|
|
||||||
normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_GEMINI_IMAGE_MODEL;
|
|
||||||
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
|
|
||||||
const parts: Array<Record<string, unknown>> = [];
|
|
||||||
const inputImages = req.inputImages ?? [];
|
|
||||||
|
|
||||||
for (const img of inputImages) {
|
|
||||||
parts.push({
|
|
||||||
inlineData: {
|
|
||||||
mimeType: img.mimeType || "image/jpeg",
|
|
||||||
data: img.buffer.toString("base64"),
|
|
||||||
},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
parts.push({
|
|
||||||
text: req.prompt,
|
|
||||||
});
|
|
||||||
|
|
||||||
const jsonBody: Record<string, unknown> = {
|
|
||||||
contents: [
|
|
||||||
{
|
|
||||||
parts,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
generationConfig: {
|
|
||||||
responseMimeType: "image/jpeg",
|
|
||||||
},
|
|
||||||
};
|
|
||||||
|
|
||||||
const normalizedBaseUrl = baseUrl.endsWith("/") ? baseUrl.slice(0, -1) : baseUrl;
|
|
||||||
const url = `${normalizedBaseUrl}/models/${model}:generateContent`;
|
|
||||||
|
|
||||||
const { response, release } = await postJsonRequest({
|
|
||||||
url,
|
|
||||||
headers,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
});
|
|
||||||
|
|
||||||
interface GeminiGenerateContentResponse {
|
|
||||||
candidates?: Array<{
|
|
||||||
content?: {
|
|
||||||
parts?: Array<{
|
|
||||||
text?: string;
|
|
||||||
inlineData?: {
|
|
||||||
mimeType?: string;
|
|
||||||
data?: string;
|
|
||||||
};
|
|
||||||
}>;
|
|
||||||
};
|
|
||||||
finishReason?: string;
|
|
||||||
}>;
|
|
||||||
error?: {
|
|
||||||
code?: number;
|
|
||||||
message?: string;
|
|
||||||
status?: string;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
let payload: GeminiGenerateContentResponse;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "Gemini image generation request failed");
|
|
||||||
payload = await readProviderJsonResponse<GeminiGenerateContentResponse>(
|
|
||||||
response,
|
|
||||||
"Gemini image generation failed",
|
|
||||||
);
|
|
||||||
} finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
|
|
||||||
if (payload.error?.message) {
|
|
||||||
throw new Error(`Gemini image generation error: ${payload.error.message}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
const candidateParts = payload.candidates?.[0]?.content?.parts ?? [];
|
|
||||||
const generatedImages: GeneratedImageAsset[] = [];
|
|
||||||
|
|
||||||
for (let i = 0; i < candidateParts.length; i++) {
|
|
||||||
const part = candidateParts[i];
|
|
||||||
if (part.inlineData?.data) {
|
|
||||||
const buffer = Buffer.from(part.inlineData.data, "base64");
|
|
||||||
const mimeType = part.inlineData.mimeType || "image/jpeg";
|
|
||||||
const ext = mimeType.includes("png") ? "png" : "jpg";
|
|
||||||
generatedImages.push({
|
|
||||||
buffer,
|
|
||||||
mimeType,
|
|
||||||
fileName: `image-${i + 1}.${ext}`,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (generatedImages.length === 0) {
|
|
||||||
const textMessage = candidateParts.find((p) => p.text)?.text;
|
|
||||||
throw new Error(
|
|
||||||
textMessage
|
|
||||||
? `Gemini returned text instead of an image: ${textMessage}`
|
|
||||||
: "Gemini image generation returned no image data",
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
images: generatedImages,
|
|
||||||
model,
|
|
||||||
metadata: {
|
|
||||||
finishReason: payload.candidates?.[0]?.finishReason,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,427 +0,0 @@
|
|||||||
import { bufferToBlobPart } from "openclaw/plugin-sdk/blob-runtime";
|
|
||||||
import { extensionForMime } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
import type {
|
|
||||||
GeneratedImageAsset,
|
|
||||||
ImageGenerationProvider,
|
|
||||||
ImageGenerationRequest,
|
|
||||||
ImageGenerationResult,
|
|
||||||
} from "openclaw/plugin-sdk/image-generation";
|
|
||||||
|
|
||||||
const DEFAULT_OPENAI_IMAGE_BASE_URL = "https://api.openai.com/v1";
|
|
||||||
const DEFAULT_OPENAI_IMAGE_MODEL = "gpt-image-2";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
const DEFAULT_SIZE = "1024x1024";
|
|
||||||
|
|
||||||
const SUPPORTED_SIZES = [
|
|
||||||
"1024x1024",
|
|
||||||
"1536x1024",
|
|
||||||
"1024x1536",
|
|
||||||
"1792x1024",
|
|
||||||
"1024x1792",
|
|
||||||
"2048x2048",
|
|
||||||
"2048x1152",
|
|
||||||
"3840x2160",
|
|
||||||
] as const;
|
|
||||||
|
|
||||||
const SUPPORTED_ASPECT_RATIOS = [
|
|
||||||
"1:1",
|
|
||||||
"16:9",
|
|
||||||
"9:16",
|
|
||||||
"4:3",
|
|
||||||
"3:4",
|
|
||||||
"3:2",
|
|
||||||
"2:3",
|
|
||||||
"21:9",
|
|
||||||
] as const;
|
|
||||||
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface ImagePluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
responseFormat?: "url" | "b64_json";
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveImageConfig(
|
|
||||||
cfg: ImageGenerationRequest["cfg"] | undefined,
|
|
||||||
apiPluginConfig?: Record<string, unknown>,
|
|
||||||
): ImagePluginConfig | undefined {
|
|
||||||
const toolsConfig = (apiPluginConfig ?? cfg?.plugins?.entries?.["1h-tools"]?.config) as
|
|
||||||
| {
|
|
||||||
openaiImage?: ImagePluginConfig;
|
|
||||||
image?: ImagePluginConfig;
|
|
||||||
}
|
|
||||||
| undefined;
|
|
||||||
|
|
||||||
return toolsConfig?.openaiImage ?? toolsConfig?.image;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveConfiguredImageBaseUrl(
|
|
||||||
cfg: ImageGenerationRequest["cfg"],
|
|
||||||
pluginCfg?: ImagePluginConfig,
|
|
||||||
): string {
|
|
||||||
return (
|
|
||||||
normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_BASE_URL) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_BASE_URL) ??
|
|
||||||
normalizeOptionalString(cfg?.models?.providers?.openai?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_BASE_URL) ??
|
|
||||||
DEFAULT_OPENAI_IMAGE_BASE_URL
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveImageSize(params: {
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
model?: string;
|
|
||||||
}): string {
|
|
||||||
const explicitSize = normalizeOptionalString(params.size);
|
|
||||||
if (explicitSize) {
|
|
||||||
return explicitSize;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Safe aspect ratio normalization preventing issue #168665 (proxy rejecting 2048x1152)
|
|
||||||
switch (normalizeOptionalString(params.aspectRatio)) {
|
|
||||||
case "1:1":
|
|
||||||
return "1024x1024";
|
|
||||||
case "16:9":
|
|
||||||
return "1792x1024";
|
|
||||||
case "9:16":
|
|
||||||
return "1024x1792";
|
|
||||||
case "4:3":
|
|
||||||
case "3:2":
|
|
||||||
return "1536x1024";
|
|
||||||
case "3:4":
|
|
||||||
case "2:3":
|
|
||||||
return "1024x1536";
|
|
||||||
case "21:9":
|
|
||||||
return "1792x1024";
|
|
||||||
default:
|
|
||||||
return DEFAULT_SIZE;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
export function buildOpenAIImageGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig,
|
|
||||||
}: {
|
|
||||||
isProviderApiKeyConfiguredAsync: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfiguredAsync"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): ImageGenerationProvider {
|
|
||||||
return {
|
|
||||||
id: "1h-openai-image",
|
|
||||||
label: "1-h OpenAI Image",
|
|
||||||
defaultModel: DEFAULT_OPENAI_IMAGE_MODEL,
|
|
||||||
models: [DEFAULT_OPENAI_IMAGE_MODEL, "gpt-image-2.5", "gpt-image-2.5-flare", "gpt-image-2.5-sunburst"],
|
|
||||||
isConfiguredAsync: async (ctx) => {
|
|
||||||
const pluginCfg = resolveImageConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (
|
|
||||||
resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_API_KEY)
|
|
||||||
) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return await isProviderApiKeyConfiguredAsync({
|
|
||||||
provider: "openai",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxCount: 4,
|
|
||||||
supportsSize: true,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: true,
|
|
||||||
},
|
|
||||||
edit: {
|
|
||||||
enabled: true,
|
|
||||||
maxCount: 4,
|
|
||||||
maxInputImages: 5,
|
|
||||||
supportsSize: true,
|
|
||||||
supportsAspectRatio: true,
|
|
||||||
supportsResolution: true,
|
|
||||||
},
|
|
||||||
geometry: {
|
|
||||||
sizes: [...SUPPORTED_SIZES],
|
|
||||||
aspectRatios: [...SUPPORTED_ASPECT_RATIOS],
|
|
||||||
resolutions: ["1K", "2K", "4K"],
|
|
||||||
},
|
|
||||||
output: {
|
|
||||||
formats: ["png", "jpeg", "webp"],
|
|
||||||
qualities: ["low", "medium", "high", "auto"],
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateImage(req: ImageGenerationRequest): Promise<ImageGenerationResult> {
|
|
||||||
const pluginCfg = resolveImageConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey =
|
|
||||||
resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_IMAGE_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.BIFROST_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_API_KEY);
|
|
||||||
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } = await import(
|
|
||||||
"openclaw/plugin-sdk/provider-auth-runtime"
|
|
||||||
);
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "openai",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
modelApi: "openai-responses",
|
|
||||||
});
|
|
||||||
if (!auth.apiKey) {
|
|
||||||
throw new Error("OpenAI / Bifrost Image API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
|
|
||||||
const [
|
|
||||||
{
|
|
||||||
assertOkOrThrowHttpError,
|
|
||||||
createProviderOperationDeadline,
|
|
||||||
createProviderOperationTimeoutResolver,
|
|
||||||
fetchWithTimeoutGuarded,
|
|
||||||
postJsonRequest,
|
|
||||||
postMultipartRequest,
|
|
||||||
readProviderJsonResponse,
|
|
||||||
resolveProviderHttpRequestConfig,
|
|
||||||
resolveProviderOperationTimeoutMs,
|
|
||||||
sanitizeConfiguredModelProviderRequest,
|
|
||||||
},
|
|
||||||
] = await Promise.all([import("openclaw/plugin-sdk/provider-http")]);
|
|
||||||
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "OpenAI Image Generation",
|
|
||||||
});
|
|
||||||
|
|
||||||
const providerConfig = req.cfg?.models?.providers?.openai;
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
...providerConfig?.request,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
|
|
||||||
const rawBaseUrl = resolveConfiguredImageBaseUrl(req.cfg, pluginCfg);
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
|
|
||||||
resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: rawBaseUrl,
|
|
||||||
defaultBaseUrl: DEFAULT_OPENAI_IMAGE_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
},
|
|
||||||
provider: "openai",
|
|
||||||
capability: "image",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
|
|
||||||
const model =
|
|
||||||
normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_OPENAI_IMAGE_MODEL;
|
|
||||||
|
|
||||||
const size = resolveImageSize({
|
|
||||||
size: req.size,
|
|
||||||
aspectRatio: req.aspectRatio,
|
|
||||||
model,
|
|
||||||
});
|
|
||||||
|
|
||||||
const count = Math.max(1, Math.min(req.count ?? 1, 4));
|
|
||||||
const inputImages = req.inputImages ?? [];
|
|
||||||
const isEdit = inputImages.length > 0;
|
|
||||||
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
|
|
||||||
let response: Response;
|
|
||||||
let release: () => Promise<void>;
|
|
||||||
|
|
||||||
const responseFormat = pluginCfg?.responseFormat ?? "url";
|
|
||||||
|
|
||||||
if (isEdit) {
|
|
||||||
const form = new FormData();
|
|
||||||
form.set("model", model);
|
|
||||||
form.set("prompt", req.prompt);
|
|
||||||
form.set("n", String(count));
|
|
||||||
form.set("size", size);
|
|
||||||
form.set("response_format", responseFormat);
|
|
||||||
|
|
||||||
inputImages.forEach((img, idx) => {
|
|
||||||
const mime = img.mimeType || "image/png";
|
|
||||||
const ext = extensionForMime(mime)?.slice(1) || "png";
|
|
||||||
const name = img.fileName || `input-${idx + 1}.${ext}`;
|
|
||||||
form.append(
|
|
||||||
idx === 0 ? "image" : "image[]",
|
|
||||||
new File([bufferToBlobPart(img.buffer)], name, { type: mime }),
|
|
||||||
);
|
|
||||||
});
|
|
||||||
|
|
||||||
const multipartHeaders = new Headers(headers);
|
|
||||||
multipartHeaders.delete("Content-Type");
|
|
||||||
|
|
||||||
({ response, release } = await postMultipartRequest({
|
|
||||||
url: `${baseUrl}/images/edits`,
|
|
||||||
headers: multipartHeaders,
|
|
||||||
body: form,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
} else {
|
|
||||||
const jsonHeaders = new Headers(headers);
|
|
||||||
jsonHeaders.set("Content-Type", "application/json");
|
|
||||||
|
|
||||||
const jsonBody: Record<string, unknown> = {
|
|
||||||
model,
|
|
||||||
prompt: req.prompt,
|
|
||||||
n: count,
|
|
||||||
size,
|
|
||||||
response_format: responseFormat,
|
|
||||||
};
|
|
||||||
|
|
||||||
if (req.quality) {
|
|
||||||
jsonBody.quality = req.quality;
|
|
||||||
}
|
|
||||||
|
|
||||||
({ response, release } = await postJsonRequest({
|
|
||||||
url: `${baseUrl}/images/generations`,
|
|
||||||
headers: jsonHeaders,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
|
|
||||||
interface OpenAIImageApiResponse {
|
|
||||||
created?: number;
|
|
||||||
data?: Array<{
|
|
||||||
b64_json?: string;
|
|
||||||
url?: string;
|
|
||||||
revised_prompt?: string;
|
|
||||||
}>;
|
|
||||||
error?: {
|
|
||||||
message?: string;
|
|
||||||
code?: string;
|
|
||||||
type?: string;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
let payload: OpenAIImageApiResponse;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "Image generation request failed");
|
|
||||||
payload = await readProviderJsonResponse<OpenAIImageApiResponse>(
|
|
||||||
response,
|
|
||||||
"OpenAI image generation failed",
|
|
||||||
);
|
|
||||||
} finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
|
|
||||||
if (payload.error?.message) {
|
|
||||||
throw new Error(`Image generation error: ${payload.error.message}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
const dataItems = payload.data ?? [];
|
|
||||||
if (dataItems.length === 0) {
|
|
||||||
throw new Error("Image generation returned no image data");
|
|
||||||
}
|
|
||||||
|
|
||||||
const generatedImages: GeneratedImageAsset[] = [];
|
|
||||||
const downloadTimeout = createProviderOperationTimeoutResolver({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
|
|
||||||
for (let i = 0; i < dataItems.length; i++) {
|
|
||||||
const item = dataItems[i];
|
|
||||||
if (item.b64_json) {
|
|
||||||
const buffer = Buffer.from(item.b64_json, "base64");
|
|
||||||
generatedImages.push({
|
|
||||||
buffer,
|
|
||||||
mimeType: "image/png",
|
|
||||||
fileName: `image-${i + 1}.png`,
|
|
||||||
revisedPrompt: item.revised_prompt,
|
|
||||||
});
|
|
||||||
} else if (item.url) {
|
|
||||||
const dlResult = await fetchWithTimeoutGuarded(
|
|
||||||
item.url,
|
|
||||||
{ method: "GET" },
|
|
||||||
downloadTimeout(),
|
|
||||||
fetchFn,
|
|
||||||
{
|
|
||||||
allowPrivateNetwork,
|
|
||||||
auditContext: "openai-image-download",
|
|
||||||
},
|
|
||||||
);
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(dlResult.response, "Failed to download generated image");
|
|
||||||
const arrayBuf = await dlResult.response.arrayBuffer();
|
|
||||||
const mime = dlResult.response.headers.get("content-type") || "image/png";
|
|
||||||
const ext = extensionForMime(mime)?.slice(1) || "png";
|
|
||||||
generatedImages.push({
|
|
||||||
buffer: Buffer.from(arrayBuf),
|
|
||||||
mimeType: mime,
|
|
||||||
fileName: `image-${i + 1}.${ext}`,
|
|
||||||
revisedPrompt: item.revised_prompt,
|
|
||||||
});
|
|
||||||
} finally {
|
|
||||||
await dlResult.release();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (generatedImages.length === 0) {
|
|
||||||
throw new Error("Failed to extract any valid image from provider response");
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
images: generatedImages,
|
|
||||||
model,
|
|
||||||
metadata: {
|
|
||||||
created: payload.created,
|
|
||||||
size,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
+5
-34
@@ -1,38 +1,9 @@
|
|||||||
import { describe, expect, it, vi } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
import entry from "./index.js";
|
import entry from "./index.js";
|
||||||
|
import { getToolPluginMetadata } from "openclaw/plugin-sdk/tool-plugin";
|
||||||
|
|
||||||
describe("1h-tools", () => {
|
describe("tools", () => {
|
||||||
it("registers tool, video provider, and both image providers", () => {
|
it("declares tool metadata", () => {
|
||||||
const registeredTools: unknown[] = [];
|
expect(getToolPluginMetadata(entry)?.tools.map((tool) => tool.name)).toEqual(["echo"]);
|
||||||
const registeredVideoProviders: unknown[] = [];
|
|
||||||
const registeredImageProviders: unknown[] = [];
|
|
||||||
|
|
||||||
const mockApi = {
|
|
||||||
runtime: {
|
|
||||||
modelAuth: {
|
|
||||||
isProviderApiKeyConfiguredAsync: vi.fn().mockResolvedValue(false),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
pluginConfig: {},
|
|
||||||
registerTool: vi.fn((tool) => registeredTools.push(tool)),
|
|
||||||
registerVideoGenerationProvider: vi.fn((provider) =>
|
|
||||||
registeredVideoProviders.push(provider),
|
|
||||||
),
|
|
||||||
registerImageGenerationProvider: vi.fn((provider) =>
|
|
||||||
registeredImageProviders.push(provider),
|
|
||||||
),
|
|
||||||
};
|
|
||||||
|
|
||||||
entry.register(mockApi as any);
|
|
||||||
|
|
||||||
expect(mockApi.registerTool).toHaveBeenCalledTimes(1);
|
|
||||||
expect(registeredTools[0]).toMatchObject({ name: "echo" });
|
|
||||||
|
|
||||||
expect(mockApi.registerVideoGenerationProvider).toHaveBeenCalledTimes(1);
|
|
||||||
expect(registeredVideoProviders[0]).toMatchObject({ id: "1h-openai-video" });
|
|
||||||
|
|
||||||
expect(mockApi.registerImageGenerationProvider).toHaveBeenCalledTimes(2);
|
|
||||||
expect(registeredImageProviders[0]).toMatchObject({ id: "1h-openai-image" });
|
|
||||||
expect(registeredImageProviders[1]).toMatchObject({ id: "1h-gemini-image" });
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
+15
-48
@@ -1,51 +1,18 @@
|
|||||||
import { jsonResult } from "openclaw/plugin-sdk/core";
|
import { Type } from "typebox";
|
||||||
import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
|
import { defineToolPlugin } from "openclaw/plugin-sdk/tool-plugin";
|
||||||
import { echoTool } from "./tools/echo.js";
|
|
||||||
import { buildOpenAIVideoGenerationProvider } from "./video/openai.js";
|
|
||||||
import { buildOpenAIImageGenerationProvider } from "./image/openai.js";
|
|
||||||
import { buildGeminiImageGenerationProvider } from "./image/gemini.js";
|
|
||||||
|
|
||||||
const entry = definePluginEntry({
|
export default defineToolPlugin({
|
||||||
id: "1h-tools",
|
id: "tools",
|
||||||
name: "1-h Tools",
|
name: "Tools",
|
||||||
description: "Custom Tools and Media Providers (OpenAI/Gemini) for OpenClaw.",
|
description: "Add Tools tools to OpenClaw.",
|
||||||
register(api) {
|
tools: (tool) => [
|
||||||
// 1. Register tools
|
tool({
|
||||||
api.registerTool({
|
name: "echo",
|
||||||
name: echoTool.name,
|
description: "Echo input text.",
|
||||||
label: echoTool.label,
|
parameters: Type.Object({
|
||||||
description: echoTool.description,
|
input: Type.String({ description: "Text to echo." }),
|
||||||
parameters: echoTool.parameters,
|
|
||||||
execute: async (_toolCallId, params) => {
|
|
||||||
const result = await echoTool.execute(params as any);
|
|
||||||
return jsonResult(result);
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
// 2. Register OpenAI-compatible video generation provider
|
|
||||||
api.registerVideoGenerationProvider(
|
|
||||||
buildOpenAIVideoGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync: api.runtime.modelAuth.isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig: () => api.pluginConfig,
|
|
||||||
}),
|
}),
|
||||||
);
|
execute: async ({ input }) => ({ input }),
|
||||||
|
}),
|
||||||
// 3. Register OpenAI-compatible image generation provider
|
],
|
||||||
api.registerImageGenerationProvider(
|
|
||||||
buildOpenAIImageGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync: api.runtime.modelAuth.isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig: () => api.pluginConfig,
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
|
|
||||||
// 4. Register Gemini-native image generation provider
|
|
||||||
api.registerImageGenerationProvider(
|
|
||||||
buildGeminiImageGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync: api.runtime.modelAuth.isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig: () => api.pluginConfig,
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
|
|
||||||
export default entry;
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
# Tools (`tools`)
|
|
||||||
|
|
||||||
基础工具集合模块。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 包含工具
|
|
||||||
|
|
||||||
### `echo`
|
|
||||||
* **说明**:回显输入的文本内容。
|
|
||||||
* **主要用途**:验证 Agent 工具链的挂载连通性、参数序列化与状态交付。
|
|
||||||
* **参数**:
|
|
||||||
* `input` (string): 需要回显的文本内容。
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
import { Type, type Static } from "typebox";
|
|
||||||
|
|
||||||
export const EchoInputSchema = Type.Object({
|
|
||||||
input: Type.String({ description: "Text to echo." }),
|
|
||||||
});
|
|
||||||
|
|
||||||
export type EchoInput = Static<typeof EchoInputSchema>;
|
|
||||||
|
|
||||||
export const echoTool = {
|
|
||||||
name: "echo",
|
|
||||||
label: "Echo",
|
|
||||||
description: "Echo input text.",
|
|
||||||
parameters: EchoInputSchema,
|
|
||||||
execute: async ({ input }: EchoInput) => ({ input }),
|
|
||||||
};
|
|
||||||
Vendored
-110
@@ -1,110 +0,0 @@
|
|||||||
declare module "openclaw/plugin-sdk/blob-runtime" {
|
|
||||||
export function bufferToBlobPart(buffer: Buffer): BlobPart;
|
|
||||||
}
|
|
||||||
|
|
||||||
declare module "openclaw/plugin-sdk/video-generation" {
|
|
||||||
export interface VideoGenerationProvider {
|
|
||||||
id: string;
|
|
||||||
aliases?: string[];
|
|
||||||
label?: string;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
isConfigured?: (ctx: { cfg?: any }) => boolean;
|
|
||||||
isConfiguredAsync?: (ctx: { cfg?: any; agentDir?: string }) => Promise<boolean>;
|
|
||||||
capabilities: any;
|
|
||||||
generateVideo: (req: VideoGenerationRequest) => Promise<any>;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface VideoGenerationRequest {
|
|
||||||
prompt: string;
|
|
||||||
model?: string;
|
|
||||||
durationSeconds?: number;
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
resolution?: string;
|
|
||||||
inputImages?: Array<{ buffer?: Buffer; mimeType?: string; fileName?: string }>;
|
|
||||||
inputVideos?: Array<{ buffer?: Buffer; mimeType?: string; fileName?: string }>;
|
|
||||||
cfg?: any;
|
|
||||||
agentDir?: string;
|
|
||||||
authStore?: any;
|
|
||||||
timeoutMs?: number;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
declare module "openclaw/plugin-sdk/image-generation" {
|
|
||||||
export interface GeneratedImageAsset {
|
|
||||||
buffer: Buffer;
|
|
||||||
mimeType: string;
|
|
||||||
fileName?: string;
|
|
||||||
revisedPrompt?: string;
|
|
||||||
metadata?: Record<string, unknown>;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface ImageGenerationRequest {
|
|
||||||
prompt: string;
|
|
||||||
model?: string;
|
|
||||||
count?: number;
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
resolution?: string;
|
|
||||||
quality?: string;
|
|
||||||
outputFormat?: string;
|
|
||||||
background?: string;
|
|
||||||
inputImages?: Array<{ buffer: Buffer; mimeType?: string; fileName?: string }>;
|
|
||||||
cfg?: any;
|
|
||||||
agentDir?: string;
|
|
||||||
authStore?: any;
|
|
||||||
timeoutMs?: number;
|
|
||||||
ssrfPolicy?: any;
|
|
||||||
providerOptions?: Record<string, unknown>;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface ImageGenerationResult {
|
|
||||||
images: GeneratedImageAsset[];
|
|
||||||
model?: string;
|
|
||||||
metadata?: Record<string, unknown>;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface ImageGenerationProvider {
|
|
||||||
id: string;
|
|
||||||
aliases?: string[];
|
|
||||||
label?: string;
|
|
||||||
defaultModel?: string;
|
|
||||||
defaultTimeoutMs?: number;
|
|
||||||
models?: string[];
|
|
||||||
capabilities: any;
|
|
||||||
isConfigured?: (ctx: { cfg?: any }) => boolean;
|
|
||||||
isConfiguredAsync?: (ctx: { cfg?: any; agentDir?: string }) => Promise<boolean>;
|
|
||||||
generateImage: (req: ImageGenerationRequest) => Promise<ImageGenerationResult>;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function parseOpenAiCompatibleImageResponse(
|
|
||||||
payload: unknown,
|
|
||||||
options?: { defaultMimeType?: string; malformedResponseError?: string; fileNamePrefix?: string; sniffMimeType?: boolean },
|
|
||||||
): GeneratedImageAsset[];
|
|
||||||
}
|
|
||||||
|
|
||||||
declare module "openclaw/plugin-sdk/provider-http" {
|
|
||||||
export function createProviderOperationDeadline(params: { timeoutMs?: number; label: string }): any;
|
|
||||||
export function createProviderOperationTimeoutResolver(params: { deadline: any; defaultTimeoutMs: number }): () => number;
|
|
||||||
export function executeProviderOperationWithRetry(params: any): Promise<any>;
|
|
||||||
export function fetchProviderDownloadResponse(params: any): Promise<any>;
|
|
||||||
export function fetchWithTimeoutGuarded(url: string, init: any, timeoutMs: number, fetchFn: any, opts: any): Promise<any>;
|
|
||||||
export function assertOkOrThrowHttpError(response: Response, message: string): Promise<void>;
|
|
||||||
export function postMultipartRequest(params: any): Promise<{ response: Response; release: () => Promise<void> }>;
|
|
||||||
export function postJsonRequest(params: any): Promise<{ response: Response; release: () => Promise<void> }>;
|
|
||||||
export function readProviderJsonResponse<T = any>(response: Response, message: string, opts?: any): Promise<T>;
|
|
||||||
export function pollProviderOperationJson<T = any>(params: any): Promise<T>;
|
|
||||||
export function resolveProviderOperationTimeoutMs(params: any): number;
|
|
||||||
export function resolveProviderHttpRequestConfig(params: any): any;
|
|
||||||
export function sanitizeConfiguredModelProviderRequest(params: any): any;
|
|
||||||
}
|
|
||||||
|
|
||||||
declare module "openclaw/plugin-sdk/provider-auth-runtime" {
|
|
||||||
export function resolveApiKeyForProvider(params: any): Promise<{ apiKey?: string; mode?: string }>;
|
|
||||||
}
|
|
||||||
|
|
||||||
declare module "openclaw/plugin-sdk/media-generation-runtime" {
|
|
||||||
export function downloadGeneratedVideoAsset(params: any): Promise<any>;
|
|
||||||
export function resolveGeneratedMediaMaxBytes(cfg: any, type: string): number;
|
|
||||||
}
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
# 1-h OpenAI Video Provider (`1h-openai-video`)
|
|
||||||
|
|
||||||
OpenClaw 视频生成驱动模块,专职适配标准 **OpenAI Videos API**(`/v1/videos` 规范)。
|
|
||||||
|
|
||||||
专为自建聚合网关或第三方中转(如 `gemini-web2api-go`、`ccLoad`)定制:
|
|
||||||
* **任务创建**:`POST /v1/videos`。纯文本任务自动发送 JSON Body,避免第三方服务因不支持 Multipart 表单报 400 错误;带参考图/垫视频时使用标准 Multipart 表单。
|
|
||||||
* **状态轮询**:`GET /v1/videos/{id}`。兼容提取普通对象错误信息与字符串错误消息。
|
|
||||||
* **视频下载**:`GET /v1/videos/{id}/content`。流式下载 MP4 二进制视频流。
|
|
||||||
* **私有网络**:原生支持私有/内网端点请求(`allowPrivateNetwork: true`),兼容 `host.incus.internal` 或局域网反代地址。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 标识符
|
|
||||||
|
|
||||||
* **Provider ID**:`1h-openai-video`
|
|
||||||
* **默认模型**:`gemini-video`
|
|
||||||
* **支持模型**:`gemini-video`, `sora-2`, `sora-2-pro`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 配置说明
|
|
||||||
|
|
||||||
在 `~/.openclaw/openclaw.json` 中配置:
|
|
||||||
|
|
||||||
```json5
|
|
||||||
{
|
|
||||||
"plugins": {
|
|
||||||
"entries": {
|
|
||||||
"1h-tools": {
|
|
||||||
"enabled": true,
|
|
||||||
"config": {
|
|
||||||
"openaiVideo": {
|
|
||||||
"baseUrl": "http://host.incus.internal:8083/v1",
|
|
||||||
"defaultModel": "gemini-video",
|
|
||||||
"apiKey": "sk-your-video-key",
|
|
||||||
"allowPrivateNetwork": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"agents": {
|
|
||||||
"defaults": {
|
|
||||||
"mediaModels": {
|
|
||||||
"video": {
|
|
||||||
"primary": "1h-openai-video/gemini-video"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 使用方法
|
|
||||||
|
|
||||||
配置完成后,OpenClaw 内置的 `video_generate` 工具即可自动调度:
|
|
||||||
* **自然语言直接对话生成**:
|
|
||||||
> “帮我生成一个 5 秒的猫咪在海滩上奔跑的 16:9 视频”
|
|
||||||
* **垫图/图生视频**:
|
|
||||||
在对话中上传或提供图片链接,提示助手基于该图生成视频。
|
|
||||||
@@ -1,509 +0,0 @@
|
|||||||
import { bufferToBlobPart } from "openclaw/plugin-sdk/blob-runtime";
|
|
||||||
import { extensionForMime, type MediaKind } from "openclaw/plugin-sdk/media-mime";
|
|
||||||
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-entry";
|
|
||||||
import type {
|
|
||||||
createProviderOperationDeadline,
|
|
||||||
postMultipartRequest,
|
|
||||||
} from "openclaw/plugin-sdk/provider-http";
|
|
||||||
import { normalizeOptionalString } from "openclaw/plugin-sdk/string-coerce-runtime";
|
|
||||||
import type {
|
|
||||||
VideoGenerationProvider,
|
|
||||||
VideoGenerationRequest,
|
|
||||||
} from "openclaw/plugin-sdk/video-generation";
|
|
||||||
|
|
||||||
const DEFAULT_OPENAI_VIDEO_BASE_URL = "https://api.openai.com/v1";
|
|
||||||
const DEFAULT_OPENAI_VIDEO_MODEL = "sora-2";
|
|
||||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
|
||||||
const POLL_INTERVAL_MS = 2_500;
|
|
||||||
const MAX_POLL_ATTEMPTS = 120;
|
|
||||||
const OPENAI_VIDEO_SECONDS = [4, 8, 12] as const;
|
|
||||||
const OPENAI_VIDEO_SIZES = ["720x1280", "1280x720", "1024x1792", "1792x1024"] as const;
|
|
||||||
|
|
||||||
export type OpenAIVideoRequestPolicy = {
|
|
||||||
allowPrivateNetwork: boolean;
|
|
||||||
dispatcherPolicy?: Parameters<typeof postMultipartRequest>[0]["dispatcherPolicy"];
|
|
||||||
};
|
|
||||||
|
|
||||||
export type OpenAIVideoStatus = "queued" | "in_progress" | "completed" | "failed";
|
|
||||||
|
|
||||||
export type OpenAIReferenceAsset = {
|
|
||||||
kind: Extract<MediaKind, "image" | "video">;
|
|
||||||
file: File;
|
|
||||||
};
|
|
||||||
|
|
||||||
export type OpenAIVideoResponse = {
|
|
||||||
id?: string;
|
|
||||||
model?: string;
|
|
||||||
status?: OpenAIVideoStatus;
|
|
||||||
prompt?: string | null;
|
|
||||||
seconds?: string;
|
|
||||||
size?: string;
|
|
||||||
error?:
|
|
||||||
| string
|
|
||||||
| {
|
|
||||||
code?: string;
|
|
||||||
message?: string;
|
|
||||||
}
|
|
||||||
| null;
|
|
||||||
};
|
|
||||||
|
|
||||||
export interface SecretRefObject {
|
|
||||||
source: string;
|
|
||||||
provider: string;
|
|
||||||
id: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
export interface VideoPluginConfig {
|
|
||||||
baseUrl?: string;
|
|
||||||
apiKey?: string | SecretRefObject;
|
|
||||||
allowPrivateNetwork?: boolean;
|
|
||||||
defaultModel?: string;
|
|
||||||
models?: string[];
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolvePluginApiKey(rawKey: string | SecretRefObject | undefined): string | undefined {
|
|
||||||
if (typeof rawKey === "string") {
|
|
||||||
const trimmed = rawKey.trim();
|
|
||||||
const envMatch = trimmed.match(/^\$\{([A-Za-z0-9_]+)\}$/);
|
|
||||||
if (envMatch) {
|
|
||||||
return normalizeOptionalString(process.env[envMatch[1]]);
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(trimmed);
|
|
||||||
}
|
|
||||||
if (rawKey && typeof rawKey === "object" && typeof rawKey.id === "string") {
|
|
||||||
return normalizeOptionalString(process.env[rawKey.id]);
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveVideoConfig(
|
|
||||||
cfg: VideoGenerationRequest["cfg"] | undefined,
|
|
||||||
apiPluginConfig?: Record<string, unknown>,
|
|
||||||
): VideoPluginConfig | undefined {
|
|
||||||
const toolsConfig = (apiPluginConfig ?? cfg?.plugins?.entries?.["1h-tools"]?.config) as
|
|
||||||
| {
|
|
||||||
openaiVideo?: VideoPluginConfig;
|
|
||||||
video?: VideoPluginConfig;
|
|
||||||
}
|
|
||||||
| undefined;
|
|
||||||
|
|
||||||
return toolsConfig?.openaiVideo ?? toolsConfig?.video;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveConfiguredOpenAIBaseUrl(
|
|
||||||
cfg: VideoGenerationRequest["cfg"],
|
|
||||||
pluginCfg?: VideoPluginConfig,
|
|
||||||
): string {
|
|
||||||
return (
|
|
||||||
normalizeOptionalString(pluginCfg?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_BASE_URL) ??
|
|
||||||
normalizeOptionalString(cfg?.models?.providers?.openai?.baseUrl) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_BASE_URL) ??
|
|
||||||
DEFAULT_OPENAI_VIDEO_BASE_URL
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function readOpenAIVideoFailureMessage(payload: OpenAIVideoResponse): string | undefined {
|
|
||||||
if (payload.status !== "failed") {
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
if (typeof payload.error === "string") {
|
|
||||||
return normalizeOptionalString(payload.error) ?? "OpenAI video generation failed";
|
|
||||||
}
|
|
||||||
return normalizeOptionalString(payload.error?.message) ?? "OpenAI video generation failed";
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveDurationSeconds(durationSeconds: number | undefined): "4" | "8" | "12" | undefined {
|
|
||||||
if (typeof durationSeconds !== "number" || !Number.isFinite(durationSeconds)) {
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
const rounded = Math.max(OPENAI_VIDEO_SECONDS[0], Math.round(durationSeconds));
|
|
||||||
const nearest = OPENAI_VIDEO_SECONDS.reduce((best, current) =>
|
|
||||||
Math.abs(current - rounded) < Math.abs(best - rounded) ? current : best,
|
|
||||||
);
|
|
||||||
return String(nearest) as "4" | "8" | "12";
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveSize(params: {
|
|
||||||
size?: string;
|
|
||||||
aspectRatio?: string;
|
|
||||||
resolution?: string;
|
|
||||||
}): (typeof OPENAI_VIDEO_SIZES)[number] | undefined {
|
|
||||||
const explicitSize = normalizeOptionalString(params.size);
|
|
||||||
const supportedSize = OPENAI_VIDEO_SIZES.find((size) => size === explicitSize);
|
|
||||||
if (supportedSize) {
|
|
||||||
return supportedSize;
|
|
||||||
}
|
|
||||||
switch (normalizeOptionalString(params.aspectRatio)) {
|
|
||||||
case "9:16":
|
|
||||||
return "720x1280";
|
|
||||||
case "16:9":
|
|
||||||
return "1280x720";
|
|
||||||
case "4:7":
|
|
||||||
return "1024x1792";
|
|
||||||
case "7:4":
|
|
||||||
return "1792x1024";
|
|
||||||
default:
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
if (params.resolution === "1080P") {
|
|
||||||
return "1792x1024";
|
|
||||||
}
|
|
||||||
return undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
export function resolveReferenceAsset(req: VideoGenerationRequest): OpenAIReferenceAsset | null {
|
|
||||||
const allAssets = [...(req.inputImages ?? []), ...(req.inputVideos ?? [])];
|
|
||||||
if (allAssets.length === 0) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
if (allAssets.length > 1) {
|
|
||||||
throw new Error("OpenAI video generation supports at most one reference image or video.");
|
|
||||||
}
|
|
||||||
const [asset] = allAssets;
|
|
||||||
if (!asset?.buffer) {
|
|
||||||
throw new Error(
|
|
||||||
"OpenAI video generation currently requires local image/video uploads for reference assets.",
|
|
||||||
);
|
|
||||||
}
|
|
||||||
const kind = (req.inputVideos?.length ?? 0) > 0 ? "video" : "image";
|
|
||||||
const mimeType =
|
|
||||||
normalizeOptionalString(asset.mimeType) || (kind === "video" ? "video/mp4" : "image/png");
|
|
||||||
const extension =
|
|
||||||
extensionForMime(mimeType)?.slice(1) ?? (mimeType.startsWith("video/") ? "mp4" : "png");
|
|
||||||
const fileName = normalizeOptionalString(asset.fileName) || `reference-${kind}.${extension}`;
|
|
||||||
return {
|
|
||||||
kind,
|
|
||||||
file: new File([bufferToBlobPart(asset.buffer)], fileName, { type: mimeType }),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function fetchOpenAIVideoDownload(
|
|
||||||
params: {
|
|
||||||
url: string;
|
|
||||||
init: RequestInit;
|
|
||||||
deadline: ReturnType<typeof createProviderOperationDeadline>;
|
|
||||||
fetchFn: typeof fetch;
|
|
||||||
} & OpenAIVideoRequestPolicy,
|
|
||||||
) {
|
|
||||||
const {
|
|
||||||
assertOkOrThrowHttpError,
|
|
||||||
createProviderOperationTimeoutResolver,
|
|
||||||
executeProviderOperationWithRetry,
|
|
||||||
fetchProviderDownloadResponse,
|
|
||||||
fetchWithTimeoutGuarded,
|
|
||||||
} = await import("openclaw/plugin-sdk/provider-http");
|
|
||||||
const timeoutMs = createProviderOperationTimeoutResolver({
|
|
||||||
deadline: params.deadline,
|
|
||||||
defaultTimeoutMs: params.deadline.timeoutMs ?? DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
if (!params.allowPrivateNetwork && !params.dispatcherPolicy) {
|
|
||||||
const response = await fetchProviderDownloadResponse({
|
|
||||||
url: params.url,
|
|
||||||
init: params.init,
|
|
||||||
deadline: params.deadline,
|
|
||||||
fetchFn: params.fetchFn,
|
|
||||||
provider: "openai",
|
|
||||||
requestFailedMessage: "OpenAI video download failed",
|
|
||||||
});
|
|
||||||
return {
|
|
||||||
response,
|
|
||||||
release: async () => {},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
return await executeProviderOperationWithRetry({
|
|
||||||
provider: "openai",
|
|
||||||
stage: "download",
|
|
||||||
operation: async () => {
|
|
||||||
const result = await fetchWithTimeoutGuarded(
|
|
||||||
params.url,
|
|
||||||
params.init,
|
|
||||||
timeoutMs(),
|
|
||||||
params.fetchFn,
|
|
||||||
{
|
|
||||||
...(params.allowPrivateNetwork ? { ssrfPolicy: { allowPrivateNetwork: true } } : {}),
|
|
||||||
...(params.dispatcherPolicy ? { dispatcherPolicy: params.dispatcherPolicy } : {}),
|
|
||||||
auditContext: "openai-video-download",
|
|
||||||
},
|
|
||||||
);
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(result.response, "OpenAI video download failed");
|
|
||||||
return result;
|
|
||||||
} catch (error) {
|
|
||||||
await result.release();
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
},
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
export function buildOpenAIVideoGenerationProvider({
|
|
||||||
isProviderApiKeyConfiguredAsync,
|
|
||||||
getPluginConfig,
|
|
||||||
}: {
|
|
||||||
isProviderApiKeyConfiguredAsync: OpenClawPluginApi["runtime"]["modelAuth"]["isProviderApiKeyConfiguredAsync"];
|
|
||||||
getPluginConfig?: () => Record<string, unknown> | undefined;
|
|
||||||
}): VideoGenerationProvider {
|
|
||||||
return {
|
|
||||||
id: "1h-openai-video",
|
|
||||||
label: "1-h OpenAI Video",
|
|
||||||
defaultModel: DEFAULT_OPENAI_VIDEO_MODEL,
|
|
||||||
models: [DEFAULT_OPENAI_VIDEO_MODEL, "sora-2-pro", "gemini-video"],
|
|
||||||
isConfiguredAsync: async (ctx) => {
|
|
||||||
const pluginCfg = resolveVideoConfig(ctx.cfg, getPluginConfig?.());
|
|
||||||
if (
|
|
||||||
resolvePluginApiKey(pluginCfg?.apiKey) ||
|
|
||||||
Boolean(pluginCfg?.apiKey) ||
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.GEMINI_WEB2API_API_KEY) ||
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY)
|
|
||||||
) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
return await isProviderApiKeyConfiguredAsync({
|
|
||||||
provider: "openai",
|
|
||||||
...ctx,
|
|
||||||
profileTypes: ["api_key"],
|
|
||||||
});
|
|
||||||
},
|
|
||||||
capabilities: {
|
|
||||||
generate: {
|
|
||||||
maxVideos: 1,
|
|
||||||
maxDurationSeconds: 12,
|
|
||||||
supportedDurationSeconds: OPENAI_VIDEO_SECONDS,
|
|
||||||
supportsSize: true,
|
|
||||||
sizes: OPENAI_VIDEO_SIZES,
|
|
||||||
},
|
|
||||||
imageToVideo: {
|
|
||||||
enabled: true,
|
|
||||||
maxVideos: 1,
|
|
||||||
maxInputImages: 1,
|
|
||||||
maxDurationSeconds: 12,
|
|
||||||
supportedDurationSeconds: OPENAI_VIDEO_SECONDS,
|
|
||||||
supportsSize: true,
|
|
||||||
sizes: OPENAI_VIDEO_SIZES,
|
|
||||||
},
|
|
||||||
videoToVideo: {
|
|
||||||
enabled: true,
|
|
||||||
maxVideos: 1,
|
|
||||||
maxInputVideos: 1,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
async generateVideo(req) {
|
|
||||||
const pluginCfg = resolveVideoConfig(req.cfg, getPluginConfig?.());
|
|
||||||
const explicitApiKey =
|
|
||||||
resolvePluginApiKey(pluginCfg?.apiKey) ??
|
|
||||||
normalizeOptionalString(process.env.OPENAI_VIDEO_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.GEMINI_WEB2API_API_KEY) ??
|
|
||||||
normalizeOptionalString(process.env.CCLOAD_API_KEY);
|
|
||||||
|
|
||||||
let apiKey = explicitApiKey;
|
|
||||||
if (!apiKey) {
|
|
||||||
const { resolveApiKeyForProvider } =
|
|
||||||
await import("openclaw/plugin-sdk/provider-auth-runtime");
|
|
||||||
const auth = await resolveApiKeyForProvider({
|
|
||||||
provider: "openai",
|
|
||||||
cfg: req.cfg,
|
|
||||||
agentDir: req.agentDir,
|
|
||||||
store: req.authStore,
|
|
||||||
modelApi: "openai-responses",
|
|
||||||
});
|
|
||||||
if (!auth.apiKey || (auth.mode !== undefined && auth.mode !== "api-key")) {
|
|
||||||
throw new Error("OpenAI API key missing");
|
|
||||||
}
|
|
||||||
apiKey = auth.apiKey;
|
|
||||||
}
|
|
||||||
|
|
||||||
const [
|
|
||||||
{
|
|
||||||
assertOkOrThrowHttpError,
|
|
||||||
createProviderOperationDeadline,
|
|
||||||
createProviderOperationTimeoutResolver,
|
|
||||||
pollProviderOperationJson,
|
|
||||||
postJsonRequest,
|
|
||||||
postMultipartRequest,
|
|
||||||
readProviderJsonResponse,
|
|
||||||
resolveProviderOperationTimeoutMs,
|
|
||||||
resolveProviderHttpRequestConfig,
|
|
||||||
sanitizeConfiguredModelProviderRequest,
|
|
||||||
},
|
|
||||||
{ downloadGeneratedVideoAsset, resolveGeneratedMediaMaxBytes },
|
|
||||||
] = await Promise.all([
|
|
||||||
import("openclaw/plugin-sdk/provider-http"),
|
|
||||||
import("openclaw/plugin-sdk/media-generation-runtime"),
|
|
||||||
]);
|
|
||||||
const fetchFn = fetch;
|
|
||||||
const deadline = createProviderOperationDeadline({
|
|
||||||
timeoutMs: req.timeoutMs,
|
|
||||||
label: "OpenAI video generation",
|
|
||||||
});
|
|
||||||
const providerConfig = req.cfg.models?.providers?.openai;
|
|
||||||
const allowPrivate = pluginCfg?.allowPrivateNetwork ?? true;
|
|
||||||
const requestConfig = sanitizeConfiguredModelProviderRequest({
|
|
||||||
...providerConfig?.request,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
});
|
|
||||||
const { baseUrl, allowPrivateNetwork, headers, dispatcherPolicy } =
|
|
||||||
resolveProviderHttpRequestConfig({
|
|
||||||
baseUrl: resolveConfiguredOpenAIBaseUrl(req.cfg, pluginCfg),
|
|
||||||
defaultBaseUrl: DEFAULT_OPENAI_VIDEO_BASE_URL,
|
|
||||||
allowPrivateNetwork: allowPrivate,
|
|
||||||
request: requestConfig,
|
|
||||||
defaultHeaders: {
|
|
||||||
Authorization: `Bearer ${apiKey}`,
|
|
||||||
},
|
|
||||||
provider: "openai",
|
|
||||||
capability: "video",
|
|
||||||
transport: "http",
|
|
||||||
});
|
|
||||||
|
|
||||||
const model =
|
|
||||||
normalizeOptionalString(req.model) ??
|
|
||||||
normalizeOptionalString(pluginCfg?.defaultModel) ??
|
|
||||||
DEFAULT_OPENAI_VIDEO_MODEL;
|
|
||||||
const seconds = resolveDurationSeconds(req.durationSeconds);
|
|
||||||
const size = resolveSize(req);
|
|
||||||
const referenceAsset = resolveReferenceAsset(req);
|
|
||||||
const isVideoEdit = referenceAsset?.kind === "video";
|
|
||||||
const submitTimeoutMs = resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
});
|
|
||||||
|
|
||||||
let response: Response;
|
|
||||||
let release: () => Promise<void>;
|
|
||||||
|
|
||||||
if (referenceAsset) {
|
|
||||||
const form = new FormData();
|
|
||||||
form.set("prompt", req.prompt);
|
|
||||||
if (isVideoEdit) {
|
|
||||||
form.set("video", referenceAsset.file);
|
|
||||||
} else {
|
|
||||||
form.set("model", model);
|
|
||||||
if (seconds) {
|
|
||||||
form.set("seconds", seconds);
|
|
||||||
}
|
|
||||||
if (size) {
|
|
||||||
form.set("size", size);
|
|
||||||
}
|
|
||||||
form.set("input_reference", referenceAsset.file);
|
|
||||||
}
|
|
||||||
const multipartHeaders = new Headers(headers);
|
|
||||||
multipartHeaders.delete("Content-Type");
|
|
||||||
({ response, release } = await postMultipartRequest({
|
|
||||||
url: `${baseUrl}/videos${isVideoEdit ? "/edits" : ""}`,
|
|
||||||
headers: multipartHeaders,
|
|
||||||
body: form,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
} else {
|
|
||||||
const jsonHeaders = new Headers(headers);
|
|
||||||
jsonHeaders.set("Content-Type", "application/json");
|
|
||||||
const jsonBody: Record<string, unknown> = {
|
|
||||||
model,
|
|
||||||
prompt: req.prompt,
|
|
||||||
...(seconds ? { seconds } : {}),
|
|
||||||
...(size ? { size } : {}),
|
|
||||||
};
|
|
||||||
({ response, release } = await postJsonRequest({
|
|
||||||
url: `${baseUrl}/videos`,
|
|
||||||
headers: jsonHeaders,
|
|
||||||
body: jsonBody,
|
|
||||||
timeoutMs: submitTimeoutMs,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
|
|
||||||
let submitted: OpenAIVideoResponse;
|
|
||||||
try {
|
|
||||||
await assertOkOrThrowHttpError(response, "OpenAI video generation failed");
|
|
||||||
submitted = await readProviderJsonResponse<OpenAIVideoResponse>(
|
|
||||||
response,
|
|
||||||
"OpenAI video generation failed",
|
|
||||||
);
|
|
||||||
} finally {
|
|
||||||
await release();
|
|
||||||
}
|
|
||||||
const failureMessage = readOpenAIVideoFailureMessage(submitted);
|
|
||||||
if (failureMessage) {
|
|
||||||
throw new Error(failureMessage);
|
|
||||||
}
|
|
||||||
const videoId = normalizeOptionalString(submitted.id);
|
|
||||||
if (!videoId) {
|
|
||||||
throw new Error("OpenAI video generation response missing video id");
|
|
||||||
}
|
|
||||||
const completed =
|
|
||||||
submitted.status === "completed"
|
|
||||||
? submitted
|
|
||||||
: await pollProviderOperationJson<OpenAIVideoResponse>({
|
|
||||||
url: `${baseUrl}/videos/${videoId}`,
|
|
||||||
headers,
|
|
||||||
deadline: createProviderOperationDeadline({
|
|
||||||
timeoutMs: resolveProviderOperationTimeoutMs({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
}),
|
|
||||||
label: `OpenAI video generation task ${videoId}`,
|
|
||||||
}),
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
fetchFn,
|
|
||||||
maxAttempts: MAX_POLL_ATTEMPTS,
|
|
||||||
pollIntervalMs: POLL_INTERVAL_MS,
|
|
||||||
requestFailedMessage: "OpenAI video status request failed",
|
|
||||||
timeoutMessage: `OpenAI video generation task ${videoId} did not finish in time`,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
auditContext: "openai-video-status",
|
|
||||||
isComplete: (payload: OpenAIVideoResponse) => payload.status === "completed",
|
|
||||||
getFailureMessage: readOpenAIVideoFailureMessage,
|
|
||||||
});
|
|
||||||
const url = new URL(`${baseUrl}/videos/${videoId}/content`);
|
|
||||||
url.searchParams.set("variant", "video");
|
|
||||||
const video = await downloadGeneratedVideoAsset({
|
|
||||||
url: url.toString(),
|
|
||||||
timeoutMs: createProviderOperationTimeoutResolver({
|
|
||||||
deadline,
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
}),
|
|
||||||
defaultTimeoutMs: DEFAULT_TIMEOUT_MS,
|
|
||||||
fetchFn,
|
|
||||||
provider: "openai",
|
|
||||||
label: "OpenAI generated video download",
|
|
||||||
requestFailedMessage: "OpenAI video download failed",
|
|
||||||
maxBytes: resolveGeneratedMediaMaxBytes(req.cfg, "video"),
|
|
||||||
validateBinaryResponse: true,
|
|
||||||
fetchResponse: async ({ deadline: downloadDeadline }: { deadline: any }) =>
|
|
||||||
await fetchOpenAIVideoDownload({
|
|
||||||
url: url.toString(),
|
|
||||||
init: {
|
|
||||||
method: "GET",
|
|
||||||
headers: new Headers({
|
|
||||||
...Object.fromEntries(headers.entries()),
|
|
||||||
Accept: "application/binary",
|
|
||||||
}),
|
|
||||||
},
|
|
||||||
deadline: downloadDeadline,
|
|
||||||
fetchFn,
|
|
||||||
allowPrivateNetwork,
|
|
||||||
dispatcherPolicy,
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
return {
|
|
||||||
videos: [video],
|
|
||||||
model: completed.model ?? submitted.model ?? model,
|
|
||||||
metadata: {
|
|
||||||
videoId,
|
|
||||||
status: completed.status,
|
|
||||||
seconds: completed.seconds ?? submitted.seconds,
|
|
||||||
size: completed.size ?? submitted.size,
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
+1
-4
@@ -10,9 +10,6 @@
|
|||||||
"skipLibCheck": true
|
"skipLibCheck": true
|
||||||
},
|
},
|
||||||
"include": [
|
"include": [
|
||||||
"src/**/*"
|
"src/index.ts"
|
||||||
],
|
|
||||||
"exclude": [
|
|
||||||
"src/**/*.test.ts"
|
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
Reference in new issue
Block a user