Avatar Runtime Skill
Security & Trust
This skill contains no bundled server code — it instructs the agent to download and execute the avatar-runtime npm package via npx at runtime. Before running:
- 1. Verify the source — Review the avatar-runtime repository and confirm you are installing the official package from the npm registry.
- Review setup scripts before running —
scripts/ensure-default-vrm-sample.sh and scripts/ensure-default-live2d-sample.sh download third-party assets at runtime. Inspect their contents before executing them in your environment. - Sandbox first — Run
npx avatar-runtime in an isolated environment and monitor outbound network connections before deploying to production. - Live2D licensing — The sample Live2D model (
live2d-widget-model-chitose) is subject to the Live2D Free Material License, which prohibits redistribution and commercial use. Use it for local development only. - API key handling —
HEYGEN_API_KEY and KUSAPICS_API_KEY are passed as environment variables directly to npx avatar-runtime. Ensure these are not logged or exposed in shared environments.
Runtime endpoint
Default: http://127.0.0.1:3721
Override: AVATAR_RUNTIME_URL env var (default applied automatically if unset).
CODEBLOCK0
First-time setup
VRM 3D avatar (free, no account required)
CODEBLOCK1
Downloads VRM1_Constraint_Twist_Sample.vrm from @pixiv/three-vrm (CC BY 4.0 — free to use with attribution).
Sets it as assets/vrm/slot/default.vrm so npm run dev:vrm-bridge serves it automatically.
Live2D avatar
The Live2D slot (assets/live2d/slot/) requires a model file before the live2d provider can render.
Option A — Use your own model (recommended):
Copy any Cubism 2 (.model.json) or Cubism 4 (.model3.json) model you hold a license for into assets/live2d/slot/, named default.model.json / default.model3.json.
Option B — Local dev bootstrap only (risk acknowledged):
CODEBLOCK2
Downloads chitose from the npm package live2d-widget-model-chitose@1.0.5 for local testing.
⚠️ The original model is subject to the Live2D Free Material License, which prohibits redistribution and commercial use. Do not deploy or distribute with this model.
Starting the server
CODEBLOCK3
Session API
CODEBLOCK4
Avatar control API (v0.2)
The runtime uses a unified control namespace replacing the legacy faceControl field.
CODEBLOCK5
Partial patches: Only supplied sub-objects are merged. Each sub-domain (avatar.face, avatar.body, avatar.emotion, scene) merges independently — patching mouth.smile does not clobber eyes.
Embedding an avatar widget (browser)
Minimal script-tag usage — vendor scripts are auto-loaded:
CODEBLOCK6
VRM 3D avatar:
CODEBLOCK7
Without a model (vector fallback — no files needed):
CODEBLOCK8
Driving avatar control manually (widget)
INLINECODE32 accepts a mediaState with a control object.
Safe to call before ready() resolves — buffered and applied on mount.
CODEBLOCK9
control from runtime state
The /v1/status response includes a control field produced by the active provider merged with agent-set values.
AvatarWidget with stateUrl polls this automatically at pollMs interval.
For manual polling:
CODEBLOCK10
Provider configuration
| Provider | Env vars | Notes |
|---|
| INLINECODE40 | — | Development default, no key needed |
| INLINECODE41 |
HEYGEN_API_KEY | Real streaming avatar.
HEYGEN_STRICT=false degrades to mock |
|
live2d |
LIVE2D_ENDPOINT | Local Cubism bridge required.
LIVE2D_STRICT=false degrades |
|
vrm |
VRM_BRIDGE_ENDPOINT | Local 3D avatar, client-side rendering. Free models from
VRoid Hub. No API key. |
|
kusapics |
KUSAPICS_API_KEY,
KUSAPICS_BASE_URL | Anime-oriented image provider |
Provider capabilities
| Provider | faceRig | lipSync | gaze | blink | bodyMotion | streaming | bodyRig | sceneControl |
|---|
| INLINECODE52 | ✓ | — | ✓ | ✓ | — | — | — | — |
| INLINECODE53 |
✓ | ✓ | — | — | ✓ | ✓ | — | — |
|
live2d | ✓ | ✓ | ✓ | ✓ | — | — | — | — |
|
vrm | ✓ | — | ✓ | ✓ | — | — | ✓ | ✓ |
|
kusapics | — | — | — | — | — | — | — | — |
INLINECODE57 and sceneControl are VRM-only features — use /v1/control/avatar/set with a body field and /v1/control/scene/set respectively.
Quick-start: VRM 3D avatar (no API key)
CODEBLOCK11
Test face + body control:
CODEBLOCK12
Embed the VRM viewer in a page:
CODEBLOCK13
Quick-start: HeyGen streaming avatar
Prerequisites: HeyGen API key + an avatar ID from your HeyGen account.
CODEBLOCK14
Start a session and stream a talking clip:
CODEBLOCK15
Graceful degradation: when HEYGEN_STRICT is unset (default false), the runtime automatically falls back to mock if the API key is missing or the request fails — useful for local development without a key.
Fallback policy
If the runtime is unavailable or returns an error:
- - Continue interaction in text mode
- Inform the user avatar mode is currently unavailable
- Do not claim rendering or voice playback succeeded
Additional reference
- - WEB-EMBEDDING.md — Renderer Registry, custom renderer implementation, npm usage
Avatar Runtime 技能
安全与信任
本技能不包含任何捆绑的服务器代码——它指示代理在运行时通过 npx 下载并执行 avatar-runtime npm 包。运行前：
- 1. 验证来源 — 查看 avatar-runtime 仓库，确认您正在安装来自 npm 注册表的官方包。
- 运行前审查设置脚本 — scripts/ensure-default-vrm-sample.sh 和 scripts/ensure-default-live2d-sample.sh 会在运行时下载第三方资源。在您的环境中执行前，请检查其内容。
- 先沙箱运行 — 在隔离环境中运行 npx avatar-runtime，并在部署到生产环境前监控出站网络连接。
- Live2D 许可 — 示例 Live2D 模型（live2d-widget-model-chitose）受 Live2D 免费素材许可 约束，禁止再分发和商业使用。仅限本地开发使用。
- API 密钥处理 — HEYGENAPIKEY 和 KUSAPICSAPIKEY 作为环境变量直接传递给 npx avatar-runtime。确保这些密钥不会在共享环境中被记录或暴露。
运行时端点
默认：http://127.0.0.1:3721
覆盖：AVATARRUNTIMEURL 环境变量（未设置时自动应用默认值）。
bash
export AVATARRUNTIMEURL=${AVATARRUNTIMEURL:-http://127.0.0.1:3721}
首次设置
VRM 3D 头像（免费，无需账户）
bash
从包根目录运行
bash scripts/ensure-default-vrm-sample.sh
从 @pixiv/three-vrm 下载 VRM1ConstraintTwist_Sample.vrm（CC BY 4.0 — 可免费使用，需注明出处）。
将其设置为 assets/vrm/slot/default.vrm，以便 npm run dev:vrm-bridge 自动提供该文件。
Live2D 头像
Live2D 插槽（assets/live2d/slot/）需要模型文件后，live2d 提供者才能进行渲染。
选项 A — 使用您自己的模型（推荐）：
将您持有许可的任何 Cubism 2（.model.json）或 Cubism 4（.model3.json）模型复制到 assets/live2d/slot/ 中，命名为 default.model.json / default.model3.json。
选项 B — 仅限本地开发引导（风险已知）：
bash
从包根目录运行（package.json 所在位置）
bash scripts/ensure-default-live2d-sample.sh
从 npm 包 live2d-widget-model-chitose@1.0.5 下载 chitose 用于本地测试。
⚠️ 原始模型受 Live2D 免费素材许可 约束，禁止再分发和商业使用。请不要使用此模型进行部署或分发。
启动服务器
bash
零配置（模拟提供者 — 无需 API 密钥）
AVATAR_PROVIDER=mock npx avatar-runtime
使用 Live2D 本地桥接
npm run dev:live2d-cubism-bridge # 终端 A — 桥接在 :3755
AVATAR
PROVIDER=live2d LIVE2DENDPOINT=http://127.0.0.1:3755 npx avatar-runtime # 终端 B
使用 VRM 3D 头像（免费模型来自 https://hub.vroid.com — 将 .vrm 文件放入 assets/vrm/slot/）
npm run dev:vrm-bridge # 终端 A — 资源服务器在 :3756
AVATAR_PROVIDER=vrm npx avatar-runtime # 终端 B
会话 API
bash
启动会话
curl -s -X POST $AVATAR
RUNTIMEURL/v1/session/start \
-H content-type: application/json \
-d {personaId:{{slug}},form:image}
向活动会话发送文本
curl -s -X POST $AVATAR
RUNTIMEURL/v1/input/text \
-H content-type: application/json \
-d {sessionId:
,text:hello}
查询当前状态（包含渲染器的控制命名空间）
curl -s $AVATARRUNTIMEURL/v1/status
头像控制 API（v0.2）
运行时使用统一的 control 命名空间，取代了旧的 faceControl 字段。
bash
设置面部表情
curl -s -X POST $AVATARRUNTIMEURL/v1/control/avatar/set \
-H content-type: application/json \
-d {
face: {
pose: { yaw: 0.2 },
mouth: { smile: 0.7 }
},
emotion: { valence: 0.8, arousal: 0.3, label: happy }
}
设置身体姿势（仅限 VRM）
curl -s -X POST $AVATARRUNTIMEURL/v1/control/avatar/set \
-H content-type: application/json \
-d {
body: {
preset: wave,
skeleton: { rightUpperArm: { x: 0, y: 0, z: 60 } }
}
}
设置场景（仅限 VRM）
curl -s -X POST $AVATARRUNTIMEURL/v1/control/scene/set \
-H content-type: application/json \
-d {
camera: { fov: 40, position: { x: 0, y: 1.4, z: 2.5 } },
world: { ambientLight: 0.5, keyLight: { intensity: 1.2 } }
}
单次调用完整控制补丁
curl -s -X POST $AVATARRUNTIMEURL/v1/control/set \
-H content-type: application/json \
-d {
avatar: {
face: { mouth: { smile: 0.5 } },
emotion: { label: neutral }
},
scene: { world: { background: #001133 } }
}
部分补丁： 仅合并提供的子对象。每个子域（avatar.face、avatar.body、avatar.emotion、scene）独立合并——修补 mouth.smile 不会覆盖 eyes。
嵌入头像小部件（浏览器）
最小化脚本标签用法——供应商脚本自动加载：
html
VRM 3D 头像：
html
无模型（矢量回退——无需文件）：
html
##
