a0c1dc6ad5
All checks were successful
Gitea Actions Demo / Explore-Gitea-Actions (push) Successful in 27s
173 lines
6.7 KiB
Markdown
173 lines
6.7 KiB
Markdown
# Doubao Frontend File ID Upload Implementation Plan
|
|
|
|
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
|
|
**Goal:** Allow the frontend to upload a video to Ark Files API, receive a `file_id`, and send that `file_id` to this app's backend so Doubao subtitle generation uses `Responses API` file references instead of inline base64 video payloads.
|
|
|
|
**Architecture:** Add a frontend Ark upload helper plus a backward-compatible extension to the subtitle request contract. The backend will accept either an uploaded `video` file or a `fileId`, and the Doubao path will prefer `file_id` while Gemini stays on the current multipart upload flow.
|
|
|
|
**Tech Stack:** React, TypeScript, Express, Vitest, Fetch API, Volcengine Ark Files API, Volcengine Ark Responses API
|
|
|
|
---
|
|
|
|
### Task 1: Add failing frontend tests for the new Doubao request flow
|
|
|
|
**Files:**
|
|
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\services\subtitleService.test.ts`
|
|
|
|
**Step 1: Write the failing tests**
|
|
|
|
Add tests that verify:
|
|
- Doubao first uploads the file to Ark Files API and then posts `fileId` to `/api/generate-subtitles`
|
|
- Gemini still uploads multipart form data with `video`
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `npm.cmd run test -- src/services/subtitleService.test.ts`
|
|
Expected: FAIL because the service does not yet upload to Ark or send `fileId`
|
|
|
|
**Step 3: Write minimal implementation**
|
|
|
|
Update the frontend subtitle service to support an Ark upload step and dual request modes.
|
|
|
|
**Step 4: Run test to verify it passes**
|
|
|
|
Run: `npm.cmd run test -- src/services/subtitleService.test.ts`
|
|
Expected: PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/services/subtitleService.test.ts src/services/subtitleService.ts
|
|
git commit -m "feat: add frontend doubao file id upload flow"
|
|
```
|
|
|
|
### Task 2: Add failing backend tests for `fileId` parsing and Doubao request shape
|
|
|
|
**Files:**
|
|
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleRequest.test.ts`
|
|
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\videoSubtitleGeneration.test.ts`
|
|
|
|
**Step 1: Write the failing tests**
|
|
|
|
Add tests that verify:
|
|
- Subtitle request parsing accepts `fileId`
|
|
- Doubao `Responses API` request uses `file_id` and `fps`
|
|
- Gemini rejects requests that provide only `fileId`
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `npm.cmd run test -- src/server/subtitleRequest.test.ts src/server/videoSubtitleGeneration.test.ts`
|
|
Expected: FAIL because parsing and generation do not yet support `fileId`
|
|
|
|
**Step 3: Write minimal implementation**
|
|
|
|
Extend server request parsing and video generation to accept `fileId`.
|
|
|
|
**Step 4: Run test to verify it passes**
|
|
|
|
Run: `npm.cmd run test -- src/server/subtitleRequest.test.ts src/server/videoSubtitleGeneration.test.ts`
|
|
Expected: PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/server/subtitleRequest.test.ts src/server/videoSubtitleGeneration.test.ts src/server/subtitleRequest.ts src/server/videoSubtitleGeneration.ts
|
|
git commit -m "feat: support doubao file id subtitle requests"
|
|
```
|
|
|
|
### Task 3: Wire the backend route to accept `fileId` without a multipart video upload
|
|
|
|
**Files:**
|
|
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\server.ts`
|
|
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleGeneration.ts`
|
|
|
|
**Step 1: Write the failing test**
|
|
|
|
If route-level coverage is already represented through unit seams, add or update a pipeline test that proves `fileId` can flow into subtitle generation without `videoPath`.
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `npm.cmd run test -- src/server/subtitleGeneration.test.ts`
|
|
Expected: FAIL because the pipeline still assumes a local video path
|
|
|
|
**Step 3: Write minimal implementation**
|
|
|
|
Allow subtitle generation to receive either:
|
|
- `videoPath`
|
|
- `fileId`
|
|
|
|
Require at least one, and keep backend cleanup safe when no uploaded file exists.
|
|
|
|
**Step 4: Run test to verify it passes**
|
|
|
|
Run: `npm.cmd run test -- src/server/subtitleGeneration.test.ts`
|
|
Expected: PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add server.ts src/server/subtitleGeneration.ts src/server/subtitleGeneration.test.ts
|
|
git commit -m "feat: accept file id subtitle generation requests"
|
|
```
|
|
|
|
### Task 4: Verify editor behavior still works with provider switching
|
|
|
|
**Files:**
|
|
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.test.tsx`
|
|
|
|
**Step 1: Write the failing test**
|
|
|
|
Add or update coverage so the editor still calls subtitle generation correctly after the service signature change.
|
|
|
|
**Step 2: Run test to verify it fails**
|
|
|
|
Run: `npm.cmd run test -- src/components/EditorScreen.test.tsx`
|
|
Expected: FAIL because mocks or call signatures need updating
|
|
|
|
**Step 3: Write minimal implementation**
|
|
|
|
Adjust the editor or tests so the new service contract is reflected without changing the visible UX.
|
|
|
|
**Step 4: Run test to verify it passes**
|
|
|
|
Run: `npm.cmd run test -- src/components/EditorScreen.test.tsx`
|
|
Expected: PASS
|
|
|
|
**Step 5: Commit**
|
|
|
|
```bash
|
|
git add src/components/EditorScreen.test.tsx src/components/EditorScreen.tsx
|
|
git commit -m "test: align editor subtitle generation with file id flow"
|
|
```
|
|
|
|
### Task 5: Run focused regression coverage
|
|
|
|
**Files:**
|
|
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\services\subtitleService.test.ts`
|
|
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleRequest.test.ts`
|
|
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleGeneration.test.ts`
|
|
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\server\videoSubtitleGeneration.test.ts`
|
|
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.test.tsx`
|
|
|
|
**Step 1: Run the focused suite**
|
|
|
|
Run: `npm.cmd run test -- src/services/subtitleService.test.ts src/server/subtitleRequest.test.ts src/server/subtitleGeneration.test.ts src/server/videoSubtitleGeneration.test.ts src/components/EditorScreen.test.tsx`
|
|
Expected: PASS
|
|
|
|
**Step 2: Fix any regressions**
|
|
|
|
Make only the minimal changes required to keep Doubao and Gemini flows working.
|
|
|
|
**Step 3: Re-run the focused suite**
|
|
|
|
Run: `npm.cmd run test -- src/services/subtitleService.test.ts src/server/subtitleRequest.test.ts src/server/subtitleGeneration.test.ts src/server/videoSubtitleGeneration.test.ts src/components/EditorScreen.test.tsx`
|
|
Expected: PASS
|
|
|
|
**Step 4: Commit**
|
|
|
|
```bash
|
|
git add src/services/subtitleService.ts src/services/subtitleService.test.ts src/server/subtitleRequest.ts src/server/subtitleRequest.test.ts src/server/subtitleGeneration.ts src/server/subtitleGeneration.test.ts src/server/videoSubtitleGeneration.ts src/server/videoSubtitleGeneration.test.ts src/components/EditorScreen.test.tsx server.ts docs/plans/2026-03-19-doubao-file-id-frontend-design.md docs/plans/2026-03-19-doubao-file-id-frontend.md
|
|
git commit -m "feat: use ark file ids for doubao subtitle generation"
|
|
```
|