commit code
All checks were successful
Gitea Actions Demo / Explore-Gitea-Actions (push) Successful in 1m6s
All checks were successful
Gitea Actions Demo / Explore-Gitea-Actions (push) Successful in 1m6s
This commit is contained in:
Song367
parent
d10c788535
commit
04072dc94b
1
.env
1
.env
@ -3,5 +3,6 @@ ARK_API_KEY="e96194a9-8eda-4a90-a211-6db288045bdc"
|
||||
MINIMAX_API_KEY="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJHcm91cE5hbWUiOiLkuIrmtbfpopzpgJTnp5HmioDmnInpmZDlhazlj7giLCJVc2VyTmFtZSI6IuadqOmqpSIsIkFjY291bnQiOiIiLCJTdWJqZWN0SUQiOiIxNzI4NzEyMzI0OTc5NjI2ODM5IiwiUGhvbmUiOiIxMzM4MTU1OTYxOCIsIkdyb3VwSUQiOiIxNzI4NzEyMzI0OTcxMjM4MjMxIiwiUGFnZU5hbWUiOiIiLCJNYWlsIjoiIiwiQ3JlYXRlVGltZSI6IjIwMjUtMDYtMDYgMTU6MDU6NTUiLCJUb2tlblR5cGUiOjEsImlzcyI6Im1pbmltYXgifQ.aw1AUJnBYxXerJ4qNUaXM3DqPTd94WSVHWRiIpnjImhuCia3Ta1AyANTQTx__2CF5eByHOaHJFHhBCg6KgHUEaR6TiWFn0fWwXaU7XgnHwbvD4pNAmF_uYxMKbi-a6IyIGNyFdEMy22V5JEqfY4okAco5U96cnSOQZH7lyIBpvOsesjZU6L9q6Tf2jvlcnO9QG8GPg2DVpeL8Q3zLuYWezN4Wk6N-ISwQmZUwBYL3BhYamsFqCdSEyMd_uYQ_aQJa5tmlQqpimtALiutFshPUXB6VsvXEO6q-lCZ6Tg8QWwlFHkmEtUMQw4pWoX25d7Us06VFUhvV6pOzvM7yqCaWw"
|
||||
VITE_BASE_URL=/video_translate/
|
||||
VITE_API_BASE_PATH=/video_translate/api
|
||||
DOUBAO_MODEL="doubao-seed-2-0-pro-260215"
|
||||
DOUBAO_TIMEOUT_MS=900000
|
||||
VITE_ARK_API_KEY="e96194a9-8eda-4a90-a211-6db288045bdc"
|
||||
|
||||
117
docs/plans/2026-03-19-bilingual-ui-design.md
Normal file
117
docs/plans/2026-03-19-bilingual-ui-design.md
Normal file
@ -0,0 +1,117 @@
|
||||
# Bilingual UI Design
|
||||
|
||||
**Goal:** Make the application UI available in both Chinese and English, defaulting to Chinese and allowing users to switch languages with a button.
|
||||
|
||||
## Context
|
||||
|
||||
The current application UI is written almost entirely in English across the upload screen, trim modal, editor, export modal, and voice market modal. The user wants the product to support both Chinese and English and to switch between them at runtime with a visible button.
|
||||
|
||||
The requested behavior is UI-only internationalization. It should not change the underlying subtitle generation workflow, target-language request payloads, or model/provider behavior.
|
||||
|
||||
## Approaches Considered
|
||||
|
||||
### Option A: Prop-based translations
|
||||
|
||||
Store the current locale in `App` and pass translated strings into every component via props.
|
||||
|
||||
**Pros**
|
||||
- Very explicit data flow
|
||||
- No new React context
|
||||
|
||||
**Cons**
|
||||
- High prop churn across many components
|
||||
- Harder to maintain as UI grows
|
||||
|
||||
### Option B: Lightweight local i18n context
|
||||
|
||||
Create a small translation module and a React context that exposes:
|
||||
|
||||
- current locale
|
||||
- locale setter
|
||||
- translation lookup helper
|
||||
|
||||
Components read strings through a shared hook.
|
||||
|
||||
**Pros**
|
||||
- Small implementation footprint
|
||||
- Easy to scale across the existing component tree
|
||||
- Avoids adding a third-party i18n library for a two-language UI
|
||||
|
||||
**Cons**
|
||||
- Requires a small amount of context plumbing
|
||||
|
||||
### Recommendation
|
||||
|
||||
Use **Option B**. It keeps the implementation simple while making future UI expansion much easier.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Locale State
|
||||
|
||||
`App` will own the current locale with these rules:
|
||||
|
||||
- supported locales: `zh`, `en`
|
||||
- default locale: `zh`
|
||||
|
||||
`App` will wrap the UI in a small provider that exposes locale state and a `t(...)` helper.
|
||||
|
||||
### Translation Dictionary
|
||||
|
||||
Create a single front-end translation dictionary that covers current user-visible strings for:
|
||||
|
||||
- upload screen
|
||||
- trim modal
|
||||
- editor
|
||||
- export modal
|
||||
- voice market modal
|
||||
|
||||
The dictionary should use stable keys rather than raw string matching.
|
||||
|
||||
### Language Switcher
|
||||
|
||||
Add a compact switcher button group in the app shell so the user can toggle:
|
||||
|
||||
- `中文`
|
||||
- `English`
|
||||
|
||||
The active locale should be visually highlighted.
|
||||
|
||||
### Language List Rendering
|
||||
|
||||
The upload screen's target-language options should display localized labels while still preserving a stable English request value for the backend. This avoids accidentally changing API behavior when the UI language changes.
|
||||
|
||||
Each supported target language entry should therefore include:
|
||||
|
||||
- code
|
||||
- backend display name or canonical request name
|
||||
- Chinese label
|
||||
- English label
|
||||
|
||||
## Scope
|
||||
|
||||
### Included
|
||||
|
||||
- Front-end UI copy
|
||||
- Language switch button
|
||||
- Localized supported-language labels in the upload screen
|
||||
- Chinese default locale
|
||||
|
||||
### Excluded
|
||||
|
||||
- Backend message localization
|
||||
- API error localization from third-party providers
|
||||
- Browser-language auto-detection
|
||||
- Persistence across refreshes
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
Add tests that verify:
|
||||
|
||||
- the app defaults to Chinese
|
||||
- clicking the language switch updates visible UI labels to English
|
||||
- upload-screen language options render localized names correctly
|
||||
- existing component behavior continues to work while localized
|
||||
|
||||
## Rollout Notes
|
||||
|
||||
This design intentionally uses a local translation system instead of a third-party i18n package. That keeps the change fast, understandable, and easy to refactor later if the app grows into a larger multilingual product.
|
||||
174
docs/plans/2026-03-19-bilingual-ui.md
Normal file
174
docs/plans/2026-03-19-bilingual-ui.md
Normal file
@ -0,0 +1,174 @@
|
||||
# Bilingual UI Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add Chinese and English UI support with Chinese as the default and a runtime switch button.
|
||||
|
||||
**Architecture:** `App` owns locale state and provides it through a lightweight translation context. UI components read localized strings from a shared dictionary and switch instantly without changing backend-facing request values.
|
||||
|
||||
**Tech Stack:** React, TypeScript, Vite, Vitest, Testing Library
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add failing tests for app-level language switching
|
||||
|
||||
**Files:**
|
||||
- Create: `src/App.test.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Add tests that verify:
|
||||
|
||||
- the upload screen defaults to Chinese copy
|
||||
- clicking the English switch updates visible text to English
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/App.test.tsx`
|
||||
Expected: FAIL because no locale state or switcher exists yet
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Do not implement here; continue to later tasks.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run after Tasks 2 through 4.
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/App.test.tsx src/App.tsx src/i18n.ts src/components/*.tsx
|
||||
git commit -m "test: cover bilingual UI switching"
|
||||
```
|
||||
|
||||
### Task 2: Add translation infrastructure
|
||||
|
||||
**Files:**
|
||||
- Create: `src/i18n.tsx`
|
||||
- Modify: `src/App.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Covered by Task 1.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/App.test.tsx`
|
||||
Expected: FAIL due to missing locale provider and switch control
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Add locale type and translation dictionary
|
||||
- Add a small React context and translation hook
|
||||
- Store locale in `App` with `zh` as default
|
||||
- Add a visible `中文 / English` switcher
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run after localized components are updated.
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/i18n.tsx src/App.tsx src/App.test.tsx
|
||||
git commit -m "feat: add lightweight UI locale infrastructure"
|
||||
```
|
||||
|
||||
### Task 3: Localize the upload flow
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/UploadScreen.tsx`
|
||||
- Modify: `src/components/TrimModal.tsx`
|
||||
- Modify: `src/components/UploadScreen.test.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Extend or add tests only if needed beyond Task 1.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/App.test.tsx src/components/UploadScreen.test.tsx`
|
||||
Expected: FAIL because upload copy and language labels remain English-only
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Localize upload-screen copy
|
||||
- Localize target-language labels while preserving stable backend names
|
||||
- Localize trim modal copy
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/App.test.tsx src/components/UploadScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/UploadScreen.tsx src/components/TrimModal.tsx src/components/UploadScreen.test.tsx src/i18n.tsx
|
||||
git commit -m "feat: localize upload flow"
|
||||
```
|
||||
|
||||
### Task 4: Localize editor and modal surfaces
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/EditorScreen.tsx`
|
||||
- Modify: `src/components/ExportModal.tsx`
|
||||
- Modify: `src/components/VoiceMarketModal.tsx`
|
||||
- Modify: `src/components/EditorScreen.test.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Reuse the app-level switch test where possible.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/App.test.tsx src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL if editor or modal copy remains untranslated
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Localize editor copy and labels
|
||||
- Localize export modal copy
|
||||
- Localize voice market modal copy
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/App.test.tsx src/components/EditorScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/EditorScreen.tsx src/components/ExportModal.tsx src/components/VoiceMarketModal.tsx src/components/EditorScreen.test.tsx src/i18n.tsx
|
||||
git commit -m "feat: localize editor and modal UI"
|
||||
```
|
||||
|
||||
### Task 5: Verify targeted bilingual behavior
|
||||
|
||||
**Files:**
|
||||
- Modify only if verification exposes gaps
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Only add one if targeted verification finds a missing translation path.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Only if needed.
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Apply the smallest follow-up fix needed.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/App.test.tsx src/components/UploadScreen.test.tsx src/components/EditorScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/App.tsx src/i18n.tsx src/components/UploadScreen.tsx src/components/TrimModal.tsx src/components/EditorScreen.tsx src/components/ExportModal.tsx src/components/VoiceMarketModal.tsx
|
||||
git commit -m "test: verify bilingual UI"
|
||||
```
|
||||
100
docs/plans/2026-03-19-subtitle-jobs-async-design.md
Normal file
100
docs/plans/2026-03-19-subtitle-jobs-async-design.md
Normal file
@ -0,0 +1,100 @@
|
||||
# Subtitle Jobs Async Design
|
||||
|
||||
## Goal
|
||||
|
||||
Replace the synchronous `/api/generate-subtitles` request with an async job flow so long-running subtitle generation can survive gateway timeouts and provide user-visible progress.
|
||||
|
||||
## Chosen Approach
|
||||
|
||||
Use a single-process in-memory job store on the backend and 5-second polling on the frontend.
|
||||
|
||||
The backend will:
|
||||
|
||||
- accept subtitle generation requests
|
||||
- create a job with `queued` status
|
||||
- return `202 Accepted` immediately with `jobId`
|
||||
- continue subtitle generation in the background
|
||||
- expose `GET /api/generate-subtitles/:jobId` for status and result lookup
|
||||
|
||||
The frontend will:
|
||||
|
||||
- submit the job
|
||||
- poll every 5 seconds
|
||||
- map backend `stage` values to a staged progress bar
|
||||
- stop polling on `succeeded`, `failed`, or timeout
|
||||
|
||||
## Job Model
|
||||
|
||||
Each subtitle job stores:
|
||||
|
||||
- `id`
|
||||
- `requestId`
|
||||
- `status`
|
||||
- `stage`
|
||||
- `progress`
|
||||
- `message`
|
||||
- `createdAt`
|
||||
- `updatedAt`
|
||||
- `provider`
|
||||
- `targetLanguage`
|
||||
- `fileId`
|
||||
- `filePath`
|
||||
- `error`
|
||||
- `result`
|
||||
|
||||
Statuses:
|
||||
|
||||
- `queued`
|
||||
- `running`
|
||||
- `succeeded`
|
||||
- `failed`
|
||||
|
||||
Stages:
|
||||
|
||||
- `queued`
|
||||
- `upload_received`
|
||||
- `preparing`
|
||||
- `calling_provider`
|
||||
- `processing_result`
|
||||
- `succeeded`
|
||||
- `failed`
|
||||
|
||||
Jobs expire from memory after 1 hour.
|
||||
|
||||
## Progress Strategy
|
||||
|
||||
Progress is stage-based, not byte-accurate. This avoids fake precision while still keeping the user informed.
|
||||
|
||||
Suggested mapping:
|
||||
|
||||
- `queued`: 5
|
||||
- `upload_received`: 15
|
||||
- `preparing`: 30
|
||||
- `calling_provider`: 70
|
||||
- `processing_result`: 90
|
||||
- `succeeded`: 100
|
||||
- `failed`: keep last reported progress
|
||||
|
||||
## Backend Flow
|
||||
|
||||
1. Parse request and validate source input.
|
||||
2. Create a job.
|
||||
3. Return `202` with the new job state.
|
||||
4. Run subtitle generation in a detached async task.
|
||||
5. Update the job on each pipeline stage.
|
||||
6. Store final result or error.
|
||||
7. Clean up uploaded temp files after background completion.
|
||||
|
||||
## Frontend Flow
|
||||
|
||||
1. Doubao still uploads to Ark Files first and waits for file readiness.
|
||||
2. Frontend posts to `/api/generate-subtitles`.
|
||||
3. Frontend polls `/api/generate-subtitles/:jobId` every 5 seconds.
|
||||
4. Progress UI updates from backend job status and stage.
|
||||
5. Final result is normalized into `SubtitlePipelineResult`.
|
||||
|
||||
## Risks
|
||||
|
||||
- In-memory jobs are lost on restart.
|
||||
- Single-instance memory state will not scale across multiple backend replicas.
|
||||
- Upload time can still be slow, but the long model invocation is no longer tied to one HTTP response.
|
||||
47
docs/plans/2026-03-19-subtitle-jobs-async.md
Normal file
47
docs/plans/2026-03-19-subtitle-jobs-async.md
Normal file
@ -0,0 +1,47 @@
|
||||
# Subtitle Jobs Async Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Convert subtitle generation to an async job flow with 5-second polling and a staged progress bar.
|
||||
|
||||
**Architecture:** Add a backend in-memory subtitle job store, make `/api/generate-subtitles` return `202` immediately, expose a job lookup endpoint, and teach the frontend subtitle service and editor UI to poll and render stage-based progress.
|
||||
|
||||
**Tech Stack:** Express, TypeScript, React, Vitest
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add backend job store helpers
|
||||
|
||||
**Files:**
|
||||
- Create: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleJobs.ts`
|
||||
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleJobs.test.ts`
|
||||
|
||||
### Task 2: Convert subtitle route to async job submission
|
||||
|
||||
**Files:**
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\server.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleGeneration.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\videoSubtitleGeneration.ts`
|
||||
|
||||
### Task 3: Poll subtitle jobs from the frontend
|
||||
|
||||
**Files:**
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\services\subtitleService.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\services\subtitleService.test.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\types.ts`
|
||||
|
||||
### Task 4: Show staged progress in the editor UI
|
||||
|
||||
**Files:**
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.tsx`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.test.tsx`
|
||||
|
||||
### Task 5: Verify focused tests and typecheck
|
||||
|
||||
**Files:**
|
||||
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleJobs.test.ts`
|
||||
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\services\subtitleService.test.ts`
|
||||
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.test.tsx`
|
||||
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleGeneration.test.ts`
|
||||
- Test: `E:\Downloads\ai-video-dubbing-&-translation\src\server\videoSubtitleGeneration.test.ts`
|
||||
|
||||
142
docs/plans/2026-03-19-subtitle-properties-panel-design.md
Normal file
142
docs/plans/2026-03-19-subtitle-properties-panel-design.md
Normal file
@ -0,0 +1,142 @@
|
||||
# Subtitle Properties Panel Design
|
||||
|
||||
**Goal:** Turn the editor's right sidebar into a real subtitle properties panel that edits the selected subtitle's display settings, shows real speaker metadata, and can optionally apply style changes to all subtitles.
|
||||
|
||||
## Context
|
||||
|
||||
The current editor already presents the right side as a properties area, but most of it is still static UI. The `Wife` and `Husband` labels are hard-coded, the text style controls only update a single top-level `textStyles` state object, and the preview overlay always renders with those shared settings instead of reading from the active subtitle.
|
||||
|
||||
That makes the panel look like a subtitle control surface while behaving more like a mock. The user expectation is that this area should control subtitle presentation for the selected item and optionally propagate those style changes across the full subtitle list.
|
||||
|
||||
## Approaches Considered
|
||||
|
||||
### Option A: Keep a single global style object
|
||||
|
||||
Continue storing one `textStyles` object in `EditorScreen` and wire the right panel to that object only.
|
||||
|
||||
**Pros**
|
||||
- Smallest code change
|
||||
- Minimal test churn
|
||||
|
||||
**Cons**
|
||||
- Does not match the "current subtitle properties" mental model
|
||||
- Makes "Apply to all subtitles" redundant
|
||||
- Prevents subtitle-level overrides later
|
||||
|
||||
### Option B: Store style state per subtitle with optional bulk apply
|
||||
|
||||
Add a style object to each subtitle, derive the right panel from the active subtitle, and let bulk apply copy style changes to every subtitle.
|
||||
|
||||
**Pros**
|
||||
- Matches the current UI language and user expectation
|
||||
- Keeps the implementation scoped to the existing editor screen
|
||||
- Leaves room for richer per-speaker or per-preset styling later
|
||||
|
||||
**Cons**
|
||||
- Requires subtitle normalization when data is loaded
|
||||
- Slightly more state update logic in the editor
|
||||
|
||||
### Recommendation
|
||||
|
||||
Use **Option B**. It gives the right sidebar a truthful interaction model without forcing a much larger style system redesign.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Data Model
|
||||
|
||||
Introduce a `SubtitleTextStyle` shape that extends the current style fields with the controls surfaced by the panel:
|
||||
|
||||
- `fontFamily`
|
||||
- `fontSize`
|
||||
- `color`
|
||||
- `strokeColor`
|
||||
- `strokeWidth`
|
||||
- `alignment`
|
||||
- `isBold`
|
||||
- `isItalic`
|
||||
- `isUnderline`
|
||||
|
||||
Each `Subtitle` will gain an optional `textStyle` field. When subtitle data is generated or loaded, the editor will normalize every subtitle to ensure a complete style object is present.
|
||||
|
||||
### Editor State
|
||||
|
||||
`EditorScreen` will:
|
||||
|
||||
1. Keep a single source of truth in `subtitles`
|
||||
2. Derive `activeSubtitle` from `activeSubtitleId`
|
||||
3. Replace the old shared `textStyles` state with helper functions that update:
|
||||
- only the active subtitle
|
||||
- or every subtitle when "Apply to all subtitles" is enabled
|
||||
|
||||
The checkbox will become real state instead of a static default value.
|
||||
|
||||
### Right Sidebar Behavior
|
||||
|
||||
The properties panel will become data-driven:
|
||||
|
||||
1. Speaker badge/label will read from the active subtitle's speaker metadata
|
||||
2. The text field will show the active subtitle's translated text as contextual reference instead of a hard-coded name
|
||||
3. Controls will read and write the active subtitle's `textStyle`
|
||||
4. The panel will show a friendly empty state if no subtitle is selected
|
||||
|
||||
This keeps the right column focused on subtitle properties without introducing an unrelated metadata editor.
|
||||
|
||||
### Preview Behavior
|
||||
|
||||
The video subtitle overlay will render the style of the subtitle currently visible at the playback position. This means:
|
||||
|
||||
- editing the selected subtitle updates the preview immediately when that subtitle is active
|
||||
- bulk apply updates every visible subtitle style consistently
|
||||
|
||||
Stroke rendering will be added via CSS `textShadow` so the preview reflects the new sidebar control.
|
||||
|
||||
## Interaction Details
|
||||
|
||||
### Size Presets
|
||||
|
||||
Keep the existing "Normal / Large / Small" control, but wire it to concrete font sizes:
|
||||
|
||||
- `Small` -> 20
|
||||
- `Normal` -> 24
|
||||
- `Large` -> 30
|
||||
|
||||
This matches the current UI affordance without adding a freeform number input yet.
|
||||
|
||||
### Apply To All
|
||||
|
||||
When checked, any style change initiated from the right panel copies the changed style keys to all subtitles. It will not overwrite subtitle text, timing, voice, or speaker data.
|
||||
|
||||
### Speaker Information
|
||||
|
||||
The top block will present the active subtitle's speaker identity:
|
||||
|
||||
- badge uses the first letter of the speaker label
|
||||
- label shows `subtitle.speaker`
|
||||
- secondary read-only field shows the active subtitle text for context
|
||||
|
||||
This keeps the section meaningful even if there is no separate speaker alias dataset yet.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- If no subtitles are loaded, the right panel stays disabled with helper text
|
||||
- If a subtitle lacks style data, normalization fills defaults before rendering controls
|
||||
- If the active subtitle disappears after regeneration, the editor selects the first subtitle as it already does today
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Component Tests
|
||||
|
||||
Add tests that verify:
|
||||
|
||||
- the right panel reflects the active subtitle's speaker and style values
|
||||
- changing a style control updates only the selected subtitle when bulk apply is off
|
||||
- changing a style control updates every subtitle when bulk apply is on
|
||||
- the preview overlay renders the visible subtitle using its style values
|
||||
|
||||
### Type Safety
|
||||
|
||||
Update shared types so exports and preview rendering both receive the richer subtitle style data without unsafe casts.
|
||||
|
||||
## Rollout Notes
|
||||
|
||||
This design intentionally stays inside `EditorScreen` and current data structures. It does not add saved presets, per-speaker style inheritance, or backend persistence, which keeps the work focused on making the existing panel real and trustworthy.
|
||||
177
docs/plans/2026-03-19-subtitle-properties-panel.md
Normal file
177
docs/plans/2026-03-19-subtitle-properties-panel.md
Normal file
@ -0,0 +1,177 @@
|
||||
# Subtitle Properties Panel Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Make the editor's right sidebar behave like a real current-subtitle properties panel with optional bulk style application.
|
||||
|
||||
**Architecture:** The implementation keeps `EditorScreen` as the orchestration layer while moving subtitle style data into each `Subtitle`. Subtitle data is normalized on load, the right sidebar binds to the active subtitle, and preview/export paths read from subtitle-level style settings instead of one shared top-level state object.
|
||||
|
||||
**Tech Stack:** React, TypeScript, Vite, Vitest, Testing Library, Tailwind utility classes
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Extend subtitle types for per-subtitle styles
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/types.ts`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Use existing component tests in the next task to drive type usage rather than adding a dedicated type-only test.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL after the next test task starts referencing `subtitle.textStyle`
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Add a `SubtitleTextStyle` interface with font family, size, color, stroke color, stroke width, alignment, and font decoration flags
|
||||
- Add optional `textStyle?: SubtitleTextStyle` to `Subtitle`
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run after Task 2 implementation.
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/types.ts src/components/EditorScreen.test.tsx src/components/EditorScreen.tsx
|
||||
git commit -m "feat: support per-subtitle style settings"
|
||||
```
|
||||
|
||||
### Task 2: Add failing editor tests for the properties panel behavior
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/EditorScreen.test.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Add tests that:
|
||||
|
||||
- render subtitles with different speakers and styles
|
||||
- verify the right panel shows the active speaker label
|
||||
- toggle bold with bulk apply disabled and assert only one subtitle changes
|
||||
- toggle bulk apply on, change a style, and assert all subtitles update
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL because the current panel is static and does not update subtitle-level style data
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Do not implement here; proceed to Tasks 3 and 4.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run after Tasks 3 and 4.
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/EditorScreen.test.tsx src/components/EditorScreen.tsx src/types.ts
|
||||
git commit -m "test: cover subtitle properties panel interactions"
|
||||
```
|
||||
|
||||
### Task 3: Normalize subtitle style data and bind the sidebar to the active subtitle
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/EditorScreen.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Covered by Task 2.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL with missing active-speaker and style-update assertions
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Add a default subtitle style constant and normalization helper
|
||||
- Normalize generated subtitles before storing them in state
|
||||
- Derive `activeSubtitle` and `activeSubtitleStyle`
|
||||
- Replace hard-coded speaker/name fields with active subtitle data
|
||||
- Add `applyToAllSubtitles` state
|
||||
- Add helper functions that update subtitle styles per subtitle or across all subtitles
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: PASS for sidebar interaction assertions
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/EditorScreen.tsx src/components/EditorScreen.test.tsx src/types.ts
|
||||
git commit -m "feat: wire subtitle properties panel to active subtitle"
|
||||
```
|
||||
|
||||
### Task 4: Render subtitle preview with subtitle-level styles
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/EditorScreen.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Covered by Task 2 by asserting the preview subtitle reflects changed style controls.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL because the overlay still reads the old shared style state
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Replace shared overlay style usage with the currently visible subtitle's style
|
||||
- Map size presets to concrete font sizes
|
||||
- Add stroke rendering through `textShadow`
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/EditorScreen.tsx src/components/EditorScreen.test.tsx
|
||||
git commit -m "feat: preview subtitle-level style changes"
|
||||
```
|
||||
|
||||
### Task 5: Verify targeted editor behavior
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/EditorScreen.tsx` if any verification fixes are needed
|
||||
- Modify: `src/components/EditorScreen.test.tsx` if any verification fixes are needed
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
No new test unless verification exposes a gap.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Only if a regression is found during verification.
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Apply only the smallest follow-up fixes needed to make the targeted suite green.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
Optional broader check:
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx src/services/subtitleService.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/EditorScreen.tsx src/components/EditorScreen.test.tsx src/types.ts
|
||||
git commit -m "test: verify subtitle properties panel flow"
|
||||
```
|
||||
41
docs/plans/2026-03-19-tts-language-contract-design.md
Normal file
41
docs/plans/2026-03-19-tts-language-contract-design.md
Normal file
@ -0,0 +1,41 @@
|
||||
# TTS Language Contract Design
|
||||
|
||||
## Goal
|
||||
|
||||
Upgrade subtitle generation so the model always produces English subtitle text for display, plus a separate TTS translation and language code for dubbing.
|
||||
|
||||
## Scope
|
||||
|
||||
- Keep current upload and editor UI unchanged for this step.
|
||||
- Add backend contract support for `ttsText` and `ttsLanguage`.
|
||||
- Let dubbing prefer `ttsText` over `translatedText`.
|
||||
- Keep existing calls backward compatible by defaulting `ttsLanguage` to the current target language when none is provided.
|
||||
|
||||
## Design
|
||||
|
||||
### Prompt contract
|
||||
|
||||
- `translatedText` becomes the on-screen subtitle text and must always be English.
|
||||
- `ttsText` becomes the spoken dubbing text in the requested TTS language.
|
||||
- `ttsLanguage` must be returned on every subtitle item and must exactly match the requested TTS language code.
|
||||
- The system and user prompts should clearly separate subtitle language from TTS language.
|
||||
|
||||
### Data model
|
||||
|
||||
- Extend `Subtitle` with optional `ttsText` and `ttsLanguage`.
|
||||
- Extend raw model subtitle parsing to accept these fields.
|
||||
- Extend pipeline result metadata to track `ttsLanguage`.
|
||||
|
||||
### Runtime behavior
|
||||
|
||||
- Subtitle generation should accept an optional `ttsLanguage`.
|
||||
- If not provided, use `targetLanguage` to avoid breaking existing flows.
|
||||
- Voice catalog selection should use the TTS language, not the subtitle language.
|
||||
- TTS generation should read `subtitle.ttsText` first, then fall back to `translatedText`, then `text`.
|
||||
|
||||
### Testing
|
||||
|
||||
- Add prompt tests asserting the new system and user prompt text references English subtitles plus TTS language.
|
||||
- Add parsing tests asserting `ttsText` and `ttsLanguage` are normalized into subtitles.
|
||||
- Add service tests asserting `ttsLanguage` is forwarded through the subtitle pipeline request body.
|
||||
|
||||
61
docs/plans/2026-03-19-tts-language-contract.md
Normal file
61
docs/plans/2026-03-19-tts-language-contract.md
Normal file
@ -0,0 +1,61 @@
|
||||
# TTS Language Contract Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add `ttsText` and `ttsLanguage` to subtitle generation so English subtitles and TTS dubbing language are handled separately.
|
||||
|
||||
**Architecture:** Keep the current subtitle pipeline shape, but thread an optional `ttsLanguage` through request parsing, prompt creation, and subtitle normalization. Dubbing should use `ttsText` when available so later UI work can choose a TTS language without reworking the playback layer.
|
||||
|
||||
**Tech Stack:** TypeScript, Vitest, React client services, Node subtitle generation pipeline
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Lock the new prompt and parsing contract with tests
|
||||
|
||||
**Files:**
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\videoSubtitleGeneration.test.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\services\subtitleService.test.ts`
|
||||
|
||||
**Step 1: Write the failing tests**
|
||||
|
||||
- Assert the user prompt mentions `Subtitle language: English` and `TTS language`.
|
||||
- Assert the system prompt includes `ttsText` and `ttsLanguage`.
|
||||
- Assert normalized subtitles preserve returned `ttsText` and `ttsLanguage`.
|
||||
- Assert the client request forwards `ttsLanguage`.
|
||||
|
||||
**Step 2: Run tests to verify they fail**
|
||||
|
||||
Run:
|
||||
`npm run test -- src/server/videoSubtitleGeneration.test.ts src/services/subtitleService.test.ts`
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
Skip commit for now.
|
||||
|
||||
### Task 2: Implement the backend contract
|
||||
|
||||
**Files:**
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\videoSubtitleGeneration.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleGeneration.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\server\subtitleRequest.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\types.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\services\subtitleService.ts`
|
||||
- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.tsx`
|
||||
|
||||
**Step 1: Write the minimal implementation**
|
||||
|
||||
- Add optional `ttsText` and `ttsLanguage` to subtitle types.
|
||||
- Accept optional `ttsLanguage` in request parsing and pipeline functions.
|
||||
- Build prompts with English subtitle language plus requested TTS language.
|
||||
- Normalize model output into subtitle objects with `ttsText` and `ttsLanguage`.
|
||||
- Use `ttsText` for TTS generation when present.
|
||||
|
||||
**Step 2: Run tests to verify they pass**
|
||||
|
||||
Run:
|
||||
`npm run test -- src/server/videoSubtitleGeneration.test.ts src/services/subtitleService.test.ts src/components/EditorScreen.test.tsx`
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
Skip commit for now.
|
||||
|
||||
112
docs/plans/2026-03-19-upload-subtitle-defaults-design.md
Normal file
112
docs/plans/2026-03-19-upload-subtitle-defaults-design.md
Normal file
@ -0,0 +1,112 @@
|
||||
# Upload Subtitle Defaults Design
|
||||
|
||||
**Goal:** Let users set the initial subtitle position and subtitle size on the upload screen, with defaults matching the current editor behavior, and carry those values into the editor as the starting subtitle style.
|
||||
|
||||
## Context
|
||||
|
||||
The upload screen currently collects the video file, trim range, and target language. The editor later applies a default subtitle style internally, which means users have no chance to define their preferred starting subtitle placement or text size before generation begins.
|
||||
|
||||
The user wants the upload page to support free adjustment of subtitle position and subtitle size, while keeping a sensible default identical to the current editor: subtitle text near the bottom of the video and a medium default size. The upload page should define the initial values only, and the editor should continue using them as the starting style.
|
||||
|
||||
## Approaches Considered
|
||||
|
||||
### Option A: Add preset-only controls on the upload screen
|
||||
|
||||
Offer fixed choices such as top / middle / bottom and small / medium / large.
|
||||
|
||||
**Pros**
|
||||
- Smallest implementation
|
||||
- Easy to understand quickly
|
||||
|
||||
**Cons**
|
||||
- Does not satisfy the request for free adjustment
|
||||
- Makes it harder for users to match personal subtitle habits
|
||||
|
||||
### Option B: Add free controls with a lightweight preview
|
||||
|
||||
Expose a vertical position control and a size slider on the upload screen, preview the result with a sample subtitle card, and pass the chosen values into the editor.
|
||||
|
||||
**Pros**
|
||||
- Matches the requested behavior
|
||||
- Keeps upload-page controls focused on the two initial settings only
|
||||
- Reuses the editor's existing subtitle style model
|
||||
|
||||
**Cons**
|
||||
- Requires new state to flow from upload to editor
|
||||
- Needs a preview so the controls feel meaningful
|
||||
|
||||
### Recommendation
|
||||
|
||||
Use **Option B**. It gives users the flexibility they asked for without overloading the upload screen with full subtitle styling.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Shared Style Data
|
||||
|
||||
Introduce a small shared shape for upload-time subtitle defaults:
|
||||
|
||||
- `fontSize`
|
||||
- `bottomOffsetPercent`
|
||||
|
||||
This can be represented either as a new dedicated interface or as part of the existing subtitle text style model plus a position field. The important part is that the upload screen stores these values and the editor uses them as its initial defaults for every generated subtitle.
|
||||
|
||||
### Upload Screen
|
||||
|
||||
Add a "Subtitle Defaults" card to the right settings column that includes:
|
||||
|
||||
- a preview frame with a sample subtitle
|
||||
- a vertical range input for subtitle position
|
||||
- a horizontal range input for subtitle size
|
||||
- a reset button that restores the current editor defaults
|
||||
|
||||
The preview should be lightweight and not depend on the selected video file. It only needs to make the chosen position and size visible.
|
||||
|
||||
### App State Flow
|
||||
|
||||
`App` will store the chosen upload defaults alongside:
|
||||
|
||||
- `videoFile`
|
||||
- `targetLanguage`
|
||||
- `trimRange`
|
||||
|
||||
When the upload is confirmed, `UploadScreen` passes the subtitle default settings up through `onUpload(...)`. `App` then forwards them to `EditorScreen`.
|
||||
|
||||
### Editor Initialization
|
||||
|
||||
`EditorScreen` will accept the upload-provided subtitle defaults and use them when normalizing generated subtitles. That ensures:
|
||||
|
||||
- all generated subtitles start with the chosen size
|
||||
- the overlay initially appears at the chosen vertical position
|
||||
- the existing editor style controls continue from those defaults
|
||||
|
||||
## Interaction Details
|
||||
|
||||
### Default Values
|
||||
|
||||
Match the current editor behavior:
|
||||
|
||||
- `fontSize: 24`
|
||||
- `bottomOffsetPercent: 10`
|
||||
|
||||
These values must be used both as initial upload-screen state and as the reset target.
|
||||
|
||||
### Position Control
|
||||
|
||||
Use a continuous slider representing distance from the bottom edge of the video preview. This is simpler and more reliable than drag-to-place in a small upload card.
|
||||
|
||||
### Size Control
|
||||
|
||||
Use a continuous slider with a bounded range suitable for subtitle readability, for example 16 to 40 pixels.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
Add tests that verify:
|
||||
|
||||
- upload-screen controls start from the same defaults as the editor
|
||||
- changing upload-screen position and size is reflected in the upload preview
|
||||
- confirming upload passes the selected subtitle defaults up to `App`
|
||||
- editor initialization applies the upload defaults to the visible subtitle overlay
|
||||
|
||||
## Rollout Notes
|
||||
|
||||
This change intentionally limits upload-time styling to size and vertical position. Full styling such as font family, color, stroke, and per-subtitle overrides remains the editor's responsibility.
|
||||
182
docs/plans/2026-03-19-upload-subtitle-defaults.md
Normal file
182
docs/plans/2026-03-19-upload-subtitle-defaults.md
Normal file
@ -0,0 +1,182 @@
|
||||
# Upload Subtitle Defaults Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add upload-screen controls for initial subtitle position and size, then apply those defaults when the editor loads generated subtitles.
|
||||
|
||||
**Architecture:** The upload screen owns temporary subtitle default settings and previews them locally. `App` stores the selected defaults during upload confirmation and passes them into `EditorScreen`, which uses them while normalizing generated subtitles and rendering the overlay.
|
||||
|
||||
**Tech Stack:** React, TypeScript, Vite, Vitest, Testing Library, Tailwind utility classes
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add failing tests for upload defaults flow
|
||||
|
||||
**Files:**
|
||||
- Create: `src/components/UploadScreen.test.tsx`
|
||||
- Modify: `src/components/EditorScreen.test.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Add tests that:
|
||||
|
||||
- verify upload-screen subtitle size and position controls start at the expected defaults
|
||||
- verify changing them updates the upload preview
|
||||
- verify upload confirmation passes subtitle defaults through `onUpload`
|
||||
- verify `EditorScreen` initializes the subtitle overlay with upload-provided defaults
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/UploadScreen.test.tsx src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL because upload defaults do not yet exist and the editor cannot consume them
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Do not implement here; continue to Tasks 2 through 4.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run after Tasks 2 through 4.
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/UploadScreen.test.tsx src/components/EditorScreen.test.tsx src/components/UploadScreen.tsx src/components/EditorScreen.tsx src/App.tsx src/types.ts
|
||||
git commit -m "test: cover upload subtitle defaults flow"
|
||||
```
|
||||
|
||||
### Task 2: Add shared subtitle-default types and app-level state
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/types.ts`
|
||||
- Modify: `src/App.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Covered by Task 1.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/UploadScreen.test.tsx src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL on missing props and missing subtitle-default state
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Add a shared interface for upload subtitle defaults
|
||||
- Store subtitle defaults in `App`
|
||||
- Extend the upload callback signature
|
||||
- Pass the chosen defaults into `EditorScreen`
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run after Tasks 3 and 4.
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/types.ts src/App.tsx src/components/UploadScreen.tsx src/components/EditorScreen.tsx
|
||||
git commit -m "feat: plumb upload subtitle defaults through app state"
|
||||
```
|
||||
|
||||
### Task 3: Implement upload-screen subtitle defaults UI
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/UploadScreen.tsx`
|
||||
- Test: `src/components/UploadScreen.test.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Covered by Task 1.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/UploadScreen.test.tsx`
|
||||
Expected: FAIL because the controls and preview do not exist
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Add subtitle size and vertical position state with current-editor default values
|
||||
- Add a preview card showing sample subtitle text
|
||||
- Add range inputs and a reset button
|
||||
- Ensure upload confirmation sends the chosen defaults via `onUpload`
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/components/UploadScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/UploadScreen.tsx src/components/UploadScreen.test.tsx
|
||||
git commit -m "feat: add upload-screen subtitle default controls"
|
||||
```
|
||||
|
||||
### Task 4: Apply upload defaults inside the editor
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/EditorScreen.tsx`
|
||||
- Modify: `src/components/EditorScreen.test.tsx`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Covered by Task 1.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: FAIL because the editor still uses hard-coded defaults
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
- Accept upload subtitle defaults as a prop
|
||||
- Use them while normalizing subtitles
|
||||
- Render the overlay at the chosen bottom offset
|
||||
- Keep existing editor controls working from these defaults
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/components/EditorScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/EditorScreen.tsx src/components/EditorScreen.test.tsx src/App.tsx src/types.ts
|
||||
git commit -m "feat: initialize editor subtitles from upload defaults"
|
||||
```
|
||||
|
||||
### Task 5: Verify targeted behavior
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/components/UploadScreen.tsx` if verification reveals issues
|
||||
- Modify: `src/components/EditorScreen.tsx` if verification reveals issues
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Only add one if targeted verification reveals a missing edge case.
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Only if a regression is found.
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Apply the smallest follow-up fix needed to make the targeted flow green.
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npm run test -- src/components/UploadScreen.test.tsx src/components/EditorScreen.test.tsx`
|
||||
Expected: PASS
|
||||
|
||||
Optional broader check:
|
||||
|
||||
Run: `npm run test -- src/components/UploadScreen.test.tsx src/components/EditorScreen.test.tsx src/lib/exportPayload.test.ts`
|
||||
Expected: PASS
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/components/UploadScreen.tsx src/components/UploadScreen.test.tsx src/components/EditorScreen.tsx src/components/EditorScreen.test.tsx src/App.tsx src/types.ts
|
||||
git commit -m "test: verify upload subtitle default behavior"
|
||||
```
|
||||
282
server.ts
282
server.ts
@ -20,7 +20,15 @@ import {
|
||||
DEFAULT_EXPORT_TEXT_STYLES,
|
||||
shiftSubtitlesToExportTimeline,
|
||||
} from './src/server/exportVideo';
|
||||
import { formatLogContext, serializeError } from './src/server/errorLogging';
|
||||
import { formatLogContext, logEvent, serializeError } from './src/server/errorLogging';
|
||||
import {
|
||||
createSubtitleJob,
|
||||
createSubtitleJobStore,
|
||||
getSubtitleJob,
|
||||
pruneExpiredSubtitleJobs,
|
||||
toSubtitleJobResponse,
|
||||
updateSubtitleJob,
|
||||
} from './src/server/subtitleJobs';
|
||||
import { TextStyles } from './src/types';
|
||||
|
||||
const upload = multer({
|
||||
@ -35,6 +43,33 @@ if (!fs.existsSync('uploads')) {
|
||||
fs.mkdirSync('uploads');
|
||||
}
|
||||
|
||||
const summarizeRequestHeaders = (headers: express.Request['headers']) => ({
|
||||
host: headers.host,
|
||||
'content-type': headers['content-type'],
|
||||
'content-length': headers['content-length'],
|
||||
'user-agent': headers['user-agent'],
|
||||
'x-forwarded-for': headers['x-forwarded-for'],
|
||||
'x-forwarded-proto': headers['x-forwarded-proto'],
|
||||
});
|
||||
|
||||
const SUBTITLE_JOB_TTL_MS = 60 * 60 * 1000;
|
||||
const subtitleJobStore = createSubtitleJobStore();
|
||||
|
||||
const deriveSubtitleErrorStatus = (message: string) => {
|
||||
const lowerMessage = message.toLowerCase();
|
||||
return lowerMessage.includes('target language') ||
|
||||
lowerMessage.includes('unsupported llm provider') ||
|
||||
lowerMessage.includes('_api_key is required') ||
|
||||
lowerMessage.includes('studio project fallback is disabled')
|
||||
? 400
|
||||
: lowerMessage.includes('unauthorized') ||
|
||||
lowerMessage.includes('authentication') ||
|
||||
lowerMessage.includes('auth fail') ||
|
||||
lowerMessage.includes('status 401')
|
||||
? 401
|
||||
: 502;
|
||||
};
|
||||
|
||||
dotenv.config();
|
||||
|
||||
const ffmpegPath = process.env.FFMPEG_PATH?.trim();
|
||||
@ -203,78 +238,235 @@ async function startServer() {
|
||||
const videoPath = req.file?.path;
|
||||
const requestId = `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const startedAt = Date.now();
|
||||
const requestContext = {
|
||||
requestId,
|
||||
method: req.method,
|
||||
path: req.originalUrl,
|
||||
fileName: req.file?.originalname,
|
||||
fileSize: req.file?.size,
|
||||
contentType: req.headers['content-type'],
|
||||
contentLength: req.headers['content-length'],
|
||||
host: req.headers.host,
|
||||
};
|
||||
|
||||
try {
|
||||
const { provider, targetLanguage, fileId } = parseSubtitleRequest(req.body);
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: '[subtitle] request received',
|
||||
context: requestContext,
|
||||
details: {
|
||||
headers: summarizeRequestHeaders(req.headers),
|
||||
bodyKeys: Object.keys(req.body || {}).slice(0, 20),
|
||||
hasUploadFile: Boolean(req.file),
|
||||
},
|
||||
});
|
||||
|
||||
const { provider, targetLanguage, ttsLanguage, fileId } = parseSubtitleRequest(req.body);
|
||||
if (!videoPath && !fileId) {
|
||||
logEvent({
|
||||
level: 'warn',
|
||||
message: '[subtitle] request rejected: missing video source',
|
||||
context: {
|
||||
...requestContext,
|
||||
durationMs: Date.now() - startedAt,
|
||||
},
|
||||
details: {
|
||||
fileId,
|
||||
parsedProvider: provider,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
},
|
||||
});
|
||||
return res.status(400).json({ error: 'No video file provided' });
|
||||
}
|
||||
console.info(
|
||||
`[subtitle] request started ${formatLogContext({
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
fileName: req.file?.originalname,
|
||||
fileSize: req.file?.size,
|
||||
})}`,
|
||||
);
|
||||
|
||||
const result = await generateSubtitlePipeline({
|
||||
videoPath,
|
||||
fileId,
|
||||
pruneExpiredSubtitleJobs(subtitleJobStore, SUBTITLE_JOB_TTL_MS);
|
||||
const subtitleJob = createSubtitleJob(subtitleJobStore, {
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
env: process.env,
|
||||
requestId,
|
||||
ttsLanguage,
|
||||
fileId,
|
||||
filePath: videoPath,
|
||||
});
|
||||
|
||||
console.info(
|
||||
`[subtitle] request succeeded ${formatLogContext({
|
||||
updateSubtitleJob(subtitleJobStore, subtitleJob.id, {
|
||||
status: 'running',
|
||||
stage: videoPath ? 'upload_received' : 'queued',
|
||||
progress: videoPath ? 15 : 5,
|
||||
message: videoPath ? 'Upload received' : 'Queued',
|
||||
});
|
||||
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: `[subtitle] job accepted ${formatLogContext({
|
||||
jobId: subtitleJob.id,
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
durationMs: Date.now() - startedAt,
|
||||
subtitleCount: result.subtitles.length,
|
||||
ttsLanguage,
|
||||
fileName: req.file?.originalname,
|
||||
fileSize: req.file?.size,
|
||||
fileId,
|
||||
})}`,
|
||||
);
|
||||
|
||||
res.json({
|
||||
...result,
|
||||
provider,
|
||||
requestId,
|
||||
context: {
|
||||
jobId: subtitleJob.id,
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
fileName: req.file?.originalname,
|
||||
fileSize: req.file?.size,
|
||||
fileId,
|
||||
},
|
||||
});
|
||||
|
||||
res.status(202).json(toSubtitleJobResponse(getSubtitleJob(subtitleJobStore, subtitleJob.id)!));
|
||||
|
||||
void (async () => {
|
||||
try {
|
||||
const result = await generateSubtitlePipeline({
|
||||
videoPath,
|
||||
fileId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
env: process.env,
|
||||
requestId,
|
||||
onProgress: (progress) => {
|
||||
updateSubtitleJob(subtitleJobStore, subtitleJob.id, {
|
||||
status: progress.status,
|
||||
stage: progress.stage,
|
||||
progress: progress.progress,
|
||||
message: progress.message,
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
updateSubtitleJob(subtitleJobStore, subtitleJob.id, {
|
||||
status: 'succeeded',
|
||||
stage: 'succeeded',
|
||||
progress: 100,
|
||||
message: 'Subtitle generation completed',
|
||||
result: {
|
||||
...result,
|
||||
provider,
|
||||
requestId,
|
||||
},
|
||||
});
|
||||
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: `[subtitle] background job succeeded ${formatLogContext({
|
||||
jobId: subtitleJob.id,
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
durationMs: Date.now() - startedAt,
|
||||
subtitleCount: result.subtitles.length,
|
||||
})}`,
|
||||
context: {
|
||||
jobId: subtitleJob.id,
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
durationMs: Date.now() - startedAt,
|
||||
subtitleCount: result.subtitles.length,
|
||||
quality: result.quality,
|
||||
alignmentEngine: result.alignmentEngine,
|
||||
},
|
||||
});
|
||||
} catch (error: any) {
|
||||
const message = error instanceof Error ? error.message : 'Failed to generate subtitles';
|
||||
const status = deriveSubtitleErrorStatus(message);
|
||||
|
||||
updateSubtitleJob(subtitleJobStore, subtitleJob.id, {
|
||||
status: 'failed',
|
||||
stage: 'failed',
|
||||
message,
|
||||
error: message,
|
||||
});
|
||||
|
||||
logEvent({
|
||||
level: 'error',
|
||||
message: `[subtitle] background job failed ${formatLogContext({
|
||||
jobId: subtitleJob.id,
|
||||
requestId,
|
||||
durationMs: Date.now() - startedAt,
|
||||
fileName: req.file?.originalname,
|
||||
fileSize: req.file?.size,
|
||||
status,
|
||||
})}`,
|
||||
context: {
|
||||
...requestContext,
|
||||
jobId: subtitleJob.id,
|
||||
durationMs: Date.now() - startedAt,
|
||||
status,
|
||||
},
|
||||
details: {
|
||||
error: serializeError(error),
|
||||
headers: summarizeRequestHeaders(req.headers),
|
||||
bodyKeys: Object.keys(req.body || {}).slice(0, 20),
|
||||
},
|
||||
});
|
||||
} finally {
|
||||
if (videoPath && fs.existsSync(videoPath)) {
|
||||
fs.unlinkSync(videoPath);
|
||||
updateSubtitleJob(subtitleJobStore, subtitleJob.id, {
|
||||
filePath: undefined,
|
||||
});
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: '[subtitle] uploaded temp file cleaned',
|
||||
context: {
|
||||
jobId: subtitleJob.id,
|
||||
requestId,
|
||||
videoPath,
|
||||
durationMs: Date.now() - startedAt,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
})();
|
||||
} catch (error: any) {
|
||||
const message = error instanceof Error ? error.message : 'Failed to generate subtitles';
|
||||
const lowerMessage = message.toLowerCase();
|
||||
const status =
|
||||
lowerMessage.includes('target language') ||
|
||||
lowerMessage.includes('unsupported llm provider') ||
|
||||
lowerMessage.includes('_api_key is required') ||
|
||||
lowerMessage.includes('studio project fallback is disabled')
|
||||
? 400
|
||||
: lowerMessage.includes('unauthorized') ||
|
||||
lowerMessage.includes('authentication') ||
|
||||
lowerMessage.includes('auth fail') ||
|
||||
lowerMessage.includes('status 401')
|
||||
? 401
|
||||
: 502;
|
||||
const status = deriveSubtitleErrorStatus(message);
|
||||
|
||||
console.error(
|
||||
`[subtitle] request failed ${formatLogContext({
|
||||
logEvent({
|
||||
level: 'error',
|
||||
message: `[subtitle] request failed ${formatLogContext({
|
||||
requestId,
|
||||
durationMs: Date.now() - startedAt,
|
||||
fileName: req.file?.originalname,
|
||||
fileSize: req.file?.size,
|
||||
status,
|
||||
})}`,
|
||||
serializeError(error),
|
||||
);
|
||||
context: {
|
||||
...requestContext,
|
||||
durationMs: Date.now() - startedAt,
|
||||
status,
|
||||
},
|
||||
details: {
|
||||
error: serializeError(error),
|
||||
headers: summarizeRequestHeaders(req.headers),
|
||||
bodyKeys: Object.keys(req.body || {}).slice(0, 20),
|
||||
},
|
||||
});
|
||||
res.status(status).json({ error: message, requestId });
|
||||
} finally {
|
||||
if (videoPath && fs.existsSync(videoPath)) fs.unlinkSync(videoPath);
|
||||
}
|
||||
});
|
||||
|
||||
app.get('/api/generate-subtitles/:jobId', (req, res) => {
|
||||
pruneExpiredSubtitleJobs(subtitleJobStore, SUBTITLE_JOB_TTL_MS);
|
||||
const subtitleJob = getSubtitleJob(subtitleJobStore, req.params.jobId);
|
||||
|
||||
if (!subtitleJob) {
|
||||
return res.status(404).json({ error: 'Subtitle job not found.' });
|
||||
}
|
||||
|
||||
return res.json(toSubtitleJobResponse(subtitleJob));
|
||||
});
|
||||
|
||||
app.post('/api/export-video', upload.single('video'), async (req, res) => {
|
||||
const tempFiles: string[] = [];
|
||||
try {
|
||||
|
||||
26
src/App.test.tsx
Normal file
26
src/App.test.tsx
Normal file
@ -0,0 +1,26 @@
|
||||
// @vitest-environment jsdom
|
||||
|
||||
import React from 'react';
|
||||
import { cleanup, fireEvent, render, screen } from '@testing-library/react';
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
import App from './App';
|
||||
|
||||
describe('App bilingual UI', () => {
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
});
|
||||
|
||||
it('defaults to Chinese and switches to English from the language toggle', async () => {
|
||||
render(<App />);
|
||||
|
||||
expect(screen.getByLabelText('switch-ui-language-zh')).toHaveAttribute('aria-pressed', 'true');
|
||||
expect(screen.getByText('字幕语言')).toBeInTheDocument();
|
||||
expect(screen.getByText('上传视频')).toBeInTheDocument();
|
||||
|
||||
fireEvent.click(screen.getByLabelText('switch-ui-language-en'));
|
||||
|
||||
expect(screen.getByLabelText('switch-ui-language-en')).toHaveAttribute('aria-pressed', 'true');
|
||||
expect(screen.getByText('Subtitle Language')).toBeInTheDocument();
|
||||
expect(screen.getByText('Upload Video')).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
61
src/App.tsx
61
src/App.tsx
@ -6,16 +6,30 @@
|
||||
import React, { useState } from 'react';
|
||||
import UploadScreen from './components/UploadScreen';
|
||||
import EditorScreen from './components/EditorScreen';
|
||||
import { DEFAULT_SUBTITLE_DEFAULTS, SubtitleDefaults } from './types';
|
||||
import { I18nProvider, useI18n } from './i18n';
|
||||
|
||||
function App() {
|
||||
function AppContent() {
|
||||
const { locale, setLocale, m } = useI18n();
|
||||
const [currentView, setCurrentView] = useState<'upload' | 'editor'>('upload');
|
||||
const [videoFile, setVideoFile] = useState<File | null>(null);
|
||||
const [targetLanguage, setTargetLanguage] = useState<string>('en');
|
||||
const [targetLanguage, setTargetLanguage] = useState<string>('English');
|
||||
const [ttsLanguage, setTtsLanguage] = useState<string>('English');
|
||||
const [trimRange, setTrimRange] = useState<{start: number, end: number} | null>(null);
|
||||
const [subtitleDefaults, setSubtitleDefaults] = useState<SubtitleDefaults>(DEFAULT_SUBTITLE_DEFAULTS);
|
||||
|
||||
const handleVideoUpload = (file: File, lang: string, startTime?: number, endTime?: number) => {
|
||||
const handleVideoUpload = (
|
||||
file: File,
|
||||
lang: string,
|
||||
selectedTtsLanguage: string,
|
||||
initialSubtitleDefaults: SubtitleDefaults,
|
||||
startTime?: number,
|
||||
endTime?: number,
|
||||
) => {
|
||||
setVideoFile(file);
|
||||
setTargetLanguage(lang);
|
||||
setTtsLanguage(selectedTtsLanguage);
|
||||
setSubtitleDefaults(initialSubtitleDefaults);
|
||||
if (startTime !== undefined && endTime !== undefined) {
|
||||
setTrimRange({ start: startTime, end: endTime });
|
||||
} else {
|
||||
@ -26,13 +40,46 @@ function App() {
|
||||
|
||||
return (
|
||||
<div className="min-h-screen bg-gray-50 text-gray-800 font-sans">
|
||||
<div className="mx-auto max-w-6xl px-8 pt-4">
|
||||
<div className="flex justify-end">
|
||||
<div
|
||||
aria-label="app-language-switcher"
|
||||
className="inline-flex rounded-full border border-gray-200 bg-white p-1 shadow-sm"
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
aria-label="switch-ui-language-zh"
|
||||
aria-pressed={locale === 'zh'}
|
||||
onClick={() => setLocale('zh')}
|
||||
className={`px-3 py-1.5 text-sm rounded-full transition-colors ${
|
||||
locale === 'zh' ? 'bg-gray-900 text-white' : 'text-gray-600 hover:bg-gray-100'
|
||||
}`}
|
||||
>
|
||||
{m.app.chinese}
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
aria-label="switch-ui-language-en"
|
||||
aria-pressed={locale === 'en'}
|
||||
onClick={() => setLocale('en')}
|
||||
className={`px-3 py-1.5 text-sm rounded-full transition-colors ${
|
||||
locale === 'en' ? 'bg-gray-900 text-white' : 'text-gray-600 hover:bg-gray-100'
|
||||
}`}
|
||||
>
|
||||
{m.app.english}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{currentView === 'upload' ? (
|
||||
<UploadScreen onUpload={handleVideoUpload} />
|
||||
) : (
|
||||
<EditorScreen
|
||||
videoFile={videoFile}
|
||||
targetLanguage={targetLanguage}
|
||||
ttsLanguage={ttsLanguage}
|
||||
trimRange={trimRange}
|
||||
initialSubtitleDefaults={subtitleDefaults}
|
||||
onBack={() => setCurrentView('upload')}
|
||||
/>
|
||||
)}
|
||||
@ -40,4 +87,12 @@ function App() {
|
||||
);
|
||||
}
|
||||
|
||||
function App() {
|
||||
return (
|
||||
<I18nProvider initialLocale="zh">
|
||||
<AppContent />
|
||||
</I18nProvider>
|
||||
);
|
||||
}
|
||||
|
||||
export default App;
|
||||
|
||||
@ -4,6 +4,13 @@ import React from 'react';
|
||||
import { cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import EditorScreen from './EditorScreen';
|
||||
import { Subtitle } from '../types';
|
||||
import { I18nProvider } from '../i18n';
|
||||
|
||||
const defaultSubtitleDefaults = {
|
||||
fontSize: 24,
|
||||
bottomOffsetPercent: 10,
|
||||
};
|
||||
|
||||
const { generateSubtitlePipelineMock, generateTTSMock } = vi.hoisted(() => ({
|
||||
generateSubtitlePipelineMock: vi.fn(),
|
||||
@ -19,6 +26,58 @@ vi.mock('../services/ttsService', () => ({
|
||||
}));
|
||||
|
||||
describe('EditorScreen', () => {
|
||||
const renderEditorScreen = (props: React.ComponentProps<typeof EditorScreen>) =>
|
||||
render(
|
||||
<I18nProvider initialLocale="en">
|
||||
<EditorScreen {...props} />
|
||||
</I18nProvider>,
|
||||
);
|
||||
|
||||
const styledSubtitles: Subtitle[] = [
|
||||
{
|
||||
id: 'sub-1',
|
||||
startTime: 0,
|
||||
endTime: 2,
|
||||
originalText: 'Original one',
|
||||
translatedText: 'Styled first subtitle',
|
||||
speaker: 'Young Woman',
|
||||
voiceId: 'voice-1',
|
||||
textStyle: {
|
||||
fontFamily: 'Arial',
|
||||
fontSize: 24,
|
||||
color: '#ffffff',
|
||||
backgroundColor: 'transparent',
|
||||
alignment: 'center',
|
||||
isBold: false,
|
||||
isItalic: false,
|
||||
isUnderline: false,
|
||||
strokeColor: '#000000',
|
||||
strokeWidth: 2,
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'sub-2',
|
||||
startTime: 2,
|
||||
endTime: 4,
|
||||
originalText: 'Original two',
|
||||
translatedText: 'Styled second subtitle',
|
||||
speaker: 'Young Man',
|
||||
voiceId: 'voice-2',
|
||||
textStyle: {
|
||||
fontFamily: 'Roboto',
|
||||
fontSize: 20,
|
||||
color: '#00ff00',
|
||||
backgroundColor: 'transparent',
|
||||
alignment: 'center',
|
||||
isBold: false,
|
||||
isItalic: false,
|
||||
isUnderline: false,
|
||||
strokeColor: '#000000',
|
||||
strokeWidth: 2,
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
});
|
||||
@ -50,28 +109,26 @@ describe('EditorScreen', () => {
|
||||
});
|
||||
|
||||
it('shows a low-precision notice for fallback subtitle results', async () => {
|
||||
render(
|
||||
<EditorScreen
|
||||
videoFile={new File(['video'], 'clip.mp4', { type: 'video/mp4' })}
|
||||
targetLanguage="en"
|
||||
trimRange={null}
|
||||
onBack={() => {}}
|
||||
/>,
|
||||
);
|
||||
renderEditorScreen({
|
||||
videoFile: new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
targetLanguage: 'en',
|
||||
trimRange: null,
|
||||
initialSubtitleDefaults: defaultSubtitleDefaults,
|
||||
onBack: () => {},
|
||||
});
|
||||
|
||||
expect(screen.getAllByLabelText(/llm/i)[0]).toHaveValue('doubao');
|
||||
expect(await screen.findByText(/low-precision/i)).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('regenerates subtitles with the selected llm provider', async () => {
|
||||
render(
|
||||
<EditorScreen
|
||||
videoFile={new File(['video'], 'clip.mp4', { type: 'video/mp4' })}
|
||||
targetLanguage="en"
|
||||
trimRange={null}
|
||||
onBack={() => {}}
|
||||
/>,
|
||||
);
|
||||
renderEditorScreen({
|
||||
videoFile: new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
targetLanguage: 'en',
|
||||
trimRange: null,
|
||||
initialSubtitleDefaults: defaultSubtitleDefaults,
|
||||
onBack: () => {},
|
||||
});
|
||||
|
||||
await waitFor(() =>
|
||||
expect(generateSubtitlePipelineMock).toHaveBeenCalledWith(
|
||||
@ -79,6 +136,9 @@ describe('EditorScreen', () => {
|
||||
'en',
|
||||
'doubao',
|
||||
null,
|
||||
undefined,
|
||||
expect.any(Function),
|
||||
undefined,
|
||||
),
|
||||
);
|
||||
|
||||
@ -92,24 +152,182 @@ describe('EditorScreen', () => {
|
||||
'en',
|
||||
'gemini',
|
||||
null,
|
||||
undefined,
|
||||
expect.any(Function),
|
||||
undefined,
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
it('only auto-generates subtitles once in StrictMode', async () => {
|
||||
render(
|
||||
<React.StrictMode>
|
||||
<EditorScreen
|
||||
videoFile={new File(['video'], 'clip.mp4', { type: 'video/mp4' })}
|
||||
targetLanguage="en"
|
||||
trimRange={null}
|
||||
onBack={() => {}}
|
||||
/>
|
||||
</React.StrictMode>,
|
||||
<I18nProvider initialLocale="en">
|
||||
<React.StrictMode>
|
||||
<EditorScreen
|
||||
videoFile={new File(['video'], 'clip.mp4', { type: 'video/mp4' })}
|
||||
targetLanguage="en"
|
||||
trimRange={null}
|
||||
initialSubtitleDefaults={defaultSubtitleDefaults}
|
||||
onBack={() => {}}
|
||||
/>
|
||||
</React.StrictMode>
|
||||
</I18nProvider>,
|
||||
);
|
||||
|
||||
await waitFor(() => expect(generateSubtitlePipelineMock).toHaveBeenCalled());
|
||||
|
||||
expect(generateSubtitlePipelineMock).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('shows subtitle generation progress updates', async () => {
|
||||
generateSubtitlePipelineMock.mockImplementation(
|
||||
(_file, _lang, _provider, _trim, _fetch, onProgress) =>
|
||||
new Promise(() => {
|
||||
onProgress?.({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'running',
|
||||
stage: 'calling_provider',
|
||||
progress: 70,
|
||||
message: 'Calling provider',
|
||||
});
|
||||
}),
|
||||
);
|
||||
|
||||
renderEditorScreen({
|
||||
videoFile: new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
targetLanguage: 'en',
|
||||
trimRange: null,
|
||||
initialSubtitleDefaults: defaultSubtitleDefaults,
|
||||
onBack: () => {},
|
||||
});
|
||||
|
||||
expect(await screen.findByText(/70%/i)).toBeInTheDocument();
|
||||
expect(screen.getByText(/Calling provider/i)).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('shows the active subtitle speaker and updates only the selected subtitle style by default', async () => {
|
||||
generateSubtitlePipelineMock.mockResolvedValue({
|
||||
subtitles: styledSubtitles,
|
||||
speakers: [],
|
||||
quality: 'full',
|
||||
});
|
||||
|
||||
renderEditorScreen({
|
||||
videoFile: new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
targetLanguage: 'en',
|
||||
trimRange: null,
|
||||
initialSubtitleDefaults: defaultSubtitleDefaults,
|
||||
onBack: () => {},
|
||||
});
|
||||
|
||||
expect(await screen.findByText('Young Woman')).toBeInTheDocument();
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /toggle bold/i }));
|
||||
|
||||
const overlay = screen.getByTestId('video-subtitle-overlay');
|
||||
expect(overlay).toHaveStyle({ fontWeight: '700' });
|
||||
|
||||
fireEvent.click(screen.getByDisplayValue('Styled second subtitle'));
|
||||
expect(await screen.findByText('Young Man')).toBeInTheDocument();
|
||||
|
||||
const colorInput = screen.getByLabelText(/subtitle color/i);
|
||||
fireEvent.change(colorInput, { target: { value: '#ff0000' } });
|
||||
|
||||
fireEvent.click(screen.getByDisplayValue('Styled first subtitle'));
|
||||
expect(screen.getByLabelText(/subtitle color/i)).toHaveValue('#ffffff');
|
||||
});
|
||||
|
||||
it('applies style changes to every subtitle when apply to all subtitles is enabled', async () => {
|
||||
generateSubtitlePipelineMock.mockResolvedValue({
|
||||
subtitles: styledSubtitles,
|
||||
speakers: [],
|
||||
quality: 'full',
|
||||
});
|
||||
|
||||
renderEditorScreen({
|
||||
videoFile: new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
targetLanguage: 'en',
|
||||
trimRange: null,
|
||||
initialSubtitleDefaults: defaultSubtitleDefaults,
|
||||
onBack: () => {},
|
||||
});
|
||||
|
||||
await screen.findByText('Young Woman');
|
||||
|
||||
fireEvent.click(screen.getByLabelText(/apply style changes to all subtitles/i));
|
||||
fireEvent.change(screen.getByLabelText(/subtitle color/i), {
|
||||
target: { value: '#ff0000' },
|
||||
});
|
||||
|
||||
fireEvent.click(screen.getByDisplayValue('Styled second subtitle'));
|
||||
expect(screen.getByLabelText(/subtitle color/i)).toHaveValue('#ff0000');
|
||||
});
|
||||
|
||||
it('initializes the overlay from upload subtitle defaults', async () => {
|
||||
generateSubtitlePipelineMock.mockResolvedValue({
|
||||
subtitles: [
|
||||
{
|
||||
id: 'sub-default',
|
||||
startTime: 0,
|
||||
endTime: 2,
|
||||
originalText: 'Original',
|
||||
translatedText: 'Uses upload defaults',
|
||||
speaker: 'Narrator',
|
||||
voiceId: 'voice-1',
|
||||
},
|
||||
],
|
||||
speakers: [],
|
||||
quality: 'full',
|
||||
});
|
||||
|
||||
renderEditorScreen({
|
||||
videoFile: new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
targetLanguage: 'en',
|
||||
trimRange: null,
|
||||
initialSubtitleDefaults: { fontSize: 32, bottomOffsetPercent: 18 },
|
||||
onBack: () => {},
|
||||
});
|
||||
|
||||
const overlay = await screen.findByTestId('video-subtitle-overlay');
|
||||
expect(overlay).toHaveStyle({ fontSize: '32px' });
|
||||
expect(overlay.parentElement).toHaveStyle({ bottom: '18%' });
|
||||
});
|
||||
|
||||
it('uses ttsText when generating dubbing audio', async () => {
|
||||
generateSubtitlePipelineMock.mockResolvedValue({
|
||||
subtitles: [
|
||||
{
|
||||
id: 'sub-tts',
|
||||
startTime: 0,
|
||||
endTime: 2,
|
||||
originalText: 'Original',
|
||||
translatedText: 'English subtitle',
|
||||
ttsText: 'Bonjour a tous',
|
||||
ttsLanguage: 'fr',
|
||||
speaker: 'Narrator',
|
||||
voiceId: 'voice-1',
|
||||
},
|
||||
],
|
||||
speakers: [],
|
||||
quality: 'full',
|
||||
});
|
||||
generateTTSMock.mockResolvedValue('data:audio/mp3;base64,AAA');
|
||||
|
||||
renderEditorScreen({
|
||||
videoFile: new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
targetLanguage: 'en',
|
||||
trimRange: null,
|
||||
initialSubtitleDefaults: defaultSubtitleDefaults,
|
||||
onBack: () => {},
|
||||
});
|
||||
|
||||
await screen.findAllByDisplayValue('English subtitle');
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /generate dubbing/i }));
|
||||
|
||||
await waitFor(() =>
|
||||
expect(generateTTSMock).toHaveBeenCalledWith('Bonjour a tous', 'voice-1'),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@ -3,13 +3,61 @@ import axios from 'axios';
|
||||
import { ChevronLeft, Play, Pause, Volume2, Settings, Download, Save, LayoutTemplate, Type, Image as ImageIcon, Music, Scissors, Plus, Trash2, Maximize2, Loader2 } from 'lucide-react';
|
||||
import VoiceMarketModal from './VoiceMarketModal';
|
||||
import ExportModal from './ExportModal';
|
||||
import { LlmProvider, PipelineQuality, Subtitle, TextStyles } from '../types';
|
||||
import { DEFAULT_SUBTITLE_DEFAULTS, LlmProvider, PipelineQuality, Subtitle, SubtitleDefaults, SubtitleGenerationProgress, TextStyles } from '../types';
|
||||
import { generateSubtitlePipeline } from '../services/subtitleService';
|
||||
import { generateTTS } from '../services/ttsService';
|
||||
import { MINIMAX_VOICES } from '../voices';
|
||||
import { apiUrl } from '../lib/apiBasePath';
|
||||
import { useI18n } from '../i18n';
|
||||
|
||||
export default function EditorScreen({ videoFile, targetLanguage, trimRange, onBack }: { videoFile: File | null; targetLanguage: string; trimRange?: {start: number, end: number} | null; onBack: () => void }) {
|
||||
const DEFAULT_TEXT_STYLE: TextStyles = {
|
||||
fontFamily: 'MiSans-Late',
|
||||
fontSize: 24,
|
||||
color: '#FFFFFF',
|
||||
backgroundColor: 'transparent',
|
||||
strokeColor: '#000000',
|
||||
strokeWidth: 2,
|
||||
alignment: 'center',
|
||||
isBold: false,
|
||||
isItalic: false,
|
||||
isUnderline: false,
|
||||
};
|
||||
|
||||
const FONT_SIZE_PRESETS = {
|
||||
small: 20,
|
||||
normal: 24,
|
||||
large: 30,
|
||||
} as const;
|
||||
|
||||
const normalizeTextStyle = (textStyle?: Partial<TextStyles>): TextStyles => ({
|
||||
...DEFAULT_TEXT_STYLE,
|
||||
...textStyle,
|
||||
});
|
||||
|
||||
const normalizeSubtitle = (subtitle: Subtitle, initialSubtitleDefaults: SubtitleDefaults): Subtitle => ({
|
||||
...subtitle,
|
||||
textStyle: normalizeTextStyle({
|
||||
fontSize: initialSubtitleDefaults.fontSize,
|
||||
...subtitle.textStyle,
|
||||
}),
|
||||
});
|
||||
|
||||
export default function EditorScreen({
|
||||
videoFile,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
trimRange,
|
||||
initialSubtitleDefaults = DEFAULT_SUBTITLE_DEFAULTS,
|
||||
onBack,
|
||||
}: {
|
||||
videoFile: File | null;
|
||||
targetLanguage: string;
|
||||
ttsLanguage?: string;
|
||||
trimRange?: {start: number, end: number} | null;
|
||||
initialSubtitleDefaults?: SubtitleDefaults;
|
||||
onBack: () => void;
|
||||
}) {
|
||||
const { m } = useI18n();
|
||||
const [subtitles, setSubtitles] = useState<Subtitle[]>([]);
|
||||
const [activeSubtitleId, setActiveSubtitleId] = useState<string>('');
|
||||
const [showVoiceMarket, setShowVoiceMarket] = useState(false);
|
||||
@ -21,6 +69,8 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
const [generationError, setGenerationError] = useState<string | null>(null);
|
||||
const [subtitleQuality, setSubtitleQuality] = useState<PipelineQuality>('fallback');
|
||||
const [llmProvider, setLlmProvider] = useState<LlmProvider>('doubao');
|
||||
const [generationProgress, setGenerationProgress] = useState<SubtitleGenerationProgress | null>(null);
|
||||
const [applyToAllSubtitles, setApplyToAllSubtitles] = useState(false);
|
||||
|
||||
// Video Player State
|
||||
const videoRef = useRef<HTMLVideoElement>(null);
|
||||
@ -84,12 +134,25 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
if (!videoFile) return;
|
||||
setIsGenerating(true);
|
||||
setGenerationError(null);
|
||||
setGenerationProgress({
|
||||
jobId: 'pending',
|
||||
requestId: 'pending',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
message: m.editor.aiAnalyzing,
|
||||
});
|
||||
try {
|
||||
const pipelineResult = await generateSubtitlePipeline(
|
||||
videoFile,
|
||||
targetLanguage,
|
||||
llmProvider,
|
||||
trimRange,
|
||||
undefined,
|
||||
(progress) => {
|
||||
setGenerationProgress(progress);
|
||||
},
|
||||
ttsLanguage,
|
||||
);
|
||||
const generatedSubs = pipelineResult.subtitles;
|
||||
setSubtitleQuality(pipelineResult.quality);
|
||||
@ -105,15 +168,17 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
}));
|
||||
}
|
||||
|
||||
setSubtitles(adjustedSubs);
|
||||
if (adjustedSubs.length > 0) {
|
||||
setActiveSubtitleId(adjustedSubs[0].id);
|
||||
const normalizedSubs = adjustedSubs.map((subtitle) => normalizeSubtitle(subtitle, initialSubtitleDefaults));
|
||||
|
||||
setSubtitles(normalizedSubs);
|
||||
if (normalizedSubs.length > 0) {
|
||||
setActiveSubtitleId(normalizedSubs[0].id);
|
||||
}
|
||||
} catch (err: any) {
|
||||
console.error("Failed to generate subtitles:", err);
|
||||
setSubtitleQuality('fallback');
|
||||
|
||||
let errorMessage = "Failed to generate subtitles. Please try again or check your API key.";
|
||||
let errorMessage = m.editor.failedToGenerateSubtitles;
|
||||
const errString = err instanceof Error ? err.message : JSON.stringify(err);
|
||||
|
||||
if (
|
||||
@ -123,7 +188,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
err?.status === 429 ||
|
||||
err?.error?.code === 429
|
||||
) {
|
||||
errorMessage = "You have exceeded your Volcengine API quota. Please check your plan and billing details.";
|
||||
errorMessage = m.editor.exceededQuota;
|
||||
} else if (err instanceof Error) {
|
||||
errorMessage = err.message;
|
||||
}
|
||||
@ -133,8 +198,9 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
setActiveSubtitleId('');
|
||||
} finally {
|
||||
setIsGenerating(false);
|
||||
setGenerationProgress(null);
|
||||
}
|
||||
}, [videoFile, targetLanguage, trimRange, llmProvider]);
|
||||
}, [videoFile, targetLanguage, ttsLanguage, trimRange, llmProvider, initialSubtitleDefaults, m.editor.aiAnalyzing, m.editor.exceededQuota, m.editor.failedToGenerateSubtitles]);
|
||||
|
||||
// Generate subtitles on mount
|
||||
useEffect(() => {
|
||||
@ -142,6 +208,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
fileName: videoFile?.name || '',
|
||||
fileSize: videoFile?.size || 0,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
trimRange,
|
||||
llmProvider,
|
||||
});
|
||||
@ -152,18 +219,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
|
||||
autoGenerationKeyRef.current = autoGenerationKey;
|
||||
fetchSubtitles();
|
||||
}, [fetchSubtitles, videoFile, targetLanguage, trimRange, llmProvider]);
|
||||
|
||||
const [textStyles, setTextStyles] = useState<TextStyles>({
|
||||
fontFamily: 'MiSans-Late',
|
||||
fontSize: 24,
|
||||
color: '#FFFFFF',
|
||||
backgroundColor: 'transparent',
|
||||
alignment: 'center',
|
||||
isBold: false,
|
||||
isItalic: false,
|
||||
isUnderline: false,
|
||||
});
|
||||
}, [fetchSubtitles, videoFile, targetLanguage, ttsLanguage, trimRange, llmProvider]);
|
||||
|
||||
const togglePlay = async () => {
|
||||
if (videoRef.current) {
|
||||
@ -228,7 +284,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
if (sub.audioUrl) continue;
|
||||
|
||||
try {
|
||||
const textToSpeak = sub.translatedText || sub.text;
|
||||
const textToSpeak = sub.ttsText || sub.translatedText || sub.originalText;
|
||||
const audioUrl = await generateTTS(textToSpeak, sub.voiceId);
|
||||
updatedSubtitles[i] = { ...sub, audioUrl };
|
||||
// Update state incrementally so user sees progress
|
||||
@ -415,12 +471,60 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
|
||||
const displayDuration = trimRange ? trimRange.end - trimRange.start : duration;
|
||||
const displayCurrentTime = trimRange ? Math.max(0, currentTime - trimRange.start) : currentTime;
|
||||
const activeSubtitle = useMemo(
|
||||
() => subtitles.find((subtitle) => subtitle.id === activeSubtitleId) ?? null,
|
||||
[subtitles, activeSubtitleId],
|
||||
);
|
||||
const activeSubtitleStyle = activeSubtitle?.textStyle ?? DEFAULT_TEXT_STYLE;
|
||||
const currentSubtitle = useMemo(
|
||||
() => subtitles.find((subtitle) => displayCurrentTime >= subtitle.startTime && displayCurrentTime <= subtitle.endTime) ?? null,
|
||||
[subtitles, displayCurrentTime],
|
||||
);
|
||||
const currentSubtitleStyle = currentSubtitle?.textStyle ?? DEFAULT_TEXT_STYLE;
|
||||
const currentSubtitleText = currentSubtitle?.translatedText || '';
|
||||
const currentFontSizePreset = (Object.entries(FONT_SIZE_PRESETS).find(
|
||||
([, fontSize]) => fontSize === activeSubtitleStyle.fontSize,
|
||||
)?.[0] ?? 'normal') as keyof typeof FONT_SIZE_PRESETS;
|
||||
const exportTextStyles = subtitles[0]?.textStyle ?? DEFAULT_TEXT_STYLE;
|
||||
const fontSizePresetLabels: Record<keyof typeof FONT_SIZE_PRESETS, string> = {
|
||||
small: m.editor.small,
|
||||
normal: m.editor.normal,
|
||||
large: m.editor.large,
|
||||
};
|
||||
const alignmentOptions: Array<{ value: TextStyles['alignment']; label: string }> = [
|
||||
{ value: 'left', label: m.editor.alignLeft },
|
||||
{ value: 'center', label: m.editor.alignCenter },
|
||||
{ value: 'right', label: m.editor.alignRight },
|
||||
];
|
||||
|
||||
// Calculate playhead position percentage
|
||||
const playheadPercent = displayDuration > 0 ? (displayCurrentTime / displayDuration) * 100 : 0;
|
||||
|
||||
// Get current subtitle text
|
||||
const currentSubtitleText = subtitles.find(s => displayCurrentTime >= s.startTime && displayCurrentTime <= s.endTime)?.translatedText || '';
|
||||
const updateSubtitleStyles = useCallback((updates: Partial<TextStyles>) => {
|
||||
if (!activeSubtitleId && !applyToAllSubtitles) {
|
||||
return;
|
||||
}
|
||||
|
||||
setSubtitles((previousSubtitles) =>
|
||||
previousSubtitles.map((subtitle) => {
|
||||
if (!applyToAllSubtitles && subtitle.id !== activeSubtitleId) {
|
||||
return subtitle;
|
||||
}
|
||||
|
||||
return {
|
||||
...subtitle,
|
||||
textStyle: {
|
||||
...normalizeTextStyle(subtitle.textStyle),
|
||||
...updates,
|
||||
},
|
||||
};
|
||||
}),
|
||||
);
|
||||
}, [activeSubtitleId, applyToAllSubtitles]);
|
||||
|
||||
const activeSpeakerInitial = activeSubtitle?.speaker?.trim().charAt(0).toUpperCase() || 'S';
|
||||
const activeSubtitleContext = activeSubtitle?.translatedText || activeSubtitle?.originalText || '';
|
||||
const subtitleTextShadow = `${currentSubtitleStyle.strokeWidth}px 0 ${currentSubtitleStyle.strokeColor}, -${currentSubtitleStyle.strokeWidth}px 0 ${currentSubtitleStyle.strokeColor}, 0 ${currentSubtitleStyle.strokeWidth}px ${currentSubtitleStyle.strokeColor}, 0 -${currentSubtitleStyle.strokeWidth}px ${currentSubtitleStyle.strokeColor}`;
|
||||
|
||||
return (
|
||||
<div className="h-[100dvh] flex flex-col bg-white overflow-hidden">
|
||||
@ -432,7 +536,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
</button>
|
||||
<div className="flex items-center gap-2">
|
||||
<div className="w-6 h-6 bg-orange-500 rounded flex items-center justify-center text-white font-bold text-xs">M</div>
|
||||
<span className="font-medium text-sm">Translate 1.0</span>
|
||||
<span className="font-medium text-sm">{m.app.productName}</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -449,18 +553,18 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
className="flex items-center gap-2 px-3 py-1.5 bg-red-50 text-red-600 rounded-md text-sm font-medium hover:bg-red-100"
|
||||
>
|
||||
<Play className="w-4 h-4 fill-current" />
|
||||
Watch Video
|
||||
{m.editor.watchVideo}
|
||||
</button>
|
||||
<button className="flex items-center gap-2 px-3 py-1.5 text-gray-600 hover:bg-gray-100 rounded-md text-sm font-medium">
|
||||
<Save className="w-4 h-4" />
|
||||
Save Editing
|
||||
{m.editor.saveEditing}
|
||||
</button>
|
||||
<button
|
||||
onClick={() => setShowExportModal(true)}
|
||||
className="flex items-center gap-2 px-4 py-1.5 bg-[#52c41a] text-white rounded-md text-sm font-medium hover:bg-[#46a616]"
|
||||
>
|
||||
<Download className="w-4 h-4" />
|
||||
Export
|
||||
{m.editor.export}
|
||||
</button>
|
||||
</div>
|
||||
</header>
|
||||
@ -471,19 +575,19 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
<div className="w-80 border-r border-gray-200 flex flex-col bg-gray-50 shrink-0">
|
||||
<div className="p-4 border-b border-gray-200 bg-white shrink-0">
|
||||
<div className="flex bg-gray-100 p-1 rounded-md mb-4">
|
||||
<button className="flex-1 py-1.5 bg-white shadow-sm rounded text-sm font-medium">AI Dub</button>
|
||||
<button className="flex-1 py-1.5 text-gray-600 text-sm font-medium">Voice Clone</button>
|
||||
<button className="flex-1 py-1.5 bg-white shadow-sm rounded text-sm font-medium">{m.editor.aiDub}</button>
|
||||
<button className="flex-1 py-1.5 text-gray-600 text-sm font-medium">{m.editor.voiceClone}</button>
|
||||
</div>
|
||||
<p className="text-xs text-gray-500 mb-4">
|
||||
Tip: After modifying the subtitle text, you need to regenerate the dubbing to take effect!
|
||||
{m.editor.tip}
|
||||
</p>
|
||||
<div className="mb-4">
|
||||
<label htmlFor="llm-provider" className="block text-xs font-medium text-gray-500 mb-1">
|
||||
LLM
|
||||
{m.editor.llm}
|
||||
</label>
|
||||
<select
|
||||
id="llm-provider"
|
||||
aria-label="LLM"
|
||||
aria-label={m.editor.llmProvider}
|
||||
value={llmProvider}
|
||||
onChange={(e) => setLlmProvider(e.target.value as LlmProvider)}
|
||||
className="w-full border border-gray-300 rounded-md px-3 py-2 text-sm focus:outline-none focus:border-blue-500 bg-white"
|
||||
@ -498,7 +602,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
className="w-full py-2 bg-blue-500 hover:bg-blue-600 disabled:bg-blue-300 text-white rounded-md text-sm font-medium transition-colors flex items-center justify-center gap-2"
|
||||
>
|
||||
{isDubbingGenerating && <Loader2 className="w-4 h-4 animate-spin" />}
|
||||
{isSeparating ? 'Separating Vocals...' : isDubbingGenerating ? 'Generating TTS...' : 'Generate Dubbing'}
|
||||
{isSeparating ? m.editor.separatingVocals : isDubbingGenerating ? m.editor.generatingTts : m.editor.generateDubbing}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
@ -506,8 +610,22 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
{isGenerating ? (
|
||||
<div className="flex flex-col items-center justify-center h-full text-gray-500 space-y-4">
|
||||
<Loader2 className="w-8 h-8 animate-spin text-blue-500" />
|
||||
<p className="text-sm font-medium">AI is analyzing and translating...</p>
|
||||
<p className="text-xs text-center px-4">This may take a minute depending on the video length.</p>
|
||||
<p className="text-sm font-medium">
|
||||
{generationProgress?.message || m.editor.aiAnalyzing}
|
||||
</p>
|
||||
<div className="w-56 space-y-2">
|
||||
<div className="h-2 w-full overflow-hidden rounded-full bg-gray-200">
|
||||
<div
|
||||
className="h-full rounded-full bg-blue-500 transition-all duration-500"
|
||||
style={{ width: `${generationProgress?.progress ?? 5}%` }}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex items-center justify-between text-xs text-gray-500">
|
||||
<span>{generationProgress?.stage || m.editor.qualityQueued}</span>
|
||||
<span>{generationProgress?.progress ?? 5}%</span>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-xs text-center px-4">{m.editor.thisMayTakeMinutes}</p>
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
@ -518,13 +636,13 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
onClick={() => fetchSubtitles()}
|
||||
className="px-3 py-1.5 bg-red-100 hover:bg-red-200 text-red-700 rounded font-medium transition-colors self-start"
|
||||
>
|
||||
Retry Generation
|
||||
{m.editor.retryGeneration}
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
{!generationError && subtitleQuality === 'fallback' && (
|
||||
<div className="p-3 mb-2 text-xs text-amber-700 bg-amber-50 border border-amber-200 rounded-md">
|
||||
Low-precision subtitle timing is active for this generation. You can still edit subtitles before dubbing.
|
||||
{m.editor.lowPrecision}
|
||||
</div>
|
||||
)}
|
||||
{subtitles.map((sub, index) => (
|
||||
@ -589,7 +707,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
<div className="w-4 h-4 rounded-full bg-orange-200 overflow-hidden">
|
||||
<img src={`https://api.dicebear.com/7.x/avataaars/svg?seed=${sub.voiceId}`} alt="avatar" />
|
||||
</div>
|
||||
{MINIMAX_VOICES.find(v => v.id === sub.voiceId)?.name || 'Select Voice'}
|
||||
{MINIMAX_VOICES.find(v => v.id === sub.voiceId)?.name || m.editor.selectVoice}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
@ -640,7 +758,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
) : (
|
||||
<div className="text-gray-500 flex flex-col items-center justify-center h-full w-full">
|
||||
<ImageIcon className="w-12 h-12 mb-2 opacity-50" />
|
||||
<p>No video loaded</p>
|
||||
<p>{m.editor.noVideoLoaded}</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
@ -660,18 +778,24 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
{/* Subtitle Overlay */}
|
||||
{currentSubtitleText && (
|
||||
<div
|
||||
className="absolute bottom-[10%] left-1/2 -translate-x-1/2 flex justify-center pointer-events-none px-4"
|
||||
style={{ width: renderedVideoWidth }}
|
||||
className="absolute left-1/2 -translate-x-1/2 flex justify-center pointer-events-none px-4"
|
||||
style={{
|
||||
width: renderedVideoWidth,
|
||||
bottom: `${initialSubtitleDefaults.bottomOffsetPercent}%`,
|
||||
}}
|
||||
>
|
||||
<p
|
||||
data-testid="video-subtitle-overlay"
|
||||
className="text-white text-base md:text-lg font-bold drop-shadow-md break-words whitespace-pre-wrap text-center max-w-[90%]"
|
||||
style={{
|
||||
fontFamily: textStyles.fontFamily,
|
||||
color: textStyles.color,
|
||||
textAlign: textStyles.alignment,
|
||||
fontWeight: textStyles.isBold ? 'bold' : 'normal',
|
||||
fontStyle: textStyles.isItalic ? 'italic' : 'normal',
|
||||
textDecoration: textStyles.isUnderline ? 'underline' : 'none',
|
||||
fontFamily: currentSubtitleStyle.fontFamily,
|
||||
fontSize: `${currentSubtitleStyle.fontSize}px`,
|
||||
color: currentSubtitleStyle.color,
|
||||
textAlign: currentSubtitleStyle.alignment,
|
||||
fontWeight: currentSubtitleStyle.isBold ? 700 : 400,
|
||||
fontStyle: currentSubtitleStyle.isItalic ? 'italic' : 'normal',
|
||||
textDecoration: currentSubtitleStyle.isUnderline ? 'underline' : 'none',
|
||||
textShadow: subtitleTextShadow,
|
||||
}}
|
||||
>
|
||||
{currentSubtitleText}
|
||||
@ -695,7 +819,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
{bgmUrl && (
|
||||
<div className="flex items-center gap-2 bg-gray-100 px-2 py-1 rounded-md">
|
||||
<Music className="w-4 h-4 text-green-600" />
|
||||
<span className="text-xs font-medium text-gray-600">BGM</span>
|
||||
<span className="text-xs font-medium text-gray-600">{m.editor.bgm}</span>
|
||||
<input
|
||||
type="range"
|
||||
min="0"
|
||||
@ -718,98 +842,144 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
<div className="w-72 border-l border-gray-200 bg-white flex flex-col shrink-0 overflow-y-auto">
|
||||
<div className="p-4 border-b border-gray-200">
|
||||
<label className="flex items-center gap-2 text-sm text-gray-700 cursor-pointer">
|
||||
<input type="checkbox" className="rounded text-blue-600 focus:ring-blue-500" defaultChecked />
|
||||
Apply to all subtitles
|
||||
<input
|
||||
type="checkbox"
|
||||
aria-label={m.editor.applyAllAria}
|
||||
className="rounded text-blue-600 focus:ring-blue-500"
|
||||
checked={applyToAllSubtitles}
|
||||
onChange={(e) => setApplyToAllSubtitles(e.target.checked)}
|
||||
/>
|
||||
{m.editor.applyAll}
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div className="p-4 border-b border-gray-200">
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
<div className="w-6 h-6 rounded-full bg-green-100 flex items-center justify-center text-green-600 text-xs font-bold">
|
||||
W
|
||||
{activeSubtitle ? (
|
||||
<>
|
||||
<div className="p-4 border-b border-gray-200">
|
||||
<div className="flex items-center gap-2 mb-3">
|
||||
<div className="w-6 h-6 rounded-full bg-green-100 flex items-center justify-center text-green-600 text-xs font-bold">
|
||||
{activeSpeakerInitial}
|
||||
</div>
|
||||
<span className="text-sm font-medium text-gray-700">{activeSubtitle.speaker}</span>
|
||||
</div>
|
||||
<input
|
||||
type="text"
|
||||
aria-label={m.editor.activeSubtitleContext}
|
||||
value={activeSubtitleContext}
|
||||
className="w-full border border-gray-300 rounded-md px-3 py-1.5 text-sm focus:outline-none focus:border-blue-500"
|
||||
readOnly
|
||||
/>
|
||||
</div>
|
||||
<span className="text-sm font-medium text-gray-700">Wife</span>
|
||||
</div>
|
||||
<input
|
||||
type="text"
|
||||
value="Husband"
|
||||
className="w-full border border-gray-300 rounded-md px-3 py-1.5 text-sm focus:outline-none focus:border-blue-500"
|
||||
readOnly
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="p-4">
|
||||
<h3 className="text-sm font-medium text-gray-700 mb-3">Text Styles</h3>
|
||||
<div className="p-4">
|
||||
<h3 className="text-sm font-medium text-gray-700 mb-3">{m.editor.textStyles}</h3>
|
||||
|
||||
{/* Style Presets Grid */}
|
||||
<div className="grid grid-cols-4 gap-2 mb-6">
|
||||
{['T', 'T', 'T', 'T', 'T', 'T', 'T', 'T', 'T', 'T', 'T', 'T'].map((item, i) => (
|
||||
<button key={i} className={`aspect-square rounded border flex items-center justify-center text-lg font-bold ${i === 8 ? 'border-blue-500 bg-blue-50 text-blue-600' : 'border-gray-200 bg-gray-50 text-gray-700 hover:bg-gray-100'}`}>
|
||||
{item}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<div className="grid grid-cols-4 gap-2 mb-6">
|
||||
{alignmentOptions.map((alignment) => (
|
||||
<button
|
||||
key={alignment.value}
|
||||
type="button"
|
||||
aria-label={alignment.label}
|
||||
className={`aspect-square rounded border flex items-center justify-center text-lg font-bold ${
|
||||
activeSubtitleStyle.alignment === alignment.value
|
||||
? 'border-blue-500 bg-blue-50 text-blue-600'
|
||||
: 'border-gray-200 bg-gray-50 text-gray-700 hover:bg-gray-100'
|
||||
}`}
|
||||
onClick={() => updateSubtitleStyles({ alignment: alignment.value })}
|
||||
>
|
||||
T
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
|
||||
{/* Font Family */}
|
||||
<div className="mb-4">
|
||||
<select
|
||||
className="w-full border border-gray-300 rounded-md px-3 py-2 text-sm focus:outline-none focus:border-blue-500"
|
||||
value={textStyles.fontFamily}
|
||||
onChange={(e) => setTextStyles({...textStyles, fontFamily: e.target.value})}
|
||||
>
|
||||
<option value="MiSans-Late">MiSans-Late</option>
|
||||
<option value="Arial">Arial</option>
|
||||
<option value="Roboto">Roboto</option>
|
||||
<option value="serif">Serif</option>
|
||||
</select>
|
||||
</div>
|
||||
<div className="mb-4">
|
||||
<select
|
||||
aria-label={m.editor.subtitleFontFamily}
|
||||
className="w-full border border-gray-300 rounded-md px-3 py-2 text-sm focus:outline-none focus:border-blue-500"
|
||||
value={activeSubtitleStyle.fontFamily}
|
||||
onChange={(e) => updateSubtitleStyles({ fontFamily: e.target.value })}
|
||||
>
|
||||
<option value="MiSans-Late">MiSans-Late</option>
|
||||
<option value="Arial">Arial</option>
|
||||
<option value="Roboto">Roboto</option>
|
||||
<option value="serif">Serif</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
{/* Font Size & Alignment */}
|
||||
<div className="flex gap-2 mb-4">
|
||||
<select className="flex-1 border border-gray-300 rounded-md px-3 py-2 text-sm focus:outline-none focus:border-blue-500">
|
||||
<option>Normal</option>
|
||||
<option>Large</option>
|
||||
<option>Small</option>
|
||||
</select>
|
||||
<div className="flex border border-gray-300 rounded-md overflow-hidden">
|
||||
<button
|
||||
className={`px-3 py-2 border-r border-gray-300 font-bold ${textStyles.isBold ? 'bg-gray-200' : 'bg-gray-50 hover:bg-gray-100'}`}
|
||||
onClick={() => setTextStyles({...textStyles, isBold: !textStyles.isBold})}
|
||||
>B</button>
|
||||
<button
|
||||
className={`px-3 py-2 border-r border-gray-300 italic ${textStyles.isItalic ? 'bg-gray-200' : 'bg-gray-50 hover:bg-gray-100'}`}
|
||||
onClick={() => setTextStyles({...textStyles, isItalic: !textStyles.isItalic})}
|
||||
>I</button>
|
||||
<button
|
||||
className={`px-3 py-2 underline ${textStyles.isUnderline ? 'bg-gray-200' : 'bg-gray-50 hover:bg-gray-100'}`}
|
||||
onClick={() => setTextStyles({...textStyles, isUnderline: !textStyles.isUnderline})}
|
||||
>U</button>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex gap-2 mb-4">
|
||||
<select
|
||||
aria-label={m.editor.subtitleSizePreset}
|
||||
className="flex-1 border border-gray-300 rounded-md px-3 py-2 text-sm focus:outline-none focus:border-blue-500"
|
||||
value={currentFontSizePreset}
|
||||
onChange={(e) =>
|
||||
updateSubtitleStyles({
|
||||
fontSize: FONT_SIZE_PRESETS[e.target.value as keyof typeof FONT_SIZE_PRESETS],
|
||||
})
|
||||
}
|
||||
>
|
||||
{Object.keys(FONT_SIZE_PRESETS).map((preset) => (
|
||||
<option key={preset} value={preset}>
|
||||
{fontSizePresetLabels[preset as keyof typeof FONT_SIZE_PRESETS]}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
<div className="flex border border-gray-300 rounded-md overflow-hidden">
|
||||
<button
|
||||
type="button"
|
||||
aria-label={m.editor.toggleBold}
|
||||
className={`px-3 py-2 border-r border-gray-300 font-bold ${activeSubtitleStyle.isBold ? 'bg-gray-200' : 'bg-gray-50 hover:bg-gray-100'}`}
|
||||
onClick={() => updateSubtitleStyles({ isBold: !activeSubtitleStyle.isBold })}
|
||||
>B</button>
|
||||
<button
|
||||
type="button"
|
||||
aria-label={m.editor.toggleItalic}
|
||||
className={`px-3 py-2 border-r border-gray-300 italic ${activeSubtitleStyle.isItalic ? 'bg-gray-200' : 'bg-gray-50 hover:bg-gray-100'}`}
|
||||
onClick={() => updateSubtitleStyles({ isItalic: !activeSubtitleStyle.isItalic })}
|
||||
>I</button>
|
||||
<button
|
||||
type="button"
|
||||
aria-label={m.editor.toggleUnderline}
|
||||
className={`px-3 py-2 underline ${activeSubtitleStyle.isUnderline ? 'bg-gray-200' : 'bg-gray-50 hover:bg-gray-100'}`}
|
||||
onClick={() => updateSubtitleStyles({ isUnderline: !activeSubtitleStyle.isUnderline })}
|
||||
>U</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Colors */}
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm text-gray-600">Color</span>
|
||||
<div className="flex items-center gap-2">
|
||||
<input
|
||||
type="color"
|
||||
value={textStyles.color}
|
||||
onChange={(e) => setTextStyles({...textStyles, color: e.target.value})}
|
||||
className="w-6 h-6 rounded border border-gray-300 p-0 cursor-pointer"
|
||||
/>
|
||||
<span className="text-sm text-gray-500">100%</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm text-gray-600">Stroke</span>
|
||||
<div className="flex items-center gap-2">
|
||||
<div className="w-6 h-6 rounded border border-gray-300 bg-black"></div>
|
||||
<span className="text-sm text-gray-500">100%</span>
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm text-gray-600">{m.editor.color}</span>
|
||||
<div className="flex items-center gap-2">
|
||||
<input
|
||||
type="color"
|
||||
aria-label={m.editor.subtitleColor}
|
||||
value={activeSubtitleStyle.color}
|
||||
onChange={(e) => updateSubtitleStyles({ color: e.target.value })}
|
||||
className="w-6 h-6 rounded border border-gray-300 p-0 cursor-pointer"
|
||||
/>
|
||||
<span className="text-sm text-gray-500">100%</span>
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm text-gray-600">{m.editor.stroke}</span>
|
||||
<div className="flex items-center gap-2">
|
||||
<input
|
||||
type="color"
|
||||
aria-label={m.editor.subtitleStrokeColor}
|
||||
value={activeSubtitleStyle.strokeColor}
|
||||
onChange={(e) => updateSubtitleStyles({ strokeColor: e.target.value })}
|
||||
className="w-6 h-6 rounded border border-gray-300 p-0 cursor-pointer"
|
||||
/>
|
||||
<span className="text-sm text-gray-500">100%</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
) : (
|
||||
<div className="p-4 text-sm text-gray-500">
|
||||
{m.editor.selectSubtitleToEdit}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -822,7 +992,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
<button className="p-1.5 hover:bg-gray-100 rounded text-gray-600"><Trash2 className="w-4 h-4" /></button>
|
||||
<div className="h-4 w-px bg-gray-300 mx-2"></div>
|
||||
<span className="text-xs text-orange-500 bg-orange-50 px-2 py-1 rounded border border-orange-200">
|
||||
Stretch the dubbing to control the speed
|
||||
{m.editor.stretchDubbing}
|
||||
</span>
|
||||
<div className="flex-1"></div>
|
||||
{/* Zoom slider placeholder */}
|
||||
@ -845,7 +1015,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
{/* Video Track */}
|
||||
<div className="h-12 border-b border-gray-200 flex items-center px-4 min-w-[1000px] relative">
|
||||
<div className="absolute left-4 right-4 h-10 bg-blue-50 rounded overflow-hidden flex border border-blue-100 items-center justify-center text-xs text-blue-400">
|
||||
Video Track
|
||||
{m.editor.videoTrack}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@ -952,7 +1122,7 @@ export default function EditorScreen({ videoFile, targetLanguage, trimRange, onB
|
||||
subtitles={subtitles}
|
||||
bgmUrl={bgmUrl}
|
||||
bgmBase64={bgmBase64}
|
||||
textStyles={textStyles}
|
||||
textStyles={exportTextStyles}
|
||||
trimRange={trimRange}
|
||||
/>
|
||||
)}
|
||||
|
||||
@ -4,6 +4,7 @@ import axios from 'axios';
|
||||
import { Subtitle, TextStyles } from '../types';
|
||||
import { buildExportPayload } from '../lib/exportPayload';
|
||||
import { apiUrl } from '../lib/apiBasePath';
|
||||
import { useI18n } from '../i18n';
|
||||
|
||||
interface ExportModalProps {
|
||||
onClose: () => void;
|
||||
@ -16,6 +17,7 @@ interface ExportModalProps {
|
||||
}
|
||||
|
||||
export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgmBase64, textStyles, trimRange }: ExportModalProps) {
|
||||
const { m } = useI18n();
|
||||
const [isExporting, setIsExporting] = useState(false);
|
||||
const [progress, setProgress] = useState(0);
|
||||
const [isDone, setIsDone] = useState(false);
|
||||
@ -81,7 +83,7 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
}
|
||||
} catch (err: any) {
|
||||
console.error('Export Error:', err);
|
||||
setError(err.response?.data?.error || err.message || 'An error occurred during export');
|
||||
setError(err.response?.data?.error || err.message || m.exportModal.errorDefault);
|
||||
} finally {
|
||||
setIsExporting(false);
|
||||
}
|
||||
@ -103,7 +105,7 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
<div className="bg-white rounded-lg w-[600px] flex flex-col shadow-xl overflow-hidden">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between p-4 border-b border-gray-100 bg-gray-50/50">
|
||||
<h2 className="text-lg font-semibold text-gray-800">Export Video</h2>
|
||||
<h2 className="text-lg font-semibold text-gray-800">{m.exportModal.title}</h2>
|
||||
<button onClick={onClose} className="text-gray-400 hover:text-gray-600">
|
||||
<X className="w-5 h-5" />
|
||||
</button>
|
||||
@ -117,7 +119,7 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
<video src={thumbnailUrl} className="w-full h-full object-cover" />
|
||||
) : (
|
||||
<div className="w-full h-full flex items-center justify-center text-gray-400">
|
||||
No Preview
|
||||
{m.exportModal.noPreview}
|
||||
</div>
|
||||
)}
|
||||
<div className="absolute inset-0 bg-black/20 flex items-center justify-center">
|
||||
@ -128,10 +130,10 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
{/* Form */}
|
||||
<div className="flex-1 space-y-4">
|
||||
<div>
|
||||
<label className="block text-xs text-gray-500 mb-1">File Name</label>
|
||||
<label className="block text-xs text-gray-500 mb-1">{m.exportModal.fileName}</label>
|
||||
<input
|
||||
type="text"
|
||||
value={videoFile?.name?.replace(/\.[^/.]+$/, "") + "_translated"}
|
||||
value={videoFile?.name?.replace(/\.[^/.]+$/, "") + m.exportModal.translatedSuffix}
|
||||
className="w-full border border-gray-300 rounded px-3 py-2 text-sm bg-gray-50 text-gray-600"
|
||||
readOnly
|
||||
/>
|
||||
@ -139,13 +141,13 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
|
||||
<div className="grid grid-cols-2 gap-4">
|
||||
<div>
|
||||
<label className="block text-xs text-gray-500 mb-1">Format</label>
|
||||
<label className="block text-xs text-gray-500 mb-1">{m.exportModal.format}</label>
|
||||
<input type="text" value="MP4 (H.264)" className="w-full border border-gray-300 rounded px-3 py-2 text-sm bg-gray-50 text-gray-600" readOnly />
|
||||
</div>
|
||||
<div>
|
||||
<label className="block text-xs text-gray-500 mb-1">Resolution</label>
|
||||
<label className="block text-xs text-gray-500 mb-1">{m.exportModal.resolution}</label>
|
||||
<select className="w-full border border-gray-300 rounded px-3 py-2 text-sm focus:outline-none focus:border-blue-500">
|
||||
<option>Original</option>
|
||||
<option>{m.exportModal.original}</option>
|
||||
<option>1080P</option>
|
||||
<option>720P</option>
|
||||
</select>
|
||||
@ -162,7 +164,7 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
{isExporting && (
|
||||
<div className="space-y-2">
|
||||
<div className="flex justify-between text-xs">
|
||||
<span className="text-blue-600 font-medium">Exporting...</span>
|
||||
<span className="text-blue-600 font-medium">{m.exportModal.exporting}</span>
|
||||
<span className="text-blue-600 font-medium">{progress}%</span>
|
||||
</div>
|
||||
<div className="w-full h-2 bg-gray-100 rounded-full overflow-hidden">
|
||||
@ -179,7 +181,7 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
{/* Footer */}
|
||||
<div className="p-4 border-t border-gray-100 bg-gray-50/50 flex flex-col items-end gap-3">
|
||||
<p className="text-xs text-gray-500 w-full text-left">
|
||||
Compositing video with subtitles and AI dubbing. This may take a few minutes depending on the video length.
|
||||
{m.exportModal.description}
|
||||
</p>
|
||||
|
||||
<div className="flex gap-3">
|
||||
@ -188,7 +190,7 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
className="px-4 py-2 text-sm text-gray-500 hover:text-gray-700 font-medium"
|
||||
disabled={isExporting}
|
||||
>
|
||||
{isDone ? 'Close' : 'Cancel'}
|
||||
{isDone ? m.exportModal.close : m.exportModal.cancel}
|
||||
</button>
|
||||
|
||||
{isDone ? (
|
||||
@ -197,7 +199,7 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
className="bg-[#52c41a] text-white px-6 py-2 rounded-md text-sm font-medium flex items-center gap-2 hover:bg-[#46a616] transition-colors shadow-sm"
|
||||
>
|
||||
<Download className="w-4 h-4" />
|
||||
Download Video
|
||||
{m.exportModal.downloadVideo}
|
||||
</button>
|
||||
) : (
|
||||
<button
|
||||
@ -208,9 +210,9 @@ export default function ExportModal({ onClose, videoFile, subtitles, bgmUrl, bgm
|
||||
{isExporting ? (
|
||||
<span className="flex items-center gap-2">
|
||||
<Loader2 className="w-4 h-4 animate-spin" />
|
||||
Processing...
|
||||
{m.exportModal.processing}
|
||||
</span>
|
||||
) : 'Start Export'}
|
||||
) : m.exportModal.startExport}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
import React, { useState, useRef, useEffect, useMemo } from 'react';
|
||||
import { X, Play, Pause } from 'lucide-react';
|
||||
import { useI18n } from '../i18n';
|
||||
|
||||
interface TrimModalProps {
|
||||
file: File;
|
||||
@ -8,6 +9,7 @@ interface TrimModalProps {
|
||||
}
|
||||
|
||||
export default function TrimModal({ file, onClose, onConfirm }: TrimModalProps) {
|
||||
const { m, format } = useI18n();
|
||||
const videoRef = useRef<HTMLVideoElement>(null);
|
||||
const timelineRef = useRef<HTMLDivElement>(null);
|
||||
|
||||
@ -251,14 +253,14 @@ export default function TrimModal({ file, onClose, onConfirm }: TrimModalProps)
|
||||
<div className="w-full flex items-center justify-between border-t border-gray-100 pt-4">
|
||||
<div className="flex items-center gap-4">
|
||||
<span className="text-sm text-gray-600">
|
||||
Selected duration: <strong className="text-gray-800">{Math.round(trimDuration)}s</strong>
|
||||
{format(m.trim.selectedDuration, { value: Math.round(trimDuration) })}
|
||||
</span>
|
||||
</div>
|
||||
<button
|
||||
onClick={() => onConfirm(file, startTime, endTime)}
|
||||
className="px-6 py-2 rounded-md text-sm font-medium transition-colors border border-gray-300 text-gray-700 hover:bg-gray-50"
|
||||
>
|
||||
Confirm
|
||||
{m.trim.confirm}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
128
src/components/UploadScreen.test.tsx
Normal file
128
src/components/UploadScreen.test.tsx
Normal file
@ -0,0 +1,128 @@
|
||||
// @vitest-environment jsdom
|
||||
|
||||
import React from 'react';
|
||||
import { cleanup, fireEvent, render, screen } from '@testing-library/react';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import UploadScreen from './UploadScreen';
|
||||
import { I18nProvider } from '../i18n';
|
||||
|
||||
vi.mock('./TrimModal', () => ({
|
||||
default: ({
|
||||
file,
|
||||
onClose,
|
||||
onConfirm,
|
||||
}: {
|
||||
file: File;
|
||||
onClose: () => void;
|
||||
onConfirm: (file: File, startTime: number, endTime: number) => void;
|
||||
}) => (
|
||||
<div>
|
||||
<button type="button" onClick={onClose}>
|
||||
Close trim
|
||||
</button>
|
||||
<button type="button" onClick={() => onConfirm(file, 1, 5)}>
|
||||
Confirm trim
|
||||
</button>
|
||||
</div>
|
||||
),
|
||||
}));
|
||||
|
||||
describe('UploadScreen', () => {
|
||||
const renderUploadScreen = (onUpload = () => {}) =>
|
||||
render(
|
||||
<I18nProvider initialLocale="en">
|
||||
<UploadScreen onUpload={onUpload} />
|
||||
</I18nProvider>,
|
||||
);
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
});
|
||||
|
||||
it('starts from the editor default subtitle size and position and updates the preview', () => {
|
||||
renderUploadScreen();
|
||||
|
||||
const sizeSlider = screen.getByLabelText(/subtitle initial size/i);
|
||||
const positionSlider = screen.getByLabelText(/subtitle initial position/i);
|
||||
const preview = screen.getByTestId('upload-subtitle-preview');
|
||||
|
||||
expect(sizeSlider).toHaveValue('24');
|
||||
expect(positionSlider).toHaveValue('10');
|
||||
expect(preview).toHaveStyle({ fontSize: '24px', bottom: '10%' });
|
||||
|
||||
fireEvent.change(sizeSlider, { target: { value: '32' } });
|
||||
fireEvent.change(positionSlider, { target: { value: '18' } });
|
||||
|
||||
expect(preview).toHaveStyle({ fontSize: '32px', bottom: '18%' });
|
||||
});
|
||||
|
||||
it('shows a fixed English subtitle language and the supported TTS languages', () => {
|
||||
renderUploadScreen();
|
||||
|
||||
expect(screen.getByText('Subtitle Language')).toBeInTheDocument();
|
||||
expect(screen.getAllByText('English')).toHaveLength(2);
|
||||
expect(screen.getByText('TTS Language')).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: 'Chinese' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: 'Cantonese' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: 'French' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: 'Indonesian' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: 'German' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: 'Filipino' })).toBeInTheDocument();
|
||||
|
||||
expect(screen.queryByRole('button', { name: 'Arabic' })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole('button', { name: 'Japanese' })).not.toBeInTheDocument();
|
||||
expect(screen.queryByRole('button', { name: 'Korean' })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('passes subtitle defaults through when upload is confirmed', () => {
|
||||
const onUpload = vi.fn();
|
||||
renderUploadScreen(onUpload);
|
||||
|
||||
fireEvent.change(screen.getByLabelText(/subtitle initial size/i), {
|
||||
target: { value: '30' },
|
||||
});
|
||||
fireEvent.change(screen.getByLabelText(/subtitle initial position/i), {
|
||||
target: { value: '16' },
|
||||
});
|
||||
fireEvent.click(screen.getByRole('button', { name: 'French' }));
|
||||
|
||||
fireEvent.change(screen.getByLabelText(/upload video file/i), {
|
||||
target: {
|
||||
files: [new File(['video'], 'clip.mp4', { type: 'video/mp4' })],
|
||||
},
|
||||
});
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /confirm trim/i }));
|
||||
|
||||
expect(onUpload).toHaveBeenCalledWith(
|
||||
expect.any(File),
|
||||
'English',
|
||||
'French',
|
||||
{
|
||||
fontSize: 30,
|
||||
bottomOffsetPercent: 16,
|
||||
},
|
||||
1,
|
||||
5,
|
||||
);
|
||||
});
|
||||
|
||||
it('clears the pending upload after closing the trim preview', () => {
|
||||
renderUploadScreen();
|
||||
|
||||
const fileInput = screen.getByLabelText(/upload video file/i);
|
||||
const file = new File(['video'], 'clip.mp4', { type: 'video/mp4' });
|
||||
|
||||
fireEvent.change(fileInput, {
|
||||
target: {
|
||||
files: [file],
|
||||
},
|
||||
});
|
||||
|
||||
expect(screen.getByRole('button', { name: /close trim/i })).toBeInTheDocument();
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /close trim/i }));
|
||||
expect(screen.queryByRole('button', { name: /close trim/i })).not.toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: /generate translated video/i })).toBeDisabled();
|
||||
});
|
||||
});
|
||||
@ -1,30 +1,45 @@
|
||||
import React, { useState } from 'react';
|
||||
import React, { useRef, useState } from 'react';
|
||||
import { Upload, CheckCircle2, Circle } from 'lucide-react';
|
||||
import TrimModal from './TrimModal';
|
||||
import { DEFAULT_SUBTITLE_DEFAULTS, SubtitleDefaults } from '../types';
|
||||
import { useI18n } from '../i18n';
|
||||
|
||||
const LANGUAGES = [
|
||||
{ code: 'ar', name: 'Arabic', group: 'A' },
|
||||
{ code: 'zh', name: 'Chinese', group: 'C' },
|
||||
{ code: 'yue', name: 'Cantonese', group: 'C' },
|
||||
{ code: 'cs', name: 'Czech', group: 'C' },
|
||||
{ code: 'zh-TW', name: 'Traditional Chinese', group: 'T' },
|
||||
{ code: 'th', name: 'Thai', group: 'T' },
|
||||
{ code: 'tr', name: 'Turkey', group: 'T' },
|
||||
{ code: 'nl', name: 'Dutch', group: 'D' },
|
||||
{ code: 'en', name: 'English', group: 'E' },
|
||||
{ code: 'ru', name: 'Russian', group: 'R' },
|
||||
{ code: 'ro', name: 'Romanian', group: 'R' },
|
||||
{ code: 'ja', name: 'Japanese', group: 'J' },
|
||||
{ code: 'ko', name: 'Korean', group: 'K' },
|
||||
{ code: 'ms', name: 'Malay', group: 'M' },
|
||||
{ code: 'fr', name: 'French', group: 'F' },
|
||||
{ code: 'zh', apiName: 'Chinese', group: 'C' },
|
||||
{ code: 'yue', apiName: 'Cantonese', group: 'C' },
|
||||
{ code: 'en', apiName: 'English', group: 'E' },
|
||||
{ code: 'id', apiName: 'Indonesian', group: 'I' },
|
||||
{ code: 'de', apiName: 'German', group: 'G' },
|
||||
{ code: 'fil', apiName: 'Filipino', group: 'F' },
|
||||
{ code: 'fr', apiName: 'French', group: 'F' },
|
||||
];
|
||||
|
||||
export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang: string, startTime?: number, endTime?: number) => void }) {
|
||||
export default function UploadScreen({
|
||||
onUpload,
|
||||
}: {
|
||||
onUpload: (
|
||||
file: File,
|
||||
targetLanguage: string,
|
||||
ttsLanguage: string,
|
||||
subtitleDefaults: SubtitleDefaults,
|
||||
startTime?: number,
|
||||
endTime?: number,
|
||||
) => void
|
||||
}) {
|
||||
const { m, format } = useI18n();
|
||||
const [mode, setMode] = useState<'editing' | 'simple'>('editing');
|
||||
const [selectedLang, setSelectedLang] = useState('en');
|
||||
const [selectedTtsLanguage, setSelectedTtsLanguage] = useState('en');
|
||||
const [showTrimModal, setShowTrimModal] = useState(false);
|
||||
const [tempFile, setTempFile] = useState<File | null>(null);
|
||||
const [subtitleDefaults, setSubtitleDefaults] = useState<SubtitleDefaults>(DEFAULT_SUBTITLE_DEFAULTS);
|
||||
const fileInputRef = useRef<HTMLInputElement | null>(null);
|
||||
|
||||
const clearPendingUpload = () => {
|
||||
setTempFile(null);
|
||||
if (fileInputRef.current) {
|
||||
fileInputRef.current.value = '';
|
||||
}
|
||||
};
|
||||
|
||||
const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
|
||||
if (e.target.files && e.target.files[0]) {
|
||||
@ -35,8 +50,8 @@ export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang
|
||||
|
||||
const handleTrimConfirm = (file: File, startTime: number, endTime: number) => {
|
||||
setShowTrimModal(false);
|
||||
const langName = LANGUAGES.find(l => l.code === selectedLang)?.name || 'English';
|
||||
onUpload(file, langName, startTime, endTime);
|
||||
const ttsLanguage = LANGUAGES.find((language) => language.code === selectedTtsLanguage)?.apiName || 'English';
|
||||
onUpload(file, 'English', ttsLanguage, subtitleDefaults, startTime, endTime);
|
||||
};
|
||||
|
||||
return (
|
||||
@ -45,20 +60,22 @@ export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang
|
||||
<div className="flex-1 bg-white rounded-lg shadow-sm border border-gray-200 p-8 flex flex-col items-center justify-center min-h-[400px]">
|
||||
<div className="w-full h-full border-2 border-dashed border-gray-300 rounded-lg flex flex-col items-center justify-center bg-gray-50 relative">
|
||||
<input
|
||||
ref={fileInputRef}
|
||||
type="file"
|
||||
aria-label="Upload video file"
|
||||
className="absolute inset-0 w-full h-full opacity-0 cursor-pointer"
|
||||
accept="video/mp4,video/quicktime,video/webm"
|
||||
onChange={handleFileChange}
|
||||
/>
|
||||
<Upload className="w-16 h-16 text-gray-400 mb-4" />
|
||||
<p className="text-gray-600 mb-6">Click to upload or drag files here</p>
|
||||
<p className="text-gray-600 mb-6">{m.upload.clickToUpload}</p>
|
||||
<button className="bg-[#52c41a] hover:bg-[#46a616] text-white px-8 py-3 rounded-md font-medium flex items-center gap-2 transition-colors w-full max-w-md justify-center pointer-events-none">
|
||||
<Upload className="w-5 h-5" />
|
||||
Upload Video
|
||||
{m.upload.uploadVideo}
|
||||
</button>
|
||||
</div>
|
||||
<p className="text-sm text-gray-500 mt-4 w-full text-left">
|
||||
Supported formats: MP4/MOV/WEBM. Maximum file size is 500MB.
|
||||
{m.upload.supportedFormats}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@ -78,9 +95,9 @@ export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang
|
||||
) : (
|
||||
<Circle className="w-5 h-5 text-gray-300" />
|
||||
)}
|
||||
<span className="font-semibold text-gray-800">Editing Mode</span>
|
||||
<span className="font-semibold text-gray-800">{m.upload.editingMode}</span>
|
||||
</div>
|
||||
<p className="text-xs text-gray-500 ml-7">Supports secondary editing and more precise translation</p>
|
||||
<p className="text-xs text-gray-500 ml-7">{m.upload.editingModeDesc}</p>
|
||||
</div>
|
||||
<div
|
||||
className={`flex-1 p-4 rounded-lg border-2 cursor-pointer transition-colors ${
|
||||
@ -94,34 +111,123 @@ export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang
|
||||
) : (
|
||||
<Circle className="w-5 h-5 text-gray-300" />
|
||||
)}
|
||||
<span className="font-semibold text-gray-800">Simple Mode</span>
|
||||
<span className="font-semibold text-gray-800">{m.upload.simpleMode}</span>
|
||||
</div>
|
||||
<p className="text-xs text-gray-500 ml-7">{m.upload.simpleModeDesc}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="bg-white rounded-lg shadow-sm border border-gray-200 p-6">
|
||||
<div className="flex items-start justify-between gap-4 mb-4">
|
||||
<div>
|
||||
<h3 className="font-semibold text-gray-800 mb-1">{m.upload.subtitleDefaults}</h3>
|
||||
<p className="text-xs text-gray-500">
|
||||
{m.upload.subtitleDefaultsDesc}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
className="text-sm text-blue-600 hover:text-blue-700 font-medium"
|
||||
onClick={() => setSubtitleDefaults(DEFAULT_SUBTITLE_DEFAULTS)}
|
||||
>
|
||||
{m.upload.reset}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div className="rounded-xl border border-gray-200 bg-slate-950 p-4 mb-4">
|
||||
<div className="relative h-48 rounded-lg overflow-hidden bg-[radial-gradient(circle_at_top,_rgba(148,163,184,0.35),_transparent_40%),linear-gradient(180deg,_#1e293b_0%,_#0f172a_100%)]">
|
||||
<div className="absolute inset-0 bg-[linear-gradient(90deg,transparent_0%,rgba(255,255,255,0.04)_50%,transparent_100%)]" />
|
||||
<p
|
||||
data-testid="upload-subtitle-preview"
|
||||
className="absolute left-1/2 -translate-x-1/2 text-white font-semibold text-center whitespace-pre-wrap max-w-[88%] px-3 py-1"
|
||||
style={{
|
||||
bottom: `${subtitleDefaults.bottomOffsetPercent}%`,
|
||||
fontSize: `${subtitleDefaults.fontSize}px`,
|
||||
textShadow: '2px 0 #000, -2px 0 #000, 0 2px #000, 0 -2px #000',
|
||||
}}
|
||||
>
|
||||
{m.upload.subtitlePreview}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="space-y-4">
|
||||
<div>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<label htmlFor="subtitle-position" className="text-sm font-medium text-gray-700">
|
||||
{m.upload.subtitleInitialPosition}
|
||||
</label>
|
||||
<span className="text-xs text-gray-500">{format(m.upload.fromBottom, { value: subtitleDefaults.bottomOffsetPercent })}</span>
|
||||
</div>
|
||||
<input
|
||||
id="subtitle-position"
|
||||
aria-label="Subtitle initial position"
|
||||
type="range"
|
||||
min="4"
|
||||
max="30"
|
||||
step="1"
|
||||
value={subtitleDefaults.bottomOffsetPercent}
|
||||
onChange={(e) =>
|
||||
setSubtitleDefaults((previous) => ({
|
||||
...previous,
|
||||
bottomOffsetPercent: Number(e.target.value),
|
||||
}))
|
||||
}
|
||||
className="w-full accent-blue-500"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<div className="flex items-center justify-between mb-2">
|
||||
<label htmlFor="subtitle-size" className="text-sm font-medium text-gray-700">
|
||||
{m.upload.subtitleInitialSize}
|
||||
</label>
|
||||
<span className="text-xs text-gray-500">{format(m.upload.pxValue, { value: subtitleDefaults.fontSize })}</span>
|
||||
</div>
|
||||
<input
|
||||
id="subtitle-size"
|
||||
aria-label="Subtitle initial size"
|
||||
type="range"
|
||||
min="16"
|
||||
max="40"
|
||||
step="1"
|
||||
value={subtitleDefaults.fontSize}
|
||||
onChange={(e) =>
|
||||
setSubtitleDefaults((previous) => ({
|
||||
...previous,
|
||||
fontSize: Number(e.target.value),
|
||||
}))
|
||||
}
|
||||
className="w-full accent-blue-500"
|
||||
/>
|
||||
</div>
|
||||
<p className="text-xs text-gray-500 ml-7">One-click video translation for beginners</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Language Selection */}
|
||||
<div className="bg-white rounded-lg shadow-sm border border-gray-200 p-6 flex-1 flex flex-col">
|
||||
<h3 className="font-semibold text-gray-800 mb-1">Select Translation Language</h3>
|
||||
<p className="text-xs text-gray-500 mb-4">AI detect video source language automatically.</p>
|
||||
<div className="mb-5">
|
||||
<h3 className="font-semibold text-gray-800 mb-1">{m.upload.subtitleLanguage}</h3>
|
||||
<p className="text-xs text-gray-500 mb-3">{m.upload.subtitleLanguageHint}</p>
|
||||
<div className="inline-flex items-center rounded-md border border-gray-200 bg-gray-50 px-3 py-2 text-sm font-medium text-gray-700">
|
||||
{m.upload.languages.en}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<h3 className="font-semibold text-gray-800 mb-1">{m.upload.ttsLanguage}</h3>
|
||||
<p className="text-xs text-gray-500 mb-4">{m.upload.ttsLanguageHint}</p>
|
||||
|
||||
{/* Alphabet Tabs */}
|
||||
<div className="flex gap-4 border-b border-gray-100 pb-2 mb-4 text-sm text-gray-500 overflow-x-auto">
|
||||
<button className="font-medium text-blue-600 border-b-2 border-blue-600 pb-2 -mb-[9px]">Popular</button>
|
||||
<button className="hover:text-gray-800">ABC</button>
|
||||
<button className="hover:text-gray-800">DEF</button>
|
||||
<button className="hover:text-gray-800">GHI</button>
|
||||
<button className="hover:text-gray-800">JKL</button>
|
||||
<button className="hover:text-gray-800">MN</button>
|
||||
<button className="hover:text-gray-800">OPQ</button>
|
||||
<button className="hover:text-gray-800">RST</button>
|
||||
<button className="hover:text-gray-800">UVW</button>
|
||||
<button className="hover:text-gray-800">XYZ</button>
|
||||
<button className="font-medium text-blue-600 border-b-2 border-blue-600 pb-2 -mb-[9px]">{m.upload.popular}</button>
|
||||
<button className="hover:text-gray-800">{m.upload.groupABC}</button>
|
||||
<button className="hover:text-gray-800">{m.upload.groupGHI}</button>
|
||||
<button className="hover:text-gray-800">{m.upload.groupDEF}</button>
|
||||
</div>
|
||||
|
||||
{/* Language List */}
|
||||
<div className="flex-1 overflow-y-auto pr-2 custom-scrollbar">
|
||||
{['A', 'C', 'T', 'D', 'E', 'R', 'J', 'K', 'M', 'F'].map((letter) => (
|
||||
{['C', 'E', 'F', 'G', 'I'].map((letter) => (
|
||||
<div key={letter} className="flex border-b border-gray-100 py-3 last:border-0">
|
||||
<div className="w-8 text-green-600 font-medium">{letter}</div>
|
||||
<div className="flex flex-wrap gap-x-6 gap-y-2 flex-1">
|
||||
@ -129,15 +235,15 @@ export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang
|
||||
<button
|
||||
key={lang.code}
|
||||
className={`text-sm hover:text-blue-600 transition-colors ${
|
||||
selectedLang === lang.code
|
||||
selectedTtsLanguage === lang.code
|
||||
? 'bg-green-600 text-white px-2 py-0.5 rounded'
|
||||
: lang.code === 'zh' || lang.code === 'yue' || lang.code === 'ja' || lang.code === 'ko'
|
||||
: lang.code === 'zh' || lang.code === 'yue'
|
||||
? 'text-orange-500'
|
||||
: 'text-gray-700'
|
||||
}`}
|
||||
onClick={() => setSelectedLang(lang.code)}
|
||||
onClick={() => setSelectedTtsLanguage(lang.code)}
|
||||
>
|
||||
{lang.name}
|
||||
{m.upload.languages[lang.code as keyof typeof m.upload.languages]}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
@ -156,7 +262,7 @@ export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang
|
||||
}}
|
||||
disabled={!tempFile}
|
||||
>
|
||||
Generate Translated Video
|
||||
{m.upload.generateTranslatedVideo}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
@ -164,7 +270,10 @@ export default function UploadScreen({ onUpload }: { onUpload: (file: File, lang
|
||||
{showTrimModal && tempFile && (
|
||||
<TrimModal
|
||||
file={tempFile}
|
||||
onClose={() => setShowTrimModal(false)}
|
||||
onClose={() => {
|
||||
setShowTrimModal(false);
|
||||
clearPendingUpload();
|
||||
}}
|
||||
onConfirm={handleTrimConfirm}
|
||||
/>
|
||||
)}
|
||||
|
||||
@ -2,6 +2,7 @@ import React, { useState } from 'react';
|
||||
import { X, Play, Search, Filter } from 'lucide-react';
|
||||
import { MINIMAX_VOICES } from '../voices';
|
||||
import { Voice } from '../types';
|
||||
import { useI18n } from '../i18n';
|
||||
|
||||
export default function VoiceMarketModal({
|
||||
onClose,
|
||||
@ -12,6 +13,7 @@ export default function VoiceMarketModal({
|
||||
onSelect: (voiceId: string) => void;
|
||||
onSelectAll: (voiceId: string) => void;
|
||||
}) {
|
||||
const { m } = useI18n();
|
||||
const [searchQuery, setSearchQuery] = useState('');
|
||||
const [selectedLanguage, setSelectedLanguage] = useState<string>('all');
|
||||
const [selectedGender, setSelectedGender] = useState<string>('all');
|
||||
@ -32,12 +34,12 @@ export default function VoiceMarketModal({
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between p-4 border-b border-gray-100 shrink-0">
|
||||
<div className="flex items-center gap-4">
|
||||
<h2 className="text-lg font-semibold text-gray-800">Voice Market</h2>
|
||||
<h2 className="text-lg font-semibold text-gray-800">{m.voiceMarket.title}</h2>
|
||||
<div className="flex items-center gap-2 bg-gray-100 px-3 py-1.5 rounded-md">
|
||||
<Search className="w-4 h-4 text-gray-400" />
|
||||
<input
|
||||
type="text"
|
||||
placeholder="Search voices..."
|
||||
placeholder={m.voiceMarket.searchVoices}
|
||||
className="bg-transparent border-none focus:ring-0 text-sm w-48"
|
||||
value={searchQuery}
|
||||
onChange={(e) => setSearchQuery(e.target.value)}
|
||||
@ -48,7 +50,7 @@ export default function VoiceMarketModal({
|
||||
value={selectedLanguage}
|
||||
onChange={(e) => setSelectedLanguage(e.target.value)}
|
||||
>
|
||||
<option value="all">All Languages</option>
|
||||
<option value="all">{m.voiceMarket.allLanguages}</option>
|
||||
{languages.map(lang => (
|
||||
<option key={lang} value={lang}>{lang.toUpperCase()}</option>
|
||||
))}
|
||||
@ -58,10 +60,10 @@ export default function VoiceMarketModal({
|
||||
value={selectedGender}
|
||||
onChange={(e) => setSelectedGender(e.target.value)}
|
||||
>
|
||||
<option value="all">All Genders</option>
|
||||
<option value="male">Male</option>
|
||||
<option value="female">Female</option>
|
||||
<option value="neutral">Neutral</option>
|
||||
<option value="all">{m.voiceMarket.allGenders}</option>
|
||||
<option value="male">{m.voiceMarket.male}</option>
|
||||
<option value="female">{m.voiceMarket.female}</option>
|
||||
<option value="neutral">{m.voiceMarket.neutral}</option>
|
||||
</select>
|
||||
</div>
|
||||
<button onClick={onClose} className="text-gray-400 hover:text-gray-600">
|
||||
@ -72,7 +74,7 @@ export default function VoiceMarketModal({
|
||||
{/* Content */}
|
||||
<div className="flex-1 p-6 overflow-y-auto bg-gray-50/50">
|
||||
<div className="bg-blue-50 text-blue-800 text-sm p-3 rounded-md mb-6 border border-blue-100">
|
||||
Tip: Select a voice for the current sentence, or apply it to all sentences in the project.
|
||||
{m.voiceMarket.tip}
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-4 gap-4">
|
||||
@ -97,13 +99,13 @@ export default function VoiceMarketModal({
|
||||
onClick={() => onSelect(voice.id)}
|
||||
className="flex-1 py-1.5 rounded text-xs font-medium border border-[#52c41a] text-[#52c41a] hover:bg-green-50 transition-colors"
|
||||
>
|
||||
Choose
|
||||
{m.voiceMarket.choose}
|
||||
</button>
|
||||
<button
|
||||
onClick={() => onSelectAll(voice.id)}
|
||||
className="flex-1 py-1.5 rounded text-xs font-medium border border-gray-300 text-gray-600 hover:bg-gray-50"
|
||||
>
|
||||
Apply All
|
||||
{m.voiceMarket.applyAll}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
@ -112,7 +114,7 @@ export default function VoiceMarketModal({
|
||||
{filteredVoices.length === 0 && (
|
||||
<div className="flex flex-col items-center justify-center py-20 text-gray-400">
|
||||
<Search className="w-12 h-12 mb-2 opacity-20" />
|
||||
<p>No voices found matching your criteria</p>
|
||||
<p>{m.voiceMarket.noVoicesFound}</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
304
src/i18n.tsx
Normal file
304
src/i18n.tsx
Normal file
@ -0,0 +1,304 @@
|
||||
import React, { createContext, useContext, useMemo, useState } from 'react';
|
||||
|
||||
export type Locale = 'zh' | 'en';
|
||||
|
||||
const messages = {
|
||||
zh: {
|
||||
app: {
|
||||
chinese: '\u4e2d\u6587',
|
||||
english: 'English',
|
||||
productName: 'Translate 1.0',
|
||||
},
|
||||
upload: {
|
||||
uploadVideoFile: '\u4e0a\u4f20\u89c6\u9891\u6587\u4ef6',
|
||||
clickToUpload: '\u70b9\u51fb\u4e0a\u4f20\u6216\u5c06\u89c6\u9891\u62d6\u62fd\u5230\u6b64\u5904',
|
||||
uploadVideo: '\u4e0a\u4f20\u89c6\u9891',
|
||||
supportedFormats: '\u652f\u6301\u683c\u5f0f\uff1aMP4 / MOV / WEBM\uff0c\u6587\u4ef6\u5927\u5c0f\u4e0d\u8d85\u8fc7 500MB\u3002',
|
||||
editingMode: '\u7f16\u8f91\u6a21\u5f0f',
|
||||
editingModeDesc: '\u652f\u6301\u4e8c\u6b21\u7f16\u8f91\u548c\u66f4\u7cbe\u7ec6\u7684\u7ffb\u8bd1\u8c03\u6574',
|
||||
simpleMode: '\u7b80\u5355\u6a21\u5f0f',
|
||||
simpleModeDesc: '\u9002\u5408\u5feb\u901f\u751f\u6210\u7ffb\u8bd1\u89c6\u9891',
|
||||
subtitleDefaults: '\u5b57\u5e55\u521d\u59cb\u6837\u5f0f',
|
||||
subtitleDefaultsDesc: '\u8bbe\u7f6e\u5b57\u5e55\u7684\u521d\u59cb\u4f4d\u7f6e\u548c\u5927\u5c0f\uff0c\u751f\u6210\u540e\u4ecd\u53ef\u5728\u7f16\u8f91\u9875\u7ee7\u7eed\u8c03\u6574\u3002',
|
||||
reset: '\u6062\u590d\u9ed8\u8ba4',
|
||||
subtitlePreview: '\u5b57\u5e55\u9884\u89c8\u793a\u4f8b',
|
||||
subtitleInitialPosition: '\u5b57\u5e55\u521d\u59cb\u4f4d\u7f6e',
|
||||
subtitleInitialSize: '\u5b57\u5e55\u521d\u59cb\u5927\u5c0f',
|
||||
fromBottom: '\u8ddd\u5e95\u90e8 {value}%',
|
||||
pxValue: '{value}px',
|
||||
subtitleLanguage: '\u5b57\u5e55\u8bed\u8a00',
|
||||
subtitleLanguageHint: '\u5b57\u5e55\u7ffb\u8bd1\u56fa\u5b9a\u4e3a\u82f1\u8bed\u3002',
|
||||
ttsLanguage: 'TTS \u8bed\u8a00',
|
||||
ttsLanguageHint: '\u9009\u62e9\u914d\u97f3\u6717\u8bfb\u4f7f\u7528\u7684\u76ee\u6807\u8bed\u8a00\u3002',
|
||||
selectTranslationLanguage: '\u9009\u62e9\u7ffb\u8bd1\u8bed\u8a00',
|
||||
detectSourceLanguage: 'AI \u4f1a\u81ea\u52a8\u8bc6\u522b\u89c6\u9891\u6e90\u8bed\u8a00\u3002',
|
||||
popular: '\u70ed\u95e8',
|
||||
groupABC: 'ABC',
|
||||
groupDEF: 'DEF',
|
||||
groupGHI: 'GHI',
|
||||
generateTranslatedVideo: '\u751f\u6210\u7ffb\u8bd1\u89c6\u9891',
|
||||
languages: {
|
||||
zh: '\u4e2d\u6587\u666e\u901a\u8bdd',
|
||||
yue: '\u4e2d\u6587\u7ca4\u8bed',
|
||||
en: '\u82f1\u8bed',
|
||||
fr: '\u6cd5\u8bed',
|
||||
id: '\u5370\u5c3c\u8bed',
|
||||
de: '\u5fb7\u8bed',
|
||||
fil: '\u83f2\u5f8b\u5bbe\u8bed',
|
||||
},
|
||||
},
|
||||
trim: {
|
||||
confirm: '\u786e\u8ba4',
|
||||
selectedDuration: '\u5df2\u9009\u65f6\u957f\uff1a{value} \u79d2',
|
||||
},
|
||||
editor: {
|
||||
watchVideo: '\u9884\u89c8\u89c6\u9891',
|
||||
saveEditing: '\u4fdd\u5b58\u7f16\u8f91',
|
||||
export: '\u5bfc\u51fa',
|
||||
aiDub: 'AI \u914d\u97f3',
|
||||
voiceClone: '\u58f0\u97f3\u514b\u9686',
|
||||
tip: '\u63d0\u793a\uff1a\u4fee\u6539\u5b57\u5e55\u6587\u672c\u540e\uff0c\u9700\u8981\u91cd\u65b0\u751f\u6210\u914d\u97f3\u624d\u4f1a\u751f\u6548\u3002',
|
||||
llm: 'LLM',
|
||||
llmProvider: '\u6a21\u578b\u63d0\u4f9b\u65b9',
|
||||
generateDubbing: '\u751f\u6210\u914d\u97f3',
|
||||
separatingVocals: '\u6b63\u5728\u5206\u79bb\u4eba\u58f0...',
|
||||
generatingTts: '\u6b63\u5728\u751f\u6210\u914d\u97f3...',
|
||||
lowPrecision: '\u672c\u6b21\u5b57\u5e55\u65f6\u95f4\u8f74\u4e3a\u4f4e\u7cbe\u5ea6\u7ed3\u679c\uff0c\u4f60\u4ecd\u53ef\u5728\u914d\u97f3\u524d\u7ee7\u7eed\u8c03\u6574\u5b57\u5e55\u3002',
|
||||
retryGeneration: '\u91cd\u65b0\u751f\u6210',
|
||||
noVideoLoaded: '\u672a\u52a0\u8f7d\u89c6\u9891',
|
||||
bgm: 'BGM',
|
||||
applyAll: '\u5e94\u7528\u5230\u5168\u90e8\u5b57\u5e55',
|
||||
applyAllAria: '\u5c06\u6837\u5f0f\u53d8\u66f4\u5e94\u7528\u5230\u5168\u90e8\u5b57\u5e55',
|
||||
activeSubtitleContext: '\u5f53\u524d\u5b57\u5e55\u4e0a\u4e0b\u6587',
|
||||
textStyles: '\u5b57\u5e55\u6837\u5f0f',
|
||||
selectSubtitleToEdit: '\u8bf7\u9009\u62e9\u4e00\u6761\u5b57\u5e55\uff0c\u518d\u7f16\u8f91\u5b83\u7684\u8bf4\u8bdd\u4eba\u4fe1\u606f\u548c\u663e\u793a\u6837\u5f0f\u3002',
|
||||
subtitleFontFamily: '\u5b57\u5e55\u5b57\u4f53',
|
||||
subtitleSizePreset: '\u5b57\u5e55\u5927\u5c0f\u9884\u8bbe',
|
||||
toggleBold: '\u5207\u6362\u7c97\u4f53',
|
||||
toggleItalic: '\u5207\u6362\u659c\u4f53',
|
||||
toggleUnderline: '\u5207\u6362\u4e0b\u5212\u7ebf',
|
||||
subtitleColor: '\u5b57\u5e55\u989c\u8272',
|
||||
subtitleStrokeColor: '\u5b57\u5e55\u63cf\u8fb9\u989c\u8272',
|
||||
color: '\u989c\u8272',
|
||||
stroke: '\u63cf\u8fb9',
|
||||
small: '\u5c0f',
|
||||
normal: '\u4e2d',
|
||||
large: '\u5927',
|
||||
alignLeft: '\u5de6\u5bf9\u9f50',
|
||||
alignCenter: '\u5c45\u4e2d\u5bf9\u9f50',
|
||||
alignRight: '\u53f3\u5bf9\u9f50',
|
||||
videoTrack: '\u89c6\u9891\u8f68\u9053',
|
||||
stretchDubbing: '\u62c9\u4f38\u914d\u97f3\u4ee5\u63a7\u5236\u8bed\u901f',
|
||||
qualityQueued: '\u6392\u961f\u4e2d',
|
||||
aiAnalyzing: 'AI \u6b63\u5728\u5206\u6790\u5e76\u7ffb\u8bd1\u89c6\u9891...',
|
||||
thisMayTakeMinutes: '\u8fd9\u53ef\u80fd\u9700\u8981\u51e0\u5206\u949f\uff0c\u5177\u4f53\u53d6\u51b3\u4e8e\u89c6\u9891\u65f6\u957f\u3002',
|
||||
selectVoice: '\u9009\u62e9\u97f3\u8272',
|
||||
failedToGenerateSubtitles: '\u751f\u6210\u5b57\u5e55\u5931\u8d25\uff0c\u8bf7\u91cd\u8bd5\u6216\u68c0\u67e5 API Key\u3002',
|
||||
exceededQuota: 'Volcengine API \u914d\u989d\u5df2\u8d85\u9650\uff0c\u8bf7\u68c0\u67e5\u5957\u9910\u548c\u8d26\u5355\u4fe1\u606f\u3002',
|
||||
},
|
||||
exportModal: {
|
||||
title: '\u5bfc\u51fa\u89c6\u9891',
|
||||
noPreview: '\u6682\u65e0\u9884\u89c8',
|
||||
fileName: '\u6587\u4ef6\u540d',
|
||||
format: '\u683c\u5f0f',
|
||||
resolution: '\u5206\u8fa8\u7387',
|
||||
original: '\u539f\u59cb',
|
||||
errorDefault: '\u5bfc\u51fa\u65f6\u53d1\u751f\u9519\u8bef',
|
||||
exporting: '\u6b63\u5728\u5bfc\u51fa...',
|
||||
description: '\u6b63\u5728\u5408\u6210\u5e26\u5b57\u5e55\u548c AI \u914d\u97f3\u7684\u89c6\u9891\uff0c\u8fd9\u53ef\u80fd\u9700\u8981\u51e0\u5206\u949f\u3002',
|
||||
close: '\u5173\u95ed',
|
||||
cancel: '\u53d6\u6d88',
|
||||
downloadVideo: '\u4e0b\u8f7d\u89c6\u9891',
|
||||
processing: '\u5904\u7406\u4e2d...',
|
||||
startExport: '\u5f00\u59cb\u5bfc\u51fa',
|
||||
translatedSuffix: '_\u7ffb\u8bd1\u7248',
|
||||
},
|
||||
voiceMarket: {
|
||||
title: '\u97f3\u8272\u5e02\u573a',
|
||||
searchVoices: '\u641c\u7d22\u97f3\u8272...',
|
||||
allLanguages: '\u5168\u90e8\u8bed\u8a00',
|
||||
allGenders: '\u5168\u90e8\u6027\u522b',
|
||||
male: '\u7537',
|
||||
female: '\u5973',
|
||||
neutral: '\u4e2d\u6027',
|
||||
tip: '\u63d0\u793a\uff1a\u53ef\u4ee5\u4e3a\u5f53\u524d\u5b57\u5e55\u9009\u62e9\u97f3\u8272\uff0c\u4e5f\u53ef\u4ee5\u4e00\u952e\u5e94\u7528\u5230\u5168\u90e8\u5b57\u5e55\u3002',
|
||||
choose: '\u9009\u62e9',
|
||||
applyAll: '\u5e94\u7528\u5168\u90e8',
|
||||
noVoicesFound: '\u6ca1\u6709\u627e\u5230\u7b26\u5408\u6761\u4ef6\u7684\u97f3\u8272',
|
||||
},
|
||||
},
|
||||
en: {
|
||||
app: {
|
||||
chinese: '\u4e2d\u6587',
|
||||
english: 'English',
|
||||
productName: 'Translate 1.0',
|
||||
},
|
||||
upload: {
|
||||
uploadVideoFile: 'Upload video file',
|
||||
clickToUpload: 'Click to upload or drag files here',
|
||||
uploadVideo: 'Upload Video',
|
||||
supportedFormats: 'Supported formats: MP4 / MOV / WEBM. Maximum file size is 500MB.',
|
||||
editingMode: 'Editing Mode',
|
||||
editingModeDesc: 'Supports secondary editing and more precise translation',
|
||||
simpleMode: 'Simple Mode',
|
||||
simpleModeDesc: 'Best for quickly generating a translated video',
|
||||
subtitleDefaults: 'Subtitle Defaults',
|
||||
subtitleDefaultsDesc: 'Set the initial subtitle position and size. You can still refine them in the editor.',
|
||||
reset: 'Reset',
|
||||
subtitlePreview: 'Subtitle preview',
|
||||
subtitleInitialPosition: 'Subtitle Initial Position',
|
||||
subtitleInitialSize: 'Subtitle Initial Size',
|
||||
fromBottom: '{value}% from bottom',
|
||||
pxValue: '{value}px',
|
||||
subtitleLanguage: 'Subtitle Language',
|
||||
subtitleLanguageHint: 'Subtitle translation is fixed to English.',
|
||||
ttsLanguage: 'TTS Language',
|
||||
ttsLanguageHint: 'Choose the target language used for dubbing playback.',
|
||||
selectTranslationLanguage: 'Select Translation Language',
|
||||
detectSourceLanguage: 'AI detects the video source language automatically.',
|
||||
popular: 'Popular',
|
||||
groupABC: 'ABC',
|
||||
groupDEF: 'DEF',
|
||||
groupGHI: 'GHI',
|
||||
generateTranslatedVideo: 'Generate Translated Video',
|
||||
languages: {
|
||||
zh: 'Chinese',
|
||||
yue: 'Cantonese',
|
||||
en: 'English',
|
||||
fr: 'French',
|
||||
id: 'Indonesian',
|
||||
de: 'German',
|
||||
fil: 'Filipino',
|
||||
},
|
||||
},
|
||||
trim: {
|
||||
confirm: 'Confirm',
|
||||
selectedDuration: 'Selected duration: {value}s',
|
||||
},
|
||||
editor: {
|
||||
watchVideo: 'Watch Video',
|
||||
saveEditing: 'Save Editing',
|
||||
export: 'Export',
|
||||
aiDub: 'AI Dub',
|
||||
voiceClone: 'Voice Clone',
|
||||
tip: 'Tip: After modifying the subtitle text, you need to regenerate the dubbing to take effect.',
|
||||
llm: 'LLM',
|
||||
llmProvider: 'LLM Provider',
|
||||
generateDubbing: 'Generate Dubbing',
|
||||
separatingVocals: 'Separating Vocals...',
|
||||
generatingTts: 'Generating TTS...',
|
||||
lowPrecision: 'Low-precision subtitle timing is active for this generation. You can still edit subtitles before dubbing.',
|
||||
retryGeneration: 'Retry Generation',
|
||||
noVideoLoaded: 'No video loaded',
|
||||
bgm: 'BGM',
|
||||
applyAll: 'Apply to all subtitles',
|
||||
applyAllAria: 'Apply style changes to all subtitles',
|
||||
activeSubtitleContext: 'Active subtitle context',
|
||||
textStyles: 'Text Styles',
|
||||
selectSubtitleToEdit: 'Select a subtitle to edit its speaker details and display style.',
|
||||
subtitleFontFamily: 'Subtitle font family',
|
||||
subtitleSizePreset: 'Subtitle size preset',
|
||||
toggleBold: 'Toggle bold',
|
||||
toggleItalic: 'Toggle italic',
|
||||
toggleUnderline: 'Toggle underline',
|
||||
subtitleColor: 'Subtitle color',
|
||||
subtitleStrokeColor: 'Subtitle stroke color',
|
||||
color: 'Color',
|
||||
stroke: 'Stroke',
|
||||
small: 'Small',
|
||||
normal: 'Normal',
|
||||
large: 'Large',
|
||||
alignLeft: 'Align left',
|
||||
alignCenter: 'Align center',
|
||||
alignRight: 'Align right',
|
||||
videoTrack: 'Video Track',
|
||||
stretchDubbing: 'Stretch the dubbing to control the speed',
|
||||
qualityQueued: 'Queued',
|
||||
aiAnalyzing: 'AI is analyzing and translating...',
|
||||
thisMayTakeMinutes: 'This may take a few minutes depending on the video length.',
|
||||
selectVoice: 'Select Voice',
|
||||
failedToGenerateSubtitles: 'Failed to generate subtitles. Please try again or check your API key.',
|
||||
exceededQuota: 'You have exceeded your Volcengine API quota. Please check your plan and billing details.',
|
||||
},
|
||||
exportModal: {
|
||||
title: 'Export Video',
|
||||
noPreview: 'No Preview',
|
||||
fileName: 'File Name',
|
||||
format: 'Format',
|
||||
resolution: 'Resolution',
|
||||
original: 'Original',
|
||||
errorDefault: 'An error occurred during export',
|
||||
exporting: 'Exporting...',
|
||||
description: 'Compositing video with subtitles and AI dubbing. This may take a few minutes depending on the video length.',
|
||||
close: 'Close',
|
||||
cancel: 'Cancel',
|
||||
downloadVideo: 'Download Video',
|
||||
processing: 'Processing...',
|
||||
startExport: 'Start Export',
|
||||
translatedSuffix: '_translated',
|
||||
},
|
||||
voiceMarket: {
|
||||
title: 'Voice Market',
|
||||
searchVoices: 'Search voices...',
|
||||
allLanguages: 'All Languages',
|
||||
allGenders: 'All Genders',
|
||||
male: 'Male',
|
||||
female: 'Female',
|
||||
neutral: 'Neutral',
|
||||
tip: 'Tip: Select a voice for the current subtitle, or apply it to all subtitles.',
|
||||
choose: 'Choose',
|
||||
applyAll: 'Apply All',
|
||||
noVoicesFound: 'No voices found matching your criteria',
|
||||
},
|
||||
},
|
||||
} as const;
|
||||
|
||||
type MessageTree = typeof messages.zh;
|
||||
|
||||
const formatMessage = (template: string, values?: Record<string, string | number>) => {
|
||||
if (!values) return template;
|
||||
return Object.entries(values).reduce(
|
||||
(result, [key, value]) => result.replaceAll(`{${key}}`, String(value)),
|
||||
template,
|
||||
);
|
||||
};
|
||||
|
||||
interface I18nContextValue {
|
||||
locale: Locale;
|
||||
setLocale: (locale: Locale) => void;
|
||||
m: MessageTree;
|
||||
format: (template: string, values?: Record<string, string | number>) => string;
|
||||
}
|
||||
|
||||
const I18nContext = createContext<I18nContextValue | null>(null);
|
||||
|
||||
export const I18nProvider = ({
|
||||
children,
|
||||
initialLocale = 'zh',
|
||||
}: {
|
||||
children: React.ReactNode;
|
||||
initialLocale?: Locale;
|
||||
}) => {
|
||||
const [locale, setLocale] = useState<Locale>(initialLocale);
|
||||
|
||||
const value = useMemo<I18nContextValue>(
|
||||
() => ({
|
||||
locale,
|
||||
setLocale,
|
||||
m: messages[locale],
|
||||
format: formatMessage,
|
||||
}),
|
||||
[locale],
|
||||
);
|
||||
|
||||
return <I18nContext.Provider value={value}>{children}</I18nContext.Provider>;
|
||||
};
|
||||
|
||||
export const useI18n = () => {
|
||||
const context = useContext(I18nContext);
|
||||
if (!context) {
|
||||
throw new Error('useI18n must be used within I18nProvider');
|
||||
}
|
||||
|
||||
return context;
|
||||
};
|
||||
@ -23,6 +23,8 @@ describe('buildExportPayload', () => {
|
||||
fontSize: 24,
|
||||
color: '#FFFFFF',
|
||||
backgroundColor: 'transparent',
|
||||
strokeColor: '#000000',
|
||||
strokeWidth: 2,
|
||||
alignment: 'center',
|
||||
isBold: true,
|
||||
isItalic: false,
|
||||
@ -70,6 +72,8 @@ describe('buildExportPayload', () => {
|
||||
fontSize: 18,
|
||||
color: '#FF0000',
|
||||
backgroundColor: 'transparent',
|
||||
strokeColor: '#000000',
|
||||
strokeWidth: 2,
|
||||
alignment: 'left',
|
||||
isBold: false,
|
||||
isItalic: true,
|
||||
|
||||
@ -51,7 +51,7 @@ describe('doubaoTranslation', () => {
|
||||
endTime: 1,
|
||||
},
|
||||
],
|
||||
model: 'doubao-seed-2-0-pro-260215',
|
||||
model: 'doubao-seed-2-0-lite-260215',
|
||||
requestResponse,
|
||||
});
|
||||
|
||||
@ -86,7 +86,7 @@ describe('doubaoTranslation', () => {
|
||||
endTime: 1,
|
||||
},
|
||||
],
|
||||
model: 'doubao-seed-2-0-pro-260215',
|
||||
model: 'doubao-seed-2-0-lite-260215',
|
||||
requestResponse,
|
||||
});
|
||||
|
||||
|
||||
@ -1,5 +1,19 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { formatLogContext, serializeError } from './errorLogging';
|
||||
import fs from 'fs';
|
||||
import os from 'os';
|
||||
import path from 'path';
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
import { appendLogEntry, formatLogContext, resolveLogFilePath, serializeError } from './errorLogging';
|
||||
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
afterEach(() => {
|
||||
while (tempDirs.length > 0) {
|
||||
const tempDir = tempDirs.pop();
|
||||
if (tempDir && fs.existsSync(tempDir)) {
|
||||
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
describe('errorLogging', () => {
|
||||
it('serializes nested causes and error codes', () => {
|
||||
@ -33,4 +47,47 @@ describe('errorLogging', () => {
|
||||
}),
|
||||
).toBe('requestId=req-1 provider=doubao durationMs=1234');
|
||||
});
|
||||
|
||||
it('defaults log file path to logs/server.log under the current working directory', () => {
|
||||
expect(resolveLogFilePath({} as NodeJS.ProcessEnv)).toBe(
|
||||
path.join(process.cwd(), 'logs', 'server.log'),
|
||||
);
|
||||
});
|
||||
|
||||
it('appends structured log entries to a file path from env', () => {
|
||||
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'subtitle-logs-'));
|
||||
tempDirs.push(tempDir);
|
||||
const logFilePath = path.join(tempDir, 'server.log');
|
||||
|
||||
appendLogEntry({
|
||||
level: 'info',
|
||||
message: 'subtitle request started',
|
||||
context: {
|
||||
requestId: 'req-123',
|
||||
provider: 'doubao',
|
||||
},
|
||||
details: {
|
||||
fileId: 'file-123',
|
||||
},
|
||||
env: {
|
||||
LOG_FILE_PATH: logFilePath,
|
||||
} as NodeJS.ProcessEnv,
|
||||
});
|
||||
|
||||
const written = fs.readFileSync(logFilePath, 'utf8').trim();
|
||||
const parsed = JSON.parse(written);
|
||||
|
||||
expect(parsed).toMatchObject({
|
||||
level: 'info',
|
||||
message: 'subtitle request started',
|
||||
context: {
|
||||
requestId: 'req-123',
|
||||
provider: 'doubao',
|
||||
},
|
||||
details: {
|
||||
fileId: 'file-123',
|
||||
},
|
||||
});
|
||||
expect(typeof parsed.timestamp).toBe('string');
|
||||
});
|
||||
});
|
||||
|
||||
@ -1,3 +1,6 @@
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
export interface ErrorLogDetails {
|
||||
message: string;
|
||||
stack?: string;
|
||||
@ -6,6 +9,87 @@ export interface ErrorLogDetails {
|
||||
name?: string;
|
||||
}
|
||||
|
||||
export type LogLevel = 'info' | 'warn' | 'error';
|
||||
|
||||
export interface LogEntry {
|
||||
timestamp: string;
|
||||
level: LogLevel;
|
||||
message: string;
|
||||
context?: Record<string, unknown>;
|
||||
details?: unknown;
|
||||
pid: number;
|
||||
}
|
||||
|
||||
export const resolveLogFilePath = (env: NodeJS.ProcessEnv = process.env) =>
|
||||
env.LOG_FILE_PATH?.trim() || path.join(process.cwd(), 'logs', 'server.log');
|
||||
|
||||
export const appendLogEntry = ({
|
||||
level,
|
||||
message,
|
||||
context,
|
||||
details,
|
||||
env = process.env,
|
||||
}: {
|
||||
level: LogLevel;
|
||||
message: string;
|
||||
context?: Record<string, unknown>;
|
||||
details?: unknown;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
}) => {
|
||||
const logFilePath = resolveLogFilePath(env);
|
||||
const logDir = path.dirname(logFilePath);
|
||||
const entry: LogEntry = {
|
||||
timestamp: new Date().toISOString(),
|
||||
level,
|
||||
message,
|
||||
pid: process.pid,
|
||||
...(context ? { context } : {}),
|
||||
...(details !== undefined ? { details } : {}),
|
||||
};
|
||||
|
||||
fs.mkdirSync(logDir, { recursive: true });
|
||||
fs.appendFileSync(logFilePath, `${JSON.stringify(entry)}\n`, 'utf8');
|
||||
};
|
||||
|
||||
export const logEvent = ({
|
||||
level,
|
||||
message,
|
||||
context,
|
||||
details,
|
||||
env = process.env,
|
||||
}: {
|
||||
level: LogLevel;
|
||||
message: string;
|
||||
context?: Record<string, unknown>;
|
||||
details?: unknown;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
}) => {
|
||||
try {
|
||||
appendLogEntry({ level, message, context, details, env });
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`[log] failed to write file log ${formatLogContext({
|
||||
message,
|
||||
configuredPath: resolveLogFilePath(env),
|
||||
})}`,
|
||||
serializeError(error),
|
||||
);
|
||||
}
|
||||
|
||||
const consoleMethod = level === 'error' ? console.error : level === 'warn' ? console.warn : console.info;
|
||||
if (details !== undefined) {
|
||||
consoleMethod(message, context || {}, details);
|
||||
return;
|
||||
}
|
||||
|
||||
if (context) {
|
||||
consoleMethod(message, context);
|
||||
return;
|
||||
}
|
||||
|
||||
consoleMethod(message);
|
||||
};
|
||||
|
||||
export const serializeError = (error: unknown): ErrorLogDetails => {
|
||||
if (error instanceof Error) {
|
||||
const details: ErrorLogDetails = {
|
||||
|
||||
@ -11,6 +11,8 @@ const defaultTextStyles: TextStyles = {
|
||||
fontSize: 24,
|
||||
color: '#FFFFFF',
|
||||
backgroundColor: 'transparent',
|
||||
strokeColor: '#000000',
|
||||
strokeWidth: 2,
|
||||
alignment: 'center',
|
||||
isBold: true,
|
||||
isItalic: false,
|
||||
|
||||
@ -6,6 +6,8 @@ export const DEFAULT_EXPORT_TEXT_STYLES: TextStyles = {
|
||||
fontSize: 24,
|
||||
color: '#FFFFFF',
|
||||
backgroundColor: 'transparent',
|
||||
strokeColor: '#000000',
|
||||
strokeWidth: 2,
|
||||
alignment: 'center',
|
||||
isBold: false,
|
||||
isItalic: false,
|
||||
@ -105,7 +107,7 @@ export const buildAssSubtitleContent = ({
|
||||
textStyles.fontSize,
|
||||
toAssColor(textStyles.color),
|
||||
'&H00000000',
|
||||
'&H00000000',
|
||||
toAssColor(textStyles.strokeColor),
|
||||
'&H64000000',
|
||||
textStyles.isBold ? -1 : 0,
|
||||
textStyles.isItalic ? -1 : 0,
|
||||
@ -116,7 +118,7 @@ export const buildAssSubtitleContent = ({
|
||||
0,
|
||||
0,
|
||||
1,
|
||||
1,
|
||||
Math.max(1, textStyles.strokeWidth),
|
||||
2,
|
||||
mapAlignmentToAss(textStyles.alignment),
|
||||
48,
|
||||
|
||||
@ -14,7 +14,7 @@ describe('createSentenceTranslator', () => {
|
||||
const translator = createSentenceTranslator({
|
||||
provider: 'doubao',
|
||||
apiKey: 'ark-key',
|
||||
model: 'doubao-seed-2-0-pro-260215',
|
||||
model: 'doubao-seed-2-0-lite-260215',
|
||||
baseUrl: 'https://ark.cn-beijing.volces.com/api/v3/responses',
|
||||
timeoutMs: 600000,
|
||||
});
|
||||
|
||||
@ -132,4 +132,34 @@ describe('generateSubtitlePipeline', () => {
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('passes tts language through to video subtitle generation', async () => {
|
||||
const subtitleResult: SubtitlePipelineResult = {
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
targetLanguage: 'English',
|
||||
};
|
||||
const generateSubtitlesFromVideo = vi.fn(async () => subtitleResult);
|
||||
|
||||
await generateSubtitlePipeline({
|
||||
videoPath: 'clip.mp4',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'fr',
|
||||
provider: 'doubao',
|
||||
env: {
|
||||
ARK_API_KEY: 'ark-key',
|
||||
},
|
||||
deps: {
|
||||
generateSubtitlesFromVideo,
|
||||
},
|
||||
} as any);
|
||||
|
||||
expect(generateSubtitlesFromVideo).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'fr',
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@ -1,4 +1,6 @@
|
||||
import { SubtitleGenerationProgress } from '../types';
|
||||
import { resolveAudioPipelineConfig } from './audioPipelineConfig';
|
||||
import { logEvent, serializeError } from './errorLogging';
|
||||
import { resolveLlmProviderConfig, normalizeLlmProvider } from './llmProvider';
|
||||
import { generateSubtitlesFromVideo as defaultGenerateSubtitlesFromVideo } from './videoSubtitleGeneration';
|
||||
|
||||
@ -6,10 +8,12 @@ export interface GenerateSubtitlePipelineOptions {
|
||||
videoPath?: string;
|
||||
fileId?: string;
|
||||
targetLanguage: string;
|
||||
ttsLanguage?: string;
|
||||
provider?: string | null;
|
||||
env: NodeJS.ProcessEnv;
|
||||
fetchImpl?: typeof fetch;
|
||||
requestId?: string;
|
||||
onProgress?: (progress: Omit<SubtitleGenerationProgress, 'jobId' | 'requestId'>) => void;
|
||||
deps?: {
|
||||
generateSubtitlesFromVideo?: typeof defaultGenerateSubtitlesFromVideo;
|
||||
};
|
||||
@ -19,16 +23,19 @@ export const generateSubtitlePipeline = async ({
|
||||
videoPath,
|
||||
fileId,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
provider,
|
||||
env,
|
||||
fetchImpl,
|
||||
requestId,
|
||||
onProgress,
|
||||
deps,
|
||||
}: GenerateSubtitlePipelineOptions) => {
|
||||
if (!videoPath && !fileId) {
|
||||
throw new Error('A video upload or fileId is required.');
|
||||
}
|
||||
|
||||
const startedAt = Date.now();
|
||||
const audioPipelineConfig = resolveAudioPipelineConfig(env);
|
||||
const selectedProvider = provider
|
||||
? normalizeLlmProvider(provider)
|
||||
@ -37,12 +44,75 @@ export const generateSubtitlePipeline = async ({
|
||||
const generateSubtitlesFromVideo =
|
||||
deps?.generateSubtitlesFromVideo || defaultGenerateSubtitlesFromVideo;
|
||||
|
||||
return generateSubtitlesFromVideo({
|
||||
providerConfig,
|
||||
videoPath,
|
||||
fileId,
|
||||
targetLanguage,
|
||||
requestId,
|
||||
...(fetchImpl ? { fetchImpl } : {}),
|
||||
onProgress?.({
|
||||
status: 'running',
|
||||
stage: 'preparing',
|
||||
progress: 30,
|
||||
message: 'Preparing subtitle generation',
|
||||
});
|
||||
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: '[subtitle] pipeline configured',
|
||||
context: {
|
||||
requestId,
|
||||
requestedProvider: provider || undefined,
|
||||
selectedProvider,
|
||||
targetLanguage,
|
||||
hasVideoPath: Boolean(videoPath),
|
||||
hasFileId: Boolean(fileId),
|
||||
},
|
||||
});
|
||||
|
||||
try {
|
||||
onProgress?.({
|
||||
status: 'running',
|
||||
stage: 'calling_provider',
|
||||
progress: 70,
|
||||
message: 'Calling subtitle provider',
|
||||
});
|
||||
|
||||
const result = await generateSubtitlesFromVideo({
|
||||
providerConfig,
|
||||
videoPath,
|
||||
fileId,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
requestId,
|
||||
...(fetchImpl ? { fetchImpl } : {}),
|
||||
});
|
||||
|
||||
onProgress?.({
|
||||
status: 'running',
|
||||
stage: 'processing_result',
|
||||
progress: 90,
|
||||
message: 'Processing subtitle result',
|
||||
});
|
||||
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: '[subtitle] pipeline completed',
|
||||
context: {
|
||||
requestId,
|
||||
provider: providerConfig.provider,
|
||||
durationMs: Date.now() - startedAt,
|
||||
subtitleCount: result.subtitles.length,
|
||||
quality: result.quality,
|
||||
},
|
||||
});
|
||||
|
||||
return result;
|
||||
} catch (error) {
|
||||
logEvent({
|
||||
level: 'error',
|
||||
message: '[subtitle] pipeline failed',
|
||||
context: {
|
||||
requestId,
|
||||
provider: providerConfig.provider,
|
||||
durationMs: Date.now() - startedAt,
|
||||
},
|
||||
details: serializeError(error),
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
|
||||
59
src/server/subtitleJobs.test.ts
Normal file
59
src/server/subtitleJobs.test.ts
Normal file
@ -0,0 +1,59 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
createSubtitleJobStore,
|
||||
createSubtitleJob,
|
||||
pruneExpiredSubtitleJobs,
|
||||
toSubtitleJobResponse,
|
||||
} from './subtitleJobs';
|
||||
|
||||
describe('subtitleJobs', () => {
|
||||
it('creates jobs with queued defaults', () => {
|
||||
const store = createSubtitleJobStore();
|
||||
|
||||
const job = createSubtitleJob(store, {
|
||||
requestId: 'req-1',
|
||||
provider: 'doubao',
|
||||
targetLanguage: 'English',
|
||||
fileId: 'file-123',
|
||||
});
|
||||
|
||||
expect(job).toMatchObject({
|
||||
requestId: 'req-1',
|
||||
provider: 'doubao',
|
||||
targetLanguage: 'English',
|
||||
fileId: 'file-123',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
});
|
||||
});
|
||||
|
||||
it('removes expired jobs past the ttl', () => {
|
||||
vi.useFakeTimers();
|
||||
const store = createSubtitleJobStore();
|
||||
|
||||
createSubtitleJob(store, {
|
||||
requestId: 'req-1',
|
||||
provider: 'doubao',
|
||||
targetLanguage: 'English',
|
||||
});
|
||||
|
||||
vi.advanceTimersByTime(60 * 60 * 1000 + 1);
|
||||
pruneExpiredSubtitleJobs(store, 60 * 60 * 1000);
|
||||
|
||||
expect(store.jobs.size).toBe(0);
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it('hides file paths from api responses', () => {
|
||||
const store = createSubtitleJobStore();
|
||||
const job = createSubtitleJob(store, {
|
||||
requestId: 'req-1',
|
||||
provider: 'gemini',
|
||||
targetLanguage: 'English',
|
||||
filePath: 'uploads/temp-file.mp4',
|
||||
});
|
||||
|
||||
expect(toSubtitleJobResponse(job)).not.toHaveProperty('filePath');
|
||||
});
|
||||
});
|
||||
168
src/server/subtitleJobs.ts
Normal file
168
src/server/subtitleJobs.ts
Normal file
@ -0,0 +1,168 @@
|
||||
import { SubtitleGenerationProgress, SubtitleJobStage, SubtitleJobStatus, SubtitlePipelineResult } from '../types';
|
||||
|
||||
const DEFAULT_PROGRESS_BY_STAGE: Record<SubtitleJobStage, number> = {
|
||||
queued: 5,
|
||||
upload_received: 15,
|
||||
preparing: 30,
|
||||
calling_provider: 70,
|
||||
processing_result: 90,
|
||||
succeeded: 100,
|
||||
failed: 0,
|
||||
};
|
||||
|
||||
export interface SubtitleJob {
|
||||
id: string;
|
||||
requestId: string;
|
||||
status: SubtitleJobStatus;
|
||||
stage: SubtitleJobStage;
|
||||
progress: number;
|
||||
message: string;
|
||||
createdAt: number;
|
||||
updatedAt: number;
|
||||
provider?: string | null;
|
||||
targetLanguage: string;
|
||||
ttsLanguage?: string;
|
||||
fileId?: string;
|
||||
filePath?: string;
|
||||
error?: string;
|
||||
result?: SubtitlePipelineResult & {
|
||||
provider?: string | null;
|
||||
requestId: string;
|
||||
};
|
||||
}
|
||||
|
||||
export interface SubtitleJobStore {
|
||||
jobs: Map<string, SubtitleJob>;
|
||||
nextId: number;
|
||||
}
|
||||
|
||||
export const createSubtitleJobStore = (): SubtitleJobStore => ({
|
||||
jobs: new Map(),
|
||||
nextId: 1,
|
||||
});
|
||||
|
||||
const createSubtitleJobId = (store: SubtitleJobStore) => {
|
||||
const id = `subtitle-job-${Date.now()}-${store.nextId}`;
|
||||
store.nextId += 1;
|
||||
return id;
|
||||
};
|
||||
|
||||
const defaultMessageForStage = (stage: SubtitleJobStage) => {
|
||||
switch (stage) {
|
||||
case 'queued':
|
||||
return 'Queued';
|
||||
case 'upload_received':
|
||||
return 'Upload received';
|
||||
case 'preparing':
|
||||
return 'Preparing subtitle generation';
|
||||
case 'calling_provider':
|
||||
return 'Calling subtitle provider';
|
||||
case 'processing_result':
|
||||
return 'Processing subtitle result';
|
||||
case 'succeeded':
|
||||
return 'Subtitle generation completed';
|
||||
case 'failed':
|
||||
return 'Subtitle generation failed';
|
||||
}
|
||||
};
|
||||
|
||||
export const createSubtitleJob = (
|
||||
store: SubtitleJobStore,
|
||||
{
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
fileId,
|
||||
filePath,
|
||||
}: {
|
||||
requestId: string;
|
||||
provider?: string | null;
|
||||
targetLanguage: string;
|
||||
ttsLanguage?: string;
|
||||
fileId?: string;
|
||||
filePath?: string;
|
||||
},
|
||||
): SubtitleJob => {
|
||||
const now = Date.now();
|
||||
const job: SubtitleJob = {
|
||||
id: createSubtitleJobId(store),
|
||||
requestId,
|
||||
provider,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
fileId,
|
||||
filePath,
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: DEFAULT_PROGRESS_BY_STAGE.queued,
|
||||
message: defaultMessageForStage('queued'),
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
};
|
||||
store.jobs.set(job.id, job);
|
||||
return job;
|
||||
};
|
||||
|
||||
export const getSubtitleJob = (store: SubtitleJobStore, jobId: string) => store.jobs.get(jobId);
|
||||
|
||||
export const updateSubtitleJob = (
|
||||
store: SubtitleJobStore,
|
||||
jobId: string,
|
||||
updates: Partial<SubtitleJob> & {
|
||||
stage?: SubtitleJobStage;
|
||||
status?: SubtitleJobStatus;
|
||||
message?: string;
|
||||
progress?: number;
|
||||
},
|
||||
) => {
|
||||
const job = store.jobs.get(jobId);
|
||||
if (!job) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const nextStage = updates.stage || job.stage;
|
||||
const nextStatus = updates.status || job.status;
|
||||
const nextProgress =
|
||||
updates.progress !== undefined
|
||||
? updates.progress
|
||||
: nextStage !== job.stage
|
||||
? DEFAULT_PROGRESS_BY_STAGE[nextStage]
|
||||
: job.progress;
|
||||
|
||||
const updated: SubtitleJob = {
|
||||
...job,
|
||||
...updates,
|
||||
stage: nextStage,
|
||||
status: nextStatus,
|
||||
progress: nextProgress,
|
||||
message: updates.message || (updates.stage ? defaultMessageForStage(nextStage) : job.message),
|
||||
updatedAt: Date.now(),
|
||||
};
|
||||
|
||||
store.jobs.set(jobId, updated);
|
||||
return updated;
|
||||
};
|
||||
|
||||
export const pruneExpiredSubtitleJobs = (store: SubtitleJobStore, ttlMs: number, now = Date.now()) => {
|
||||
for (const [jobId, job] of store.jobs.entries()) {
|
||||
if (now - job.updatedAt > ttlMs) {
|
||||
store.jobs.delete(jobId);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
export const toSubtitleGenerationProgress = (job: SubtitleJob): SubtitleGenerationProgress => ({
|
||||
jobId: job.id,
|
||||
requestId: job.requestId,
|
||||
status: job.status,
|
||||
stage: job.stage,
|
||||
progress: job.progress,
|
||||
message: job.message,
|
||||
});
|
||||
|
||||
export const toSubtitleJobResponse = (job: SubtitleJob) => ({
|
||||
...toSubtitleGenerationProgress(job),
|
||||
...(job.error ? { error: job.error } : {}),
|
||||
...(job.result ? { result: job.result } : {}),
|
||||
});
|
||||
@ -6,6 +6,7 @@ describe('parseSubtitleRequest', () => {
|
||||
expect(parseSubtitleRequest({ targetLanguage: 'English' })).toEqual({
|
||||
provider: 'doubao',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'English',
|
||||
});
|
||||
});
|
||||
|
||||
@ -18,6 +19,7 @@ describe('parseSubtitleRequest', () => {
|
||||
).toEqual({
|
||||
provider: 'gemini',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'English',
|
||||
});
|
||||
});
|
||||
|
||||
@ -36,7 +38,21 @@ describe('parseSubtitleRequest', () => {
|
||||
).toEqual({
|
||||
provider: 'doubao',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'English',
|
||||
fileId: 'file-123',
|
||||
});
|
||||
});
|
||||
|
||||
it('preserves a tts language override when provided', () => {
|
||||
expect(
|
||||
parseSubtitleRequest({
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'fr',
|
||||
} as any),
|
||||
).toEqual({
|
||||
provider: 'doubao',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'fr',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@ -3,12 +3,14 @@ import { LlmProvider, normalizeLlmProvider } from './llmProvider';
|
||||
export interface SubtitleRequestBody {
|
||||
provider?: string | null;
|
||||
targetLanguage?: string | null;
|
||||
ttsLanguage?: string | null;
|
||||
fileId?: string | null;
|
||||
}
|
||||
|
||||
export interface ParsedSubtitleRequest {
|
||||
provider: LlmProvider;
|
||||
targetLanguage: string;
|
||||
ttsLanguage: string;
|
||||
fileId?: string;
|
||||
}
|
||||
|
||||
@ -20,9 +22,12 @@ export const parseSubtitleRequest = (
|
||||
throw new Error('Target language is required.');
|
||||
}
|
||||
|
||||
const ttsLanguage = body.ttsLanguage?.trim() || targetLanguage;
|
||||
|
||||
return {
|
||||
provider: normalizeLlmProvider(body.provider),
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
...(body.fileId?.trim() ? { fileId: body.fileId.trim() } : {}),
|
||||
};
|
||||
};
|
||||
|
||||
@ -20,8 +20,10 @@ describe('generateSubtitlesFromVideo', () => {
|
||||
sourceLanguage: 'zh',
|
||||
subtitles: [
|
||||
{
|
||||
originalText: '你好',
|
||||
originalText: 'hello there',
|
||||
translatedText: 'Hello',
|
||||
ttsText: 'Bonjour',
|
||||
ttsLanguage: 'fr',
|
||||
startTime: 0,
|
||||
endTime: 1,
|
||||
},
|
||||
@ -49,8 +51,9 @@ describe('generateSubtitlesFromVideo', () => {
|
||||
},
|
||||
videoPath: 'clip.mp4',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'fr',
|
||||
fetchImpl,
|
||||
});
|
||||
} as any);
|
||||
|
||||
expect(fetchImpl).toHaveBeenCalledWith(
|
||||
'https://ark.cn-beijing.volces.com/api/v3/responses',
|
||||
@ -73,8 +76,10 @@ describe('generateSubtitlesFromVideo', () => {
|
||||
sourceLanguage: 'zh',
|
||||
subtitles: [
|
||||
{
|
||||
originalText: '你好',
|
||||
originalText: 'hello there',
|
||||
translatedText: 'Hello',
|
||||
ttsText: 'Bonjour',
|
||||
ttsLanguage: 'fr',
|
||||
startTime: 0,
|
||||
endTime: 1,
|
||||
},
|
||||
@ -102,8 +107,9 @@ describe('generateSubtitlesFromVideo', () => {
|
||||
},
|
||||
fileId: 'file-123',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'fr',
|
||||
fetchImpl,
|
||||
});
|
||||
} as any);
|
||||
|
||||
const [, request] = fetchImpl.mock.calls[0] as [string, RequestInit];
|
||||
const payload = JSON.parse(String(request.body));
|
||||
@ -112,6 +118,12 @@ describe('generateSubtitlesFromVideo', () => {
|
||||
expect(payload.input[0].content[0].type).toBe('input_text');
|
||||
expect(payload.input[0].content[0].text).toContain('# Role');
|
||||
expect(payload.input[0].content[0].text).toContain('Voice Selection');
|
||||
expect(payload.input[0].content[0].text).toContain('# Output Contract');
|
||||
expect(payload.input[0].content[0].text).toContain('The first character of your response must be {.');
|
||||
expect(payload.input[0].content[0].text).toContain('The last character of your response must be }.');
|
||||
expect(payload.input[0].content[0].text).toContain('"ttsText":');
|
||||
expect(payload.input[0].content[0].text).toContain('"ttsLanguage":');
|
||||
expect(payload.input[0].content[0].text).toContain('translatedText must always be English');
|
||||
|
||||
expect(payload.input[1].role).toBe('user');
|
||||
expect(payload.input[1].content[0]).toEqual({
|
||||
@ -119,8 +131,55 @@ describe('generateSubtitlesFromVideo', () => {
|
||||
file_id: 'file-123',
|
||||
});
|
||||
expect(payload.input[1].content[1].type).toBe('input_text');
|
||||
expect(payload.input[1].content[1].text).toContain('Target language: English');
|
||||
expect(payload.input[1].content[1].text).toContain('Available voices');
|
||||
expect(payload.input[1].content[1].text).toContain('Sweet_Girl');
|
||||
expect(payload.input[1].content[1].text).toContain('Subtitle language: English');
|
||||
expect(payload.input[1].content[1].text).toContain('TTS language: fr');
|
||||
expect(payload.input[1].content[1].text).toContain('Available voices for the TTS language');
|
||||
});
|
||||
|
||||
it('extracts tts fields when doubao returns prose around a fenced payload', async () => {
|
||||
const fetchImpl = vi.fn<typeof fetch>(async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
output: [
|
||||
{
|
||||
content: [
|
||||
{
|
||||
text: 'Here is the JSON result:\n```json\n{"sourceLanguage":"zh","subtitles":[{"originalText":"hello there","translatedText":"Hello","ttsText":"Bonjour","ttsLanguage":"fr","startTime":0,"endTime":1,"speaker":"Speaker 1","voiceId":"male-qn-qingse"}]}\n```\nUse it directly.',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
},
|
||||
),
|
||||
);
|
||||
|
||||
const result = await generateSubtitlesFromVideo({
|
||||
providerConfig: {
|
||||
provider: 'doubao',
|
||||
apiKey: 'ark-key',
|
||||
model: 'doubao-seed-2-0-pro-260215',
|
||||
baseUrl: 'https://ark.cn-beijing.volces.com/api/v3/responses',
|
||||
timeoutMs: 600000,
|
||||
},
|
||||
fileId: 'file-123',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'fr',
|
||||
fetchImpl,
|
||||
} as any);
|
||||
|
||||
expect(result.sourceLanguage).toBe('zh');
|
||||
expect(result.subtitles).toHaveLength(1);
|
||||
expect(result.subtitles[0]).toMatchObject({
|
||||
originalText: 'hello there',
|
||||
translatedText: 'Hello',
|
||||
ttsText: 'Bonjour',
|
||||
ttsLanguage: 'fr',
|
||||
speaker: 'Speaker 1',
|
||||
voiceId: 'male-qn-qingse',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@ -2,7 +2,7 @@ import fs from 'fs';
|
||||
import { GoogleGenAI } from '@google/genai';
|
||||
import { SubtitlePipelineResult } from '../types';
|
||||
import { MINIMAX_VOICES } from '../voices';
|
||||
import { formatLogContext, serializeError } from './errorLogging';
|
||||
import { formatLogContext, logEvent, serializeError } from './errorLogging';
|
||||
import { DoubaoProviderConfig, GeminiProviderConfig, LlmProviderConfig } from './llmProvider';
|
||||
|
||||
interface RawModelSubtitle {
|
||||
@ -11,6 +11,8 @@ interface RawModelSubtitle {
|
||||
endTime?: number | string;
|
||||
originalText?: string;
|
||||
translatedText?: string;
|
||||
ttsText?: string;
|
||||
ttsLanguage?: string;
|
||||
speaker?: string;
|
||||
voiceId?: string;
|
||||
}
|
||||
@ -25,18 +27,78 @@ const SUPPORTED_VOICE_IDS = new Set(MINIMAX_VOICES.map((voice) => voice.id));
|
||||
|
||||
const stripJsonFences = (text: string) => text.replace(/```json\n?|\n?```/g, '').trim();
|
||||
|
||||
const extractBalancedJsonBlock = (text: string) => {
|
||||
const startIndexes = [text.indexOf('{'), text.indexOf('[')].filter((index) => index >= 0);
|
||||
const start = startIndexes.length > 0 ? Math.min(...startIndexes) : -1;
|
||||
if (start < 0) {
|
||||
return text;
|
||||
}
|
||||
|
||||
const opening = text[start];
|
||||
const closing = opening === '{' ? '}' : ']';
|
||||
let depth = 0;
|
||||
let inString = false;
|
||||
let escaped = false;
|
||||
|
||||
for (let index = start; index < text.length; index += 1) {
|
||||
const char = text[index];
|
||||
|
||||
if (inString) {
|
||||
if (escaped) {
|
||||
escaped = false;
|
||||
} else if (char === '\\') {
|
||||
escaped = true;
|
||||
} else if (char === '"') {
|
||||
inString = false;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (char === '"') {
|
||||
inString = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (char === opening) {
|
||||
depth += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (char === closing) {
|
||||
depth -= 1;
|
||||
if (depth === 0) {
|
||||
return text.slice(start, index + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return text.slice(start);
|
||||
};
|
||||
|
||||
const extractJson = (text: string): RawModelResponse => {
|
||||
const cleaned = stripJsonFences(text);
|
||||
if (!cleaned) {
|
||||
return { subtitles: [] };
|
||||
}
|
||||
|
||||
const parsed = JSON.parse(cleaned);
|
||||
if (Array.isArray(parsed)) {
|
||||
return { subtitles: parsed };
|
||||
}
|
||||
const directCandidate = cleaned.replace(/^\uFEFF/, '').trim();
|
||||
|
||||
return parsed as RawModelResponse;
|
||||
try {
|
||||
const parsed = JSON.parse(directCandidate);
|
||||
if (Array.isArray(parsed)) {
|
||||
return { subtitles: parsed };
|
||||
}
|
||||
|
||||
return parsed as RawModelResponse;
|
||||
} catch {
|
||||
const extractedCandidate = extractBalancedJsonBlock(directCandidate).trim();
|
||||
const parsed = JSON.parse(extractedCandidate);
|
||||
if (Array.isArray(parsed)) {
|
||||
return { subtitles: parsed };
|
||||
}
|
||||
|
||||
return parsed as RawModelResponse;
|
||||
}
|
||||
};
|
||||
|
||||
const toSeconds = (value: unknown, fallback: number) => {
|
||||
@ -93,12 +155,13 @@ const formatVoiceCatalogForPrompt = (targetLanguage: string) => {
|
||||
|
||||
const createSystemPrompt = () => `# Role
|
||||
You are a senior film and TV subtitle expert and an advanced localization translator.
|
||||
You deeply understand screen reading experience.
|
||||
Subtitles must be short, easy to read, precisely timed to the visuals and speech rhythm, and must never cause viewer reading fatigue.
|
||||
You deeply understand subtitle readability, dubbing naturalness, and multilingual localization for short-form video.
|
||||
|
||||
# Task
|
||||
Listen to and watch the user-provided audio or video.
|
||||
Transcribe the spoken content and translate it into the target language specified by the user.
|
||||
Transcribe the spoken content.
|
||||
Translate every subtitle into English for on-screen subtitles.
|
||||
Also translate every subtitle into the user-specified TTS language for dubbing playback.
|
||||
Extract highly accurate start and end timestamps, speaker labels, and speaker gender.
|
||||
Select the most suitable voiceId for each subtitle item by matching the speaker's gender, tone, style, and delivery to the voice options provided by the user.
|
||||
Return the result strictly in the required JSON format.
|
||||
@ -110,9 +173,8 @@ You must split subtitles according to the speaker's actual breathing, pauses, co
|
||||
|
||||
2. Screen-Friendly Length:
|
||||
Each subtitle item must be short.
|
||||
Chinese text should ideally stay within 15 to 20 characters.
|
||||
English text should ideally stay within 7 to 10 words.
|
||||
If a sentence is too long, you must split it into multiple subtitle objects with consecutive timestamps.
|
||||
English subtitle text should ideally stay within 7 to 10 words.
|
||||
If a sentence is too long, split it into multiple subtitle objects with consecutive timestamps.
|
||||
|
||||
3. Highly Precise Timestamps:
|
||||
Timestamps must align closely with the actual speech.
|
||||
@ -123,18 +185,27 @@ The duration of a single subtitle item should usually not exceed 3 to 5 seconds.
|
||||
Accurately identify the speaker label and speaker gender.
|
||||
Gender must be either "male" or "female".
|
||||
|
||||
5. Voice Selection:
|
||||
The user will provide the target language and a list of available voices.
|
||||
5. Translation Rules:
|
||||
- translatedText must always be English subtitle text for on-screen display.
|
||||
- ttsText must be translated into the user-provided TTS language.
|
||||
- translatedText and ttsText must preserve the same meaning as the original speech.
|
||||
- translatedText should prioritize subtitle readability.
|
||||
- ttsText should prioritize natural spoken dubbing in the target TTS language.
|
||||
|
||||
6. Voice Selection:
|
||||
The user will provide a TTS language and a list of available voices.
|
||||
Each voice includes a voiceId and descriptive metadata.
|
||||
You must analyze the user-provided voice list and choose the best matching voiceId for each subtitle item.
|
||||
Only return a voiceId that exists in the user-provided voice list.
|
||||
Do not invent new voiceId values.
|
||||
|
||||
6. Output Format:
|
||||
Return only valid JSON.
|
||||
Do not output markdown, code fences, explanations, or any extra text.
|
||||
# Output Contract
|
||||
You must return exactly one JSON object.
|
||||
The first character of your response must be {.
|
||||
The last character of your response must be }.
|
||||
Do not output markdown, code fences, comments, headings, explanations, or any text before or after the JSON.
|
||||
|
||||
Return an object with this exact structure:
|
||||
Return a JSON object with this exact top-level structure:
|
||||
{
|
||||
"sourceLanguage": "detected language code",
|
||||
"subtitles": [
|
||||
@ -143,7 +214,9 @@ Return an object with this exact structure:
|
||||
"startTime": 0.0,
|
||||
"endTime": 1.2,
|
||||
"originalText": "source dialogue",
|
||||
"translatedText": "translated dialogue in the target language",
|
||||
"translatedText": "english subtitle text",
|
||||
"ttsText": "translated dubbing text in the requested TTS language",
|
||||
"ttsLanguage": "requested tts language code",
|
||||
"speaker": "short speaker label",
|
||||
"gender": "male or female",
|
||||
"voiceId": "one of the user-provided voice ids"
|
||||
@ -151,23 +224,39 @@ Return an object with this exact structure:
|
||||
]
|
||||
}
|
||||
|
||||
Additional rules:
|
||||
1. Use video timeline seconds for startTime and endTime.
|
||||
2. Keep subtitles chronological and non-overlapping.
|
||||
3. Do not invent dialogue if it is not actually audible.
|
||||
4. Preserve meaning naturally while keeping subtitle lines short and readable.
|
||||
5. If a long utterance must be split, preserve continuity across consecutive subtitle items.
|
||||
6. Output JSON only.`;
|
||||
# JSON Rules
|
||||
1. sourceLanguage must be a JSON string.
|
||||
2. subtitles must be a JSON array.
|
||||
3. Every subtitle item must include all of these fields: id, startTime, endTime, originalText, translatedText, ttsText, ttsLanguage, speaker, gender, voiceId.
|
||||
4. startTime and endTime must be JSON numbers, not strings.
|
||||
5. id, originalText, translatedText, ttsText, ttsLanguage, speaker, gender, and voiceId must be JSON strings.
|
||||
6. gender must be exactly "male" or "female".
|
||||
7. translatedText must always be English.
|
||||
8. ttsLanguage must exactly match the user-requested TTS language code.
|
||||
9. voiceId must exactly match one of the user-provided voiceId values.
|
||||
10. Do not include any extra fields.
|
||||
11. Do not use trailing commas.
|
||||
12. Use video timeline seconds for startTime and endTime.
|
||||
13. Keep subtitles chronological and non-overlapping.
|
||||
14. Do not invent dialogue if it is not actually audible.
|
||||
15. Preserve meaning naturally while keeping subtitle lines short and readable.
|
||||
16. If a long utterance must be split, preserve continuity across consecutive subtitle items.
|
||||
17. Output JSON only.`;
|
||||
|
||||
const createUserPrompt = (targetLanguage: string) => `Target language: ${targetLanguage}
|
||||
const createUserPrompt = (ttsLanguage: string) => `Subtitle language: English
|
||||
TTS language: ${ttsLanguage}
|
||||
|
||||
Available voices:
|
||||
${formatVoiceCatalogForPrompt(targetLanguage)}
|
||||
Available voices for the TTS language:
|
||||
${formatVoiceCatalogForPrompt(ttsLanguage)}
|
||||
|
||||
Please watch and listen to the provided video.
|
||||
Transcribe the dialogue, translate it into ${targetLanguage}, and assign the best matching voiceId from the available voices for each subtitle item.`;
|
||||
Transcribe the dialogue.
|
||||
Generate English subtitle text in translatedText.
|
||||
Generate dubbing text in ${ttsLanguage} inside ttsText.
|
||||
Set ttsLanguage to ${ttsLanguage} for every subtitle item.
|
||||
Assign the best matching voiceId from the available voices for each subtitle item.`;
|
||||
|
||||
const normalizeSubtitles = (raw: RawModelSubtitle[]) => {
|
||||
const normalizeSubtitles = (raw: RawModelSubtitle[], fallbackTtsLanguage: string) => {
|
||||
let lastEnd = 0;
|
||||
const subtitles = raw
|
||||
.map((entry, index) => {
|
||||
@ -186,6 +275,8 @@ const normalizeSubtitles = (raw: RawModelSubtitle[]) => {
|
||||
endTime,
|
||||
originalText,
|
||||
translatedText: (entry.translatedText || originalText).trim(),
|
||||
ttsText: (entry.ttsText || entry.translatedText || originalText).trim(),
|
||||
ttsLanguage: (entry.ttsLanguage || fallbackTtsLanguage).trim(),
|
||||
speaker: (entry.speaker || `Speaker ${index + 1}`).trim(),
|
||||
speakerId: `speaker-${index + 1}`,
|
||||
voiceId: sanitizeVoiceId(entry.voiceId),
|
||||
@ -218,6 +309,7 @@ const generateWithDoubao = async ({
|
||||
videoDataUrl,
|
||||
fileId,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
fetchImpl = fetch,
|
||||
requestId,
|
||||
}: {
|
||||
@ -225,6 +317,7 @@ const generateWithDoubao = async ({
|
||||
videoDataUrl?: string;
|
||||
fileId?: string;
|
||||
targetLanguage: string;
|
||||
ttsLanguage: string;
|
||||
fetchImpl?: typeof fetch;
|
||||
requestId?: string;
|
||||
}) => {
|
||||
@ -234,9 +327,21 @@ const generateWithDoubao = async ({
|
||||
provider: 'doubao',
|
||||
timeoutMs: config.timeoutMs,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
});
|
||||
|
||||
console.info(`[subtitle] doubao request started ${logContext}`);
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: `[subtitle] doubao request started ${logContext}`,
|
||||
context: {
|
||||
requestId,
|
||||
provider: 'doubao',
|
||||
timeoutMs: config.timeoutMs,
|
||||
targetLanguage,
|
||||
hasFileId: Boolean(fileId),
|
||||
hasVideoDataUrl: Boolean(videoDataUrl),
|
||||
},
|
||||
});
|
||||
|
||||
let response: Response;
|
||||
try {
|
||||
@ -262,45 +367,65 @@ const generateWithDoubao = async ({
|
||||
fileId
|
||||
? { type: 'input_video', file_id: fileId }
|
||||
: { type: 'input_video', video_url: videoDataUrl },
|
||||
{ type: 'input_text', text: createUserPrompt(targetLanguage) },
|
||||
{ type: 'input_text', text: createUserPrompt(ttsLanguage) },
|
||||
],
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`[subtitle] doubao request failed ${formatLogContext({
|
||||
logEvent({
|
||||
level: 'error',
|
||||
message: `[subtitle] doubao request failed ${formatLogContext({
|
||||
requestId,
|
||||
provider: 'doubao',
|
||||
timeoutMs: config.timeoutMs,
|
||||
durationMs: Date.now() - startedAt,
|
||||
})}`,
|
||||
serializeError(error),
|
||||
);
|
||||
context: {
|
||||
requestId,
|
||||
provider: 'doubao',
|
||||
timeoutMs: config.timeoutMs,
|
||||
durationMs: Date.now() - startedAt,
|
||||
},
|
||||
details: serializeError(error),
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
const payload = await response.text();
|
||||
console.error(
|
||||
`[subtitle] doubao request returned non-200 ${formatLogContext({
|
||||
logEvent({
|
||||
level: 'error',
|
||||
message: `[subtitle] doubao request returned non-200 ${formatLogContext({
|
||||
requestId,
|
||||
status: response.status,
|
||||
durationMs: Date.now() - startedAt,
|
||||
})}`,
|
||||
payload,
|
||||
);
|
||||
context: {
|
||||
requestId,
|
||||
provider: 'doubao',
|
||||
status: response.status,
|
||||
durationMs: Date.now() - startedAt,
|
||||
},
|
||||
details: payload,
|
||||
});
|
||||
throw new Error(`Doubao subtitle request failed (${response.status}): ${payload}`);
|
||||
}
|
||||
|
||||
const payload = await response.json();
|
||||
console.info(
|
||||
`[subtitle] doubao request finished ${formatLogContext({
|
||||
logEvent({
|
||||
level: 'info',
|
||||
message: `[subtitle] doubao request finished ${formatLogContext({
|
||||
requestId,
|
||||
durationMs: Date.now() - startedAt,
|
||||
})}`,
|
||||
);
|
||||
context: {
|
||||
requestId,
|
||||
provider: 'doubao',
|
||||
durationMs: Date.now() - startedAt,
|
||||
},
|
||||
});
|
||||
const text = extractDoubaoTextOutput(payload);
|
||||
return extractJson(text);
|
||||
};
|
||||
@ -309,10 +434,12 @@ const generateWithGemini = async ({
|
||||
config,
|
||||
videoBase64,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
}: {
|
||||
config: GeminiProviderConfig;
|
||||
videoBase64: string;
|
||||
targetLanguage: string;
|
||||
ttsLanguage: string;
|
||||
}) => {
|
||||
const ai = new GoogleGenAI({ apiKey: config.apiKey });
|
||||
const response = await ai.models.generateContent({
|
||||
@ -327,7 +454,7 @@ const generateWithGemini = async ({
|
||||
data: videoBase64,
|
||||
},
|
||||
},
|
||||
{ text: `${createSystemPrompt()}\n\n${createUserPrompt(targetLanguage)}` },
|
||||
{ text: `${createSystemPrompt()}\n\n${createUserPrompt(ttsLanguage)}` },
|
||||
],
|
||||
},
|
||||
],
|
||||
@ -341,6 +468,7 @@ export const generateSubtitlesFromVideo = async ({
|
||||
videoPath,
|
||||
fileId,
|
||||
targetLanguage,
|
||||
ttsLanguage,
|
||||
fetchImpl = fetch,
|
||||
requestId,
|
||||
}: {
|
||||
@ -348,6 +476,7 @@ export const generateSubtitlesFromVideo = async ({
|
||||
videoPath?: string;
|
||||
fileId?: string;
|
||||
targetLanguage: string;
|
||||
ttsLanguage?: string;
|
||||
fetchImpl?: typeof fetch;
|
||||
requestId?: string;
|
||||
}): Promise<SubtitlePipelineResult> => {
|
||||
@ -359,6 +488,8 @@ export const generateSubtitlesFromVideo = async ({
|
||||
const videoBase64 = videoBuffer?.toString('base64');
|
||||
const videoDataUrl = videoBase64 ? `data:video/mp4;base64,${videoBase64}` : undefined;
|
||||
|
||||
const resolvedTtsLanguage = ttsLanguage?.trim() || targetLanguage;
|
||||
|
||||
const raw =
|
||||
providerConfig.provider === 'doubao'
|
||||
? await generateWithDoubao({
|
||||
@ -366,6 +497,7 @@ export const generateSubtitlesFromVideo = async ({
|
||||
videoDataUrl,
|
||||
fileId,
|
||||
targetLanguage,
|
||||
ttsLanguage: resolvedTtsLanguage,
|
||||
fetchImpl,
|
||||
requestId,
|
||||
})
|
||||
@ -373,9 +505,10 @@ export const generateSubtitlesFromVideo = async ({
|
||||
config: providerConfig,
|
||||
videoBase64: videoBase64!,
|
||||
targetLanguage,
|
||||
ttsLanguage: resolvedTtsLanguage,
|
||||
});
|
||||
|
||||
const subtitles = normalizeSubtitles(Array.isArray(raw.subtitles) ? raw.subtitles : []);
|
||||
const subtitles = normalizeSubtitles(Array.isArray(raw.subtitles) ? raw.subtitles : [], resolvedTtsLanguage);
|
||||
|
||||
return {
|
||||
subtitles,
|
||||
@ -383,6 +516,7 @@ export const generateSubtitlesFromVideo = async ({
|
||||
quality: subtitles.length > 0 ? 'full' : 'fallback',
|
||||
sourceLanguage: raw.sourceLanguage,
|
||||
targetLanguage,
|
||||
ttsLanguage: resolvedTtsLanguage,
|
||||
duration: subtitles.length > 0 ? subtitles[subtitles.length - 1].endTime : 0,
|
||||
alignmentEngine: `llm-video-${providerConfig.provider}`,
|
||||
};
|
||||
|
||||
@ -6,6 +6,7 @@ import { generateSubtitlePipeline } from './subtitleService';
|
||||
describe('generateSubtitlePipeline', () => {
|
||||
beforeEach(() => {
|
||||
vi.stubEnv('VITE_ARK_API_KEY', 'ark-key');
|
||||
vi.stubEnv('VITE_API_BASE_PATH', '/api');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
@ -14,29 +15,59 @@ describe('generateSubtitlePipeline', () => {
|
||||
});
|
||||
|
||||
it('posts the selected provider to the server for gemini', async () => {
|
||||
const fetchMock = vi.fn(async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
vi.useFakeTimers();
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
message: 'Queued',
|
||||
}),
|
||||
{
|
||||
status: 202,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'succeeded',
|
||||
stage: 'succeeded',
|
||||
progress: 100,
|
||||
result: {
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
|
||||
await generateSubtitlePipeline(
|
||||
const promise = generateSubtitlePipeline(
|
||||
new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
'English',
|
||||
'gemini',
|
||||
null,
|
||||
fetchMock as unknown as typeof fetch,
|
||||
);
|
||||
await vi.runAllTimersAsync();
|
||||
await promise;
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledWith(
|
||||
'/api/generate-subtitles',
|
||||
@ -50,6 +81,68 @@ describe('generateSubtitlePipeline', () => {
|
||||
const formData = requestInit.body as FormData;
|
||||
expect(formData.get('targetLanguage')).toBe('English');
|
||||
expect(formData.get('provider')).toBe('gemini');
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(2, '/api/generate-subtitles/job-1', { method: 'GET' });
|
||||
});
|
||||
|
||||
it('forwards the tts language in subtitle requests', async () => {
|
||||
vi.useFakeTimers();
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
}),
|
||||
{
|
||||
status: 202,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'succeeded',
|
||||
stage: 'succeeded',
|
||||
progress: 100,
|
||||
result: {
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
|
||||
const promise = generateSubtitlePipeline(
|
||||
new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
'English',
|
||||
'gemini',
|
||||
null,
|
||||
fetchMock as unknown as typeof fetch,
|
||||
undefined,
|
||||
'fr',
|
||||
);
|
||||
await vi.runAllTimersAsync();
|
||||
await promise;
|
||||
|
||||
const [, requestInit] = fetchMock.mock.calls[0] as unknown as [string, RequestInit];
|
||||
const formData = requestInit.body as FormData;
|
||||
expect(formData.get('ttsLanguage')).toBe('fr');
|
||||
});
|
||||
|
||||
it('uploads doubao videos to ark files before requesting subtitles', async () => {
|
||||
@ -100,9 +193,33 @@ describe('generateSubtitlePipeline', () => {
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
}),
|
||||
{
|
||||
status: 202,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'succeeded',
|
||||
stage: 'succeeded',
|
||||
progress: 100,
|
||||
result: {
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
@ -171,7 +288,9 @@ describe('generateSubtitlePipeline', () => {
|
||||
fileId: 'file-123',
|
||||
provider: 'doubao',
|
||||
targetLanguage: 'English',
|
||||
ttsLanguage: 'English',
|
||||
});
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(5, '/api/generate-subtitles/job-1', { method: 'GET' });
|
||||
});
|
||||
|
||||
it('stops when ark reports file preprocessing failure', async () => {
|
||||
@ -222,32 +341,61 @@ describe('generateSubtitlePipeline', () => {
|
||||
});
|
||||
|
||||
it('keeps multipart uploads for gemini requests', async () => {
|
||||
const fetchMock = vi.fn(async () =>
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
vi.useFakeTimers();
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
}),
|
||||
{
|
||||
status: 202,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'succeeded',
|
||||
stage: 'succeeded',
|
||||
progress: 100,
|
||||
result: {
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
|
||||
await generateSubtitlePipeline(
|
||||
const promise = generateSubtitlePipeline(
|
||||
new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
'English',
|
||||
'gemini',
|
||||
null,
|
||||
fetchMock as unknown as typeof fetch,
|
||||
);
|
||||
await vi.runAllTimersAsync();
|
||||
await promise;
|
||||
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
expect(fetchMock).toHaveBeenCalledWith(
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
'/api/generate-subtitles',
|
||||
expect.objectContaining({
|
||||
method: 'POST',
|
||||
@ -255,4 +403,98 @@ describe('generateSubtitlePipeline', () => {
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('polls every 5 seconds and reports progress updates', async () => {
|
||||
vi.useFakeTimers();
|
||||
const onProgress = vi.fn();
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
message: 'Queued',
|
||||
}),
|
||||
{
|
||||
status: 202,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'running',
|
||||
stage: 'calling_provider',
|
||||
progress: 70,
|
||||
message: 'Calling provider',
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
new Response(
|
||||
JSON.stringify({
|
||||
jobId: 'job-1',
|
||||
requestId: 'req-1',
|
||||
status: 'succeeded',
|
||||
stage: 'succeeded',
|
||||
progress: 100,
|
||||
message: 'Done',
|
||||
result: {
|
||||
subtitles: [],
|
||||
speakers: [],
|
||||
quality: 'fallback',
|
||||
},
|
||||
}),
|
||||
{
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
},
|
||||
),
|
||||
);
|
||||
|
||||
const promise = generateSubtitlePipeline(
|
||||
new File(['video'], 'clip.mp4', { type: 'video/mp4' }),
|
||||
'English',
|
||||
'gemini',
|
||||
null,
|
||||
fetchMock as unknown as typeof fetch,
|
||||
onProgress,
|
||||
);
|
||||
await vi.runAllTimersAsync();
|
||||
await promise;
|
||||
|
||||
expect(onProgress).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
jobId: 'job-1',
|
||||
status: 'queued',
|
||||
stage: 'queued',
|
||||
progress: 5,
|
||||
}),
|
||||
);
|
||||
expect(onProgress).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
jobId: 'job-1',
|
||||
status: 'running',
|
||||
stage: 'calling_provider',
|
||||
progress: 70,
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
import { LlmProvider, PipelineQuality, SubtitlePipelineResult } from '../types';
|
||||
import { LlmProvider, PipelineQuality, SubtitleGenerationProgress, SubtitlePipelineResult } from '../types';
|
||||
import { apiUrl } from '../lib/apiBasePath';
|
||||
|
||||
type JsonResponseResult<T> =
|
||||
@ -8,6 +8,13 @@ type JsonResponseResult<T> =
|
||||
const ARK_FILES_URL = 'https://ark.cn-beijing.volces.com/api/v3/files';
|
||||
const ARK_FILE_STATUS_POLL_INTERVAL_MS = 1000;
|
||||
const ARK_FILE_STATUS_TIMEOUT_MS = 120000;
|
||||
const SUBTITLE_JOB_POLL_INTERVAL_MS = 5000;
|
||||
const SUBTITLE_JOB_TIMEOUT_MS = 20 * 60 * 1000;
|
||||
|
||||
interface SubtitleJobResponse extends SubtitleGenerationProgress {
|
||||
error?: string;
|
||||
result?: Partial<SubtitlePipelineResult>;
|
||||
}
|
||||
|
||||
const normalizePipelineQuality = (value: unknown): PipelineQuality => {
|
||||
if (value === 'full' || value === 'partial' || value === 'fallback') {
|
||||
@ -82,6 +89,59 @@ const sleep = (durationMs: number) =>
|
||||
setTimeout(resolve, durationMs);
|
||||
});
|
||||
|
||||
const toPipelineResult = (
|
||||
parsed: Partial<SubtitlePipelineResult>,
|
||||
targetLanguage: string,
|
||||
ttsLanguage: string,
|
||||
): SubtitlePipelineResult => ({
|
||||
subtitles: Array.isArray(parsed.subtitles) ? parsed.subtitles : [],
|
||||
speakers: Array.isArray(parsed.speakers) ? parsed.speakers : [],
|
||||
quality: normalizePipelineQuality(parsed.quality),
|
||||
sourceLanguage: parsed.sourceLanguage,
|
||||
targetLanguage: parsed.targetLanguage || targetLanguage,
|
||||
ttsLanguage: parsed.ttsLanguage || ttsLanguage,
|
||||
duration:
|
||||
typeof parsed.duration === 'number' ? parsed.duration : undefined,
|
||||
alignmentEngine: parsed.alignmentEngine,
|
||||
});
|
||||
|
||||
const pollSubtitleJob = async (
|
||||
jobId: string,
|
||||
targetLanguage: string,
|
||||
ttsLanguage: string,
|
||||
fetchImpl: typeof fetch,
|
||||
onProgress?: (progress: SubtitleGenerationProgress) => void,
|
||||
): Promise<SubtitlePipelineResult> => {
|
||||
const deadline = Date.now() + SUBTITLE_JOB_TIMEOUT_MS;
|
||||
|
||||
while (true) {
|
||||
const resp = await fetchImpl(apiUrl(`/generate-subtitles/${jobId}`), {
|
||||
method: 'GET',
|
||||
});
|
||||
|
||||
const parsed = await readJsonResponseOnce<SubtitleJobResponse>(resp);
|
||||
if (parsed.ok === false) {
|
||||
throw new Error(parsed.error);
|
||||
}
|
||||
|
||||
onProgress?.(parsed.data);
|
||||
|
||||
if (parsed.data.status === 'succeeded') {
|
||||
return toPipelineResult(parsed.data.result || {}, targetLanguage, ttsLanguage);
|
||||
}
|
||||
|
||||
if (parsed.data.status === 'failed') {
|
||||
throw new Error(parsed.data.error || 'Subtitle generation failed.');
|
||||
}
|
||||
|
||||
if (Date.now() >= deadline) {
|
||||
throw new Error('Timed out while waiting for subtitle generation to complete.');
|
||||
}
|
||||
|
||||
await sleep(SUBTITLE_JOB_POLL_INTERVAL_MS);
|
||||
}
|
||||
};
|
||||
|
||||
const waitForArkFileToBecomeActive = async (
|
||||
fileId: string,
|
||||
apiKey: string,
|
||||
@ -132,11 +192,15 @@ export const generateSubtitlePipeline = async (
|
||||
provider: LlmProvider = 'doubao',
|
||||
trimRange?: { start: number; end: number } | null,
|
||||
fetchImpl: typeof fetch = fetch,
|
||||
onProgress?: (progress: SubtitleGenerationProgress) => void,
|
||||
ttsLanguage?: string,
|
||||
): Promise<SubtitlePipelineResult> => {
|
||||
if (!targetLanguage.trim()) {
|
||||
throw new Error('Target language is required.');
|
||||
}
|
||||
|
||||
const resolvedTtsLanguage = ttsLanguage?.trim() || targetLanguage;
|
||||
|
||||
if (provider === 'doubao') {
|
||||
const { fileId, apiKey } = await uploadDoubaoVideoFile(videoFile, fetchImpl);
|
||||
await waitForArkFileToBecomeActive(fileId, apiKey, fetchImpl);
|
||||
@ -148,33 +212,27 @@ export const generateSubtitlePipeline = async (
|
||||
body: JSON.stringify({
|
||||
fileId,
|
||||
targetLanguage,
|
||||
ttsLanguage: resolvedTtsLanguage,
|
||||
provider,
|
||||
...(trimRange ? { trimRange } : {}),
|
||||
}),
|
||||
});
|
||||
|
||||
const parsed = await readJsonResponseOnce<Partial<SubtitlePipelineResult>>(resp);
|
||||
const parsed = await readJsonResponseOnce<SubtitleJobResponse>(resp);
|
||||
if (parsed.ok === false) {
|
||||
const error = new Error(parsed.error);
|
||||
(error as any).status = resp.status;
|
||||
throw error;
|
||||
}
|
||||
|
||||
return {
|
||||
subtitles: Array.isArray(parsed.data.subtitles) ? parsed.data.subtitles : [],
|
||||
speakers: Array.isArray(parsed.data.speakers) ? parsed.data.speakers : [],
|
||||
quality: normalizePipelineQuality(parsed.data.quality),
|
||||
sourceLanguage: parsed.data.sourceLanguage,
|
||||
targetLanguage: parsed.data.targetLanguage || targetLanguage,
|
||||
duration:
|
||||
typeof parsed.data.duration === 'number' ? parsed.data.duration : undefined,
|
||||
alignmentEngine: parsed.data.alignmentEngine,
|
||||
};
|
||||
const job = parsed.data as unknown as SubtitleJobResponse;
|
||||
onProgress?.(job);
|
||||
return pollSubtitleJob(job.jobId, targetLanguage, resolvedTtsLanguage, fetchImpl, onProgress);
|
||||
}
|
||||
|
||||
const formData = new FormData();
|
||||
formData.append('video', videoFile);
|
||||
formData.append('targetLanguage', targetLanguage);
|
||||
formData.append('ttsLanguage', resolvedTtsLanguage);
|
||||
formData.append('provider', provider);
|
||||
if (trimRange) {
|
||||
formData.append('trimRange', JSON.stringify(trimRange));
|
||||
@ -185,21 +243,12 @@ export const generateSubtitlePipeline = async (
|
||||
body: formData,
|
||||
});
|
||||
|
||||
const parsed = await readJsonResponseOnce<Partial<SubtitlePipelineResult>>(resp);
|
||||
const parsed = await readJsonResponseOnce<SubtitleJobResponse>(resp);
|
||||
if (parsed.ok === false) {
|
||||
const error = new Error(parsed.error);
|
||||
(error as any).status = resp.status;
|
||||
throw error;
|
||||
}
|
||||
|
||||
return {
|
||||
subtitles: Array.isArray(parsed.data.subtitles) ? parsed.data.subtitles : [],
|
||||
speakers: Array.isArray(parsed.data.speakers) ? parsed.data.speakers : [],
|
||||
quality: normalizePipelineQuality(parsed.data.quality),
|
||||
sourceLanguage: parsed.data.sourceLanguage,
|
||||
targetLanguage: parsed.data.targetLanguage || targetLanguage,
|
||||
duration:
|
||||
typeof parsed.data.duration === 'number' ? parsed.data.duration : undefined,
|
||||
alignmentEngine: parsed.data.alignmentEngine,
|
||||
};
|
||||
onProgress?.(parsed.data);
|
||||
return pollSubtitleJob(parsed.data.jobId, targetLanguage, resolvedTtsLanguage, fetchImpl, onProgress);
|
||||
};
|
||||
|
||||
36
src/types.ts
36
src/types.ts
@ -4,6 +4,8 @@ export interface Subtitle {
|
||||
endTime: number;
|
||||
originalText: string;
|
||||
translatedText: string;
|
||||
ttsText?: string;
|
||||
ttsLanguage?: string;
|
||||
speaker: string;
|
||||
speakerId?: string;
|
||||
words?: WordTiming[];
|
||||
@ -14,12 +16,24 @@ export interface Subtitle {
|
||||
age?: string;
|
||||
voiceCharacteristics?: string;
|
||||
emotion?: string;
|
||||
textStyle?: TextStyles;
|
||||
}
|
||||
|
||||
export type LlmProvider = 'doubao' | 'gemini';
|
||||
|
||||
export type PipelineQuality = 'full' | 'partial' | 'fallback';
|
||||
|
||||
export type SubtitleJobStatus = 'queued' | 'running' | 'succeeded' | 'failed';
|
||||
|
||||
export type SubtitleJobStage =
|
||||
| 'queued'
|
||||
| 'upload_received'
|
||||
| 'preparing'
|
||||
| 'calling_provider'
|
||||
| 'processing_result'
|
||||
| 'succeeded'
|
||||
| 'failed';
|
||||
|
||||
export interface WordTiming {
|
||||
text: string;
|
||||
startTime: number;
|
||||
@ -40,21 +54,43 @@ export interface SubtitlePipelineResult {
|
||||
quality: PipelineQuality;
|
||||
sourceLanguage?: string;
|
||||
targetLanguage?: string;
|
||||
ttsLanguage?: string;
|
||||
duration?: number;
|
||||
alignmentEngine?: string;
|
||||
}
|
||||
|
||||
export interface SubtitleGenerationProgress {
|
||||
jobId: string;
|
||||
requestId: string;
|
||||
status: SubtitleJobStatus;
|
||||
stage: SubtitleJobStage;
|
||||
progress: number;
|
||||
message: string;
|
||||
}
|
||||
|
||||
export interface TextStyles {
|
||||
fontFamily: string;
|
||||
fontSize: number;
|
||||
color: string;
|
||||
backgroundColor: string;
|
||||
strokeColor: string;
|
||||
strokeWidth: number;
|
||||
alignment: 'left' | 'center' | 'right';
|
||||
isBold: boolean;
|
||||
isItalic: boolean;
|
||||
isUnderline: boolean;
|
||||
}
|
||||
|
||||
export interface SubtitleDefaults {
|
||||
fontSize: number;
|
||||
bottomOffsetPercent: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_SUBTITLE_DEFAULTS: SubtitleDefaults = {
|
||||
fontSize: 24,
|
||||
bottomOffsetPercent: 10,
|
||||
};
|
||||
|
||||
export interface Voice {
|
||||
id: string;
|
||||
name: string;
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user