Low-precision timing detected. Manual review recommended.
+)} +``` + +**Step 4: Run test to verify it passes** + +Run: `npm test -- --run src/components/EditorScreen.test.tsx` +Expected: PASS. + +**Step 5: Commit** + +```bash +git add src/components/EditorScreen.tsx src/components/EditorScreen.test.tsx +git commit -m "feat: surface subtitle precision status in editor" +``` + +### Task 9: Add Word-Level Playback Helpers + +**Files:** +- Create: `E:\Downloads\ai-video-dubbing-&-translation\src\lib\playback\wordHighlight.ts` +- Create: `E:\Downloads\ai-video-dubbing-&-translation\src\lib\playback\wordHighlight.test.ts` +- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.tsx` + +**Step 1: Write the failing test** + +Test the active-word lookup helper. + +```ts +it('returns the active word for the current playback time', () => { + const activeWord = getActiveWord([ + { text: 'Hello', startTime: 1, endTime: 1.3, speakerId: 'spk_0', confidence: 0.9 }, + ], 1.1); + + expect(activeWord?.text).toBe('Hello'); +}); +``` + +**Step 2: Run test to verify it fails** + +Run: `npm test -- --run src/lib/playback/wordHighlight.test.ts` +Expected: FAIL because playback helpers do not exist. + +**Step 3: Write minimal implementation** + +1. Create a pure helper for active-word lookup. +2. Use it in `EditorScreen.tsx` to render highlighted word spans when `words` are present. + +```ts +export const getActiveWord = (words: WordTiming[], currentTime: number) => + words.find((word) => currentTime >= word.startTime && currentTime <= word.endTime); +``` + +**Step 4: Run test to verify it passes** + +Run: `npm test -- --run src/lib/playback/wordHighlight.test.ts` +Expected: PASS. + +**Step 5: Commit** + +```bash +git add src/lib/playback/wordHighlight.ts src/lib/playback/wordHighlight.test.ts src/components/EditorScreen.tsx +git commit -m "feat: add word-level playback highlighting" +``` + +### Task 10: Snap Timeline Edges to Word Boundaries + +**Files:** +- Create: `E:\Downloads\ai-video-dubbing-&-translation\src\lib\timeline\snapToWords.ts` +- Create: `E:\Downloads\ai-video-dubbing-&-translation\src\lib\timeline\snapToWords.test.ts` +- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.tsx` + +**Step 1: Write the failing test** + +Test snapping to nearest word edges. + +```ts +it('snaps a dragged start edge to the nearest word boundary', () => { + const next = snapTimeToNearestWordBoundary( + 1.34, + [ + { text: 'Hello', startTime: 1.0, endTime: 1.3, speakerId: 'spk_0', confidence: 0.9 }, + { text: 'world', startTime: 1.35, endTime: 1.8, speakerId: 'spk_0', confidence: 0.9 }, + ], + ); + + expect(next).toBe(1.35); +}); +``` + +**Step 2: Run test to verify it fails** + +Run: `npm test -- --run src/lib/timeline/snapToWords.test.ts` +Expected: FAIL because no snapping helper exists. + +**Step 3: Write minimal implementation** + +1. Add a pure snapping helper with a small tolerance window. +2. Use it in the left and right resize timeline handlers. + +```ts +export const snapTimeToNearestWordBoundary = (time: number, words: WordTiming[]) => { + // choose nearest start or end boundary within tolerance +}; +``` + +**Step 4: Run test to verify it passes** + +Run: `npm test -- --run src/lib/timeline/snapToWords.test.ts` +Expected: PASS. + +**Step 5: Commit** + +```bash +git add src/lib/timeline/snapToWords.ts src/lib/timeline/snapToWords.test.ts src/components/EditorScreen.tsx +git commit -m "feat: snap subtitle edits to word boundaries" +``` + +### Task 11: Add Speaker-Aware UI State + +**Files:** +- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\components\EditorScreen.tsx` +- Modify: `E:\Downloads\ai-video-dubbing-&-translation\src\voices.ts` +- Create: `E:\Downloads\ai-video-dubbing-&-translation\src\lib\speakers\speakerPresentation.ts` +- Create: `E:\Downloads\ai-video-dubbing-&-translation\src\lib\speakers\speakerPresentation.test.ts` + +**Step 1: Write the failing test** + +Test stable color and label generation for speaker tracks. + +```ts +it('creates stable display metadata for each speaker id', () => { + const speaker = buildSpeakerPresentation({ speakerId: 'spk_0', label: 'Speaker 1' }); + expect(speaker.color).toMatch(/^#/); +}); +``` + +**Step 2: Run test to verify it fails** + +Run: `npm test -- --run src/lib/speakers/speakerPresentation.test.ts` +Expected: FAIL because no speaker presentation helper exists. + +**Step 3: Write minimal implementation** + +1. Create a helper that derives display color and fallback label from `speakerId`. +2. Use it to color sentence chips or timeline items. +3. Keep voice assignment behavior backward compatible. + +```ts +export const buildSpeakerPresentation = ({ speakerId, label }: SpeakerTrack) => ({ + speakerId, + label, + color: '#1677ff', +}); +``` + +**Step 4: Run test to verify it passes** + +Run: `npm test -- --run src/lib/speakers/speakerPresentation.test.ts` +Expected: PASS. + +**Step 5: Commit** + +```bash +git add src/components/EditorScreen.tsx src/voices.ts src/lib/speakers/speakerPresentation.ts src/lib/speakers/speakerPresentation.test.ts +git commit -m "feat: add speaker-aware editor presentation" +``` + +### Task 12: Verify End-to-End Behavior and Update Docs + +**Files:** +- Modify: `E:\Downloads\ai-video-dubbing-&-translation\README.md` +- Modify: `E:\Downloads\ai-video-dubbing-&-translation\docs\plans\2026-03-17-precise-dialogue-localization-design.md` + +**Step 1: Write the failing test** + +Write down the manual verification checklist before changing docs so the release criteria are explicit. + +```md +- [ ] Single-speaker clip returns `quality: full` +- [ ] Two-speaker clip shows distinct speaker IDs +- [ ] Fallback path shows low-precision notice +- [ ] Timeline resize snaps to word boundaries +``` + +**Step 2: Run test to verify it fails** + +Run: `npm run lint` +Expected: PASS or FAIL depending on in-progress code, but manual verification is still incomplete until the checklist is executed. + +**Step 3: Write minimal implementation** + +1. Update `README.md` with new environment requirements and pipeline description. +2. Record the manual verification results in the design document or a linked note. + +```md +## High-Precision Subtitle Mode + +Set the alignment backend environment variables before running the app. +``` + +**Step 4: Run test to verify it passes** + +Run: `npm test -- --run` +Expected: PASS. + +Run: `npm run lint` +Expected: PASS. + +Run: `npm run build` +Expected: PASS. + +**Step 5: Commit** + +```bash +git add README.md docs/plans/2026-03-17-precise-dialogue-localization-design.md +git commit -m "docs: document precise dialogue localization workflow" +``` + +## Notes for Execution + +1. This workspace currently has no `.git` directory, so commit steps cannot be executed until the project is placed in a real Git checkout. +2. Introduce the alignment backend behind environment-based configuration so existing demos can still use the current fallback path. +3. Prefer pure functions for sentence reconstruction, speaker assignment, snapping, and word-highlighting logic so they remain easy to test. diff --git a/docs/plans/2026-03-18-alignment-fallback-safety.md b/docs/plans/2026-03-18-alignment-fallback-safety.md new file mode 100644 index 0000000..794eafe --- /dev/null +++ b/docs/plans/2026-03-18-alignment-fallback-safety.md @@ -0,0 +1,75 @@ +# Alignment Fallback Safety Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Stop subtitle generation from silently falling back into the Studio project workflow unless an explicit environment flag enables it. + +**Architecture:** Keep the existing alignment adapter and Studio fallback code path, but gate that fallback behind a parsed boolean config from `.env`. When the alignment endpoint returns `404` or `405` and the flag is not enabled, fail fast with a clear error instead of returning unrelated subtitles. + +**Tech Stack:** TypeScript, Vitest, Express, Node fetch/FormData APIs + +--- + +### Task 1: Add failing tests for safe-by-default fallback behavior + +**Files:** +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/src/server/alignmentAdapter.test.ts` +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/src/server/audioPipelineConfig.test.ts` +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/src/server/subtitleGeneration.test.ts` + +**Step 1: Write the failing tests** + +- Add a test asserting `requestAlignedTranscript()` throws a clear error on `404` when Studio fallback is not explicitly enabled. +- Update the existing fallback test to pass only when `allowStudioProjectFallback` is set to `true`. +- Add config tests for parsing `ALLOW_STUDIO_PROJECT_FALLBACK` with a default of `false`. +- Add a pipeline test asserting `generateSubtitlePipeline()` forwards the parsed fallback flag into `requestAlignedTranscript()`. + +**Step 2: Run targeted tests to verify they fail** + +Run: `node ./node_modules/vitest/vitest.mjs run src/server/alignmentAdapter.test.ts src/server/audioPipelineConfig.test.ts src/server/subtitleGeneration.test.ts` + +Expected: FAIL because the fallback flag does not exist yet and `requestAlignedTranscript()` still auto-falls back. + +### Task 2: Implement the minimal fallback gate + +**Files:** +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/src/server/audioPipelineConfig.ts` +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/src/server/alignmentAdapter.ts` +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/src/server/subtitleGeneration.ts` + +**Step 1: Add config parsing** + +- Extend `AudioPipelineConfig` with `allowStudioProjectFallback: boolean`. +- Parse `ALLOW_STUDIO_PROJECT_FALLBACK` from `.env`, defaulting to `false`. + +**Step 2: Gate fallback execution** + +- Extend `RequestAlignedTranscriptOptions` with `allowStudioProjectFallback`. +- When the alignment root returns `404` or `405` and the flag is `false`, throw a clear error that points to `ALLOW_STUDIO_PROJECT_FALLBACK=true` or a proper alignment backend. +- When the flag is `true`, keep the current Studio project workflow unchanged. + +**Step 3: Thread config through the pipeline** + +- Pass `allowStudioProjectFallback` from `resolveAudioPipelineConfig()` into `requestAlignedTranscript()` inside `generateSubtitlePipeline()`. + +### Task 3: Update docs and verify + +**Files:** +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/.env.example` +- Modify: `E:/Downloads/ai-video-dubbing-&-translation/README.md` + +**Step 1: Document the new flag** + +- Add `ALLOW_STUDIO_PROJECT_FALLBACK="false"` to `.env.example`. +- Clarify in `README.md` that subtitle generation now uses the local Studio workflow by default, and that fail-fast mode is opt-in. + +**Step 2: Run verification** + +Run: `node ./node_modules/vitest/vitest.mjs run src/server/alignmentAdapter.test.ts src/server/audioPipelineConfig.test.ts src/server/subtitleGeneration.test.ts` +Expected: PASS + +Run: `node ./node_modules/vitest/vitest.mjs run` +Expected: PASS + +Run: `cmd /c npm run lint` +Expected: PASS diff --git a/index.html b/index.html new file mode 100644 index 0000000..21dfe69 --- /dev/null +++ b/index.html @@ -0,0 +1,13 @@ + + + + + ++ Tip: After modifying the subtitle text, you need to regenerate the dubbing to take effect! +
+AI is analyzing and translating...
+This may take a minute depending on the video length.
+{generationError}
+ +