Add optional TwelveLabs Pegasus transcription backend - #68
Open
mohit-twelvelabs wants to merge 1 commit into
Open
Add optional TwelveLabs Pegasus transcription backend#68mohit-twelvelabs wants to merge 1 commit into
mohit-twelvelabs wants to merge 1 commit into
Conversation
Adds Pegasus as an opt-in transcription backend alongside Whisper. When TWELVELABS_API_KEY is set, a 'Pegasus (TwelveLabs)' option appears in the model dropdown; selecting it uploads and indexes the file with TwelveLabs and writes .srt/.vtt/.txt subtitles via the same completion pipeline as Whisper. Default Whisper behavior is unchanged when no key is configured.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi! I'm Mohit, I work at TwelveLabs (@mohit-twelvelabs).
This PR adds TwelveLabs Pegasus as an optional transcription backend alongside Whisper.
What it adds
transcribe/transcribe-pegasus.jsmodule that uploads a file to TwelveLabs, indexes it with the Pegasus video-understanding model, and asks it to return a WebVTT transcript with timestamps..vtt/.srt/.txtfiles that Whisper produces, so the existing completion pipeline (file moving, translation, websocket updates,processing_data.json) is reused unchanged. I refactored the Whisperclosehandler into a namedonTranscriptionCompletefunction so both backends share one code path.Why it helps
Pegasus runs in the cloud, so it can transcribe uploads without a local GPU or even a local Whisper install — useful for low-powered hosts or as a fallback. It also handles video natively.
Opt-in / non-breaking
TWELVELABS_API_KEYis set; with no key the app behaves exactly as before (the dropdown and defaultmediumWhisper model are unchanged).which.sync('whisper')use{ nothrow: true }so the app can still boot on Pegasus-only setups; Whisper users are unaffected.How it was tested
node transcribe/transcribe-pegasus.test.js— no-network unit tests for the WebVTT→SRT/TXT conversion and cue-number cleanup (all assertions pass).npm run lintpasses clean on the full repo..vtt/.srt/.txtfiles were produced. A speech sample returns correctly timestamped WebVTT cues.You can grab a free API key at https://twelvelabs.io — there's a generous free tier.