Engineering - Junior Engineer Instruction from Video Feedback
Transform video feedback sessions into actionable Product Requirement Documents (PRD) with extracted video segments. Perfect for code reviews, technical explanations, and mentoring sessions. Automatically transcribes videos, identifies key moments using AI, extracts important segments, and generates comprehensive documentation.
Features
- ✓Automatic video transcription using AssemblyAI
- ✓AI-powered important moment identification
- ✓Intelligent video segment extraction with context buffers
- ✓Comprehensive PRD generation with visual insights
- ✓Cursor movement and code change tracking
- ✓Performance monitoring and detailed logging
- ✓Support for existing transcripts
- ✓Visual analysis of video frames using OpenAI Vision
Use Cases
- •Code review feedback sessions
- •Technical mentoring and pair programming
- •Onboarding new engineers
- •Documenting architectural decisions
- •Creating training materials from video sessions
- •Converting video tutorials into structured documentation
Prerequisites
- →Node.js v18 or higher
- →FFmpeg installed on your system
- →OpenAI API key (for AI processing and visual analysis)
- →AssemblyAI API key (for automatic video transcription)
Workflow & Tools
Video Upload & Transcription
If no transcript is provided, video is uploaded to AssemblyAI, transcription job is started, and system polls for completion every 3 seconds until transcript is ready.
Important Segment Identification
OpenAI GPT-4 analyzes the transcript to identify technical explanations, code walkthroughs, problem-solving discussions, key decisions, corrections, action items, and moments of emphasis.
Visual Analysis
Extracts frames from video segments and uses OpenAI Vision API to analyze cursor movements, code changes, IDE context, dialogs, and visual flow.
Video Segment Extraction
FFmpeg extracts video segments at important moments with 2-second buffers before and after each segment. Segments are saved as individual MP4 files named by importance score.
PRD Generation
OpenAI generates a comprehensive Product Requirement Document combining transcript insights, visual analysis, and timing information. Includes overview, objectives, requirements, technical specs, implementation steps, acceptance criteria, and visual insights.
Benefits
- ★Saves hours of manual documentation work
- ★Captures every important detail from video sessions
- ★Creates searchable, structured documentation
- ★Extracts key video moments for easy reference
- ★Combines audio and visual insights for comprehensive understanding
- ★Tracks cursor movements and code changes automatically
- ★Generates actionable PRDs ready for implementation
Examples
Process a code review video
Automatically transcribe, analyze, and document a code review session
npm run tools:run junior-engineer-video-feedback -v ~/Videos/code-review-2024-01-15.mp4 -o ./output/code-review-jan-15Process with existing transcript
Skip transcription step if you already have a transcript file
npm run tools:run junior-engineer-video-feedback -v ~/Videos/feedback.mp4 -t ~/Documents/transcript.json -o ./output/feedback-analysisBest Practices
- 💡Use automatic transcription when possible - it's more accurate and saves time
- 💡Keep videos under 2GB for faster processing and better reliability
- 💡Use descriptive output directories for better organization
- 💡Check logs regularly for performance insights and troubleshooting
- 💡Monitor API usage - OpenAI and AssemblyAI have rate limits
- 💡Ensure stable internet connection for video uploads
Troubleshooting
FFmpeg not found
Install FFmpeg: macOS (brew install ffmpeg), Ubuntu/Debian (sudo apt-get install ffmpeg), or download from ffmpeg.org for Windows
API key errors
Ensure your config.toml file or .env contains valid OPENAI_API_KEY and ASSEMBLYAI_API_KEY. Run 'npm run tools:configure' to set up.
Video upload fails
Check video file exists and is readable, verify AssemblyAI API key is valid, check video file size (keep under 2GB), ensure stable internet connection
Transcript generation takes too long
Large videos take longer to process. Check AssemblyAI dashboard for job status. Default timeout is 1 hour.