diff --git a/CLAUDE.md b/CLAUDE.md index 653eff4..3c80820 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,9 +2,22 @@ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +## Project Overview + +**Soundboard V2** - A full-stack web application for managing and playing sound files with streaming capabilities. + +### Core Features +- **Sound Management**: Upload, organize, and play sound files +- **Stream Processing**: Download and process audio from YouTube, SoundCloud, and other platforms using yt-dlp +- **Music Player**: Full-featured VLC-based music player with playlist support +- **Real-time Updates**: WebSocket support for live player state synchronization +- **Multi-Provider OAuth**: Google and GitHub authentication with JWT tokens +- **Credits System**: Usage tracking with tiered subscription plans +- **Sound Normalization**: Automatic audio processing and normalization + ## Project Structure -This is a full-stack application with separate backend (Flask) and frontend (Next.js) components: +This is a full-stack application with separate backend (Flask) and frontend (React) components: - **Backend** (`backend/`): Flask application with clean architecture - Entry point: `main.py` @@ -13,18 +26,20 @@ This is a full-stack application with separate backend (Flask) and frontend (Nex - Uses modern Python tooling (Black, Ruff, pytest) - CORS enabled for frontend integration - Python 3.12+ required + - Real-time features via Flask-SocketIO - **Frontend** (`frontend/`): React application with Vite - Uses React 19, TypeScript, React Router v7, and Tailwind CSS v4 - Shadcn/ui components with consistent sidebar layout for authenticated users - Built with Vite for fast development and production builds + - Real-time updates via Socket.IO client ## Development Commands ### Backend (Flask) ```bash cd backend -uv run python main.py # Run the Flask development server (localhost:5000) +uv run python main.py # Run the Flask development server with SocketIO (localhost:5000) uv run pytest # Run tests uv run pytest tests/ # Run specific test directory uv run ruff check # Lint code @@ -39,6 +54,13 @@ uv run flask --app app db migrate # Create new migration uv run flask --app app db upgrade # Apply migrations ``` +#### Key Dependencies +- **yt-dlp**: YouTube and streaming platform downloader +- **python-vlc**: VLC media player Python bindings +- **flask-socketio**: Real-time WebSocket communication +- **ffmpeg-python**: Audio processing and normalization +- **apscheduler**: Background task scheduling + #### Environment Variables ```bash # Required for sessions and JWT @@ -58,10 +80,10 @@ export GITHUB_CLIENT_SECRET="your_github_client_secret" export STREAM_MAX_CONCURRENT="2" # Number of concurrent downloads ``` -### Frontend (Next.js) +### Frontend (React/Vite) ```bash cd frontend -bun dev # Start development server (preferred) +bun dev # Start development server (preferred) on localhost:3000 # Alternative package managers: npm run dev / yarn dev / pnpm dev bun run build # Build for production @@ -69,6 +91,14 @@ bun run start # Start production server bun run lint # Run ESLint ``` +#### Key Dependencies +- **@number-flow/react**: Animated number transitions +- **socket.io-client**: Real-time WebSocket communication +- **next-themes**: Dark/light theme support +- **react-router**: Client-side routing (v7) +- **lucide-react**: Icon library +- **@radix-ui**: Headless UI components + ## Code Style ### Backend @@ -326,4 +356,174 @@ def ai_generation(): { "error": "Insufficient credits. Required: 10, Available: 5" } -``` \ No newline at end of file +``` + +## Soundboard System + +The application includes a comprehensive sound management system with multiple sound types and real-time playback capabilities. + +### Sound Types + +- **SDB (Soundboard)**: Traditional soundboard effects and clips stored in `sounds/soundboard/` +- **SAY (Text-to-Speech)**: Generated speech audio files stored in `sounds/say/` +- **STR (Stream)**: Downloaded audio from streaming platforms stored in `sounds/stream/` + +### Database Models + +#### Sound Model +- **Core fields**: id, type, name, filename, duration, size, hash +- **Normalization**: normalized_filename, normalized_duration, normalized_size, normalized_hash +- **Properties**: is_normalized, is_music, is_deletable, play_count +- **Relationships**: playlist_sounds, streams +- **Methods**: increment_play_count(), set_normalized_info(), find_by_hash() + +#### Stream Model +- **Metadata**: service, service_id, url, title, track, artist, album, genre +- **Status tracking**: status (pending/processing/completed/failed), error +- **Relationships**: Links to Sound model when processing completes +- **Constraints**: Unique constraint on service + service_id + +#### Playlist Models +- **Playlist**: id, name, description, genre, user_id, is_main, is_current +- **PlaylistSound**: Junction table with playlist_id, sound_id, order +- **Methods**: add_sound(), find_current_playlist(), find_main_playlist() + +### API Endpoints + +#### Soundboard Routes (`/api/soundboard/`) +- `GET /sounds` - Get all soundboard sounds with type filtering +- `POST /sounds//play` - Play a sound (costs 1 credit) +- `POST /stop-all` - Stop all currently playing sounds +- `POST /force-stop` - Force stop with aggressive cleanup +- `GET /status` - Get current playback status + +#### Stream Routes (`/api/stream/`) +- `POST /add-url` - Add streaming URL to processing queue + +#### Music Player Routes (`/api/player/`) +- `GET /state` - Get current player state +- `POST /play` - Start playback +- `POST /pause` - Pause playback +- `POST /stop` - Stop playback +- `POST /next` - Skip to next track +- `POST /previous` - Skip to previous track +- `POST /seek` - Seek to position (0.0-1.0) +- `POST /volume` - Set volume (0-100) +- `POST /mode` - Set play mode (continuous/loop-playlist/loop-one/random/single) +- `POST /playlist` - Load playlist into player +- `POST /play-track` - Play track at specific index + +#### Admin Routes (`/api/admin/`) +- Sound management endpoints for admin users +- User management endpoints + +### Services + +#### StreamProcessingService +- **Queue-based processing**: Multi-threaded download queue with configurable concurrency +- **Metadata extraction**: Uses yt-dlp to extract video/audio metadata +- **Download processing**: Downloads audio in opus format with thumbnails +- **File management**: Moves files to organized directory structure +- **Database integration**: Creates Sound entries and links to Stream records +- **Playlist integration**: Automatically adds processed sounds to main playlist +- **Normalization**: Triggers automatic audio normalization after download + +#### MusicPlayerService +- **VLC integration**: Uses python-vlc bindings for audio playback +- **Playlist management**: Full playlist support with multiple play modes +- **Real-time sync**: Background thread syncs player state with VLC +- **Play tracking**: Tracks listening time and play counts (20% completion threshold) +- **WebSocket updates**: Emits real-time player state via SocketIO +- **Multi-format support**: Plays normalized and original audio files + +#### VLCService +- **Process management**: Handles multiple VLC instances for sound effects +- **Force stop capabilities**: Aggressive cleanup for stuck processes +- **Sound tracking**: Records play counts and user interactions + +#### SoundNormalizerService +- **Audio processing**: FFmpeg-based normalization with ReplayGain +- **Two-pass processing**: Optional high-quality normalization +- **Batch operations**: Queue-based processing for multiple files + +#### SocketIOService +- **Real-time communication**: WebSocket server for live updates +- **Player state sync**: Broadcasts player state changes to all clients +- **Sound event notifications**: Play count updates and status changes + +### File Organization + +``` +sounds/ +├── soundboard/ # Original soundboard files +├── say/ # Text-to-speech files +├── stream/ # Downloaded streaming audio +│ └── thumbnails/ # Video thumbnails +├── normalized/ # Normalized versions +│ ├── soundboard/ +│ ├── say/ +│ └── stream/ +└── temp/ # Temporary download files +``` + +### Real-time Features + +#### WebSocket Events +- **player_state_update**: Complete player state including current track, position, volume +- **sound_play_count_changed**: When sound play counts are updated +- **stream_processing_update**: Status updates for stream downloads + +#### Player State Structure +```json +{ + "is_playing": true, + "current_time": 45000, + "duration": 180000, + "volume": 80, + "play_mode": "continuous", + "current_track": { + "id": 123, + "title": "Song Title", + "artist": "Artist Name", + "duration": 180000, + "thumbnail": "http://localhost:5000/api/sounds/str/thumbnails/image.jpg", + "service_url": "https://youtube.com/watch?v=...", + "type": "STR" + }, + "current_track_index": 2, + "playlist": [...], + "playlist_id": 1 +} +``` + +### Stream Processing Workflow + +1. **URL Submission**: User submits streaming URL via `/api/stream/add-url` +2. **Validation**: Check for duplicate URLs and extract basic metadata +3. **Queue Addition**: Add stream to processing queue with "pending" status +4. **Background Processing**: Worker thread processes stream: + - Extract full metadata with yt-dlp + - Download audio file (opus format preferred) + - Download thumbnail if available + - Move files to organized directory structure + - Create Sound database entry + - Link Stream to Sound + - Add to main playlist + - Trigger normalization +5. **Status Updates**: Stream status updated throughout process +6. **Completion**: Stream marked as "completed" with sound_id + +### Play Modes + +- **single**: Play one track and stop +- **continuous**: Play through playlist once +- **loop-playlist**: Repeat entire playlist +- **loop-one**: Repeat current track +- **random**: Random track selection + +### Error Handling + +- **Stream processing errors**: Detailed error messages stored in Stream.error +- **VLC playback errors**: Graceful fallbacks and process cleanup +- **File system errors**: Validation and cleanup of incomplete downloads +- **Database errors**: Transaction rollbacks and error logging \ No newline at end of file