quaik8/sdb-claude
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update CLAUDE.md to enhance project documentation and clarify soundboard system details

Browse Source
This commit is contained in:
JSC
2025-07-19 09:40:05 +02:00
parent 0c15c3f1a9
commit 8f47f25b4b
1 changed files with 205 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

208
CLAUDE.md
View File

@@ -2,9 +2,22 @@
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
**Soundboard V2** - A full-stack web application for managing and playing sound files with streaming capabilities.
### Core Features
- **Sound Management**: Upload, organize, and play sound files
- **Stream Processing**: Download and process audio from YouTube, SoundCloud, and other platforms using yt-dlp
- **Music Player**: Full-featured VLC-based music player with playlist support
- **Real-time Updates**: WebSocket support for live player state synchronization
- **Multi-Provider OAuth**: Google and GitHub authentication with JWT tokens
- **Credits System**: Usage tracking with tiered subscription plans
- **Sound Normalization**: Automatic audio processing and normalization
## Project Structure ## Project Structure
This is a full-stack application with separate backend (Flask) and frontend (Next.js) components: This is a full-stack application with separate backend (Flask) and frontend (React) components:
- **Backend** (`backend/`): Flask application with clean architecture - **Backend** (`backend/`): Flask application with clean architecture
- Entry point: `main.py` - Entry point: `main.py`
@@ -13,18 +26,20 @@ This is a full-stack application with separate backend (Flask) and frontend (Nex
- Uses modern Python tooling (Black, Ruff, pytest) - Uses modern Python tooling (Black, Ruff, pytest)
- CORS enabled for frontend integration - CORS enabled for frontend integration
- Python 3.12+ required - Python 3.12+ required
- Real-time features via Flask-SocketIO
- **Frontend** (`frontend/`): React application with Vite - **Frontend** (`frontend/`): React application with Vite
- Uses React 19, TypeScript, React Router v7, and Tailwind CSS v4 - Uses React 19, TypeScript, React Router v7, and Tailwind CSS v4
- Shadcn/ui components with consistent sidebar layout for authenticated users - Shadcn/ui components with consistent sidebar layout for authenticated users
- Built with Vite for fast development and production builds - Built with Vite for fast development and production builds
- Real-time updates via Socket.IO client
## Development Commands ## Development Commands
### Backend (Flask) ### Backend (Flask)
```bash ```bash
cd backend cd backend
uv run python main.py # Run the Flask development server (localhost:5000) uv run python main.py # Run the Flask development server with SocketIO (localhost:5000)
uv run pytest # Run tests uv run pytest # Run tests
uv run pytest tests/ # Run specific test directory uv run pytest tests/ # Run specific test directory
uv run ruff check # Lint code uv run ruff check # Lint code
@@ -39,6 +54,13 @@ uv run flask --app app db migrate # Create new migration
uv run flask --app app db upgrade # Apply migrations uv run flask --app app db upgrade # Apply migrations
``` ```
#### Key Dependencies
- **yt-dlp**: YouTube and streaming platform downloader
- **python-vlc**: VLC media player Python bindings
- **flask-socketio**: Real-time WebSocket communication
- **ffmpeg-python**: Audio processing and normalization
- **apscheduler**: Background task scheduling
#### Environment Variables #### Environment Variables
```bash ```bash
# Required for sessions and JWT # Required for sessions and JWT
@@ -58,10 +80,10 @@ export GITHUB_CLIENT_SECRET="your_github_client_secret"
export STREAM_MAX_CONCURRENT="2" # Number of concurrent downloads export STREAM_MAX_CONCURRENT="2" # Number of concurrent downloads
``` ```
### Frontend (Next.js) ### Frontend (React/Vite)
```bash ```bash
cd frontend cd frontend
bun dev # Start development server (preferred) bun dev # Start development server (preferred) on localhost:3000
# Alternative package managers: # Alternative package managers:
npm run dev / yarn dev / pnpm dev npm run dev / yarn dev / pnpm dev
bun run build # Build for production bun run build # Build for production
@@ -69,6 +91,14 @@ bun run start # Start production server
bun run lint # Run ESLint bun run lint # Run ESLint
``` ```
#### Key Dependencies
- **@number-flow/react**: Animated number transitions
- **socket.io-client**: Real-time WebSocket communication
- **next-themes**: Dark/light theme support
- **react-router**: Client-side routing (v7)
- **lucide-react**: Icon library
- **@radix-ui**: Headless UI components
## Code Style ## Code Style
### Backend ### Backend
@@ -327,3 +357,173 @@ def ai_generation():
"error": "Insufficient credits. Required: 10, Available: 5" "error": "Insufficient credits. Required: 10, Available: 5"
} }
``` ```
## Soundboard System
The application includes a comprehensive sound management system with multiple sound types and real-time playback capabilities.
### Sound Types
- **SDB (Soundboard)**: Traditional soundboard effects and clips stored in `sounds/soundboard/`
- **SAY (Text-to-Speech)**: Generated speech audio files stored in `sounds/say/`
- **STR (Stream)**: Downloaded audio from streaming platforms stored in `sounds/stream/`
### Database Models
#### Sound Model
- **Core fields**: id, type, name, filename, duration, size, hash
- **Normalization**: normalized_filename, normalized_duration, normalized_size, normalized_hash
- **Properties**: is_normalized, is_music, is_deletable, play_count
- **Relationships**: playlist_sounds, streams
- **Methods**: increment_play_count(), set_normalized_info(), find_by_hash()
#### Stream Model
- **Metadata**: service, service_id, url, title, track, artist, album, genre
- **Status tracking**: status (pending/processing/completed/failed), error
- **Relationships**: Links to Sound model when processing completes
- **Constraints**: Unique constraint on service + service_id
#### Playlist Models
- **Playlist**: id, name, description, genre, user_id, is_main, is_current
- **PlaylistSound**: Junction table with playlist_id, sound_id, order
- **Methods**: add_sound(), find_current_playlist(), find_main_playlist()
### API Endpoints
#### Soundboard Routes (`/api/soundboard/`)
- `GET /sounds` - Get all soundboard sounds with type filtering
- `POST /sounds/<id>/play` - Play a sound (costs 1 credit)
- `POST /stop-all` - Stop all currently playing sounds
- `POST /force-stop` - Force stop with aggressive cleanup
- `GET /status` - Get current playback status
#### Stream Routes (`/api/stream/`)
- `POST /add-url` - Add streaming URL to processing queue
#### Music Player Routes (`/api/player/`)
- `GET /state` - Get current player state
- `POST /play` - Start playback
- `POST /pause` - Pause playback
- `POST /stop` - Stop playback
- `POST /next` - Skip to next track
- `POST /previous` - Skip to previous track
- `POST /seek` - Seek to position (0.0-1.0)
- `POST /volume` - Set volume (0-100)
- `POST /mode` - Set play mode (continuous/loop-playlist/loop-one/random/single)
- `POST /playlist` - Load playlist into player
- `POST /play-track` - Play track at specific index
#### Admin Routes (`/api/admin/`)
- Sound management endpoints for admin users
- User management endpoints
### Services
#### StreamProcessingService
- **Queue-based processing**: Multi-threaded download queue with configurable concurrency
- **Metadata extraction**: Uses yt-dlp to extract video/audio metadata
- **Download processing**: Downloads audio in opus format with thumbnails
- **File management**: Moves files to organized directory structure
- **Database integration**: Creates Sound entries and links to Stream records
- **Playlist integration**: Automatically adds processed sounds to main playlist
- **Normalization**: Triggers automatic audio normalization after download
#### MusicPlayerService
- **VLC integration**: Uses python-vlc bindings for audio playback
- **Playlist management**: Full playlist support with multiple play modes
- **Real-time sync**: Background thread syncs player state with VLC
- **Play tracking**: Tracks listening time and play counts (20% completion threshold)
- **WebSocket updates**: Emits real-time player state via SocketIO
- **Multi-format support**: Plays normalized and original audio files
#### VLCService
- **Process management**: Handles multiple VLC instances for sound effects
- **Force stop capabilities**: Aggressive cleanup for stuck processes
- **Sound tracking**: Records play counts and user interactions
#### SoundNormalizerService
- **Audio processing**: FFmpeg-based normalization with ReplayGain
- **Two-pass processing**: Optional high-quality normalization
- **Batch operations**: Queue-based processing for multiple files
#### SocketIOService
- **Real-time communication**: WebSocket server for live updates
- **Player state sync**: Broadcasts player state changes to all clients
- **Sound event notifications**: Play count updates and status changes
### File Organization
```
sounds/
├── soundboard/ # Original soundboard files
├── say/ # Text-to-speech files
├── stream/ # Downloaded streaming audio
│ └── thumbnails/ # Video thumbnails
├── normalized/ # Normalized versions
│ ├── soundboard/
│ ├── say/
│ └── stream/
└── temp/ # Temporary download files
```
### Real-time Features
#### WebSocket Events
- **player_state_update**: Complete player state including current track, position, volume
- **sound_play_count_changed**: When sound play counts are updated
- **stream_processing_update**: Status updates for stream downloads
#### Player State Structure
```json
{
"is_playing": true,
"current_time": 45000,
"duration": 180000,
"volume": 80,
"play_mode": "continuous",
"current_track": {
"id": 123,
"title": "Song Title",
"artist": "Artist Name",
"duration": 180000,
"thumbnail": "http://localhost:5000/api/sounds/str/thumbnails/image.jpg",
"service_url": "https://youtube.com/watch?v=...",
"type": "STR"
},
"current_track_index": 2,
"playlist": [...],
"playlist_id": 1
}
```
### Stream Processing Workflow
1. **URL Submission**: User submits streaming URL via `/api/stream/add-url`
2. **Validation**: Check for duplicate URLs and extract basic metadata
3. **Queue Addition**: Add stream to processing queue with "pending" status
4. **Background Processing**: Worker thread processes stream:
- Extract full metadata with yt-dlp
- Download audio file (opus format preferred)
- Download thumbnail if available
- Move files to organized directory structure
- Create Sound database entry
- Link Stream to Sound
- Add to main playlist
- Trigger normalization
5. **Status Updates**: Stream status updated throughout process
6. **Completion**: Stream marked as "completed" with sound_id
### Play Modes
- **single**: Play one track and stop
- **continuous**: Play through playlist once
- **loop-playlist**: Repeat entire playlist
- **loop-one**: Repeat current track
- **random**: Random track selection
### Error Handling
- **Stream processing errors**: Detailed error messages stored in Stream.error
- **VLC playback errors**: Graceful fallbacks and process cleanup
- **File system errors**: Validation and cleanup of incomplete downloads
- **Database errors**: Transaction rollbacks and error logging