quaik8/sdb-claude
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
sdb-claude/CLAUDE.md

529 lines
22 KiB
Markdown
Raw Permalink Blame History

# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
**Soundboard V2** - A full-stack web application for managing and playing sound files with streaming capabilities.
### Core Features
- **Sound Management**: Upload, organize, and play sound files
- **Stream Processing**: Download and process audio from YouTube, SoundCloud, and other platforms using yt-dlp
- **Music Player**: Full-featured VLC-based music player with playlist support
- **Real-time Updates**: WebSocket support for live player state synchronization
- **Multi-Provider OAuth**: Google and GitHub authentication with JWT tokens
- **Credits System**: Usage tracking with tiered subscription plans
- **Sound Normalization**: Automatic audio processing and normalization
## Project Structure
This is a full-stack application with separate backend (Flask) and frontend (React) components:
- **Backend** (`backend/`): Flask application with clean architecture
- Entry point: `main.py`
- App structure: `app/` with separate `routes/` and `services/` directories
- Business logic in services, routes handle HTTP concerns only
- Uses modern Python tooling (Black, Ruff, pytest)
- CORS enabled for frontend integration
- Python 3.12+ required
- Real-time features via Flask-SocketIO
- **Frontend** (`frontend/`): React application with Vite
- Uses React 19, TypeScript, React Router v7, and Tailwind CSS v4
- Shadcn/ui components with consistent sidebar layout for authenticated users
- Built with Vite for fast development and production builds
- Real-time updates via Socket.IO client
## Development Commands
### Backend (Flask)
```bash
cd backend
uv run python main.py # Run the Flask development server with SocketIO (localhost:5000)
uv run pytest # Run tests
uv run pytest tests/ # Run specific test directory
uv run ruff check # Lint code
uv run ruff format # Format code
uv run black . # Format code (alternative to ruff format)
# Database commands
uv run python migrate_db.py init-db # Initialize database tables and seed data
uv run python migrate_db.py reset-db # Reset database (drop and recreate all tables with seed data)
uv run flask --app app db init # Initialize migrations (first time only)
uv run flask --app app db migrate # Create new migration
uv run flask --app app db upgrade # Apply migrations
```
#### Key Dependencies
- **yt-dlp**: YouTube and streaming platform downloader
- **python-vlc**: VLC media player Python bindings
- **flask-socketio**: Real-time WebSocket communication
- **ffmpeg-python**: Audio processing and normalization
- **apscheduler**: Background task scheduling
#### Environment Variables
```bash
# Required for sessions and JWT
export SECRET_KEY="your_secret_key_for_sessions"
export JWT_SECRET_KEY="your_jwt_secret_key"
# OAuth Providers (configure as needed)
# Google OAuth
export GOOGLE_CLIENT_ID="your_google_client_id"
export GOOGLE_CLIENT_SECRET="your_google_client_secret"
# GitHub OAuth
export GITHUB_CLIENT_ID="your_github_client_id"
export GITHUB_CLIENT_SECRET="your_github_client_secret"
# Stream Processing (optional - defaults provided)
export STREAM_MAX_CONCURRENT="2" # Number of concurrent downloads
```
### Frontend (React/Vite)
```bash
cd frontend
bun dev # Start development server (preferred) on localhost:3000
# Alternative package managers:
npm run dev / yarn dev / pnpm dev
bun run build # Build for production
bun run start # Start production server
bun run lint # Run ESLint
```
#### Key Dependencies
- **@number-flow/react**: Animated number transitions
- **socket.io-client**: Real-time WebSocket communication
- **next-themes**: Dark/light theme support
- **react-router**: Client-side routing (v7)
- **lucide-react**: Icon library
- **@radix-ui**: Headless UI components
## Code Style
### Backend
- Line length: 80 characters
- Uses Ruff with comprehensive rule set (ALL rules enabled with specific ignores)
- Black formatting enforced
- Ignores: D100 (module docstrings), D104 (package docstrings)
### Frontend
- TypeScript with strict configuration
- ESLint with Next.js configuration
- Tailwind CSS for styling
- Uses Geist font family (sans and mono variants)
## Backend Architecture
The Flask backend follows a clean architecture pattern:
- **Routes** (`app/routes/`): Handle HTTP requests/responses, delegate to services
- **Services** (`app/services/`): Contain all business logic, pure functions when possible
- **Application Factory** (`app/__init__.py`): Creates and configures Flask app
Key principles:
- Routes should be thin - only handle HTTP concerns
- Business logic lives in services
- Services are testable in isolation
- Clear separation of concerns
## Authentication
The backend implements multi-provider OAuth authentication (Google, GitHub) with JWT tokens using Flask-JWT-Extended:
### Available Endpoints
**OAuth Authentication:**
- `GET /api/auth/providers` - Get list of available OAuth providers
- `GET /api/auth/login/<provider>` - Initiate OAuth login for specified provider (google, github)
- `GET /api/auth/callback/<provider>` - Handle OAuth callback from specified provider
**Password Authentication:**
- `POST /api/auth/register` - Register new user with email, password, and name
- `POST /api/auth/login` - Login with email and password
**Session Management:**
- `GET /api/auth/logout` - Logout current user (clears cookies)
- `GET /api/auth/me` - Get current user information (requires JWT)
- `POST /api/auth/refresh` - Refresh access token using refresh token
**Multi-Provider Management:**
- `GET /api/auth/link/<provider>` - Link additional OAuth provider to current user (requires JWT)
- `GET /api/auth/link/callback/<provider>` - Handle OAuth callback for linking provider
- `DELETE /api/auth/unlink/<provider>` - Unlink OAuth provider from current user (requires JWT)
**API Token Management:**
- `POST /api/auth/regenerate-api-token` - Generate new API token for current user (requires JWT)
**Example Endpoints:**
- `GET /api/protected` - Example protected endpoint (requires authentication)
- `GET /api/api-protected` - Example endpoint that accepts JWT or API token authentication
- `GET /api/admin` - Example admin-only endpoint (requires admin role)
- `GET /api/use-credits/<amount>` - Example endpoint that costs 5 credits to use
- `GET /api/expensive-operation` - Example endpoint that costs 10 credits to use
### Flask-JWT-Extended Features
- **Access Token**: Short-lived (15 minutes), contains user identity and claims
- **Refresh Token**: Long-lived (7 days), used to generate new access tokens
- **HTTP-only Cookies**: Automatic secure storage with Flask-JWT-Extended
- **Built-in Security**: CSRF protection, secure cookie settings
- **Cookie Paths**: Access tokens work on `/api/`, refresh tokens on `/api/auth/refresh`
### Usage Flow
**Password Authentication:**
1. **Register**: `POST /api/auth/register` with `{"email": "...", "password": "...", "name": "..."}`
2. **Login**: `POST /api/auth/login` with `{"email": "...", "password": "..."}`
3. JWT tokens are automatically set as HTTP-only cookies
4. Access protected endpoints - Flask-JWT-Extended handles token validation
**OAuth Authentication:**
1. Call `/api/auth/providers` to get available OAuth providers
2. Call `/api/auth/login/<provider>` to initiate OAuth flow (e.g., `/api/auth/login/google` or `/api/auth/login/github`)
3. Redirect user to the returned OAuth URL
4. Provider redirects back to `/api/auth/callback/<provider>` with authorization code
5. JWT tokens are automatically set as HTTP-only cookies
6. Access protected endpoints - Flask-JWT-Extended handles token validation
**Multi-Provider Management:**
1. User authenticates with password or OAuth provider
2. Call `/api/auth/link/<provider>` to link additional OAuth providers
3. Users can sign in with password or any linked OAuth provider
4. Call `DELETE /api/auth/unlink/<provider>` to remove OAuth providers (minimum 1 auth method required)
5. User data includes `providers` array showing all available authentication methods
### Authentication Components
- **User Model**: Stores user profile information (email, name, picture, role)
- **UserOAuth Model**: Stores provider-specific authentication data
- **AuthService**: Handles multi-provider OAuth flow and Flask-JWT-Extended integration
- **TokenService**: Simplified token generation using Flask-JWT-Extended
- **OAuthProviderRegistry**: Manages available OAuth providers based on environment variables
- **OAuthProvider**: Abstract base class for OAuth providers (Google, GitHub, etc.)
- **Decorators**: `@require_auth`, `@require_admin`, `@require_role()` for access control
- **Flask-JWT-Extended**: Handles cookie management, validation, and security
### Database Schema
- **users** table: Core user information (id, email, name, picture, password_hash, role, is_active, api_token, api_token_expires_at, timestamps)
- **user_oauth** table: Provider-specific data (user_id, provider, provider_id, email, name, picture)
- **Relationships**: One user can have multiple OAuth providers, enabling flexible authentication
- **Authentication**: Users can authenticate via password OR OAuth providers (or both)
- **User Status**: Users have an `is_active` field (default: true) for account management
- **API Tokens**: Each user automatically gets an API token with no expiration for programmatic access
- **Roles**: First user gets "admin" role, subsequent users get "user" role by default
### Supported Authentication Methods
- **Password**: Traditional email/password authentication with Werkzeug secure hashing
- **Google OAuth**: OpenID Connect with `openid email profile` scopes
- **GitHub OAuth**: OAuth 2.0 with `user:email` scope to access user profile and email
- **API Token**: Long-lived tokens for programmatic access (no expiration by default)
### Role-Based Access Control
- **Admin Role**: First user automatically gets admin privileges
- **User Role**: Default role for all subsequent users
- **Role Assignment**: Automatic during registration (password or OAuth)
- **Access Control**: Use decorators to protect routes by role
- **JWT Integration**: Role included in JWT tokens for stateless authorization
### Available Authentication Decorators
- `@require_auth` - Requires authentication (JWT or API token)
- `@require_role("role_name")` - Requires specific role (chainable with require_auth)
- `@require_credits(amount)` - Requires and deducts specified credits from user
### Multi-Authentication Benefits
- **Flexible Registration**: Register with email/password or OAuth providers
- **Account Linking**: Link multiple authentication methods to one account
- **Flexible Sign-in**: Sign in with password, Google, or GitHub
- **API Access**: Programmatic access via API tokens for scripts and applications
- **Account Recovery**: Multiple authentication options prevent lockout
- **Data Consistency**: Single user profile updated from any authentication method
- **Security**: Werkzeug secure password hashing and OAuth security best practices
- **Role-Based Access**: Fine-grained permission control with automatic admin assignment
### API Token Usage
**Getting Your API Token:**
1. Authenticate via password or OAuth to get JWT tokens
2. Call `POST /api/auth/regenerate-api-token` to get a new API token
3. Use the returned token for programmatic access
**Using API Tokens:**
```bash
# Include API token in Authorization header
curl -H "Authorization: Bearer YOUR_API_TOKEN" http://localhost:5000/api/api-protected
# Alternative format
curl -H "Authorization: Token YOUR_API_TOKEN" http://localhost:5000/api/api-protected
```
**API Token Features:**
- **No Expiration**: Tokens don't expire by default (can be configured per token)
- **Regeneration**: Users can regenerate tokens at any time via `/api/auth/regenerate-api-token`
- **Automatic Creation**: New tokens generated automatically during user registration
- **Role Support**: Tokens inherit user's role for role-based access control
- **Security**: 32-byte URL-safe tokens using `secrets.token_urlsafe()`
## Plan System
The backend includes a subscription plan system that assigns users to different plans with varying credit limits:
### Available Plans
- **Free Plan** (`free`): 25 credits (75 max) - Default plan for new users
- **Premium Plan** (`premium`): 50 credits (150 max) - Enhanced features with increased limits
- **Pro Plan** (`pro`): 100 credits (300 max) - Full access with unlimited usage
### Plan Assignment
- **First User**: Automatically assigned to Pro plan with admin role
- **Subsequent Users**: Automatically assigned to Free plan with user role
- **Plan Information**: Included in all authentication responses (login, register, /me endpoint)
### Database Schema
- **plans** table: id, code, name, description, credits, max_credits
- **users.plan_id**: Foreign key to plans table (required)
- **users.credits**: Current user credits (initialized from plan.credits)
- **Relationship**: Each user belongs to exactly one plan
### Automatic Initialization
The plan system is automatically initialized when the Flask app starts:
- **Database Creation**: `db.create_all()` creates all tables including plans and users
- **Plan Seeding**: Automatically seeds the three default plans if the plans table is empty
- **User Migration**: Automatically assigns plans and credits to existing users who don't have them
- **First User**: Gets Pro plan (100 credits) and admin role automatically
- **Subsequent Users**: Get Free plan (25 credits) and user role automatically
No manual migration scripts are needed - everything happens automatically on app startup.
### Plan Information in API Responses
All user data returned by authentication endpoints includes plan and credits information:
```json
{
"user": {
"id": "1",
"email": "user@example.com",
"name": "User Name",
"role": "user",
"credits": 25,
"plan": {
"id": 1,
"code": "free",
"name": "Free Plan",
"description": "Basic features with limited usage",
"credits": 25,
"max_credits": 75
}
}
}
```
### Credits System
- **User Credits**: Each user has a `credits` field tracking their current available credits
- **Initial Credits**: Set to the plan's `credits` value when user is created
- **Plan Credits**: The default credits amount for the plan (what new users get)
- **Max Credits**: The maximum credits a user on this plan can have
- **Authentication**: Credits are included in JWT tokens and all auth responses
### Credit Usage Decorator
Use the `@require_credits(amount)` decorator to protect endpoints that consume credits:
```python
from app.services.decorators import require_auth, require_credits
@bp.route("/ai-generation")
@require_auth
@require_credits(10) # Costs 10 credits to use
def ai_generation():
"""AI generation endpoint that costs 10 credits."""
return {"result": "AI generated content"}
```
**Features:**
- **Automatic Deduction**: Credits are deducted from user's balance before endpoint execution
- **Insufficient Credits**: Returns HTTP 402 (Payment Required) with clear error message
- **Database Updates**: Credits are updated in real-time in the database
- **Authentication**: Works with both JWT and API token authentication
- **Error Handling**: If endpoint fails, credits are still deducted (transaction-like behavior)
**Example Error Response:**
```json
{
"error": "Insufficient credits. Required: 10, Available: 5"
}
```
## Soundboard System
The application includes a comprehensive sound management system with multiple sound types and real-time playback capabilities.
### Sound Types
- **SDB (Soundboard)**: Traditional soundboard effects and clips stored in `sounds/soundboard/`
- **SAY (Text-to-Speech)**: Generated speech audio files stored in `sounds/say/`
- **STR (Stream)**: Downloaded audio from streaming platforms stored in `sounds/stream/`
### Database Models
#### Sound Model
- **Core fields**: id, type, name, filename, duration, size, hash
- **Normalization**: normalized_filename, normalized_duration, normalized_size, normalized_hash
- **Properties**: is_normalized, is_music, is_deletable, play_count
- **Relationships**: playlist_sounds, streams
- **Methods**: increment_play_count(), set_normalized_info(), find_by_hash()
#### Stream Model
- **Metadata**: service, service_id, url, title, track, artist, album, genre
- **Status tracking**: status (pending/processing/completed/failed), error
- **Relationships**: Links to Sound model when processing completes
- **Constraints**: Unique constraint on service + service_id
#### Playlist Models
- **Playlist**: id, name, description, genre, user_id, is_main, is_current
- **PlaylistSound**: Junction table with playlist_id, sound_id, order
- **Methods**: add_sound(), find_current_playlist(), find_main_playlist()
### API Endpoints
#### Soundboard Routes (`/api/soundboard/`)
- `GET /sounds` - Get all soundboard sounds with type filtering
- `POST /sounds/<id>/play` - Play a sound (costs 1 credit)
- `POST /stop-all` - Stop all currently playing sounds
- `POST /force-stop` - Force stop with aggressive cleanup
- `GET /status` - Get current playback status
#### Stream Routes (`/api/stream/`)
- `POST /add-url` - Add streaming URL to processing queue
#### Music Player Routes (`/api/player/`)
- `GET /state` - Get current player state
- `POST /play` - Start playback
- `POST /pause` - Pause playback
- `POST /stop` - Stop playback
- `POST /next` - Skip to next track
- `POST /previous` - Skip to previous track
- `POST /seek` - Seek to position (0.0-1.0)
- `POST /volume` - Set volume (0-100)
- `POST /mode` - Set play mode (continuous/loop-playlist/loop-one/random/single)
- `POST /playlist` - Load playlist into player
- `POST /play-track` - Play track at specific index
#### Admin Routes (`/api/admin/`)
- Sound management endpoints for admin users
- User management endpoints
### Services
#### StreamProcessingService
- **Queue-based processing**: Multi-threaded download queue with configurable concurrency
- **Metadata extraction**: Uses yt-dlp to extract video/audio metadata
- **Download processing**: Downloads audio in opus format with thumbnails
- **File management**: Moves files to organized directory structure
- **Database integration**: Creates Sound entries and links to Stream records
- **Playlist integration**: Automatically adds processed sounds to main playlist
- **Normalization**: Triggers automatic audio normalization after download
#### MusicPlayerService
- **VLC integration**: Uses python-vlc bindings for audio playback
- **Playlist management**: Full playlist support with multiple play modes
- **Real-time sync**: Background thread syncs player state with VLC
- **Play tracking**: Tracks listening time and play counts (20% completion threshold)
- **WebSocket updates**: Emits real-time player state via SocketIO
- **Multi-format support**: Plays normalized and original audio files
#### VLCService
- **Process management**: Handles multiple VLC instances for sound effects
- **Force stop capabilities**: Aggressive cleanup for stuck processes
- **Sound tracking**: Records play counts and user interactions
#### SoundNormalizerService
- **Audio processing**: FFmpeg-based normalization with ReplayGain
- **Two-pass processing**: Optional high-quality normalization
- **Batch operations**: Queue-based processing for multiple files
#### SocketIOService
- **Real-time communication**: WebSocket server for live updates
- **Player state sync**: Broadcasts player state changes to all clients
- **Sound event notifications**: Play count updates and status changes
### File Organization
```
sounds/
├── soundboard/ # Original soundboard files
├── say/ # Text-to-speech files
├── stream/ # Downloaded streaming audio
│ └── thumbnails/ # Video thumbnails
├── normalized/ # Normalized versions
│ ├── soundboard/
│ ├── say/
│ └── stream/
└── temp/ # Temporary download files
```
### Real-time Features
#### WebSocket Events
- **player_state_update**: Complete player state including current track, position, volume
- **sound_play_count_changed**: When sound play counts are updated
- **stream_processing_update**: Status updates for stream downloads
#### Player State Structure
```json
{
"is_playing": true,
"current_time": 45000,
"duration": 180000,
"volume": 80,
"play_mode": "continuous",
"current_track": {
"id": 123,
"title": "Song Title",
"artist": "Artist Name",
"duration": 180000,
"thumbnail": "http://localhost:5000/api/sounds/str/thumbnails/image.jpg",
"service_url": "https://youtube.com/watch?v=...",
"type": "STR"
},
"current_track_index": 2,
"playlist": [...],
"playlist_id": 1
}
```
### Stream Processing Workflow
1. **URL Submission**: User submits streaming URL via `/api/stream/add-url`
2. **Validation**: Check for duplicate URLs and extract basic metadata
3. **Queue Addition**: Add stream to processing queue with "pending" status
4. **Background Processing**: Worker thread processes stream:
- Extract full metadata with yt-dlp
- Download audio file (opus format preferred)
- Download thumbnail if available
- Move files to organized directory structure
- Create Sound database entry
- Link Stream to Sound
- Add to main playlist
- Trigger normalization
5. **Status Updates**: Stream status updated throughout process
6. **Completion**: Stream marked as "completed" with sound_id
### Play Modes
- **single**: Play one track and stop
- **continuous**: Play through playlist once
- **loop-playlist**: Repeat entire playlist
- **loop-one**: Repeat current track
- **random**: Random track selection
### Error Handling
- **Stream processing errors**: Detailed error messages stored in Stream.error
- **VLC playback errors**: Graceful fallbacks and process cleanup
- **File system errors**: Validation and cleanup of incomplete downloads
- **Database errors**: Transaction rollbacks and error logging
Reference in New Issue View Git Blame Copy Permalink