sdb-claude/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Soundboard V2 - A full-stack web application for managing and playing sound files with streaming capabilities.

Core Features

• Sound Management: Upload, organize, and play sound files
• Stream Processing: Download and process audio from YouTube, SoundCloud, and other platforms using yt-dlp
• Music Player: Full-featured VLC-based music player with playlist support
• Real-time Updates: WebSocket support for live player state synchronization
• Multi-Provider OAuth: Google and GitHub authentication with JWT tokens
• Credits System: Usage tracking with tiered subscription plans
• Sound Normalization: Automatic audio processing and normalization

Project Structure

This is a full-stack application with separate backend (Flask) and frontend (React) components:

• Backend (backend/): Flask application with clean architecture

  • Entry point: main.py
  • App structure: app/ with separate routes/ and services/ directories
  • Business logic in services, routes handle HTTP concerns only
  • Uses modern Python tooling (Black, Ruff, pytest)
  • CORS enabled for frontend integration
  • Python 3.12+ required
  • Real-time features via Flask-SocketIO

• Frontend (frontend/): React application with Vite

  • Uses React 19, TypeScript, React Router v7, and Tailwind CSS v4
  • Shadcn/ui components with consistent sidebar layout for authenticated users
  • Built with Vite for fast development and production builds
  • Real-time updates via Socket.IO client

Development Commands

Backend (Flask)

cd backend
uv run python main.py         # Run the Flask development server with SocketIO (localhost:5000)
uv run pytest                 # Run tests
uv run pytest tests/          # Run specific test directory
uv run ruff check             # Lint code
uv run ruff format            # Format code
uv run black .                # Format code (alternative to ruff format)

# Database commands
uv run python migrate_db.py init-db    # Initialize database tables and seed data
uv run python migrate_db.py reset-db   # Reset database (drop and recreate all tables with seed data)
uv run flask --app app db init         # Initialize migrations (first time only)
uv run flask --app app db migrate      # Create new migration
uv run flask --app app db upgrade      # Apply migrations

Key Dependencies

• yt-dlp: YouTube and streaming platform downloader
• python-vlc: VLC media player Python bindings
• flask-socketio: Real-time WebSocket communication
• ffmpeg-python: Audio processing and normalization
• apscheduler: Background task scheduling

Environment Variables

# Required for sessions and JWT
export SECRET_KEY="your_secret_key_for_sessions"
export JWT_SECRET_KEY="your_jwt_secret_key"

# OAuth Providers (configure as needed)
# Google OAuth
export GOOGLE_CLIENT_ID="your_google_client_id"
export GOOGLE_CLIENT_SECRET="your_google_client_secret"

# GitHub OAuth  
export GITHUB_CLIENT_ID="your_github_client_id"
export GITHUB_CLIENT_SECRET="your_github_client_secret"

# Stream Processing (optional - defaults provided)
export STREAM_MAX_CONCURRENT="2"              # Number of concurrent downloads

Frontend (React/Vite)

cd frontend
bun dev                        # Start development server (preferred) on localhost:3000
# Alternative package managers:
npm run dev / yarn dev / pnpm dev
bun run build                  # Build for production
bun run start                  # Start production server
bun run lint                   # Run ESLint

Key Dependencies

• @number-flow/react: Animated number transitions
• socket.io-client: Real-time WebSocket communication
• next-themes: Dark/light theme support
• react-router: Client-side routing (v7)
• lucide-react: Icon library
• @radix-ui: Headless UI components

Code Style

Backend

• Line length: 80 characters
• Uses Ruff with comprehensive rule set (ALL rules enabled with specific ignores)
• Black formatting enforced
• Ignores: D100 (module docstrings), D104 (package docstrings)

Frontend

• TypeScript with strict configuration
• ESLint with Next.js configuration
• Tailwind CSS for styling
• Uses Geist font family (sans and mono variants)

Backend Architecture

The Flask backend follows a clean architecture pattern:

• Routes (app/routes/): Handle HTTP requests/responses, delegate to services
• Services (app/services/): Contain all business logic, pure functions when possible
• Application Factory (app/__init__.py): Creates and configures Flask app

Key principles:

• Routes should be thin - only handle HTTP concerns
• Business logic lives in services
• Services are testable in isolation
• Clear separation of concerns

Authentication

The backend implements multi-provider OAuth authentication (Google, GitHub) with JWT tokens using Flask-JWT-Extended:

Available Endpoints

OAuth Authentication:

• GET /api/auth/providers - Get list of available OAuth providers
• GET /api/auth/login/<provider> - Initiate OAuth login for specified provider (google, github)
• GET /api/auth/callback/<provider> - Handle OAuth callback from specified provider

Password Authentication:

• POST /api/auth/register - Register new user with email, password, and name
• POST /api/auth/login - Login with email and password

Session Management:

• GET /api/auth/logout - Logout current user (clears cookies)
• GET /api/auth/me - Get current user information (requires JWT)
• POST /api/auth/refresh - Refresh access token using refresh token

Multi-Provider Management:

• GET /api/auth/link/<provider> - Link additional OAuth provider to current user (requires JWT)
• GET /api/auth/link/callback/<provider> - Handle OAuth callback for linking provider
• DELETE /api/auth/unlink/<provider> - Unlink OAuth provider from current user (requires JWT)

API Token Management:

• POST /api/auth/regenerate-api-token - Generate new API token for current user (requires JWT)

Example Endpoints:

• GET /api/protected - Example protected endpoint (requires authentication)
• GET /api/api-protected - Example endpoint that accepts JWT or API token authentication
• GET /api/admin - Example admin-only endpoint (requires admin role)
• GET /api/use-credits/<amount> - Example endpoint that costs 5 credits to use
• GET /api/expensive-operation - Example endpoint that costs 10 credits to use

Flask-JWT-Extended Features

• Access Token: Short-lived (15 minutes), contains user identity and claims
• Refresh Token: Long-lived (7 days), used to generate new access tokens
• HTTP-only Cookies: Automatic secure storage with Flask-JWT-Extended
• Built-in Security: CSRF protection, secure cookie settings
• Cookie Paths: Access tokens work on /api/, refresh tokens on /api/auth/refresh

Usage Flow

Password Authentication:

1. Register: POST /api/auth/register with {"email": "...", "password": "...", "name": "..."}
2. Login: POST /api/auth/login with {"email": "...", "password": "..."}
3. JWT tokens are automatically set as HTTP-only cookies
4. Access protected endpoints - Flask-JWT-Extended handles token validation

OAuth Authentication:

1. Call /api/auth/providers to get available OAuth providers
2. Call /api/auth/login/<provider> to initiate OAuth flow (e.g., /api/auth/login/google or /api/auth/login/github)
3. Redirect user to the returned OAuth URL
4. Provider redirects back to /api/auth/callback/<provider> with authorization code
5. JWT tokens are automatically set as HTTP-only cookies
6. Access protected endpoints - Flask-JWT-Extended handles token validation

Multi-Provider Management:

1. User authenticates with password or OAuth provider
2. Call /api/auth/link/<provider> to link additional OAuth providers
3. Users can sign in with password or any linked OAuth provider
4. Call DELETE /api/auth/unlink/<provider> to remove OAuth providers (minimum 1 auth method required)
5. User data includes providers array showing all available authentication methods

Authentication Components

• User Model: Stores user profile information (email, name, picture, role)
• UserOAuth Model: Stores provider-specific authentication data
• AuthService: Handles multi-provider OAuth flow and Flask-JWT-Extended integration
• TokenService: Simplified token generation using Flask-JWT-Extended
• OAuthProviderRegistry: Manages available OAuth providers based on environment variables
• OAuthProvider: Abstract base class for OAuth providers (Google, GitHub, etc.)
• Decorators: @require_auth, @require_admin, @require_role() for access control
• Flask-JWT-Extended: Handles cookie management, validation, and security

Database Schema

• users table: Core user information (id, email, name, picture, password_hash, role, is_active, api_token, api_token_expires_at, timestamps)
• user_oauth table: Provider-specific data (user_id, provider, provider_id, email, name, picture)
• Relationships: One user can have multiple OAuth providers, enabling flexible authentication
• Authentication: Users can authenticate via password OR OAuth providers (or both)
• User Status: Users have an is_active field (default: true) for account management
• API Tokens: Each user automatically gets an API token with no expiration for programmatic access
• Roles: First user gets "admin" role, subsequent users get "user" role by default

Supported Authentication Methods

• Password: Traditional email/password authentication with Werkzeug secure hashing
• Google OAuth: OpenID Connect with openid email profile scopes
• GitHub OAuth: OAuth 2.0 with user:email scope to access user profile and email
• API Token: Long-lived tokens for programmatic access (no expiration by default)

Role-Based Access Control

• Admin Role: First user automatically gets admin privileges
• User Role: Default role for all subsequent users
• Role Assignment: Automatic during registration (password or OAuth)
• Access Control: Use decorators to protect routes by role
• JWT Integration: Role included in JWT tokens for stateless authorization

Available Authentication Decorators

• @require_auth - Requires authentication (JWT or API token)
• @require_role("role_name") - Requires specific role (chainable with require_auth)
• @require_credits(amount) - Requires and deducts specified credits from user

Multi-Authentication Benefits

• Flexible Registration: Register with email/password or OAuth providers
• Account Linking: Link multiple authentication methods to one account
• Flexible Sign-in: Sign in with password, Google, or GitHub
• API Access: Programmatic access via API tokens for scripts and applications
• Account Recovery: Multiple authentication options prevent lockout
• Data Consistency: Single user profile updated from any authentication method
• Security: Werkzeug secure password hashing and OAuth security best practices
• Role-Based Access: Fine-grained permission control with automatic admin assignment

API Token Usage

Getting Your API Token:

1. Authenticate via password or OAuth to get JWT tokens
2. Call POST /api/auth/regenerate-api-token to get a new API token
3. Use the returned token for programmatic access

Using API Tokens:

# Include API token in Authorization header
curl -H "Authorization: Bearer YOUR_API_TOKEN" http://localhost:5000/api/api-protected

# Alternative format
curl -H "Authorization: Token YOUR_API_TOKEN" http://localhost:5000/api/api-protected

API Token Features:

• No Expiration: Tokens don't expire by default (can be configured per token)
• Regeneration: Users can regenerate tokens at any time via /api/auth/regenerate-api-token
• Automatic Creation: New tokens generated automatically during user registration
• Role Support: Tokens inherit user's role for role-based access control
• Security: 32-byte URL-safe tokens using secrets.token_urlsafe()

Plan System

The backend includes a subscription plan system that assigns users to different plans with varying credit limits:

Available Plans

• Free Plan (free): 25 credits (75 max) - Default plan for new users
• Premium Plan (premium): 50 credits (150 max) - Enhanced features with increased limits
• Pro Plan (pro): 100 credits (300 max) - Full access with unlimited usage

Plan Assignment

• First User: Automatically assigned to Pro plan with admin role
• Subsequent Users: Automatically assigned to Free plan with user role
• Plan Information: Included in all authentication responses (login, register, /me endpoint)

Database Schema

• plans table: id, code, name, description, credits, max_credits
• users.plan_id: Foreign key to plans table (required)
• users.credits: Current user credits (initialized from plan.credits)
• Relationship: Each user belongs to exactly one plan

Automatic Initialization

The plan system is automatically initialized when the Flask app starts:

• Database Creation: db.create_all() creates all tables including plans and users
• Plan Seeding: Automatically seeds the three default plans if the plans table is empty
• User Migration: Automatically assigns plans and credits to existing users who don't have them
• First User: Gets Pro plan (100 credits) and admin role automatically
• Subsequent Users: Get Free plan (25 credits) and user role automatically

No manual migration scripts are needed - everything happens automatically on app startup.

Plan Information in API Responses

All user data returned by authentication endpoints includes plan and credits information:

{
  "user": {
    "id": "1",
    "email": "user@example.com",
    "name": "User Name",
    "role": "user",
    "credits": 25,
    "plan": {
      "id": 1,
      "code": "free",
      "name": "Free Plan", 
      "description": "Basic features with limited usage",
      "credits": 25,
      "max_credits": 75
    }
  }
}

Credits System

• User Credits: Each user has a credits field tracking their current available credits
• Initial Credits: Set to the plan's credits value when user is created
• Plan Credits: The default credits amount for the plan (what new users get)
• Max Credits: The maximum credits a user on this plan can have
• Authentication: Credits are included in JWT tokens and all auth responses

Credit Usage Decorator

Use the @require_credits(amount) decorator to protect endpoints that consume credits:

from app.services.decorators import require_auth, require_credits

@bp.route("/ai-generation")
@require_auth
@require_credits(10)  # Costs 10 credits to use
def ai_generation():
    """AI generation endpoint that costs 10 credits."""
    return {"result": "AI generated content"}

Features:

• Automatic Deduction: Credits are deducted from user's balance before endpoint execution
• Insufficient Credits: Returns HTTP 402 (Payment Required) with clear error message
• Database Updates: Credits are updated in real-time in the database
• Authentication: Works with both JWT and API token authentication
• Error Handling: If endpoint fails, credits are still deducted (transaction-like behavior)

Example Error Response:

{
  "error": "Insufficient credits. Required: 10, Available: 5"
}

Soundboard System

The application includes a comprehensive sound management system with multiple sound types and real-time playback capabilities.

Sound Types

• SDB (Soundboard): Traditional soundboard effects and clips stored in sounds/soundboard/
• SAY (Text-to-Speech): Generated speech audio files stored in sounds/say/
• STR (Stream): Downloaded audio from streaming platforms stored in sounds/stream/

Database Models

Sound Model

• Core fields: id, type, name, filename, duration, size, hash
• Normalization: normalized_filename, normalized_duration, normalized_size, normalized_hash
• Properties: is_normalized, is_music, is_deletable, play_count
• Relationships: playlist_sounds, streams
• Methods: increment_play_count(), set_normalized_info(), find_by_hash()

Stream Model

• Metadata: service, service_id, url, title, track, artist, album, genre
• Status tracking: status (pending/processing/completed/failed), error
• Relationships: Links to Sound model when processing completes
• Constraints: Unique constraint on service + service_id

Playlist Models

• Playlist: id, name, description, genre, user_id, is_main, is_current
• PlaylistSound: Junction table with playlist_id, sound_id, order
• Methods: add_sound(), find_current_playlist(), find_main_playlist()

API Endpoints

Soundboard Routes (/api/soundboard/)

• GET /sounds - Get all soundboard sounds with type filtering
• POST /sounds/<id>/play - Play a sound (costs 1 credit)
• POST /stop-all - Stop all currently playing sounds
• POST /force-stop - Force stop with aggressive cleanup
• GET /status - Get current playback status

Stream Routes (/api/stream/)

• POST /add-url - Add streaming URL to processing queue

Music Player Routes (/api/player/)

• GET /state - Get current player state
• POST /play - Start playback
• POST /pause - Pause playback
• POST /stop - Stop playback
• POST /next - Skip to next track
• POST /previous - Skip to previous track
• POST /seek - Seek to position (0.0-1.0)
• POST /volume - Set volume (0-100)
• POST /mode - Set play mode (continuous/loop-playlist/loop-one/random/single)
• POST /playlist - Load playlist into player
• POST /play-track - Play track at specific index

Admin Routes (/api/admin/)

• Sound management endpoints for admin users
• User management endpoints

Services

StreamProcessingService

• Queue-based processing: Multi-threaded download queue with configurable concurrency
• Metadata extraction: Uses yt-dlp to extract video/audio metadata
• Download processing: Downloads audio in opus format with thumbnails
• File management: Moves files to organized directory structure
• Database integration: Creates Sound entries and links to Stream records
• Playlist integration: Automatically adds processed sounds to main playlist
• Normalization: Triggers automatic audio normalization after download

MusicPlayerService

• VLC integration: Uses python-vlc bindings for audio playback
• Playlist management: Full playlist support with multiple play modes
• Real-time sync: Background thread syncs player state with VLC
• Play tracking: Tracks listening time and play counts (20% completion threshold)
• WebSocket updates: Emits real-time player state via SocketIO
• Multi-format support: Plays normalized and original audio files

VLCService

• Process management: Handles multiple VLC instances for sound effects
• Force stop capabilities: Aggressive cleanup for stuck processes
• Sound tracking: Records play counts and user interactions

SoundNormalizerService

• Audio processing: FFmpeg-based normalization with ReplayGain
• Two-pass processing: Optional high-quality normalization
• Batch operations: Queue-based processing for multiple files

SocketIOService

• Real-time communication: WebSocket server for live updates
• Player state sync: Broadcasts player state changes to all clients
• Sound event notifications: Play count updates and status changes

File Organization

sounds/
├── soundboard/           # Original soundboard files
├── say/                  # Text-to-speech files  
├── stream/               # Downloaded streaming audio
│   └── thumbnails/       # Video thumbnails
├── normalized/           # Normalized versions
│   ├── soundboard/
│   ├── say/
│   └── stream/
└── temp/                 # Temporary download files

Real-time Features

WebSocket Events

• player_state_update: Complete player state including current track, position, volume
• sound_play_count_changed: When sound play counts are updated
• stream_processing_update: Status updates for stream downloads

Player State Structure

{
  "is_playing": true,
  "current_time": 45000,
  "duration": 180000,
  "volume": 80,
  "play_mode": "continuous",
  "current_track": {
    "id": 123,
    "title": "Song Title",
    "artist": "Artist Name",
    "duration": 180000,
    "thumbnail": "http://localhost:5000/api/sounds/str/thumbnails/image.jpg",
    "service_url": "https://youtube.com/watch?v=...",
    "type": "STR"
  },
  "current_track_index": 2,
  "playlist": [...],
  "playlist_id": 1
}

Stream Processing Workflow

1. URL Submission: User submits streaming URL via /api/stream/add-url
2. Validation: Check for duplicate URLs and extract basic metadata
3. Queue Addition: Add stream to processing queue with "pending" status
4. Background Processing: Worker thread processes stream:
   • Extract full metadata with yt-dlp
   • Download audio file (opus format preferred)
   • Download thumbnail if available
   • Move files to organized directory structure
   • Create Sound database entry
   • Link Stream to Sound
   • Add to main playlist
   • Trigger normalization
5. Status Updates: Stream status updated throughout process
6. Completion: Stream marked as "completed" with sound_id

Play Modes

• single: Play one track and stop
• continuous: Play through playlist once
• loop-playlist: Repeat entire playlist
• loop-one: Repeat current track
• random: Random track selection

Error Handling

• Stream processing errors: Detailed error messages stored in Stream.error
• VLC playback errors: Graceful fallbacks and process cleanup
• File system errors: Validation and cleanup of incomplete downloads
• Database errors: Transaction rollbacks and error logging