Back to DocumentationAPI Reference

SCORM API System Design

SCORM API System Design

Introduction

This document provides detailed system design specifications for the AllureLMS SCORM API, including component interactions, data models, algorithms, and design patterns.

System Components

1. API Layer

Route Handlers

The API layer consists of Next.js API route handlers organized by functionality:

Public API Routes (/api/v1/*):

  • Package management endpoints
  • Session management endpoints
  • Webhook management endpoints
  • Dispatch package endpoints
  • xAPI telemetry endpoints

Customer Routes (/api/customer/*):

  • Self-service dashboard endpoints
  • API key management
  • Usage reports
  • Billing integration

Admin Routes (/api/admin/*):

  • System health monitoring
  • Tenant management
  • SaaS metrics
  • Churn detection

Request Processing Pipeline

Request → Middleware Chain:
  1. CORS Handler
  2. Authentication Middleware
     - Extract API key or Clerk session
     - Validate credentials
     - Resolve tenant_id
  3. Authorization Middleware
     - Check API key scopes
     - Verify tenant access
  4. Rate Limiting Middleware
     - Check rate limits (local or Redis)
     - Return 429 if exceeded
  5. Input Validation
     - Validate request body (Zod)
     - Validate query parameters
  6. Business Logic Handler
  7. Response Formatter
  8. Error Handler

2. Authentication & Authorization Layer

API Key Authentication

interface APIKey {
  id: string;
  tenant_id: string;
  key_hash: string; // SHA-256 hash
  name: string;
  scopes: string[]; // ['read', 'write', 'admin']
  expires_at?: string;
  last_used_at?: string;
  created_at: string;
}

Authentication Flow:

  1. Extract API key from X-API-Key header or Authorization: Bearer header
  2. Hash the provided key with SHA-256
  3. Query database for matching key hash
  4. Validate expiration and active status
  5. Extract tenant_id and scopes
  6. Attach to request context

Clerk Authentication

Authentication Flow:

  1. Extract Clerk session token from cookies
  2. Verify token with Clerk API
  3. Extract userId from token
  4. Query user_tenants table for tenant association
  5. Attach tenant_id to request context

Authorization

Scope-Based Authorization:

  • read: GET requests only
  • write: POST, PUT requests
  • admin: All operations including DELETE

Tenant Isolation:

  • All database queries automatically filtered by tenant_id
  • RLS policies enforce isolation at database level

3. Business Logic Layer

Package Management

Package Upload Process:

async function uploadPackage(file: File, tenantId: string, uploadedBy: string) {
  // 1. Validate tenant quota
  await checkTenantQuota(tenantId);
  
  // 2. Upload file to storage
  const storagePath = await uploadToStorage(file, tenantId);
  
  // 3. Process package
  const manifest = await extractAndParseManifest(storagePath);
  
  // 4. Create package record
  const package = await createPackageRecord({
    tenant_id: tenantId,
    title: manifest.title,
    version: manifest.version,
    manifest_url: manifest.url,
    launch_url: manifest.launchUrl,
    storage_path: storagePath,
    uploaded_by: uploadedBy,
    file_size_bytes: file.size
  });
  
  // 5. Trigger webhook
  await sendWebhookEvent({
    event: 'package.processing.completed',
    tenant_id: tenantId,
    data: { package_id: package.id }
  });
  
  return package;
}

Package Processing:

  1. Extract ZIP file
  2. Locate imsmanifest.xml
  3. Parse SCORM manifest (1.2 or 2004)
  4. Validate manifest structure
  5. Extract metadata (title, version, launch URL)
  6. Store extracted files in organized structure

Session Management

Session Creation:

interface Session {
  id: string;
  package_id: string;
  tenant_id: string;
  user_id: string;
  cmi_data: Record<string, any>;
  completion_status: 'not_attempted' | 'incomplete' | 'completed';
  success_status: 'unknown' | 'passed' | 'failed';
  score: {
    scaled?: number;
    raw?: number;
    min?: number;
    max?: number;
  };
  version: number; // Optimistic locking
  created_at: string;
  updated_at: string;
}

Optimistic Locking:

  • Each session has a version field
  • Updates must include current version
  • Database checks version before update
  • Returns 409 Conflict if version mismatch
  • Client retries with fresh version

CMI Data Merging:

async function updateSession(sessionId: string, updates: Partial<Session>) {
  // 1. Get current session with version
  const current = await getSession(sessionId);
  
  // 2. Merge CMI data
  const mergedCmiData = {
    ...current.cmi_data,
    ...updates.cmi_data
  };
  
  // 3. Attempt update with version check
  try {
    return await updateSessionWithVersion(sessionId, {
      ...updates,
      cmi_data: mergedCmiData,
      version: current.version
    });
  } catch (error) {
    if (error.code === 'VERSION_CONFLICT') {
      // Retry with fresh data
      return updateSession(sessionId, updates);
    }
    throw error;
  }
}

Webhook System

Webhook Delivery:

interface WebhookEvent {
  event: string;
  tenant_id: string;
  data: Record<string, any>;
  timestamp: string;
  delivery_id: string;
}

async function sendWebhookEvent(payload: WebhookEventPayload) {
  // 1. Get active webhooks for event type
  const webhooks = await getActiveWebhooks(payload.tenant_id, payload.event);
  
  // 2. Queue delivery for each webhook
  const deliveries = await Promise.all(
    webhooks.map(webhook => queueWebhookDelivery(webhook, payload))
  );
  
  // 3. Process deliveries asynchronously
  processPendingDeliveries();
  
  return deliveries;
}

Retry Logic:

  • Exponential backoff with jitter
  • Max retries: 5 attempts
  • Retry intervals: 1s, 2s, 4s, 8s, 16s
  • Jitter: ±20% random variation

HMAC Signature:

function generateWebhookSignature(body: string, secret: string): string {
  return crypto
    .createHmac('sha256', secret)
    .update(body)
    .digest('hex');
}

// Verification on client side
const signature = request.headers['x-allure-signature'];
const expectedSignature = generateWebhookSignature(request.body, webhookSecret);
const isValid = crypto.timingSafeEqual(
  Buffer.from(signature),
  Buffer.from(expectedSignature)
);

4. Storage Layer

Storage Abstraction

interface StorageAdapter {
  upload(file: Buffer, path: string): Promise<string>;
  download(path: string): Promise<Buffer>;
  delete(path: string): Promise<void>;
  getPresignedUrl(path: string, expiresIn: number): Promise<string>;
}

class SupabaseStorageAdapter implements StorageAdapter {
  // Supabase Storage implementation
}

class R2StorageAdapter implements StorageAdapter {
  // Cloudflare R2 implementation
}

Storage Selection:

  • If R2 credentials provided → Use R2
  • Else → Use Supabase Storage
  • Automatic failover if R2 unavailable

Multipart Upload

Flow:

  1. Client requests multipart upload initialization
  2. API creates upload record
  3. Client requests presigned URLs for each part
  4. Client uploads parts directly to storage
  5. Client completes upload with ETags
  6. API validates and processes package
interface MultipartUpload {
  id: string;
  tenant_id: string;
  filename: string;
  total_size: number;
  parts: MultipartPart[];
  status: 'pending' | 'completed' | 'failed';
}

interface MultipartPart {
  part_number: number;
  size: number;
  etag?: string;
  presigned_url?: string;
}

5. Database Layer

Schema Design

Core Tables:

-- Packages
scorm_packages (
  id UUID PRIMARY KEY,
  tenant_id UUID NOT NULL,
  title VARCHAR(255),
  version VARCHAR(10), -- '1.2' or '2004'
  manifest_url TEXT,
  launch_url TEXT,
  storage_path TEXT,
  uploaded_by VARCHAR(255),
  file_size_bytes BIGINT,
  metadata JSONB,
  created_at TIMESTAMPTZ,
  updated_at TIMESTAMPTZ
)

-- Sessions
scorm_sessions (
  id UUID PRIMARY KEY,
  package_id UUID REFERENCES scorm_packages(id),
  tenant_id UUID NOT NULL,
  user_id UUID NOT NULL,
  cmi_data JSONB,
  completion_status VARCHAR(20),
  success_status VARCHAR(20),
  score JSONB,
  version INTEGER DEFAULT 1,
  created_at TIMESTAMPTZ,
  updated_at TIMESTAMPTZ
)

-- Events (Audit Log)
scorm_events (
  id UUID PRIMARY KEY,
  tenant_id UUID NOT NULL,
  event_type VARCHAR(50),
  entity_type VARCHAR(50), -- 'package', 'session'
  entity_id UUID,
  user_id UUID,
  metadata JSONB,
  created_at TIMESTAMPTZ
)

Row Level Security (RLS)

RLS Policy Example:

-- Enable RLS
ALTER TABLE scorm_packages ENABLE ROW LEVEL SECURITY;

-- Policy: Users can only see their tenant's packages
CREATE POLICY "tenant_isolation" ON scorm_packages
  FOR ALL
  USING (tenant_id = current_setting('app.tenant_id')::UUID);

-- Function to set tenant context
CREATE FUNCTION set_tenant_context(tenant_uuid UUID)
RETURNS void AS $$
BEGIN
  PERFORM set_config('app.tenant_id', tenant_uuid::text, false);
END;
$$ LANGUAGE plpgsql;

RLS Enforcement:

  • All queries automatically filtered by tenant_id
  • Service role can bypass RLS for admin operations
  • Policies applied at database level (cannot be bypassed by application)

Indexes

Critical Indexes:

-- Tenant isolation
CREATE INDEX idx_packages_tenant ON scorm_packages(tenant_id);
CREATE INDEX idx_sessions_tenant ON scorm_sessions(tenant_id);

-- Lookups
CREATE INDEX idx_sessions_package ON scorm_sessions(package_id);
CREATE INDEX idx_sessions_user ON scorm_sessions(user_id);

-- Composite indexes
CREATE INDEX idx_sessions_tenant_user ON scorm_sessions(tenant_id, user_id);
CREATE INDEX idx_events_tenant_type ON scorm_events(tenant_id, event_type);

6. Rate Limiting

Local Rate Limiting

interface RateLimitStore {
  [key: string]: {
    count: number;
    resetAt: number;
  };
}

class LocalRateLimiter {
  private store: RateLimitStore = {};
  
  async checkLimit(identifier: string, maxRequests: number, windowSeconds: number): Promise<boolean> {
    const key = identifier;
    const now = Date.now();
    const windowMs = windowSeconds * 1000;
    
    const record = this.store[key];
    
    if (!record || now > record.resetAt) {
      // New window
      this.store[key] = {
        count: 1,
        resetAt: now + windowMs
      };
      return true;
    }
    
    if (record.count >= maxRequests) {
      return false; // Rate limit exceeded
    }
    
    record.count++;
    return true;
  }
}

Distributed Rate Limiting (Redis)

class RedisRateLimiter {
  async checkLimit(identifier: string, maxRequests: number, windowSeconds: number): Promise<boolean> {
    const key = `rate_limit:${identifier}`;
    const now = Date.now();
    const windowMs = windowSeconds * 1000;
    
    // Use Redis sliding window log
    const pipeline = redis.pipeline();
    pipeline.zremrangebyscore(key, 0, now - windowMs); // Remove old entries
    pipeline.zcard(key); // Count current entries
    pipeline.zadd(key, now, `${now}-${Math.random()}`); // Add current request
    pipeline.expire(key, windowSeconds); // Set expiration
    
    const results = await pipeline.exec();
    const count = results[1][1] as number;
    
    return count < maxRequests;
  }
}

7. SCORM Player Integration

SCORM API Bridge

The SCORM API Bridge is a JavaScript library that implements the SCORM API specification and communicates with the backend:

class ScormAPIBridge {
  private sessionId: string;
  private apiEndpoint: string;
  
  // SCORM API Methods
  LMSInitialize(): string {
    // Initialize session
    return 'true';
  }
  
  LMSGetValue(element: string): string {
    // Get CMI data from session
    return this.getCmiValue(element);
  }
  
  LMSSetValue(element: string, value: string): string {
    // Update CMI data in session
    return this.setCmiValue(element, value);
  }
  
  LMSCommit(): string {
    // Persist changes to backend
    return this.commitChanges();
  }
  
  LMSFinish(): string {
    // Finalize session
    return 'true';
  }
}

Communication Flow:

  1. Player loads SCORM content
  2. Content calls SCORM API methods
  3. Bridge intercepts calls
  4. Bridge makes HTTP requests to backend API
  5. Backend updates session CMI data
  6. Bridge returns values to content

8. xAPI Integration

SCORM to xAPI Conversion

function convertScormToXAPI(session: Session): xAPIStatement {
  return {
    actor: {
      mbox: `mailto:${session.user_id}@tenant.com`
    },
    verb: {
      id: getVerbFromStatus(session.completion_status),
      display: { 'en-US': getVerbDisplay(session.completion_status) }
    },
    object: {
      id: `https://api.scorm.com/packages/${session.package_id}`,
      definition: {
        type: 'http://adlnet.gov/expapi/activities/course',
        name: { 'en-US': session.package.title }
      }
    },
    result: {
      score: session.score,
      success: session.success_status === 'passed',
      completion: session.completion_status === 'completed',
      duration: `PT${session.time_spent_seconds}S`
    },
    context: {
      registration: session.id
    },
    timestamp: session.updated_at
  };
}

9. Dispatch Package System

Dispatch Package Generation

interface DispatchPackage {
  id: string;
  package_id: string;
  tenant_id: string;
  dispatch_name?: string;
  client_name?: string;
  launch_url: string;
  token: string; // Signed JWT
  registration_limit?: number;
  license_limit?: number;
  expires_at?: string;
  allowed_domains?: string[];
}

async function createDispatchPackage(packageId: string, options: DispatchOptions): Promise<DispatchPackage> {
  // 1. Verify package access
  const package = await getPackage(packageId);
  
  // 2. Generate dispatch token
  const token = generateDispatchToken({
    dispatch_id: uuid(),
    package_id: packageId,
    tenant_id: package.tenant_id,
    expires_in: options.expires_in_hours * 3600
  });
  
  // 3. Create launch URL
  const launchUrl = `${API_BASE_URL}/api/v1/dispatches/launch?token=${token}`;
  
  // 4. Generate SCORM-compatible ZIP
  const zipBuffer = await generateDispatchZip({
    dispatchId: dispatch.id,
    launchUrl: launchUrl,
    version: package.version
  });
  
  // 5. Create dispatch record
  const dispatch = await createDispatchRecord({
    package_id: packageId,
    tenant_id: package.tenant_id,
    launch_url: launchUrl,
    token: token,
    ...options
  });
  
  return dispatch;
}

Dispatch ZIP Structure:

  • imsmanifest.xml: SCORM manifest pointing to launch URL
  • index.html: Redirect page that loads launch URL
  • SCORM API Bridge embedded

Design Patterns

1. Repository Pattern

interface PackageRepository {
  findById(id: string, tenantId: string): Promise<Package | null>;
  findByTenant(tenantId: string): Promise<Package[]>;
  create(package: CreatePackageInput): Promise<Package>;
  update(id: string, updates: Partial<Package>): Promise<Package>;
  delete(id: string): Promise<void>;
}

class SupabasePackageRepository implements PackageRepository {
  // Database implementation
}

2. Strategy Pattern

interface StorageStrategy {
  upload(file: Buffer, path: string): Promise<string>;
}

class SupabaseStorageStrategy implements StorageStrategy { }
class R2StorageStrategy implements StorageStrategy { }

class StorageManager {
  private strategy: StorageStrategy;
  
  constructor(strategy: StorageStrategy) {
    this.strategy = strategy;
  }
  
  async upload(file: Buffer, path: string): Promise<string> {
    return this.strategy.upload(file, path);
  }
}

3. Factory Pattern

class StorageFactory {
  static create(): StorageStrategy {
    if (process.env.CLOUDFLARE_R2_ACCOUNT_ID) {
      return new R2StorageStrategy();
    }
    return new SupabaseStorageStrategy();
  }
}

4. Observer Pattern (Webhooks)

class WebhookNotifier {
  private subscribers: Webhook[] = [];
  
  subscribe(webhook: Webhook): void {
    this.subscribers.push(webhook);
  }
  
  async notify(event: WebhookEvent): Promise<void> {
    await Promise.all(
      this.subscribers
        .filter(w => w.eventType === event.type)
        .map(w => w.deliver(event))
    );
  }
}

Performance Optimizations

1. Database Query Optimization

  • Indexes: Strategic indexes on frequently queried columns
  • Query Batching: Batch multiple queries when possible
  • Connection Pooling: Supabase handles connection pooling
  • Read Replicas: Use read replicas for analytics queries

2. Caching Strategy

  • Response Caching: Cache read-heavy endpoints
  • CDN Caching: Static assets cached at edge
  • Redis Caching: Optional Redis for session data caching

3. Async Processing

  • Webhook Delivery: Asynchronous webhook processing
  • Package Processing: Background job for large packages
  • Event Logging: Async event logging to avoid blocking

Error Handling Strategy

Error Classification

enum ErrorCode {
  // Client Errors (4xx)
  VALIDATION_ERROR = 'VALIDATION_ERROR',
  UNAUTHORIZED = 'UNAUTHORIZED',
  FORBIDDEN = 'FORBIDDEN',
  NOT_FOUND = 'NOT_FOUND',
  CONFLICT = 'CONFLICT',
  RATE_LIMIT_EXCEEDED = 'RATE_LIMIT_EXCEEDED',
  
  // Server Errors (5xx)
  INTERNAL_ERROR = 'INTERNAL_ERROR',
  DATABASE_ERROR = 'DATABASE_ERROR',
  STORAGE_ERROR = 'STORAGE_ERROR'
}

Error Response Format

interface ErrorResponse {
  error: {
    code: string;
    message: string;
    details?: Record<string, any>;
    request_id?: string;
  };
}

Retry Logic

  • Transient Errors: Automatic retry with exponential backoff
  • Permanent Errors: Return error immediately
  • Idempotency: Safe to retry idempotent operations

Security Considerations

1. Input Validation

  • Zod Schemas: All inputs validated with Zod
  • SQL Injection Prevention: Parameterized queries only
  • XSS Prevention: Sanitize all user inputs
  • File Upload Validation: Validate file types and sizes

2. Authentication Security

  • API Key Hashing: SHA-256 hashing of API keys
  • Token Expiration: Short-lived tokens with expiration
  • Secret Rotation: Support for secret rotation
  • Rate Limiting: Prevent brute force attacks

3. Data Protection

  • Encryption at Rest: Database encryption
  • Encryption in Transit: HTTPS only
  • Tenant Isolation: RLS enforcement
  • Audit Logging: Complete audit trail

Last Updated: 2025-01-12
Version: 1.0

For architecture overview, see Architecture Overview.