API Overview

Architecture

Segra uses a dual-process architecture to separate concerns and maximize performance:

Backend Process (C#/.NET): Built on OBS, handles recording, video processing, file operations, and system integration
Frontend Process (React/TypeScript): Provides the user interface through a WebView

Communication Layer

The frontend and backend communicate through several channels:

WebSocket

Bidirectional real-time channel on ws://localhost:44030/ for state updates, progress notifications, and system events. Frontend-to-backend messages can also be sent over the same socket.

window.external

Photino IPC bridge used by the frontend to deliver commands to the backend (window.external.sendMessage).

Content HTTP server

Local HTTP endpoint on http://localhost:2222/ exposing /api/content (video/JSON streaming with byte-range support) and /api/thumbnail (frame extraction via FFmpeg).

Game integration listeners

Per-game local listeners such as the Counter-Strike 2 GSI endpoint on http://127.0.0.1:1340/.

Frontend to Backend

The frontend sends commands to the backend using the window.external.sendMessage() interface, wrapped by the sendMessageToBackend utility:

Frontend/src/Utils/MessageUtils.ts

export const sendMessageToBackend = (method: string, parameters?: any) => {
  const message = { Method: method, Parameters: parameters };
  if ((window as any).external && typeof (window as any).external.sendMessage === 'function') {
    const messageString = JSON.stringify(message);
    (window as any).external.sendMessage(messageString);
  } else {
    console.error('window.external.sendMessage is not available.');
  }
};

Message Format

All messages sent to the backend follow this structure:

Method

string

required

The command/action to execute (e.g., “StartRecording”, “CreateClip”)

Parameters

object

Optional parameters specific to the method being called

Example: Starting a Recording

import { sendMessageToBackend } from '../Utils/MessageUtils';

// Start recording manually
sendMessageToBackend('StartRecording');

// Login with authentication tokens
sendMessageToBackend('Login', {
  accessToken: session.access_token,
  refreshToken: session.refresh_token,
});

Backend to Frontend

The backend sends real-time updates to the frontend via WebSocket on ws://localhost:44030/.

WebSocket Connection

The frontend establishes a WebSocket connection using the WebSocketProvider context:

Frontend/src/Context/WebSocketContext.tsx

const { readyState } = useWebSocket('ws://localhost:44030/', {
  onOpen: () => {
    sendMessageToBackend('NewConnection');
  },
  onMessage: (event) => {
    const data: WebSocketMessage = JSON.parse(event.data);
    // Dispatch to listeners
    window.dispatchEvent(
      new CustomEvent('websocket-message', {
        detail: data,
      }),
    );
  },
  shouldReconnect: () => true,
  reconnectAttempts: Infinity,
  reconnectInterval: 3000,
  heartbeat: {
    message: 'ping',
    returnMessage: 'pong',
    timeout: 30000,
    interval: 15000,
  },
});

Message Format

All WebSocket messages from the backend follow this structure:

method

string

required

The event type (e.g., “Settings”, “ClipProgress”, “ShowModal”)

content

object

required

The payload data specific to the method

Heartbeat Protocol

The WebSocket connection maintains a heartbeat to detect disconnections:

Frontend → Backend: Sends "ping" every 15 seconds
Backend → Frontend: Responds with {"method": "pong", "content": {}}
Timeout: Connection considered dead after 30 seconds without a pong

Connection Lifecycle

Frontend Starts

React app initializes and the WebSocketProvider mounts

WebSocket Connects

Connection established to ws://localhost:44030/

NewConnection Message

Frontend sends NewConnection command to backend

Initial Sync

Backend responds with:

Persisted settings (Settings message)
Runtime application state (State message)
Available games list (GameList message)
App version (AppVersion message)

Session Sync

If user is authenticated, frontend sends Login message with tokens

Heartbeat Begins

Frontend starts sending ping every 15 seconds

The backend automatically closes any existing WebSocket connection when a new one is established, ensuring only one active connection at a time.

State Management

The backend splits application state across two singletons:

Settings.Instance (Backend/Core/Models/Settings.cs) — persisted user preferences (video quality, storage locations, keybindings, whitelists/blacklists, etc.). Synced to the frontend via the Settings WebSocket message.
AppState.Instance (Backend/Core/Models/AppState.cs) — runtime, non-persisted state (active recording, pre-recording, content library, detected devices/displays/codecs, available OBS versions, current folder size, GPU vendor). Synced to the frontend via the State WebSocket message.

Whenever settings change, the backend sends a Settings message to sync the frontend:

Backend/App/MessageService.cs

public static async Task SendSettingsToFrontend(string cause)
{
    if (!Program.hasLoadedInitialSettings || Settings.Instance._isBulkUpdating)
        return;

    Log.Information("Sending settings to frontend ({Cause})", cause);
    await SendFrontendMessage("Settings", Settings.Instance);
}

public static async Task SendStateToFrontend(string cause)
{
    if (!Program.hasLoadedInitialSettings || Settings.Instance._isBulkUpdating)
        return;

    Log.Information("Sending state to frontend ({Cause})", cause);
    await SendFrontendMessage("State", AppState.Instance);
}

Bulk operations set _isBulkUpdating = true to prevent spamming the frontend with updates. A single update is sent after the operation completes.

Error Handling

Both communication channels implement error handling:

Frontend Errors

if ((window as any).external && typeof (window as any).external.sendMessage === 'function') {
  // Send message
} else {
  console.error('window.external.sendMessage is not available.');
}

Backend Errors

The backend catches and logs exceptions in the message handler:

try {
    var jsonDoc = JsonDocument.Parse(message);
    // Handle message
}
catch (JsonException ex) {
    Log.Error($"Failed to parse message as JSON: {ex.Message}");
}
catch (Exception ex) {
    Log.Error($"Unhandled exception in message handler: {ex.Message}");
}

Critical errors are displayed to users via the ShowModal message:

await MessageService.ShowModal(
    "Error", 
    $"Failed to create clip: {ex.Message}", 
    "error"
);

API Overview

Architecture

Communication Layer

WebSocket

window.external

Content HTTP server

Game integration listeners

Frontend to Backend

Message Format

Example: Starting a Recording

Backend to Frontend

WebSocket Connection

Message Format

Heartbeat Protocol

Connection Lifecycle

State Management

Error Handling

Frontend Errors

Backend Errors

Next Steps

WebSocket API

Recording Service

​Architecture

​Communication Layer

WebSocket

window.external

Content HTTP server

Game integration listeners

​Frontend to Backend

​Message Format

​Example: Starting a Recording

​Backend to Frontend

​WebSocket Connection

​Message Format

​Heartbeat Protocol

​Connection Lifecycle

​State Management

​Error Handling

​Frontend Errors

​Backend Errors

​Next Steps

WebSocket API

Recording Service

Architecture

Communication Layer

Frontend to Backend

Message Format

Example: Starting a Recording

Backend to Frontend

WebSocket Connection

Message Format

Heartbeat Protocol

Connection Lifecycle

State Management

Error Handling

Frontend Errors

Backend Errors

Next Steps