System Architecture

Architecture Overview

FirmwareCI uses a distributed architecture with three main components: the FirmwareCI Server, Devices-under-Test (DUTs), and Clients. The server coordinates all testing activities, managing hardware resources and executing workflows triggered by clients.

┌─────────────────────────────────────────┐
│                Clients                  │
│  ┌──────────┐  ┌────────┐  ┌─────────┐  │
│  │ GitHub   │  │  Web   │  │   CLI   │  │
│  │ Actions  │  │   UI   │  │  (fwci) │  │
│  └─────┬────┘  └────┬───┘  └────┬────┘  │
└────────┼────────────┼───────────┼───────┘
         │            │           │
         └────────────┼───────────┘
                      │
         ┌────────────▼────────────┐
         │   FirmwareCI Server     │
         │  ┌──────────────────┐   │
         │  │  Job Scheduler   │   │
         │  └──────────────────┘   │
         │  ┌──────────────────┐   │
         │  │  DUT Manager     │   │
         │  └──────────────────┘   │
         │  ┌──────────────────┐   │
         │  │  Test Executor   │   │
         │  └──────────────────┘   │
         └────────────┬────────────┘
                      │
         ┌────────────▼────────────┐
         │         DUTs            │
         │  ┌─────┐  ┌─────┐       │
         │  │ Pi  │  │ Pi  │ ...   │
         │  │  ↓  │  │  ↓  │       │
         │  │Board│  │Board│       │
         │  └─────┘  └─────┘       │
         └─────────────────────────┘

FirmwareCI Server

The FirmwareCI Server is the central coordinator managing the entire testing lifecycle.

Core Functions

Job Scheduling: Queues and schedules test jobs based on workflow triggers.

DUT Management: Tracks available hardware, handles reservations, and ensures DUTs are properly configured. Integrates with external reservation systems when configured.

Test Execution: Runs test workflows in isolated containers. Manages pre-stage, test stages, and post-stage operations. Handles timeouts, failures, and cleanup.

Artifact Storage: Stores test artifacts, firmware binaries, and logs. Provides access to historical test results and reports.

Configuration Storage

The server stores configuration per project, organized into branches. Each branch mirrors a branch of your repository and holds its own independent configuration, so main and a feature branch can define different workflows, DUTs, or storage:

Project
└── Branch (e.g. "main")
    ├── Workflows      Define which tests run for which events
    ├── DUT definitions Hardware specifications and connection details
    ├── Test files     Test stages and steps
    └── Storage items  Firmware binaries, tools, and test data

Secrets (API tokens and sensitive configuration) are stored at the project level and shared across its branches.

See the Branches reference for branch types, default-branch resolution, and how a trigger maps to a branch.

Devices Under Test (DUTs)

DUTs are hardware devices connected through control interfaces that enable remote management and testing.

Interface Capabilities

Control interfaces (typically single-board computers like Raspberry Pi) provide:

  • Power Control: Turn devices on/off, perform hard resets
  • Serial Access: Monitor boot logs, access console
  • Flash Operations: Read and write firmware to flash chips
  • Binary Execution: Deploy and run test binaries
  • Network Access: SSH connectivity for command execution

DUT Configuration

Each DUT has a configuration defining:

Attributes: Key-value pairs for device-specific data (hostnames, addresses, credentials).

Label: Groups DUTs of the same hardware type. Workflows target labels, not individual DUTs.

Pre-stage: Optional setup steps run before tests (flash firmware, configure BIOS).

Post-stage: Optional cleanup steps run after tests (power off, reset settings).

Reservation System: Optional integration with external hardware reservation systems.

Hardware Control

FirmwareCI delegates low-level device management to control interfaces. While design is flexible, FirmwareCI is intended to integrate with DUT‑Control (dutctl), an open‑source CLI for remote hardware interaction: DUT‑Control (dutctl). The dutctl command provides the standardized operations FirmwareCI relies on — power control, flashing, serial console access, resets.

Clients

Clients interact with the FirmwareCI Server to trigger workflows, monitor progress, and retrieve results.

Client Types

  • CI System Integrations: GitHub Actions, GitLab CI, and other CI platforms trigger workflows automatically on commits, pull requests, and releases.
  • Web Interface: Browser UI for monitoring runs, viewing reports, and managing configurations.
  • CLI Tool (fwci): Command-line tool for setup and configuration.

Workflows

Workflows connect all components, defining how tests execute when triggered.

Workflow Components

Hardware Requirements: Defines which DUT labels the workflow targets. Multiple hardware types can be specified.

Test Selection: Lists test files to execute. Tests run sequentially in definition order.

Input Requirements: Declares required inputs like firmware binaries that must be provided when triggering.

Secrets: Stores sensitive data (API tokens, credentials) accessible through template variables.

Execution Flow

  1. Trigger: Client sends workflow trigger with required inputs (firmware binary, parameters)
  2. Queue: Server queues the job and reserves appropriate DUTs based on labels
  3. Preparation: DUTs execute pre-stage operations (flash firmware, configure hardware)
  4. Testing: Each test file runs sequentially, executing all stages and steps
  5. Cleanup: DUTs execute post-stage operations and return to the available pool
  6. Reporting: Server generates reports and notifies the triggering client

Workflow Overview Workflow Overview

Next Steps