Writing a Simple Boot Test

Create a boot test that power cycles your board, flashes firmware, and verifies the device boots.

Prerequisites

  • fwci CLI installed and authenticated
  • Git repository with .firmwareci infrastructure initialized (see Repository Setup)
  • A DUT with a pre-stage and a post-stage, set up as described in Connecting Hardware with dutctl
  • Access to target device over network

Quick Example

The boot test runs on the DUT from Connecting Hardware with dutctl, which has label: board, a Host attribute, and a pre-stage that powers the board off, flashes the firmware, and powers it on again.

Workflow (.firmwareci/workflows/board/workflow.yaml):

name: Board-Workflow
description: Boot tests for the board
runs-on: board

Boot Test (.firmwareci/workflows/board/tests/boot.yaml):

name: Boot Test
description: Verify board boots and accepts SSH connections

stages:
  - name: Boot Verification
    steps:
      - cmd: ping
        name: Wait for SSH service
        options:
          timeout: 2m
        parameters:
          host: "[[attributes.Host]]"

      - cmd: cmd
        name: Verify device responds
        transport:
          proto: ssh
          options:
            host: "[[attributes.Host]]"
            user: root
            # Uses auto-discovery for SSH keys
        parameters:
          executable: echo
          args: ["Hello, World!"]

Step-by-Step Setup

1. Set Up the DUT

Set up the DUT with its dut.yaml, pre.yaml, and post.yaml as described in Connecting Hardware with dutctl. The pre-stage runs before the test and prepares the board; the post-stage runs after it, even if the test fails.

The test uses the DUT’s Host attribute, the address of the board itself. See DUT Configuration Reference for the complete schema.

2. Update Workflow

Edit .firmwareci/workflows/<workflow-name>/workflow.yaml:

name: Board-Workflow
description: Boot tests for the board
runs-on: board

The runs-on field must match the DUT’s label value.

See Workflow Reference for complete configuration options.

3. Create Boot Test

Create .firmwareci/workflows/<workflow-name>/tests/boot.yaml:

name: Boot Test
description: Verify board boots and accepts SSH connections

stages:
  - name: Boot Verification
    steps:
      - cmd: ping
        name: Wait for SSH service
        options:
          timeout: 2m
        parameters:
          host: "[[attributes.Host]]"

      - cmd: cmd
        name: Verify device responds
        transport:
          proto: ssh
          options:
            host: "[[attributes.Host]]"
            user: root
        parameters:
          executable: echo
          args: ["Hello, World!"]

Test execution flow:

  1. Pre-stage: Power off → Flash → Power on
  2. Ping command waits up to 2 minutes for SSH service
  3. SSH command executes on target device
  4. Post-stage: Power off

4. SSH Authentication

Option 1: Auto-Discovery (Recommended)

Omit identity_file and FirmwareCI automatically tries all organization SSH keys:

transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: root
    # No identity_file specified

Option 2: Specific Key

Reference a specific SSH key by name:

transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: root
    identity_file: "[[ssh-keys.board_key]]"

Add SSH keys in the web interface under Settings → SSH Keys. See SSH Key Management.

Option 3: Password Authentication

Use for testing only - not recommended for production:

transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: root
    password: root

5. Validate and Deploy

Validate configuration:

fwci validate

Expected output:

- duts
  - all validations passed
- storage
  - all validations passed
- workflows
  - board-workflow

To try the test on the board before you push it, run it on your machine (see Running Tests Locally). A local run doesn’t have your organization’s SSH keys, so pass the key the board accepts with --ssh-key:

fwci run -w Board-Workflow --ssh-key board_key=$HOME/.ssh/board_key Binary=./coreboot.rom

Commit and push:

git add .firmwareci
git commit -m "Add board boot test"
git push

FirmwareCI automatically syncs changes and the workflow becomes available for triggering.

Test Images

Use pre-configured test container images with required tooling:

FirmwareCI Base Images: firmwareci-base-image repository

These images include all test step dependencies and can be customized for your DUT requirements. To run a workflow’s jobs in your own image, name it in the workflow: see Job Images.

Handling Step Failures

By default, a test stops at the first failed step: the rest of its stage and all following stages are skipped, and the Post-stage still runs. Two options change this:

  • on-failure: continue runs the remaining steps of the same stage. Later stages are still skipped.
  • A last stage with run-condition: failure runs only when the test failed, for example to collect logs.

To keep a stage running after a failed step:

stages:
  - name: Boot Verification
    steps:
      - cmd: ping
        name: Wait for SSH service
        options:
          timeout: 2m
          on-failure: continue  # Continue to next step even if this fails
        parameters:
          host: "[[attributes.Host]]"

See Changing the Default for where to set on-failure, and for the failure stage.

Common Issues

ProblemSolution
Ping timeoutIncrease timeout in ping step options. Verify network connectivity to DUT. Check attributes.Host address.
SSH authentication failsVerify SSH key exists in organization settings. Check username is correct. Confirm SSH service runs on DUT.
Flash command failsVerify the dutagent is reachable. Check the attributes.Agent address and port and the attributes.Device name. Ensure firmware binary path is correct. See When the Step Fails.
Job stays queuedCheck the device list for the board’s status. A manual dutctl lock or a local run may hold the board: release your manual lock, or wait for the local run. See How a Job Gets a DUT.
DUT doesn’t boot after flashCheck power cycle commands execute successfully. Verify firmware binary is valid. Increase power-on timeout.
Template not resolvedVerify template name matches exactly (case-sensitive). Check attribute exists in dut.yaml. Ensure input provided when triggering.
Validation failsCheck YAML syntax (use YAML linter). Verify all referenced templates exist. Ensure runs-on matches DUT label.
Step fails but test should continueAdd on-failure: continue at step, stage, or test level. It only continues within the same stage: later stages are still skipped after a failure, so keep steps that must run together in one stage.

See Also