Running a Job

Prerequisites

• Ensure FirmwareCI is set up by following the steps in the Installation, Repository Setup, and Configuration guides.

What is a Job?

A job is part of a job-request executing tests in a workflow on a Device Under Test (DUT) with a matching label. Results are reported to your FirmwareCI dashboard. When integrations like the GitHub App are enabled, results also appear on the relevant commit or pull request.

Each job requires two inputs:

1. Workflow: Defines which tests to run
2. Binaries: Maps template keys to file paths (e.g., BINARY=~/Downloads/coreboot.rom or OS_IMAGE=~/Downloads/ubuntu.img)

The template keys correspond to variables in your test files, allowing you to specify which binaries are used during test execution. You can run all tests in a workflow or select a subset.

Job Triggers

We offer three different methods to trigger a job:

1. Command-Line Interface: Using the fwci job or fwci exec commands.

   By default fwci job auto-follows the submitted request: it uploads the binary, creates the job request, and then streams live output until all jobs reach a terminal state. Pass --no-follow to submit and exit immediately.

   • Interactive Mode: This mode prompts you to select the workflow and specify whether to run all tests or only a subset. If only one workflow is available, it will be automatically selected.

     # Login using your credentials
     $ fwci login <email> <password>
     
     # Start a job
     $ fwci job Binary=~/Downloads/my-rom.rom
     
     Selected workflow: fwci-crystal-core
     Do you want to select partial tests to be run? (y/n): n
     Selected all tests
     
     Uploading binaries. This may take a while...
       Binary (1/1)          [████████████████████] 100%  64.00 MB  0:11
     
     Successfully started
     Request ID: 01KPDF6MT63NM59NEPXVG66X62
     Waiting for jobs to start…
     
     Workflow fwci-crystal-core
     JOB ID                       TEST                   STATUS    DURATION
     01KPDF6MTCRATNF7JH4ARQT7GH   Web: Interface Access  running   44s
     01KPDF6MTNKJ6GBVY6Y6STT69Y   User: Session Mgmt     queued
     …
     polling: 0/12 done  •  refreshing every 3s

   • Non-Interactive Mode: The CLI can also operate non-interactively by providing the workflow name as the second argument:

     $ fwci job Binary=~/Downloads/coreboot.rom -w fwci-crystal-core --no-partial-tests

     Caution: You might be required to provide the project-name as well, if the workflow-name is not unique across projects.

   • CI / Scripted Mode: For pipelines, combine --no-follow with --no-partial-tests so the CLI submits and exits without any interaction:

     $ fwci job Binary=./coreboot.rom -w fwci-crystal-core --no-partial-tests --no-follow
     Submitted job request: 01KPDF6MT63NM59NEPXVG66X62

     Or capture a machine-readable completion report with --json:

     $ fwci job Binary=./coreboot.rom -w fwci-crystal-core --no-partial-tests --json > report.json

Viewing and Inspecting Jobs

Once a request has been submitted you can always come back to it:

While a job is queued, fwci view shows the full request table live so you can see queue position; once the job actually starts it switches to log streaming automatically.

Ad-hoc Tests (fwci exec)

fwci exec submits one or more test YAML files on-the-fly, without committing them to the workflow. Useful for iterating on a new test locally.

You can pass individual test files, multiple files, or entire directories. When a directory is provided, it is scanned recursively for .yml/.yaml files that match the test schema — non-test files (e.g. dut.yaml, workflow.yaml) are automatically skipped.

# Run a single test file
$ fwci exec ./tests/boot.yaml -w fwci-crystal-core -b Binary=./coreboot.rom

# Run multiple test files
$ fwci exec test1.yaml test2.yaml -w fwci-crystal-core -b Binary=./coreboot.rom

# Run every test in a directory (recursive)
$ fwci exec ./tests/ -w fwci-crystal-core -b Binary=./coreboot.rom

# Multiple binaries
$ fwci exec ./tests/ -w fwci-crystal-core -b Binary=./coreboot.rom -b Data=./data.bin

# Submit and exit (for CI)
$ fwci exec ./tests/boot.yaml -w fwci-crystal-core --no-follow

Follows by default (same --no-follow / --json / --long flags as fwci job).

Tests submitted via fwci exec appear on the FirmwareCI dashboard as OneShot triggers and are not associated with any VCS commit.

2. Integrations: Integration with version control systems (GitHub or GitLab).

   Getting Your User/Org Token:

   The TOKEN is required for all CI/CD integrations. To obtain it:

   1. Log in to the FirmwareCI web UI
   2. Navigate to your profile
   3. Click the tokens tab inside the profile page
   4. Create a new user token or ask you administrator if an org token is required.
   5. Use this Token in your CI/CD configuration (see examples below)

   Referencing Your Workflow:

   Every CI/CD integration requires you to identify which workflow to run. There are two ways to do this — use whichever fits your setup best:

   Variable	Description
   WORKFLOW_NAME	Recommended. The human-readable name of the workflow (e.g. coreboot-tests). Easy to read in CI configs. The name must match exactly and be unique within the project.
   WORKFLOW_ID	(Deprecated — still supported.) The internal ULID of the workflow. Does not change when the workflow is renamed, but makes CI configs harder to read and requires copying the ID from the UI.

   WORKFLOW_ID and WORKFLOW_NAME are mutually exclusive — set exactly one.

   Getting Your Workflow Name:

   1. Log in to the FirmwareCI web UI
   2. Navigate to your workflow
   3. The name shown in the header is the value to use for WORKFLOW_NAME

   • GitHub Action:

     - name: Upload to FirmwareCI
       uses: docker://firmwareci/action:v5.2
       with:
         TOKEN: "${{ secrets.FWCI_TOKEN }}"
         # EMAIL: ${{ secrets.FWCI_EMAIL }}
         # PASSWORD: ${{ secrets.FWCI_PASSWORD }}
     
         WORKFLOW_NAME: "coreboot-tests"
         # WORKFLOW_ID: "${{ secrets.FWCI_WORKFLOW_ID }}"  # deprecated, still supported
     
         BINARIES: BINARY=./coreboot.rom
         # GITHUB_INSTALLATION_ID: 12345678 (only required if it differs from the integration source)

   • GitLab CI/CD:

     # Push binary artifacts to FirmwareCI
     push_firmwareci:
       stage: push_firmwareci
       needs:
         - job: build_coreboot
     
       image: firmwareci/action:v5.2
       variables:
         TOKEN: $FWCI_TOKEN
         # EMAIL: $FWCI_EMAIL
         # PASSWORD: $FWCI_PASSWORD
     
         WORKFLOW_NAME: "coreboot-tests"
         # WORKFLOW_ID: $FWCI_WORKFLOW_ID  # deprecated, still supported
     
         # GITLAB_TOKEN: $FWCI_GITLAB_TOKEN  # required for back-reporting; can be omitted if already stored via `fwci integration --gitlab-auth`
         BINARIES: BINARY=/builds/$CI_PROJECT_PATH/build/coreboot.rom;Data=/builds/$CI_PROJECT_PATH/build/data
       script:
         - /bin/action

   • Gerrit CI/CD: Please refer to Gerrit Integration Guide.

Cross-Repository Workflow Configuration

By default, the action resolves the workflow by looking up the project that matches the repository where the CI pipeline runs. This works well when the FirmwareCI configuration (.firmwareci/) lives in the same repository as the triggering pipeline.

However, a common pattern is to keep FirmwareCI configuration in a dedicated infrastructure repository (e.g. my-org/firmware-tests) while triggering jobs from a product repository (e.g. my-org/my-product). In this case the action cannot find the workflow automatically, because the trigger repository does not have a matching FirmwareCI project.

Use PROJECT_LINK to tell the action which project to look up the workflow in.

Getting Your PROJECT_LINK:

1. Log in to the FirmwareCI web UI
2. Navigate to the Workflows page
3. Find the project that contains your workflow — the value to use (in instance/org/repo format) is shown next to the project name and can be copied with the button beside it

PROJECT_LINK only affects workflow resolution. Back-reporting (commit statuses, PR checks) always targets the repository that triggered the pipeline — this is unchanged.

Token Requirements

When using PROJECT_LINK, verify that all tokens have the required scopes for both the trigger repository and the project referenced by PROJECT_LINK:

GitHub:

• The FirmwareCI GitHub App must be installed in the organization that owns the workflow project (i.e. my-org in my-org/firmware-tests), not just the trigger repository.
• The GITHUB_INSTALLATION_ID must correspond to the installation in the workflow project’s organization. Set it explicitly if the trigger repo and the workflow project belong to different organizations.

GitLab:

• The GITLAB_TOKEN (or the token stored in the FirmwareCI integration) must have at least read_repository scope on the repository specified in PROJECT_LINK.
• For self-hosted instances (https://gitlab.example.com/...), ensure the token is valid for that instance and not scoped to a different GitLab URL.

Example — GitHub Action, config in a different GitHub repo:

- name: Upload to FirmwareCI
  uses: docker://firmwareci/action:v5.2
  with:
    TOKEN: "${{ secrets.FWCI_TOKEN }}"
    WORKFLOW_NAME: "coreboot-tests"
    PROJECT_LINK: "github.com/my-org/firmware-tests"
    BINARIES: BINARY=./coreboot.rom

Example — GitLab CI/CD, config on a self-hosted GitLab instance:

push_firmwareci:
  stage: push_firmwareci
  image: firmwareci/action:v5.2
  variables:
    TOKEN: $FWCI_TOKEN
    WORKFLOW_NAME: "coreboot-tests"
    PROJECT_LINK: "gitlab.example.com/my-group/firmware-tests"
    BINARIES: BINARY=/builds/$CI_PROJECT_PATH/build/coreboot.rom
  script:
    - /bin/action

Example — GitLab CI/CD, referencing a GitHub project:

push_firmwareci:
  stage: push_firmwareci
  image: firmwareci/action:v5.2
  variables:
    TOKEN: $FWCI_TOKEN
    WORKFLOW_NAME: "coreboot-tests"
    PROJECT_LINK: "github.com/my-org/firmware-tests"
    BINARIES: BINARY=/builds/$CI_PROJECT_PATH/build/coreboot.rom
  script:
    - /bin/action

3. REST API: Making HTTP requests to trigger jobs. For further information, refer to FirmwareCI API Documentation.

Backreporting

Backreporting ensures that test results from FirmwareCI are automatically communicated back to your version control system. We support backreporting for both GitHub, GitLab & Gerrit. Jobs can be initiated from any repository, not limited to the integration source, allowing you to trigger tests across platforms (e.g., GitHub and GitLab).

GitHub Backreporting

For GitHub, FirmwareCI provides an official GitHub App that can be installed in your repository. This app automatically reports:

• Pull Requests: Adds test result comments directly on the PR.
• Default Branch Commits: Updates the commit status with test results.

Installation & Setup

1. Install the FirmwareCI GitHub App.
2. Grant it access to the repositories where you want backreporting enabled.
3. Once a job is triggered through the CI, it will report back into this repository.

GitLab Backreporting

For GitLab, users need to manually provide an access token for FirmwareCI to authenticate and post test results. The results are reported as:

• Commit Comments: Test status is added as a comment and status to the commit.
• Merge Requests: Adds results as a comment and status on the MR.

Setup Instructions

1. Create a Gitlab access token (select one method):

   Option 1: User Access Token

   • Navigate to your Gitlab account settings
   • Select “Access Tokens” in the left navigation panel

   

   • Generate a new token with the required scopes: api, read_repository

   Option 2: Project Access Token

   • Access your Gitlab project repository
   • Navigate to Settings > Access Tokens

   

   • Create a project access token with minimum developer role permissions and both api and read_repository scopes

   

   • Integrate the repository using the following command:

   fwci integration --gitlab-auth="YOUR_ACCESS_TOKEN"

2. Visit the webpage of your project.

3. Scroll down to the Webhook Settings.

   

4. Add the webhook as shown and save.

Note: The secret token for the webhook will be provided upon request. Please contact the FirmwareCI team to obtain your specific webhook token.

Default-Webhook-URL:

   https://api.firmwareci.9esec.dev:8443

Gerrit Backreporting

Please refer to Gerrit Integration Guide.

Linking Back to Job Triggers

For details on how to trigger a job, refer to the Job Triggers section.