Running a Job

Prerequisites

Ensure FirmwareCI is set up by following the steps in the Installation, Repository Setup, and Configuration guides.

What is a Job?

A job is part of a job-request executing tests in a workflow on a Device Under Test (DUT) with a matching label. Results are reported to your FirmwareCI dashboard. When integrations like the GitHub App are enabled, results also appear on the relevant commit or pull request.

Each job requires two inputs:

Workflow: Defines which tests to run
Binaries: Maps template keys to file paths (e.g., BINARY=~/Downloads/coreboot.rom or OS_IMAGE=~/Downloads/ubuntu.img)

The template keys correspond to variables in your test files, allowing you to specify which binaries are used during test execution. You can run all tests in a workflow or select a subset. Optional metadata like commit hashes can be included.

Job Triggers

We offer three different methods to trigger a job:

Command-Line Interface: Using the fwci job command.

Interactive Mode: This mode prompts you to select the workflow and specify whether to run all tests or only a subset. If only one workflow is available, it will be automatically selected.

# Login using your credentials
$ fwci login <email> <password>

# Start a job
$ fwci job Binary=~/Downloads/my-rom.rom

Selected workflow: fwci-crystal-core
Do you want to select partial tests to be run? (y/n): n
Selected all tests
Uploading binary: /home/l1x0r/Downloads/coreboot.rom
This may take a while...

Non-Interactive Mode: The CLI can also operate non-interactively by providing the workflow name as the second argument:
```
$ fwci job Binary=~/Downloads/coreboot.rom -w fwci-crystal-core

Selected workflow: fwci-crystal-core
Selected all tests
Uploading binary: /home/l1x0r/Downloads/coreboot.rom
This may take a while...
```
Caution: You might be required to provide the project-name as well, if the workflow-name is not unique across projects.

Integrations: Integration with version control systems (GitHub or GitLab).

Getting Your User/Org Token:

The TOKEN is required for all CI/CD integrations. To obtain it:

Log in to the FirmwareCI web UI
Navigate to your profile
Click the tokens tab inside the profile page
Create a new user token or ask you administrator if an org token is required.
Use this Token in your CI/CD configuration (see examples below)

Getting Your Workflow ID:

The WORKFLOW_ID is required for all CI/CD integrations. To obtain it:

Log in to the FirmwareCI web UI
Navigate to your workflow
Click the Copy ID button inside the workflow
Use this ID in your CI/CD configuration (see examples below)

GitHub Action:

- name: Upload to FirmwareCI
  uses: docker://firmwareci/action:v5.1
  with:
    TOKEN: "${{ secrets.TOKEN}}"
    # EMAIL: ${{ secrets.FWCI_EMAIL }}
    # PASSWORD: ${{ secrets.FWCI_PASSWORD }}
    WORKFLOW_ID: "${{ secrets.FWCI_WORKFLOW_ID}}"
    COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }}
    BINARIES: BINARY=./coreboot.rom
    # GITHUB_INSTALLATION_ID: 12345678 (Only required if differs from the integration source)

GitLab CI/CD:

# Push binary artifacts to FirmwareCI
push_firmwareci:
  stage: push_firmwareci
  needs:
    - job: build_coreboot

  image: firmwareci/action:v5.1
  variables:
    TOKEN: $FWCI_TOKEN
    # EMAIL: $FWCI_EMAIL
    # PASSWORD: $FWCI_PASSWORD
    WORKFLOW_ID: $FWCI_WORKFLOW_ID
  # GITLAB_TOKEN: glpa_.... (Only required if it differs from the integration source)
    BINARIES: BINARY=/builds/$CI_PROJECT_PATH/build/coreboot.rom;Data=/builds/$CI_PROJECT_PATH/build/data
  script:
    - /bin/action

Gerrit CI/CD: Please refer to Gerrit Integration Guide.

REST API: Making HTTP requests to trigger jobs. For further information, refer to FirmwareCI API Documentation.

Backreporting

Backreporting ensures that test results from FirmwareCI are automatically communicated back to your version control system. We support backreporting for both GitHub, GitLab & Gerrit. Jobs can be initiated from any repository, not limited to the integration source, allowing you to trigger tests across platforms (e.g., GitHub and GitLab).

GitHub Backreporting

For GitHub, FirmwareCI provides an official GitHub App that can be installed in your repository. This app automatically reports:

Pull Requests: Adds test result comments directly on the PR.
Default Branch Commits: Updates the commit status with test results.

Installation & Setup

Install the FirmwareCI GitHub App.
Grant it access to the repositories where you want backreporting enabled.
Once a job is triggered through the CI, it will report back into this repository.

GitLab Backreporting

For GitLab, users need to manually provide an access token for FirmwareCI to authenticate and post test results. The results are reported as:

Commit Comments: Test status is added as a comment and status to the commit.
Merge Requests: Adds results as a comment and status on the MR.

Setup Instructions

Create a Gitlab access token (select one method):
Option 1: User Access Token
- Navigate to your Gitlab account settings
- Select “Access Tokens” in the left navigation panel
- Generate a new token with the required scopes: api, read_repository
Option 2: Project Access Token
- Access your Gitlab project repository
- Navigate to Settings > Access Tokens
- Create a project access token with minimum developer role permissions and both api and read_repository scopes
- Integrate the repository using the following command:
```
fwci integration --gitlab-auth="YOUR_ACCESS_TOKEN"
```
Visit the webpage of your project.
Scroll down to the Webhook Settings.
Add the webhook as shown and save.

Note: The secret token for the webhook will be provided upon request. Please contact the FirmwareCI team to obtain your specific webhook token.

Default-Webhook-URL:

   https://api.firmwareci.9esec.dev:8443

Gerrit Backreporting

Please refer to Gerrit Integration Guide.

Linking Back to Job Triggers

For details on how to trigger a job, refer to the Job Triggers section.