Running a Job

Prerequisites

Ensure FirmwareCI is set up by following the steps in the Initial Setup guide.

What is a Job?

A job in FirmwareCI is the execution of all tests specified within a workflow on a Device Under Test (DUT) that matches the designated label. Upon completion, job results are reported to your FirmwareCI dashboard. If you have enabled integrations—such as the GitHub App—results are also communicated directly to the relevant commit or pull request in your version control system.

Each job requires a workflow and a set of binaries, provided as a map of template keys to file paths. These keys correspond to variables referenced in your test definitions, allowing you to specify which files are used for each part of the workflow. For example, you might provide BINARY=~/Downloads/coreboot.rom or OS_IMAGE=~/Downloads/ubuntu.img. This flexible approach supports various use cases, such as flashing firmware, booting devices with specific OS images, or supplying additional required files. Users can also choose to run all tests in a workflow or select only a subset. Additional job metadata, such as file names and commit hashes, may be included as needed.

Job Triggers

We offer three different methods to trigger a job:

Command-Line Interface: Using the fwci job command.

Interactive Mode: This mode prompts you to select the workflow and specify whether to run all tests or only a subset. If only one workflow is available, it will be automatically selected.

# Login using your credentials
$ fwci login <email> <password>

# Start a job
$ fwci job Binary=~/Downloads/my-rom.rom

Selected workflow: fwci-crystal-core
Do you want to select partial tests to be run? (y/n): n
Selected all tests
Uploading binary: /home/l1x0r/Downloads/coreboot.rom
This may take a while...

Non-Interactive Mode: The CLI can also operate non-interactively by providing the workflow name as the second argument:
```
$ fwci job Binary=~/Downloads/coreboot.rom -w fwci-crystal-core

Selected workflow: fwci-crystal-core
Selected all tests
Uploading binary: /home/l1x0r/Downloads/coreboot.rom
This may take a while...
```
Caution: You might be required to provide the project-name as well, if the workflow-name is not unique across projects.

Integrations: Integration with version control systems (GitHub or GitLab).

Create an user/org token to authenticate.

GitHub Action:

- name: Upload to FirmwareCI
  uses: docker://firmwareci/action:v5.1
  with:
    TOKEN: "${{ secrets.TOKEN}}"
    # EMAIL: ${{ secrets.FWCI_EMAIL }}
    # PASSWORD: ${{ secrets.FWCI_PASSWORD }}
    WORKFLOW_ID: "${{ secrets.FWCI_WORKFLOW_ID}}"
    COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }}
    BINARIES: BINARY=./coreboot.rom
    # GITHUB_INSTALLATION_ID: 12345678 (Only required if differs from the integration source)

GitLab CI/CD:

# Push binary artifacts to FirmwareCI
push_firmwareci:
  stage: push_firmwareci
  needs:
    - job: build_coreboot

  image: firmwareci/action:v5.1
  variables:
    TOKEN: $FWCI_TOKEN
    # EMAIL: $FWCI_EMAIL
    # PASSWORD: $FWCI_PASSWORD
    WORKFLOW_ID: $FWCI_WORKFLOW_ID
  # GITLAB_TOKEN: glpa_.... (Only required if it differs from the integration source)
    BINARIES: BINARY=/builds/$CI_PROJECT_PATH/build/coreboot.rom;Data=/builds/$CI_PROJECT_PATH/build/data
  script:
    - /bin/action

Gerrit CI/CD: Please refer to Gerrit Integration Guide.

REST API: Making HTTP requests to trigger jobs. For further information, refer to FirmwareCI API Documentation.

Backreporting

Backreporting ensures that test results from FirmwareCI are automatically communicated back to your version control system. We support backreporting for both GitHub, GitLab & Gerrit. Jobs can be initiated from any repository, not limited to the integration source, allowing you to trigger tests across platforms (e.g., GitHub and GitLab).

GitHub Backreporting

For GitHub, FirmwareCI provides an official GitHub App that can be installed in your repository. This app automatically reports:

Pull Requests: Adds test result comments directly on the PR.
Default Branch Commits: Updates the commit status with test results.

Installation & Setup

Install the FirmwareCI GitHub App.
Grant it access to the repositories where you want backreporting enabled.
Once a job is triggered through the CI, it will report back into this repository. It’s mandatory for this that the test definitions are also within the same repository.

Example GitHub PR Report Screenshot:

GitLab Backreporting

For GitLab, users need to manually provide an access token for FirmwareCI to authenticate and post test results. The results are reported as:

Commit Comments: Test status is added as a comment and status to the commit.
Merge Requests: Adds results as a comment and status on the MR.

Setup Instructions

Create a Gitlab access token (select one method):
Option 1: User Access Token
- Navigate to your Gitlab account settings
- Select “Access Tokens” in the left navigation panel
- Generate a new token with the required scopes: api, read_repository
Option 2: Project Access Token
- Access your Gitlab project repository
- Navigate to Settings > Access Tokens
- Create a project access token with minimum developer role permissions and both api and read_repository scopes
- Integrate the repository using the following command:
```
fwci integration --gitlab-auth="YOUR_ACCESS_TOKEN"
```
Visit the webpage of your project.
Scroll down to the Webhook Settings.
Add the webhook as shown and save.

Note: The secret token for the webhook will be provided upon request. Please contact the FirmwareCI team to obtain your specific webhook token.

Default-Webhook-URL:

   https://api.firmwareci.9esec.dev:8443

Example GitLab MR Report Screenshot:

Gerrit Backreporting

Please refer to Gerrit Integration Guide.

Linking Back to Job Triggers

For details on how to trigger a job, refer to the Job Triggers section.