Running a Job

Prerequisites

• Ensure FirmwareCI is set up by following the steps in the Installation, Repository Setup, and Configuration guides.

What is a Job?

A job is part of a job-request executing tests in a workflow on a Device Under Test (DUT) with a matching label. Results are reported to your FirmwareCI dashboard. When integrations like the GitHub App are enabled, results also appear on the relevant commit or pull request.

Each job requires two inputs:

1. Workflow: Defines which tests to run
2. Binaries: Maps template keys to file paths (e.g., BINARY=~/Downloads/coreboot.rom or OS_IMAGE=~/Downloads/ubuntu.img)

The template keys correspond to variables in your test files, allowing you to specify which binaries are used during test execution. You can run all tests in a workflow or select a subset. Optional metadata like commit hashes can be included.

Job Triggers

We offer three different methods to trigger a job:

1. Command-Line Interface: Using the fwci job command.

   • Interactive Mode: This mode prompts you to select the workflow and specify whether to run all tests or only a subset. If only one workflow is available, it will be automatically selected.

     # Login using your credentials
     $ fwci login <email> <password>
     
     # Start a job
     $ fwci job Binary=~/Downloads/my-rom.rom
     
     Selected workflow: fwci-crystal-core
     Do you want to select partial tests to be run? (y/n): n
     Selected all tests
     Uploading binary: /home/l1x0r/Downloads/coreboot.rom
     This may take a while...

   • Non-Interactive Mode: The CLI can also operate non-interactively by providing the workflow name as the second argument:

     $ fwci job Binary=~/Downloads/coreboot.rom -w fwci-crystal-core
     
     Selected workflow: fwci-crystal-core
     Selected all tests
     Uploading binary: /home/l1x0r/Downloads/coreboot.rom
     This may take a while...

     Caution: You might be required to provide the project-name as well, if the workflow-name is not unique across projects.

2. Integrations: Integration with version control systems (GitHub or GitLab).

   Create an user/org token to authenticate.

   Getting Your Workflow ID:

   The WORKFLOW_ID is required for all CI/CD integrations. To obtain it:

   1. Log in to the FirmwareCI web UI
   2. Navigate to your workflow
   3. Click the Copy ID button inside the workflow
   4. Use this ID in your CI/CD configuration (see examples below)

   • GitHub Action:

     - name: Upload to FirmwareCI
       uses: docker://firmwareci/action:v5.1
       with:
         TOKEN: "${{ secrets.TOKEN}}"
         # EMAIL: ${{ secrets.FWCI_EMAIL }}
         # PASSWORD: ${{ secrets.FWCI_PASSWORD }}
         WORKFLOW_ID: "${{ secrets.FWCI_WORKFLOW_ID}}"
         COMMIT_HASH: ${{ github.event.pull_request.head.sha || github.sha }}
         BINARIES: BINARY=./coreboot.rom
         # GITHUB_INSTALLATION_ID: 12345678 (Only required if differs from the integration source)

   • GitLab CI/CD:

     # Push binary artifacts to FirmwareCI
     push_firmwareci:
       stage: push_firmwareci
       needs:
         - job: build_coreboot
     
       image: firmwareci/action:v5.1
       variables:
         TOKEN: $FWCI_TOKEN
         # EMAIL: $FWCI_EMAIL
         # PASSWORD: $FWCI_PASSWORD
         WORKFLOW_ID: $FWCI_WORKFLOW_ID
       # GITLAB_TOKEN: glpa_.... (Only required if it differs from the integration source)
         BINARIES: BINARY=/builds/$CI_PROJECT_PATH/build/coreboot.rom;Data=/builds/$CI_PROJECT_PATH/build/data
       script:
         - /bin/action

   • Gerrit CI/CD: Please refer to Gerrit Integration Guide.

3. REST API: Making HTTP requests to trigger jobs. For further information, refer to FirmwareCI API Documentation.

Backreporting

Backreporting ensures that test results from FirmwareCI are automatically communicated back to your version control system. We support backreporting for both GitHub, GitLab & Gerrit. Jobs can be initiated from any repository, not limited to the integration source, allowing you to trigger tests across platforms (e.g., GitHub and GitLab).

GitHub Backreporting

For GitHub, FirmwareCI provides an official GitHub App that can be installed in your repository. This app automatically reports:

• Pull Requests: Adds test result comments directly on the PR.
• Default Branch Commits: Updates the commit status with test results.

Installation & Setup

1. Install the FirmwareCI GitHub App.
2. Grant it access to the repositories where you want backreporting enabled.
3. Once a job is triggered through the CI, it will report back into this repository.

GitLab Backreporting

For GitLab, users need to manually provide an access token for FirmwareCI to authenticate and post test results. The results are reported as:

• Commit Comments: Test status is added as a comment and status to the commit.
• Merge Requests: Adds results as a comment and status on the MR.

Setup Instructions

1. Create a Gitlab access token (select one method):

   Option 1: User Access Token

   • Navigate to your Gitlab account settings
   • Select “Access Tokens” in the left navigation panel

   

   • Generate a new token with the required scopes: api, read_repository

   Option 2: Project Access Token

   • Access your Gitlab project repository
   • Navigate to Settings > Access Tokens

   

   • Create a project access token with minimum developer role permissions and both api and read_repository scopes

   

   • Integrate the repository using the following command:

   fwci integration --gitlab-auth="YOUR_ACCESS_TOKEN"

2. Visit the webpage of your project.

3. Scroll down to the Webhook Settings.

   

4. Add the webhook as shown and save.

Note: The secret token for the webhook will be provided upon request. Please contact the FirmwareCI team to obtain your specific webhook token.

Default-Webhook-URL:

   https://api.firmwareci.9esec.dev:8443

Gerrit Backreporting

Please refer to Gerrit Integration Guide.

Linking Back to Job Triggers

For details on how to trigger a job, refer to the Job Triggers section.