Running a Job

Prerequisites

• Ensure FirmwareCI is set up by following the steps in the Initial Setup guide.

What is a Job?

A job represents the execution of all tests defined within a workflow on a Device Under Test (DUT) that shares the same label. The results of the job will be communicated back to your FirmwareCI dashboard. Additionally, if you are utilizing an integration such as our GitHub App, the results will also be reported back to the corresponding GitHub commit page.

Each job requires a workflow and a binary file, typically the firmware binary used to flash the devices. However, it could also be a host OS image used to boot the device or any other binary file required for your specific use case. Instead of running all the tests in a workflow, users can choose to run only a subset of tests. A job might also contain additional information such as the file name, the commit hash used to trigger the job, and other relevant information.

Job Triggers

We offer three different methods to trigger a job:

1. Command-Line Interface: Using the fwci job command.

   • Interactive Mode: This mode prompts you to select the workflow and specify whether to run all tests or only a subset. If only one workflow is available, it will be automatically selected.

     # Login using your credentials
     $ fwci login <email> <password>
     
     # Start a job
     $ fwci job ~/Downloads/my-rom.rom
     
     Selected workflow: fwci-crystal-core
     Do you want to select partial tests to be run? (y/n): n
     Selected all tests
     Uploading binary: /home/l1x0r/Downloads/coreboot.rom
     This may take a while...

   • Non-Interactive Mode: The CLI can also operate non-interactively by providing the workflow name as the second argument:

     $ fwci job ~/Downloads/coreboot.rom fwci-crystal-core
     
     Selected workflow: fwci-crystal-core
     Selected all tests
     Uploading binary: /home/l1x0r/Downloads/coreboot.rom
     This may take a while...

2. Integrations: Integrating with version control systems (GitHub or GitLab).

   • GitHub Action:

     - name: Upload to FirmwareCI
       uses: docker://firmwareci/action:v3.0
       with:
         EMAIL: ${{ secrets.FWCI_EMAIL }}
         PASSWORD: ${{ secrets.FWCI_PASSWORD }}
         WORKFLOW_ID: "${{ secrets.FWCI_WORKFLOW_ID}}"
         COMMIT_HASH: ${{ github.sha }}
         URI: ./coreboot.rom

   • GitLab CI/CD:

     # Push binary artifacts to FirmwareCI
     push_firmwareci:
       stage: push_firmwareci
       needs:
         - job: build_coreboot
     
       image: firmwareci/action:v3.0
       variables:
         EMAIL: $FWCI_EMAIL
         PASSWORD: $FWCI_PASSWORD
         WORKFLOW_ID: $FWCI_WORKFLOW_ID
         URI: /builds/$CI_PROJECT_PATH/build/coreboot.rom
       script:
         - /bin/action

3. REST API: Making HTTP requests to trigger jobs. For further information, refer to FirmwareCI API Documentation.

Backreporting

Backreporting ensures that test results from FirmwareCI are automatically communicated back to your version control system. We support backreporting for both GitHub and GitLab.

GitHub Backreporting

For GitHub, FirmwareCI provides an official GitHub App that can be installed in your repository. This app automatically reports:

• Pull Requests: Adds test result comments directly on the PR.
• Default Branch Commits: Updates the commit status with test results.

Installation & Setup

1. Install the FirmwareCI GitHub App.
2. Grant it access to the repositories where you want backreporting enabled.
3. Once a job is triggered through the CI, it will report back into this repository. It’s mandatory for this that the test definitions are also within the same repository.

Example GitHub PR Report Screenshot:

GitLab Backreporting

For GitLab, users need to manually provide an access token for FirmwareCI to authenticate and post test results. The results are reported as:

• Commit Comments: Test status is added as a comment to the commit.
• Pull Requests (Merge Requests): Adds results as a comment on the MR.

Setup Instructions

1. Generate a GitLab Access Token with API write permissions.
2. TBD

Example GitLab MR Report Screenshot:

Linking Back to Job Triggers

For details on how to trigger a job, refer to the Job Triggers section.

By setting up backreporting, you ensure that firmware testing results are always visible where developers need them most—directly in their repositories.