GitHub Actions and GitLab CI
Your pipeline runs a workflow’s tests with the FirmwareCI action in GitHub Actions, or with a template job in
GitLab CI. Both run fwci run and take the same switches. By default the tests run on
FirmwareCI, on one of your organization’s DUTs with the workflow’s label, and FirmwareCI reports them on the
commit. With local: true they run on the runner itself, next to your board.
Before the first run on FirmwareCI, set up:
- The repository as a FirmwareCI project, with the workflow pushed, see Repository Setup.
- A DUT with the workflow’s label, see Connecting Hardware.
- An API token for the pipeline, see Authentication.
GitHub Actions
This runs every test of the workflow crystal-core, as FirmwareCI synced it from the commit the run is on, with
build/coreboot.rom for [[input.Binary]]. The step ends once the tests are queued, and the
FirmwareCI GitHub App reports their results on the commit.
| You want | Add |
|---|---|
| Some tests, not all | tests: boot, flash, by the name in their files |
| The step to wait for the results | follow: true |
| This checkout’s tests on FirmwareCI, not the synced ones | synced: false |
| The run on this runner, not FirmwareCI | local: true |
| A particular DUT for a run on this runner | dut: board-2; without it, the first free DUT with the workflow’s label |
The workflow is the name in its workflow.yaml. The Workflow filter on the Job Requests page of the
web UI lists the names too.
On FirmwareCI
FirmwareCI finds the workflow in this repository’s project, on the branch or tag the run is on, and runs each
test as a job on any free DUT with the workflow’s label. The FirmwareCI app reports each test as a check on the
commit, which links the job and the run’s JUnit report. With follow: true the
step waits for the jobs: it fails on a failed test, the job’s own check gets an annotation for each failed step,
the run’s summary page a table of the tests, and fwci.log and report.xml are kept as an artifact. The runner
is held while the jobs queue. Each job and matrix leg keeps its own artifact; when the action runs twice in one
job, set artifact, such as artifact: fwci-${{ matrix.board }}.
synced: false sends this checkout’s test files instead, as a one-shot run that no commit reports. The
workflow, its label and its DUTs still come from FirmwareCI, so the workflow has to be pushed once.
FirmwareCI picks the DUT, so dut is refused there: give a board a label of its own to run on it alone.
On pull_request events, leave commit_hash unset: the action reports on the pull request’s head.
${{ github.sha }} is GitHub’s merge commit, not the commit under test.
On the Runner
With local: true the tests come from this checkout’s .firmwareci and run one after another on one DUT, as
in Running Tests Locally, with no FirmwareCI login. The runner needs Docker and has to reach
the board: a GitHub-hosted runner reaches only virtual and mock DUTs, so a board needs a self-hosted runner
next to it. Results show as with follow: true, and the artifact is kept also when a test fails. A test’s
[[secrets.X]] comes from FWCI_SECRET_X in the step’s env.
In a public repository, a pull request from a fork brings its own .firmwareci, and its tests would run on your
board with the runner’s Docker. Run such a job only on push, or require approval for workflows from outside
contributors. Artifacts of a public repository are public too.
The step’s exit code says how it went: 0 every test passed, 1 a test failed, 6 a test couldn’t start, 10 every
DUT is held by someone else or its agent can’t be reached. wait waits only for held DUTs.
Inputs
| Input | Description |
|---|---|
workflow | The workflow to run, by name. Required. |
tests | Test names, separated by commas or newlines. Default: all of the workflow’s |
binaries | name=file, or name=URL on FirmwareCI, separated by ; or newlines, for [[input.name]]. |
local | true runs the tests on this runner instead of FirmwareCI. Default: false |
synced | On FirmwareCI, the tests as FirmwareCI synced them, reported on the commit. false sends this checkout’s tests, as a one-shot run not reported on a commit. Default: true |
args | More arguments for fwci run, such as --verbose. |
working_directory | Where fwci runs, and binaries are relative to. With local: true or synced: false, inside the repository whose .firmwareci holds the workflow. Default: . |
artifact | Name of the artifact with fwci.log and report.xml, with local: true or follow. Empty keeps none. Default: fwci-<job>-<matrix index> |
version | The fwci to install, 0.21.0 or newer. installed uses the one on the PATH, as on a self-hosted runner that has it. Default: latest |
| On FirmwareCI | |
token | A FirmwareCI organization or user API token. |
email, password | Log in with these instead of a token. |
organization | The organization to submit to, when the credential reaches more than one. An organization token names its own. |
follow | true waits for the jobs, see above. Default: false |
priority | low, mid or high. Unset, FirmwareCI queues the run at mid, or at high with synced: false, see Queue Priority. |
branch | The branch to find the workflow on. Defaults to the branch or tag the run is on. |
project_link | The repository holding the FirmwareCI config, when it isn’t this one, see Config in Another Repository. Needs synced (the default). |
commit_hash | The commit to report on. Defaults to the pull request’s head, or the commit the run is on. Needs synced. |
action | A label for the run, such as nightly or release. Needs synced. |
github_installation_id | The FirmwareCI GitHub App’s installation, when project_link is in another GitHub organization. Needs synced. |
api | The FirmwareCI API, for a self-hosted instance. Default: https://api.firmware-ci.com |
| On the runner | |
dut | The DUT, by its directory under .firmwareci/duts or its name. It must carry the workflow’s label. |
wait | true waits while someone else holds the DUTs, 30m at most that long. |
storage | name=directory, separated by ; or newlines, for [[storage.name]]. |
ssh_keys | name=private key file, separated by ; or newlines, for [[ssh-keys.name]]. |
local, synced and follow take true or false, and empty keeps the default. Anything else stops the step,
and so does local: true with synced: false.
The action installs fwci on Linux x64 runners. On another runner, install fwci yourself, see
Installation, and set version: installed.
Authentication
In the web UI, Profile in the account menu opens your settings. Create a token under API Tokens there.
An organization admin can create an organization token under the organization’s API Tokens on the same page.
Add the token as the secret FWCI_TOKEN (Settings, Secrets and variables, Actions). Pass it as token. The
action hands it to fwci for that step alone, so no login stays on the runner. email and password work too.
A run on the runner needs none.
An organization token is the better CI credential: it names its organization and can’t reach another. A user
token, or email and password, for an account in several organizations needs organization, by the name
fwci org list shows.
GitLab CI
A template has a job to extend, with the same switches as variables:
The job runs the firmwareci/action:v8 image, which follows the newest fwci within a day;
firmwareci/action:0.21.0 and so on keep one. Set FWCI_TOKEN as a masked CI/CD variable.
| Variable | Description |
|---|---|
FWCI_WORKFLOW | The workflow to run, by name. Required. |
FWCI_TESTS | Test names, separated by commas or newlines. Default: all of the workflow’s |
FWCI_BINARIES | name=file, or name=URL on FirmwareCI, separated by ; or newlines. |
FWCI_LOCAL | true runs the tests on this runner instead of FirmwareCI. Default: false |
FWCI_SYNCED | On FirmwareCI, the tests as FirmwareCI synced them, reported on the commit. false sends this checkout’s tests. Default: true |
FWCI_FOLLOW | true waits for the jobs and fails on a failed test. Default: false |
FWCI_DUT | With FWCI_LOCAL: "true", the DUT. Default: the first free one with the workflow’s label |
FWCI_ARGS | More fwci run flags, such as --priority low, -b main, --wait=30m or --storage OS=images. |
FWCI_TOKEN, FWCI_EMAIL, FWCI_PASSWORD | The login, as for the action. |
FWCI_ORG | The organization, when the credential reaches more than one. |
FWCI_API | The FirmwareCI API, for a self-hosted instance. Default: https://api.firmware-ci.com |
FWCI_GITLAB_TOKEN | Only if the project’s GitLab integration has no token: a project access token with api scope, for FirmwareCI to report back. |
FWCI_LOCAL, FWCI_SYNCED and FWCI_FOLLOW take true, 1 or yes, or false, 0 or no, and empty
keeps the default. Anything else stops the job, and so does FWCI_LOCAL true with FWCI_SYNCED false. Quote the
value, as in FWCI_LOCAL: "true", since GitLab refuses an unquoted true or false. When the job waits for the
tests, GitLab shows report.xml in the merge request. Otherwise fwci writes neither fwci.log nor
report.xml, so GitLab’s warning that it found no files to upload is expected.
To run the tests on a runner next to your board, pick that runner with tags:
A test’s [[secrets.X]] comes from the CI/CD variable FWCI_SECRET_X.
A run on the runner with a docker executor needs the host’s network and Docker socket, and a cache directory at
the same path in the job as on the host, as fwci hands paths in it to the host’s Docker. In the runner’s
config.toml:
A shell executor with fwci and Docker installed needs nothing more.
Config in Another Repository
FirmwareCI looks for the workflow in the project of the repository the pipeline runs in. A common setup keeps
the FirmwareCI config (.firmwareci/) in a dedicated repository, such as my-org/firmware-tests, and runs the
pipeline in a product repository, such as my-org/my-product. Name the config’s repository with project_link,
or --project-link in FWCI_ARGS. A job request’s page in the web UI shows the value under its title, such as
github.com/my-org/firmware-tests.
| Value | Config on |
|---|---|
github.com/org/repo | GitHub, from any CI |
gitlab.com/group/name | GitLab.com, from any CI |
gitlab.example.com/group/name, or with https:// | A self-hosted GitLab |
org/repo | GitHub in GitHub Actions, GitLab.com in GitLab CI |
project_link only decides where the workflow comes from. The results still go back to the commit the pipeline
runs on. The credentials need both repositories:
- GitHub: the FirmwareCI GitHub App must be
installed in the organization that owns the config’s repository. When that is another organization than the
pipeline’s, set
github_installation_idto that installation. - GitLab: the project’s GitLab integration token, or
FWCI_GITLAB_TOKEN, needs at leastread_repositoryon the config’s repository, on the same GitLab instance.
Without the Action
In GitHub Actions and GitLab CI, a step that runs fwci run --synced does the same: it logs in with FWCI_TOKEN
for that one command and reports on the commit, see Running a Job. In another CI it runs the
tests as synced too, but FirmwareCI reports them on no commit.
Moving from v7
docker://firmwareci/action:v7.0 keeps working as before, with its upper-case inputs, and gets no more changes.
To move to v8, use BlindspotSoftware/actions@v8 in GitHub Actions; the v8 image fails a docker:// step. In
GitLab CI, extend .fwci instead of running /bin/action in the v7.0 image.
| v7 | v8 input | v8 GitLab variable |
|---|---|---|
WORKFLOW_NAME | workflow | FWCI_WORKFLOW |
INDIVIDUAL_TESTS | tests | FWCI_TESTS |
BINARIES | binaries | FWCI_BINARIES |
TOKEN, EMAIL, PASSWORD | token, email, password | FWCI_TOKEN, FWCI_EMAIL, FWCI_PASSWORD |
ORGANIZATION | organization | FWCI_ORG |
API | api | FWCI_API |
GITLAB_TOKEN | FWCI_GITLAB_TOKEN | |
PRIORITY | priority | --priority in FWCI_ARGS |
BRANCH | branch | -b in FWCI_ARGS |
PROJECT_LINK | project_link | --project-link in FWCI_ARGS |
COMMIT_HASH | commit_hash | --commit in FWCI_ARGS |
ACTION | action | --action in FWCI_ARGS |
GITHUB_INSTALLATION_ID | github_installation_id | |
CI | gone, fwci tells the CI itself | gone |