Commands

This page describes the structure of of test step commands.

Test Step Commands

Each test step in a stage has this structure:

Attribute	Type	Required	Description
`cmd`	`string`	Yes	Identifies which test step to execute
`name`	`string`	Yes	Uniquely identifies the step for reports and logs
`transport`	`TransportObject`	Yes	Defines how the command is executed
`parameters`	`object`	Yes	Command-specific parameters
`options`	`OptionsObject`	No	Additional command options

Examples

- cmd: dutctl
  name: Shutdown the DUT.
  parameters:
    host: "[[attributes.Flasher]]"
    command: power
    args: ["off"]

- cmd: dutctl
  name: Flash the provided binary.
  parameters:
    host: "[[attributes.Flasher]]"
    command: flash
    args: [write, "[[defaults.Binary]]"]

- cmd: sleep
  name: Wait for the DUT to settle
  parameters:
    duration: 15s

- cmd: dutctl
  name: Turn the DUT on
  parameters:
    host: "[[attributes.Flasher]]"
    command: power
    args: ["hardreset"]
  options:
    timeout: 2m

- cmd: ping
  name: Wait for the device to become online.
  options:
    timeout: 4m
  parameters:
    host: "[[attributes.Host]]"

Transport

The transport object defines how test steps are executed, either locally or on a remote machine. This configuration ensures proper command execution and authentication.

Attribute	Type	Required	Description
`proto`	`string`	Yes	Protocol for command execution: `local` (local execution) or `ssh` (remote execution)
`options`	`TransportOptions`	Required for SSH	Protocol-specific configuration options

Transport Options

When using the SSH protocol (proto: "ssh"), the following options are available:

Attribute	Type	Required	Default	Description
`host`	`string`	Yes		Hostname or IP address of remote machine
`user`	`string`	Yes		Username for authentication
`port`	`integer`	No	`22`	SSH port number
`password`	`string`	One of these required*		User’s password
`identity_file`	`string`	One of these required*		Path to SSH private key

* You must provide either password or identity_file, but not both.

Protocol-Specific Requirements

Local Protocol

When using proto: "local":

No options are required or allowed
Commands execute on the local machine

SSH Protocol

When using proto: "ssh":

The options object is required
Must provide host and user
Must provide either password or identity_file
Cannot provide both password and identity_file

Examples

Local Execution

transport:
  proto: local

SSH with Password

transport:
  proto: ssh
  options:
    host: "192.168.1.100"
    user: "admin"
    password: "secretpassword"

SSH with Identity File

transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: "root"
    identity_file: "/root/.ssh/id_ecdsa"
    port: 2222 # Optional custom port

Options

The options object provides common configuration settings that can be applied to any test step. These settings control the test step’s execution behavior.

Attribute	Type	Required	Default	Description
`timeout`	`string`	No	`6h00m00s`	Maximum duration for test step execution. If the step takes longer than this timeout, it will fail. Format: hours(h), minutes(m), seconds(s)

Timeout Format

The timeout value must follow this pattern: ^((\\d+h)?(\\d+m)?(\\d+s)?)?$

Valid examples:

"30s" - 30 seconds
"5m" - 5 minutes
"2h30m" - 2 hours and 30 minutes
"1h30m45s" - 1 hour, 30 minutes, and 45 seconds

Examples

Basic Timeout

options:
  timeout: "30s"

Complex Timeout

options:
  timeout: "1h30m45s"

General

Command

Run any executable or bash-cmd on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`cmd`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	No		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`CommandOptions`	Yes		Options passed into the `cmd` cmd

Command Options

Attribute	Type	Required	Default	Description
`executable`	`string`	Yes		Executable file
`Args`	[]`string`	No		Array of arguments passed to executable
`working_dir`	`bool`	No	`false`	Specify the working directory
`report_only`	bool	No	`false`	Do not fail on exit-code non-equal to zero
`expect`	[]`string`	No		Check the stdout for any regexs

Command Example

cmd: cmd
name: Run echo test
transport: *transport
options:
  timeout: 20m
parameters:
  executable: "echo"
  args:
    [
      "Hello World!"
    ]
  expect:
    - "Hello World!"

Copy

Copy files to the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`copy`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	No		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`CopyOptions`	Yes		Options passed into the `copy` cmd

Copy Options

Attribute	Type	Required	Default	Description
`source`	`string`	Yes		Source Location of the file to copy to the DUT.
`destination`	`string`	Yes		Destination Location of the file to copy to the DUT
`recursive`	`bool`	No	`false`	Copy recursively

Copy Example

cmd: copy
name: Copy some tooling to DUT
transport: *transport
parameters:
  source: "[[storage.my-tooling]]/tool"
  destination: /tmp/tool
  recursive: true

DUTCTL

Run DUTCTL for the DUT. This command can power-cycle andflash the DUT aswell as read the serial of the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`dutctl`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`options`	`OptionsObject`	No		Additional command options
`parameters`	`DUTCTLOptions`	Yes		Options passed into the `dutctl` cmd

DUTCTL Options

Attribute	Type	Required	Description
`host`	`string`	Yes	Hostname or IP address of the device.
`command`	`string`	Yes	Command to execute on the device. Available commands: `power`, `flash`, `serial`.
`args`	`[]string`	No	Arguments for the command. Arguments depend on the command that is executed.
`ìnput`	`string`	No	Input for the command, used only if command is set to `serial`.
`uart`	`integer`	No	UART number to use for serial communication.

DUTCTL Expect Options

Attribute	Type	Required	Default	Description
`regex`	`string`	Yes	`expect`	Regular expression to match against the output of the device for the `serial` command.

DUTCTL Example

- cmd: dutctl
  name: Shutdown the DUT.
  parameters:
    host: "[[attributes.Flasher]]"
    command: power
    args: ["off"]

- cmd: dutctl
  name: Flash the provided binary.
  parameters:
    host: "[[attributes.Flasher]]"
    command: flash
    args: [write, "[[defaults.Binary]]"]

- cmd: dutctl
  name: Turn the DUT on
  parameters:
    host: "[[attributes.Flasher]]"
    command: power
    args: ["hardreset"]
  options:
    timeout: 2m

Firmware Version

Check the ESRT Firmware version entry on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`firmware version`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`FirmwareVersionOptions`	Yes		Options passed into the `firmware version` cmd

Firmware Version Options

Attribute	Type	Required	Default	Description
`tool_path`	`string`	Yes		Path to the tool used to retrieve the firmware version.
`format`	`string`	No	`number`	Format of the tool’s output. Available options: `number`, `hex`, `pair`, `triplet`, `quad`.
`expect`	`Firmware Version Expect Options`	Yes		Expected Options

Firmware Version Expect Options

Attribute	Type	Required	Default	Description
`version`	`string`	Yes	`expect`	Expected firmware version in the specified format.

Firmware Version Example

cmd: firmware version
name: Get firmware version (ESRT)
transport: *transport
parameters:
  tool_path: /tmp/system-suite/system-suite
  format: pair
  expect:
    version: "1.11"

PiKVM

Run a ping on any host.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`pikvm`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`options`	`OptionsObject`	No		Additional command options
`parameters`	`PiKVMOptions`	Yes		Options passed into the `ping` cmd

PiKVM Options

Boot an OS image on your PiKVM connected to the DUT.

Attribute	Type	Required	Description
`host`	`string`	Yes	Hostname of the PiKVM
`image`	`string`	Yes	Path to OS image to boot
`username`	`string`	Yes	PiKVM username
`password`	`string`	Yes	PiKVM pasword

PiKVM Example

cmd: pikvm
name: Setup Host Image
parameters:
  host: "[[attributes.PiKVM]]"
  image: "[[storage.Image]]/nixos.img"
  username: root
  password: root

Ping

Run a ping on any host.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`ping`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`options`	`OptionsObject`	No		Additional command options
`parameters`	`PingOptions`	Yes		Options passed into the `ping` cmd

Ping Options

Attribute	Type	Required	Default	Description
`host`	`string`	Yes		Hostname or IP address used for the ping.
`port`	`integer`	No	`22`	Port to use for the ping.
`expect`	`PingExpectOptions`	Yes		`ping` expect options

Ping Expect Options

Attribute	Type	Required	Default	Description
`should_fail`	`boolean`	No	`false`	If set to true, the expected behavior is that the ping won’t work. Otherwise, a successful ping is expected.

Ping Example

cmd: ping
name: Wait for the device to become online.
options:
  timeout: 4m
parameters:
  host: "[[attributes.Host]]"

Sleep

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`sleep`
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`parameters`	`SleepOptions`	Yes		Options passed into the `sleep` cmd

Sleep Options

Attribute	Type	Required	Default	Description
`duration`	`string`	Yes		Duration to sleep. Format: 1h2m3s. Units: h, m, s.

Sleep Example

cmd: sleep
name: Wait for the DUT to settle
parameters:
  duration: 15s

BIOS

Different Commands related to the BIOS.

BIOS Certificate

Check BIOS Options on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`bios certificate management`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`BIOS Certificate Options`	Yes		Options passed into ChipSec

BIOS Certificate Options

Attribute	Type	Required	Description
`tool_path`	`string`	Yes	Required tooling systemsuite
`password`	`string`	Either of*	BIOS Password
`cert_path`	`string`	Either of*	BIOS Certificate
`key_path`	`string`	Either of*	BIOS Certificate Private Key
`expect`	`BIOS Certificate Expect Options`	No	Array of expected BIOS Certificate Options

BIOS Certificate Expect Options

Attribute	Type	Required	Default	Description
`should_fail`	`bool`	Yes		BIOS Option

BIOS Certificate Example

- cmd: bios certificate management
  name: Enable certificate
  transport: *transport
  parameters:
    command: enable
    tool_path: /tmp/system-suite/system-suite
    password: secunet
    cert_path: /tmp/system-suite/4k/new/cert.pem
- cmd: bios certificate management
  name: Check certificate thumbprint
  transport: *transport
  parameters:
    command: check
    tool_path: /tmp/system-suite/system-suite
    cert_path: /tmp/system-suite/4k/new/cert.pem
- cmd: bios certificate management
  name: Disable certificate
  transport: *transport
  parameters:
    command: disable
    tool_path: /tmp/system-suite/system-suite
    key_path: /tmp/system-suite/4k/new/key.pem
    password: secunet

BIOS Settings Get

Check BIOS Options on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`get bios setting`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`BIOS Settings Get Options`	Yes		Options passed into ChipSec

BIOS Settings Get Options

Attribute	Type	Required	Default	Description
`tool_path`	`string`	Yes		Required tooling systemsuite
`expect`	`[]BIOS Settings Get Expect Options`	Yes		Array of expected BIOS Settings

BIOS Settings Get Expect Options

Attribute	Type	Required	Default	Description
`option`	`string`	Yes		BIOS Option
`value`	`string`	Yes		Expected Value (supports regex)

BIOS Settings Get Example

cmd: get bios setting
name: "Check BIOS Settings"
transport: *transport
parameters:
  tool_path: /tmp/system-suite/system-suite
  expect:
    - option: BootDisplayDevice
      value: LCD
    - option: BootMode
      value: Quick
    - option: BootOrderLock
      value: Disable

BIOS Settings Set

Try to set BIOS Options on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`get bios setting`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`BIOS Settings Set Options`	Yes		Options passed into ChipSec

BIOS Settings Set Options

Attribute	Type	Required	Description
`tool_path`	`string`	Yes	Required tooling systemsuite
`password`	`string`	No	BIOS Password
`key_path`	`string`	No	BIOS Cert Key
`bios_options`	`[]BIOS Options`	Yes	Array of BIOS Options to set

BIOS Options

Attribute	Type	Required	Description
`option`	`string`	Yes	BIOS Option
`value`	`string`	Yes	Value to set
`should_fail`	`bool`	No	Set to true, if the cmd should fail

BIOS Settings Set Example

cmd: set bios setting
name: "Set Target BIOS Settings"
transport: *transport
parameters:
  tool_path: /tmp/system-suite/system-suite
  password: <bios-password>
  bios_options:
    - option: AlwaysOnUSB
      value: Disable
    - option: FingerprintPasswordAuthentication
      value: Disable

CPU

CPULoad

Run a CPU-Load test on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`cpuload`	Command Name
`name`	`string`			Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	No		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`CPULoadOptions`	No	`parameters`	`cpuload` Options

CPULoad Options

Attribute	Type	Required	Description
`tool_path`	`string`	Yes	Path to the tool on the DUT
`args`	`[]string`	Yes	Array of `strings` with arguments passed into the `cpuload` tool.
`cpus`	`[]integer`	Yes	Array of CPUs that should be put under load.
`duration`	`string`	Yes	Duration of the load. Format: `^((\\d+h)?(\\d+m)?(\\d+s)?)?$`
`expect`	`CPUExpectOptions`	Yes	Expected values

CPULoad Example

cmd: cpuload
name: Check cpu frequencies with load
transport: *transport
options:
  timeout: 2m
parameters:
  tool_path: /tmp/system-suite/system-suite
  duration: 1m
  expect:
    individual:
      - cpus: [0, 1, 2, 3, 4, 5, 6, 7]
        option: AverageFrequency
        value: ">1500"
      - cpus: [8, 9, 10, 11, 12, 13, 14, 15]
        option: AverageFrequency
        value: ">1200"

CPUSet

Set cpu profiles on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`cpuset`	Command Name
`name`	`string`			Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	No		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`CPUSetOptions`	Yes		Options passed into the `cpuset` cmd

CPUSet Options

Attribute	Type	Required	Description
`tool_path`	`string`	Yes	Path to the tool on the DUT
`command`	`string`	Yes	Command to run. Available options: `core`, or `profile`.
`cores`	`[]integer`	No	Array of Cores where the command should be applied to. If not set apply to all
`arg`	`string`	Yes	Value of the command option

CPUSet Example

cmd: cpuset
name: Set ACPI Platform Profile to low-power
transport: *transport
parameters:
  tool_path: /tmp/system-suite/system-suite
  command: profile
  arg: low-power

CPUStats

Run a CPU-Load test on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`cpuload`	Command Name
`name`	`string`			Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	No		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`CPUStatsOptions`	Yes	`parameters`	`cpuload` Options

CPUStats Options

Attribute	Type	Required	Description
`tool_path`	`string`	Yes	Path to the tool on the DUT
`interval`	`string`	No	Testing interval time
`expect`	`CPUExpectOptions`	Yes	Expected values

CPU Expect Options

Attribute	Type	Required	Default	Description
`general`	`[]GeneralObject`	No	`expect`	General Expected Values
`individual`	`[]IndividualObject`	No		Individual CPU Expected Values

General Object

Attribute	Type	Required	Default	Description
`option`	`string`	No	`general`	Name of the option
`value`	`string`	No	`general`	Value of the option

Individual Object

Attribute	Type	Required	Default	Description
`cpu`	`[]integer`	Yes	`individual`	Array of CPUs where the option should be applied to
`option`	`string`	Yes	`individual`	Name of the option
`value`	`string`	Yes	`individual`	Value of the option

CPUStats Example

cmd: cpustats
name: Check cpu frequencies in idle
transport: *transport
parameters:
  tool_path: /tmp/system-suite/system-suite
  interval: 10s
  expect:
    individual:
      - cpus: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
        option: AverageFrequency
        value: "<500"

Various

Binarly

The Binarly test step executes a scan using the Binarly service to analyze firmware binaries.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	binarly	Must be `binarly report`
`name`	`string`	Yes	Uniquely identifies the test step in reports and logs
`parameters`	`BinarlyParameters`	Yes	Binarly-specific parameters
`options`	`OptionsObject`	No	Additional command options

Binarly Parameters

Attribute	Type	Required	Description
`url`	`string`	Yes	URL of the Binarly endpoint
`token`	`string`	Yes	Authentication token for Binarly service
`file_path`	`string`	Yes	Path to the binary file to be scanned

Binarly Example

cmd: binarly report
name: "Scan BIOS for vulnerabilities"
parameters:
  url: "https://api.binarly.io/v1"
  token: "your-binarly-token"
  file_path: "/path/to/firmware.bin"
options:
  timeout: "30m" # Scan might take a while

ChipSec

Run ChipSec on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`chipsec`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	No		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`ChipSecOptions`	Yes		Options passed into ChipSec

ChipSec Options

Attribute	Type	Required	Description
`modules`	`[]string`	Yes	List of modules that get executed from the `ChipSec` test suite
`pch`	`string`	No	PCH parameter that gets passed into `ChipSec` as `--pch` parameter
`platform`	`string`	No	Platform parameter that gets passed into `ChipSec` as `--platform` parameter

ChipSec Example

cmd: chipsec
name: Run chipsec tests
transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: root
    password: root
parameters:
  modules:
    [
      common.bios_kbrd_buffer,
      common.bios_smi,
      common.bios_ts,
      common.bios_wp,
      common.ia32cfg,
      common.me_mfg_mode,
      common.memlock,
      common.rtclock,
      common.smm,
      common.smrr,
      common.spi_access,
      common.spi_desc,
      common.spi_fdopss,
      common.spi_lock,
      common.uefi.access_uefispec,
      common.memconfig,
      common.remap,
      common.smm_dma,
    ]

FWHunt

Run Firmware Hunt on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`fwhunt`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`options`	`OptionsObject`	No		Additional command options
`parameters`	`FWHuntOptions`	Yes		Options passed into the `fwhunt` cmd

FWHunt Options

Attribute	Type	Required	Default	Description
`rules_dirs`	`[]string`	No		List of directories to search for rules. If not specified, the default rules directory will be used.
`rules`	`[]string`	No		List of rules to run.
`report_only`	`boolean`	No	`false`	Only report findings, do not interpret the output.

FWHunt Example

name: FwHunt Test
steps:
  - cmd: fwhunt
    name: Run FwHunt tests
    parameters:
      report_only: true

FWTS

Run Firmware Test-Suite on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`fwts`	Command Name
`name`	`string`			Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`FWTS Options`			Options passed into the `fwts` cmd

FWTS Options

Attribute	Type	Required	Default	Description
`flags`	`[]string`	No		List of flags to pass to `fwts`.
`report_only`	`boolean`	No	`false`	Only report findings, do not interpret the output.

FWTS Example

  cmd: fwts
  name: Run Firmware Testsuite tests
  options:
    timeout: 10m
  transport: *transport
  parameters:
    flags: [-b]
    report_only: true

HSI

Report Host Firmware Security on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`hsi`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options

HSI Example

  cmd: hsi
  name: Report Host Firmware Security
  options:
    timeout: 10m
  transport: *transport

Qemu

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`qemu`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`options`	`OptionsObject`	No		Additional command options
`parameters`	`QemuOptions`	Yes		Options passed into the `qemu` cmd

Qemu Options

Attribute	Type	Required	Description
`executable`	`string`	Yes	The executable that should be executed.
`firmware`	`string`	Yes	The firmware that should be used.
`image`	`string`	Yes	The image that should be used.
`logfile`	`string`	No	The logfile that should be used.
`nproc`	`integer`	No	The number of processors that should be used.
`mem`	`integer`	No	The amount of memory that should be used.
`steps`	`QemuStepsOptions`	Yes	Steps to interact with during test execution.

Qemu Steps Options

Attribute	Type	Required	Description
`send`	`string`	No	A string that should be sent as input.
`timeout`	`string`	No	Timeout for the interaction. Format: h for hours, m for minutes, s for seconds (e.g., 1h30m20s).
`expect`	`QemuExpectOptions`	No	Expectation for the interaction output.

Qemu Expect Options

Attribute	Type	Required	Default	Description
`regex`	`string`	Yes	`expect`	Regular expression that should be matched against the output of the interaction.

Robot Test Framework

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`robot`	Command Name
`name`	`string`			Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`Robot Options`	Yes		Options passed into the `fwts` cmd

Robot Options

Attribute	Type	Required	Default	Description
`file_path`	`string`	Yes		Path to the robot test file.
`args`	`[]string`		No	Arguments to pass to the robot test file.
`report_only`	`boolean`	No	`false`	If true, the output won’t be validated.

Robot Example

cmd: robot
name: Run OpenBMC Test Automation Setup Test
transport:
  proto: local
parameters:
  file_path: "[[storage.openbmc_test_auto]]/openbmc-test-automation/templates/test_openbmc_setup.robot"
  args:
    - "OPENBMC_HOST:[[attributes.Host]]"
    - "OPENBMC_USERNAME:root"
    - "OPENBMC_PASSWORD:root"
    - "IPMI_USERNAME:root"
    - "IPMI_PASSWORD:root"
    - "REDFISH_SUPPORT_TRANS_STATE:1"

S0ix-Selftest

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`s0ix-selftest`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options

S0ix-Selftest Example

cmd: s0ix-selftest
name: Run s0ix selftest
transport: *transport

SecureBoot Management

Run different commands, based on sbctl on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`Secure Boot Management`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`SecureBoot Options`	Yes		Options passed into the `Secure Boot Management` cmd

SecureBoot Options

Attribute	Type	Required	Default	Description
`command`	`string`	Yes		Command to execute. Can be one of: `status`, `enroll-key`, `rotate-key`, `reset`, `custom-key`.
`tool_path`	`string`	Yes		Path to the tool that should be executed to run Secure Boot commands.
`hierarchy`	`string`	Yes		Hierarchy to use for the key. Can be one of: `db`, `dbx`, `KEK`, `PK`.
`key_file`	`string`	No		Path to the key file used for `enroll-key` and `rotate-key` commands.
`custom_key_file`	`string`	No		Path to the custom key file used for `custom-key` command.
`cert_file`	`string`	No		Path to the certificate file used for `enroll-key` and `rotate-key` commands.
`signing_key_file`	`string`	No		Path to the signing key file used for `enroll-key` and `rotate-key` commands.
`signing_cert_file`	`string`	No		Path to the signing certificate file used for `enroll-key` and `rotate-key` commands.
`append`	`boolean`	No	`false`	Append the key to the existing key database.
`expect`	`SecureBoot Expect Options`	No	`parameters`	Expectations for the command execution.

SecureBoot Expect Options

Attribute	Type	Required	Default	Description
`should_fail`	`boolean`	No	`false`	If set to true, the test step will fail if the command does not fail.
`secure_boot`	`boolean`	Yes	`false`	If set to true, the test step will fail if Secure Boot is not enabled.
`setup_mode`	`boolean`	Yes	`false`	If set to true, the test step will fail if Secure Boot is not in setup mode.

SecureBoot Management Examples

- cmd: Secure Boot Management
name: Check if SetupMode is enabled
transport: *transport
parameters:
  command: status
  tool_path: /tmp/sbctl/sbctl
  expect:
    secure_boot: false
    setup_mode: true

- cmd: Secure Boot Management
name: Enroll invalid signed PK key
transport: *transport
parameters:
  command: custom-key
  tool_path: /tmp/sbctl/sbctl
  hierarchy: PK
  custom_key_file: /tmp/sbctl/keys/invalidPK.auth
  expect:
    should_fail: true

- cmd: Secure Boot Management
name: Enroll db key
transport: *transport
parameters:
  command: enroll-key
  tool_path: /tmp/sbctl/sbctl
  hierarchy: db
  key_file: /tmp/sbctl/keys/DB.key
  cert_file: /tmp/sbctl/keys/DB.crt
  signing_key_file: /tmp/sbctl/keys/KEK.key
  signing_cert_file: /tmp/sbctl/keys/KEK.crt
  append: true

- cmd: Secure Boot Management
name: Rotate db key
transport: *transport
parameters:
  command: rotate-key
  tool_path: /tmp/sbctl/sbctl
  hierarchy: db
  key_file: /tmp/sbctl/keys-new/DB.key
  cert_file: /tmp/sbctl/keys-new/DB.crt
  signing_key_file: /tmp/sbctl/keys/KEK.key
  signing_cert_file: /tmp/sbctl/keys/KEK.crt

- cmd: Secure Boot Management
  name: Reset db key
  transport: *transport
  parameters:
    command: reset
    tool_path: /tmp/sbctl/sbctl
    hierarchy: db
    signing_key_file: /tmp/sbctl/keys/KEK.key
    signing_cert_file: /tmp/sbctl/keys/KEK.crt
    cert_file: /tmp/sbctl/keys/DB.crt

SysBench

Run SysBench on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`sysbench`	Command Name
`name`	`string`			Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options

SysBench Options

Attribute	Type	Required	Default	Description
`args`	`[]string`	No		Arguments to pass to `sysbench`.
`expect`	`[]ExpectOptions`	Yes		Expectations to check against the output of `sysbench`.

SysBench Expect Options

Attribute	Type	Required	Default	Description
`option`	`string`	Yes		Option to check
`value`	`string`	Yes		Expected value

SysBench Example

cmd: sysbench
name: Benchmark
transport: *transport
parameters:
  args: [cpu, --cpu-max-prime=20000, --threads=18, --time=60, run]
  expect:
    - option: EventsPerSecond
      value: <4500

Commands

Table of Contents

Test Step Commands

Examples

Transport

Transport Options

Protocol-Specific Requirements

Local Protocol

SSH Protocol

Examples

Local Execution

SSH with Password

SSH with Identity File

Options

Timeout Format

Examples

Basic Timeout

Complex Timeout

General

Command

Command Options

Command Example

Copy

Copy Options

Copy Example

DUTCTL

DUTCTL Options

DUTCTL Expect Options

DUTCTL Example

Firmware Version

Firmware Version Options

Firmware Version Expect Options

Firmware Version Example

PiKVM

PiKVM Options

PiKVM Example

Ping

Ping Options

Ping Expect Options

Ping Example

Sleep

Sleep Options

Sleep Example

BIOS

BIOS Certificate

BIOS Certificate Options

BIOS Certificate Expect Options

BIOS Certificate Example

BIOS Settings Get

BIOS Settings Get Options

BIOS Settings Get Expect Options

BIOS Settings Get Example

BIOS Settings Set

BIOS Settings Set Options

BIOS Options

BIOS Settings Set Example

CPU

CPULoad

CPULoad Options

CPULoad Example

CPUSet

CPUSet Options

CPUSet Example

CPUStats

CPUStats Options

CPU Expect Options

General Object

Individual Object

CPUStats Example

Various

Binarly

Binarly Parameters

Binarly Example

ChipSec

ChipSec Options

ChipSec Example

FWHunt

FWHunt Options

FWHunt Example

FWTS