Benchmarking

System performance benchmarking and power state testing commands.

AMD S2Idle

Test AMD S2Idle (suspend-to-idle) functionality using the AMD debug tools. This command runs comprehensive suspend/resume cycle testing with detailed validation and reporting.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`amd-s2idle`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options
`parameters`	`AMDS2IdleOptions`	No		Options passed into the `amd-s2idle` tool

AMD S2Idle Options

Attribute	Type	Required	Default	Description
`count`	`integer`	No	`10`	Number of suspend/resume cycles to run
`duration`	`integer`	No	`10`	Duration of each suspend cycle in seconds
`wait`	`integer`	No	`4`	Time to wait before starting each cycle in seconds
`format`	`string`	No	`stdout`	Format of the report to produce. Options: `html`, `txt`, `md`, `stdout`
`report_file`	`string`	No		File path to write the report to (only applicable if format is `html`, `txt`, or `md`)
`force`	`boolean`	No	`false`	Run test cycle even if the system fails prerequisite checks
`random`	`boolean`	No	`false`	Run sleep cycles with random durations and waits (using `duration` and `wait` as upper bounds)
`logind`	`boolean`	No	`false`	Use systemd-logind to suspend the system
`tool_debug`	`boolean`	No	`false`	Enable debug logging for the tool
`bios_debug`	`boolean`	No	`false`	Enable BIOS/firmware debug logging

AMD S2Idle Examples

Basic Usage

cmd: amd-s2idle
name: Run AMD S2Idle test with defaults
transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: root

Custom Parameters

cmd: amd-s2idle
name: Run extended AMD S2Idle test
transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: root
options:
  timeout: 30m
parameters:
  count: 20
  duration: 15
  wait: 5
  format: html
  report_file: /tmp/s2idle-report.html

With Debug Logging

cmd: amd-s2idle
name: Run AMD S2Idle test with debug logging
transport:
  proto: ssh
  options:
    host: "[[attributes.Host]]"
    user: root
parameters:
  count: 5
  duration: 10
  tool_debug: true
  bios_debug: true

S0ix-Selftest

Test S0ix (suspend-to-idle) power state functionality.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`s0ix-selftest`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options

S0ix-Selftest Example

cmd: s0ix-selftest
name: Run s0ix selftest
transport: *transport

Expect Comparator Syntax

The fio, iperf3, and stress-ng steps assert measured metrics against comparator strings. Empty (or omitted) means “no assertion”.

Form	Meaning
`<N`	Measured value must be ≤ N
`>N`	Measured value must be ≥ N
`=N`	Measured value must equal N
`N-M`	Measured value must be within `[N, M]`

All comparisons are integer. Float metrics (bandwidth, latency, ops/sec) are truncated before comparison.

Fio

Run fio on the DUT and assert against parsed metrics. The fio binary must be installed on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`fio`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options (default timeout 2m)
`parameters`	`FioOptions`	Yes		Options passed into the `fio` cmd

Fio Options

Attribute	Type	Required	Default	Description
`args`	`[]string`	No	randread 4k 1G 30s job	Raw arguments passed to `fio`. If omitted, runs a default randread workload against `/tmp/contest-fio-<nano>.dat`. `--output-format=json` auto-added.
`expect`	`FioExpect`	No		Per-metric assertions. See Expect Comparator Syntax.

Fio Expect Options

Assertions run against the first job in fio’s JSON output.

Attribute	Type	Required	Description
`ReadBW`	`string`	No	Read bandwidth in KiB/s
`ReadIOPS`	`string`	No	Read IOPS
`ReadLatMean`	`string`	No	Mean read latency in nanoseconds
`WriteBW`	`string`	No	Write bandwidth in KiB/s
`WriteIOPS`	`string`	No	Write IOPS
`WriteLatMean`	`string`	No	Mean write latency in nanoseconds

Fio Example

cmd: fio
name: Random read bandwidth
transport: *transport
parameters:
  args:
    - --name=randread
    - --filename=/tmp/fio.dat
    - --rw=randread
    - --bs=4k
    - --size=512M
    - --runtime=20
    - --time_based
  expect:
    ReadIOPS: ">5000"
    ReadLatMean: "<200000"

Iperf3

Run iperf3 in client mode against a remote server and assert against the bandwidth summary. The iperf3 binary must be installed on the DUT, and an iperf3 -s peer must be reachable.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`iperf3`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options (default timeout 2m)
`parameters`	`Iperf3Options`	Yes		Options passed into the `iperf3` cmd

Iperf3 Options

Attribute	Type	Required	Default	Description
`args`	`[]string`	Yes		Raw arguments passed to `iperf3`. Must include `-c <host>` (client mode); server mode is out of scope. `-J` auto-added.
`expect`	`Iperf3Expect`	No		Per-metric assertions. See Expect Comparator Syntax.

Iperf3 Expect Options

Assertions run against end.sum_received (TCP) or end.sum (UDP).

Attribute	Type	Required	Description
`BitsPerSecond`	`string`	No	Throughput in bits per second
`Bytes`	`string`	No	Total bytes transferred
`Seconds`	`string`	No	Test duration in seconds
`Retransmits`	`string`	No	Retransmit count (TCP only; absent for UDP)

Iperf3 Example

cmd: iperf3
name: Throughput to lab server
transport: *transport
parameters:
  args: [-c, 10.0.0.5, -t, "10"]
  expect:
    BitsPerSecond: ">900000000"
    Retransmits: "<10"

Stress-NG

Run stress-ng on the DUT and assert against the --metrics-brief rows aggregated across all stressors. The stress-ng binary must be installed on the DUT.

Attribute	Type	Required	Default	Description
`cmd`	`const`	Yes	`stress-ng`	Command Name
`name`	`string`	Yes		Name to uniquely identify the teststep. This name will show up in reports, logs and the WebUI
`transport`	`TransportObject`	Yes		Additional transport options
`options`	`OptionsObject`	No		Additional command options (default timeout 2m)
`parameters`	`StressNGOptions`	Yes		Options passed into the `stress-ng` cmd

Stress-NG Options

Attribute	Type	Required	Default	Description
`args`	`[]string`	No	`--cpu 0 --timeout 30s --metrics-brief`	Raw arguments passed to `stress-ng`. If supplied, must include `--metrics-brief` so per-stressor metric rows are emitted.
`expect`	`StressNGExpect`	No		Per-metric assertions. See Expect Comparator Syntax.

Stress-NG Expect Options

Values are summed across every stressor row in the output.

Attribute	Type	Required	Description
`BogoOps`	`string`	No	Sum of bogo-ops across all stressors
`BogoOpsPerSecRealTime`	`string`	No	Sum of bogo-ops/sec (real-time column) across all stressors
`BogoOpsPerSecUsrSys`	`string`	No	Sum of bogo-ops/sec (usr+sys column) across all stressors

Stress-NG Example

cmd: stress-ng
name: CPU stress baseline
transport: *transport
parameters:
  args: [--cpu, "0", --timeout, 60s, --metrics-brief]
  expect:
    BogoOpsPerSecRealTime: ">1000"