Module key

The key for a benchmarking run–all parameters modifying the output

The representation of the parameters (given or recorded) for a benchmarking run. This does not necessarily include all recorded metainformation. The aim is to allow to automatically group runs for which a sensible summary can be calculated, and to allow to specify sets of runs for which the change should be calculated. Maybe be fine-grained enough to e.g. allow to include runs from different hosts if their CPU and memory configuration is identical. And maybe provide for the means to allow for manual overrides, to include all runs in a summary with keys “close enough”.

Some parameters, e.g. hostname, may be irrelevant when the hardware and software versions are given; or it may turn out controlling for those is not enough; thus, some key pieces are redundant, or not?

Time-of-day may be relevant (rather: were other processes shut down or not), strongly or weakly, but can’t be part of the key or grouping would not work; this is a piece of information to track separately for verification.

Custom parameters can be given and be relevant, e.g. whether providing input data to an application sorted or not.

Structs

BenchmarkingJobParameters

Only the parts of a BenchmarkingJob that determine results–but also excluding schedule_condition, which does have a conscious influence on results, but comes from the configured pipeline, not the insertion. This here is used for insertion uniqueness checking, but maybe also for key determination lager (XX todo).

BenchmarkingJobParametersHash

CustomParameters

Custom key/value pairings, passed on as environment variables when executing the benchmarking runner of the target project. These are checked for allowed and required values.

EarlyContext

As determined by evobench (but should compare to duplicates of some of the fields in the bench log file resulting from a run)

Host

HostInfo

Information that together should allow a host to be determined to be equivalent.

Key

LateContext

As output by the benchmark runner of the target project (currently always the evobench-probes library)

OsInfo

RunParameters

UncheckedCustomParameters

Same as CustomParameters but not checked against a config. Keys are checked for basic correctness but not whether usable for a particular benchmark, and the values are of unknown type.

Traits

ExtendPath