5badc8a7d2
In commit 0c1a216ce82b ("ci: define time limits for runtime testing job
limits"), we've enabled time limits for runtime testing jobs to prevent
problematic test jobs from occupying precious testbed resources.
It turned out that the initial approach with the CI job limit wasn't
ideal, as it led to the immediate termination of CI jobs, preventing the
collection of useful artifacts like logs, thus making post-mortem
debugging of problematic tests impossible.
This commit introduces TEST_CUTOFF_TIMEOUT to limit test execution while
preserving time for artifact collection. The timeout is computed as:
CI_JOB_TIMEOUT - DUT_SLEEP_AFTER_BOOT - TEST_SETUP_TEARDOWN_DURATION
where TEST_SETUP_TEARDOWN_DURATION (6 minutes) reserves time for DUT
setup and artifact collection.
Two timeout mechanisms are implemented:
- CDRouter tests: custom timeout in testbed-cdrouter.py that monitors
execution time and calls package_stop() for graceful termination when
the deadline is reached, exiting with code 124 (used by `timeout`
command).
- Testbed tests: The `timeout` command wraps test execution, sending a
TERM signal at the deadline and a KILL signal after 10 seconds if
needed.
Both approaches assure that artifact collection can proceed after test
execution is terminated, enabling debugging of timeout issues.
While at it, I've removed SimpleNamespace import in testbed-cdrouter.py
as reported by flake8.
References: PCF-2157
Link: https://gitlab.com/prpl-foundation/prplos/prplos/-/jobs/11818939729#L136
Link: https://gitlab.com/prpl-foundation/prplos/prplos/-/jobs/11882363749#L238
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
|
||
|---|---|---|
| .. | ||
| build.d | ||
| dut-init | ||
| prplMesh | ||
| cdrouter-pretty-failures.awk | ||
| prpl-jira.py | ||
| README.md | ||
| requirements.txt | ||
| testbed-cdrouter.py | ||
| testbed-device.py | ||
Content
This directory contains files used mainly during testing on the GitLab CI.
requirements.txt
This requirements.txt file is used for specifying what Python packages are required to run the scripts in this directory. Usually used with pip install -f requirements.txt command.
prpl-jira.py
This helper Python script creates new or updates existing Jira ticket when something goes wrong during GitLab CI testing process. Currently only supports reporting of build test failures.
Needs Python version 3.6+ due to use of f-strings feature.
testbed-device.py
This helper Python scripts allows managing of device under test (DUT) via labgrid framework.