The lcm_data cleanup erases the first 0x400 blocks (the ext4 filesystem
metadata) of the lcm_data partition in U-Boot, so on the next boot the
block-mount layer finds a bad superblock on /lcm (mmcblk0p16, a ~3 GiB
partition on OSPv2) and runs a full mke2fs before it can be mounted:
e2fsck: Bad magic number in super-block while trying to open /dev/mmcblk0p16
mount: mounting /dev/mmcblk0p16 on /lcm failed: Invalid argument
mke2fs 1.47.0 (5-Feb-2023)
Creating filesystem with 786432 4k blocks and 196608 inodes
That reformat is the whole point (it is what keeps /lcm clean between CI
runs), but it adds enough boot time to push the login prompt just past
the ShellDriver default login_timeout of 60s on the OSPv2 (mxl25641-hdk-6)
eMMC target, which has no override, so board bringup fails with:
pexpect.exceptions.TIMEOUT: Timeout of 60 seconds exceeded during waiting for login
Bump the OSPv2 login_timeout to 80s, which is enough headroom to absorb
the reformat on top of its usual boot while still failing bringup fast on
a genuinely dead board. The Freedom (wnc-freedom) eMMC target keeps its
own 120s, which reflects its longer/different boot time expectations
rather than the lcm_data reformat.
Fixes: 5432e4a864 ("ci: testbed: labgrid: restore lcm_data cleanup dropped by FEAT-29 flow")
Fixes: CI-2
References: PPW-718
Link: https://gitlab.com/prpl-foundation/prplos/prplos/-/jobs/15140103104#L697
Assisted-by: claude:opus-4-8:1m-context
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
The FEAT-29 update_prpl conversion switched both the Freedom (wnc-freedom)
and OSPv2 (mxl25641-hdk-6) eMMC flash flows to erase only the rootfs_data
overlay, and in doing so dropped the lcm_data partition cleanup that
PPW-718 had introduced. The persistent /lcm (lcm_data) partition is not
cleared between CI runs, so stale containers from a previous test survive
the reboot/flash procedure and make the LCM cram suite non-idempotent.
So lets restore the cleanup by erasing the first 0x400 blocks (the eMMC
erase-group size, i.e. just the ext4 filesystem metadata, not the whole
multi-GiB partition) of lcm_data right after the rootfs_data erase, so
every refresh starts from a clean /lcm. The offsets are taken from each
board's current prpl Standard Flash Layout GPT, verified live in U-Boot:
Freedom (IPQ9574#):
28 0x00134022 0x00534021 "lcm_data"
29 0x00534022 0x00574021 "rootfs_data"
OSPv2 (Lightning #):
14 0x00086c00 0x00096bff "rootfs_data"
16 0x000a6c00 0x006a6bff "lcm_data"
Fixes: CI-2
References: 78b11d2d5b ("ci: testbed: labgrid: flash OSPv2 and Freedom via FEAT-29 update_prpl")
References: PPW-718
Assisted-by: claude:opus-4-8:1m-context
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Nokia Valyrian board should be integrated into the testbeds, so lets add
labgrid support bits for initramfs based boot into testbed-01 for the
start.
References: PCF-2338
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Signed-off-by: Peter Steinhaeuser <ps@embedd.com>
Switch the OSPv2 (mxl25641-hdk-6) and Freedom (wnc-freedom) eMMC flash
flow to the FEAT-29 U-Boot update script, replacing the legacy OSPv2
update_fullimage path and the Freedom untar/flash sysupgrade path. This
exercises the image/update tooling introduced in prplos!1964.
The flow tftpboots the per-board update_script.itb, sources it and runs
update_prpl against the FEAT-29 OS artifacts:
- OSPv2: PRPL_OSP_v2-wgrtd159be_b_wav700-ext4.img (4G variant)
- Freedom: prpl_freedom-kernel.itb + prpl_freedom-rootfs.itb
update_rescue_bank is set to yes so both OS banks carry the same image
and a bank failover cannot boot a stale rootfs. U-Boot/RBE/XBL/SMD are
left unset: those are one-time migration inputs and must not be
reflashed on every refresh. Boards are expected to already be on the
prpl flash layout. Hand-off to Linux uses run bootcmd (bootprpl/bootipq)
instead of reset, which avoids a power cycle and the labgrid autoboot
re-interrupt hazard.
init_command_timeout bounds the whole init_commands sequence and is set
deliberately tight, per board, so a network/perf regression (e.g. image
TFTP throughput roughly halving) trips the timeout instead of silently
passing, while still absorbing TFTP first-packet jitter. Validated runs
executed the init_commands in:
- OSPv2: ~30s (update_script TFTP ~10s + update_prpl ~18s) -> 60s
- Freedom: ~37s (sleep 5 + update_script TFTP ~3.5s + update_prpl ~24s,
two images to both banks) -> 75s
The rootfs_data overlay is erased before update_prpl so every refresh
boots from a clean overlay; update_prpl itself leaves rootfs_data
untouched. The block offsets are hardcoded from each board's GPT because
the target U-Boot has no part command to derive them:
- OSPv2: mmc erase 0x86c00 0x10000 (32 MiB, UPDK 9.2.0)
- Freedom: mmc erase 0x534022 0x40000 (128 MiB)
Because update_prpl needs both the update script and the OS image over
TFTP, generalise the testbed/cdrouter firmware copy to a list via the
new TFTP_IMAGE_FILENAMES variable (falling back to TFTP_IMAGE_FILENAME),
rather than overloading TFTP_IMAGE_UNPACK_COMMAND. The Freedom testbed
job now drives the wnc-freedom-emmc target instead of initramfs, so the
now-unused wnc-freedom-initramfs labgrid target is dropped.
Flip the wnc-freedom and mxl25641-hdk-6 testbed runner tags to the -sfl
variant (dut-wnc-freedom-sfl, dut-mxl25641-hdk-6-b-sfl), to make it
clear, that those devices are converted to the prpl Standard Flash
Layout.
References: FEAT-29
Closes: PCF-2644, PCF-2645
Assisted-by: claude:opus-4-8:1m-context
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently all CDRouter USP protocol tests in prplOS use STOMP MTP
(100%). This no longer matches the FEAT-63 delivery plan and industry
adoption.
MQTTv5 should become the default MTP for the majority of CDRouter test
runs, while keeping WebSocket and STOMP coverage in some jobs as smaller
regression slices.
So lets switch the shared template to MQTTv5 by default, parameterising
the shipped CDRouter configs instead of cloning them, and adding guarded
DUT init helpers that stop obuspa, install the requested reset file,
clear /etc/obuspa.db, start obuspa again, and wait for the USP data
model to come back.
Use the existing job matrix to keep protocol coverage aligned with the
agreed distribution:
- Wireless smoke tests use WebSocket
- prplOS Network Test Plan Candidate jobs use WebSocket
- IPv6 Top-100+mDNS VLAN jobs use STOMP
- all remaining jobs use MQTTv5
- MQTTv3 stays out of scope
References: PCF-1323, PCF-2322
Assisted-by: codex:gpt-5.4:xhigh
Assisted-by: claude:opus-4-6:1m-context
Co-developed-by: Sergio Panseri <sergio.panseri@consult.red>
Co-developed-by: Piotr Nakraszewicz <piotr.nakraszewicz@consult.red>
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently all 16 CDRouter test packages are stored as opaque .gz
binary exports that cannot be reviewed, diffed, or maintained as
code. The .gz format carries implicit runtime fields (config_id,
device_id, schedule, agent_id) that do not belong in version
control.
So lets add unversioned YAML manifests for all 16 packages
converted from their .gz source files using
cdrouter-package-convert.py, and remove all .gz binary artifacts
from the repository. Each manifest contains name, description,
and tests in original order. Options use the compact form where
only non-default overrides are present (2 of 16 packages have
explicit options; the remaining 14 use CDRouter defaults
entirely).
All 16 packages passed round-trip fidelity verification:
CDRouter-IPv6-Top-100, CDRouter-IPv6-Top-100+mDNS,
CDRouter-IPv6-Top-100+mDNS+dslite, CDRouter-Top-100,
CDRouter-Top-100+mDNS, CDRouter-Top-100+mDNS-OSP,
CDRouter-TR-069, CDRouter-UPnP-IPv46,
CDRouter-UPnP-IPv46-Failing, CDRouter-USP-prpl,
CDRouter-Wireless, CDRouter-Wireless-OSPv1,
prpl-Certification, prplWare-Smoke-Test-Suite,
prplOS-Network-Test-Plan-Candidate-v1.2.0,
prplOS-Network-Test-Plan-Mandatory-v1.2.0
References: PCF-2533
Assisted-by: codex:gpt-5.4:xhigh
Assisted-by: claude:opus-4-6:1m-context
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently CDRouter test packages are stored as opaque .gz binary
exports in .testbed/cdrouter/packages/. These binary blobs are not
human-readable, not diffable, and carry implicit fields that leak
runtime state into the repository.
So lets add a converter script that reads the two-line JSON format
inside .gz exports and emits clean unversioned YAML manifests. The
converter extracts name, description, testlist (as tests), and
options from each .gz file. By default it compacts the options map
by omitting keys matching known CDRouter defaults, keeping only
semantic overrides. The --preserve-all-options flag emits the full
source options map when needed.
Fidelity verification expands both source and generated options
with known defaults before comparison, so compact manifests with
omitted defaults still pass. Testlist order is compared exactly.
The converter rejects packages where the internal name does not
match the .gz filename.
CLI supports --input-dir, --output-dir, --filter, --dry-run,
--overwrite, --preserve-all-options, and --verbose flags. Exits
non-zero if any package fails fidelity.
References: PCF-2531, PCF-2535
Assisted-by: codex:gpt-5.4:xhigh
Assisted-by: claude:opus-4-6:1m-context
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Changes in !1920 (c66bc572) introduced issues with longer boot times.
CI tests failed because the board was not ready within the 80 second
timeout.
pexpect.exceptions.TIMEOUT: Timeout of 80 seconds exceeded during waiting for login
Increase login_timeout by 40 seconds (from 80s to 120s) for both
wnc-freedom-initramfs and wnc-freedom-emmc targets to accommodate
the longer boot times.
References: PCF-2346
Link: https://gitlab.com/prpl-foundation/prplos/prplos/-/jobs/12946876755#L73
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently testbed-01 labgrid still controls power via the SSH-based
ExternalPowerDriver, which became unreliable around the time we replaced
the faulty relay board and can leave multiple DUTs powered at the same
time.
So lets update the testbed-01 labgrid config to use the new Papouch
Quido relay board via Modbus/TCP, following the pattern already used in
testbed-02.
Closes: PCF-2361
References: PCF-2003
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Update references to OSPv2 image names in YAML configurations to match
the new naming convention, which was introduced in UPDK 9.2.3, providing
single build for both A-Step and B-Step OSPv2 hardware revisions.
New A-Step: prplos-intel_x86-lgm-PRPL_OSP_v2-tb341_wav700_fullimage.fit
New B-Step: prplos-intel_x86-lgm-PRPL_OSP_v2-wgrtd159be_b_wav700_fullimage.fit
References: PCF-2202
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
OSPv1 board was removed quite some time ago and replaced with OSPv2
successor, so lets cleanup the CI bits a bit to keep the tree tidy.
References: PCF-1134
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
prplOS Network Functionality Test Plan v1.2.0 added following prplOS
4.1.0 mandatory tests which were enabled in Mandatory package and
removed from Candidate package.
* 1.2.10
* 1.2.25
* 1.2.27
* 1.2.28
* 1.3.7
* 1.4.2
* 1.4.3
* 1.4.4
* 1.5.4
* 1.5.5
* 1.5.6
* 1.5.7
* 1.5.14
* 1.5.15
* 1.5.16
* 1.5.17
Test plan packages v1.1.0 are being removed as they're not being used
anymore.
prplOS Network Test Plan timeout was increased to 1h 45min as the test
runtime increased to 1h 40min, 5 min is kept as buffer. The prplOS
Network Test Plan v1.2.0 mandatory tests require more time to complete
than the previous 70 minute timeout allowed. Candidate tests are
shorter, approx. 47 minutes on average, so having 52 minute timeout.
Closes: PCF-2275
Link: https://prplfoundation.org/wp-content/uploads/prplOS-Network-Functionality-Test-Plan-v1.2.0.pdf
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We've upgraded CDRouter hardware from legacy NTA-1000 hardware platform
to the new NTA-3000 hardware platform.
Some of the network interface names changed, so we need to update
network interface names of the NTA1000 to match the corresponding
interface on the new NTA3000 platform.
References: PCF-2267
Link: https://support.qacafe.com/cdrouter/hardware/cdrouter-backup-and-restore-tools/#nta1000---nta3000-interface-mapping-table
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Since OSPv2 B-Step devices are de-facto a standard devices currently
being sold, it was decided to start using them for testing in prplOS CI
instead of current OSPv2 A-Step boards.
References: PCF-2200
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
It seems, that recent changes in init scripts are causing different
behavior of serial console, thus the current timeout of 75s for login
prompt detection is not enough anymore:
File "/usr/local/lib/python3.7/dist-packages/labgrid-0.4.1.dev239+g1ca30de-py3.7.egg/labgrid/driver/shelldriver.py", line 210, in _await_login
raise TIMEOUT(f"Timeout of {self.login_timeout} seconds exceeded during waiting for login") # pylint: disable=line-too-long
pexpect.exceptions.TIMEOUT: Timeout of 75 seconds exceeded during waiting for login
Currently it needs 77s:
expect state='stop', result=(0, b"crond[5305]: crond (busybox 1.36.1) started, log level 5\r\ncrond[5305]: ignoring file 'logrotate' (no such user)\r\n\r\n", <re.Match object; span=(115, 130), match=b'root@prplOS:~# '>, b'root@prplOS:~# '), duration=0.004
_await_login state='stop', duration=77.550
So lets increase the timeout to 80s.
References: PCF-1990, FEAT-67
Link: https://gitlab.com/prpl-foundation/prplos/prplos/-/jobs/11649231947
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
There is now dedicated LCM data partition mounted under /lcm which is
not cleared up after each test, thus containers used in previous tests
persist during the DUT reboot and flashing procedure, violating test
imdepotence and making consecutive test fail.
So lets fix it by clearing up the LCM data partition during DUT firmware
flashing procedure, clearing just ext4 filesystem metadata, not erasing
complete 3 GiB and saving some time.
16 0x000a6c00 0x006a6bff "lcm_data"
attrs: 0x0000000000000000
type: ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
guid: 777a7bb0-4d48-4b39-8cc1-da40d9f28205
The 0xa6c00 is the starting eMMC block of that partition and 0x400 is
eMMC erase group size, thus minimal size we can erase.
Reported-by: Matthias Franck <matthias.franck@softathome.com>
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We need to make sure, that prplWare passes prplOS Network Test Plan
certification test suite. We've added separate test package with
Mandatory test cases, but we should be able to test as well the
remaining, currently Optional or Privisionary Excluded test cases, so we
get the test results and work towards making all of them Mandatory.
The test package contains only mandatory tests from the test plan, so 64
tests:
prplos.1.1.1, prplos.1.1.2, prplos.1.1.8, prplos.1.1.9, prplos.1.1.10,
prplos.1.1.11, prplos.1.1.12, prplos.1.1.13, prplos.1.2.10,
prplos.1.2.20, prplos.1.2.21, prplos.1.2.22, prplos.1.2.23,
prplos.1.2.24, prplos.1.2.25, prplos.1.2.27, prplos.1.2.28,
prplos.1.2.30, prplos.1.3.4, prplos.1.3.5, prplos.1.3.6, prplos.1.3.7,
prplos.1.3.9, prplos.1.3.11, prplos.1.3.14, prplos.1.3.15,
prplos.1.3.16, prplos.1.3.18, prplos.1.3.19, prplos.1.3.20,
prplos.1.3.21, prplos.1.3.22, prplos.1.3.23, prplos.1.3.24,
prplos.1.3.25, prplos.1.3.26, prplos.1.3.27, prplos.1.4.2, prplos.1.4.3,
prplos.1.4.4, prplos.1.5.4, prplos.1.5.5, prplos.1.5.6, prplos.1.5.7,
prplos.1.5.14, prplos.1.5.15, prplos.1.5.16, prplos.1.5.17,
prplos.1.5.18, prplos.1.5.19, prplos.1.6.1, prplos.1.6.2, prplos.1.6.6,
prplos.1.6.7, prplos.1.6.8, prplos.1.6.13, prplos.1.6.15, prplos.1.6.19,
prplos.1.6.24, prplos.1.6.25, prplos.1.6.26, prplos.1.6.27,
prplos.1.6.28, prplos.1.6.29
References: PCF-1609
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently we're seeing prplOS.1.4.9 test failure which is happening due
to disabled config options:
* ipv6WanRDNSS - This option enables or disables the IPv6 RDNSS option
for router advertisements on the WAN. When set to yes, CDRouter will
announce the configured IPv6 DNS servers using the RDNSS option in
router advertisements.
* ipv6WanDNSSL - This option enables or disables the IPv6 DNSSL option
for router advertisements on the WAN. When set to yes, CDRouter will
announce the domain name, as configured by the testvar wanDomainName,
using the DNSSL option in router advertisements.
So lets fix it by enabling those config options.
References: PPW-362
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We need to make sure, that prplWare passes prplOS Network Test Plan
certification test suite and for that we need to configure CDRouter and
DUT.
CDRouter config changes in new prplos-certification config:
* ipv6WanMode was changed from autoconf to DHCP
* ipv6LanMode was changed from autoconf to DHCP
DUTs are being dynamically configured with following datamodel
configuration:
* DHCPv6Client.Client.wan.RequestAddresses=1 so DUT requests IA_NA
* RouterAdvertisement.InterfaceSetting.lan.AdvManagedFlag=1 so CDRouter
can start DHCPv6 client
- WARNING(lan): The Mbit in the last router advertisement is set to 0 indicating that DHCPv6 is not allowed
- WARNING(lan): Cannot start DHCPv6 client
References: PCF-1609
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We need to create a new configuration for prplOS Certification testing,
so lets start from generic-ipv6 config, so the CDRouter configuration
changes needed for prplOS Certification tests are visible in the next
Git commit (diff).
References: PCF-1609
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We need to make sure, that prplWare passes prplOS Network Test Plan
certification test suite. So lets add a new `certification tests` stage
with a new prplOS-Network-Test-Plan-Mandatory-v1.1.0 CI test job which
should help us QA this.
The test package contains only mandatory tests from the test plan, so 63
tests:
prplos.1.1.3, prplos.1.1.4, prplos.1.1.5, prplos.1.1.6, prplos.1.1.7,
prplos.1.2.1, prplos.1.2.2, prplos.1.2.3, prplos.1.2.4, prplos.1.2.5,
prplos.1.2.6, prplos.1.2.7, prplos.1.2.8, prplos.1.2.9, prplos.1.2.11,
prplos.1.2.12, prplos.1.2.13, prplos.1.2.14, prplos.1.2.15,
prplos.1.2.16, prplos.1.2.17, prplos.1.2.18, prplos.1.2.19,
prplos.1.2.26, prplos.1.2.29, prplos.1.3.1, prplos.1.3.2, prplos.1.3.3,
prplos.1.3.8, prplos.1.3.10, prplos.1.3.12, prplos.1.3.13,
prplos.1.3.17, prplos.1.4.1, prplos.1.4.5, prplos.1.4.6, prplos.1.4.7,
prplos.1.4.8, prplos.1.4.9, prplos.1.5.1, prplos.1.5.2, prplos.1.5.3,
prplos.1.5.8, prplos.1.5.9, prplos.1.5.10, prplos.1.5.11, prplos.1.5.12,
prplos.1.5.13, prplos.1.6.3, prplos.1.6.4, prplos.1.6.5, prplos.1.6.9,
prplos.1.6.10, prplos.1.6.11, prplos.1.6.12, prplos.1.6.14,
prplos.1.6.16, prplos.1.6.17, prplos.1.6.18, prplos.1.6.20,
prplos.1.6.21, prplos.1.6.22, prplos.1.6.23
References: PCF-1609
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
In CDRouter-UPnP-IPv46 we've now all passing UPnP tests from both upnp
and upnp-v6 modules, which are known to be working as of today.
From the list of available 111 tests in CDRouter version 15.8.1 in both
upnp-v6.tcl (60) and upnp.tcl(51) modules we've removed following 30
tests currently known to be not failing:
cdrouter_upnp_203, cdrouter_upnp_204, cdrouter_upnp_220,
cdrouter_upnp_igd2_203, cdrouter_upnp_igd2_204, cdrouter_upnp_igd2_220,
ipv6_upnp_35, ipv6_upnp_36, ipv6_upnp_40, ipv6_upnp_41, ipv6_upnp_45,
ipv6_upnp_50, ipv6_upnp_100, ipv6_upnp_203, ipv6_upnp_204,
ipv6_upnp_220, ipv6_upnp_igd2_20, ipv6_upnp_igd2_25, ipv6_upnp_igd2_35,
ipv6_upnp_igd2_36, ipv6_upnp_igd2_40, ipv6_upnp_igd2_41,
ipv6_upnp_igd2_45, ipv6_upnp_igd2_50, ipv6_upnp_igd2_100, ipv6_upnp_igd2_200,
ipv6_upnp_igd2_203, ipv6_upnp_igd2_204, ipv6_upnp_igd2_220,
ipv6_upnp_igd2_307
So lets introduce a new CDRouter-UPnP-IPv46-Failing package, with the
above mentioned failing tests (29) which would allow us tracking the
potentially remaining bits which needs to be fixed.
Link: https://support.qacafe.com/cdrouter/knowledge-base/upnp-testing-guide
References: PCF-720, PPW-336, FEAT-85, FEAT-244
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We're currently using just a subset of UPnP test cases as being
available in CDRouter via PCF-720.
We need to increase the QA of UPnP, so lets add complete upnp and
upnp-v6 modules, which are known to be working as of today. From the
list of available 111 tests in CDRouter version 15.8.1 in both upnp-v6.tcl
(60) and upnp.tcl(51) modules we're removing following 30 tests currently known
to be not passing:
cdrouter_upnp_203, cdrouter_upnp_204, cdrouter_upnp_220,
cdrouter_upnp_igd2_203, cdrouter_upnp_igd2_204, cdrouter_upnp_igd2_220,
ipv6_upnp_35, ipv6_upnp_36, ipv6_upnp_40, ipv6_upnp_41, ipv6_upnp_45,
ipv6_upnp_50, ipv6_upnp_100, ipv6_upnp_203, ipv6_upnp_204,
ipv6_upnp_220, ipv6_upnp_igd2_20, ipv6_upnp_igd2_25, ipv6_upnp_igd2_35,
ipv6_upnp_igd2_36, ipv6_upnp_igd2_40, ipv6_upnp_igd2_41,
ipv6_upnp_igd2_45, ipv6_upnp_igd2_50, ipv6_upnp_igd2_200, ipv6_upnp_igd2_203,
ipv6_upnp_igd2_204, ipv6_upnp_igd2_220, ipv6_upnp_igd2_307
So we can regression test UPnP stack in current state and thus improve the QA.
Cap the CI job runtime to 60 minutes as currently it seems, that it
takes around 53 minutes to complete.
References: PCF-720, PPW-336, FEAT-85, FEAT-244
Link: https://support.qacafe.com/cdrouter/knowledge-base/upnp-testing-guide
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Co-authored-by: Nikolai Merinov <n.merinov@inango-systems.com>
We now have proper TLS support, configuration and certificates in the
system and tr069-manager, so lets switch all CDRouter tests to TR-609
over TLS v1.3.
References: PCF-2021
Signed-off-by: Sergio Panseri <sergio.panseri@consult.red>
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org> [tls1.3, certs, all configs]
Support for 24.10 based prplOS comes with UPDK 9.2.0/9.1.95 which brings
as well prpl Standard Flash Layout, so lets adapt the flashing
procedure using new bootloader and FIT image.
References: PCF-1832
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
To QA instance and search expression based permissions we're adding
following USP conformance test cases into smoke test suite (adding ~13s
of additional test runtime):
* usp_conformance_2_10 - Adding a Role
* usp_conformance_2_11 - Permissions - Object Creation Allowed
* usp_conformance_2_12 - Permissions - Object Creation Not Allowed
* usp_conformance_2_13 - Permissions - Object Deletion Allowed
* usp_conformance_2_14 - Permissions - Object Deletion Not Allowed
* usp_conformance_2_15 - Permissions - Parameter Update Allowed
* usp_conformance_2_16 - Permissions - Parameter Update Not Allowed
* usp_conformance_2_18 - Permissions - Operation Not Allowed
* usp_conformance_2_19 - Permissions - Value Change Notification Allowed on Parameter
* usp_conformance_2_20 - Permissions - Value Change Notification Not Allowed on Parameter
* usp_conformance_2_21 - Permissions - Overlapping Permissions
* usp_conformance_2_22 - Using Get when no read permissions are available on some parameters
* usp_conformance_2_25 - Permissions - Parameter within added object not allowed, omitted
* usp_conformance_2_26 - Permissions - Parameter within added object not allowed, included
References: FEAT-199
Tests: PCF-1748, PCF-1749
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
New MediaTek/Arcadyan Mozart regular track reference platform is being
introduced, so lets enable the runtime testing using cram/CDRouter tests
on one Mozart device currently plugged into testbed-02.
References: PCF-1460, PCF-1461
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
After changes done in commit cd0b15563161 ("profiles: prpl:
tr069-manager: bump to v1.35.5"), we've started seeing increased number
of CDRouter test failures related to TR-069 testing:
FAIL: Did not receive TR-069 Inform RPC from CPE after 120 seconds » no-wan-tr069-inform
FAIL: Did not receive TR-069 Inform RPC from CPE after 240 seconds » no-wan-tr069-inform
Until this gets fixed, lets increase the default acsWaitForInform timeout
from 120 seconds to 480 seconds, which hopefully should provide enough
room for cwmpd to connect with ACS.
References: PPW-544
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently the build for Haze produces images for ath79 target (default).
This is happening, because there is no ipq807x target in the tree
anymore, because upstream in commit f02f6aaa8d ("ipq807x: rename
target to qualcommax") renamed the target for following reasons:
Currently, ipq807x only covers Qualcomm IPQ807x SoC-s.
However, Qualcomm also has IPQ60xx and IPQ50xx SoC-s under the AX WiSoC-s
and they share a lot of stuff with IPQ807x, especially IPQ60xx so to avoid
duplicating kernel patches and everything lets make a common target with
per SoC subtargets.
Start doing that by renaming ipq807x to qualcommax so that dependencies
on ipq807x target can be updated.
So lets fix the profile using the new target/subtarget structure.
Fixes: PCF-1605
References: PCF-1574
References: https://github.com/openwrt/openwrt/commit/f02f6aaa8d4e
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently we've only "prplOS Test Plan for prpl devices" included in the
prpl-Certification package, but we need as well "prpl High-Level API
Tests" (USP) and "prpl High-Level API Tests for CWMP" as we need to make
sure, that prplWare is usable via both USP and CWMP protocols.
So lets add those currently missing test cases:
* prpl High-Level API Tests (USP)
* prpl_hl-api.1.1 - Profile GetSupportedDM
* prpl_hl-api.1.2 - Profile Parameters Write
* prpl_hl-api.1.3 - Get
* prpl_hl-api.1.4 - Set
* prpl_hl-api.1.5 - Add and Delete
* prpl High-Level API Tests for CWMP
* prpl_hl-api-cwmp.2.1 - Profile GetParameterNames
* prpl_hl-api-cwmp.2.2 - Profile Parameters Write
* prpl_hl-api-cwmp.2.3 - Get
* prpl_hl-api-cwmp.2.4 - Set
* prpl_hl-api-cwmp.2.5 - Add and Delete
References: PCF-1611
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
CDRouter offers a dedicated prpl Certification expansion for the prpl
Foundation’s prplWare and High-Level API Certification program.
As an officially approved testing tool, CDRouter is an authorized test
tool for use in conducting the prplWare and High-Level API self-testing
certification.
prplOS should be routinely tested using the CDRouter prpl Certification
expansion, so lets add the prpl Certification CI jobs for all devices
and new prpl Certification package.
Closes: PCF-1611
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
CDRouter offers a dedicated prpl Certification expansion for the prpl
Foundation’s prplWare and High-Level API Certification program.
As an officially approved testing tool, CDRouter is an authorized test
tool for use in conducting the prplWare and High-Level API self-testing
certification.
prplOS should be routinely tested using the CDRouter prpl Certification
expansion, so lets add the needed configuration:
prplosLanInterfaceAlias - This parameter specifies the LAN interface
Alias used to verify bridge functions on the prplOS implementation. This
parameter must be set to the value of Device.Bridging.Bridge.{i}.Port.{i}.Alias
that corresponds to the primary LAN test interface used for prplOS
Certification testing.
useSameLanInterface - If this testvar is set to no, CDRouter will
change LAN interfaces each time a test is run when multiple LAN
interfaces are configured. You can disable this behavior by configuring
the testvar useSameLanInterface to yes.
We set this value to `yes`, because we're only utilizing one LAN port.
References: PCF-1611
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
usp_conformance_1_20 has a known issue in test implementation, caused by
an erroneous change to the USP Spec. So until its fixed, lets remove
the test.
References: PPW-363, PCF-1612
References: https://issues.broadband-forum.org/browse/USP-917
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Current CI testing pipeline is very comprehensive and unusable for
testing of more granular deliveries and thus a new prplWare Smoke Test
Suite is needed.
The idea behind this new prplWare Smoke Test Suite is, that the complete
test runtime should fit into the range of 60-90 minutes.
The new Smoke Test Suite should contain testing a bit of everything, so
we cover most of the areas, while being able to finish a test within a
reasonable timeframe.
Thus this new test suite combines the current:
* CDRouter TOP 100 IPv4 and IPv6 packages, minus:
- the longest running TCP/UDP firewall port scan tests:
* ipv6_firewall_100
* ipv6_firewall_101
* cdrouter_firewall_100
* cdrouter_firewall_101
* cdrouter_firewall_110
- the failing tests on OSP boards:
* cdrouter_nat_201 tracked in PCF-972
* cdrouter_l2tppt_2 tracked in PCF-971
* CDRouter TR-069 package, minus the reboot test od128_test_14.1
* CDRouter USP certification test cases + USP basic test + prpl HL-API
tests (minus the problematic and currently always failing prpl_hl-api.1.4 Set test)
* cdrouter_dyndns_1 - DynDNS client sends an update request when the WAN IP address changes
* cdrouter_dyndns_2 - DynDNS client does not update if the WAN reestablishes with the same IP address
So the package contains in total 310 CDRouter test cases.
Closes: PCF-1645
References: PCF-971, PCF-972
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
There is currently a lot of old and already very obsolete stuff in the
tree for the past devices like glinet-b1300, urx851-b0-dk, nec-wx3000hp.
So lets cleanup all those unused bits to make the tree more tidy and
easier to maintain.
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We're going to use single smoke-test config to test bit of everything in
single test suite, so lets sync the DynDNS config from the generic
config.
References: PCF-1645
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
We're going to use single config for both IPv4 and IPv6 testing, so lets
sync the dhcpLeaseTime of 180 seconds from generic config, currently
being used for IPv4 based tests only.
References: PCF-1645
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Copy over generic-ipv6 configuration to smoke-test, so later changes are
clearly visible.
References: PCF-1645
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
The main purpose of this upstep is the integration of the Wifi7/MLO
feature.
Additional updates :
Feeds :
- Update libswlc to v5.30.2
CRAM tests :
- obuspa.t/ubus-datamodel-list.t : add new objects introduced by the
pwhm v7.x upstep
- 100-pwhm.t : update script to test MLO main link interface status
- 101-mlo.t : new script to test MLO deactivation
CDRouter updates (tri-band MLO prequesties) :
- Security mode is now set to WPA3-WPA2-Personal
- Add SAE in the expected key management mode list.
- Remove 'be' from the expected phy mode list on OSPv2. As it shouldn't
be broadcasted if MLO is not supported or when the access point is not
part of an MLD.
PWHM reference commit : 0627886f993c25b0de4ee628d9d07e7f67f01008
Closes: PPM-2984
References: PPW-228
Signed-off-by: Houssem Dafdouf <houssem.dafdouf_ext@softathome.com>
We have been running CDRouter wireless tests on the 2.4GHz band, using default
settings except for changing the SSID to prplOS-2g-6 and channel to 6. In multiple
test loops, the majority of observed failures occur specifically in CDRouter’s
wifi_10 test when running on the OSPv1 board. Removing the wifi_10 test, however,
leads to near 100% passing results, indicating that the issue is specifically tied
to how the AP handles the 4-way EAPOL handshake under wifi_10.
Packet captures and hostapd logs suggest the AP resets its handshake state machine
prematurely, returning to PTKSTART and eventually causing deauthentication (reason code 15).
This breaks the test procedure and leads to a high failure rate on OSPv1.
Analysis shows that other boards (e.g., MXL25641-HDK-6, Haze, Freedom) do not exhibit
the same level of instability. For OSPv1 (URX851-HDK-3), the failure rate can reach about
40-78% during wifi_10 runs. Removing wifi_10 reduces failures drastically.
Until the root cause in the EAPOL 4-way handshake process is identified and resolved,
we will remove the wifi_10 test from the test suite. Doing so mitigates the frequent
failures and allows the rest of the suite to execute reliably.
References: PCF-1586
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Three issues were identified with CDRouter wireless tests:
1. The wireless configuration handling changed, making SSIDs no longer
available at fixed datamodel indexes.
2. The default Channel=1 frequency spectrum changed in the testbed environment,
causing increased interference with other devices.
3. Both testbeds and DUTs in them were using identical default settings,
resulting in crosstalk between them when CDRouter Wireless tests were
run on both testbeds at the same time.
So lets fix it by moving each testbed into different channel and properly
configuring the wireless settings for each testbed and DUT corresponding
to the testbed:
- testbed-01: RegulatoryDomain=CZ, Channel=6, SSID=prplOS-2g-6
- testbed-02: RegulatoryDomain=CZ, Channel=12, SSID=prplOS-2g-12
While at it, switch from ubus-cli to ba-cli, which is now preferred due
to its bus agnostic nature, thus making the commands more future proof.
Closes: PPM-3069
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
This reverts commit e19787c7e531ce7ce29fd3ae681a94bfa38ec306 as the
issues should be fixed and thus the workaround shouldn't be needed.
References: PCF-1168
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
This reverts commit 65b6e4b990993b7e760acff7d309c67744f16f2b as the
issues should be fixed and thus the workaround shouldn't be needed.
References: PCF-1252
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
With the new wnc-freedom wifi7 box, the wifiBeaconPhy testvar of CDRouter needs to include "be" as well. Let's make this configurable in the CI by introducing a CDROUTER_CONFIG_WIFI_BEACON_PHY variable that can be set uniquely for every box.
Signed-off-by: Matthias FRANCK <matthias.franck@softathome.com>
