Add initial cram tests for tr181-wireguard, including an end-to-end tunnel
test: the remote peer lives in a network namespace behind a veth pair
emulating the WAN link and is configured with wireguard-tools, so the
test is self-contained and is not tied to a specific board. The
end-to-end test drives the CPE side through the data
model (WireGuard.Tunnel/Peer + IP.Interface + Firewall.Service), verifies
the handshake on the peer, ping in both directions, a short iperf3 session
and the NetModel flags.
Implements: PCF-2023
References: FEAT-150
Signed-off-by: Patrick Havelange <patrick.havelange_ext@softathome.com>
Signed-off-by: Alexandre Fiset <alexandre.fiset@softathome.com>
Add CRAM tests for tr181-gre, including an end-to-end tunnel test: the
remote peer lives in a network namespace behind a veth pair emulating
the WAN link, so the test is self-contained and is not tied to a
specific board. The end-to-end test drives the tunnel
through the data model (GRE.Tunnel + Interface + IP.Interface), verifies
the kernel route, reply behaviour with and without a peer return route,
the UseSequenceNumber/UseChecksum toggles and runs a short iperf3 session
through the tunnel.
References: PCF-2294, FEAT-150
Signed-off-by: Khaled Hasni <khaled.hasni_ext@softathome.com>
Signed-off-by: Alexandre Fiset <alexandre.fiset@softathome.com>
Cram supplies each generated test script to the shell through standard
input. The shared SSH command inherits that input and consumes the
remaining script, leaving the shell at EOF inside a quoted command.
Commit 5fd42aa540 ("ci: cram: use legacy SCP protocol for remote
copies") rewrote CRAM_REMOTE_COMMAND while adding legacy SCP support and
dropped the -n option added for PCF-2222. Job 15640024463 then failed
130-periodicfileupload.t with:
/bin/sh: 152: Syntax error: Unterminated quoted string (no-eol)
Job 15640024462 leaked this salt marker in the same test:
CRAM1 (no-eol)
Restore -n so SSH reads from /dev/null instead of the generated test
script.
Assisted-by: codex:gpt-5.6-sol
Assisted-by: claude:opus-5:1m-context
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Fixes: 5fd42aa540 ("ci: cram: use legacy SCP protocol for remote copies")
References: PCF-2222
Quote expanded command substitutions in remote logger commands
to ensure the expanded value is passed as a single literal
argument and prevent remote shell interpretation of special
characters.
Closes: PPW-1315
Signed-off-by: Houssem Dafdouf <houssem.dafdouf_ext@softathome.com>
Currently, the 036-lcm-usp-restrictions test fails during the
TrustedProvider model validation; this happened because
TrustedProvider required some time (up to 5 seconds) to register its model:
da53b18a65/src/main.c (L159)
New version of TrustedProvider service registers data model over ODL and
amxrt mechanism and does not contain the registration timeouts.
Fixes: PPW-2037
References: FEAT-28, FEAT-120
Signed-off-by: Mykola Hlukhenkyi <m.hlukhenkyi@inango-systems.com>
Signed-off-by: Artem Trokhymchuk <a.trokhymchuk@inango-systems.com>
Debian 13 ships pyOpenSSL without the deprecated X509Req API used by
servefile certificate generation.
Generate a short-lived certificate with openssl and pass it to both
HTTPS server instances.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PPW-1258
The Debian 13 testbed image runs cram as an unprivileged user, so its
ping process cannot open a raw socket with the executor capabilities.
Use the existing passwordless sudo setup for the host-side probe.
GitLab runner hostnames are not present in the container hosts file,
so sudo also emits a lookup warning before the asserted ping result.
Discard stderr while retaining the ping and grep exit behavior.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PPW-1258
The separately released testbed image can update MiniDLNA without
changing the behavior under test. Matching one package version makes
the DUT discovery test fail whenever that host-side fixture changes.
Keep the server description fields exact, but match the MiniDLNA
version with a cram glob. This verifies that the DUT discovers the
fixture without coupling the expectation to a Debian package version.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PPW-1258
OpenSSH in the Debian 13 testbed container uses SFTP for scp transfers
by default, while the DUT images do not provide an SFTP server.
Force the legacy SCP protocol in the shared cram copy command, the
security certificate fixture, and the remaining direct scp calls so
every test can stage its helper files and packages.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Fixes: 854ef0eafb ("ci: cram: add new LCM tests")
References: PPW-1258
The shared testbed setup installs MiniDLNA, Avahi, and dbus before
every cram and CDRouter job. This adds network and package mirror
dependencies to hardware tests and allows the runtime environment to
drift independently of the selected image.
The separately released testbed image now carries these packages,
together with the Python dependencies used by cram. Remove the runtime
apt update and install command so jobs consume only the image contents.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Fixes: 6792ba6b95 ("PROVISORY: ci: testbed: install minidlna and avahi packages at runtime")
References: PPW-1258
The testbed setup rewrites the Debian mirror to archive.debian.org
before installing runtime cram dependencies. This was needed for the
old Buster container, but Debian 13 uses current mirrors and stores its
configuration in deb822 files instead of /etc/apt/sources.list.
Both focused cram jobs stop at the first rewrite before DUT bring-up:
```text
$ sudo sed -i 's|http://deb.debian.org/debian|http://archive.debian.org/debian|g' /etc/apt/sources.list 2>/dev/null
Running after_script
```
Remove both obsolete rewrites and let apt use the sources supplied by
the versioned Debian 13 testbed image.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Fixes: 6792ba6b95 ("PROVISORY: ci: testbed: install minidlna and avahi packages at runtime")
References: PPW-1258
Debian 13 marks its system Python environment as externally managed,
so cram-time pip installs fail before the tests can exercise the DUT.
Python 3.13 also removed cgi, which the preinstalled servefile 0.5.4
still imports.
Use async-upnp-client, servefile, scapy, and legacy cgi from the
testbed image. Remove the per-test pip calls and PATH overrides, and
document that scapy is an image dependency. Preserve the newer
CABundle fixture setup from the source change.
The later 018-dhcpv4-client-options.t test reintroduced the same pip
call, so give it the same treatment.
Assisted-by: codex:gpt-5
Assisted-by: claude:opus-5:orchestrator
Assisted-by: codex:gpt-5.6-sol:medium-fast
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PPW-1258
ntpsec-ntpdate 1.2.3 reports successful queries as a timestamp
followed by numeric offset and uncertainty values instead of the
former textual offset wording.
Match the numeric ntpsec query format so the time test accepts the
output provided by the Debian 13 testbed image.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PPW-1258
The testbed container now has its own prpl Foundation project, release
pipeline, and versioned registry image. Keeping the old Debian Buster
image in prplos leaves two sources for the same CI runtime and continues
to expose obsolete build, test, and deploy jobs.
Remove the in-tree Dockerfile and its CI job definitions. Drop the root
include because the testbed and CDRouter templates now consume the
external containers/testbed release directly.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PPW-1258
The testbed and CDRouter templates currently derive their image from the
designated prplos branch. This couples image changes to an in-tree build
and prevents consumers from using the separately released container.
Use the released ci/containers/testbed v1.3.0 image in both templates,
pinning it to its published immutable manifest digest. This keeps cram
and CDRouter on the same Debian 13 environment with the runtime test
dependencies preinstalled, while every pipeline pulls the exact released
content.
The previous image declared /home/testbed as a volume. GitLab's Docker
executor replaced it with an empty, root-owned, persistent runner cache
that shadowed the writable home created in the image. OpenSSH resolves
~/.ssh through the passwd database instead of $HOME, so redirecting HOME
could not make bare ssh read the host key written by ssh-keyscan.
The v1.3.0 image drops that volume. Remove the ineffective HOME
overrides while retaining the .ssh directory setup that ssh-keyscan
needs.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PPW-1258
Cram tests conventionally define R directly from CRAM_REMOTE_COMMAND.
Commit 43f1696033 ("ci: cram: periodicfileupload tests sometimes fails
(PPW-1766)") introduced a sed pipeline for this test.
The substitution only matches ssh when it has a leading space, while
CRAM_REMOTE_COMMAND begins with ssh. The pipeline is therefore a silent
no-op.
Keeping a per-test rewrite would duplicate the stdin handling. The
preceding "ci: cram: keep remote commands off the script stdin" puts
ssh -n into CRAM_REMOTE_COMMAND at its source for every test.
Restore the canonical alias so this test follows the suite convention
and relies on the centralized remote command definition.
References: PCF-2222, PPW-1766
Assisted-by: claude:opus-5:orchestrator
Assisted-by: codex:gpt-5.6-sol:medium-fast
Fixes: 43f1696033 ("ci: cram: periodicfileupload tests sometimes fails (PPW-1766)")
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Cram supplies the complete generated test script through the shell's
stdin. SSH inherits that input from the shared remote-command alias and
drains the script beyond the shell's read-ahead buffer. The shell then
reaches EOF in the middle of a quoted command near byte 8,192:
```text
/bin/sh: 74: Syntax error: Unterminated quoted string
```
This is independent of the Python interpreter. Freedom job 15484314070
failed the 49-command probe at chunk 0036 under Python 3.14.6. With the
same probe and firmware, job 15484856804 passed all 49 chunks under
Python 3.13 after adding the SSH stdin protection.
Pass -n to the shared SSH command so it reads from /dev/null instead of
the test script. Apply the same safe default to the MXL fallback alias.
Assisted-by: codex:gpt-5
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
References: PCF-2222
Problem:
reboot TimeStamp precision has changed: unnecessary fraction of seconds have been removed
This simplification change has been introduced in PPW-1605 in the reboot-service when
the reboot timestamp is now set to unknow when the HGW isn't NTP synced.
Solution:
change the cram test timestamp check
Reference: PPW-1605
Signed-off-by: Florian Rouzaud <florian.rouzaud@softathome.com>
As part of PCF-2632, when no default Trusted Element is available,
the TrustedElement.SIM data model is now populated using the active
IMSI obtained from the modem.
Since a prepopulated Trusted SIM is now available, removed the
creation of a new Trusted Element during validation. This prevents
duplicate entries and resolves the cram test failure.
Updated 018-cellular-trusted-elements.t to validate Trusted SIM
based on the prepopulated active IMSI.
References: PCF-2632
Signed-off-by: Sathishkumar Selvam <sathishkumar.selvam@capgemini.com>
Add two dhcpv4 client tests. first one check if option provided by a
client is correctly provided in the DHCPv4.Server DM. Second one check
that if multiple client provide the same option with different value,
the value are not mixed up.
References: PCF-2262
Signed-off-by: Alexandre Fiset <alexandre.fiset@softathome.com>
Cram tests emit .t.err files on failure; ignore them so they don't
accumulate as untracked noise.
References: PCF-2262
Signed-off-by: Alexandre Fiset <alexandre.fiset@softathome.com>
In 05a30b1404 we removed "# CONFIG_VERSION_CODE_FILENAMES is not set" from the cram tests after aligning with openwrt 24.10.5.
However, this was forgotten for the freedom and valyrian board.
References: PPW-2167
Signed-off-by: Matthias Franck <matthias.franck@softathome.com>
Adding cram testcase to subscribe to ubus events for the
notify flow created and verify no other unexpected events
listed in ubus notifications subscribed for enumerate
connection tracking object.
References: PPW-1201
Signed-off-by: Hari Sriram <hari.s@capgemini.com>
Align the Easymesh R1 smoke test with the DUT flashing procedure from FEAT-29.
This commit aligns the DUT deploy scripts from the prplMesh repository.
Reference: PPM-4139
Signed-off-by: Maarten De Decker <maarten.dedecker@mind.be>
During the course of the test, containers are created and tested to properly survive a simulated software upgrade.
However, pcm does not guarantee that indexes will be the same after SWU, especially if there were any gaps
in numbering prior to the upgrade.
This change to the test makes it index-agnostic by replacing all possibly affected indexes with placeholders.
References: PPW-1787
Closes: PPW-1787
Signed-off-by: a.kuznetsov <a.kuznetsov@inango-systems.com>
Currently the final DNS host check after configuration restore
accepts only one digit in the generated alias. The failing
testbed-02 job reaches instance 10, restores a valid object, and
reports this Cram difference:
- "Alias": "cpe-X_PRPLWARE-COM_Host-\d", (re)
+ "Alias": "cpe-X_PRPLWARE-COM_Host-10",
The comparable testbed-01 control job reaches instance 9 and passes
the same check. Both jobs successfully restore the DNS object, so
the failure depends on the allocator crossing the single-digit
boundary, not on invalid restored state.
The additional testbed-02 topology from PCF-2504 is the supported
trigger for the one-instance delta. It exposes the restrictive
expectation but does not make the restored alias invalid.
So lets fix the final alias expectation to accept one or more
digits, matching the equivalent check earlier in the test. This
keeps the test independent of the current multi-instance allocator
position.
Closes: PPW-1934
References: PCF-2504
Link: https://gitlab.com/prpl-foundation/prplos/prplos/-/jobs/15294595185
Link: https://gitlab.com/prpl-foundation/prplos/prplos/-/jobs/15300965299
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Currently the BulkData MQTT cram test assumes that every DUT uses the
legacy SN[A-F0-9]+ serial-number format. SFL devices instead expose the
manufacturing serial through readmfg, including values such as
ospv2_A8C2463D425C and freedom_58E403D21127.
This makes a valid report fail the complete-payload grep even though the
reported SN is exactly DeviceInfo.SerialNumber. Both OSPv2 and Freedom
jobs show the following assertion failure:
$ R 'cat /tmp/mqtt_subscriber' | \
> grep -cP '^\{"Report":\[\{"SN":"SN[A-F0-9]+","CollectionTime":"[0-9]{13}"\}\]\}$'
-[1-9][0-9]* (re)
+0
+[1]
So lets fix the test by reading DeviceInfo.SerialNumber from the DUT and
passing that value to jq. Require exactly one report and compare its SN
with the live serial. Validate CollectionTime independently as a string
of exactly 13 digits. Print the payload when validation fails so future
cram diffs preserve the malformed message.
Closes: PPW-2102
References: PCF-2643
Assisted-by: codex:gpt-5.6-sol:high
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org>
Problem:
network_security.t fails deterministically on latest-24.10. The
service-readiness block added by !2697 (adc82ee3, PPW-887) starts
with "$ R '" without the two-space indent cram requires for a
command line.
Root cause:
cram does not parse the malformed line as a command, so it is
absorbed as expected output of the preceding nmap check. Its
continuation lines then attach to that check and run in the local
testbed shell instead of on the DUT via the R alias:
/bin/sh: 6: pgrep: not found
/bin/sh: 14: Syntax error: Unterminated quoted string (no-eol)
adc82ee3 was reverted (72f0994e) once DUT jobs were finally
triggered and exposed the failure; no DUT cram job ran against any
revision of !2697 before it merged.
Solution:
Re-add the service-readiness wait, indenting "$ R '" by two spaces
so cram parses it as a command and dispatches the wait script to the
DUT through the R alias, matching every other "$ R '...'" usage in
this test suite.
Reference: PPW-2150, PPW-887, PPW-455
Signed-off-by: Alexandre Fiset <alexandre.fiset@softathome.com>
When finishing the pwhm-wpa3-cm.t test script, MLDUnits are currently
restored to the same values as in Freedom (0: Private MLD, 1: Guest
MLD).
However, MLO is not yet supported on OSPv2, so MLDUnits should instead
remain at their default value (-1, as set through MXL UPDK).
Closes: PPW-1713
References: PCF-2395
Signed-off-by: Houssem Dafdouf <houssem.dafdouf_ext@softathome.com>
After re-enabling the wifi-scheduler script, update the test to:
- Add backhaul interfaces
- Use the correct prplMesh AccessPoint creation method
- Use the generic wifi_dm helper
- Adapt AP status checks to keep the script generic
References: PPW-774
Signed-off-by: Houssem Dafdouf <houssem.dafdouf_ext@softathome.com>
The test was hardcoded to expect wan and br-lan as interface names,
but the actual kernel/netdev interface name that showed up in the mangle table was eth1.
Instead of resolving the interface name at test time (hardcoded),
we resolve it at test run time as a dynamic resolution by
walking through the data model to extract the real netdev name whatever the project (ospv2/freedom)
the test now emit OK if this check passes or FAIL with explicit message if the check fails.
Tracked-by: PPW-1998
Signed-off-by: Yousfi Neji <yousfi.neji_ext@softathome.com>
Remove the deprecated Device.DeviceInfo.Reboots.reboot() RPC and its input
parameters (Cause, Reason) from the obuspa.expected fixture.
This change aligns the test expectations with the current reboot-service
model where the reboot() RPC is no longer exposed.
References: PPW-1960
Signed-off-by: ahmed sassi <ahmed.sassi_ext@softathome.com>
uboot-tools (pylibfdt) failed to compile with:
fatal error: Python.h: No such file or directory
Add python3-dev to the apt-get install step to provide the
missing header.
Reference: PPW-2121
Signed-off-by: Mouna Aloui <mouna.aloui@softathome.com>
Integrates:
-Update the CRAM test expectations to use the correct parameter name Stats.NACK instead of the misspelled Stats.NAK
References: PPW-1952
Signed-off-by: FAOUZI YAHYA <faouzi.yahya_ext@softathome.com>
Integrates:
-Update the obuspa expected fixture to add:
Device.SoftwareModules.ExecutionUnit.{i}.EnvVariableNumberOfEntries
Device.SoftwareModules.ExecutionUnit.{i}.HostObjectNumberOfEntries
References: PPW-1954
Signed-off-by: Antoine Philippe <antoine.philippe@softathome.com>
Integrates:
-Update the obuspa expected fixture to include
Device.SoftwareModules.ExecEnv.{i}.ApplicationDataNumberOfEntries
for timingila.
References : PPW-1954
Signed-off-by: ahmed sassi <ahmed.sassi_ext@softathome.com>
Context:
========
A first attempt to reorganize the pwhm usp tests for PPW-1606 was
introduced in commit 7560c16e81.
However, CI was failing because some required alias definitions were
missing from the OSPv2 cram tests. The change was reverted in commit
c7ea8021e0.
Let's rework the change and include the missing alias definitions so
the test reorganization is handled properly.
Initial problem:
================
During the integration of PPM-3671, the 103-apmld.t script started
failing due to an incorrect security mode set by previous tests.
Cause:
======
The security mode was unexpectedly modified during the execution of
102-pwhm-usp.t while toggling AP1. This happens because prplMesh now
runs in controller mode as part of PPM-3671.
Solution:
=========
When testing pure PWHM features, prplMesh should not be running to
avoid conflicts. This was previously the case, except for the AP1
toggling step executed in wifi-usp.t.
Move the USP-related test cases to more appropriate locations:
- USP socket creation and PWHM USP event checks (AP1 toggling):
moved to 100-pwhm.t where prplMesh is kept stopped
- prplMesh connection to the PWHM direct socket:
moved to 110-prplmesh-dataelements.t
- 102-pwhm-usp.t is so removed from fast track test scripts
Closes: PPW-1606
References: PPM-3671
Signed-off-by: Houssem Dafdouf <houssem.dafdouf_ext@softathome.com>
Move AssociatedDeviceNumberOfEntries to match the current HomePlug data model output ordering in obuspa.expected.
Refrences: PPW-1953
Signed-off-by: ahmed sassi <ahmed.sassi_ext@softathome.com>
Updating the test 005-build-config during PCF-2624.
Update the test to add CONFIG_FWI_FLASH_PARTITIONS="u-boot,kernel,rootfs"
References: PCF-2624, PPW-2154
Signed-off-by: Hela Saadani <hela.saadani_ext@softathome.com>
New packages ivykis and liburing appear in the expected build config
because upstream openwrt/packages commit 8fcf45d05a changed syslog-ng
to depend on an external ivykis package (instead of the previously
embedded copy). ivykis in turn pulls in liburing when io_uring support
is enabled in the kernel.
References: PCF-1982
Signed-off-by: Hela Saadani <hela.saadani_ext@softathome.com>
This reverts commit 693e5596d9.
No longer needed after fixing the original bug.
References: PPW-1656
Signed-off-by: Nabil BIZID <nabil.bizid@softathome.com>
The test expected a trailing blank line after each one-shot usp-cli
command. That blank line was never real CLI output: pre-PCF-2445, amx-cli
(which usp-cli, like ba-cli, is a front-end of) kept reading the stdin
inherited from the cram-over-ssh R wrapper and echoed its trailing newline
to stdout. Cram tests written in that era encoded the stray blank line as
expected output.
e893e4d4f2 ("profiles: prpl_core.yml: feed_amx: amx-cli: bump stdin fix")
bumped feed_amx to the PCF-2445 fix, so positional one-shot commands no
longer consume or echo the inherited stdin. The trailing blank line is now
deterministically gone for every usp-cli call, so the failure is not
intermittent ("in some cases ... but not always") but reproducible.
This test was added in 5c664a92d9 ("ci: cram: add new tests for inter
container communication"), authored before the amx-cli fix and merged
without being rebased onto it, so it kept the old blank-line expectations
and now fails.
Drop the stale blank-line expectations so the test matches the corrected
one-shot output, the same way the other cram tests were realigned in
e893e4d4f2 (e.g. 015-nat-reverse-portmapping.t), instead of masking them
with a per-command sed filter.
Fixes: 5c664a92d9 ("ci: cram: add new tests for inter container communication")
Closes: PPW-2028
References: FEAT-18, PCF-2445
Signed-off-by: Jef Steelant <jef.steelant_ext@softathome.com>
