When rebuilding a package, OpenWRT build system uses a complex set of
routines to check weither it must rebuild the dependent packages or skip
them and only rebuild the package. This smartly optimizes the build
time.
To do so, each dependent package calls the macro Build/Autoclean
(defined in include/package.mk).
This macro calls twice the `rdep` macro (defined in include/depends.mk):
1. To check if a reconfiguration is needed:
* if one of its files have changed (handled by .deps_file hash)
* if timestamp of any of its files is newer than the stamp .configured_<hash>
When this check is done, a new stamp file ".configured_<hash>_check is
created. If a configuration should take place, the timestamp of the
stamp check file becomes newer than the .configured_<hash> file: it
triggers a new package reconfiguration.
2. To check if a rebuild is needed:
* if timestamp of any of its files is newer than the stamp .built
When this check is done, a new stamp file ".built_check is created. If a
rebuild should take place, the timestamp of the stamp check file becomes
newer than .built triggering anew package build.
The rdep check already excludes a set of files that are generated after
the .built stamp file (like ipkg*).
When the 'security' profiles is loaded,
CONFIG_JSON_CHECKSEC_MITIGATIONS_REPORT is set and make all
packages generate a checksec report in their package build di (e.g.
<pkg_build_dir>/CHECKSEC).
This folder and its children files are generated after the .built file.
On next build, OpenWRT detects that checksec reports are newer than
.built file and trigger a rebuild of that component. This should not be
the case since this is not a file generated by the package itself!
Let's exclude the files of the CHECKSEC folder from the build check
mechanism to avoid unecessary rebuilds.
References: PPW-1414
Signed-off-by: Cedric CHEDALEUX <cedric.chedaleux@orange.com>
The code to find OVERLAY_MAKEFILE_APPENDS for each package took
about 0.3 seconds which can create a big overhad during
"./scripts/feeds update" processing, where OpenWrt process each
package that exists in all feeds, not only packages targeted for
installation.
To reduce processing time we will re-use the
"$(TMP_DIR)/info/.files-packageinfo-$(SCAN_COOKIE)" file that
created on each run of "make" in OpenWrt root directory and,
after the 1ff5427954 commit,
contains list of all relevant Makefile.append files.
In cases when this file missed, the plain search implementation
would be used to get list of Makefile.append's.
References: PPW-1162
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
Problem: When package configuration fragments defined in a separate file included from base package Makefile
OpenWRT can't track changes in Package/<name>/config variable correctly.
Cause: In OpenWrt when you move part of package confgiuration to separate files you should use the following
trick in Makefile:
SCAN_DEPS=part_of_package.mk
-include part_of_package.mk
In this case changes in "part_of_pacakge.mk" would trigger regenration of "tmp/info/.packageinfo-<path_to_package>"
file. The "tmp/info/.packageinfo-<path_to_package>" file used in generation of "tmp/.packageinfo" and
"tmp/.config-package.in" files that used for OpenWRT defconfig infrastructure.
Unfortunately OpenWrt implementation can't correctly track cases when such "part_of_package.mk" deleted.
Solution: Add md5sum from string with dependency names as dependecy for "tmp/info/.packageinfo-<path_to_package>"
file. The code reuse OpenWRT trick that already used for "tmp/.packageinfo" dependencies
References: PCF-1613
Upstream-Status: Submitted [https://github.com/openwrt/openwrt/pull/19408]
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
Problem: With the new Overlay Feeds feature Makefile.append files would be appended to <package>/Makefile
code, as result OpenWRT build system should interpret this files as <package>/Makefile dependency in same
manner as includes defined with SCAN_DEPS variable.
Cause: Overlay Feeds is a new feature and requires support for new files.
Solution: Scan all enabled feeds for Makefile.append files and add "$(TOPDIR)/feeds/<feed_name>/path/to/<package>/Makefie.append"
as dependency for "tmp/info/.packageinfo-<path_to_package>" files.
References: PCF-1616
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
Problem: OpenWRT Build system provide no way to extend package from 3rd party feed.
As a result in order to manage CVE patches or small modification to a single package
we should make copy of package in feeds that we own and monitor any changes in the
package original feed to not miss upstream updates.
Solution: Include all $(PKG_DIR_NAME)/Makefile.append files as part of first
$(eval $(call BuildPackage,packagename)) call in original $(PKG_DIR_NAME)/Makefile.
Add append directories to PKG_FILE_DEPENDS in order to invalidate build when
append files changed.
Together with this change provide several useful functions and variables:
Variable $(THISDIR) -- defined in context of Makefile.append files to and points to
the directory where Makefile.append recipe. Can be used to find additional patches,
config fragments, etc.
Variable $(OVERLAY_PATCHES) -- can be used to provide additional patch files for package
sources in the following manner:
OVERLAY_PATCHES := $(OVERLAY_PATCHES) $(THISDIR)/patches/filename.patch
OVERLAY_PATCHES := $(OVERLAY_PATCHES) $(wildcard $(THISDIR)/patches/*.patch)
Variable $(newline) -- contains single "\n" symbol. Required to append/prepend
OpenWRT multiline variables with additional line.
Functions (append,VARNAME,value[,override]) -- used to append "value" string to variable VARNAME.
Behave like "+=" with two exceptions:
1. Append "value" without extra space before the value
2. The "value" expanded immediately before the assignment, while VARNAME is treated
as recursive expand variable. Such behavior allow you to use $(THISDIR) variable
together with "append" function. For example the following call will expand
$(THISDIR) immediately:
$(call append,Package/helloworld/config,$(newline)source "$(THISDIR)/Config.in")
Function (prepend,VARNAME,value[,override]) -- used to prepend "value" string to variable VARNAME.
Shares "append" function behavior regards expansion time. In order to append string
with delayed expansion time you can replace $(...) with $$(...). For exaple the
following call will delay $(Build/IncludeOverlay) expansion:
$(call prepend,BuildPackage,$$(Build/IncludeOverlay)$(newline))
Function (remove,VARNAME,value[,override]) -- behave like "filter-out" function, but allow to
control expansion time of the passed "value".
Function (rewrite,VARNAME,value[,override]) -- generalized function that allow arbitrary change
of VARNAME variable value controlling "value" expansion time.
All this functions support optional last argument that allow to modify "override VARNAME" instead
of non-override version of VARNAME.
References: PCF-1615
Signed-off-by: Ivan Efimov <i.efimov@inango-systems.com>
Signed-off-by: Nicolai Scurtu <n.scurtu@inango-systems.com>
Signed-off-by: Vadym Omelchuk <v.omelchuk@inango-systems.com>
Signed-off-by: Vladimir Shulika <v.shulika@inango-systems.com>
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
In the case where an external feed overwrite the generic target
directory, for kernel_menuconfig the kconfig.pl use the GENERIC_PLATFORM_DIR
variable to search for the config-filter file.
The config-filter file is a file to exclude some kernel config from the
final target config file and doesn't depend on specific kernel version.
Also it's generic enough and it's possible to use the TOPDIR generic
target instead of requiring the external target feed to provide one.
To fix the kernel_menuconfig and prevent external feed from having to
duplicate this additional file, use the TOPDIR generic target as
fallback if the external target feed doesn't provide one.
References: PCF-1881
Fixes: e4635d24b21b ("target: introduce default location for external target linux generic")
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This was already done in 91e1d4b44ef03adb59072f0b00ae0dd877223ca8 for openwrt23.05,
but the 6.6 kernel file is new in openwrt24.
So let's move it as well.
Signed-off-by: Matthias Franck <matthias.franck@softathome.com>
An external target provided by a custom feed might depend on a specific
kernel version and might need to apply generic patches for the specific
kernel version.
Now that the kernel version file is defined in the generic target
director, it's possible to handle this scenario for an external target
from feed.
To do this, we introduce additional logic for GENERIC_PLATFORM_DIR where
we check if the target feed have in his upper directory a linux/generic
path and use that instead of the default TOPLEVEL linux/generic path.
An example for an external target from feed would have the following
path structure:
mediatek/Makefile
linux/generic
linux/generic/config-6.6
...
An external target from feed is also defined by creating a symbolic link
in target/linux/${BOARD} to the feed directory with board as the name of
the external target.
This is to try to enforce a more consistent way to handle external
target in downstream project that makes use of OpenWrt build system.
Upstream-Status: Submitted [https://github.com/openwrt/openwrt/pull/18820]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
eneric other-files directory path hardly needs to be overwritten as 99%
of the time the mainline generic one is usable.
Don't depend on GENERIC_PLATFORM_DIR variable (that can point to the
external target feed path) but introduce a dedicated define and set it
to the default TOPLEVEL location by default.
Upstream-Status: Submitted [https://github.com/openwrt/openwrt/pull/18820]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Move kernel-version.mk include to proper place to properly set patch folders for test kernel.
Fixes: 8865dadea7b94e7859b416d3b1931b897ea43c48
Signed-off-by: Mieczyslaw Nalewaj <namiltd@yahoo.com>
(cherry picked from commit 2c8fb6fa9ff67bdc4cc4ce3c38f372d23ba4377f)
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
To further improve cleanup and maintenance usage, move the kernel
version files to target/linux/generic directory. This permits to self
contain any change to the specific generic directory instead of having
to bload the include directory of periodic changes.
In kernel-version.mk we now use GENERIC_PLATFORM_DIR provided by
target.mk. To make this work, we need to move the inclusion of
kernel-version.mk in target.mk right after GENERIC_PLATFORM_DIR is
defined.
This also comes to permit downstream project to provide a custom generic
directory and specify the kernel version complete of the hash and the
minor version without having to affect other feeds.
In such case both generic and the target directory are provided as feeds
and OpenWrt reference these specific one instead of the generic one.
For downstream it's still suggested and preferable to all match the
shipped generic kernel minor version but this change permits to at least
enforce good practice instead of having to bloat OpenWrt include file of
all kind of downstream changes (making porting to OpenWrt mainline even
more difficult)
(cherry picked from commit 8865dadea7b94e7859b416d3b1931b897ea43c48)
[ rebased on top of openwrt-23.04 ]
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Linux 5.15 kernel is in use on OSPv2 and OSPv1, Adding generic linux
5.15 kernel config so that we can build images for OSPv2 and OSPv1 with
Openwrt 24.10.
References: PCF-1574
Signed-off-by: Bhavesh Bharadiya <bnbharadiya@maxlinear.com>
Checksec is a bash script to check the properties of executables (like
PIE, RELRO, Canaries, ASLR, Fortify Source).
Add an optional build feature, which when enabled is going to create a
JSON report about the state of mitigations in the produced binaries.
Upstream-Status: Submitted [https://github.com/openwrt/openwrt/pull/14569]
Signed-off-by: Cedric DOURLENT <cedric.dourlent@softathome.com>
Netifd is not longer responsible for configuring the gateway. The TR181 components took over this task.
Signed-off-by: Matthias FRANCK <matthias.franck@softathome.com>
ATH 12.4.0 CS is based on 6.1 kernel and thus we need to add a tarball
hash, so we could build the kernel.
References: PCF-1176
Signed-off-by: Marvin Lu <marvin.lu@wnc.com.tw>
Signed-off-by: Petr Štetiar <petr.stetiar@prplfoundation.org> [commit description]
(cherry picked from commit ac99cc80a976b9d807048d9781b4d02b6202d796)
Define the complete Linux kernel 5.4 version which is needed for WNC
Freedom board support as its using older 5.4 kernel.
Signed-off-by: Marvin Lu <marvin.lu@wnc.com.tw>
Signed-off-by: Petr Štetiar <ynezz@true.cz> [commit facelift]
(cherry picked from commit 1e3ed96bc295d6f37dce032f71736c1988603d03)
When we try to add additional membership in `foo`, `bar` and `baz` groups to
the dnsmasq user via `USERID` variable, we're supposed to do so via comma
separated delimiter:
```
USERID:=dnsmasq=453:dnsmasq=453:foo,bar,baz
```
thus the `USERID` variable now contains comma separated values which Make
parses/interprets as separate arguments of the underlying `addfield`
function/macro, but they should be passed verbatim, otherwise we get
content in the `/etc/group`:
```
dnsmasq❌453:dnsmasq
fooSourceDateEpoch❌65536:dnsmasq
```
and `dnsmasq.control` files mangled:
```
Require-User: dnsmasq=453:dnsmasq=453:fooSourceDateEpoch: 1697028391
```
So lets fix it by making the expansion of `USERID` happening in the `addfield`
function/macro as expected.
Upstream-Status: Pending [https://github.com/openwrt/openwrt/pull/14337]
Signed-off-by: Cedric DOURLENT <cedric.dourlent@softathome.com>
(cherry picked from commit 108af4a7bec74151822a539a75de2c76e7f2c295)
OpenSSL should be used as the only TLS provider, so lets replace
target's default WolfSSL with OpenSSL as well, otherwise we still might
end up with WolfSSL in the final image.
Fixes: PCF-1006
Fixes: e206607d0232 ("profiles: prpl: make OpenSSL default and only TLS provider")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 195e2ebe453401615c3dd62dd7e46a3148f958cb)
Henceforth, we will utilize the prplOS version throughout the
development cycle, determined by the output of the `git describe`
command. The version will be composed of:
- The fixed version number of the upcoming release currently in
development.
- The abbreviated (8 characters) object hash of the most recent commit.
For instance, with the ongoing development of prplOS version 3.0.0 and
having the Git hash `5a2a69d`, the resulting version would be
`3.0.0-5a2a69d`.
References: PCF-691
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit d2499f3e10aa6a8982122c0def4dc7997bbc1753)
We’ve got bug reports, that the new mod-ppp-daemon-v0.5.4 fails to build:
x86_64-openwrt-linux-musl-gcc -Os <snip>
In file included from ppp.c:70:
prplos/staging_dir/target-x86_64_musl/usr/include/pppd/pppd.h:60:10: fatal error: net/ppp_defs.h: No such file or directory
60 | #include <net/ppp_defs.h>
| ^~~~~~~~~~~~~~~~
The problem is that we now have two ppp packages with
different/conflicting staging dir content, `ppp` and `ppp-prpl`, both are
being compiled now, but just `ppp-prpl` ends in the final image.
This can result likely in the following build race:
Working case:
ppp (installs incompatible pppd.h header)
|
`ppp-prpl (installs compatible pppd.h header)
|
`mod-ppp-daemon (builds fine)
Broken case:
ppp-prpl (installs compatible header)
|
`ppp (installs incompatible header)
|
`mod-ppp-daemon (build fails)
So workaround it for now by removing `ppp` and `ppp-mod-pppoe` from
default packages, as we clearly can't have two packages for `ppp` in the
system.
Fixes: PCF-979
Signed-off-by: Petr Štetiar <ynezz@true.cz>
(cherry picked from commit 251911e39aa471b3047c3c2db67b0044c370e772)
TF-A LTS releases have an lts- prefix in their version tag. Introduce a
PKG_LTS variable to support these.
As the non-LTS tarbals do not contain the version prefix in the
directory, we need different PKG_BUILD_DIR for each variant:
➜ tar --list --file dl/trusted-firmware-a-v2.12.tar.gz | head -n1
trusted-firmware-a-2.12/
➜ tar --list --file dl/trusted-firmware-a-lts-v2.12.1.tar.gz | head -n1
trusted-firmware-a-lts-v2.12.1/
Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
cherry picked from commit 06c4e8a262e4618e61866ed92e296e46a1b6b671
Signed-off-by: Thomas Richard <thomas.richard@bootlin.com>
Link: https://github.com/openwrt/openwrt/pull/19076
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
Changelog: https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.6.92
No patches needed a rebase with this release/only archive hash updated.
Build system: x86/64
Build-tested: flogic/xiaomi_redmi-router-ax6000-ubootmod, ramips/tplink_archer-a6-v3
Run-tested: flogic/xiaomi_redmi-router-ax6000-ubootmod, ramips/tplink_archer-a6-v3
Signed-off-by: John Audia <therealgraysky@proton.me>
Link: https://github.com/openwrt/openwrt/pull/18835
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
(cherry picked from commit 2a7cfbbbef76446bc0340fecda79a2d88c9375f5)
Link: https://github.com/openwrt/openwrt/pull/18924
Signed-off-by: Robert Marko <robimarko@gmail.com>
Currently, we are filtering out images if DEFAULT:=n or BROKEN:=y are set,
so if you are building from scratch and want to build custom images that
are stripped down to fit, you must edit the image recipe or its just
filtered out.
So, to allow this behaviour when building from scratch as we can assume
that person doing that knows what they are attempting to do lets just limit
the filtering to ImageBuilder.
Fixes: f060615a78e5 ("image: respect DEFAULT and BROKEN when Default profile is selected")
Signed-off-by: Robert Marko <robimarko@gmail.com>
Currently, when you select the Default profile it does not honor DEFAULT:=n
nor BROKEN:=y in device profiles but rather just tries to build all of them.
This may work when building directly, but when using Image Builder it will
always fail since no kernel or anything else is present for devices that
have DEFAULT:=n or BROKEN:=Y set since those are skipped during build.
So, lets look for DEFAULT being set to "n" or BROKEN being set to "y" and
then remove clear _PROFILE_SET so they dont end up being marked for
installation.
Fixes: #18410
Link: https://github.com/openwrt/openwrt/pull/18814
Signed-off-by: Robert Marko <robimarko@gmail.com>
(cherry picked from commit f060615a78e5c5e86829b2e40c3f77e5cf7033bf)
Do not assume that the various tools like llc can be found under the
same path as clang; instead, look them up through BPF_PATH (while still
preferring ones found next to clang).
This fixes build in common setups with ccache, where clang resolves to a
path like /usr/lib/ccache/bin/clang, but no other tools can be found at
that location.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Link: https://github.com/openwrt/openwrt/pull/18422
Signed-off-by: Robert Marko <robimarko@gmail.com>
(cherry picked from commit 89f1d56a7eab7d530be4bbb5cf21dfc80b3485b1)
CLANG_MIN_VER is 12, so there is no point in looking for clang-11.
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Link: https://github.com/openwrt/openwrt/pull/18422
Signed-off-by: Robert Marko <robimarko@gmail.com>
(cherry picked from commit 45f0eb106bed8868c02e0ccfb545b7e1c4279841)
In some cases hostpkg python from packages feed is used (hostpkg has higher
priority in PATH) which causes build failure (cryptography module is
missing). So override PATH to not use hostpkg python.
Signed-off-by: Thomas Richard <thomas.richard@bootlin.com>
Link: https://github.com/openwrt/openwrt/pull/18102
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
(cherry picked from commit 8c3a43be27d989ef42d4148949e43266694a5168)
