With the related change, we build manauals in the debian package of osmocom-bb. The osmocom:nightly:asan feed gets built with manuals disabled. Fix that building the osmocom-bb package in the asan feed currently fails with:
[ 160s] /usr/bin/make -C doc/manuals [ 160s] make[2]: Entering directory '/usr/src/packages/BUILD/doc/manuals' [ 160s] OSMO_GSM_MANUALS_DIR: /usr/src/packages/osmo-gsm-manuals (fallback) [ 160s] ERROR: OSMO_GSM_MANUALS_DIR does not exist! [ 160s] Install osmo-gsm-manuals or set OSMO_GSM_MANUALS_DIR. [ 160s] Makefile:6: /build/Makefile.asciidoc.inc: No such file or directory [ 160s] make[2]: *** No rule to make target '/build/Makefile.asciidoc.inc'. Stop.
Building limesuite is not needed anymore, as it is available at least since debian 10 in the debian and ubuntu repositories. We have been building the upstream version without changes.
After looking at this again, it makes more sense to keep /proc/sys/kernel/core_pattern as "core" on build4 and build5, which we use for jenkins jobs instead of installing systemd's coredumpctl and making all coredumps go through that. The motivation for using coredumpctl in testenv was that lots of developers may have it already installed (though I'm not sure about that anymore), and that it gives a nice API for retrieving the related executable name to a coredump instead of having to parse the output of e.g. "file".
Using coredumpctl had the disadvantage that coredumps for all other jobs, such as the master-builds are also caught by that and cannot be easily placed in the workspace directory. I have started implementing this by extending testenv-coredump-helper to communicate through a socket and mounting that socket inside docker containers running contrib/jenkins.sh (host -> lxc -> docker), and it works, but this complexity is not useful here.
Instead the related patch will make testenv pick the core files when coredumpctl is not available, and read the path from "file", making testenv-coredump-helper obsolete.
Add a wrapper script that runs a given program, and checks the exit code. If it is not 0, and a core file appears, then load the coredump in gdb and display its backtrace.
Now that it is also possible to run the HLR tests against PyHSS, add "-c osmo_hlr" to run the existing ttcn3-hlr-test jobs against osmo-hlr. I plan to add jobs for PyHSS later in a separate patch.
Adjust to MME tests now having two configurations in osmo-ttcn3-hacks:
[testenv] Found multiple testenv.cfg files, use one of: [testenv] -c emergency [testenv] -c generic [testenv] You can also select all of them (-c all) or use the * character as wildcard.
The osmocom-latest, osmocom-nightly etc. meta-packages depend on the osmocom-keyring package. Fix that the osmocom-keyring package was also depending on these meta packages.
This fixes that when uninstalling all osmocom:latest packages by removing the osmocom-latest meta package, the osmocom-keyring package would also get removed. This is undesirable for switching between latest and nightly, as the osmocom-keyring package must remain installed to be able to install new packages from the repository (as it has the repo key).
We have recently noticed that with current settings the nightly arm packages have still not finished the next morning. Schedule nightly OBS packages 2-3 hours earlier to avoid this.
Print all shell commands and their output executed by build_srcpkgs.py and build_binpkgs.py, instead of only printing them on error. This is useful to see the output of the scripts downloading dependencies when building sourcepackages.
Build source packages for the python project pyhss similar to how we do it for erlang projects: by downloading all dependencies ahead of time and vendoring them in the source package.
Add the Osmocom fork of PyHSS to scripts/obs/lib/config.py, so the Osmocom_OBS_* jobs build source packages for it and push them to our OBS where binary packages get built. We need to use a fork for now, as the official repository can't run outside of a git tree yet and doesn't have debian packaging. These changes are being upstreamed, the goal is that we don't need to use our fork in the long run.
For most C projects, running checkpatch works best with the coding style we use. Run clang-format only in specific projects that seem to be fully formatted with that tool.
Fix that the Osmocom-release-manuals job was still using debian-bookworm-build instead of debian-trixie-build to build the manuals. We use the latter for building manuals in master-builds and gerrit-verifications.
Without this patch building pyosmocom manuals failed with the following, as python3-typing-extensions in debian bookworm is at version 4.4.0 but typing_extensions >= 4.6.0 is needed for TypeAliasType. Trixie has version 4.13.2.
AttributeError: module 'typing_extensions' has no attribute 'TypeAliasType'. Did you mean: 'TypeAlias'?
Create an /etc/security/limits.d file to allow the jenkins user to set rtprio 30. This is required for running the ttcn3 bts testsuite through testenv, as (unlike the previous approach with docker containers) osmo-bts-trx runs without root rights there.
The debian 10 repository has been officially disabled: https://osmocom.org/news/308
However we just re-enabled a subset of the packages, osmo-gbproxy + dependencies, because currently they are relevant for a customer. Adjust the repo-install-test to deal with this subset of packages for debian 10 to fix that it is currently failing.
I have verified that repo-install-test works with this change for debian 10, 11 and 12. It currently doesn't run for debian 13 yet (OS#6934).
Add a script that maintains a linux repository in one place on jenkins nodes, so we need less git clones from git.kernel.org and less disk space. All jobs that need a kernel tree can now clone the relevant branch directly from the jenkins node.
Follow up patches will add a jenkins job that runs the script daily and adjust the existing jobs to make use of this instead of doing their own clones.
Currently this script produces a 396M bare git repository.
Run checkpatch with any .checkpatch*.conf found in the project dir. This is in preparation for having two .checkpatch.conf files in osmo-trx, in order to use different linting rules for C++ code: * .checkpatch.c.conf * .checkpatch.c++.conf
repo-install-test: fix test_conflict for debian 13
Two changes are necessary to make this test work with debian 13:
* Installing libosmocore from osmocom-latest, then switching to osmocom-nightly and attempting to install another package is not enough anymore to trigger a conflict. apt is now able to resolve this by uninstalling the osmocom-latest package and upgrading libosmocore to the nightly version. Force the conflict by explicitly marking osmocom-latest (osmocom-$FEED) as installed and for hold.
* The apt conflict message has been reworked, so the string to look for needs to be adjusted.
The following packages have unmet dependencies: sdcc-dbgsym : Depends: sdcc (= 4.2.0~osmocom3.113.9edd) but 4.5.0+dfsg-1 is to be installed E: Unable to correct problems, you have held broken packages. E: The following information from --solver 3.0 may provide additional context: Unable to satisfy dependencies. Reached two conflicting decisions: 1. sdcc:amd64=4.2.0~osmocom3.113.9edd is not selected for install 2. sdcc:amd64=4.2.0~osmocom3.113.9edd is selected for install because: 1. sdcc-dbgsym:amd64=4.2.0~osmocom3.113.9edd is selected for install 2. sdcc-dbgsym:amd64 Depends sdcc (= 4.2.0~osmocom3.113.9edd)
This package from strongswan-epdg causes the SSH connection to QEMU to break when installed in debian 13. Don't install it. Use the wildcard, because there is also a debug symbols package that pulls in charon-systemd.
PyHSS listens on the same port as OsmoHLR, which causes the test to fail with debian 13 because OsmoHLR can't start up properly. PyHSS wasn't built for earlier debian versions in the Osmocom binary repositories.
I have applied a patch in our OBS server, which adjusts the version check so we can build packages with newer LXC versions and sent it upstream: https://github.com/openSUSE/obs-build/pull/1127
Remove the lxc version constraint (which finally allowed upgrading to a newer opensuse version).
Fix errors in lxc-start related to not having securityfs available:
[ 14s] booting lxc... [ 14s] lxc-start: obsbuild:root_6: lsm/apparmor.c: apparmor_prepare: 1132 If you really want to start this container, set [ 14s] lxc-start: obsbuild:root_6: lsm/apparmor.c: apparmor_prepare: 1133 lxc.apparmor.allow_incomplete = 1 [ 14s] lxc-start: obsbuild:root_6: lsm/apparmor.c: apparmor_prepare: 1134 in your container configuration file [ 14s] lxc-start: obsbuild:root_6: start.c: lxc_init: 879 Failed to initialize LSM [ 14s] lxc-start: obsbuild:root_6: start.c: __lxc_start: 2008 Failed to initialize container "obsbuild:root_6" [ 14s] lxc-start: obsbuild:root_6: tools/lxc_start.c: main: 306 The container failed to start [ 14s] lxc-start: obsbuild:root_6: tools/lxc_start.c: main: 312 Additional information can be obtained by setting the --logfile and --logpriority options [ 14s] '/var/cache/obs/worker/root_6/.build/_exitcode' not found or symlink
Usually systemd would mount the securityfs automatically as I understand it, but it doesn't work anymore without this service in our use case where osmocom-obs-worker runs inside an LXC itself. This is probably related to upgrading the opensuse version for the osmocom-obs-worker containers.
Logic for building the osmo-s1gw-usermanual.pdf has just been added to master. The Osmocom-release-manuals job tries to build old manuals as well because now the osmo-s1gw directory exists at https://ftp.osmocom.org/docs. Ignore old tags, as we can't build usermanuals for those.
Allow configuring the email notifications before starting a job, so no notifications can be set while testing changes to the jenkins job. Add jenkins-notifications@lists.osmocom.org while at it, as we have it with almost all other jobs.
* move name and desc up (in preparation to adding another job in the same file) * remove "concurrent: false": the octsimtest node is already configured to only accept one job. if we had multiple octsimtest nodes, we wouldn't need this limitation of not running concurrently, so this isn't useful. * remove options without effect (block-downstream, block-upstream, disabled, build-blocker, etc.) * add "scm checkout" comment to retry-count as we have it in other files
This job already gets triggered once a day from master-osmo-ccid-firmware. Running it once a day like all other master jobs should be enough, remove the timed trigger that caused it to run twice a day.
scripts/verify_value_string_arrays_are_terminated.py:22: SyntaxWarning: "\s" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\s"? A raw string is also an option. scripts/verify_value_string_arrays_are_terminated.py:22: SyntaxWarning: "\s" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\s"? A raw string is also an option. scripts/verify_value_string_arrays_are_terminated.py:23: SyntaxWarning: "\s" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\s"? A raw string is also an option. scripts/verify_value_string_arrays_are_terminated.py:23: SyntaxWarning: "\s" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\s"? A raw string is also an option. scripts/verify_value_string_arrays_are_terminated.py:30: DeprecationWarning: codecs.open() is deprecated. Use open() instead.
New jenkins versions require newer java versions to be installed on the jenkins nodes. Install openjdk from download.java.net instead of using distribution packages, so we are not limited by what the distribution offers (which is oftentimes not recent enough).
Use the openntpd package, as there isn't a ntp package anymore. The service gets enabled and started automatically after installing it, so remove the block that did it manually.
Add a script for downloading artifacts from the build job in a gerrit pipeline, so it can be used in the hwtest job job.
pipeline: <--- URL to the pipeline is passed to all jobs parallel: sequential: build <--- artifacts are here hwtest <--- will call the new script with the pipeline URL lint binpkgs ...
Add a new test that flashes the firmware from the previous build job and runs tests against it. Tests are intentionally from the master branch for security reasons.
Ensure the JDK can always be installed to /home/{{ jenkins_user }}/jenkins/jdk/bin/java, even if ~/jenkins does not exist yet at this point. This could be because jenkins didn't connect to the node yet, or because the jenkins dir isn't set to ~/jenkins.
The previous logic failed when there are no artifacts for archiving, for multiple reasons. Fix all of them:
* "allow-empty" needs to be set, so jenkins doesn't fail the job when it can't find any artifacts to archive. * The "artifacts" string must not be empty even if "allow-empty" is set, as otherwise it assumes a configuration error. * The jenkins jobs fails in the archiving step if the workspace dir doesn't exist.
Skip tests requiring physical cards ("card-test") for commits having a special marker in the COMMIT_MSG:
Jenkins: skip-card-tests
This speeds up the build verification significantly, and may be useful for patches that do not modify the core logic (such as documentation improvements and misc fixes).
jobs/master-builds.yml: add missing job type 'distcheck'
We do have this job type in gerrit-verifications.yml, so it's executed for each patch submitted to Gerrit. However, we are not executing 'distcheck' during the daily master builds - fix this.
I've already implemented this logic in 6a3b8a0, however by mistake I added it to master-builds.yml instead of gerrit-verifications.yml. This is wrong because we specifically need this feature for Jenkins verification, not for the master builds.
For daily master builds, on the other hand, we specifically want to execute all existing job types, so revert my recent changes from master-builds.yml, but keep the new job type.
Which in turn is the result of "apt update" failing to download files in an earlier job:
E: Failed to fetch https://downloads.osmocom.org/packages/osmocom:/nightly/Debian_13/./Packages.gz File has unexpected size (210767 != 210491). Mirror sync in progress? [IP: 148.251.236.141 443] Hashes of expected file: - Filesize:210491 [weak] - SHA256:a0cdea838b724880f455d68eb433f6331a7d4113c2eb677ca2f7f6dbdf9a72e1 - SHA1:71780896257ca550e51b95e77fee4c63fcc9980c [weak] - MD5Sum:531a2d49d4e9f3ff121ca6aa5be602fc [weak] Release file created at: Sun, 26 Apr 2026 08:48:04 +0000 E: Some index files failed to download. They have been ignored, or old ones used instead.
#define DEFUN_CMD_FUNC_TEXT(funcname) \ static int funcname \ (struct cmd_element *self, struct vty *vty, int argc, const char *argv[])
So the pointer to argv is const, but the elements in the array are not. Functions can in theory modify the elements of argv (just like argv in main()). I don't know if we have users of this API that are actually doing that, but as long as the API is defined like this it doesn't make sense for the linter to complain about using it correctly. Therefore, ignore STATIC_CONST_CHAR_ARRAY.
Run the start-hook.sh / stop-hook.sh scripts that store pcscd logs and a usbmon capture. The jenkins node is configured to allow running these two scripts with sudo as jenkins user. Store the resulting files as artifacts.
When the server returns an error, the curl command does not fail - probably because the server sends a wrong http status code:
curl \ --form token="$token" \ --form email=holger@freyther.de --form file=@Osmocom.tgz \ --form version=Version --form description=AutoUpload \ https://scan.coverity.com/builds?project=Osmocom + set +x % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed … <html> <head><title>413 Request Entity Too Large</title></head> <body> <center><h1>413 Request Entity Too Large</h1></center> <hr><center>nginx/1.19.10</center> </body> </html> Finished: SUCCESS
Fail if the output doesn't contain the "Build successfully submitted" string, so the job fails if there was an error.
(The request entity too large error is probably an error on their end, scan.coverity.com still says they are performing upgrades on their servers.)
When running the linter against dahdi-linux.git or potentially other repositories with out-of-tree linux kernel modules, it should not complain about the LINUX_VERSION_CODE being used there:
> drivers/dahdi/dahdi-base.c:6397: WARNING:LINUX_VERSION_CODE: > LINUX_VERSION_CODE should be avoided, code should be for the version to > which it is merged
Set wipe-workspace for the onomondo-eim and onomondo-ipa jobs to true, so previous failures cannot influence new jenkins job runs. E.g. currently we had gerrit-onomondo-ipa-build failing with:
Most existing jobs call scripts/osmo-clean-workspace.sh from osmo-ci.git at the beginning of their jenkins.sh instead of using wipe-workspace. But this doesn't have any advantage here while making jenkins.sh more complex. Let's use the built-in feature from jenkins here.
