Commit
9f2b10968915cca7b7e44b1834321908b18a58b3
by Vadim Yanitskiy
OBS: add tag_pattern(), make tag patterns stricter
* add a convenience helper to avoid regexp body duplication * r'[0-9]*' is not strict enough, we want one or more digits * replace r'[0-9]' with r'\d' to make statements a bit shorter
Commit
8dd8e811961771bcf81066c5baba1fe36b856b32
by Oliver Smith
registry-rebuild-upload-testenv: new job
Add a job that updates the one podman image used by ttcn3 testsuites with testenv to the registry. The new testsuite jobs will not build their own images anymore, just pull this one at the start.
Commit
3700f45f4d49903a12820761c7e47d06489df118
by Oliver Smith
jobs/ttcn3-testsuites-testenv: initial
Move the first testsuites over from the old docker-playground based architecture of running testsuites to testenv. I'm doing this iteratively to avoid regressions.
Commit
5b75d6be44107b02b9ea7057239f1a3aee0361ce
by Oliver Smith
jobs/osmocom-depcheck: disable mail notifications
This is a manually triggered job that users may use if they want to verify minimal versions of dependencies mentioned in configure.ac result in a successful build. Don't notify the mailing list about it.
Commit
8fe1efa6d4441e655d477e8e5837592e0d6d910c
by Oliver Smith
ansible: podman: use fuse-overlayfs
The default driver "vfs" leads to container crashes on heavy i/o load on our jenkins machines. Use "fuse-overlayfs" instead, which does not have these problems.
Add a systemd service that creates /dev/fuse when the LXC container starts and configure the driver via ~/.config/containers/storage.conf.
Note: when changing the storage driver, the previous storage database needs to be removed (~/.local/share/containers/storage/). I've done this manually before rolling out this change.
Commit
c98cd9bcee9677db08817d74fffca01be1c3c641
by Oliver Smith
jobs/ttcn3-testsuites-testenv: add s1gw
Remove the docker-playground based job and add a new testenv based job instead. Add one for -latest too as we recently tagged the first releases.
Do not limit the job to a specific node, looks like this was pasted from the above job by mistake. I've successfully ran this on another node build2-deb11build-ansible.
Commit
cabaaa9e7a4455cb9763eba303f9ec3680f28a35
by laforge
jobs/{gerrit,master,update-osmo-ci}: add aarch64 case
The newly set up rpi4-raspbian11build-ansible-1 returns aarch64 for arch (it has an aarch64 OS instead of 32-bit arm like the others currently have). Add "aarch64" to the cases next to arm*, so these work as expected again.
Commit
914b58cb79f85a3956b7c38fc51b143864a331e6
by laforge
jobs/{master,gerrit}: rpi4-raspbian11 -> raspbian
Rename the jenkins label rpi4-raspbian11 to raspbian so the name is not misleading with how we currently use it. One raspberry pi has been reinstalled with raspbian12 (OS#6549) and we also use it for LX2 with a raspbian container (OS#5873, therefore remove rpi4 from the name).
The OS running directly on the rpi / in the LXC is not that important anyway since we run the actual jobs in docker with the latest debian version (currently 12).
Commit
c0fd46590420d57636a381c8b113a1fb8d1ecd40
by laforge
jobs/{gerrit,master}: aarch64: use setarch linux32
Pretend to be a 32 bit arm machine when running the arm jobs, so in combination with the related commit we can build test for 32 bit arm on rpi4s with a 64 bit raspberry pi OS.
Commit
3a26fd299b052858a64d6243492a95c76e940a97
by laforge
Stop building openbsc
With recent changes in libosmo-sccp, openbsc does not build anymore. The openbsc.git repo has been unmaintained/unsupported for 7 years, so let's finally remove related build jobs instead of fixing this.
Commit
b2e064a44ebf59bdacf4bc5d8ac890625ce6cb4f
by Oliver Smith
jobs/osmo-gsm-tester-build: fix Missing parameter
Use OSMO_GSM_TESTER_BRANCH as shell variable ('$...'), not as jenkins job builder parameter (${...}). This is how we do it in other jobs as well, e.g. jobs/osmocom-obs.yml.
Apparently the workaround in the file happened to work earlier, but it does not work anymore with Jenkins Job Builder version 6.3.0. Remove it.
Commit
2410e86b2a832f1345445f700df1a665f9394397
by Oliver Smith
libosmo-sccp -> libosmo-sigtran
Adjust to: * libosmo-sccp.git has been renamed to libosmo-sigtran.git * libosmo-sccp static libraries have been split into libosmo-sccp-legacy.git
build-old-tags-against-master: libosmo-sccp is intentionally kept in REPOS_TAGS, so old libosmo-sccp versions are built against current master of Osmocom libraries.
Various Osmocom components now include support for this family of Themyscira specs, but the linter outright rejects any and all patches that have a legitimate need to mention this acronym.
Commit
4c773bee97fad8a05f59ec4f8ca0e760fb0e7525
by Oliver Smith
jobs/osmo-gsm-tester-runner: fix syntax errors
Fix the syntax of this file, so it is compatible with Jenkins Job Builder 6.3.0.
* In shell blocks, {{ and }} needs to be used instead of { and } if the string in brackets does not refer to a JJB variable. * "${OSMO_GSM_TESTER_BRANCH}" would refer to a non-existing JJB var, instead we want the shell variable here (that jenkins sets), so omit the {}.
Commit
68ddbceeeb9fd47e001021d02e13eb1f114e4242
by Oliver Smith
jobs/gerrit: osmo-ttcn3-hacks: tweak make lines
* The "make clean" is not needed, as the osmo-ttcn3-hacks.git repository is a fresh clone of code that was submitted to gerrit. * The "make deps" is not needed, as "make compile" depends on the deps target. * Combine the cd and make lines. * Remove ; at the end of lines, as it is not needed * Switch " and ', so variables can be used with less escaping in future patches in this block.
Commit
65c3275be06603a29311ba2160ffbb4500b1c206
by Oliver Smith
jobs/gerrit: use deps cache from docker image
Instead of freshly cloning the dependencies each time, use the cache we already have in the debian-bookworm-titan docker image. This way only the dependency repositories that were changed in the patch submitted to gerrit need to be updated.
Commit
9679c5d20778cba7e479e3e5bd75eac7cbb4f85f
by Oliver Smith
OBS: build_srcpkg.Dockerfile: add sphinx-common
Install sphinx-common, as it is needed to build the source package for pyosmocom.
Fix for: dh clean --with python3,sphinxdoc --buildsystem=pybuild dh: error: unable to load addon sphinxdoc: Can't locate Debian/Debhelper/Sequence/sphinxdoc.pm in @INC (you may need to install the Debian::Debhelper::Sequence::sphinxdoc module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.36.0 /usr/local/share/perl/5.36.0 /usr/lib/x86_64-linux-gnu/perl5/5.36 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.36 /usr/share/perl/5.36 /usr/local/lib/site_perl) at (eval 13) line 1. BEGIN failed--compilation aborted at (eval 13) line 1.
Commit
072b1eae88a89bba4585b38ef0e47ccc6c60bea9
by Oliver Smith
manuals: publish pyosmocom manuals for releases
Adjust the building and publishing commands to work for pyosmocom. Remove the 0.0.3 release from TAGS_IGNORE, because it already has manuals.
Skip "apt-get -y build-dep" and adjusting "debian/control" if that file does not exist, as it is the case with the 0.0.3 and 0.0.4 releases of pyosmocom.
Commit
a4ba4ea461db88f739c73dbdce4800b1d40ecf9e
by Oliver Smith
jobs/master-builds: add param BRANCH
Sometimes it is desirable to run master jobs from a different branch: * As discussed with Philipp: running pysim CI on a branch with many patches once before submitting to gerrit (it takes much longer in gerrit to run on each individual patch) * When adding/modifying master jobs to test if they still work as expected with additional patches in a branch.
It is undesirable to have mail notifications if not running on master, the next patch will make it easy to disable those.
Commit
7a80d1597a506b3de5839d0c169ff07007aec3fc
by Oliver Smith
jobs/build-kernels-testenv: new jobs
Prepare to replace the docker-playground based jobs for running osmo-ggsn in QEMU with a kernel built from source with testenv.
Split the part that builds the kernels into extra jobs that export the built kernels as artifacts. This makes it easier to use kernels outside of jenkins too and it allows retriggering only a testsuite run without also building the kernel again.
The fragment.config is imported from docker-playground current master 978adc, path: ttcn3-ggsn-test/osmo-ggsn-kernel/fragment.config
Commit
85eb09d778ee0ec12a2d09e46904662bde9c99a7
by Oliver Smith
jobs/build-kernels-testenv: add build-kernel-none
Create a dummy output/linux file, that the jobs generated by ttcn3-testsuites-testenv can copy (via copyartifact) when setting the kernel to "none".
Without this, the ttcn3-testsuites-testenv.yml would need to become more complex, as it is not possible in jenkins job builder templates to only do the copyartifact block conditionally if kernel != "none".
Another solution would be using shell logic like the following:
if [ "$KERNEL" != "none" ]; then wget -q -O .linux "https://jenkins.osmocom.org/..." fi
But this has the disadvantage that jenkins is not aware of the file and therefore we cannot track easily which linux kernel build was used in the job. With the copyartifact method, we can click on "See Fingerprints" and find which job built the kernel. Example: https://jenkins.osmocom.org/jenkins/job/ttcn3-ggsn-test-kernel-net-next/147/fingerprints/
Commit
660f83108aa90bb6161ddda48a37782e091a8d38
by Oliver Smith
jobs/osmo-gsm-tester-runner: use debian bookworm
* Use bookworm during clean up instead of bullseye * Don't build debian-buster-jenkins anymore, the osmo-gsm-tester container now uses debian-bookworm-build which gets deployed via the update-osmo-ci-on-slaves job (see commit in Depends:)
Commit
c0046ab4a5aa97a05b2d067702fad9586429ec52
by Oliver Smith
jobs/osmo-gsm-tester-builder: build in docker
Build Osmocom programs in docker (debian bookworm), so they link against libraries available when trying to run the programs later on.
Without this we get errors like to following when osmo-gsm-tester-virtual runs in docker with debian bookworm (after recent change, before that it used debian buster): osmo-msc: error while loading shared libraries: libasan.so.5: cannot open shared object file: No such file or directory
Commit
0023c20cc06373cbbac43ba70b055fecad491892
by Oliver Smith
jobs/ttcn3-testsuites-testenv: tweak description
Don't link to osmo-ttcn3-hacks.git directly anymore, the wiki page that is also linked already refers to it. This shortens the boilerplate of the description with every one of these jenkins jobs.
Instead of only displaying the arguments, display the full command so users can easily run the testsuite the same way it is running on jenkins. Omit the arguments --cache, --ccache, --log-dir, which only need to be adjusted for jenkins, for users the defaults make sense.
Let the kernel line link to the related build-kernel job.
These testsuites have been disabled for > 3 years, and the related SUT is not developed any further. Remove the entries to unclutter the list for now, we could add them back later easily if it makes sense.
Add the current "last duration" time from the jenkins jobs as comment, so we have a rough idea of how long these jobs take and can distribute them more evenly throughout the days. Even though the durations will change over time, this gives a good indicator of whether a job runs a few minutes or 1 or even 2 hours.
Commit
9abdfe5307e4a682278a4174e73dcb0512a61dd5
by Oliver Smith
scripts/osmo-ci-docker-rebuild: fix rpi4 failures
It is no longer possible to pull the arm32 image by just using "docker build --pull". Use a separate pull command with --platform and set an env var to not use --pull with "docker build".
Commit
0ba4ce01b652c74832817ddd8eb893abc791c3a5
by Oliver Smith
build-old-tags: ignore failing due to api break
Ignore osmo-bsc and osmo-msc versions that are known to fail due to intentional API break. osmo-bsc 1.12.2 and osmo-msc 1.12.2 build are forward-compatible.
Note that previous releases don't need to be listed in tags_to_ignore() as this script only builds the last 3 release tags.
Store the apk after successful run of the android-apdu-proxy job. It can be found by clicking on a job at [1] and then clicking on "default" (since the job is behind a matrix, as all master jobs).
Commit
89a3b3f049c13c8d7e3ba17e305990fc9ea21f3d
by laforge
jobs/osmocom-release-android-apks: new job
Add a job similar to the release tarballs job, that iterates over the tags of the android app repositories (currently only android-apdu-proxy) and builds/uploads an apk for each of them.
Commit
b7e67c1df52050ebaa23437db8dc405dd223c446
by Oliver Smith
jobs/ttcn3-testsuites: adjust timer for bts
BTS testsuites tend to fail with timeouts on heavy loads. Group them in a timeslot where jenkins nodes are otherwise not busy according to stats in grafana to hopefully make the test results more stable. Once we can replace faketrx with a rewrite (OS#6672), this should not be necessary anymore.
Commit
0ba4dac82b83a8e6515c75e1b792014375595f37
by Oliver Smith
jobs/master,gerrit: use io_uring seccomp profile
Use the seccomp profile not only in ttcn3 testsuites, but also in master-builds and gerrit-verifications so we can test io_uring there too.
CI happened to work without this for libosmocore on build4 because of a specific docker version where io_uring was not yet (fully?) disabled in the default seccomp profile. It did not work without this patch on build5 where we currently have a newer docker version.
Commit
ef498e7efe3a5801bbfbf4b6e90a4e54bb3ee6e5
by Oliver Smith
jobs/{master,gerrit}: set ulimit filesize to 1 GiB
Set a max file size limit of 1 GiB via ulimit. This prevents jobs from filling up the disk by generating one huge log file.
I have chosen 1 GiB as limit, because jobs that clone linux kernel repos may have packed git artifacts that are several 100 MiB in size.
It would be even better to set the overall max disk space that can be used from one "docker run" call, but this is currently not supported for ext4: https://github.com/moby/moby/issues/29364
Commit
a71ccaa13598d4d5627b4e16e497f4d7d076b53e
by Oliver Smith
jobs: manually trigger "git gc" for linux.git
We already use shallow clones for linux.git, and git runs a garbage collector automatically to ensure no longer needed artifacts are removed. But this did not work anymore on build4 for build-kernel-net-next, and it increased up to 8.4 GiB:
8,4 GiB /build-kernel-net-next
After running "git gc" again:
3,3 GiB /build-kernel-net-next
The reason was that "git gc" did not exit successfully once and wrote a log file, then refused to run again. Run "git gc --auto" explicitly in foreground to let abort in such a case, so it does not go unnoticed in the future.
Commit
fa7a94dd4e75e8ead20d4cc5d1ed79a64a85f4ff
by Oliver Smith
scripts/docker-cleanup: add "docker system prune"
On build4 this freed up another 8.223GB when I just ran it:
$ docker system prune WARNING! This will remove: - all stopped containers - all networks not used by at least one container - all dangling images - all dangling build cache
Commit
7b4109ff906dd933d0db425bf2fb686740046646
by Oliver Smith
ansible/setup-jenkins-slave: add mingw job deps
Support running the legacy job Osmocom-OBS_MinGW_weekly_publish by installing its dependencies directly in the LXC. This is a workaround until we modernize the job, see the related issue.
Commit
e22ca0edcf020d1eca4cf20a729e50faa13cb9e7
by Oliver Smith
ansible: update build5-obsworker ssh key
I've reinstalled the build5-obsworker lxc after first upgrading to openSUSE 15.6 and then discovering that the lxc version is incompatible with obs-build: 15.6 doesn't have lxc 4 anymore, but a higher version is not supported by obs-build. I've added a big warning in the wiki so we don't run into this again until obs-build has been adjusted.
Commit
31db96c312b953afc26ae4b16b266e367a0fdcef
by Oliver Smith
lint/lint_diff: run clang-format if config was found
The osmo-asf4-dfu and osmo-ccid-firmware now have clang-format configs. Adjust the lint script to run clang-format if such a config is found in the repository for which the lint script is running.
Commit
3dffb34587dc0bba9ce46c7ef0d7e07c8167a8d4
by Oliver Smith
lint/lint_diff: allow skipping with OSMO_LINT=0
When this script is set as pre-commit hook, it can be useful to skip the linter with this env var to make a commit that intentionally does not pass the linter (e.g. to test if it will be caught in gerrit + jenkins).
There is also "git commit --no-verify", but with that the hook that applies the Change-Id does not run.
Commit
9a18ebaa78fc255a0aea42186a2743829d69162e
by Oliver Smith
obs: gerrit_binpkgs: set --allow-unknown-package
The gerrit_binpkgs.sh script can be used to quickly test building of debian packages in a container even for packages that we don't have in the OBS osmocom repositories. For example I'm currently using it to test-build kamailio packages. Use the --allow-unknown-package argument so build_srcpkg.py doesn't complain if this gets called with a package that isn't a known Osmocom package.
Commit
e28454ea4b120a7ab96e133589a7b7b3ee8c0846
by Oliver Smith
*.py: ignore: E722 Do not use bare `except`
Let the ruff linter not complain about the existing cases of "Do not use bare `except`" so we get the linter to pass without changing the existing logic here. With this we don't disable the check globally, so we can consider making stricter excepts in new code.
Commit
c8ec5d58507718165b863c36b51e29e9afd1a4a1
by Oliver Smith
scripts/verify_log_statements: fix undefined name
Append it to "errors_found" instead of undefined variable "ignored". This makes it work similarly to further above:
# Skip 'LOG("bla" fmt )' strings that typically appear as #defines. if fmt_re.match(quoted): if debug: errors_found.append(error_found(f, log.start(), 'Skipping define', log.group(0))) continue
Fix the following bug:
scripts/verify_log_statements.py:92:11: F821 Undefined name `ignored` | 90 | # hopefully eval broke because of some '## args' macro def 91 | if debug: 92 | ignored.append(error_found(f, log.start(), 'Ignoring', log.group(0))) | ^^^^^^^ F821 93 | continue |
Commit
b97a04f431503a5aa0a0f07a68910153c13721be
by Oliver Smith
jobs/osmocom-build-tags…: set seccomp profile
Set the seccomp profile to make io_uring work in docker, so the regression test in libosmocore using io_uring does not fail. This happened to work previously on the build4 jenkins node because of an older docker version.
Commit
01602b16c59838be23259762a5c1d99a0240e764
by Oliver Smith
OBS: cleanup: new script
Add a script for cleaning old sources uploaded to OBS. This was needed as we were running out of space on the OBS server, due to source tarballs being generated in nightly jobs (e.g. Osmocom_OBS_nightly and never getting deleted). This script is now running automatically every day.
The script does the following: * Iterate over all OBS projects and packages inside the projects. * Delete all source files matching *.dsc, *.tar.xz in old revisions from disk, which are not in the most recent revision anymore. * Create an empty file instead (as otherwise OBS becomes unstable) * Write a marker file with the revisions already iterated through (per project+package) and skip those on the next run.
Since osmo-pcap.git Change-Id I72e8a6ceb4fb1eb70372e13bb139ead0e2bc0860, osmo-pcap-client uses osmo_stream which in turn uses osmo_io (which can u io_uring backend). Hence, add io-uring backend to the test matrix for this project.
Commit
06b815c32d7a639e716f657720795baff2bb4c9c
by Oliver Smith
jobs: free up runners from 01:00 - 03:00
When looking at grafana, the build hosts (build4 and build5) are not busy during 00:00 and 03:00. Binary packages seem to be finished at 00:00 already. Shorten the time for "Jobs that don't need binary packages" by two hours to make room for running ttcn3-bts-test with less load in a follow-up patch.
Commit
e5493058eb750818c1460346c92bd7700255d2a1
by Oliver Smith
jobs: run ttcn3-bts-test-* between 01:00 - 06:00
The automatic respawns of osmo-bts-trx have been intentionally removed in the related docker-playground patch, so we can better notice when osmo-bts-trx crashes instead of just restarting it and moving on with the testsuite.
The result is that now it is even more important when ttcn3-bts-test jobs are running, because when the builders are under high load then faketrx cannot send the time quickly enough and it also causes osmo-bts-trx to shutdown.
In a discussion with Vadim we've decided to schedule the ttcn3-bts-tests so that they don't only run at a time where no other jobs are running. But also that they don't run at the same time on the same builder anymore to give them the least unrelated load possible. Other solutions are also being investigated, but this should make the job in its current form more stable already.
Move misc ttcn3 jobs to run from 06:00 to 08:00 and run ttcn3-bts-tests between 01:00 and 06:00. Pin them to specific runners to make sure we don't have both running on the same node.
Commit
88bc9b1e07c1f31c410780288658cbe0c4076737
by Oliver Smith
ansible: build-hosts: add testenv-coredump-helper
The Osmocom jenkins nodes run inside LXCs. When we get a coredump it appears on the host. Add a helper script to the hosts so the jenkins jobs can fetch the coredumps in case an Osmocom program crashes while running a ttcn3 testsuite.
The helper script has the following safety features to ensure jenkins can't just fetch any coredump:
* Only fetch coredumps within the last 3 seconds and only if the executable matches osmo-* or open5gs-* * Only listen on the lxc IP
Commit
56e77437c64f252765302ee9eb5739da21e9af2d
by Oliver Smith
jobs/master: pass osmo-ci to docker
Let jobs such as the upcoming one for osmo-rp2040-firmware use the known_hosts file from osmo-ci when uploading binaries. In other jobs we've copied the known_hosts into the contrib/jenkins.sh file, but let's try to have it in a central place at least for new jobs.
Fix that installing openjdk-21 as builddep of eclipse-titan fails with: Setting up openjdk-21-jre-headless:amd64 (21.0.7+6~us1-0ubuntu1~24.04) ... update-alternatives: using /usr/lib/jvm/java-21-openjdk-amd64/bin/java to provide /usr/bin/java (java) in auto mode update-alternatives: error: error creating symbolic link '/usr/share/man/man1/java.1.gz.dpkg-tmp': No such file or directory
Commit
9db271c7b325fd57e84adaaedba8644b2db03d23
by Oliver Smith
OBS: gerrit_binpkgs: add FEED parameter
Allow switching to another feed easily. I'm using this to build packages for ubuntu quickly, for which we don't have a master feed (only for debian 10, 11, 12 currently).
Commit
3b348d5effcd8b6cfde611b00473dd37545cebbb
by Oliver Smith
OBS: remove pdf draft watermark for latest pkgs
Add the line ASCIIDOCSTYLE = $(BUILDDIR)\/custom-dblatex.sty before the line include $(OSMO_GSM_MANUALS_DIR)/build/Makefile.asciidoc.inc when building source packages for the latest feed, so there is no "DRAFT" watermark.
Commit
9f5b48a2978e652671a51b6511a068e05798cffc
by Oliver Smith
scripts/manuals: use new method for removing DRAFT
osmo-gsm-manuals now has two "ASCIIDOCSTYLE" files: * custom-dblatex.sty (without draft) * custom-dblatex-draft.sty (new default)
This change was necessary to make it feasible to build the manuals inside the debian binary packages without the watermark for the osmocom:latest feed (see next patch).
Adjust the method for removing the "DRAFT" watermark: instead of patching custom-dblatex.sty, switch from custom-dblatex-draft.sty to custom-dblatex.sty.
Commit
f7e3533392ef98ba46ed1e1b4ac3a60363de3bd2
by laforge
jobs/ttcn3-testsuites-testenv: no rm on error
Do not delete the cache dir if testenv returns with error, so we can inspect it. Keep the linux kernel too, it doesn't make much of a size difference and makes the code slightly simpler.
Commit
d0c3e3bf9bb540cd82f71b2a47c109b3889c401a
by Oliver Smith
OBS: update_obs_eclipse_titan: update
* Set the branch as argument instead of hardcoding it. * Don't use --version-append, the ~osmocom part will be added to debian/changelog instead when pushing new versions. * Add usage description.
Commit
43c09c91818d9c62e60896c3680b1bea05a48604
by Oliver Smith
OBS: add the new osmocom-keyring package
The public key for OBS packages needs to be extended every two years. When the previous key expires, it is not possible to update packages anymore until users replace the old key with the new, extended key.
To make this less inconvenient for users, introduce a new osmocom-keyring package that holds the current key (valid for 2 years), which will get updated every year with an extended key (valid for another 2 years, I'll add a jenkins job to remind us). This means as long as users upgrade their packages once a year, they will automatically receive the new key.
Commit
baf71c6784a5ad9b4508396d5e467555f2e5e520
by Oliver Smith
OBS: let meta package depend on osmocom-keyring
Let the meta package (osmocom-latest, osmocom-nightly etc.) depend on osmocom-keyring. This means that after the user has enabled any of the official Osmocom OBS repositories and installs a package from there, the osmocom-keyring package will get installed as well, which results in having an up-to-date /usr/share/osmocom-keyring/osmocom.asc.
Commit
8b28d63e980d051e5d39893549fe985218bae28f
by Oliver Smith
jobs/reminder-extend-obs-key: new job
Add a script that downloads the current OBS pubkey, checks its expiration date and fails if it is in less than a year. The idea is to extend it for two years every year (so users have enough time to update their osmocom-keyring package).
Commit
af02d1fd3d6c840df49cc7dce2f8e59cf15c4f58
by Oliver Smith
jobs: make references to master branch consistent
Use "master" instead of: - "refs/remotes/origin/master" - "origin/master" - "*/master"
This improves consistency and makes clear what should be used when adding new jobs. It also fixes the problem with */master that it may actually not only use the master branch, but also e.g. osmith/master which can lead to unintentionally building and publishing artifacts from the wrong branch. If we want to run a job on a different branch intentionally, such a job should have an extra argument for branch that can be changed when starting the job.
Commit
3d1e68dd2303c19130e13a192d6d6bc72a6d4841
by Oliver Smith
repo-install-test: ignore eclipse-titan-optdir
The osmocom:latest repository now has eclipse-titan-optdir-9.0.0 and eclipse-titan-optdir-11.1.0 packages, which allow installing multiple titan versions side-by-side. The related -dbgsym packages can conflict with the main eclipse-titan-dbgsym package. Ignore these packages in the repo-install test.
Fixes: Unpacking eclipse-titan-optdir-9.0.0-dbgsym (9.0.0~osmocom1.2263.c685) ... dpkg: error processing archive /tmp/apt-dpkg-install-4XXdZ1/205-eclipse-titan-optdir-9.0.0-dbgsym_9.0.0~osmocom1.2263.c685_amd64.deb (--unpack): trying to overwrite '/usr/lib/debug/.build-id/0a/072e12f92a7b564adec3a0896c18aa6c942391.debug', which is also in package eclipse-titan-dbgsym 9.0.0.3.0bbfc~osmocom
Commit
1262888d49057d212a16a29506e8a30b79202510
by Oliver Smith
jobs/registry…: fix syntax error
Use $REGISTRY instead of ${REGISTRY}, so jenkins-job-builder 6.3.0 doesn't interpret it as JJB variable. It needs to be interpreted as shell variable here. This probably worked with older JJB versions.
Commit
80a9abae512b4f28c7531c27bb91ea8757bb1e72
by Oliver Smith
contrib/obs-mirror/obs-mirror-include: update
Update the daily backup/mirror configuration.
* Add Debian 13 * Add Raspbian 12 (not sure why it was not there yet) * Add nightly/CentOS_7 (we build a few packages for it and we also have the mirror configured for latest) * Remove CentOS 8 (dropped 1 year ago: https://osmocom.org/news/255) * Remove Debian 10, Raspbian 10 (https://osmocom.org/news/308)
Commit
f071c697887247030b8ec43c1a9dc2d873984d35
by Oliver Smith
OBS: support building debian trixie (13) pkgs
Translate "debian:13" to "debian:trixie", as for some reason no docker tag is getting pushed for "debian:13". With this we can keep using the usual short naming scheme in pipeline_binpkgs lines in gerrit-verifications.yml (see next patch).
Commit
5c0b9984b437759fea55fb9101ccde49acb27d25
by Oliver Smith
jobs/gerrit: binpkgs: update debian versions
Change the default from "debian:10 debian:12" to "debian:11 debian:13", as Debian 13 has just been released and we are about to remove Debian 10 binary packages.
Commit
6d19e82eedcd4852cce3b43482ab54f1df195ef2
by Oliver Smith
OBS: sync_obs_projects: support security.d.o
The Debian 13 meta config in the OpenSUSE OBS now lists security.debian.org. Adjust the script that syncs the config to support this URL too, rewrite it to HTTPS just like the other mirror URL.
Fix for: changing url to https: http://security.debian.org/debian-security?dist=trixie-security&component=main Traceback (most recent call last): File "/obs/sync_obs_projects.py", line 322, in <module> main() File "/obs/sync_obs_projects.py", line 312, in main rewrite_meta(project) File "/obs/sync_obs_projects.py", line 190, in rewrite_meta assert url.startswith("http://ftp.de.debian.org/debian"), \ AssertionError: unexpected mirror URL
Commit
bc5dbb3093b47a5dfdca8f956d85e2cd5a7bcba1
by Oliver Smith
jenkins-gerrit: binpkgs: ignore decode errors
The output of recent osmo-s1gw jobs for building binary packages contains invalid utf-8 characters, even though it should be utf-8 according to the HTTP header: content-type: text/plain;charset=utf-8
We only need to get the distro name from the output here, so simply ignore any decode errors.
Fix for: $ ./comment_generate.py https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-s1gw/417/ -o gerrit_report.json -t result Reading https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-s1gw/417//consoleText Reading https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs/17255/consoleText Reading https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs/17260/consoleText Traceback (most recent call last): File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 270, in <module> main() File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 257, in main comment = get_comment_result(args.build_url, args.notify_on_success) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 198, in get_comment_result pipeline = parse_pipeline(build_url) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 89, in parse_pipeline stage = stage_from_job_name(job_name, job_url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 59, in stage_from_job_name return stage_binpkgs_from_url(job_url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 46, in stage_binpkgs_from_url content = response.read().decode("utf-8") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xef in position 237446: invalid continuation byte
Commit
cf88fab8cda55362233afbbd54aba0b99a263d44
by Vadim Yanitskiy
jobs/gerrit-verifications: osmo-ttcn3-hacks: do make clean
It's easy to break the build verification [for everybody] by submitting a patch that creates a symlink to a non-existent file. The workspace is not wiped, so the stale symlink remains and results in failures like this:
Commit
bf3ef9d95af939d581f10f8a35e952e1a8131b21
by Oliver Smith
OBS: --version-append: never override version
We have some branches set up that when pushed to, directly result in building OBS packages. We append a string to the package version there, such as ~rhizomatica.testing. The expectation is that all packages built from that branch have this string in the version.
Recently an edge case was found: when amending the commit of the last tagged release, e.g. osmo-sgsn 1.12.0, then the generated version is based on the previous git tag (as the last git tag's commit is not part of the current branch anymore) while having the newer version in debian/changelog. This then results in the newer version being used, without appending the version string:
osmo-sgsn: WARNING: version from changelog (1.12.0) is higher than version based on git tag (1.11.1.20-e000e~rhizomatica.production), using version from changelog (git tag not pushed yet?)
While IMHO it is a bad idea to just amend the commit of the last release instead of making a separate commit, this results in unexpected behavior so adjust the script to not override the version if --version-append is used.
Commit
ffef33c677f7bf9c76a99a6dfa7ab7eb29a49c23
by Oliver Smith
jobs: run ttcn3-remsim-test-asan with bookworm
The default distribution to run ttcn3 testsuites and SUTs with has recently been changed from debian 12 (bookworm) to 13 (trixie). Since then we have seen 27 additional failures with osmocom:nightly:asan (while osmocom:nightly and osmocom:latest work fine). As the reason for this is currently unclear, revert to using debian 12 for this job for now, so this doesn't mask other errors that the job would find otherwise.
Commit
60bf602f6f8b0f4d06329b14767c4c49f8d42ded
by Oliver Smith
OBS: disable_manuals: adjust for osmocom-bb
With the related change, we build manauals in the debian package of osmocom-bb. The osmocom:nightly:asan feed gets built with manuals disabled. Fix that building the osmocom-bb package in the asan feed currently fails with:
[ 160s] /usr/bin/make -C doc/manuals [ 160s] make[2]: Entering directory '/usr/src/packages/BUILD/doc/manuals' [ 160s] OSMO_GSM_MANUALS_DIR: /usr/src/packages/osmo-gsm-manuals (fallback) [ 160s] ERROR: OSMO_GSM_MANUALS_DIR does not exist! [ 160s] Install osmo-gsm-manuals or set OSMO_GSM_MANUALS_DIR. [ 160s] Makefile:6: /build/Makefile.asciidoc.inc: No such file or directory [ 160s] make[2]: *** No rule to make target '/build/Makefile.asciidoc.inc'. Stop.
Commit
bd2cf1f881aea5dd5ca4a0f8822c5467762d1166
by Oliver Smith
OBS: stop building limesuite
Building limesuite is not needed anymore, as it is available at least since debian 10 in the debian and ubuntu repositories. We have been building the upstream version without changes.
Commit
77eaf6179803cf8c9bd71df39beb375e46205a52
by Oliver Smith
testenv-coredump-helper: remove
After looking at this again, it makes more sense to keep /proc/sys/kernel/core_pattern as "core" on build4 and build5, which we use for jenkins jobs instead of installing systemd's coredumpctl and making all coredumps go through that. The motivation for using coredumpctl in testenv was that lots of developers may have it already installed (though I'm not sure about that anymore), and that it gives a nice API for retrieving the related executable name to a coredump instead of having to parse the output of e.g. "file".
Using coredumpctl had the disadvantage that coredumps for all other jobs, such as the master-builds are also caught by that and cannot be easily placed in the workspace directory. I have started implementing this by extending testenv-coredump-helper to communicate through a socket and mounting that socket inside docker containers running contrib/jenkins.sh (host -> lxc -> docker), and it works, but this complexity is not useful here.
Instead the related patch will make testenv pick the core files when coredumpctl is not available, and read the path from "file", making testenv-coredump-helper obsolete.
Commit
dfa9fcca49ae8b8ce6c691196c3883206933e503
by Oliver Smith
scripts/wrapper_core_bt_on_error: new script
Add a wrapper script that runs a given program, and checks the exit code. If it is not 0, and a core file appears, then load the coredump in gdb and display its backtrace.
Now that it is also possible to run the HLR tests against PyHSS, add "-c osmo_hlr" to run the existing ttcn3-hlr-test jobs against osmo-hlr. I plan to add jobs for PyHSS later in a separate patch.
Commit
b93642c7c66e94ea5532720bc222393505b24f4e
by Oliver Smith
jobs/ttcn3-testsuites-testenv: mme: use -c all
Adjust to MME tests now having two configurations in osmo-ttcn3-hacks:
[testenv] Found multiple testenv.cfg files, use one of: [testenv] -c emergency [testenv] -c generic [testenv] You can also select all of them (-c all) or use the * character as wildcard.
Commit
46075b796ded3c014346fe5dcc18a6082a792f89
by Oliver Smith
OBS: let osmocom-keyring not depend on other pgks
The osmocom-latest, osmocom-nightly etc. meta-packages depend on the osmocom-keyring package. Fix that the osmocom-keyring package was also depending on these meta packages.
This fixes that when uninstalling all osmocom:latest packages by removing the osmocom-latest meta package, the osmocom-keyring package would also get removed. This is undesirable for switching between latest and nightly, as the osmocom-keyring package must remain installed to be able to install new packages from the repository (as it has the repo key).
Commit
ce3a6cb2b1ad650e19b9a6b38453c02128db5cf5
by Oliver Smith
jobs/osmocom-obs: build binary packages earlier
We have recently noticed that with current settings the nightly arm packages have still not finished the next morning. Schedule nightly OBS packages 2-3 hours earlier to avoid this.
Commit
22349d63faf9ac702eab64a448e1cafdf4bd6ef6
by Oliver Smith
OBS: gerrit_binpkgs: print verbose output
Print all shell commands and their output executed by build_srcpkgs.py and build_binpkgs.py, instead of only printing them on error. This is useful to see the output of the scripts downloading dependencies when building sourcepackages.
Commit
0a410e539b7d7c2ef7c68a1ecebbc51ba9aba30d
by Oliver Smith
OBS: configure Osmocom pyhss fork
Add the Osmocom fork of PyHSS to scripts/obs/lib/config.py, so the Osmocom_OBS_* jobs build source packages for it and push them to our OBS where binary packages get built. We need to use a fork for now, as the official repository can't run outside of a git tree yet and doesn't have debian packaging. These changes are being upstreamed, the goal is that we don't need to use our fork in the long run.
![[Jenkins]](/jenkins/static/433051ca/images/svgs/logo.svg){kind=logo}
