Commit
637e5c1a89beb6353b7a6aff6b409037d73ad2c6
by Oliver Smith
CI: OBS: centos7: use vault.centos.org
The centos7 images try to use mirrorlist.centos.org to determine the fastest mirror, and then download binary packages from there. But mirrorlist.centos.org isn't online anymore. Use the vault instead.
Without this patch, the mirrors that can't be reached are ignored, and later on it fails to properly install packages because of missing dependencies.
Commit
0014a5c1c18e057b037a479cb19df90cf22581ad
by Oliver Smith
gerrit-verifications-dahdi: set concurrent: false
Each of the gerrit-dahdi-linux-* jobs has a 3-4 GiB workspace directory with a full linux tree. We keep them on disk so we don't need to clone the linux tree with each build.
This makes sense, but set concurrent to false to not create more of these than necessary. These directories stood out when investigating why build4 was running out of space.
Commit
dc413b56dfebcd9acd66761103783ee9c82b0158
by Oliver Smith
jobs/osmocom-build-tags…: fix workspace clean
As this job tries to archive the build logs after running, don't remove the entire workspace. Instead remove the paths that take up the most space, which are the one with the git clones and build artifacts (_deps) as well as the one with the temporary installation dirs (_temp/_inst*).
Fixes: b20e8e8b ("jobs/various: clean workspace on success") Change-Id: Ied86ba80b38a9b10a7a96631e6dedee9f3762c3b
Commit
3633d1f62e9615549a7fd32056f6b544825019dd
by Oliver Smith
jobs/update-osmo-…: only trigger container rebuild
Installing osmo-python-tests to the jenkins nodes should not be needed anymore, as builds of almost all jenkins jobs are done in docker where we install osmo-python-tests inside the containers where it is needed. We already removed the initial deployment to jenkins nodes via ansible in I7c87bd516d1bcdbcc21f65909dabbd3ccc38eede ("ansible: osmocom_build_deps: remove"), let's adjust the job that updates osmo-python-tests too. Instead of deploying it directly, only trigger a rebuild of containers.
I'm doing this now because the jenkins job started failing after I42ba71e2fd0ceb8466758ea46e9ae33e3d4fcb91 ("contrib/jenkins.sh: use virtualenv and pip") on some nodes where virtualenv is not available. The contrib/jenkins.sh script was used for both CI verification and to actually deploy this python package to the jenkins nodes, which was not really obvious and also as noted above isn't needed anymore.
Commit
d854bf573923aebafb3125bc021cb8914c2f6237
by Oliver Smith
OBS: osmocom:master: ignore missing commit_*.txt
Do not abort if the commit_*.txt file has been deleted from a package in osmocom:master. Instead, assume the package is outdated, create a new source package (with commit_*.txt) and replace the existing source package.
This is useful to force rebuilding a package, e.g. if the previous source package was built when the git tag was not pushed yet, and therefore the version in the package and for libraries, in the pkg-config data, don't have the proper version yet.
Commit
85359ba81d4f39fa6367659552a506a9a9f411dd
by Oliver Smith
OBS: replace - with . in version for d/changelog
Fix for: dpkg-buildpackage: info: source version 2:0.1.2-osmo-epdg ... dpkg-source: error: can't build with source format '3.0 (native)': native package version may not have a revision
Commit
9b36d2a8163d7b880c5d0f7ab888d91d6c4ac06b
by Oliver Smith
OBS: fix up-to-date check for strongswan-epdg
Move the code to transform the version from the git tag to the version that we can use in debian/changelog to a separate function. Use it not only when generating the version for debian/changelog, but also when checking if the current git tag is up-to-date.
Without this patch, it always considers strongswan-epdg outdated: strongswan-epdg: building source package (outdated: osmo-epdg-0.1.2 <=> 0.1.2.osmo.epdg in OBS)
Commit
0441e6d88b48bfd0d025044b2f154ee4e0cc57ba
by Oliver Smith
OBS: latest: handle "version is None"
Fix the Osmocom_OBS_latest job, which currently fails on git repositories where no tagged release can be found with:
File "/obs/lib/debian.py", line 96, in transform_version if "osmo-epdg-" in version: ^^^^^^^^^^^^^^^^^^^^^^^ TypeError: argument of type 'NoneType' is not iterable
Fixes: 9b36d2a8 ("OBS: fix up-to-date check for strongswan-epdg") Change-Id: I69cb884d499d1f6a8cae4b4585d31b5bc095fd6e
Commit
9f2b10968915cca7b7e44b1834321908b18a58b3
by Vadim Yanitskiy
OBS: add tag_pattern(), make tag patterns stricter
* add a convenience helper to avoid regexp body duplication * r'[0-9]*' is not strict enough, we want one or more digits * replace r'[0-9]' with r'\d' to make statements a bit shorter
Commit
8dd8e811961771bcf81066c5baba1fe36b856b32
by Oliver Smith
registry-rebuild-upload-testenv: new job
Add a job that updates the one podman image used by ttcn3 testsuites with testenv to the registry. The new testsuite jobs will not build their own images anymore, just pull this one at the start.
Commit
3700f45f4d49903a12820761c7e47d06489df118
by Oliver Smith
jobs/ttcn3-testsuites-testenv: initial
Move the first testsuites over from the old docker-playground based architecture of running testsuites to testenv. I'm doing this iteratively to avoid regressions.
Commit
5b75d6be44107b02b9ea7057239f1a3aee0361ce
by Oliver Smith
jobs/osmocom-depcheck: disable mail notifications
This is a manually triggered job that users may use if they want to verify minimal versions of dependencies mentioned in configure.ac result in a successful build. Don't notify the mailing list about it.
Commit
8fe1efa6d4441e655d477e8e5837592e0d6d910c
by Oliver Smith
ansible: podman: use fuse-overlayfs
The default driver "vfs" leads to container crashes on heavy i/o load on our jenkins machines. Use "fuse-overlayfs" instead, which does not have these problems.
Add a systemd service that creates /dev/fuse when the LXC container starts and configure the driver via ~/.config/containers/storage.conf.
Note: when changing the storage driver, the previous storage database needs to be removed (~/.local/share/containers/storage/). I've done this manually before rolling out this change.
Commit
c98cd9bcee9677db08817d74fffca01be1c3c641
by Oliver Smith
jobs/ttcn3-testsuites-testenv: add s1gw
Remove the docker-playground based job and add a new testenv based job instead. Add one for -latest too as we recently tagged the first releases.
Do not limit the job to a specific node, looks like this was pasted from the above job by mistake. I've successfully ran this on another node build2-deb11build-ansible.
Commit
cabaaa9e7a4455cb9763eba303f9ec3680f28a35
by laforge
jobs/{gerrit,master,update-osmo-ci}: add aarch64 case
The newly set up rpi4-raspbian11build-ansible-1 returns aarch64 for arch (it has an aarch64 OS instead of 32-bit arm like the others currently have). Add "aarch64" to the cases next to arm*, so these work as expected again.
Commit
914b58cb79f85a3956b7c38fc51b143864a331e6
by laforge
jobs/{master,gerrit}: rpi4-raspbian11 -> raspbian
Rename the jenkins label rpi4-raspbian11 to raspbian so the name is not misleading with how we currently use it. One raspberry pi has been reinstalled with raspbian12 (OS#6549) and we also use it for LX2 with a raspbian container (OS#5873, therefore remove rpi4 from the name).
The OS running directly on the rpi / in the LXC is not that important anyway since we run the actual jobs in docker with the latest debian version (currently 12).
Commit
c0fd46590420d57636a381c8b113a1fb8d1ecd40
by laforge
jobs/{gerrit,master}: aarch64: use setarch linux32
Pretend to be a 32 bit arm machine when running the arm jobs, so in combination with the related commit we can build test for 32 bit arm on rpi4s with a 64 bit raspberry pi OS.
Commit
3a26fd299b052858a64d6243492a95c76e940a97
by laforge
Stop building openbsc
With recent changes in libosmo-sccp, openbsc does not build anymore. The openbsc.git repo has been unmaintained/unsupported for 7 years, so let's finally remove related build jobs instead of fixing this.
Commit
b2e064a44ebf59bdacf4bc5d8ac890625ce6cb4f
by Oliver Smith
jobs/osmo-gsm-tester-build: fix Missing parameter
Use OSMO_GSM_TESTER_BRANCH as shell variable ('$...'), not as jenkins job builder parameter (${...}). This is how we do it in other jobs as well, e.g. jobs/osmocom-obs.yml.
Apparently the workaround in the file happened to work earlier, but it does not work anymore with Jenkins Job Builder version 6.3.0. Remove it.
Commit
2410e86b2a832f1345445f700df1a665f9394397
by Oliver Smith
libosmo-sccp -> libosmo-sigtran
Adjust to: * libosmo-sccp.git has been renamed to libosmo-sigtran.git * libosmo-sccp static libraries have been split into libosmo-sccp-legacy.git
build-old-tags-against-master: libosmo-sccp is intentionally kept in REPOS_TAGS, so old libosmo-sccp versions are built against current master of Osmocom libraries.
Various Osmocom components now include support for this family of Themyscira specs, but the linter outright rejects any and all patches that have a legitimate need to mention this acronym.
Commit
4c773bee97fad8a05f59ec4f8ca0e760fb0e7525
by Oliver Smith
jobs/osmo-gsm-tester-runner: fix syntax errors
Fix the syntax of this file, so it is compatible with Jenkins Job Builder 6.3.0.
* In shell blocks, {{ and }} needs to be used instead of { and } if the string in brackets does not refer to a JJB variable. * "${OSMO_GSM_TESTER_BRANCH}" would refer to a non-existing JJB var, instead we want the shell variable here (that jenkins sets), so omit the {}.
Commit
68ddbceeeb9fd47e001021d02e13eb1f114e4242
by Oliver Smith
jobs/gerrit: osmo-ttcn3-hacks: tweak make lines
* The "make clean" is not needed, as the osmo-ttcn3-hacks.git repository is a fresh clone of code that was submitted to gerrit. * The "make deps" is not needed, as "make compile" depends on the deps target. * Combine the cd and make lines. * Remove ; at the end of lines, as it is not needed * Switch " and ', so variables can be used with less escaping in future patches in this block.
Commit
65c3275be06603a29311ba2160ffbb4500b1c206
by Oliver Smith
jobs/gerrit: use deps cache from docker image
Instead of freshly cloning the dependencies each time, use the cache we already have in the debian-bookworm-titan docker image. This way only the dependency repositories that were changed in the patch submitted to gerrit need to be updated.
Commit
9679c5d20778cba7e479e3e5bd75eac7cbb4f85f
by Oliver Smith
OBS: build_srcpkg.Dockerfile: add sphinx-common
Install sphinx-common, as it is needed to build the source package for pyosmocom.
Fix for: dh clean --with python3,sphinxdoc --buildsystem=pybuild dh: error: unable to load addon sphinxdoc: Can't locate Debian/Debhelper/Sequence/sphinxdoc.pm in @INC (you may need to install the Debian::Debhelper::Sequence::sphinxdoc module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.36.0 /usr/local/share/perl/5.36.0 /usr/lib/x86_64-linux-gnu/perl5/5.36 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.36 /usr/share/perl/5.36 /usr/local/lib/site_perl) at (eval 13) line 1. BEGIN failed--compilation aborted at (eval 13) line 1.
Commit
072b1eae88a89bba4585b38ef0e47ccc6c60bea9
by Oliver Smith
manuals: publish pyosmocom manuals for releases
Adjust the building and publishing commands to work for pyosmocom. Remove the 0.0.3 release from TAGS_IGNORE, because it already has manuals.
Skip "apt-get -y build-dep" and adjusting "debian/control" if that file does not exist, as it is the case with the 0.0.3 and 0.0.4 releases of pyosmocom.
Commit
a4ba4ea461db88f739c73dbdce4800b1d40ecf9e
by Oliver Smith
jobs/master-builds: add param BRANCH
Sometimes it is desirable to run master jobs from a different branch: * As discussed with Philipp: running pysim CI on a branch with many patches once before submitting to gerrit (it takes much longer in gerrit to run on each individual patch) * When adding/modifying master jobs to test if they still work as expected with additional patches in a branch.
It is undesirable to have mail notifications if not running on master, the next patch will make it easy to disable those.
Commit
7a80d1597a506b3de5839d0c169ff07007aec3fc
by Oliver Smith
jobs/build-kernels-testenv: new jobs
Prepare to replace the docker-playground based jobs for running osmo-ggsn in QEMU with a kernel built from source with testenv.
Split the part that builds the kernels into extra jobs that export the built kernels as artifacts. This makes it easier to use kernels outside of jenkins too and it allows retriggering only a testsuite run without also building the kernel again.
The fragment.config is imported from docker-playground current master 978adc, path: ttcn3-ggsn-test/osmo-ggsn-kernel/fragment.config
Commit
85eb09d778ee0ec12a2d09e46904662bde9c99a7
by Oliver Smith
jobs/build-kernels-testenv: add build-kernel-none
Create a dummy output/linux file, that the jobs generated by ttcn3-testsuites-testenv can copy (via copyartifact) when setting the kernel to "none".
Without this, the ttcn3-testsuites-testenv.yml would need to become more complex, as it is not possible in jenkins job builder templates to only do the copyartifact block conditionally if kernel != "none".
Another solution would be using shell logic like the following:
if [ "$KERNEL" != "none" ]; then wget -q -O .linux "https://jenkins.osmocom.org/..." fi
But this has the disadvantage that jenkins is not aware of the file and therefore we cannot track easily which linux kernel build was used in the job. With the copyartifact method, we can click on "See Fingerprints" and find which job built the kernel. Example: https://jenkins.osmocom.org/jenkins/job/ttcn3-ggsn-test-kernel-net-next/147/fingerprints/
Commit
660f83108aa90bb6161ddda48a37782e091a8d38
by Oliver Smith
jobs/osmo-gsm-tester-runner: use debian bookworm
* Use bookworm during clean up instead of bullseye * Don't build debian-buster-jenkins anymore, the osmo-gsm-tester container now uses debian-bookworm-build which gets deployed via the update-osmo-ci-on-slaves job (see commit in Depends:)
Commit
c0046ab4a5aa97a05b2d067702fad9586429ec52
by Oliver Smith
jobs/osmo-gsm-tester-builder: build in docker
Build Osmocom programs in docker (debian bookworm), so they link against libraries available when trying to run the programs later on.
Without this we get errors like to following when osmo-gsm-tester-virtual runs in docker with debian bookworm (after recent change, before that it used debian buster): osmo-msc: error while loading shared libraries: libasan.so.5: cannot open shared object file: No such file or directory
Commit
0023c20cc06373cbbac43ba70b055fecad491892
by Oliver Smith
jobs/ttcn3-testsuites-testenv: tweak description
Don't link to osmo-ttcn3-hacks.git directly anymore, the wiki page that is also linked already refers to it. This shortens the boilerplate of the description with every one of these jenkins jobs.
Instead of only displaying the arguments, display the full command so users can easily run the testsuite the same way it is running on jenkins. Omit the arguments --cache, --ccache, --log-dir, which only need to be adjusted for jenkins, for users the defaults make sense.
Let the kernel line link to the related build-kernel job.
These testsuites have been disabled for > 3 years, and the related SUT is not developed any further. Remove the entries to unclutter the list for now, we could add them back later easily if it makes sense.
Add the current "last duration" time from the jenkins jobs as comment, so we have a rough idea of how long these jobs take and can distribute them more evenly throughout the days. Even though the durations will change over time, this gives a good indicator of whether a job runs a few minutes or 1 or even 2 hours.
Commit
9abdfe5307e4a682278a4174e73dcb0512a61dd5
by Oliver Smith
scripts/osmo-ci-docker-rebuild: fix rpi4 failures
It is no longer possible to pull the arm32 image by just using "docker build --pull". Use a separate pull command with --platform and set an env var to not use --pull with "docker build".
Commit
0ba4ce01b652c74832817ddd8eb893abc791c3a5
by Oliver Smith
build-old-tags: ignore failing due to api break
Ignore osmo-bsc and osmo-msc versions that are known to fail due to intentional API break. osmo-bsc 1.12.2 and osmo-msc 1.12.2 build are forward-compatible.
Note that previous releases don't need to be listed in tags_to_ignore() as this script only builds the last 3 release tags.
Store the apk after successful run of the android-apdu-proxy job. It can be found by clicking on a job at [1] and then clicking on "default" (since the job is behind a matrix, as all master jobs).
Commit
89a3b3f049c13c8d7e3ba17e305990fc9ea21f3d
by laforge
jobs/osmocom-release-android-apks: new job
Add a job similar to the release tarballs job, that iterates over the tags of the android app repositories (currently only android-apdu-proxy) and builds/uploads an apk for each of them.
Commit
b7e67c1df52050ebaa23437db8dc405dd223c446
by Oliver Smith
jobs/ttcn3-testsuites: adjust timer for bts
BTS testsuites tend to fail with timeouts on heavy loads. Group them in a timeslot where jenkins nodes are otherwise not busy according to stats in grafana to hopefully make the test results more stable. Once we can replace faketrx with a rewrite (OS#6672), this should not be necessary anymore.
Commit
0ba4dac82b83a8e6515c75e1b792014375595f37
by Oliver Smith
jobs/master,gerrit: use io_uring seccomp profile
Use the seccomp profile not only in ttcn3 testsuites, but also in master-builds and gerrit-verifications so we can test io_uring there too.
CI happened to work without this for libosmocore on build4 because of a specific docker version where io_uring was not yet (fully?) disabled in the default seccomp profile. It did not work without this patch on build5 where we currently have a newer docker version.
Commit
ef498e7efe3a5801bbfbf4b6e90a4e54bb3ee6e5
by Oliver Smith
jobs/{master,gerrit}: set ulimit filesize to 1 GiB
Set a max file size limit of 1 GiB via ulimit. This prevents jobs from filling up the disk by generating one huge log file.
I have chosen 1 GiB as limit, because jobs that clone linux kernel repos may have packed git artifacts that are several 100 MiB in size.
It would be even better to set the overall max disk space that can be used from one "docker run" call, but this is currently not supported for ext4: https://github.com/moby/moby/issues/29364
Commit
a71ccaa13598d4d5627b4e16e497f4d7d076b53e
by Oliver Smith
jobs: manually trigger "git gc" for linux.git
We already use shallow clones for linux.git, and git runs a garbage collector automatically to ensure no longer needed artifacts are removed. But this did not work anymore on build4 for build-kernel-net-next, and it increased up to 8.4 GiB:
8,4 GiB /build-kernel-net-next
After running "git gc" again:
3,3 GiB /build-kernel-net-next
The reason was that "git gc" did not exit successfully once and wrote a log file, then refused to run again. Run "git gc --auto" explicitly in foreground to let abort in such a case, so it does not go unnoticed in the future.
Commit
fa7a94dd4e75e8ead20d4cc5d1ed79a64a85f4ff
by Oliver Smith
scripts/docker-cleanup: add "docker system prune"
On build4 this freed up another 8.223GB when I just ran it:
$ docker system prune WARNING! This will remove: - all stopped containers - all networks not used by at least one container - all dangling images - all dangling build cache
Commit
7b4109ff906dd933d0db425bf2fb686740046646
by Oliver Smith
ansible/setup-jenkins-slave: add mingw job deps
Support running the legacy job Osmocom-OBS_MinGW_weekly_publish by installing its dependencies directly in the LXC. This is a workaround until we modernize the job, see the related issue.
![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
