Commit
8125b33b9a15dc0497ebe4697fa3758831b9430b
by Oliver Smith
ansible: coverity: update version to 22.06
Coverity doesn't make the old version available for download and will probably deprecate the previous version at some point. I'm about to install it on new hosts, so upgrade to the latest version while at it.
Commit
6bb5f4ce84bd8d9337da0b97501ef9c2a00e37b7
by Oliver Smith
hosts: update coverity-slaves
Remove the deb9 lxcs as they are being retired, and add two deb11 lxcs. Coverity is quite big with 1.2 GiB, so deploying and upgrading this takes some time. Having it on two hosts should be enough.
Commit
3307a90691ec14726f4f64b2d52126dbf4b1abcd
by Oliver Smith
jobs/osmo-gsm-tester virtual: fix default branch
Change it to "master" instead of "origin/master" to prepare for building the docker image right before running it. The Makefile from docker-playground.git picks up the OSMO_GSM_TESTER_BRANCH environment variable and expects it to be in that format.
Commit
ce5875fd077e3c9e3fa1dffd441ec334f3a9f58a
by Oliver Smith
jobs/osmo-gsm-tester virtual: fix docker img
Build the docker image right before using it, instead of building it in another jenkins job update-osmo-ci-on-slaves via osmo-ci-docker-rebuild.sh.
The logic in osmo-ci-docker-rebuild.sh was broken. I didn't realize at the time that this image is only used for the virtual osmo-gsm-tester, not the physical ones. But only the machines running the physical osmo-gsm-tester have the /var/tmp/osmo-gsm-tester/state path. The virtual osmo-gsm-tester isn't running on these machines but on generic jenkins nodes.
Building the image right before using it makes sense for this job, as it is the only user of the image. If it was already built from the same Dockerfile, a cached version is used.
Fix for: Unable to find image 'osmocom-build/osmo-gsm-tester:latest' locally
Commit
23cf9b4b05c534b45ed02eca6c4e663565178478
by Oliver Smith
jobs/osmo-gsm-tester: remove old test result early
The virtual osmo-gsm-tester has been failing for a long time, but the test results were still green. This was because the job failed before it removed old test results, and so those were used. Remove the old results as early as possible to prevent this in the future.
Commit
0428da0f6d35ae10aafb50cd4437b4c5149c579a
by Oliver Smith
jobs/osmo-gsm-tester virtual: fix permissions
The docker container creates the trial dir and some files inside of it as root, therefore the rm -rf trial-* line fails. Add logic to fix the permissions before attempting to remove old trial-* and when the script exits.
Commit
3b5464d0ed79ae8c21bf748701a078f04fbf15de
by Oliver Smith
jobs: retire osmocom-gerrit-debian* labels
Let the gerrit-binpkgs and gerrit-pipeline-result jobs use the "osmocom-gerrit" label. Having the -debian10/11 label previously was useful to make sure this doesn't run on an ancient python version that fails on f-strings. But since the debian9 builders were retired, this is not needed anymore. These are the only jobs that used these, so now the labels in the README can be simplified (following patch).
Commit
4fe56c096803d9a081670232269dca095e1d0605
by Oliver Smith
obs: build_deb/rpm: show contents after build
After building packages with the OBS scripts, show their contents. This allows easy checking if config files were correctly packaged etc. By adding it here, it will also show up at the end of the related jenkins jobs for gerrit verifications.
Commit
a7df704d4da50a2e179d4e613da76bceff058ea6
by Oliver Smith
scripts/docker-cleanup: fix timing problems
Don't delete images while they are being used, to fix these errors we see from time to time in the middle of "docker build" on jenkins: unknown parent image ID sha256:1b072e35048cd8b680eddabdc641ac678edb1184d222d5e7b3fbe0b3c333129a
This happens because "docker build" creates so-called dangling images for each step processed of a Dockerfile. The "docker system prune" call deletes these dangling images (among other things).
Remove the "docker system prune" call. We already have the docuum daemon to deal with unused images (dangling and not dangling), it removes them based on last use date so that the used space is always below a configured limit. As it deletes images that haven't been used the longest when it reaches the limit, it will not result in the problem explained above.
Besides images, "docker system prune" also removes unused containers (instances of images created with 'docker run' without --rm) and networks. Add "docker container prune" and "docker network prune" commands to remove them from now on.
Also remove the redundant container removal logic (previous it was redundant with "docker system prune", now redundant with "docker container prune").
Commit
21a641d6c238c10f4b2d0eb3351c21d2e8c098e9
by Oliver Smith
scripts/docker-cleanup: remove fallback code
Remove the fallback clean up code, as it also may lead to images getting removed right before we need to use them. Besides that, it should be dead code by now since docuum should be running on all our jenkins nodes to clean up old images based on last use date.
Commit
048492d1c837d32c507853f3719888b6b95c0f63
by Oliver Smith
repo-install-test: run inside qemu + kvm
Change repo-install-test to run inside of qemu instead of docker. This job needs to run systemd to verify that the systemd services start up properly. Running systemd inside docker was never officially supported, it worked with cgroups1 but does not work anymore with cgroups2.
An alternative approach was running inside podman instead of docker (running systemd inside of podman is officially supported). However we would have needed various workarounds with podman and wouldn't be able to test all Osmocom systemd services in the end, due to lack of permissions (see review of I394918fc61de36acce65ffb33defcb8fc21801c4).
By running with a separate kernel inside qemu we can run all Osmocom services.
Commit
f5bd2740180921974dad998e13a7d4fc1d808e0c
by Oliver Smith
repo-install-test: update services list
Add services from new projects and enable previously disabled services, now that this test runs in qemu and services have more permissions like setting realtime priorities.
Commit
36215cfff222b9d50be0654372f0ca70dfef97bf
by Oliver Smith
repo-install-test: fix osmo-trx-usrp1 check
The DISTRO variable is either debian10 or debian11, fix the broken check. This condition is there in the first place, because we don't build the usrp1 backend for centos8.
Commit
94a57624b422a412debd5ad28542ed8a8294b1c8
by laforge
obs-mirror: Keep nightly packages for 22.04, not 21.04
We typically keep nightly builds for the latest version of each distribution. That is correct for CentOS and Debian, but we forgot to update Ubuntu from 21.04 to 22.04 it seems :(
Commit
47e90f10d22ab81fa06b9c62aa8044446bbb444c
by laforge
obs-mirror: Also keep nightly builds for Debian 10
It seems that some users are running nightly builds in production on EOL Debian 10, which we never imagined and hence didn't keep an archive of those builds :/
Commit
4f1201ff06ad3c9d2c32971f2c9a20845e336877
by Oliver Smith
jobs/master-builds,gerrit-verifications: fix node
Do not set a node for master-builds and gerrit-verifications to run on, as this leads to needlessly blocking slots on the jenkins nodes.
Each of these jobs is a matrix job with a slave_axis. The matrix job itself can run on the "builtin" node, as it practically doesn't cause load - all it does is start other jobs in the matrix (even if there is only one other job, which is the default). Jobs started in the matrix job do run on a configured node already, as the slave_axis parameter is set.
I've just noticed this problem because currently the raspberry pi nodes are offline, and so the master-osmo-trx, master-osmo-pcu and other jobs take up lots of build slots just to wait until the raspberry pi nodes come online and an actual build can start.
Commit
50a54c923b6b1054486422859c0a46f3498da3a0
by Oliver Smith
jobs/update-osmo-ci-on-slaves: don't run on ogt
Don't run this job on the osmo-gsm-tester machines. Right now this job takes quite some time to build various docker images which we don't even need on osmo-gsm-tester. It also uploads osmo-ci.git and osmo-gsm-manuals.git, which don't need to be updated on these if we don't run the osmo-gsm-tester jenkins jobs on them (see previous commit). According to Pau they would be needed for osmo-gsm-tester_ttcn3.
Commit
48614942c698f14a954beaa632885c5da6ca2626
by laforge
jobs/master-builds: use osmo-iuh/master for asn1c
As discussed with Pau, let's build the "osmo-iuh/master" branch of our asn1c.git repository in jenkins. This branch is what we actually use for osmo-iuh and "master" should track upstream master.
Commit
b44f768b7bfe8b7c7f0eb3da9dbd7bd87724c4a5
by Oliver Smith
jobs/gerrit-verifications: osmo-trx: fix matrix
Add the missing comma between "--without-sse, --without-neon", so the matrix works as expected. Without this patch, the two builds with these flags didn't run.
Commit
30ccbd3f7065c6bb707bea35ae03eaed4b2cb549
by Oliver Smith
jobs/ttcn3: don't block on same testsuite running
Don't block jobs if other jobs with the same testsuite are already running. This used to be necessary as we had subnets hardcoded per testsuite, and therefore couldn't run it twice on the same host (e.g. once against latest, once against nightly).
Commit
ed604a0bb30812e9f928c8ecad8939c4528f7ae7
by Oliver Smith
jobs/gerrit-verifications-comment: rename
Rename the jenkins job from gerrit-pipeline-result to gerrit-verifications-comment, as the next patch will not only use this job to post a comment at the end of the gerrit verification, but also at the beginning when the pipeline starts.
Give the pipeline_summary* scripts more generic names as well.
Commit
9f1889a494db55649f22322697e65d4b8cfb5d07
by Oliver Smith
jobs/gerrit-verifications: post comment on start
Post a link to gerrit when starting the build that contains the link to the pipeline, without sending mail notifications.
This is useful when a gerrit verification build takes rather long, and it's not clear if a build for gerrit verification was actually started or not. Also I find this useful when debugging the CI scripts.
Commit
9e24272311712b9d49a9ac0073bfc466c5402e81
by Oliver Smith
lint: enable BRACES_NOT_NECESSARY check
As seen in code review here: https://gerrit.osmocom.org/c/libosmocore/+/30934
... we do care about not using braces for single statement blocks. Let the linter comment on it.
src/osmo_io.c:143: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks src/osmo_io.c:271: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks src/osmo_io.c:306: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks src/osmo_io_poll.c:63: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks src/osmo_io_poll.c:117: WARNING:BRACES_NOT_NECESSARY: braces {} are not necessary for single statement blocks
Commit
6a290081c578da84451c5b7f7c19f9ecbd6009b0
by laforge
ansible: replace deprecated include by import_tasks
include_tasks and import_tasks have similar functionality. Use import_tasks when importing non-variable depending files. Further use import_tasks when not used within a loop.
Commit
989c47bc915e5a7fbca92c984cecde3d59ad3f5e
by Oliver Smith
scripts/osmo-build.sh / ARTIFACT_STORE: remove
This script was supposed to cache already built Osmocom libraries. But it was only used with openbsc, didn't work as expected anymore (looking at openbsc-master, it just builds all deps from source without caching) and will be replaced with logic that allows using ccache in a future patch.
Using ccache has the big advantage that we can cache all build objects where the source files did not change, and not just do it on a dependency level. Oftentimes only few source files change in our patches, so we can really cache everything else that is not affected within any git repository and not just a dependency that didn't change.
ccache also does automatic cache size management to ensure a cache stays within a given size limit.
Commit
c914967c532b14a08623e44fec264af34149870f
by Oliver Smith
jobs/master,gerrit: use minimal image for arm
Building debian-buster-jenkins on the raspberry pis may take > 1h, depending on what changed. It has a lot of things we don't use, so build and use a minimal image with the rpis instead.
Commit
a788a8b14f15f32710444226d48207d745c234d2
by Oliver Smith
jobs/master,gerrit: add docker_run_ccache
Put arguments for ccache into a new docker_run_ccache variable, and use it with all projects where it makes sense to use ccache.
Other projects use docker_run without the ccache parameters again: * osmo_ss7, osmo_dia2gsup, pysim: Erlang/Python projects * osmo-e1-hardware, ice40-usbtrace, no2bootloader, osmo-opencm3-projects: Don't use the default docker image * osmo-gsm-tester: Only builds manuals (jenkins-build-manuals.sh)
Jenkins job builder's variable substitution doesn't replace {docker_run} inside docker_run_ccache when used in cmd, so just duplicate the lines.
Commit
4773f26d2d2a256ff5f3d27c7c369695c3a8f831
by Oliver Smith
jobs/master,gerrit: fix pysim, sysmo-usim-tool
Don't attempt to create the CCACHE_DIR and chown it to osmocom-build for the simtester jenkins node, and more generally all that were not created with the ansible provisioning scripts. In case of simtester, there is no osmocom-build user and we don't have the space there to use ccache.
Commit
7bd02257c9f198f709350aa0f04eec7f5df6e09c
by Oliver Smith
coverity/local_test: fix copy error
coverity/common.sh is a symlink to ../scripts/common.sh in this git repository. The local_test.sh already adjusts the symlink so it points to the right place after copying related scripts to /tmp/coverity. However when running the script again, cp would fail with:
cp: '/home/user/code/osmo-dev/src/osmo-ci/coverity/common.sh' and '/tmp/coverity/common.sh' are the same file
Commit
378abe1844d569e6026ba6f237556401f85994de
by Oliver Smith
repo-install-test: add DOMAIN variable
Allow overriding the downloads.osmocom.org domain of the url where packages are downloaded from, so we can download them from people.osmocom.org instead while developing / debugging, e.g.: https://people.osmocom.org/packages/home:/osmith:/nightly/
Commit
51748a77a7858be10b1b99d48a434254405e2bae
by Oliver Smith
jenkins-gerrit: make links clickable again
After gerrit was upgraded, the comment formatting changed. Links inside blocks indented with two spaces cannot be clicked anymore, so change it to a listing.
Commit
3e5c5d4a75075fe3975e1c687509c4e7c7652c9f
by Oliver Smith
build-old-tags: tags_to_ignore: add osmo-hlr
Add the osmo-hlr releases where tests are not foward-compatible to the related libosmocore change. Fixup releases 1.6.1 and 1.5.1 have been made that are foward compatible.
Let the check only run on the main repository, not on any submodules. This fixes that it would currently fail in osmo-trx: the osmocom-bb submodule has a copy of libosmocore where the endianness check is known to be failing (see https://gerrit.osmocom.org/c/osmocom-bb/+/31403).
Commit
719ff9760808acb1fe944f4c496c356c7dc07c91
by Oliver Smith
jobs: tweak build-discarder values
Remove num-to-keep from most jobs, as this leads to keeping the build logs for a much shorter timeline than desired. For example the gerrit-binpkgs-deb job that runs for most projects when pushing patches to jenkins reaches the 120 limit in less than 24h - and so when clicking the link on a failed build from yesterday it is already deleted.
Instead just keep the logs for the last 30 days, no matter how many were submitted on one day. Storing logs doesn't take up much space.
Remove the artifact-days-to-keep and artifact-num-to-keep lines, as they don't have an effect. For jobs that do have artifacts, the actual value is min(days-to-keep,artifact-days-to-keep) and same with num-to-keep.
While at it, increase the ttcn3-testsuites build-discarder to 120 days as this means more data will show up in the test result analyzer at which we look frequently.
Commit
9337c6c03581c1330895a8fa32a81957056be69e
by Oliver Smith
obs: lib.args: store argparse result here
Prepare to add more arguments to the OBS scripts, so they can be used for building the wireshark package in its own OBS project.
It was not so clear to me when writing this initially, but with more and more arguments getting added: it's not very maintainable to pass each variable from the argparse result several functions down to where it gets used... so unfortunately this means the code needs to be refactored to just use libs.args in the few lines of code where the arguments actually get used.
This patch adds lib.args, and the next patches refactor existing code to use it instead of passing parameters, where it makes sense. After that follow the patches related to adding the wireshark package.
During refactoring, conflict_version was changed to version here, which was wrong. The packages need to depend on the conflict_version of the meta package, version is the version of the package itself.
Fix for: The following packages have unmet dependencies: libosmo-netif-dev : Depends: osmocom-nightly (= 202303160009) but it is not going to be installed Depends: libosmocore but it is not going to be installed Depends: libosmonetif11 (= 1.3.0.8.9e65.202303160009) but it is not going to be installed libosmocore-dev : Depends: osmocom-nightly (= 1.8.0.79-b394d.202303170006) but it is not going to be installed Depends: libosmocore but it is not going to be installed
Fixes: 0ed0e464 ("obs: don't pass conflict_version through functions") Change-Id: I2326e3817c6f6887ef1196e603c3877768119a66
Commit
dc1415959fb47190a5a5f5e2e48a871548e242b6
by Oliver Smith
obs: add --allow-unknown-package
Prepare to add a wireshark package. It will be in its own osmocom:wireshark repository, and not in osmocom:nightly, :latest, :master, so it's not added to lib/config.py.
Move lib.set_args() before parse_packages(), so the latter is able to use lib.args.allow_unknown_package.
Commit
e7eba153157e228c7e03ab2d58880f3d4489fbec
by Oliver Smith
jobs: add Osmocom_OBS_wireshark
Add a nightly jenkins job that does the following: * Clone Wireshark from upstream * Merge several Osmocom branches on top * Build a source package * Submit it to the osmocom:wireshark OBS project
Commit
a052c13c12a73ae195e9367aed43c30f7a0837c1
by laforge
Revert "jobs: tweak build-discarder values"
This reverts commit 719ff9760808acb1fe944f4c496c356c7dc07c91.
The disk consumption of jenkins has grown by almost 500GB since we merged this patch. Clearly this is not expected, and we'd have ran out of disk space in a few weeks.
I personally think the current allocation of 1.5TB of disk space to jenkins should be more than sufficient; we just need to manage it better.
Commit
b126b77e813c545ab330fa8ed2eb78ea17b17a1d
by Oliver Smith
jobs: remove artifacts-days/num-to-keep: -1
Remove the "artifact-days-to-keep: -1" and "artifact-num-to-keep: -1" lines, as they don't have an effect. -1 is the default value, which means "infinite" and causes the value from "days-to-keep" / "num-to-keep" to get used instead.
This patch re-applies part of 719ff976 ("jobs: tweak build-discarder values"), which had been reverted in a052c13c.
Commit
d9f32325e20e6f4e627608599849096ddf3ed58e
by Oliver Smith
jobs/gerrit: increase num-to-keep to 1000
Increase the logs to keep for gerrit related builds, as they may exceed the previous limit of 120 within a day. All these jobs don't save build artifacts, so they should not cause a noticable size increase.
This is a much more careful version of 719ff976 ("jobs: tweak build-discarder values"), which had been reverted in a052c13c. The big problem with the previous patch was that it increased the number of jobs to keep even for jobs that had build artifacts, and so the required storage exploded.
Commit
78ca65b5d30676ae4aa064e08797fe0482e798b0
by Oliver Smith
obs: wireshark: user proper OBS api url
Set the proper https://obs.osmocom.org api URL, not "obs.osmocom.org". The latter only works on jenkins because we have set up an alias there, but not locally without setting up the same alias.
Commit
ff8ec46ac5f2fa0301fcf36fb1d0b178f33c4422
by Oliver Smith
obs: add --conflict-pkgname arg
Prepare to create new feeds for the rhizomatica/testing and rhizomatica/production branches, which will be based on osmocom:latest. They will depend on osmocom-latest (the "conflict package"/meta package), even though their feed name is not latest.
Commit
bdbc48a3728cc9ae2a91474a1cae72b2302ffc2c
by Oliver Smith
obs: update_obs_project: add ALL_OSMOCOM_PACKAGES
Add this alias to use all Osmocom packages, but no the misc packages that would also be selected if not specifying the packages argument at all (limesuite, neocon, open5gs).
This will be used for building packages from the rhizomatica branches, so we don't check the git repositories of these misc projects for the git branches.
Commit
f75d09a79e7fefca4589fe080edfa4bfec2e8074
by Oliver Smith
jobs/osmocom-obs: drop obs.osmocom.org suffix
Having the OBS URL at the end of the jenkins job was useful when we were uploading to obs.osmocom.org and build.opensuse.org at the same time during the transitional phase. Remove the URL from the job name as this isn't the case anymore, and so the jobs look consistent with new Osmocom_OBS jobs.
Commit
684a8f2f9cc120c2775327f833b164c119fbbfd8
by Oliver Smith
jobs/master,gerrit: build manuals with IU
Build the manuals with --enable-iu for osmo-msc and osmo-sgsn, so we don't miss IU-related VTY commands in the manuals.
As of writing, osmo-sgsn includes cs7-instance-iu only if building with --enable-iu. osmo-msc includes it regardless of the build flag (and returns an error if trying to use the command if built without IU support). Build the manuals with --enable-iu for both for consistency, and just in case we decide to change this behavior / add more commands.
Commit
badcf3d055ab744c3dd284c0ef9059a565c03adf
by Oliver Smith
obs: support all debian/ubuntu/almalinux versions
Adjust the code so one can pass all versions, instead of hardcoding only debian:11, ubuntu:22.04, almalinux:8. I'll use this for reproducing a build error that happens on e.g. ubuntu:18.04.
Commit
106120cace3630a1b99c26dba5b8d646dd962a79
by Oliver Smith
obs: build_binpkg: add feed argument
Make it possible to configure a different feed than master inside the docker container that gets used to build the packages. This way we can build ubuntu packages against nightly.
We don't build the Osmocom packages in the master feed for Ubuntu as we rarely have a build error that only happens on ubuntu. With this patch, it can be easily reproduced if it happens.
Commit
cdd4eece45ccf1cf4ac60e96ff6483c9b53a4324
by Oliver Smith
data/build_rpm: force refresh of package indexes
The yum-builddep command already has a mechanism to update the package index data if the last update (when the docker image was built) was some time ago. However when the packages we use happened to be updated in that timeframe, the indexes are not updated and attempting to download them leads to 404 errors. Force refresh all package indexes to avoid this.
Commit
818d29d9e8a0640b310bfb83aa2af70f15f7bcb7
by Oliver Smith
jobs/master: add workspace.tar.xz to artifacts
Upload workspace.tar.xz if it was created by osmo-bsc or osmo-msc, after the testsuite failed. This should help with figuring out why sometimes we get a coredump.
Commit
5bc3e0587ac3b2fbb47913f3af4704f7d13a8a58
by Oliver Smith
Fix failing Osmocom-release-manuals job
Add a workaround that upgrades the osmocom-nightly package before installing build dependencies for the given package.
This is not very elegant, it would make more sense if the docker image we use here did not have the nightly Osmocom repository configured in the first place (as this job is about creating manuals for tagged releases). But the image is used by jenkins build verification too and I don't think it's a good use of time to change this right now.
Fix for: + apt-get -y build-dep /build Note, using directory '/build' to get the build dependencies Reading package lists... Building dependency tree... Reading state information... Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation:
The following packages have unmet dependencies: osmo-gsm-manuals-dev : Depends: osmocom-nightly (= 202305300026) but 202305290026 is to be installed
Commit
79c5c2d7f7791dff7a0ff293b142501eba4f6e8d
by laforge
DAHDI master build diversification
* we never really wanted to build against fixed tags but branches, i.e. "the latest tag within a given stable series", so switch from a fixed tag like "v5.10" in torvalds/linux.git to "linux-5.10.y" in stable/linux.git
* we also want to build against 6.1.y, as that is what upcoming Debian bookworm will ship
Commit
910a04d39cd0d5eecf926c53f52af4a6a0528e28
by laforge
DAHDI gerrit verification job diversification
* we never really wanted to build against fixed tags but branches, i.e. "the latest tag within a given stable series", so switch from a fixed tag like "v5.10" in torvalds/linux.git to "linux-5.10.y" in stable/linux.git
* we also want to build against 6.1.y, as that is what upcoming Debian bookworm will ship
Commit
f71fceff65b96073e3245708b6f29e1eb9b0576b
by Oliver Smith
jobs: master/gerrit: use debian bookworm (12)
Upgrade from debian 11 for master and debian 10 for gerrit verifications to using debian 12 for both.
Previously we intentionally built against the older debian 10 version to ensure that our programs still build there. However it is easier to maintain the docker containers if we just use the most recent debian version for both and it makes the build environment more consistent - if a patch passes in gerrit verifications, we expect it to pass in master builds as well. And the other way around, I can just run CI of all master jobs when developing a change and assume that if they pass, gerrit verifications will run as well.
As long as we provide binary packages in OBS for debian 11, 10, ... we will still notice if a build breaks on an older debian release. I think this is good enough given that it will probably not happen that often, but if we decide that we really want to ensure it still builds on older distros at gerrit-verification time then the more suitable place to add this would be in the deb-build verification test. It is more maintainable there, because the dependencies just get installed from the debian/control file, no need to add all of them to a docker container beforehand.
The new container is debian-bookworm-build, see the docker-playground commit for reasoning why it is not debian-bookworm-jenkins.
Commit
127e79f171c3ad700e12668a2a23d23ee03c58a7
by Oliver Smith
jobs/update-osmo-ci: add branch parameters
Add parameters to set the osmo-ci and docker-playground branch. I'm using this to test the debian 12 based containers before merging the changes to master.
Commit
bd763747adab4be4d500ff9e109446d9b156301d
by Oliver Smith
jobs/gerrit: refactor pipeline script
* Deduplicate the code to run a job by moving it to a function. * Print the status of jobs right after they finished, instead of waiting until all jobs are done * Make the status print messages more readable
Commit
1dcb49a8110e65351f7e3253b98e7d55141865b0
by Oliver Smith
jobs/gerrit-binpkgs: don't hardcode distros
Make it possible to set a list of distributions to check in gerrit-verifications.yml instead of only having one boolean for testing two hardcoded rpm and deb distributions.
Commit
6d9f0843a95b5b2222625fabdf11035243b10819
by Oliver Smith
jobs/gerrit-verifications: fix binpkgs for libusrp
Don't build libusrp for centos8/almalinux. libusrp does have a .spec.in file, but it needs SDCC for the build and SDCC is not available there.
In theory we could build the rpm in CI for opensuse tumbleweed, but it requires adding support for building for opensuse in scripts/obs/data/build_binpkg.Dockerfile and scripts/obs/data/build_rpm.sh first, and since it works quite differently than centos8 and libusrp changes rarely (last code change in 2021) I've decided to not do that now.
Commit
8920d737305edf88c8f6c9db744cd004414294a6
by Oliver Smith
jobs/osmocom-release-tarballs: modernize
Rewrite the osmocom-release-manuals script (previous version is in docker-playground Ic35a28a386170b85d32aab8f2bd33e48e6d45392):
* Instead of using a separate docker container for this, that also lists all dependencies for all packages (as needed to pass ./configure), use debian-bookworm-build and install missing packages at time of generating the tarballs with "apt-get build-dep". Missing dependencies are typically other Osmocom libraries.
* This allows removing the debian 11 based release-tarball-build-dist container. As the script doesn't depend on a separate docker container anymore, move it to osmo-ci.git.
* Make it similar to scripts/manuals/publish-manuals-for-tags.sh, so it is easier to maintain both.
Commit
228efc3f13a638635a1b37d49b6c0943f403384d
by Oliver Smith
ansible: osmocom_build_deps: remove
The builds for almost all jenkins jobs are done in docker now, we definitely don't need to install all of these dependencies on all jenkins nodes anymore.
Remove the file for now, so it doesn't look like they still get used and so we don't need to keep it up-to-date (e.g. inkscape was just replaced with rsvg-convert).
When we deploy a new jenkins node and notice that we still do need a few of these dependencies, we can bring them back later.
Commit
e4ac3ae199fcac6c345ae56cd88a7ae7ffc9347b
by Oliver Smith
jobs/gerrit: osmo-dia2gsup: test debian 11 and 12
Don't try to build the debian package for debian 10. It fails as rebar3 is not in debian 10. Test the build for debian 11 and 12, which is what we build the package for on obs.osmocom.org.
Commit
451cbe7d86be71935e4614c4530517055e81bc61
by Oliver Smith
jobs/master-builds: add xgoldmon
Replace the legacy xgoldmon job (not done with jenkins-job-builder) with a new master-xgoldmon job that works more similar to other master jobs: - build in docker - have build commands in a jenkins.sh script
Put the jenkins.sh script into osmo-ci, as the upstream repository is outside of Osmocom infrastructure.
The motivation for this change is, that the current xgoldmon job is failing since libosmocore depends on liburing by default. This uncovered that the job is still running outside of docker, where the dependency has already been added. The following patches will modernize other jobs which have the same problem.
Commit
1b6970edace2c9bbfa993e9bcfd229aa9e1e953f
by Oliver Smith
jobs/master-builds: add simtrace
Replace the legacy SIMtrace job (not done with jenkins-job-builder) with a new master-simtrace job that works just like the other master jobs: - build in docker - have build commands in contrib/jenkins.sh
Commit
9ba26aa8fbc61f5ab8fd64e2d97a29080efdc725
by Oliver Smith
jobs/master-builds: add osmo-ir77
Replace the legacy osmo-ir77 job (not done with jenkins-job-builder) with a new master-osmo-ir77 job that works just like the other master jobs: - build in docker - have build commands in contrib/jenkins.sh
Commit
eac517c1af280b47370488e84e501721dff725e0
by Oliver Smith
jobs/master-builds: xgoldmon: remove osmo-ci dir
Fix running the job multiple times in a row. As the master jobs don't wipe the git repositories, we need to remove the osmo-ci dir first before attempting another shallow clone.
Commit
aa2e3cc5c016228e0f5600b8fe6573e9692b91d3
by Oliver Smith
OBS: lib/osc: add private proj variable
Adjust lib/osc.py to take the OBS project from a global variable that can be set with lib.osc.set_apiurl(), instead of using lib.args.obs_project. Move the sanity check to set_apiurl().
This is in preparation for adding a new sync script, which will operate on multiple OBS instances.
Commit
4cfe5d42b51b012fc0d6b1c7490b29d09f9dc38c
by Oliver Smith
OBS: add sync_obs_projects.py
Add a new script that syncronizes base distributions from openSUSE's OBS, like Debian:12. This is done by getting "meta" and "prjconf" from the source OBS, adjusting it and writing it to our OBS instance.
The motivation for this is making our OBS more resilient, by using these synchronized projects we can build packages even if the openSUSE OBS API is not reachable.
Commit
d27ba27a55effaf13f51fb77cf5e3401b6244537
by Oliver Smith
OBS: exit() -> sys.exit()
As pointed out in code review, sys.exit() should be used instead of exit() from site.
https://docs.python.org/3/library/constants.html#exit says: > The site module (which is imported automatically during startup, > except if the -S command-line option is given) adds several > constants to the built-in namespace. They are useful for the > interactive interpreter shell and should not be used in programs.
Commit
03cb83374624abb1474f595b97aaf44120e1ab22
by laforge
jobs: update trigger timers
Ensure we trigger building new OBS packages first, then wait plenty of time until the binary packages are available (and run jobs in the meantime that don't need them), and only after that we run the jobs that need the binary packages.
Otherwise TTCN3 jobs may test the packages from the previous day, and some jobs may fail completely due to packages not being completely built yet. For example, yesterday the new Osmocom CNI releases were tagged, which means the :latest packages also need to be rebuilt (-> building all OBS packages takes longer). The osmocom-release-manuals and -tarballs jobs failed, because the new binary packages were not available yet when they ran.
Change all timers to the format "H 20 * * *" to have a deterministic hour and semi-random minute based on the job name.
Commit
62693be148ccf8c44fd38316bdc52efc2948ffca
by Oliver Smith
OBS: fix generating wrong limesuite version
Instead of using git-version-gen in all git repositories, only use it where it is part of the repository. Use git directly to build the version string otherwise. To fix generating the wrong version string for limesuite, use "git describe --tags" instead of "git describe" as in git-version-gen. This is needed because limesuite releases > 20.01 only have lightweight (non-annotated) tags.
Fix for: limesuite: WARNING: version from changelog (22.09.1-1) is higher than version based on git tag (20.01.0.182-4828.202309140026), using version from changelog (git tag not pushed yet?)
Commit
9df253bb3405b5819806767c7fefa7fa2433f3e2
by Oliver Smith
repo-install-test: skip soapysdr-module-lms7
Fix for: dpkg: error processing archive /tmp/apt-dpkg-install-4OjX3P/0924-soapysdr0.8-module-lms7_22.09.1+dfsg-1+b2_amd64.deb (--unpack): trying to overwrite '/usr/lib/x86_64-linux-gnu/SoapySDR/modules0.8/libLMS7Support.so', which is also in package soapysdr0.7-module-lms7:amd64 22.09.1.17.4828.202309142026
Commit
1e748b38a45689eac743da92a4a6ea0d8e34e120
by Oliver Smith
OBS: add git tag pattern for wireshark
Since previous patch 62693be1 ("OBS: fix generating wrong limesuite version"), the scripts rely on having the correct pattern for the tag in the config. Add the pattern for wireshark.
Fix for: fatal: No tags can describe '6d412b991d8c2f50b7854611b1a9ef567bab9962'.
Commit
9aab3f54ad6e981effffed91f442e474a47135c9
by Oliver Smith
ansible: increase docker_max_image_space
The purpose of the limit is to remove images that are not being used in a long time. With the current limit of 40 GB it leads to images being removed just a few hours after they were last used, resulting in lots of needless rebuilds. Also unexpected errors with images that we just assume to be there (debian-bookworm-build).
Increases the limit for the nodes that run most of the jobs to 80 GB.
I have verified that the nodes have enough free space.
Commit
b206b2f1d22def7197c1913feeba167314ff2564
by Oliver Smith
scripts/docker-cleanup: remove containers > 24h
Remove containers starting with jenkins- or having ttcn3 in the name, if they have been running for more than 24 hours. This can happen with the ttcn3 testsuites, as they typically start multiple docker containers in the background (one per Osmocom program) before they start the testsuite docker container in the foreground. Usually the clean up trap makes sure that all containers get killed, but we have seen that a few containers have been running for a few months. One reason for this could be temporary loss of connection between the jenkins server and the node running the job.
Extend the clean script to remove the containers that were not properly removed by the clean up trap.
Historically we used to kill docker containers of the same name before starting a testsuite, but this had the downside that we could not start the same testsuite multiple times in parallel. This was refactored in docker-playground Ifcd384272c56d585e220e2588f2186dc110902ed.
Commit
3f037f4a73e30747dc991270262d51b1ea06fe02
by Oliver Smith
obs: srcpkg.py: Run generate_build_dep.sh for all projects
Adjust srcpkg.py to run a generate_build_dep.sh not only for osmo_dia2gsup, but for any Osmocom project that has such a script. This is in preparation for osmo-epdg, which will also have one.
Commit
f8e2ee24edf9abb581cb609a904d63313582c028
by Oliver Smith
obs: srcpkg: support having no git tags
Instead of failing when a project has no git-version-gen script, and no git tags, use 0.0.0 as version. This is needed for the new osmo-epdg project, which doesn't have tags yet.
Commit
15777116dd4942c47ca7c8f22a4801d2c4a3fb0e
by Oliver Smith
obs: build_binpkg: no network during build
Simulate that we don't have network during package builds in gerrit verifications, like it is the case when the packages will be built by OBS.
I've also considered using network namespaces via "unshare -n" or bubblewrap, but it wouldn't be possible to create a loopback device inside the network namespace inside docker (maybe with --privileged, but not worth it) and e.g. the osmo-mgw tests rely on lo.
Commit
119bc1f44a9de92254e7052ce309fd32388af2ec
by Oliver Smith
ansible: jenkins: use default-jre-headless
Use default-jre-headless instead of giving the openjdk version, so we don't need to adjust it to whatever the current version in debian is.
The default-jre-headless package is at least available since debian 10: https://packages.debian.org/search?suite=buster&arch=any&searchon=names&keywords=default-jre-headless
Commit
a779121725e2080174a128c1d1af0296e1870f61
by Oliver Smith
scripts/repo-install-test: fix sftp compat
When running scp, Debian 12 uses the SFTP protocol by default instead of the legacy SCP protocol. For some reason, this means it doesn't create the /repo-install-test directory inside QEMU:
+ … scp … -r scripts/repo-install-test root@127.0.0.1:/repo-install-test Warning: Permanently added '[127.0.0.1]:22793' (ED25519) to the list of known hosts. scp: realpath /repo-install-test: No such file scp: upload "/repo-install-test": path canonicalization failed scp: failed to upload directory scripts/repo-install-test to /repo-install-test
Changing to the legacy protocol with "scp -O" would make it work again, but instead of doing that add the mkdir so it works with the new protocol too.
Commit
3d2c3dd14cdccb35c27ee1766e00fd71868ca36f
by Oliver Smith
scripts/repo-install-test: fix sftp compat, part 2
Copy files inside repo-install-test, instead of copying the directory again. Otherwise it creates a /repo-install-test/repo-install-test/ path inside qemu and then fails to run the script inside the directory:
Fix for: sh: 0: cannot open /repo-install-test/run-inside-env.sh: No such file
Commit
867f2d28fd7edb2fbfa45c7f51a7273ccc4b5ed1
by Oliver Smith
OBS: add workaround for limesuite 23.10 release
The previous workaround for building on debian 12 is not needed anymore, as the related patch has been merged. Replace it with a new workaround needed for building the limesuite 23.10 release.
Commit
25f74d3d51bcab67ed94e06765d9ec539ccba29e
by Oliver Smith
Update git tag patterns for rtl-sdr, osmo-fl2k
With the v2.0.0 release of rtl-sdr, it was decided to include the v in the tag pattern again, so adjust it in the related scripts. Adjust osmo-fl2k too in the OBS scripts because there the tag pattern was not set.
Commit
8ba48eb28a7b0631dd4f3539c815ffb7d80de122
by laforge
repo-install-test: fix ca-certificates-java error
The repo-install-test started to fail on debian 10 with the following error. Apparently this happens when installing ca-certificates-java after installing java. Add a workaround.
Setting up ca-certificates-java (20190405) ... head: cannot open '/etc/ssl/certs/java/cacerts' for reading: No such file or directory Exception in thread "main" java.lang.InternalError: Error loading java.security file
Commit
a9e9c366b1bcf93acc92cc56dff0dacae8307ed6
by laforge
repo-install-test: ignore limesuite-images
Don't attempt to install limesuite-images. It runs a post-install script that downloads files from an external server and fails currently, as there are no images for 23.10. While we have limesuite-images packages in our OBS repository, this is just a side-effect of building limesuite. What we are really interested in is liblimesuite for osmo-trx, as I understand.
Add --no-install-recommends to the apt-get install line in install_repo_packages_debian, because the main limesuite package has limesuite-images in recommends and would pull it in otherwise.
Commit
0c79a15162b067822f561259e8b35fafbac286ac
by Oliver Smith
tarballs: git archive: put files in directory
Make the releases created from the "git archive" code path more similar to the autotools releases, by having all files in a subdirectory with the project name and tag name instead of having them directly in the root of the tarball.
Commit
38707a473cea49f39c815ac8bab1864dc11cd9dc
by Oliver Smith
tarballs: git archive: make proper tar.bz2
Compress the tarball from "git archive" with bzip2 to turn it into a proper .tar.bz2. Previously the archive was named .tar.bz2, but it was just a regular tarball as git archive didn't guess the type from the filename (supposedly this works for .tar.gz and some others) and just defaulted to tar.
Move the code that adds build_dep.tar.gz to the archive before the bzip2 call, because tar can't add files to the archive if it is compressed.
Commit
0afdbcf8bd96e77bd233b31e2d089a175f89f7ab
by laforge
obs-mirror: keep nightly builds for two years
As we're moving the archive to a machine with sufficient storage, let's extend the time we keep an archive of nightly builds from one month to two years.
+ make -C ../debian-bookworm-build-arm BUILD_ARGS=--pull UPSTREAM_DISTRO=debian:bookworm DISTRO=debian-bookworm IMAGE=osmocom-build/debian-bookworm-build-arm OSMO_TTCN3_BRANCH=master ./scripts/osmo-ci-docker-rebuild.sh: 203: make: not found
Commit
2031b3d747d8d1bb952a6d565aea104a6d1ae315
by Oliver Smith
repo-install-test: update package filtering logic
Instead of filtering with several blacklist_* files that must contain the exact names of packages to be filtered, add a shell function that uses fnmatch for filtering. Combine all lists into one.
This fixes the error we get with each limesuite release, without the need to increase the version in the txt files every time. Currently the repo-install-tests are failing for all debian versions because of this.
Commit
c4ee740d57d283ae76dd1f3ae9497c9e926642a0
by Oliver Smith
jobs: fix undefined parameter log warnings
Adjust all jobs in the gerrit verifications pipeline to use the same parameters from a new include file, to prevent undefined parameter warnings in the log.
Commit
5db0807a64f70e75af8c5e7d2e050930a1062e5e
by Oliver Smith
jobs/gerrit: osmo-upf: don't for build debian 10
osmo-upf currently can't build against debian 10, as the required libnftables-dev version there is too low and we don't provide a backport for it. As discussed, disable it for debian 10.
Adjust CI to build against debian 11 instead (in addition to the usual almalinux 8, debian 12).
Commit
6b136f74d18a243b79fe01cc9ce2f345aae99751
by Oliver Smith
OBS: lib.osc.get_projects: print apiurl
Print the apiurl, to make more obvious what is going on when running the function twice in a row with two different apiurls (as done in the next patch).
Commit
11f9724c3992a0a1c920c5b103cff0a5095c5303
by Oliver Smith
OBS: add jenkins job to check for new distros
Add a new jenkins job to notify us about new versions of Debian, Raspbian and Ubuntu becoming available in the openSUSE OBS. This allows us to consistently follow along and make the repositories available in the Osmocom OBS. I've decided to check the openSUSE OBS instead of checking somewhere upstream, because it takes time between a new release of a distribution and the version becoming available in openSUSE OBS, and we have another job that syncs the configuration with that OBS instance.
Commit
1fc751429621810c78122c418dd8e2fc7db295d7
by Oliver Smith
ansible: fix updating /opt/coverity/current
Remove the previous /opt/coverity/current symlink, before creating the new one. Otherwise the new symlink would be created inside "current" pointing at the old version, instead of replacing "current" with the new symlink.
Commit
a9c93850c35e3bac01b6d58e6c00df0e83806617
by Oliver Smith
jobs/osmo-gsm-tester-virtual: kill old instances
Make sure osmo-gsm-tester gets killed eventually, even if a bug causes it to run forever or if aborted manually.
* add a name to the docker container * kill the docker container if it runs longer than 24h with docker-cleanup.sh * rename fix_permissions_trap to clean_up_trap and kill it there, when it is still running before the job starts and after it is done (in my testing this did not kill it after pressing abort, but it would be killed either at the start of the next job running on the same jenkins node, or after 24h by docker-cleanup.sh)
Commit
33ddf03dfefd3844328b803cbe96bad513b7bc37
by Oliver Smith
OBS: srcpkg Dockerfile: add lsb-release, quilt
Wireshark with Osmocom patches on top of wireshark.git master failed to build. I've reworked the packaging to use the "debian" dir from upstream instead of having our own (now outdated) copy: https://gitea.osmocom.org/osmocom/wireshark/commits/branch/osmith/deb-packaging
Add the additional dependencies that are now required to build the wireshark source package to the Dockerfile.
Commit
61723f812cdfc6cea3d22bc0ef448f9b07dab6ae
by Pau Espin Pedrol
jobs/ttcn: Run ttcn3-epdg-test only under debian12
Test EPDG_Tests.TC_mt_ipv4_echo_req fails when running under debian11 (node build2-deb11build-ansible), most probably due to some kernel bug fixed later on. Since anyway osmo-epdg requires OTP from debian12, it makes no sense to attempt to support older kernel version from debian11, so simply avoid picking debian11 to run the tests.
Commit
467a96c28115352b0af9500f46b58658f6767ad5
by Oliver Smith
jobs/ttcn3: run io_uring tests on specific nodes
io_uring_queue_init fails on some of the jenkins nodes running the TTCN3 testsuites with io_uring. It seems that the kernel is too old on these nodes. Run the testsuites only on jenkins nodes with the new ttcn3-io_uring label, which have a recent kernel.
Commit
cd0de5983124a23bb8c8f85651a8a1bfba317a6b
by Oliver Smith
jobs/gerrit: libosmocore: run on io_uring nodes
We have at least one node where the io_uring backend does not work. Allow running the osmo_io tests with the io_uring backend in CI by limiting the jenkins nodes used to the ones where we know that io_uring works.
Commit
c1fe54aaf884549a3831b83848fb3cf8c5d24bbf
by Oliver Smith
jobs: add osmocom-obs-ewild
Similar to osmocom-obs-nhofmeyr, add a jenkins job that automatically picks up changes in Hoernchen/mssdr branches in the Osmocom projects, and builds binary packages in home:ewild:mssdr.
Commit
afd9d56e0568a61d0b2de010f710baffb71bff25
by Oliver Smith
checkpatch: recognize json_array_foreach etc.
Treat json_array_foreach, json_object_foreach, json_object_foreach_safe etc. the same as if, while, for, switch, and require exactly one space after using it:
Commit
0cc68159f632e4320054290df3baeee517cda71a
by Oliver Smith
jobs/ttcn3: io_uring: set memlock to unlimited
io_uring in the kernel memlocks a big amount of space with older kernels, such as the kernel on host2-deb11build-ansible. It seemed to be enough to increase it to 1 GiB. However now we see it failing again with:
failure during io_uring_queue_init(): Cannot allocate memory Aborted (core dumped)
Set it to unlimited to ensure that a low limit is not the reason we see it failing. If it still happens after this, we probably need to make sure that we don't run too many SUTs with io_uring in parallel on jenkins nodes with older kernels.
Commit
ec9efbf6494a1bad8a16f0c5b33f878a62822500
by Oliver Smith
repo-install-test: centos8: fix almalinux keys
Run "dnf upgrade almalinux-release" inside centos8/almalinux8 to upgrade the gpg keys first, in order to prevent gpg key errors while installing packages. We are not running repo-install-test for centos8 in CI anymore (SYS#6759), but I'm currently using this configuration to manually test that fixing permissions during upgrade and running systemd services as users works as expected on an rpm-based system.
Commit
b2e128511d5aeae1eb27a7465197012fac643da2
by Oliver Smith
OBS: move gerrit-binpkgs logic to separate script
As I'm preparing slides for a OsmoDevCon talk, I'm realizing that it is quite cumbersome for users to build debian packages from Osmocom git repositories (the same way gerrit does it, to reproduce errors locally).
Move the logic from the gerrit-binpkgs job to a separate script that can be run from an Osmocom git repository, will figure out the repository name from there and only needs the target distribution as argument.
Usage example: $ git clone https://gitea.osmocom.org/osmocom/osmo-ci $ cd libosmocore $ ../osmo-ci/scripts/obs/gerrit_binpkgs.sh debian:12 # or debian:11, debian:10, almalinux:8
Commit
ef2d95d0a298da20043e3ec37f343c6661eedb01
by Oliver Smith
ttcn3-bsc-test-io_uring: limit jenkins nodes again
Run this job only on jenkins nodes with a linux kernel of 6.1.x or higher. The io_uring implementation consumes a lot more memory on e.g. the 5.10.x kernel of host2, which causes osmo-bsc to run out of memory in other places:
<0002> system_information.c:1641 bts 0: Error while generating SI4: Cannot allocate memory (-12) <0002> bts_trx.c:505 Cannot generate SI4 for BTS 0: error <Cannot allocate memory>, most likely a problem with neighbor cell list generation
Commit
0aa60b04a5193fc3d79ce64c9ce94ebed42a96bb
by Oliver Smith
ansible/setup-jenkins: install rsync
After recent refactoring of the gerrit-binpkgs job, rsync is now requried on the jenkins nodes. It was installed on the ones where I tested, but not on all of them.
Fixes: b2e12851 ("OBS: move gerrit-binpkgs logic to separate script") Change-Id: Ic8ffd61fc2c89574085e9c3abbceb554f3c5da6a
Commit
91a79bd8aa8e7901a7d687b6222df360db1bcdc5
by Oliver Smith
jobs: gerrit: build osmo-pcap + dep for centos7
We are about to remove the rpm spec files for almost all Osmocom git repositories. The only two git repositories where we still need them, because we have a customer using them on centos 7, are libosmocore and osmo-pcap (SYS#6760). Add CI to ensure that these still build fine, and that we don't remove these by mistake.
Commit
902fb23413dbb3795dce7a39d86e854f4144d31f
by Oliver Smith
OBS: gerrit_binpkgs: use --no-meta
Don't let packages built with gerrit_binpkgs.sh depend on an osmocom-master meta package. This makes it easier to install and test the built packages locally.
Commit
d974243ef9bf4d2f7374dd98d4c9c525cad12cda
by Oliver Smith
jobs/ttcn3: run ttcn3-msc-test earlier
It seems like various tests are failing from time to time due to the builder having a too high load. As several ttcn3 testsuites run around 4 am already, move ttcn3-msc-test to 1 hour earlier where we don't have anything currently.
osmo-bts in theory can also benefit from io_uring, at least because it can maintain multiple RTP connections for voice calls. Let's execute the respective testsuite with io_uring too.
Commit
3bd97cc86a24f4ad34864fd38c21c6f8e6ba58f8
by Oliver Smith
jobs/ttcn3: update ttcn3-ggsn-test-kernel
Remove ttcn3-ggsn-test-kernel-pablo-gtp, as the IPv6 GTP patches have been merged. Add ttcn3-ggsn-test-kernel-net-next and ttcn3-ggsn-test-kernel-torvalds instead, which run the master/nightly version of osmo-ggsn against HEAD of the kernel trees in netdev/net-next and torvalds/linux.
Commit
3b10b6c6d295801aca78ee2a3c05ffb10c8891c4
by Oliver Smith
OBS: add update_obs_eclipse_titan.sh
Add a script to easily update the eclipse-titan package from a git repository. This replaces the previous workflow of downloading the previous source package, modifying it, and re-uploading it. With the git repository, it is easier to see which patches are included, and it is easier to modify the packaging (e.g. add new patches to fix build with latest gcc), and easier to upgrade the eclipse-titan version.
I've done this change because it failed to build with the most recent GCC in Debian Unstable. Backporting Vadim's patch from upstream fixes it.
Commit
e6d4b42cea8c54b5b20637262a16ed8abf06102f
by laforge
jenkins-jobs-osmo: new file
Do not recommend to the user anymore to put the Osmocom jenkins server in their global jenkins-job-builder config, as this may lead to jobs from other repositories being deployed to the Osmocom jenkins by accident.
Instead of that, let the user put the config in this path: ~/.config/jenkins_jobs/jenkins_jobs_osmo-ci.ini
Add a wrapper script that users this path, and makes sure that no global config is present.
Commit
0674a258c1c5726d7cd05e1e38990487800859ea
by Oliver Smith
ansible/setup-jenkins: add more utils
* etckeeper: to automatically have /etc changes versioned in git; we already use it in various other places * ncdu: to figure out what consumes space * tree: list directory trees, useful to quickly check if e.g. ccache and jenkins workspace dirs are populated properly
Commit
b8c6a5024658f0a1ad628b1ab319a483d7c55ef5
by Oliver Smith
OBS: fix python 3.12 compat
Fix for:
File "/home/user/code/osmo-dev/src/osmo-ci/scripts/obs/lib/__init__.py", line 112, in check_required_programs if not importlib.find_loader(module): ^^^^^^^^^^^^^^^^^^^^^ AttributeError: module 'importlib' has no attribute 'find_loader
Commit
67a5fc886a176ae9485400e3e957ab728a9007d3
by Oliver Smith
jobs: add osmocom-obs-nightly-asan
Add a new job that builds source packages for osmocom:nightly:asan, and uploads them there.
It was considered whether we should instead of building a binary repository with asan packages in OBS, just build all packages and libraries from source at the jenkins builders. We decided against it, see the related discussion.
Commit
3b20d2a281430c44e2f2bb95109708aa85eeff99
by Oliver Smith
OBS: get_git_version: debian/changelog as fallback
Fix osmocom-bb getting 0.1.0 as version:
osmocom-bb: has no git tags, using 0.0.0 as version osmocom-bb: building source package 0.0.0.3621-d70e osmocom-bb: WARNING: version from changelog (0.1.0) is higher than version based on git tag (0.0.0.3621-d70e), using version from changelog (git tag not pushed yet?)
With this patch:
osmocom-bb: has no git tags osmocom-bb: getting version from debian/changelog osmocom-bb: building source package 0.1.0.3621-d70e osmocom-bb: adding debian/changelog entry (0.1.0 => 0.1.0.3621-d70e)
Commit
6a2451a0ad3dd2f5e3179d64446d9d553a37016d
by Oliver Smith
jobs/coverity: adjust mount path to new home dir
With the related docker-playground patch, the home dir for the build user has actually been changed from /home/build to /build. This doesn't matter for all other jobs it seems, but the coverity job here fails because it tries to access $HOME/osmo-ci/coverity.
Adjust the mount path, so it works again as expected.
Fix for: ./jenkins.sh: line 37: /build/osmo-ci/coverity/get_token.sh: No such file or directory
Commit
afae3d79b5656fa24a4eeb6208e8f2ca379967b4
by Oliver Smith
jobs/coverity: fix clean up permission errors
Don't mount the host's osmo-ci/coverity dir in /build/osmo-ci/coverity. The current working dir is mounted as /build, and because docker creates the /build/osmo-ci directory structure as root, we end up with having subdirectories created by root in our current working dir. These then cause an error on the next run when jenkins tries to clean them up.
Fixes: 6a2451a0 ("jobs/coverity: adjust mount path to new home dir") Change-Id: Ie0aa02d3821df6d0505748a7dd70ff67e4cf1405
Commit
637e5c1a89beb6353b7a6aff6b409037d73ad2c6
by Oliver Smith
CI: OBS: centos7: use vault.centos.org
The centos7 images try to use mirrorlist.centos.org to determine the fastest mirror, and then download binary packages from there. But mirrorlist.centos.org isn't online anymore. Use the vault instead.
Without this patch, the mirrors that can't be reached are ignored, and later on it fails to properly install packages because of missing dependencies.
Commit
0014a5c1c18e057b037a479cb19df90cf22581ad
by Oliver Smith
gerrit-verifications-dahdi: set concurrent: false
Each of the gerrit-dahdi-linux-* jobs has a 3-4 GiB workspace directory with a full linux tree. We keep them on disk so we don't need to clone the linux tree with each build.
This makes sense, but set concurrent to false to not create more of these than necessary. These directories stood out when investigating why build4 was running out of space.
Commit
dc413b56dfebcd9acd66761103783ee9c82b0158
by Oliver Smith
jobs/osmocom-build-tags…: fix workspace clean
As this job tries to archive the build logs after running, don't remove the entire workspace. Instead remove the paths that take up the most space, which are the one with the git clones and build artifacts (_deps) as well as the one with the temporary installation dirs (_temp/_inst*).
Fixes: b20e8e8b ("jobs/various: clean workspace on success") Change-Id: Ied86ba80b38a9b10a7a96631e6dedee9f3762c3b
Commit
3633d1f62e9615549a7fd32056f6b544825019dd
by Oliver Smith
jobs/update-osmo-…: only trigger container rebuild
Installing osmo-python-tests to the jenkins nodes should not be needed anymore, as builds of almost all jenkins jobs are done in docker where we install osmo-python-tests inside the containers where it is needed. We already removed the initial deployment to jenkins nodes via ansible in I7c87bd516d1bcdbcc21f65909dabbd3ccc38eede ("ansible: osmocom_build_deps: remove"), let's adjust the job that updates osmo-python-tests too. Instead of deploying it directly, only trigger a rebuild of containers.
I'm doing this now because the jenkins job started failing after I42ba71e2fd0ceb8466758ea46e9ae33e3d4fcb91 ("contrib/jenkins.sh: use virtualenv and pip") on some nodes where virtualenv is not available. The contrib/jenkins.sh script was used for both CI verification and to actually deploy this python package to the jenkins nodes, which was not really obvious and also as noted above isn't needed anymore.
Commit
d854bf573923aebafb3125bc021cb8914c2f6237
by Oliver Smith
OBS: osmocom:master: ignore missing commit_*.txt
Do not abort if the commit_*.txt file has been deleted from a package in osmocom:master. Instead, assume the package is outdated, create a new source package (with commit_*.txt) and replace the existing source package.
This is useful to force rebuilding a package, e.g. if the previous source package was built when the git tag was not pushed yet, and therefore the version in the package and for libraries, in the pkg-config data, don't have the proper version yet.
Commit
85359ba81d4f39fa6367659552a506a9a9f411dd
by Oliver Smith
OBS: replace - with . in version for d/changelog
Fix for: dpkg-buildpackage: info: source version 2:0.1.2-osmo-epdg ... dpkg-source: error: can't build with source format '3.0 (native)': native package version may not have a revision
Commit
9b36d2a8163d7b880c5d0f7ab888d91d6c4ac06b
by Oliver Smith
OBS: fix up-to-date check for strongswan-epdg
Move the code to transform the version from the git tag to the version that we can use in debian/changelog to a separate function. Use it not only when generating the version for debian/changelog, but also when checking if the current git tag is up-to-date.
Without this patch, it always considers strongswan-epdg outdated: strongswan-epdg: building source package (outdated: osmo-epdg-0.1.2 <=> 0.1.2.osmo.epdg in OBS)
Commit
0441e6d88b48bfd0d025044b2f154ee4e0cc57ba
by Oliver Smith
OBS: latest: handle "version is None"
Fix the Osmocom_OBS_latest job, which currently fails on git repositories where no tagged release can be found with:
File "/obs/lib/debian.py", line 96, in transform_version if "osmo-epdg-" in version: ^^^^^^^^^^^^^^^^^^^^^^^ TypeError: argument of type 'NoneType' is not iterable
Fixes: 9b36d2a8 ("OBS: fix up-to-date check for strongswan-epdg") Change-Id: I69cb884d499d1f6a8cae4b4585d31b5bc095fd6e
Commit
9f2b10968915cca7b7e44b1834321908b18a58b3
by Vadim Yanitskiy
OBS: add tag_pattern(), make tag patterns stricter
* add a convenience helper to avoid regexp body duplication * r'[0-9]*' is not strict enough, we want one or more digits * replace r'[0-9]' with r'\d' to make statements a bit shorter
Commit
8dd8e811961771bcf81066c5baba1fe36b856b32
by Oliver Smith
registry-rebuild-upload-testenv: new job
Add a job that updates the one podman image used by ttcn3 testsuites with testenv to the registry. The new testsuite jobs will not build their own images anymore, just pull this one at the start.
Commit
3700f45f4d49903a12820761c7e47d06489df118
by Oliver Smith
jobs/ttcn3-testsuites-testenv: initial
Move the first testsuites over from the old docker-playground based architecture of running testsuites to testenv. I'm doing this iteratively to avoid regressions.
Commit
5b75d6be44107b02b9ea7057239f1a3aee0361ce
by Oliver Smith
jobs/osmocom-depcheck: disable mail notifications
This is a manually triggered job that users may use if they want to verify minimal versions of dependencies mentioned in configure.ac result in a successful build. Don't notify the mailing list about it.
Commit
8fe1efa6d4441e655d477e8e5837592e0d6d910c
by Oliver Smith
ansible: podman: use fuse-overlayfs
The default driver "vfs" leads to container crashes on heavy i/o load on our jenkins machines. Use "fuse-overlayfs" instead, which does not have these problems.
Add a systemd service that creates /dev/fuse when the LXC container starts and configure the driver via ~/.config/containers/storage.conf.
Note: when changing the storage driver, the previous storage database needs to be removed (~/.local/share/containers/storage/). I've done this manually before rolling out this change.
Commit
c98cd9bcee9677db08817d74fffca01be1c3c641
by Oliver Smith
jobs/ttcn3-testsuites-testenv: add s1gw
Remove the docker-playground based job and add a new testenv based job instead. Add one for -latest too as we recently tagged the first releases.
Do not limit the job to a specific node, looks like this was pasted from the above job by mistake. I've successfully ran this on another node build2-deb11build-ansible.
Commit
cabaaa9e7a4455cb9763eba303f9ec3680f28a35
by laforge
jobs/{gerrit,master,update-osmo-ci}: add aarch64 case
The newly set up rpi4-raspbian11build-ansible-1 returns aarch64 for arch (it has an aarch64 OS instead of 32-bit arm like the others currently have). Add "aarch64" to the cases next to arm*, so these work as expected again.
Commit
914b58cb79f85a3956b7c38fc51b143864a331e6
by laforge
jobs/{master,gerrit}: rpi4-raspbian11 -> raspbian
Rename the jenkins label rpi4-raspbian11 to raspbian so the name is not misleading with how we currently use it. One raspberry pi has been reinstalled with raspbian12 (OS#6549) and we also use it for LX2 with a raspbian container (OS#5873, therefore remove rpi4 from the name).
The OS running directly on the rpi / in the LXC is not that important anyway since we run the actual jobs in docker with the latest debian version (currently 12).
Commit
c0fd46590420d57636a381c8b113a1fb8d1ecd40
by laforge
jobs/{gerrit,master}: aarch64: use setarch linux32
Pretend to be a 32 bit arm machine when running the arm jobs, so in combination with the related commit we can build test for 32 bit arm on rpi4s with a 64 bit raspberry pi OS.
Commit
3a26fd299b052858a64d6243492a95c76e940a97
by laforge
Stop building openbsc
With recent changes in libosmo-sccp, openbsc does not build anymore. The openbsc.git repo has been unmaintained/unsupported for 7 years, so let's finally remove related build jobs instead of fixing this.
Commit
b2e064a44ebf59bdacf4bc5d8ac890625ce6cb4f
by Oliver Smith
jobs/osmo-gsm-tester-build: fix Missing parameter
Use OSMO_GSM_TESTER_BRANCH as shell variable ('$...'), not as jenkins job builder parameter (${...}). This is how we do it in other jobs as well, e.g. jobs/osmocom-obs.yml.
Apparently the workaround in the file happened to work earlier, but it does not work anymore with Jenkins Job Builder version 6.3.0. Remove it.
Commit
2410e86b2a832f1345445f700df1a665f9394397
by Oliver Smith
libosmo-sccp -> libosmo-sigtran
Adjust to: * libosmo-sccp.git has been renamed to libosmo-sigtran.git * libosmo-sccp static libraries have been split into libosmo-sccp-legacy.git
build-old-tags-against-master: libosmo-sccp is intentionally kept in REPOS_TAGS, so old libosmo-sccp versions are built against current master of Osmocom libraries.
Various Osmocom components now include support for this family of Themyscira specs, but the linter outright rejects any and all patches that have a legitimate need to mention this acronym.
Commit
4c773bee97fad8a05f59ec4f8ca0e760fb0e7525
by Oliver Smith
jobs/osmo-gsm-tester-runner: fix syntax errors
Fix the syntax of this file, so it is compatible with Jenkins Job Builder 6.3.0.
* In shell blocks, {{ and }} needs to be used instead of { and } if the string in brackets does not refer to a JJB variable. * "${OSMO_GSM_TESTER_BRANCH}" would refer to a non-existing JJB var, instead we want the shell variable here (that jenkins sets), so omit the {}.
Commit
68ddbceeeb9fd47e001021d02e13eb1f114e4242
by Oliver Smith
jobs/gerrit: osmo-ttcn3-hacks: tweak make lines
* The "make clean" is not needed, as the osmo-ttcn3-hacks.git repository is a fresh clone of code that was submitted to gerrit. * The "make deps" is not needed, as "make compile" depends on the deps target. * Combine the cd and make lines. * Remove ; at the end of lines, as it is not needed * Switch " and ', so variables can be used with less escaping in future patches in this block.
Commit
65c3275be06603a29311ba2160ffbb4500b1c206
by Oliver Smith
jobs/gerrit: use deps cache from docker image
Instead of freshly cloning the dependencies each time, use the cache we already have in the debian-bookworm-titan docker image. This way only the dependency repositories that were changed in the patch submitted to gerrit need to be updated.
Commit
9679c5d20778cba7e479e3e5bd75eac7cbb4f85f
by Oliver Smith
OBS: build_srcpkg.Dockerfile: add sphinx-common
Install sphinx-common, as it is needed to build the source package for pyosmocom.
Fix for: dh clean --with python3,sphinxdoc --buildsystem=pybuild dh: error: unable to load addon sphinxdoc: Can't locate Debian/Debhelper/Sequence/sphinxdoc.pm in @INC (you may need to install the Debian::Debhelper::Sequence::sphinxdoc module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.36.0 /usr/local/share/perl/5.36.0 /usr/lib/x86_64-linux-gnu/perl5/5.36 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.36 /usr/share/perl/5.36 /usr/local/lib/site_perl) at (eval 13) line 1. BEGIN failed--compilation aborted at (eval 13) line 1.
Commit
072b1eae88a89bba4585b38ef0e47ccc6c60bea9
by Oliver Smith
manuals: publish pyosmocom manuals for releases
Adjust the building and publishing commands to work for pyosmocom. Remove the 0.0.3 release from TAGS_IGNORE, because it already has manuals.
Skip "apt-get -y build-dep" and adjusting "debian/control" if that file does not exist, as it is the case with the 0.0.3 and 0.0.4 releases of pyosmocom.
Commit
a4ba4ea461db88f739c73dbdce4800b1d40ecf9e
by Oliver Smith
jobs/master-builds: add param BRANCH
Sometimes it is desirable to run master jobs from a different branch: * As discussed with Philipp: running pysim CI on a branch with many patches once before submitting to gerrit (it takes much longer in gerrit to run on each individual patch) * When adding/modifying master jobs to test if they still work as expected with additional patches in a branch.
It is undesirable to have mail notifications if not running on master, the next patch will make it easy to disable those.
Commit
7a80d1597a506b3de5839d0c169ff07007aec3fc
by Oliver Smith
jobs/build-kernels-testenv: new jobs
Prepare to replace the docker-playground based jobs for running osmo-ggsn in QEMU with a kernel built from source with testenv.
Split the part that builds the kernels into extra jobs that export the built kernels as artifacts. This makes it easier to use kernels outside of jenkins too and it allows retriggering only a testsuite run without also building the kernel again.
The fragment.config is imported from docker-playground current master 978adc, path: ttcn3-ggsn-test/osmo-ggsn-kernel/fragment.config
Commit
85eb09d778ee0ec12a2d09e46904662bde9c99a7
by Oliver Smith
jobs/build-kernels-testenv: add build-kernel-none
Create a dummy output/linux file, that the jobs generated by ttcn3-testsuites-testenv can copy (via copyartifact) when setting the kernel to "none".
Without this, the ttcn3-testsuites-testenv.yml would need to become more complex, as it is not possible in jenkins job builder templates to only do the copyartifact block conditionally if kernel != "none".
Another solution would be using shell logic like the following:
if [ "$KERNEL" != "none" ]; then wget -q -O .linux "https://jenkins.osmocom.org/..." fi
But this has the disadvantage that jenkins is not aware of the file and therefore we cannot track easily which linux kernel build was used in the job. With the copyartifact method, we can click on "See Fingerprints" and find which job built the kernel. Example: https://jenkins.osmocom.org/jenkins/job/ttcn3-ggsn-test-kernel-net-next/147/fingerprints/
Commit
660f83108aa90bb6161ddda48a37782e091a8d38
by Oliver Smith
jobs/osmo-gsm-tester-runner: use debian bookworm
* Use bookworm during clean up instead of bullseye * Don't build debian-buster-jenkins anymore, the osmo-gsm-tester container now uses debian-bookworm-build which gets deployed via the update-osmo-ci-on-slaves job (see commit in Depends:)
Commit
c0046ab4a5aa97a05b2d067702fad9586429ec52
by Oliver Smith
jobs/osmo-gsm-tester-builder: build in docker
Build Osmocom programs in docker (debian bookworm), so they link against libraries available when trying to run the programs later on.
Without this we get errors like to following when osmo-gsm-tester-virtual runs in docker with debian bookworm (after recent change, before that it used debian buster): osmo-msc: error while loading shared libraries: libasan.so.5: cannot open shared object file: No such file or directory
Commit
0023c20cc06373cbbac43ba70b055fecad491892
by Oliver Smith
jobs/ttcn3-testsuites-testenv: tweak description
Don't link to osmo-ttcn3-hacks.git directly anymore, the wiki page that is also linked already refers to it. This shortens the boilerplate of the description with every one of these jenkins jobs.
Instead of only displaying the arguments, display the full command so users can easily run the testsuite the same way it is running on jenkins. Omit the arguments --cache, --ccache, --log-dir, which only need to be adjusted for jenkins, for users the defaults make sense.
Let the kernel line link to the related build-kernel job.
These testsuites have been disabled for > 3 years, and the related SUT is not developed any further. Remove the entries to unclutter the list for now, we could add them back later easily if it makes sense.
Add the current "last duration" time from the jenkins jobs as comment, so we have a rough idea of how long these jobs take and can distribute them more evenly throughout the days. Even though the durations will change over time, this gives a good indicator of whether a job runs a few minutes or 1 or even 2 hours.
Commit
9abdfe5307e4a682278a4174e73dcb0512a61dd5
by Oliver Smith
scripts/osmo-ci-docker-rebuild: fix rpi4 failures
It is no longer possible to pull the arm32 image by just using "docker build --pull". Use a separate pull command with --platform and set an env var to not use --pull with "docker build".
Commit
0ba4ce01b652c74832817ddd8eb893abc791c3a5
by Oliver Smith
build-old-tags: ignore failing due to api break
Ignore osmo-bsc and osmo-msc versions that are known to fail due to intentional API break. osmo-bsc 1.12.2 and osmo-msc 1.12.2 build are forward-compatible.
Note that previous releases don't need to be listed in tags_to_ignore() as this script only builds the last 3 release tags.
Store the apk after successful run of the android-apdu-proxy job. It can be found by clicking on a job at [1] and then clicking on "default" (since the job is behind a matrix, as all master jobs).
![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
