Commit
1262888d49057d212a16a29506e8a30b79202510
by Oliver Smith
jobs/registry…: fix syntax error
Use $REGISTRY instead of ${REGISTRY}, so jenkins-job-builder 6.3.0 doesn't interpret it as JJB variable. It needs to be interpreted as shell variable here. This probably worked with older JJB versions.
Commit
80a9abae512b4f28c7531c27bb91ea8757bb1e72
by Oliver Smith
contrib/obs-mirror/obs-mirror-include: update
Update the daily backup/mirror configuration.
* Add Debian 13 * Add Raspbian 12 (not sure why it was not there yet) * Add nightly/CentOS_7 (we build a few packages for it and we also have the mirror configured for latest) * Remove CentOS 8 (dropped 1 year ago: https://osmocom.org/news/255) * Remove Debian 10, Raspbian 10 (https://osmocom.org/news/308)
Commit
f071c697887247030b8ec43c1a9dc2d873984d35
by Oliver Smith
OBS: support building debian trixie (13) pkgs
Translate "debian:13" to "debian:trixie", as for some reason no docker tag is getting pushed for "debian:13". With this we can keep using the usual short naming scheme in pipeline_binpkgs lines in gerrit-verifications.yml (see next patch).
Commit
5c0b9984b437759fea55fb9101ccde49acb27d25
by Oliver Smith
jobs/gerrit: binpkgs: update debian versions
Change the default from "debian:10 debian:12" to "debian:11 debian:13", as Debian 13 has just been released and we are about to remove Debian 10 binary packages.
Commit
6d19e82eedcd4852cce3b43482ab54f1df195ef2
by Oliver Smith
OBS: sync_obs_projects: support security.d.o
The Debian 13 meta config in the OpenSUSE OBS now lists security.debian.org. Adjust the script that syncs the config to support this URL too, rewrite it to HTTPS just like the other mirror URL.
Fix for: changing url to https: http://security.debian.org/debian-security?dist=trixie-security&component=main Traceback (most recent call last): File "/obs/sync_obs_projects.py", line 322, in <module> main() File "/obs/sync_obs_projects.py", line 312, in main rewrite_meta(project) File "/obs/sync_obs_projects.py", line 190, in rewrite_meta assert url.startswith("http://ftp.de.debian.org/debian"), \ AssertionError: unexpected mirror URL
Commit
bc5dbb3093b47a5dfdca8f956d85e2cd5a7bcba1
by Oliver Smith
jenkins-gerrit: binpkgs: ignore decode errors
The output of recent osmo-s1gw jobs for building binary packages contains invalid utf-8 characters, even though it should be utf-8 according to the HTTP header: content-type: text/plain;charset=utf-8
We only need to get the distro name from the output here, so simply ignore any decode errors.
Fix for: $ ./comment_generate.py https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-s1gw/417/ -o gerrit_report.json -t result Reading https://jenkins.osmocom.org/jenkins/job/gerrit-osmo-s1gw/417//consoleText Reading https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs/17255/consoleText Reading https://jenkins.osmocom.org/jenkins/job/gerrit-binpkgs/17260/consoleText Traceback (most recent call last): File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 270, in <module> main() File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 257, in main comment = get_comment_result(args.build_url, args.notify_on_success) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 198, in get_comment_result pipeline = parse_pipeline(build_url) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 89, in parse_pipeline stage = stage_from_job_name(job_name, job_url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 59, in stage_from_job_name return stage_binpkgs_from_url(job_url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/user/code/osmo-dev/src/osmo-ci/scripts/jenkins-gerrit/./comment_generate.py", line 46, in stage_binpkgs_from_url content = response.read().decode("utf-8") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xef in position 237446: invalid continuation byte
Commit
cf88fab8cda55362233afbbd54aba0b99a263d44
by Vadim Yanitskiy
jobs/gerrit-verifications: osmo-ttcn3-hacks: do make clean
It's easy to break the build verification [for everybody] by submitting a patch that creates a symlink to a non-existent file. The workspace is not wiped, so the stale symlink remains and results in failures like this:
Commit
bf3ef9d95af939d581f10f8a35e952e1a8131b21
by Oliver Smith
OBS: --version-append: never override version
We have some branches set up that when pushed to, directly result in building OBS packages. We append a string to the package version there, such as ~rhizomatica.testing. The expectation is that all packages built from that branch have this string in the version.
Recently an edge case was found: when amending the commit of the last tagged release, e.g. osmo-sgsn 1.12.0, then the generated version is based on the previous git tag (as the last git tag's commit is not part of the current branch anymore) while having the newer version in debian/changelog. This then results in the newer version being used, without appending the version string:
osmo-sgsn: WARNING: version from changelog (1.12.0) is higher than version based on git tag (1.11.1.20-e000e~rhizomatica.production), using version from changelog (git tag not pushed yet?)
While IMHO it is a bad idea to just amend the commit of the last release instead of making a separate commit, this results in unexpected behavior so adjust the script to not override the version if --version-append is used.
Commit
ffef33c677f7bf9c76a99a6dfa7ab7eb29a49c23
by Oliver Smith
jobs: run ttcn3-remsim-test-asan with bookworm
The default distribution to run ttcn3 testsuites and SUTs with has recently been changed from debian 12 (bookworm) to 13 (trixie). Since then we have seen 27 additional failures with osmocom:nightly:asan (while osmocom:nightly and osmocom:latest work fine). As the reason for this is currently unclear, revert to using debian 12 for this job for now, so this doesn't mask other errors that the job would find otherwise.
Commit
60bf602f6f8b0f4d06329b14767c4c49f8d42ded
by Oliver Smith
OBS: disable_manuals: adjust for osmocom-bb
With the related change, we build manauals in the debian package of osmocom-bb. The osmocom:nightly:asan feed gets built with manuals disabled. Fix that building the osmocom-bb package in the asan feed currently fails with:
[ 160s] /usr/bin/make -C doc/manuals [ 160s] make[2]: Entering directory '/usr/src/packages/BUILD/doc/manuals' [ 160s] OSMO_GSM_MANUALS_DIR: /usr/src/packages/osmo-gsm-manuals (fallback) [ 160s] ERROR: OSMO_GSM_MANUALS_DIR does not exist! [ 160s] Install osmo-gsm-manuals or set OSMO_GSM_MANUALS_DIR. [ 160s] Makefile:6: /build/Makefile.asciidoc.inc: No such file or directory [ 160s] make[2]: *** No rule to make target '/build/Makefile.asciidoc.inc'. Stop.
Commit
bd2cf1f881aea5dd5ca4a0f8822c5467762d1166
by Oliver Smith
OBS: stop building limesuite
Building limesuite is not needed anymore, as it is available at least since debian 10 in the debian and ubuntu repositories. We have been building the upstream version without changes.
Commit
77eaf6179803cf8c9bd71df39beb375e46205a52
by Oliver Smith
testenv-coredump-helper: remove
After looking at this again, it makes more sense to keep /proc/sys/kernel/core_pattern as "core" on build4 and build5, which we use for jenkins jobs instead of installing systemd's coredumpctl and making all coredumps go through that. The motivation for using coredumpctl in testenv was that lots of developers may have it already installed (though I'm not sure about that anymore), and that it gives a nice API for retrieving the related executable name to a coredump instead of having to parse the output of e.g. "file".
Using coredumpctl had the disadvantage that coredumps for all other jobs, such as the master-builds are also caught by that and cannot be easily placed in the workspace directory. I have started implementing this by extending testenv-coredump-helper to communicate through a socket and mounting that socket inside docker containers running contrib/jenkins.sh (host -> lxc -> docker), and it works, but this complexity is not useful here.
Instead the related patch will make testenv pick the core files when coredumpctl is not available, and read the path from "file", making testenv-coredump-helper obsolete.
Commit
dfa9fcca49ae8b8ce6c691196c3883206933e503
by Oliver Smith
scripts/wrapper_core_bt_on_error: new script
Add a wrapper script that runs a given program, and checks the exit code. If it is not 0, and a core file appears, then load the coredump in gdb and display its backtrace.
Now that it is also possible to run the HLR tests against PyHSS, add "-c osmo_hlr" to run the existing ttcn3-hlr-test jobs against osmo-hlr. I plan to add jobs for PyHSS later in a separate patch.
Commit
b93642c7c66e94ea5532720bc222393505b24f4e
by Oliver Smith
jobs/ttcn3-testsuites-testenv: mme: use -c all
Adjust to MME tests now having two configurations in osmo-ttcn3-hacks:
[testenv] Found multiple testenv.cfg files, use one of: [testenv] -c emergency [testenv] -c generic [testenv] You can also select all of them (-c all) or use the * character as wildcard.
Commit
46075b796ded3c014346fe5dcc18a6082a792f89
by Oliver Smith
OBS: let osmocom-keyring not depend on other pgks
The osmocom-latest, osmocom-nightly etc. meta-packages depend on the osmocom-keyring package. Fix that the osmocom-keyring package was also depending on these meta packages.
This fixes that when uninstalling all osmocom:latest packages by removing the osmocom-latest meta package, the osmocom-keyring package would also get removed. This is undesirable for switching between latest and nightly, as the osmocom-keyring package must remain installed to be able to install new packages from the repository (as it has the repo key).
Commit
ce3a6cb2b1ad650e19b9a6b38453c02128db5cf5
by Oliver Smith
jobs/osmocom-obs: build binary packages earlier
We have recently noticed that with current settings the nightly arm packages have still not finished the next morning. Schedule nightly OBS packages 2-3 hours earlier to avoid this.
Commit
22349d63faf9ac702eab64a448e1cafdf4bd6ef6
by Oliver Smith
OBS: gerrit_binpkgs: print verbose output
Print all shell commands and their output executed by build_srcpkgs.py and build_binpkgs.py, instead of only printing them on error. This is useful to see the output of the scripts downloading dependencies when building sourcepackages.
Commit
14b2f70e5c79f9ca0b70f498e1ef3518a776ae5b
by Oliver Smith
OBS: support building pyhss
Build source packages for the python project pyhss similar to how we do it for erlang projects: by downloading all dependencies ahead of time and vendoring them in the source package.
Commit
070d2c3bbfcb96a66dd6abdfb4af2e2d614e07d7
by Oliver Smith
OBS: configure Osmocom pyhss fork
Add the Osmocom fork of PyHSS to scripts/obs/lib/config.py, so the Osmocom_OBS_* jobs build source packages for it and push them to our OBS where binary packages get built. We need to use a fork for now, as the official repository can't run outside of a git tree yet and doesn't have debian packaging. These changes are being upstreamed, the goal is that we don't need to use our fork in the long run.
Commit
6e40c44983be020e4c66338b8dbeaa038e3517e7
by Oliver Smith
lint: run clang-format only on specific projects
For most C projects, running checkpatch works best with the coding style we use. Run clang-format only in specific projects that seem to be fully formatted with that tool.
Commit
fc08cf87d7651af2f2755d37c8916accf689a333
by Oliver Smith
scripts/manuals: use debian-trixie-build
Fix that the Osmocom-release-manuals job was still using debian-bookworm-build instead of debian-trixie-build to build the manuals. We use the latter for building manuals in master-builds and gerrit-verifications.
Without this patch building pyosmocom manuals failed with the following, as python3-typing-extensions in debian bookworm is at version 4.4.0 but typing_extensions >= 4.6.0 is needed for TypeAliasType. Trixie has version 4.13.2.
AttributeError: module 'typing_extensions' has no attribute 'TypeAliasType'. Did you mean: 'TypeAlias'?
Commit
08bdbb702c8f36c85975d6800998583e71a905c2
by Oliver Smith
ansible: allow jenkins users to set rtprio 30
Create an /etc/security/limits.d file to allow the jenkins user to set rtprio 30. This is required for running the ttcn3 bts testsuite through testenv, as (unlike the previous approach with docker containers) osmo-bts-trx runs without root rights there.
Commit
53b9bc7032d1204a17274826ad666c483a9d2d92
by Oliver Smith
repo-install-test: debian 10: osmo-gbproxy only
We only build osmo-gbproxy for debian 10, because a customer requires it. Adjust the repo-install-test to only verify that osmo-gbproxy can be installed and runs as expected.
