Started by upstream project "osmo-gsm-tester_manual-build-all" build number 63 originally caused by: Started by user Pau Espin Pedrol Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_manual-run-all Cloning the remote Git repository Cloning repository git://git.osmocom.org/osmo-gsm-tester > git init /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester # timeout=10 Fetching upstream changes from git://git.osmocom.org/osmo-gsm-tester > git --version # timeout=10 > git fetch --tags --progress git://git.osmocom.org/osmo-gsm-tester +refs/heads/*:refs/remotes/origin/* > git config remote.origin.url git://git.osmocom.org/osmo-gsm-tester # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url git://git.osmocom.org/osmo-gsm-tester # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 No valid HEAD. Skipping the resetting > git clean -fdx # timeout=10 Fetching upstream changes from git://git.osmocom.org/osmo-gsm-tester > git fetch --tags --progress git://git.osmocom.org/osmo-gsm-tester +refs/heads/*:refs/remotes/origin/* > git rev-parse pespin/mainloop^{commit} # timeout=10 > git rev-parse refs/remotes/origin/pespin/mainloop^{commit} # timeout=10 Checking out Revision 796bf3954170ad9515af9cb37b1abdf1fce1774b (refs/remotes/origin/pespin/mainloop) > git config core.sparsecheckout # timeout=10 > git checkout -f 796bf3954170ad9515af9cb37b1abdf1fce1774b Commit message: "nanobts: Attemt at fixing race condition using ipaccess-config after power up" > git rev-list --no-walk 796bf3954170ad9515af9cb37b1abdf1fce1774b # timeout=10 Copied 26 artifacts from "osmo-gsm-tester_manual-build-all" build number 63 [osmo-gsm-tester_manual-run-all] $ /bin/sh /tmp/jenkins900714297724889761.sh + export OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/example + unlink osmo-gsm-tester/example/resources.conf unlink: cannot unlink 'osmo-gsm-tester/example/resources.conf': No such file or directory + true + ln -s resources.conf.prod osmo-gsm-tester/example/resources.conf + PATH=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games ./osmo-gsm-tester/contrib/jenkins-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all + date +%F %T + time_start=2018-06-27 13:06:47 + trial_dir_prefix=trial- + rm -rf trial-* + trial_dir=trial-683 + mkdir -p trial-683 + mv osmo-bsc.build-63.tgz osmo-bts-sysmo.build-63.tgz osmo-bts.build-63.tgz osmo-ggsn.build-63.tgz osmo-hlr.build-63.tgz osmo-mgw.build-63.tgz osmo-msc.build-63.tgz osmo-nitb.build-63.tgz osmo-pcu-sysmo.build-63.tgz osmo-pcu.build-63.tgz osmo-sgsn.build-63.tgz osmo-stp.build-63.tgz osmo-trx.build-63.tgz trial-683 + cat osmo-bsc.build-63.md5 osmo-bts-sysmo.build-63.md5 osmo-bts.build-63.md5 osmo-ggsn.build-63.md5 osmo-hlr.build-63.md5 osmo-mgw.build-63.md5 osmo-msc.build-63.md5 osmo-nitb.build-63.md5 osmo-pcu-sysmo.build-63.md5 osmo-pcu.build-63.md5 osmo-sgsn.build-63.md5 osmo-stp.build-63.md5 osmo-trx.build-63.md5 + rm osmo-bsc.build-63.md5 osmo-bts-sysmo.build-63.md5 osmo-bts.build-63.md5 osmo-ggsn.build-63.md5 osmo-hlr.build-63.md5 osmo-mgw.build-63.md5 osmo-msc.build-63.md5 osmo-nitb.build-63.md5 osmo-pcu-sysmo.build-63.md5 osmo-pcu.build-63.md5 osmo-sgsn.build-63.md5 osmo-stp.build-63.md5 osmo-trx.build-63.md5 + exit_code=1 + which osmo-gsm-tester.py + python3 -u /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src/osmo-gsm-tester.py trial-683 -s aoip_ussd:nanobts+band-900 -s aoip_ussd:nanobts+band-1900 combinations: ['aoip_ussd:nanobts+band-900', 'aoip_ussd:nanobts+band-1900'] series: None trial: 'trial-683' tests: None -------------------------------------------------------------------------------------------- trial-683 -------------------------------------------------------------------------------------------- 13:06:48.482542 tst trial-683: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683/run.2018-06-27_13-06-48/log --------------------------------------------------------------------- trial-683 aoip_ussd:nanobts+band-900 --------------------------------------------------------------------- 13:06:49.757927 tst aoip_ussd:nanobts+band-900: reserving resources in /var/tmp/osmo-gsm-tester/state ... 13:06:49.777724 tst aoip_ussd:nanobts+band-900: Reserving 1 x bts (candidates: 6) 13:06:49.802271 tst aoip_ussd:nanobts+band-900: Reserving 6 x ip_address (candidates: 8) 13:06:49.836186 tst aoip_ussd:nanobts+band-900: Reserving 1 x modem (candidates: 4) ---------------------------------------------- trial-683 aoip_ussd:nanobts+band-900 assert_extension.py ---------------------------------------------- 13:06:50.049603 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 13:06:50.062078 tst aoip_ussd:nanobts+band-900: Using 1 x bts (candidates: 1) 13:06:50.101417 tst aoip_ussd:nanobts+band-900: using LAC 4997 13:06:50.109870 tst aoip_ussd:nanobts+band-900: using RAC 152 13:06:50.119662 tst aoip_ussd:nanobts+band-900: using CellId 4997 13:06:50.129417 tst aoip_ussd:nanobts+band-900: using BVCI 4998 13:06:50.130879 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 13:06:50.145488 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 13:06:50.157865 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 13:06:50.170143 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 13:06:50.182196 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 13:06:50.194710 tst aoip_ussd:nanobts+band-900: Using 1 x modem (candidates: 1) 13:06:50.264080 run osmo-hlr_10.42.42.2: Starting osmo-hlr 13:06:51.284225 run create_hlr_db(pid=13660): Launched 13:06:52.289246 run create_hlr_db(pid=13660): Terminated: ok {rc=0} 13:06:52.305507 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13662): Launched 13:06:52.321222 run osmo-hlr_10.42.42.2(pid=13663): Launched 13:06:52.322691 run osmo-stp_10.42.42.5: Starting osmo-stp 13:06:54.543567 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13664): Launched 13:06:54.557180 run osmo-stp_10.42.42.5(pid=13665): Launched 13:06:54.558837 run osmo-msc_10.42.42.6: Starting osmo-msc 13:06:57.813605 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13671): Launched 13:06:57.832526 run osmo-msc_10.42.42.6(pid=13672): Launched 13:06:57.833719 run osmo-mgw_10.42.42.3: Starting osmo-mgw 13:06:59.493705 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13673): Launched 13:06:59.513043 run osmo-mgw_10.42.42.3(pid=13674): Launched 13:06:59.514169 run osmo-mgw_10.42.42.4: Starting osmo-mgw 13:06:59.730369 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13676): Launched 13:06:59.752048 run osmo-mgw_10.42.42.4(pid=13677): Launched 13:06:59.753900 run osmo-bsc_10.42.42.7: Starting osmo-bsc 13:07:03.244622 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13678): Launched 13:07:03.260996 run osmo-bsc_10.42.42.7(pid=13679): Launched 13:07:04.794430 run pcap-recorder_any(filters='host 10.42.42.54 and port not 22')(pid=13694): Launched 13:07:04.796522 run nanobts_10.42.42.54: Finding nanobts 10.42.42.54, binding on 10.42.42.1... 13:07:04.812135 run abisip-find-preconf(pid=13695): Launched 13:12:05.206291 tst assert_extension.py:24: ERR: Error: Wait timeout [trial-683↪aoip_ussd:nanobts+band-900↪assert_extension.py:24] 13:12:05.212505 tst assert_extension.py:24: Test FAILED (315.2 sec) 13:12:05.216188 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13662): Terminating (SIGINT) 13:12:05.386335 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13662): Terminated: ok {rc=0} 13:12:05.387367 run osmo-hlr_10.42.42.2(pid=13663): Terminating (SIGINT) 13:12:05.561466 run osmo-hlr_10.42.42.2(pid=13663): Terminated {rc=1} 13:12:05.562473 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13664): Terminating (SIGINT) 13:12:05.733512 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13664): Terminated: ok {rc=0} 13:12:05.734462 run osmo-stp_10.42.42.5(pid=13665): Terminating (SIGINT) 13:12:05.904499 run osmo-stp_10.42.42.5(pid=13665): Terminated: ok {rc=0} 13:12:05.905417 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13671): Terminating (SIGINT) 13:12:06.076201 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13671): Terminated: ok {rc=0} 13:12:06.077179 run osmo-msc_10.42.42.6(pid=13672): Terminating (SIGINT) 13:12:09.484463 run osmo-msc_10.42.42.6(pid=13672): Terminated {rc=1} 13:12:09.485487 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13673): Terminating (SIGINT) 13:12:09.657261 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13673): Terminated: ok {rc=0} 13:12:09.659135 run osmo-mgw_10.42.42.3(pid=13674): Terminating (SIGINT) 13:12:09.668134 run osmo-mgw_10.42.42.3(pid=13674): Terminated {rc=-2} 13:12:09.669817 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13676): Terminating (SIGINT) 13:12:09.840810 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13676): Terminated: ok {rc=0} 13:12:09.842603 run osmo-mgw_10.42.42.4(pid=13677): Terminating (SIGINT) 13:12:09.852485 run osmo-mgw_10.42.42.4(pid=13677): Terminated {rc=-2} 13:12:09.854192 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13678): Terminating (SIGINT) 13:12:10.025372 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13678): Terminated: ok {rc=0} 13:12:10.026410 run osmo-bsc_10.42.42.7(pid=13679): Terminating (SIGINT) 13:12:13.434118 run osmo-bsc_10.42.42.7(pid=13679): Terminated {rc=1} 13:12:13.435123 run pcap-recorder_any(filters='host 10.42.42.54 and port not 22')(pid=13694): Terminating (SIGINT) 13:12:13.605417 run pcap-recorder_any(filters='host 10.42.42.54 and port not 22')(pid=13694): Terminated: ok {rc=0} 13:12:13.606444 run abisip-find-preconf(pid=13695): Terminating (SIGINT) 13:12:13.777513 run abisip-find-preconf(pid=13695): Terminated {rc=-2} 13:12:13.787153 bus /sierra_1: Setting Powered False --------------------------------------------------------------------- trial-683 aoip_ussd:nanobts+band-900 FAIL --------------------------------------------------------------------- --------------------------------------------------------------------- trial-683 aoip_ussd:nanobts+band-1900 --------------------------------------------------------------------- 13:12:15.085965 tst aoip_ussd:nanobts+band-1900: reserving resources in /var/tmp/osmo-gsm-tester/state ... 13:12:15.100329 tst aoip_ussd:nanobts+band-1900: Reserving 1 x bts (candidates: 6) 13:12:15.125351 tst aoip_ussd:nanobts+band-1900: Reserving 6 x ip_address (candidates: 8) 13:12:15.160352 tst aoip_ussd:nanobts+band-1900: Reserving 1 x modem (candidates: 4) ---------------------------------------------- trial-683 aoip_ussd:nanobts+band-1900 assert_extension.py ---------------------------------------------- 13:12:15.298949 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 13:12:15.311687 tst aoip_ussd:nanobts+band-1900: Using 1 x bts (candidates: 1) 13:12:15.348237 tst aoip_ussd:nanobts+band-1900: using LAC 4998 13:12:15.356127 tst aoip_ussd:nanobts+band-1900: using RAC 153 13:12:15.364369 tst aoip_ussd:nanobts+band-1900: using CellId 4998 13:12:15.373680 tst aoip_ussd:nanobts+band-1900: using BVCI 4999 13:12:15.375418 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 13:12:15.388546 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 13:12:15.401323 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 13:12:15.414654 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 13:12:15.427098 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 13:12:15.439735 tst aoip_ussd:nanobts+band-1900: Using 1 x modem (candidates: 1) 13:12:15.505665 run osmo-hlr_10.42.42.2: Starting osmo-hlr 13:12:15.633118 run create_hlr_db(pid=13792): Launched 13:12:15.635945 bus /sierra_1: Setting Powered False 13:12:16.638765 run create_hlr_db(pid=13792): Terminated: ok {rc=0} 13:12:16.663545 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13794): Launched 13:12:16.678977 run osmo-hlr_10.42.42.2(pid=13795): Launched 13:12:16.680904 run osmo-stp_10.42.42.5: Starting osmo-stp 13:12:16.818940 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13796): Launched 13:12:16.833252 run osmo-stp_10.42.42.5(pid=13797): Launched 13:12:16.835036 run osmo-msc_10.42.42.6: Starting osmo-msc 13:12:16.974715 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13798): Launched 13:12:16.990070 run osmo-msc_10.42.42.6(pid=13799): Launched 13:12:16.992375 run osmo-mgw_10.42.42.3: Starting osmo-mgw 13:12:17.130337 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13800): Launched 13:12:17.148294 run osmo-mgw_10.42.42.3(pid=13801): Launched 13:12:17.153191 run osmo-mgw_10.42.42.4: Starting osmo-mgw 13:12:17.317197 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13802): Launched 13:12:17.332854 run osmo-mgw_10.42.42.4(pid=13803): Launched 13:12:17.334615 run osmo-bsc_10.42.42.7: Starting osmo-bsc 13:12:17.686766 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13804): Launched 13:12:17.701704 run osmo-bsc_10.42.42.7(pid=13805): Launched 13:12:19.174304 run pcap-recorder_any(filters='host 10.42.42.53 and port not 22')(pid=13819): Launched 13:12:19.176323 run nanobts_10.42.42.53: Finding nanobts 10.42.42.53, binding on 10.42.42.1... 13:12:19.192274 run abisip-find-preconf(pid=13820): Launched 13:17:19.588271 tst assert_extension.py:24: ERR: Error: Wait timeout [trial-683↪aoip_ussd:nanobts+band-1900↪assert_extension.py:24] 13:17:19.594024 tst assert_extension.py:24: Test FAILED (304.3 sec) 13:17:19.597808 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13794): Terminating (SIGINT) 13:17:19.768710 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13794): Terminated: ok {rc=0} 13:17:19.770475 run osmo-hlr_10.42.42.2(pid=13795): Terminating (SIGINT) 13:17:19.946483 run osmo-hlr_10.42.42.2(pid=13795): Terminated {rc=1} 13:17:19.947423 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13796): Terminating (SIGINT) 13:17:20.119285 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13796): Terminated: ok {rc=0} 13:17:20.120421 run osmo-stp_10.42.42.5(pid=13797): Terminating (SIGINT) 13:17:20.293599 run osmo-stp_10.42.42.5(pid=13797): Terminated: ok {rc=0} 13:17:20.294672 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13798): Terminating (SIGINT) 13:17:20.465531 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13798): Terminated: ok {rc=0} 13:17:20.466593 run osmo-msc_10.42.42.6(pid=13799): Terminating (SIGINT) 13:17:23.874269 run osmo-msc_10.42.42.6(pid=13799): Terminated {rc=1} 13:17:23.875297 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13800): Terminating (SIGINT) 13:17:24.045811 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13800): Terminated: ok {rc=0} 13:17:24.046858 run osmo-mgw_10.42.42.3(pid=13801): Terminating (SIGINT) 13:17:24.217049 run osmo-mgw_10.42.42.3(pid=13801): Terminated {rc=-2} 13:17:24.218124 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13802): Terminating (SIGINT) 13:17:24.388420 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13802): Terminated: ok {rc=0} 13:17:24.389574 run osmo-mgw_10.42.42.4(pid=13803): Terminating (SIGINT) 13:17:24.559703 run osmo-mgw_10.42.42.4(pid=13803): Terminated {rc=-2} 13:17:24.560872 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13804): Terminating (SIGINT) 13:17:24.731615 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13804): Terminated: ok {rc=0} 13:17:24.733369 run osmo-bsc_10.42.42.7(pid=13805): Terminating (SIGINT) 13:17:28.141263 run osmo-bsc_10.42.42.7(pid=13805): Terminated {rc=1} 13:17:28.142287 run pcap-recorder_any(filters='host 10.42.42.53 and port not 22')(pid=13819): Terminating (SIGINT) 13:17:28.312589 run pcap-recorder_any(filters='host 10.42.42.53 and port not 22')(pid=13819): Terminated: ok {rc=0} 13:17:28.313641 run abisip-find-preconf(pid=13820): Terminating (SIGINT) 13:17:28.483838 run abisip-find-preconf(pid=13820): Terminated {rc=-2} 13:17:28.493689 bus /sierra_1: Setting Powered False --------------------------------------------------------------------- trial-683 aoip_ussd:nanobts+band-1900 FAIL --------------------------------------------------------------------- 13:17:29.794991 tst trial-683: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683/run.2018-06-27_13-06-48/trial-683.xml -------------------------------------------------------------------------------------------- trial-683 FAIL -------------------------------------------------------------------------------------------- 13:17:29.826511 tst trial-683: trial-683: FAIL, 2 suites failed FAIL: aoip_ussd:nanobts+band-900 (fail: 1) FAIL: assert_extension.py (315.2 sec) Error: Wait timeout FAIL: aoip_ussd:nanobts+band-1900 (fail: 1) FAIL: assert_extension.py (304.3 sec) Error: Wait timeout + rm -rf trial-683/inst + cd trial-683 + readlink last_run + journalctl -u ofono -o short-precise --since 2018-06-27 13:06:47 + readlink last_run + tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683-run.tgz run.2018-06-27_13-06-48 + tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683-bin.tgz checksums.md5 osmo-bsc.build-63.tgz osmo-bts-sysmo.build-63.tgz osmo-bts.build-63.tgz osmo-ggsn.build-63.tgz osmo-hlr.build-63.tgz osmo-mgw.build-63.tgz osmo-msc.build-63.tgz osmo-nitb.build-63.tgz osmo-pcu-sysmo.build-63.tgz osmo-pcu.build-63.tgz osmo-sgsn.build-63.tgz osmo-stp.build-63.tgz osmo-trx.build-63.tgz + exit 1 Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Finished: FAILURE