Started by upstream project "osmo-gsm-tester_manual-build-all" build number 60 originally caused by: Started by user Pau Espin Pedrol Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_manual-run-all > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url git://git.osmocom.org/osmo-gsm-tester # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Fetching upstream changes from git://git.osmocom.org/osmo-gsm-tester > git --version # timeout=10 > git fetch --tags --progress git://git.osmocom.org/osmo-gsm-tester +refs/heads/*:refs/remotes/origin/* > git rev-parse pespin/mainloop^{commit} # timeout=10 > git rev-parse refs/remotes/origin/pespin/mainloop^{commit} # timeout=10 Checking out Revision 796bf3954170ad9515af9cb37b1abdf1fce1774b (refs/remotes/origin/pespin/mainloop) > git config core.sparsecheckout # timeout=10 > git checkout -f 796bf3954170ad9515af9cb37b1abdf1fce1774b Commit message: "nanobts: Attemt at fixing race condition using ipaccess-config after power up" > git rev-list --no-walk 796bf3954170ad9515af9cb37b1abdf1fce1774b # timeout=10 Copied 26 artifacts from "osmo-gsm-tester_manual-build-all" build number 60 [osmo-gsm-tester_manual-run-all] $ /bin/sh /tmp/jenkins2186669696291103395.sh + export OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/example + unlink osmo-gsm-tester/example/resources.conf unlink: cannot unlink 'osmo-gsm-tester/example/resources.conf': No such file or directory + true + ln -s resources.conf.prod osmo-gsm-tester/example/resources.conf + PATH=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games ./osmo-gsm-tester/contrib/jenkins-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all + date +%F %T + time_start=2018-04-16 17:34:47 + trial_dir_prefix=trial- + rm -rf trial-681 trial-681-bin.tgz trial-681-run.tgz + trial_dir=trial-682 + mkdir -p trial-682 + mv osmo-bsc.build-60.tgz osmo-bts-sysmo.build-60.tgz osmo-bts.build-60.tgz osmo-ggsn.build-60.tgz osmo-hlr.build-60.tgz osmo-mgw.build-60.tgz osmo-msc.build-60.tgz osmo-nitb.build-60.tgz osmo-pcu-sysmo.build-60.tgz osmo-pcu.build-60.tgz osmo-sgsn.build-60.tgz osmo-stp.build-60.tgz osmo-trx.build-60.tgz trial-682 + cat osmo-bsc.build-60.md5 osmo-bts-sysmo.build-60.md5 osmo-bts.build-60.md5 osmo-ggsn.build-60.md5 osmo-hlr.build-60.md5 osmo-mgw.build-60.md5 osmo-msc.build-60.md5 osmo-nitb.build-60.md5 osmo-pcu-sysmo.build-60.md5 osmo-pcu.build-60.md5 osmo-sgsn.build-60.md5 osmo-stp.build-60.md5 osmo-trx.build-60.md5 + rm osmo-bsc.build-60.md5 osmo-bts-sysmo.build-60.md5 osmo-bts.build-60.md5 osmo-ggsn.build-60.md5 osmo-hlr.build-60.md5 osmo-mgw.build-60.md5 osmo-msc.build-60.md5 osmo-nitb.build-60.md5 osmo-pcu-sysmo.build-60.md5 osmo-pcu.build-60.md5 osmo-sgsn.build-60.md5 osmo-stp.build-60.md5 osmo-trx.build-60.md5 + exit_code=1 + which osmo-gsm-tester.py + python3 -u /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src/osmo-gsm-tester.py trial-682 -s aoip_ussd:nanobts+band-900 -s aoip_ussd:nanobts+band-1900 combinations: ['aoip_ussd:nanobts+band-900', 'aoip_ussd:nanobts+band-1900'] series: None trial: 'trial-682' tests: None -------------------------------------------------------------------------------------------- trial-682 -------------------------------------------------------------------------------------------- 17:34:48.933358 tst trial-682: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682/run.2018-04-16_17-34-48/log --------------------------------------------------------------------- trial-682 aoip_ussd:nanobts+band-900 --------------------------------------------------------------------- 17:34:50.103407 tst aoip_ussd:nanobts+band-900: reserving resources in /var/tmp/osmo-gsm-tester/state ... 17:34:50.128974 tst aoip_ussd:nanobts+band-900: Reserving 1 x bts (candidates: 6) 17:34:50.152073 tst aoip_ussd:nanobts+band-900: Reserving 6 x ip_address (candidates: 8) 17:34:50.184768 tst aoip_ussd:nanobts+band-900: Reserving 1 x modem (candidates: 4) ---------------------------------------------- trial-682 aoip_ussd:nanobts+band-900 assert_extension.py ---------------------------------------------- 17:34:50.399820 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 17:34:50.411776 tst aoip_ussd:nanobts+band-900: Using 1 x bts (candidates: 1) 17:34:50.451202 tst aoip_ussd:nanobts+band-900: using LAC 2843 17:34:50.465644 tst aoip_ussd:nanobts+band-900: using RAC 38 17:34:50.480073 tst aoip_ussd:nanobts+band-900: using CellId 2843 17:34:50.493551 tst aoip_ussd:nanobts+band-900: using BVCI 2844 17:34:50.495014 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 17:34:50.508048 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 17:34:50.520021 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 17:34:50.532637 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 17:34:50.544473 tst aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6) 17:34:50.557132 tst aoip_ussd:nanobts+band-900: Using 1 x modem (candidates: 1) 17:34:50.627428 run osmo-hlr_10.42.42.2: Starting osmo-hlr 17:34:51.604950 run create_hlr_db(pid=27520): Launched 17:34:52.609886 run create_hlr_db(pid=27520): Terminated: ok {rc=0} 17:34:52.626176 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27522): Launched 17:34:52.639664 run osmo-hlr_10.42.42.2(pid=27523): Launched 17:34:52.640946 run osmo-stp_10.42.42.5: Starting osmo-stp 17:34:54.891875 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27524): Launched 17:34:54.905744 run osmo-stp_10.42.42.5(pid=27525): Launched 17:34:54.907470 run osmo-msc_10.42.42.6: Starting osmo-msc 17:34:58.201340 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27526): Launched 17:34:58.215856 run osmo-msc_10.42.42.6(pid=27527): Launched 17:34:58.217139 run osmo-mgw_10.42.42.3: Starting osmo-mgw 17:34:59.466162 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27528): Launched 17:34:59.482315 run osmo-mgw_10.42.42.3(pid=27529): Launched 17:34:59.485888 run osmo-mgw_10.42.42.4: Starting osmo-mgw 17:34:59.612784 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27530): Launched 17:34:59.628146 run osmo-mgw_10.42.42.4(pid=27531): Launched 17:34:59.629928 run osmo-bsc_10.42.42.7: Starting osmo-bsc 17:35:03.003654 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27532): Launched 17:35:03.018288 run osmo-bsc_10.42.42.7(pid=27533): Launched 17:35:03.409781 run osmo-msc_10.42.42.6(pid=27527): ERR: Terminated: ERROR {rc=1} [trial-682↪aoip_ussd:nanobts+band-900↪osmo-msc_10.42.42.6↪osmo-msc_10.42.42.6(pid=27527)] 17:35:03.411418 run osmo-msc_10.42.42.6(pid=27527): stderr: | Global redzone: f9 | Global init order: f6 | Poisoned by user: f7 | Container overflow: fc | Array cookie: ac | Intra object redzone: bb | ASan internal: fe | Left alloca redzone: ca | Right alloca redzone: cb | ==27527==ABORTING 17:35:03.412819 run osmo-msc_10.42.42.6(pid=27527): stdout: | (launched: 2018-04-16_17:34:58.205822) 17:35:03.413990 run osmo-msc_10.42.42.6(pid=27527): stderr: | Global redzone: f9 | Global init order: f6 | Poisoned by user: f7 | Container overflow: fc | Array cookie: ac | Intra object redzone: bb | ASan internal: fe | Left alloca redzone: ca | Right alloca redzone: cb | ==27527==ABORTING 17:35:03.419372 tst assert_extension.py:24: ERR: Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27527) [trial-682↪aoip_ussd:nanobts+band-900↪assert_extension.py:24] 17:35:03.425586 tst assert_extension.py:24: Test FAILED (13.0 sec) 17:35:03.429833 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27522): Terminating (SIGINT) 17:35:03.600547 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27522): Terminated: ok {rc=0} 17:35:03.601566 run osmo-hlr_10.42.42.2(pid=27523): Terminating (SIGINT) 17:35:03.771619 run osmo-hlr_10.42.42.2(pid=27523): Terminated {rc=1} 17:35:03.772766 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27524): Terminating (SIGINT) 17:35:03.943554 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27524): Terminated: ok {rc=0} 17:35:03.944724 run osmo-stp_10.42.42.5(pid=27525): Terminating (SIGINT) 17:35:04.117577 run osmo-stp_10.42.42.5(pid=27525): Terminated: ok {rc=0} 17:35:04.118578 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27526): Terminating (SIGINT) 17:35:04.288772 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27526): Terminated: ok {rc=0} 17:35:04.289809 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27528): Terminating (SIGINT) 17:35:04.459910 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27528): Terminated: ok {rc=0} 17:35:04.461107 run osmo-mgw_10.42.42.3(pid=27529): Terminating (SIGINT) 17:35:04.469348 run osmo-mgw_10.42.42.3(pid=27529): Terminated {rc=-2} 17:35:04.470313 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27530): Terminating (SIGINT) 17:35:04.640624 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27530): Terminated: ok {rc=0} 17:35:04.641676 run osmo-mgw_10.42.42.4(pid=27531): Terminating (SIGINT) 17:35:04.817119 run osmo-mgw_10.42.42.4(pid=27531): Terminated {rc=-2} 17:35:04.818068 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27532): Terminating (SIGINT) 17:35:04.988793 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27532): Terminated: ok {rc=0} 17:35:04.989781 run osmo-bsc_10.42.42.7(pid=27533): Terminating (SIGINT) 17:35:08.397601 run osmo-bsc_10.42.42.7(pid=27533): Terminated: ok {rc=0} 17:35:08.407042 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-682 aoip_ussd:nanobts+band-900 FAIL --------------------------------------------------------------------- --------------------------------------------------------------------- trial-682 aoip_ussd:nanobts+band-1900 --------------------------------------------------------------------- 17:35:09.722002 tst aoip_ussd:nanobts+band-1900: reserving resources in /var/tmp/osmo-gsm-tester/state ... 17:35:09.737788 tst aoip_ussd:nanobts+band-1900: Reserving 1 x bts (candidates: 6) 17:35:09.761778 tst aoip_ussd:nanobts+band-1900: Reserving 6 x ip_address (candidates: 8) 17:35:09.795851 tst aoip_ussd:nanobts+band-1900: Reserving 1 x modem (candidates: 4) ---------------------------------------------- trial-682 aoip_ussd:nanobts+band-1900 assert_extension.py ---------------------------------------------- 17:35:09.933667 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 17:35:09.946265 tst aoip_ussd:nanobts+band-1900: Using 1 x bts (candidates: 1) 17:35:09.987126 tst aoip_ussd:nanobts+band-1900: using LAC 2844 17:35:10.001291 tst aoip_ussd:nanobts+band-1900: using RAC 39 17:35:10.015537 tst aoip_ussd:nanobts+band-1900: using CellId 2844 17:35:10.029351 tst aoip_ussd:nanobts+band-1900: using BVCI 2845 17:35:10.031194 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 17:35:10.045005 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 17:35:10.056808 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 17:35:10.068632 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 17:35:10.080292 tst aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6) 17:35:10.091962 tst aoip_ussd:nanobts+band-1900: Using 1 x modem (candidates: 1) 17:35:10.159701 run osmo-hlr_10.42.42.2: Starting osmo-hlr 17:35:10.290048 run create_hlr_db(pid=27551): Launched 17:35:10.292819 bus /sierra_2: Setting Powered False 17:35:11.294778 run create_hlr_db(pid=27551): Terminated: ok {rc=0} 17:35:11.321621 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27553): Launched 17:35:11.336444 run osmo-hlr_10.42.42.2(pid=27554): Launched 17:35:11.337609 run osmo-stp_10.42.42.5: Starting osmo-stp 17:35:11.464360 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27555): Launched 17:35:11.480076 run osmo-stp_10.42.42.5(pid=27556): Launched 17:35:11.481302 run osmo-msc_10.42.42.6: Starting osmo-msc 17:35:11.611486 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27557): Launched 17:35:11.627392 run osmo-msc_10.42.42.6(pid=27558): Launched 17:35:11.628741 run osmo-mgw_10.42.42.3: Starting osmo-mgw 17:35:11.758354 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27559): Launched 17:35:11.778561 run osmo-mgw_10.42.42.3(pid=27560): Launched 17:35:11.779677 run osmo-mgw_10.42.42.4: Starting osmo-mgw 17:35:11.912991 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27561): Launched 17:35:11.927555 run osmo-mgw_10.42.42.4(pid=27562): Launched 17:35:11.928845 run osmo-bsc_10.42.42.7: Starting osmo-bsc 17:35:12.290052 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27563): Launched 17:35:12.305636 run osmo-bsc_10.42.42.7(pid=27564): Launched 17:35:12.646093 run osmo-msc_10.42.42.6(pid=27558): ERR: Terminated: ERROR {rc=1} [trial-682↪aoip_ussd:nanobts+band-1900↪osmo-msc_10.42.42.6↪osmo-msc_10.42.42.6(pid=27558)] 17:35:12.647529 run osmo-msc_10.42.42.6(pid=27558): stderr: | Global redzone: f9 | Global init order: f6 | Poisoned by user: f7 | Container overflow: fc | Array cookie: ac | Intra object redzone: bb | ASan internal: fe | Left alloca redzone: ca | Right alloca redzone: cb | ==27558==ABORTING 17:35:12.648614 run osmo-msc_10.42.42.6(pid=27558): stdout: | (launched: 2018-04-16_17:35:11.616891) 17:35:12.649589 run osmo-msc_10.42.42.6(pid=27558): stderr: | Global redzone: f9 | Global init order: f6 | Poisoned by user: f7 | Container overflow: fc | Array cookie: ac | Intra object redzone: bb | ASan internal: fe | Left alloca redzone: ca | Right alloca redzone: cb | ==27558==ABORTING 17:35:12.652085 tst assert_extension.py:24: ERR: Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27558) [trial-682↪aoip_ussd:nanobts+band-1900↪assert_extension.py:24] 17:35:12.658180 tst assert_extension.py:24: Test FAILED (2.7 sec) 17:35:12.662517 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27553): Terminating (SIGINT) 17:35:12.832550 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27553): Terminated: ok {rc=0} 17:35:12.833535 run osmo-hlr_10.42.42.2(pid=27554): Terminating (SIGINT) 17:35:13.003530 run osmo-hlr_10.42.42.2(pid=27554): Terminated {rc=1} 17:35:13.004611 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27555): Terminating (SIGINT) 17:35:13.175368 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27555): Terminated: ok {rc=0} 17:35:13.176448 run osmo-stp_10.42.42.5(pid=27556): Terminating (SIGINT) 17:35:13.350476 run osmo-stp_10.42.42.5(pid=27556): Terminated: ok {rc=0} 17:35:13.351406 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27557): Terminating (SIGINT) 17:35:13.522020 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27557): Terminated: ok {rc=0} 17:35:13.522978 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27559): Terminating (SIGINT) 17:35:13.693632 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27559): Terminated: ok {rc=0} 17:35:13.694592 run osmo-mgw_10.42.42.3(pid=27560): Terminating (SIGINT) 17:35:13.865909 run osmo-mgw_10.42.42.3(pid=27560): Terminated {rc=-2} 17:35:13.866922 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27561): Terminating (SIGINT) 17:35:14.037822 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27561): Terminated: ok {rc=0} 17:35:14.038873 run osmo-mgw_10.42.42.4(pid=27562): Terminating (SIGINT) 17:35:14.047167 run osmo-mgw_10.42.42.4(pid=27562): Terminated {rc=-2} 17:35:14.048142 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27563): Terminating (SIGINT) 17:35:14.218822 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27563): Terminated: ok {rc=0} 17:35:14.219857 run osmo-bsc_10.42.42.7(pid=27564): Terminating (SIGINT) 17:35:17.628094 run osmo-bsc_10.42.42.7(pid=27564): Terminated: ok {rc=0} 17:35:17.637501 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-682 aoip_ussd:nanobts+band-1900 FAIL --------------------------------------------------------------------- 17:35:18.951206 tst trial-682: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682/run.2018-04-16_17-34-48/trial-682.xml -------------------------------------------------------------------------------------------- trial-682 FAIL -------------------------------------------------------------------------------------------- 17:35:18.983087 tst trial-682: trial-682: FAIL, 2 suites failed FAIL: aoip_ussd:nanobts+band-900 (fail: 1) FAIL: assert_extension.py (13.0 sec) Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27527) FAIL: aoip_ussd:nanobts+band-1900 (fail: 1) FAIL: assert_extension.py (2.7 sec) Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27558) + rm -rf trial-682/inst + cd trial-682 + readlink last_run + journalctl -u ofono -o short-precise --since 2018-04-16 17:34:47 + readlink last_run + tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682-run.tgz run.2018-04-16_17-34-48 + tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682-bin.tgz checksums.md5 osmo-bsc.build-60.tgz osmo-bts-sysmo.build-60.tgz osmo-bts.build-60.tgz osmo-ggsn.build-60.tgz osmo-hlr.build-60.tgz osmo-mgw.build-60.tgz osmo-msc.build-60.tgz osmo-nitb.build-60.tgz osmo-pcu-sysmo.build-60.tgz osmo-pcu.build-60.tgz osmo-sgsn.build-60.tgz osmo-stp.build-60.tgz osmo-trx.build-60.tgz + exit 1 Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Finished: FAILURE