FailedConsole Output

Started by upstream project "osmo-gsm-tester_manual-build-all" build number 60
originally caused by:
 Started by user Pau Espin Pedrol
Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_manual-run-all
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git://git.osmocom.org/osmo-gsm-tester # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
Fetching upstream changes from git://git.osmocom.org/osmo-gsm-tester
 > git --version # timeout=10
 > git fetch --tags --progress git://git.osmocom.org/osmo-gsm-tester +refs/heads/*:refs/remotes/origin/*
 > git rev-parse pespin/mainloop^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/pespin/mainloop^{commit} # timeout=10
Checking out Revision 796bf3954170ad9515af9cb37b1abdf1fce1774b (refs/remotes/origin/pespin/mainloop)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 796bf3954170ad9515af9cb37b1abdf1fce1774b
Commit message: "nanobts: Attemt at fixing race condition using ipaccess-config after power up"
 > git rev-list --no-walk 796bf3954170ad9515af9cb37b1abdf1fce1774b # timeout=10
Copied 26 artifacts from "osmo-gsm-tester_manual-build-all" build number 60
[osmo-gsm-tester_manual-run-all] $ /bin/sh /tmp/jenkins2186669696291103395.sh
+ export OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/example
+ unlink osmo-gsm-tester/example/resources.conf
unlink: cannot unlink 'osmo-gsm-tester/example/resources.conf': No such file or directory
+ true
+ ln -s resources.conf.prod osmo-gsm-tester/example/resources.conf
+ PATH=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games ./osmo-gsm-tester/contrib/jenkins-run.sh
+ base=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all
+ date +%F %T
+ time_start=2018-04-16 17:34:47
+ trial_dir_prefix=trial-
+ rm -rf trial-681 trial-681-bin.tgz trial-681-run.tgz
+ trial_dir=trial-682
+ mkdir -p trial-682
+ mv osmo-bsc.build-60.tgz osmo-bts-sysmo.build-60.tgz osmo-bts.build-60.tgz osmo-ggsn.build-60.tgz osmo-hlr.build-60.tgz osmo-mgw.build-60.tgz osmo-msc.build-60.tgz osmo-nitb.build-60.tgz osmo-pcu-sysmo.build-60.tgz osmo-pcu.build-60.tgz osmo-sgsn.build-60.tgz osmo-stp.build-60.tgz osmo-trx.build-60.tgz trial-682
+ cat osmo-bsc.build-60.md5 osmo-bts-sysmo.build-60.md5 osmo-bts.build-60.md5 osmo-ggsn.build-60.md5 osmo-hlr.build-60.md5 osmo-mgw.build-60.md5 osmo-msc.build-60.md5 osmo-nitb.build-60.md5 osmo-pcu-sysmo.build-60.md5 osmo-pcu.build-60.md5 osmo-sgsn.build-60.md5 osmo-stp.build-60.md5 osmo-trx.build-60.md5
+ rm osmo-bsc.build-60.md5 osmo-bts-sysmo.build-60.md5 osmo-bts.build-60.md5 osmo-ggsn.build-60.md5 osmo-hlr.build-60.md5 osmo-mgw.build-60.md5 osmo-msc.build-60.md5 osmo-nitb.build-60.md5 osmo-pcu-sysmo.build-60.md5 osmo-pcu.build-60.md5 osmo-sgsn.build-60.md5 osmo-stp.build-60.md5 osmo-trx.build-60.md5
+ exit_code=1
+ which osmo-gsm-tester.py
+ python3 -u /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src/osmo-gsm-tester.py trial-682 -s aoip_ussd:nanobts+band-900 -s aoip_ussd:nanobts+band-1900
combinations: ['aoip_ussd:nanobts+band-900', 'aoip_ussd:nanobts+band-1900']
series: None
trial: 'trial-682'
tests: None

--------------------------------------------------------------------------------------------
trial-682
--------------------------------------------------------------------------------------------
17:34:48.933358 tst                        trial-682: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682/run.2018-04-16_17-34-48/log

---------------------------------------------------------------------
trial-682 aoip_ussd:nanobts+band-900
---------------------------------------------------------------------
17:34:50.103407 tst       aoip_ussd:nanobts+band-900: reserving resources in /var/tmp/osmo-gsm-tester/state ...
17:34:50.128974 tst       aoip_ussd:nanobts+band-900: Reserving 1 x bts (candidates: 6)
17:34:50.152073 tst       aoip_ussd:nanobts+band-900: Reserving 6 x ip_address (candidates: 8)
17:34:50.184768 tst       aoip_ussd:nanobts+band-900: Reserving 1 x modem (candidates: 4)

----------------------------------------------
trial-682 aoip_ussd:nanobts+band-900 assert_extension.py
----------------------------------------------
17:34:50.399820 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
17:34:50.411776 tst       aoip_ussd:nanobts+band-900: Using 1 x bts (candidates: 1)
17:34:50.451202 tst       aoip_ussd:nanobts+band-900: using LAC 2843
17:34:50.465644 tst       aoip_ussd:nanobts+band-900: using RAC 38
17:34:50.480073 tst       aoip_ussd:nanobts+band-900: using CellId 2843
17:34:50.493551 tst       aoip_ussd:nanobts+band-900: using BVCI 2844
17:34:50.495014 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
17:34:50.508048 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
17:34:50.520021 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
17:34:50.532637 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
17:34:50.544473 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
17:34:50.557132 tst       aoip_ussd:nanobts+band-900: Using 1 x modem (candidates: 1)
17:34:50.627428 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
17:34:51.604950 run         create_hlr_db(pid=27520): Launched
17:34:52.609886 run         create_hlr_db(pid=27520): Terminated: ok {rc=0}
17:34:52.626176 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27522): Launched
17:34:52.639664 run   osmo-hlr_10.42.42.2(pid=27523): Launched
17:34:52.640946 run              osmo-stp_10.42.42.5: Starting osmo-stp
17:34:54.891875 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27524): Launched
17:34:54.905744 run   osmo-stp_10.42.42.5(pid=27525): Launched
17:34:54.907470 run              osmo-msc_10.42.42.6: Starting osmo-msc
17:34:58.201340 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27526): Launched
17:34:58.215856 run   osmo-msc_10.42.42.6(pid=27527): Launched
17:34:58.217139 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
17:34:59.466162 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27528): Launched
17:34:59.482315 run   osmo-mgw_10.42.42.3(pid=27529): Launched
17:34:59.485888 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
17:34:59.612784 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27530): Launched
17:34:59.628146 run   osmo-mgw_10.42.42.4(pid=27531): Launched
17:34:59.629928 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
17:35:03.003654 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27532): Launched
17:35:03.018288 run   osmo-bsc_10.42.42.7(pid=27533): Launched
17:35:03.409781 run   osmo-msc_10.42.42.6(pid=27527): ERR: Terminated: ERROR {rc=1}  [trial-682↪aoip_ussd:nanobts+band-900↪osmo-msc_10.42.42.6↪osmo-msc_10.42.42.6(pid=27527)]
17:35:03.411418 run   osmo-msc_10.42.42.6(pid=27527): stderr:
 |   Global redzone:          f9
|   Global init order:       f6
|   Poisoned by user:        f7
|   Container overflow:      fc
|   Array cookie:            ac
|   Intra object redzone:    bb
|   ASan internal:           fe
|   Left alloca redzone:     ca
|   Right alloca redzone:    cb
| ==27527==ABORTING 
17:35:03.412819 run   osmo-msc_10.42.42.6(pid=27527): stdout:
 | (launched: 2018-04-16_17:34:58.205822) 
17:35:03.413990 run   osmo-msc_10.42.42.6(pid=27527): stderr:
 |   Global redzone:          f9
|   Global init order:       f6
|   Poisoned by user:        f7
|   Container overflow:      fc
|   Array cookie:            ac
|   Intra object redzone:    bb
|   ASan internal:           fe
|   Left alloca redzone:     ca
|   Right alloca redzone:    cb
| ==27527==ABORTING 
17:35:03.419372 tst           assert_extension.py:24: ERR: Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27527)  [trial-682↪aoip_ussd:nanobts+band-900↪assert_extension.py:24]
17:35:03.425586 tst           assert_extension.py:24: Test FAILED (13.0 sec)
17:35:03.429833 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27522): Terminating (SIGINT)
17:35:03.600547 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27522): Terminated: ok {rc=0}
17:35:03.601566 run   osmo-hlr_10.42.42.2(pid=27523): Terminating (SIGINT)
17:35:03.771619 run   osmo-hlr_10.42.42.2(pid=27523): Terminated {rc=1}
17:35:03.772766 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27524): Terminating (SIGINT)
17:35:03.943554 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27524): Terminated: ok {rc=0}
17:35:03.944724 run   osmo-stp_10.42.42.5(pid=27525): Terminating (SIGINT)
17:35:04.117577 run   osmo-stp_10.42.42.5(pid=27525): Terminated: ok {rc=0}
17:35:04.118578 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27526): Terminating (SIGINT)
17:35:04.288772 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27526): Terminated: ok {rc=0}
17:35:04.289809 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27528): Terminating (SIGINT)
17:35:04.459910 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27528): Terminated: ok {rc=0}
17:35:04.461107 run   osmo-mgw_10.42.42.3(pid=27529): Terminating (SIGINT)
17:35:04.469348 run   osmo-mgw_10.42.42.3(pid=27529): Terminated {rc=-2}
17:35:04.470313 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27530): Terminating (SIGINT)
17:35:04.640624 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27530): Terminated: ok {rc=0}
17:35:04.641676 run   osmo-mgw_10.42.42.4(pid=27531): Terminating (SIGINT)
17:35:04.817119 run   osmo-mgw_10.42.42.4(pid=27531): Terminated {rc=-2}
17:35:04.818068 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27532): Terminating (SIGINT)
17:35:04.988793 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27532): Terminated: ok {rc=0}
17:35:04.989781 run   osmo-bsc_10.42.42.7(pid=27533): Terminating (SIGINT)
17:35:08.397601 run   osmo-bsc_10.42.42.7(pid=27533): Terminated: ok {rc=0}
17:35:08.407042 bus                        /sierra_2: Setting Powered False
---------------------------------------------------------------------
trial-682 aoip_ussd:nanobts+band-900 FAIL
---------------------------------------------------------------------

---------------------------------------------------------------------
trial-682 aoip_ussd:nanobts+band-1900
---------------------------------------------------------------------
17:35:09.722002 tst      aoip_ussd:nanobts+band-1900: reserving resources in /var/tmp/osmo-gsm-tester/state ...
17:35:09.737788 tst      aoip_ussd:nanobts+band-1900: Reserving 1 x bts (candidates: 6)
17:35:09.761778 tst      aoip_ussd:nanobts+band-1900: Reserving 6 x ip_address (candidates: 8)
17:35:09.795851 tst      aoip_ussd:nanobts+band-1900: Reserving 1 x modem (candidates: 4)

----------------------------------------------
trial-682 aoip_ussd:nanobts+band-1900 assert_extension.py
----------------------------------------------
17:35:09.933667 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
17:35:09.946265 tst      aoip_ussd:nanobts+band-1900: Using 1 x bts (candidates: 1)
17:35:09.987126 tst      aoip_ussd:nanobts+band-1900: using LAC 2844
17:35:10.001291 tst      aoip_ussd:nanobts+band-1900: using RAC 39
17:35:10.015537 tst      aoip_ussd:nanobts+band-1900: using CellId 2844
17:35:10.029351 tst      aoip_ussd:nanobts+band-1900: using BVCI 2845
17:35:10.031194 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
17:35:10.045005 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
17:35:10.056808 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
17:35:10.068632 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
17:35:10.080292 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
17:35:10.091962 tst      aoip_ussd:nanobts+band-1900: Using 1 x modem (candidates: 1)
17:35:10.159701 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
17:35:10.290048 run         create_hlr_db(pid=27551): Launched
17:35:10.292819 bus                        /sierra_2: Setting Powered False
17:35:11.294778 run         create_hlr_db(pid=27551): Terminated: ok {rc=0}
17:35:11.321621 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27553): Launched
17:35:11.336444 run   osmo-hlr_10.42.42.2(pid=27554): Launched
17:35:11.337609 run              osmo-stp_10.42.42.5: Starting osmo-stp
17:35:11.464360 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27555): Launched
17:35:11.480076 run   osmo-stp_10.42.42.5(pid=27556): Launched
17:35:11.481302 run              osmo-msc_10.42.42.6: Starting osmo-msc
17:35:11.611486 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27557): Launched
17:35:11.627392 run   osmo-msc_10.42.42.6(pid=27558): Launched
17:35:11.628741 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
17:35:11.758354 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27559): Launched
17:35:11.778561 run   osmo-mgw_10.42.42.3(pid=27560): Launched
17:35:11.779677 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
17:35:11.912991 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27561): Launched
17:35:11.927555 run   osmo-mgw_10.42.42.4(pid=27562): Launched
17:35:11.928845 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
17:35:12.290052 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27563): Launched
17:35:12.305636 run   osmo-bsc_10.42.42.7(pid=27564): Launched
17:35:12.646093 run   osmo-msc_10.42.42.6(pid=27558): ERR: Terminated: ERROR {rc=1}  [trial-682↪aoip_ussd:nanobts+band-1900↪osmo-msc_10.42.42.6↪osmo-msc_10.42.42.6(pid=27558)]
17:35:12.647529 run   osmo-msc_10.42.42.6(pid=27558): stderr:
 |   Global redzone:          f9
|   Global init order:       f6
|   Poisoned by user:        f7
|   Container overflow:      fc
|   Array cookie:            ac
|   Intra object redzone:    bb
|   ASan internal:           fe
|   Left alloca redzone:     ca
|   Right alloca redzone:    cb
| ==27558==ABORTING 
17:35:12.648614 run   osmo-msc_10.42.42.6(pid=27558): stdout:
 | (launched: 2018-04-16_17:35:11.616891) 
17:35:12.649589 run   osmo-msc_10.42.42.6(pid=27558): stderr:
 |   Global redzone:          f9
|   Global init order:       f6
|   Poisoned by user:        f7
|   Container overflow:      fc
|   Array cookie:            ac
|   Intra object redzone:    bb
|   ASan internal:           fe
|   Left alloca redzone:     ca
|   Right alloca redzone:    cb
| ==27558==ABORTING 
17:35:12.652085 tst           assert_extension.py:24: ERR: Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27558)  [trial-682↪aoip_ussd:nanobts+band-1900↪assert_extension.py:24]
17:35:12.658180 tst           assert_extension.py:24: Test FAILED (2.7 sec)
17:35:12.662517 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27553): Terminating (SIGINT)
17:35:12.832550 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27553): Terminated: ok {rc=0}
17:35:12.833535 run   osmo-hlr_10.42.42.2(pid=27554): Terminating (SIGINT)
17:35:13.003530 run   osmo-hlr_10.42.42.2(pid=27554): Terminated {rc=1}
17:35:13.004611 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27555): Terminating (SIGINT)
17:35:13.175368 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27555): Terminated: ok {rc=0}
17:35:13.176448 run   osmo-stp_10.42.42.5(pid=27556): Terminating (SIGINT)
17:35:13.350476 run   osmo-stp_10.42.42.5(pid=27556): Terminated: ok {rc=0}
17:35:13.351406 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27557): Terminating (SIGINT)
17:35:13.522020 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=27557): Terminated: ok {rc=0}
17:35:13.522978 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27559): Terminating (SIGINT)
17:35:13.693632 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27559): Terminated: ok {rc=0}
17:35:13.694592 run   osmo-mgw_10.42.42.3(pid=27560): Terminating (SIGINT)
17:35:13.865909 run   osmo-mgw_10.42.42.3(pid=27560): Terminated {rc=-2}
17:35:13.866922 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27561): Terminating (SIGINT)
17:35:14.037822 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27561): Terminated: ok {rc=0}
17:35:14.038873 run   osmo-mgw_10.42.42.4(pid=27562): Terminating (SIGINT)
17:35:14.047167 run   osmo-mgw_10.42.42.4(pid=27562): Terminated {rc=-2}
17:35:14.048142 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27563): Terminating (SIGINT)
17:35:14.218822 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=27563): Terminated: ok {rc=0}
17:35:14.219857 run   osmo-bsc_10.42.42.7(pid=27564): Terminating (SIGINT)
17:35:17.628094 run   osmo-bsc_10.42.42.7(pid=27564): Terminated: ok {rc=0}
17:35:17.637501 bus                        /sierra_2: Setting Powered False
---------------------------------------------------------------------
trial-682 aoip_ussd:nanobts+band-1900 FAIL
---------------------------------------------------------------------
17:35:18.951206 tst                        trial-682: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682/run.2018-04-16_17-34-48/trial-682.xml

--------------------------------------------------------------------------------------------
trial-682 FAIL
--------------------------------------------------------------------------------------------
17:35:18.983087 tst                        trial-682: trial-682: FAIL, 2 suites failed
FAIL: aoip_ussd:nanobts+band-900 (fail: 1)
    FAIL: assert_extension.py (13.0 sec) Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27527)
FAIL: aoip_ussd:nanobts+band-1900 (fail: 1)
    FAIL: assert_extension.py (2.7 sec) Error: Process ended prematurely: osmo-msc_10.42.42.6(pid=27558)
+ rm -rf trial-682/inst
+ cd trial-682
+ readlink last_run
+ journalctl -u ofono -o short-precise --since 2018-04-16 17:34:47
+ readlink last_run
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682-run.tgz run.2018-04-16_17-34-48
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-682-bin.tgz checksums.md5 osmo-bsc.build-60.tgz osmo-bts-sysmo.build-60.tgz osmo-bts.build-60.tgz osmo-ggsn.build-60.tgz osmo-hlr.build-60.tgz osmo-mgw.build-60.tgz osmo-msc.build-60.tgz osmo-nitb.build-60.tgz osmo-pcu-sysmo.build-60.tgz osmo-pcu.build-60.tgz osmo-sgsn.build-60.tgz osmo-stp.build-60.tgz osmo-trx.build-60.tgz
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Finished: FAILURE