FailedConsole Output

Started by upstream project "osmo-gsm-tester_manual-build-all" build number 63
originally caused by:
 Started by user Pau Espin Pedrol
Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_manual-run-all
Cloning the remote Git repository
Cloning repository git://git.osmocom.org/osmo-gsm-tester
 > git init /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester # timeout=10
Fetching upstream changes from git://git.osmocom.org/osmo-gsm-tester
 > git --version # timeout=10
 > git fetch --tags --progress git://git.osmocom.org/osmo-gsm-tester +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url git://git.osmocom.org/osmo-gsm-tester # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://git.osmocom.org/osmo-gsm-tester # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
No valid HEAD. Skipping the resetting
 > git clean -fdx # timeout=10
Fetching upstream changes from git://git.osmocom.org/osmo-gsm-tester
 > git fetch --tags --progress git://git.osmocom.org/osmo-gsm-tester +refs/heads/*:refs/remotes/origin/*
 > git rev-parse pespin/mainloop^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/pespin/mainloop^{commit} # timeout=10
Checking out Revision 796bf3954170ad9515af9cb37b1abdf1fce1774b (refs/remotes/origin/pespin/mainloop)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 796bf3954170ad9515af9cb37b1abdf1fce1774b
Commit message: "nanobts: Attemt at fixing race condition using ipaccess-config after power up"
 > git rev-list --no-walk 796bf3954170ad9515af9cb37b1abdf1fce1774b # timeout=10
Copied 26 artifacts from "osmo-gsm-tester_manual-build-all" build number 63
[osmo-gsm-tester_manual-run-all] $ /bin/sh /tmp/jenkins900714297724889761.sh
+ export OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/example
+ unlink osmo-gsm-tester/example/resources.conf
unlink: cannot unlink 'osmo-gsm-tester/example/resources.conf': No such file or directory
+ true
+ ln -s resources.conf.prod osmo-gsm-tester/example/resources.conf
+ PATH=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games ./osmo-gsm-tester/contrib/jenkins-run.sh
+ base=/home/jenkins/workspace/osmo-gsm-tester_manual-run-all
+ date +%F %T
+ time_start=2018-06-27 13:06:47
+ trial_dir_prefix=trial-
+ rm -rf trial-*
+ trial_dir=trial-683
+ mkdir -p trial-683
+ mv osmo-bsc.build-63.tgz osmo-bts-sysmo.build-63.tgz osmo-bts.build-63.tgz osmo-ggsn.build-63.tgz osmo-hlr.build-63.tgz osmo-mgw.build-63.tgz osmo-msc.build-63.tgz osmo-nitb.build-63.tgz osmo-pcu-sysmo.build-63.tgz osmo-pcu.build-63.tgz osmo-sgsn.build-63.tgz osmo-stp.build-63.tgz osmo-trx.build-63.tgz trial-683
+ cat osmo-bsc.build-63.md5 osmo-bts-sysmo.build-63.md5 osmo-bts.build-63.md5 osmo-ggsn.build-63.md5 osmo-hlr.build-63.md5 osmo-mgw.build-63.md5 osmo-msc.build-63.md5 osmo-nitb.build-63.md5 osmo-pcu-sysmo.build-63.md5 osmo-pcu.build-63.md5 osmo-sgsn.build-63.md5 osmo-stp.build-63.md5 osmo-trx.build-63.md5
+ rm osmo-bsc.build-63.md5 osmo-bts-sysmo.build-63.md5 osmo-bts.build-63.md5 osmo-ggsn.build-63.md5 osmo-hlr.build-63.md5 osmo-mgw.build-63.md5 osmo-msc.build-63.md5 osmo-nitb.build-63.md5 osmo-pcu-sysmo.build-63.md5 osmo-pcu.build-63.md5 osmo-sgsn.build-63.md5 osmo-stp.build-63.md5 osmo-trx.build-63.md5
+ exit_code=1
+ which osmo-gsm-tester.py
+ python3 -u /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/osmo-gsm-tester/src/osmo-gsm-tester.py trial-683 -s aoip_ussd:nanobts+band-900 -s aoip_ussd:nanobts+band-1900
combinations: ['aoip_ussd:nanobts+band-900', 'aoip_ussd:nanobts+band-1900']
series: None
trial: 'trial-683'
tests: None

--------------------------------------------------------------------------------------------
trial-683
--------------------------------------------------------------------------------------------
13:06:48.482542 tst                        trial-683: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683/run.2018-06-27_13-06-48/log

---------------------------------------------------------------------
trial-683 aoip_ussd:nanobts+band-900
---------------------------------------------------------------------
13:06:49.757927 tst       aoip_ussd:nanobts+band-900: reserving resources in /var/tmp/osmo-gsm-tester/state ...
13:06:49.777724 tst       aoip_ussd:nanobts+band-900: Reserving 1 x bts (candidates: 6)
13:06:49.802271 tst       aoip_ussd:nanobts+band-900: Reserving 6 x ip_address (candidates: 8)
13:06:49.836186 tst       aoip_ussd:nanobts+band-900: Reserving 1 x modem (candidates: 4)

----------------------------------------------
trial-683 aoip_ussd:nanobts+band-900 assert_extension.py
----------------------------------------------
13:06:50.049603 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
13:06:50.062078 tst       aoip_ussd:nanobts+band-900: Using 1 x bts (candidates: 1)
13:06:50.101417 tst       aoip_ussd:nanobts+band-900: using LAC 4997
13:06:50.109870 tst       aoip_ussd:nanobts+band-900: using RAC 152
13:06:50.119662 tst       aoip_ussd:nanobts+band-900: using CellId 4997
13:06:50.129417 tst       aoip_ussd:nanobts+band-900: using BVCI 4998
13:06:50.130879 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
13:06:50.145488 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
13:06:50.157865 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
13:06:50.170143 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
13:06:50.182196 tst       aoip_ussd:nanobts+band-900: Using 1 x ip_address (candidates: 6)
13:06:50.194710 tst       aoip_ussd:nanobts+band-900: Using 1 x modem (candidates: 1)
13:06:50.264080 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
13:06:51.284225 run         create_hlr_db(pid=13660): Launched
13:06:52.289246 run         create_hlr_db(pid=13660): Terminated: ok {rc=0}
13:06:52.305507 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13662): Launched
13:06:52.321222 run   osmo-hlr_10.42.42.2(pid=13663): Launched
13:06:52.322691 run              osmo-stp_10.42.42.5: Starting osmo-stp
13:06:54.543567 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13664): Launched
13:06:54.557180 run   osmo-stp_10.42.42.5(pid=13665): Launched
13:06:54.558837 run              osmo-msc_10.42.42.6: Starting osmo-msc
13:06:57.813605 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13671): Launched
13:06:57.832526 run   osmo-msc_10.42.42.6(pid=13672): Launched
13:06:57.833719 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
13:06:59.493705 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13673): Launched
13:06:59.513043 run   osmo-mgw_10.42.42.3(pid=13674): Launched
13:06:59.514169 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
13:06:59.730369 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13676): Launched
13:06:59.752048 run   osmo-mgw_10.42.42.4(pid=13677): Launched
13:06:59.753900 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
13:07:03.244622 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13678): Launched
13:07:03.260996 run   osmo-bsc_10.42.42.7(pid=13679): Launched
13:07:04.794430 run pcap-recorder_any(filters='host 10.42.42.54 and port not 22')(pid=13694): Launched
13:07:04.796522 run              nanobts_10.42.42.54: Finding nanobts 10.42.42.54, binding on 10.42.42.1...
13:07:04.812135 run   abisip-find-preconf(pid=13695): Launched
13:12:05.206291 tst           assert_extension.py:24: ERR: Error: Wait timeout  [trial-683↪aoip_ussd:nanobts+band-900↪assert_extension.py:24]
13:12:05.212505 tst           assert_extension.py:24: Test FAILED (315.2 sec)
13:12:05.216188 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13662): Terminating (SIGINT)
13:12:05.386335 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13662): Terminated: ok {rc=0}
13:12:05.387367 run   osmo-hlr_10.42.42.2(pid=13663): Terminating (SIGINT)
13:12:05.561466 run   osmo-hlr_10.42.42.2(pid=13663): Terminated {rc=1}
13:12:05.562473 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13664): Terminating (SIGINT)
13:12:05.733512 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13664): Terminated: ok {rc=0}
13:12:05.734462 run   osmo-stp_10.42.42.5(pid=13665): Terminating (SIGINT)
13:12:05.904499 run   osmo-stp_10.42.42.5(pid=13665): Terminated: ok {rc=0}
13:12:05.905417 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13671): Terminating (SIGINT)
13:12:06.076201 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13671): Terminated: ok {rc=0}
13:12:06.077179 run   osmo-msc_10.42.42.6(pid=13672): Terminating (SIGINT)
13:12:09.484463 run   osmo-msc_10.42.42.6(pid=13672): Terminated {rc=1}
13:12:09.485487 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13673): Terminating (SIGINT)
13:12:09.657261 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13673): Terminated: ok {rc=0}
13:12:09.659135 run   osmo-mgw_10.42.42.3(pid=13674): Terminating (SIGINT)
13:12:09.668134 run   osmo-mgw_10.42.42.3(pid=13674): Terminated {rc=-2}
13:12:09.669817 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13676): Terminating (SIGINT)
13:12:09.840810 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13676): Terminated: ok {rc=0}
13:12:09.842603 run   osmo-mgw_10.42.42.4(pid=13677): Terminating (SIGINT)
13:12:09.852485 run   osmo-mgw_10.42.42.4(pid=13677): Terminated {rc=-2}
13:12:09.854192 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13678): Terminating (SIGINT)
13:12:10.025372 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13678): Terminated: ok {rc=0}
13:12:10.026410 run   osmo-bsc_10.42.42.7(pid=13679): Terminating (SIGINT)
13:12:13.434118 run   osmo-bsc_10.42.42.7(pid=13679): Terminated {rc=1}
13:12:13.435123 run pcap-recorder_any(filters='host 10.42.42.54 and port not 22')(pid=13694): Terminating (SIGINT)
13:12:13.605417 run pcap-recorder_any(filters='host 10.42.42.54 and port not 22')(pid=13694): Terminated: ok {rc=0}
13:12:13.606444 run   abisip-find-preconf(pid=13695): Terminating (SIGINT)
13:12:13.777513 run   abisip-find-preconf(pid=13695): Terminated {rc=-2}
13:12:13.787153 bus                        /sierra_1: Setting Powered False
---------------------------------------------------------------------
trial-683 aoip_ussd:nanobts+band-900 FAIL
---------------------------------------------------------------------

---------------------------------------------------------------------
trial-683 aoip_ussd:nanobts+band-1900
---------------------------------------------------------------------
13:12:15.085965 tst      aoip_ussd:nanobts+band-1900: reserving resources in /var/tmp/osmo-gsm-tester/state ...
13:12:15.100329 tst      aoip_ussd:nanobts+band-1900: Reserving 1 x bts (candidates: 6)
13:12:15.125351 tst      aoip_ussd:nanobts+band-1900: Reserving 6 x ip_address (candidates: 8)
13:12:15.160352 tst      aoip_ussd:nanobts+band-1900: Reserving 1 x modem (candidates: 4)

----------------------------------------------
trial-683 aoip_ussd:nanobts+band-1900 assert_extension.py
----------------------------------------------
13:12:15.298949 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
13:12:15.311687 tst      aoip_ussd:nanobts+band-1900: Using 1 x bts (candidates: 1)
13:12:15.348237 tst      aoip_ussd:nanobts+band-1900: using LAC 4998
13:12:15.356127 tst      aoip_ussd:nanobts+band-1900: using RAC 153
13:12:15.364369 tst      aoip_ussd:nanobts+band-1900: using CellId 4998
13:12:15.373680 tst      aoip_ussd:nanobts+band-1900: using BVCI 4999
13:12:15.375418 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
13:12:15.388546 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
13:12:15.401323 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
13:12:15.414654 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
13:12:15.427098 tst      aoip_ussd:nanobts+band-1900: Using 1 x ip_address (candidates: 6)
13:12:15.439735 tst      aoip_ussd:nanobts+band-1900: Using 1 x modem (candidates: 1)
13:12:15.505665 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
13:12:15.633118 run         create_hlr_db(pid=13792): Launched
13:12:15.635945 bus                        /sierra_1: Setting Powered False
13:12:16.638765 run         create_hlr_db(pid=13792): Terminated: ok {rc=0}
13:12:16.663545 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13794): Launched
13:12:16.678977 run   osmo-hlr_10.42.42.2(pid=13795): Launched
13:12:16.680904 run              osmo-stp_10.42.42.5: Starting osmo-stp
13:12:16.818940 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13796): Launched
13:12:16.833252 run   osmo-stp_10.42.42.5(pid=13797): Launched
13:12:16.835036 run              osmo-msc_10.42.42.6: Starting osmo-msc
13:12:16.974715 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13798): Launched
13:12:16.990070 run   osmo-msc_10.42.42.6(pid=13799): Launched
13:12:16.992375 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
13:12:17.130337 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13800): Launched
13:12:17.148294 run   osmo-mgw_10.42.42.3(pid=13801): Launched
13:12:17.153191 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
13:12:17.317197 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13802): Launched
13:12:17.332854 run   osmo-mgw_10.42.42.4(pid=13803): Launched
13:12:17.334615 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
13:12:17.686766 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13804): Launched
13:12:17.701704 run   osmo-bsc_10.42.42.7(pid=13805): Launched
13:12:19.174304 run pcap-recorder_any(filters='host 10.42.42.53 and port not 22')(pid=13819): Launched
13:12:19.176323 run              nanobts_10.42.42.53: Finding nanobts 10.42.42.53, binding on 10.42.42.1...
13:12:19.192274 run   abisip-find-preconf(pid=13820): Launched
13:17:19.588271 tst           assert_extension.py:24: ERR: Error: Wait timeout  [trial-683↪aoip_ussd:nanobts+band-1900↪assert_extension.py:24]
13:17:19.594024 tst           assert_extension.py:24: Test FAILED (304.3 sec)
13:17:19.597808 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13794): Terminating (SIGINT)
13:17:19.768710 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13794): Terminated: ok {rc=0}
13:17:19.770475 run   osmo-hlr_10.42.42.2(pid=13795): Terminating (SIGINT)
13:17:19.946483 run   osmo-hlr_10.42.42.2(pid=13795): Terminated {rc=1}
13:17:19.947423 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13796): Terminating (SIGINT)
13:17:20.119285 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13796): Terminated: ok {rc=0}
13:17:20.120421 run   osmo-stp_10.42.42.5(pid=13797): Terminating (SIGINT)
13:17:20.293599 run   osmo-stp_10.42.42.5(pid=13797): Terminated: ok {rc=0}
13:17:20.294672 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13798): Terminating (SIGINT)
13:17:20.465531 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13798): Terminated: ok {rc=0}
13:17:20.466593 run   osmo-msc_10.42.42.6(pid=13799): Terminating (SIGINT)
13:17:23.874269 run   osmo-msc_10.42.42.6(pid=13799): Terminated {rc=1}
13:17:23.875297 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13800): Terminating (SIGINT)
13:17:24.045811 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13800): Terminated: ok {rc=0}
13:17:24.046858 run   osmo-mgw_10.42.42.3(pid=13801): Terminating (SIGINT)
13:17:24.217049 run   osmo-mgw_10.42.42.3(pid=13801): Terminated {rc=-2}
13:17:24.218124 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13802): Terminating (SIGINT)
13:17:24.388420 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13802): Terminated: ok {rc=0}
13:17:24.389574 run   osmo-mgw_10.42.42.4(pid=13803): Terminating (SIGINT)
13:17:24.559703 run   osmo-mgw_10.42.42.4(pid=13803): Terminated {rc=-2}
13:17:24.560872 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13804): Terminating (SIGINT)
13:17:24.731615 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13804): Terminated: ok {rc=0}
13:17:24.733369 run   osmo-bsc_10.42.42.7(pid=13805): Terminating (SIGINT)
13:17:28.141263 run   osmo-bsc_10.42.42.7(pid=13805): Terminated {rc=1}
13:17:28.142287 run pcap-recorder_any(filters='host 10.42.42.53 and port not 22')(pid=13819): Terminating (SIGINT)
13:17:28.312589 run pcap-recorder_any(filters='host 10.42.42.53 and port not 22')(pid=13819): Terminated: ok {rc=0}
13:17:28.313641 run   abisip-find-preconf(pid=13820): Terminating (SIGINT)
13:17:28.483838 run   abisip-find-preconf(pid=13820): Terminated {rc=-2}
13:17:28.493689 bus                        /sierra_1: Setting Powered False
---------------------------------------------------------------------
trial-683 aoip_ussd:nanobts+band-1900 FAIL
---------------------------------------------------------------------
13:17:29.794991 tst                        trial-683: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683/run.2018-06-27_13-06-48/trial-683.xml

--------------------------------------------------------------------------------------------
trial-683 FAIL
--------------------------------------------------------------------------------------------
13:17:29.826511 tst                        trial-683: trial-683: FAIL, 2 suites failed
FAIL: aoip_ussd:nanobts+band-900 (fail: 1)
    FAIL: assert_extension.py (315.2 sec) Error: Wait timeout
FAIL: aoip_ussd:nanobts+band-1900 (fail: 1)
    FAIL: assert_extension.py (304.3 sec) Error: Wait timeout
+ rm -rf trial-683/inst
+ cd trial-683
+ readlink last_run
+ journalctl -u ofono -o short-precise --since 2018-06-27 13:06:47
+ readlink last_run
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683-run.tgz run.2018-06-27_13-06-48
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_manual-run-all/trial-683-bin.tgz checksums.md5 osmo-bsc.build-63.tgz osmo-bts-sysmo.build-63.tgz osmo-bts.build-63.tgz osmo-ggsn.build-63.tgz osmo-hlr.build-63.tgz osmo-mgw.build-63.tgz osmo-msc.build-63.tgz osmo-nitb.build-63.tgz osmo-pcu-sysmo.build-63.tgz osmo-pcu.build-63.tgz osmo-sgsn.build-63.tgz osmo-stp.build-63.tgz osmo-trx.build-63.tgz
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Finished: FAILURE