jenkins LTS upgrade
Failed
handover_2G:sysmo+bts1-trx-b200.handover.py (from handover_2G_sysmo+bts1-trx-b200)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fd2c451b400>, timeout=10, timestep=1} [trial-822↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45↪handover.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-822 handover_2G:sysmo+bts1-trx-b200 handover.py ---------------------------------------------- 08:22:27.059734 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 08:22:27.072498 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x bts (candidates: 2) 08:22:27.198990 tst handover.py: using LAC 8826 08:22:27.327999 tst handover.py: using RAC 156 08:22:27.452848 tst handover.py: using CellId 8826 08:22:27.578723 tst handover.py: using BVCI 8822 08:22:27.602849 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x bts (candidates: 2) 08:22:27.741367 tst handover.py: using LAC 8827 08:22:27.865904 tst handover.py: using RAC 157 08:22:27.991991 tst handover.py: using CellId 8827 08:22:28.120276 tst handover.py: using BVCI 8823 08:22:28.144441 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 08:22:28.155561 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 08:22:28.166354 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 08:22:28.177288 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 08:22:28.188246 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x ip_address (candidates: 6) 08:22:28.199285 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x modem (candidates: 2) 08:22:28.306297 tst handover_2G:sysmo+bts1-trx-b200: Using 1 x modem (candidates: 2) 08:22:28.507556 run osmo-hlr_10.42.42.2: Starting osmo-hlr 08:22:29.021980 run create_hlr_db(pid=13085): Launched 08:22:29.160859 bus /sierra_1: Setting Powered False 08:22:30.230074 run create_hlr_db(pid=13085): Terminated: ok {rc=0} 08:22:30.657667 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13087): Launched 08:22:31.000121 run osmo-hlr_10.42.42.2(pid=13088): Launched 08:22:31.123404 run osmo-stp_10.42.42.5: Starting osmo-stp 08:22:31.689525 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13089): Launched 08:22:32.034211 run osmo-stp_10.42.42.5(pid=13090): Launched 08:22:32.155788 run osmo-msc_10.42.42.6: Starting osmo-msc 08:22:32.750619 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13091): Launched 08:22:33.095802 run osmo-msc_10.42.42.6(pid=13092): Launched 08:22:36.278427 run osmo-mgw_10.42.42.3: Starting osmo-mgw 08:22:36.842620 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13093): Launched 08:22:37.184944 run osmo-mgw_10.42.42.3(pid=13094): Launched 08:22:37.306774 run osmo-mgw_10.42.42.4: Starting osmo-mgw 08:22:37.859697 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13095): Launched 08:22:38.202716 run osmo-mgw_10.42.42.4(pid=13096): Launched 08:22:38.325046 run osmo-bsc_10.42.42.7: Starting osmo-bsc 08:22:39.063279 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13097): Launched 08:22:39.405088 run osmo-bsc_10.42.42.7(pid=13098): Launched 08:22:46.132538 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 08:22:46.677717 run rm-remote-dir(pid=13099): Launched 08:22:48.875466 run rm-remote-dir(pid=13099): Terminated: ok {rc=0} 08:22:49.219299 run mk-remote-dir(pid=13100): Launched 08:22:51.416587 run mk-remote-dir(pid=13100): Terminated: ok {rc=0} 08:22:51.707497 run scp-inst-to-remote(pid=13101): Launched 08:23:04.933975 run scp-inst-to-remote(pid=13101): Terminated: ok {rc=0} 08:23:05.294411 run mk-remote-dir(pid=13103): Launched 08:23:07.500425 run mk-remote-dir(pid=13103): Terminated: ok {rc=0} 08:23:07.809897 run scp-cfg-to-remote(pid=13104): Launched 08:23:10.017568 run scp-cfg-to-remote(pid=13104): Terminated: ok {rc=0} 08:23:10.371720 run reload-dsp-firmware(pid=13106): Launched 08:23:13.566898 run reload-dsp-firmware(pid=13106): Terminated: ok {rc=0} 08:23:13.893341 run osmo-bts-sysmo(pid=13107): Launched 08:23:19.212423 tst handover.py:42: ----- Report fragment: handover_2G:sysmo+bts1-trx-b200/handover.py/01_bts0_started: pass (33.0s) 08:23:19.360171 tst handover.py: using MSISDN 5718 08:23:19.539801 bus /sierra_1: Setting Powered True 08:23:23.037831 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5718', subscriber_id=1} 08:23:23.186669 tst handover.py: using MSISDN 5719 08:23:23.373054 bus /sierra_2: Setting Powered True 08:23:35.753619 tst handover.py:45: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fd2c451b400>, timeout=10, timestep=1} [trial-822↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45↪handover.py↪/sierra_2] [trial-822↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45] 08:23:35.760325 tst handover.py:45: Test FAILED (68.7 sec) 08:23:35.794905 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13087): Terminating (SIGTERM) 08:23:35.834314 run osmo-hlr_10.42.42.2(pid=13088): Terminating (SIGTERM) 08:23:35.874508 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13089): Terminating (SIGTERM) 08:23:35.918438 run osmo-stp_10.42.42.5(pid=13090): Terminating (SIGTERM) 08:23:35.969568 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13091): Terminating (SIGTERM) 08:23:36.012270 run osmo-msc_10.42.42.6(pid=13092): Terminating (SIGTERM) 08:23:36.053495 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13093): Terminating (SIGTERM) 08:23:36.092913 run osmo-mgw_10.42.42.3(pid=13094): Terminating (SIGTERM) 08:23:36.132967 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13095): Terminating (SIGTERM) 08:23:36.172402 run osmo-mgw_10.42.42.4(pid=13096): Terminating (SIGTERM) 08:23:36.213429 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13097): Terminating (SIGTERM) 08:23:36.257935 run osmo-bsc_10.42.42.7(pid=13098): Terminating (SIGTERM) 08:23:36.299690 run osmo-bts-sysmo(pid=13107): Terminating (SIGTERM) 08:23:36.311700 --- ParallelTerminationStrategy: PID 13087 died... 08:23:36.371101 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13087): Terminated: ok {rc=0} 08:23:36.381303 --- ParallelTerminationStrategy: PID 13088 died... 08:23:36.435213 run osmo-hlr_10.42.42.2(pid=13088): Terminated: ok {rc=0} 08:23:36.444684 --- ParallelTerminationStrategy: PID 13089 died... 08:23:36.497704 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13089): Terminated: ok {rc=0} 08:23:36.507798 --- ParallelTerminationStrategy: PID 13090 died... 08:23:36.562026 run osmo-stp_10.42.42.5(pid=13090): Terminated: ok {rc=0} 08:23:36.572402 --- ParallelTerminationStrategy: PID 13091 died... 08:23:36.626805 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13091): Terminated: ok {rc=0} 08:23:36.637532 --- ParallelTerminationStrategy: PID 13092 died... 08:23:36.691326 run osmo-msc_10.42.42.6(pid=13092): Terminated: ok {rc=0} 08:23:36.701373 --- ParallelTerminationStrategy: PID 13093 died... 08:23:36.754588 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13093): Terminated: ok {rc=0} 08:23:36.764170 --- ParallelTerminationStrategy: PID 13094 died... 08:23:36.817083 run osmo-mgw_10.42.42.3(pid=13094): Terminated {rc=15} 08:23:36.826758 --- ParallelTerminationStrategy: PID 13095 died... 08:23:36.879668 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13095): Terminated: ok {rc=0} 08:23:36.889579 --- ParallelTerminationStrategy: PID 13096 died... 08:23:36.942225 run osmo-mgw_10.42.42.4(pid=13096): Terminated {rc=15} 08:23:36.952017 --- ParallelTerminationStrategy: PID 13097 died... 08:23:37.005512 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13097): Terminated: ok {rc=0} 08:23:37.015163 --- ParallelTerminationStrategy: PID 13098 died... 08:23:37.067795 run osmo-bsc_10.42.42.7(pid=13098): Terminated: ok {rc=0} 08:23:37.077712 --- ParallelTerminationStrategy: PID 13107 died... 08:23:37.130832 run osmo-bts-sysmo(pid=13107): Terminated: ok {rc=0} 08:23:37.239714 bus /sierra_2: Setting Online False 08:23:38.322572 bus /sierra_2: Setting Powered False 08:23:43.963348 bus /sierra_2: Setting Powered False 08:23:45.070112 bus /sierra_2: Setting Powered False 08:23:46.197965 bus /sierra_1: Setting Online False 08:23:47.285510 bus /sierra_1: Setting Powered False 08:23:52.968392 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/handover_2G/handover.py", line 45, in <module> hlr.subscriber_add(ms1) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fd2c451b400>, timeout=10, timestep=1} [trial-822↪handover_2G:sysmo+bts1-trx-b200↪handover.py:45↪handover.py↪/sierra_2]