jenkins LTS upgrade
Failed
voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1.mo_mt_call.py (from voice_sysmo+mod-bts0-ts-tchh+cfg-codec-hr1)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb34020cd08>, timeout=10, timestep=1} [trial-820↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-820 voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1 mo_mt_call.py ---------------------------------------------- 05:41:19.970283 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x ip_address (candidates: 6) 05:41:19.981377 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x bts (candidates: 1) 05:41:20.129614 tst mo_mt_call.py: using LAC 8598 05:41:20.251632 tst mo_mt_call.py: using RAC 183 05:41:20.376757 tst mo_mt_call.py: using CellId 8598 05:41:20.501395 tst mo_mt_call.py: using BVCI 8594 05:41:20.525089 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x ip_address (candidates: 6) 05:41:20.536249 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x ip_address (candidates: 6) 05:41:20.547147 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x ip_address (candidates: 6) 05:41:20.558296 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x ip_address (candidates: 6) 05:41:20.569464 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x ip_address (candidates: 6) 05:41:20.580631 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x modem (candidates: 2) 05:41:20.685000 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1: Using 1 x modem (candidates: 2) 05:41:20.876011 run osmo-hlr_10.42.42.2: Starting osmo-hlr 05:41:21.373599 run create_hlr_db(pid=7479): Launched 05:41:21.555941 bus /sierra_1: Setting Powered False 05:41:22.581083 run create_hlr_db(pid=7479): Terminated: ok {rc=0} 05:41:23.029436 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7481): Launched 05:41:23.367555 run osmo-hlr_10.42.42.2(pid=7482): Launched 05:41:23.490014 run osmo-stp_10.42.42.5: Starting osmo-stp 05:41:24.069769 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7483): Launched 05:41:24.406517 run osmo-stp_10.42.42.5(pid=7484): Launched 05:41:24.525707 run osmo-msc_10.42.42.6: Starting osmo-msc 05:41:25.133132 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7485): Launched 05:41:25.470339 run osmo-msc_10.42.42.6(pid=7486): Launched 05:41:28.648822 run osmo-mgw_10.42.42.3: Starting osmo-mgw 05:41:29.228231 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7487): Launched 05:41:29.565112 run osmo-mgw_10.42.42.3(pid=7488): Launched 05:41:29.684532 run osmo-mgw_10.42.42.4: Starting osmo-mgw 05:41:30.255153 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7489): Launched 05:41:30.590178 run osmo-mgw_10.42.42.4(pid=7490): Launched 05:41:30.710020 run osmo-bsc_10.42.42.7: Starting osmo-bsc 05:41:31.405916 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7491): Launched 05:41:31.741536 run osmo-bsc_10.42.42.7(pid=7492): Launched 05:41:38.025244 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 05:41:38.620537 run rm-remote-dir(pid=7493): Launched 05:41:40.826257 run rm-remote-dir(pid=7493): Terminated: ok {rc=0} 05:41:41.184023 run mk-remote-dir(pid=7494): Launched 05:41:43.388771 run mk-remote-dir(pid=7494): Terminated: ok {rc=0} 05:41:43.692259 run scp-inst-to-remote(pid=7495): Launched 05:41:56.904632 run scp-inst-to-remote(pid=7495): Terminated: ok {rc=0} 05:41:57.235452 run mk-remote-dir(pid=7497): Launched 05:41:59.435264 run mk-remote-dir(pid=7497): Terminated: ok {rc=0} 05:41:59.736927 run scp-cfg-to-remote(pid=7498): Launched 05:42:01.939628 run scp-cfg-to-remote(pid=7498): Terminated: ok {rc=0} 05:42:02.287026 run reload-dsp-firmware(pid=7506): Launched 05:42:05.486990 run reload-dsp-firmware(pid=7506): Terminated: ok {rc=0} 05:42:05.828753 run osmo-bts-sysmo(pid=7507): Launched 05:42:10.085834 tst mo_mt_call.py: using MSISDN 5576 05:42:10.261926 bus /sierra_1: Setting Powered True 05:42:13.762367 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5576', subscriber_id=1} 05:42:13.899293 tst mo_mt_call.py: using MSISDN 5577 05:42:14.069865 bus /sierra_2: Setting Powered True 05:42:26.439446 tst mo_mt_call.py:8: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb34020cd08>, timeout=10, timestep=1} [trial-820↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2] [trial-820↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1↪mo_mt_call.py:8] 05:42:26.447442 tst mo_mt_call.py:8: Test FAILED (66.5 sec) 05:42:26.483441 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7481): Terminating (SIGTERM) 05:42:26.524938 run osmo-hlr_10.42.42.2(pid=7482): Terminating (SIGTERM) 05:42:26.567867 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7483): Terminating (SIGTERM) 05:42:26.611140 run osmo-stp_10.42.42.5(pid=7484): Terminating (SIGTERM) 05:42:26.668092 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7485): Terminating (SIGTERM) 05:42:26.725062 run osmo-msc_10.42.42.6(pid=7486): Terminating (SIGTERM) 05:42:26.773061 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7487): Terminating (SIGTERM) 05:42:26.817309 run osmo-mgw_10.42.42.3(pid=7488): Terminating (SIGTERM) 05:42:26.862945 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7489): Terminating (SIGTERM) 05:42:26.906092 run osmo-mgw_10.42.42.4(pid=7490): Terminating (SIGTERM) 05:42:26.961000 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7491): Terminating (SIGTERM) 05:42:27.004763 run osmo-bsc_10.42.42.7(pid=7492): Terminating (SIGTERM) 05:42:27.046827 run osmo-bts-sysmo(pid=7507): Terminating (SIGTERM) 05:42:27.056568 --- ParallelTerminationStrategy: PID 7481 died... 05:42:27.113270 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7481): Terminated: ok {rc=0} 05:42:27.123699 --- ParallelTerminationStrategy: PID 7482 died... 05:42:27.180055 run osmo-hlr_10.42.42.2(pid=7482): Terminated: ok {rc=0} 05:42:27.190366 --- ParallelTerminationStrategy: PID 7483 died... 05:42:27.246429 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7483): Terminated: ok {rc=0} 05:42:27.256680 --- ParallelTerminationStrategy: PID 7484 died... 05:42:27.312883 run osmo-stp_10.42.42.5(pid=7484): Terminated: ok {rc=0} 05:42:27.322997 --- ParallelTerminationStrategy: PID 7485 died... 05:42:27.378754 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7485): Terminated: ok {rc=0} 05:42:27.388898 --- ParallelTerminationStrategy: PID 7486 died... 05:42:27.444672 run osmo-msc_10.42.42.6(pid=7486): Terminated: ok {rc=0} 05:42:27.454950 --- ParallelTerminationStrategy: PID 7487 died... 05:42:27.510972 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7487): Terminated: ok {rc=0} 05:42:27.521335 --- ParallelTerminationStrategy: PID 7488 died... 05:42:27.577306 run osmo-mgw_10.42.42.3(pid=7488): Terminated {rc=15} 05:42:27.587599 --- ParallelTerminationStrategy: PID 7489 died... 05:42:27.643989 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7489): Terminated: ok {rc=0} 05:42:27.654349 --- ParallelTerminationStrategy: PID 7490 died... 05:42:27.710446 run osmo-mgw_10.42.42.4(pid=7490): Terminated {rc=15} 05:42:27.720741 --- ParallelTerminationStrategy: PID 7491 died... 05:42:27.777262 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7491): Terminated: ok {rc=0} 05:42:27.788250 --- ParallelTerminationStrategy: PID 7507 died... 05:42:27.845816 run osmo-bts-sysmo(pid=7507): Terminated: ok {rc=0} 05:42:28.024528 --- ParallelTerminationStrategy: PID 7492 died... 05:42:28.082245 run osmo-bsc_10.42.42.7(pid=7492): Terminated: ok {rc=0} 05:42:28.188287 bus /sierra_2: Setting Online False 05:42:29.274338 bus /sierra_2: Setting Powered False 05:42:34.966781 bus /sierra_2: Setting Powered False 05:42:36.073430 bus /sierra_2: Setting Powered False 05:42:37.202297 bus /sierra_1: Setting Online False 05:42:38.285818 bus /sierra_1: Setting Powered False 05:42:43.964820 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/mo_mt_call.py", line 8, in <module> test_mo_mt_call(False, False) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/lib/testlib.py", line 39, in test_mo_mt_call hlr.subscriber_add(ms_mt) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb34020cd08>, timeout=10, timestep=1} [trial-820↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr1↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]