jenkins LTS upgrade
Failed
voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1.mo_mt_call.py (from voice_sysmo+mod-bts0-ts-tchf+cfg-codec-fr1)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb34029ff28>, timeout=10, timestep=1} [trial-820↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-820 voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1 mo_mt_call.py ---------------------------------------------- 05:32:46.927231 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x ip_address (candidates: 6) 05:32:46.938303 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x bts (candidates: 1) 05:32:47.086973 tst mo_mt_call.py: using LAC 8592 05:32:47.210663 tst mo_mt_call.py: using RAC 177 05:32:47.334755 tst mo_mt_call.py: using CellId 8592 05:32:47.459206 tst mo_mt_call.py: using BVCI 8588 05:32:47.482982 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x ip_address (candidates: 6) 05:32:47.494077 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x ip_address (candidates: 6) 05:32:47.504887 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x ip_address (candidates: 6) 05:32:47.516035 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x ip_address (candidates: 6) 05:32:47.527186 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x ip_address (candidates: 6) 05:32:47.538279 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x modem (candidates: 2) 05:32:47.641935 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1: Using 1 x modem (candidates: 2) 05:32:47.833020 run osmo-hlr_10.42.42.2: Starting osmo-hlr 05:32:48.324418 run create_hlr_db(pid=7292): Launched 05:32:48.503366 bus /sierra_1: Setting Powered False 05:32:49.524705 run create_hlr_db(pid=7292): Terminated: ok {rc=0} 05:32:49.965417 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7294): Launched 05:32:50.300129 run osmo-hlr_10.42.42.2(pid=7295): Launched 05:32:50.420728 run osmo-stp_10.42.42.5: Starting osmo-stp 05:32:50.996761 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7296): Launched 05:32:51.332417 run osmo-stp_10.42.42.5(pid=7297): Launched 05:32:51.452271 run osmo-msc_10.42.42.6: Starting osmo-msc 05:32:52.060220 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7298): Launched 05:32:52.395732 run osmo-msc_10.42.42.6(pid=7299): Launched 05:32:55.573286 run osmo-mgw_10.42.42.3: Starting osmo-mgw 05:32:56.152034 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7301): Launched 05:32:56.487677 run osmo-mgw_10.42.42.3(pid=7302): Launched 05:32:56.606841 run osmo-mgw_10.42.42.4: Starting osmo-mgw 05:32:57.173338 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7303): Launched 05:32:57.508995 run osmo-mgw_10.42.42.4(pid=7304): Launched 05:32:57.628591 run osmo-bsc_10.42.42.7: Starting osmo-bsc 05:32:58.322199 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7305): Launched 05:32:58.674700 run osmo-bsc_10.42.42.7(pid=7306): Launched 05:33:04.943254 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 05:33:05.504679 run rm-remote-dir(pid=7307): Launched 05:33:07.698575 run rm-remote-dir(pid=7307): Terminated: ok {rc=0} 05:33:08.034058 run mk-remote-dir(pid=7308): Launched 05:33:10.226530 run mk-remote-dir(pid=7308): Terminated: ok {rc=0} 05:33:10.511446 run scp-inst-to-remote(pid=7309): Launched 05:33:23.720502 run scp-inst-to-remote(pid=7309): Terminated: ok {rc=0} 05:33:24.055075 run mk-remote-dir(pid=7311): Launched 05:33:26.253393 run mk-remote-dir(pid=7311): Terminated: ok {rc=0} 05:33:26.554137 run scp-cfg-to-remote(pid=7312): Launched 05:33:28.756276 run scp-cfg-to-remote(pid=7312): Terminated: ok {rc=0} 05:33:29.104858 run reload-dsp-firmware(pid=7314): Launched 05:33:32.303505 run reload-dsp-firmware(pid=7314): Terminated: ok {rc=0} 05:33:32.642392 run osmo-bts-sysmo(pid=7315): Launched 05:33:36.898301 tst mo_mt_call.py: using MSISDN 5564 05:33:37.080321 bus /sierra_1: Setting Powered True 05:33:40.563302 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5564', subscriber_id=1} 05:33:40.699574 tst mo_mt_call.py: using MSISDN 5565 05:33:40.870010 bus /sierra_2: Setting Powered True 05:33:53.226623 tst mo_mt_call.py:8: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb34029ff28>, timeout=10, timestep=1} [trial-820↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2] [trial-820↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1↪mo_mt_call.py:8] 05:33:53.234669 tst mo_mt_call.py:8: Test FAILED (66.3 sec) 05:33:53.271124 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7294): Terminating (SIGTERM) 05:33:53.313394 run osmo-hlr_10.42.42.2(pid=7295): Terminating (SIGTERM) 05:33:53.357261 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7296): Terminating (SIGTERM) 05:33:53.402216 run osmo-stp_10.42.42.5(pid=7297): Terminating (SIGTERM) 05:33:53.468183 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7298): Terminating (SIGTERM) 05:33:53.512010 run osmo-msc_10.42.42.6(pid=7299): Terminating (SIGTERM) 05:33:53.557801 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7301): Terminating (SIGTERM) 05:33:53.601281 run osmo-mgw_10.42.42.3(pid=7302): Terminating (SIGTERM) 05:33:53.644354 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7303): Terminating (SIGTERM) 05:33:53.688809 run osmo-mgw_10.42.42.4(pid=7304): Terminating (SIGTERM) 05:33:53.732549 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7305): Terminating (SIGTERM) 05:33:53.777530 run osmo-bsc_10.42.42.7(pid=7306): Terminating (SIGTERM) 05:33:53.820842 run osmo-bts-sysmo(pid=7315): Terminating (SIGTERM) 05:33:53.830484 --- ParallelTerminationStrategy: PID 7294 died... 05:33:53.888242 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7294): Terminated: ok {rc=0} 05:33:53.898917 --- ParallelTerminationStrategy: PID 7295 died... 05:33:53.956267 run osmo-hlr_10.42.42.2(pid=7295): Terminated: ok {rc=0} 05:33:53.966876 --- ParallelTerminationStrategy: PID 7296 died... 05:33:54.024227 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7296): Terminated: ok {rc=0} 05:33:54.034770 --- ParallelTerminationStrategy: PID 7297 died... 05:33:54.092308 run osmo-stp_10.42.42.5(pid=7297): Terminated: ok {rc=0} 05:33:54.103030 --- ParallelTerminationStrategy: PID 7298 died... 05:33:54.160164 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7298): Terminated: ok {rc=0} 05:33:54.170466 --- ParallelTerminationStrategy: PID 7299 died... 05:33:54.227037 run osmo-msc_10.42.42.6(pid=7299): Terminated: ok {rc=0} 05:33:54.237067 --- ParallelTerminationStrategy: PID 7301 died... 05:33:54.293635 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7301): Terminated: ok {rc=0} 05:33:54.303750 --- ParallelTerminationStrategy: PID 7302 died... 05:33:54.359943 run osmo-mgw_10.42.42.3(pid=7302): Terminated {rc=15} 05:33:54.370090 --- ParallelTerminationStrategy: PID 7303 died... 05:33:54.426600 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7303): Terminated: ok {rc=0} 05:33:54.436738 --- ParallelTerminationStrategy: PID 7304 died... 05:33:54.493739 run osmo-mgw_10.42.42.4(pid=7304): Terminated {rc=15} 05:33:54.503908 --- ParallelTerminationStrategy: PID 7305 died... 05:33:54.560219 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7305): Terminated: ok {rc=0} 05:33:54.570409 --- ParallelTerminationStrategy: PID 7306 died... 05:33:54.626586 run osmo-bsc_10.42.42.7(pid=7306): Terminated: ok {rc=0} 05:33:54.636879 --- ParallelTerminationStrategy: PID 7315 died... 05:33:54.693404 run osmo-bts-sysmo(pid=7315): Terminated: ok {rc=0} 05:33:54.800792 bus /sierra_2: Setting Online False 05:33:55.888325 bus /sierra_2: Setting Powered False 05:34:01.968354 bus /sierra_2: Setting Powered False 05:34:03.070693 bus /sierra_2: Setting Powered False 05:34:04.204176 bus /sierra_1: Setting Online False 05:34:05.292295 bus /sierra_1: Setting Powered False 05:34:10.963212 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/mo_mt_call.py", line 8, in <module> test_mo_mt_call(False, False) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/lib/testlib.py", line 39, in test_mo_mt_call hlr.subscriber_add(ms_mt) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb34029ff28>, timeout=10, timestep=1} [trial-820↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr1↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]