jenkins LTS upgrade
Failed
voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any.mo_mt_call.py (from voice_sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a9772a60>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-816 voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any mo_mt_call.py ---------------------------------------------- 00:29:08.988324 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x ip_address (candidates: 6) 00:29:08.999656 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x bts (candidates: 1) 00:29:09.149256 tst mo_mt_call.py: using LAC 8282 00:29:09.280040 tst mo_mt_call.py: using RAC 122 00:29:09.413804 tst mo_mt_call.py: using CellId 8282 00:29:09.547487 tst mo_mt_call.py: using BVCI 8278 00:29:09.573234 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x ip_address (candidates: 6) 00:29:09.585210 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x ip_address (candidates: 6) 00:29:09.596898 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x ip_address (candidates: 6) 00:29:09.608525 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x ip_address (candidates: 6) 00:29:09.620138 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x ip_address (candidates: 6) 00:29:09.631750 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x modem (candidates: 2) 00:29:09.737975 tst voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any: Using 1 x modem (candidates: 2) 00:29:09.940922 run osmo-hlr_10.42.42.2: Starting osmo-hlr 00:29:10.475772 run create_hlr_db(pid=28795): Launched 00:29:10.669103 bus /sierra_1: Setting Powered False 00:29:11.690915 run create_hlr_db(pid=28795): Terminated: ok {rc=0} 00:29:12.115910 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28797): Launched 00:29:12.434332 run osmo-hlr_10.42.42.2(pid=28798): Launched 00:29:12.548624 run osmo-stp_10.42.42.5: Starting osmo-stp 00:29:13.100562 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28799): Launched 00:29:13.419805 run osmo-stp_10.42.42.5(pid=28800): Launched 00:29:13.531988 run osmo-msc_10.42.42.6: Starting osmo-msc 00:29:14.111443 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28801): Launched 00:29:14.431342 run osmo-msc_10.42.42.6(pid=28802): Launched 00:29:17.600603 run osmo-mgw_10.42.42.3: Starting osmo-mgw 00:29:18.150771 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28803): Launched 00:29:18.470613 run osmo-mgw_10.42.42.3(pid=28804): Launched 00:29:18.583005 run osmo-mgw_10.42.42.4: Starting osmo-mgw 00:29:19.126809 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28805): Launched 00:29:19.446196 run osmo-mgw_10.42.42.4(pid=28806): Launched 00:29:19.558562 run osmo-bsc_10.42.42.7: Starting osmo-bsc 00:29:20.222538 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28807): Launched 00:29:20.541002 run osmo-bsc_10.42.42.7(pid=28808): Launched 00:29:26.812655 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 00:29:27.382565 run rm-remote-dir(pid=28809): Launched 00:29:29.579017 run rm-remote-dir(pid=28809): Terminated: ok {rc=0} 00:29:29.920029 run mk-remote-dir(pid=28810): Launched 00:29:32.114033 run mk-remote-dir(pid=28810): Terminated: ok {rc=0} 00:29:32.400009 run scp-inst-to-remote(pid=28811): Launched 00:29:45.610935 run scp-inst-to-remote(pid=28811): Terminated: ok {rc=0} 00:29:45.945737 run mk-remote-dir(pid=28813): Launched 00:29:48.138891 run mk-remote-dir(pid=28813): Terminated: ok {rc=0} 00:29:48.428377 run scp-cfg-to-remote(pid=28814): Launched 00:29:50.622970 run scp-cfg-to-remote(pid=28814): Terminated: ok {rc=0} 00:29:50.974724 run reload-dsp-firmware(pid=28816): Launched 00:29:53.176615 run reload-dsp-firmware(pid=28816): Terminated: ok {rc=0} 00:29:53.520102 run osmo-bts-sysmo(pid=28817): Launched 00:29:58.769640 tst mo_mt_call.py: using MSISDN 5372 00:29:58.955486 bus /sierra_1: Setting Powered True 00:30:02.433105 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5372', subscriber_id=1} 00:30:02.578038 tst mo_mt_call.py: using MSISDN 5373 00:30:02.760169 bus /sierra_2: Setting Powered True 00:30:15.123927 tst mo_mt_call.py:8: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a9772a60>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2] [trial-816↪voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any↪mo_mt_call.py:8] 00:30:15.131923 tst mo_mt_call.py:8: Test FAILED (66.2 sec) 00:30:15.168365 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28797): Terminating (SIGTERM) 00:30:15.209665 run osmo-hlr_10.42.42.2(pid=28798): Terminating (SIGTERM) 00:30:15.251124 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28799): Terminating (SIGTERM) 00:30:15.294965 run osmo-stp_10.42.42.5(pid=28800): Terminating (SIGTERM) 00:30:15.342909 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28801): Terminating (SIGTERM) 00:30:15.387868 run osmo-msc_10.42.42.6(pid=28802): Terminating (SIGTERM) 00:30:15.432568 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28803): Terminating (SIGTERM) 00:30:15.479137 run osmo-mgw_10.42.42.3(pid=28804): Terminating (SIGTERM) 00:30:15.521785 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28805): Terminating (SIGTERM) 00:30:15.564509 run osmo-mgw_10.42.42.4(pid=28806): Terminating (SIGTERM) 00:30:15.606803 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28807): Terminating (SIGTERM) 00:30:15.646819 run osmo-bsc_10.42.42.7(pid=28808): Terminating (SIGTERM) 00:30:15.687632 run osmo-bts-sysmo(pid=28817): Terminating (SIGTERM) 00:30:15.697675 --- ParallelTerminationStrategy: PID 28797 died... 00:30:15.752900 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28797): Terminated: ok {rc=0} 00:30:15.763278 --- ParallelTerminationStrategy: PID 28798 died... 00:30:15.819725 run osmo-hlr_10.42.42.2(pid=28798): Terminated: ok {rc=0} 00:30:15.830207 --- ParallelTerminationStrategy: PID 28799 died... 00:30:15.885618 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28799): Terminated: ok {rc=0} 00:30:15.896780 --- ParallelTerminationStrategy: PID 28800 died... 00:30:15.951715 run osmo-stp_10.42.42.5(pid=28800): Terminated: ok {rc=0} 00:30:15.961644 --- ParallelTerminationStrategy: PID 28801 died... 00:30:16.016248 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28801): Terminated: ok {rc=0} 00:30:16.026186 --- ParallelTerminationStrategy: PID 28802 died... 00:30:16.080064 run osmo-msc_10.42.42.6(pid=28802): Terminated: ok {rc=0} 00:30:16.090577 --- ParallelTerminationStrategy: PID 28803 died... 00:30:16.144743 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28803): Terminated: ok {rc=0} 00:30:16.155274 --- ParallelTerminationStrategy: PID 28804 died... 00:30:16.209824 run osmo-mgw_10.42.42.3(pid=28804): Terminated {rc=15} 00:30:16.219811 --- ParallelTerminationStrategy: PID 28805 died... 00:30:16.274675 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28805): Terminated: ok {rc=0} 00:30:16.284618 --- ParallelTerminationStrategy: PID 28806 died... 00:30:16.338814 run osmo-mgw_10.42.42.4(pid=28806): Terminated {rc=15} 00:30:16.349354 --- ParallelTerminationStrategy: PID 28807 died... 00:30:16.403366 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28807): Terminated: ok {rc=0} 00:30:16.414048 --- ParallelTerminationStrategy: PID 28808 died... 00:30:16.468050 run osmo-bsc_10.42.42.7(pid=28808): Terminated: ok {rc=0} 00:30:16.477995 --- ParallelTerminationStrategy: PID 28817 died... 00:30:16.531707 run osmo-bts-sysmo(pid=28817): Terminated: ok {rc=0} 00:30:16.641770 bus /sierra_2: Setting Online False 00:30:17.726777 bus /sierra_2: Setting Powered False 00:30:23.065872 bus /sierra_2: Setting Powered False 00:30:24.168921 bus /sierra_2: Setting Powered False 00:30:25.298167 bus /sierra_1: Setting Online False 00:30:26.383556 bus /sierra_1: Setting Powered False 00:30:31.959671 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/mo_mt_call.py", line 8, in <module> test_mo_mt_call(False, False) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/lib/testlib.py", line 39, in test_mo_mt_call hlr.subscriber_add(ms_mt) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a9772a60>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-dynts-ipa+cfg-codec-fr-any↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]