jenkins LTS upgrade
Failed
voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3.mo_mt_call.py (from voice_sysmo+mod-bts0-ts-tchf+cfg-codec-fr3)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f64d5c30598>, timeout=10, timestep=1} [trial-809↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-809 voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3 mo_mt_call.py ---------------------------------------------- 14:24:35.995676 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 14:24:36.007164 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x bts (candidates: 1) 14:24:36.159358 tst mo_mt_call.py: using LAC 7716 14:24:36.285693 tst mo_mt_call.py: using RAC 66 14:24:36.412649 tst mo_mt_call.py: using CellId 7716 14:24:36.540713 tst mo_mt_call.py: using BVCI 7712 14:24:36.565898 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 14:24:36.577407 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 14:24:36.588668 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 14:24:36.599976 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 14:24:36.611282 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 14:24:36.622662 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x modem (candidates: 2) 14:24:36.728341 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x modem (candidates: 2) 14:24:36.927182 run osmo-hlr_10.42.42.2: Starting osmo-hlr 14:24:37.451947 run create_hlr_db(pid=3846): Launched 14:24:37.641818 bus /sierra_1: Setting Powered False 14:24:38.663247 run create_hlr_db(pid=3846): Terminated: ok {rc=0} 14:24:39.080498 run pcap-recorder_any(filters='host 10.42.42.2')(pid=3848): Launched 14:24:39.396209 run osmo-hlr_10.42.42.2(pid=3849): Launched 14:24:39.507293 run osmo-stp_10.42.42.5: Starting osmo-stp 14:24:40.041637 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=3850): Launched 14:24:40.350348 run osmo-stp_10.42.42.5(pid=3851): Launched 14:24:40.464796 run osmo-msc_10.42.42.6: Starting osmo-msc 14:24:41.029354 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=3852): Launched 14:24:41.340247 run osmo-msc_10.42.42.6(pid=3853): Launched 14:24:44.505854 run osmo-mgw_10.42.42.3: Starting osmo-mgw 14:24:45.039863 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=3854): Launched 14:24:45.351025 run osmo-mgw_10.42.42.3(pid=3855): Launched 14:24:45.460276 run osmo-mgw_10.42.42.4: Starting osmo-mgw 14:24:45.987724 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=3856): Launched 14:24:46.296410 run osmo-mgw_10.42.42.4(pid=3857): Launched 14:24:46.405670 run osmo-bsc_10.42.42.7: Starting osmo-bsc 14:24:47.049624 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=3858): Launched 14:24:47.360022 run osmo-bsc_10.42.42.7(pid=3859): Launched 14:24:53.631052 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 14:24:54.220211 run rm-remote-dir(pid=3860): Launched 14:24:56.424811 run rm-remote-dir(pid=3860): Terminated: ok {rc=0} 14:24:56.775619 run mk-remote-dir(pid=3861): Launched 14:24:57.975685 run mk-remote-dir(pid=3861): Terminated: ok {rc=0} 14:24:58.273268 run scp-inst-to-remote(pid=3862): Launched 14:25:11.493078 run scp-inst-to-remote(pid=3862): Terminated: ok {rc=0} 14:25:11.841030 run mk-remote-dir(pid=3872): Launched 14:25:14.039995 run mk-remote-dir(pid=3872): Terminated: ok {rc=0} 14:25:14.337408 run scp-cfg-to-remote(pid=3873): Launched 14:25:16.537727 run scp-cfg-to-remote(pid=3873): Terminated: ok {rc=0} 14:25:16.880077 run reload-dsp-firmware(pid=3875): Launched 14:25:19.065423 run reload-dsp-firmware(pid=3875): Terminated: ok {rc=0} 14:25:19.384758 run osmo-bts-sysmo(pid=3876): Launched 14:25:24.634380 tst mo_mt_call.py: using MSISDN 4989 14:25:24.806071 bus /sierra_1: Setting Powered True 14:25:28.283468 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='4989', subscriber_id=1} 14:25:28.420877 tst mo_mt_call.py: using MSISDN 4990 14:25:28.589929 bus /sierra_2: Setting Powered True 14:25:40.934138 tst mo_mt_call.py:8: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f64d5c30598>, timeout=10, timestep=1} [trial-809↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2] [trial-809↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8] 14:25:40.941531 tst mo_mt_call.py:8: Test FAILED (65.0 sec) 14:25:40.976331 run pcap-recorder_any(filters='host 10.42.42.2')(pid=3848): Terminating (SIGTERM) 14:25:41.015586 run osmo-hlr_10.42.42.2(pid=3849): Terminating (SIGTERM) 14:25:41.055838 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=3850): Terminating (SIGTERM) 14:25:41.098353 run osmo-stp_10.42.42.5(pid=3851): Terminating (SIGTERM) 14:25:41.140506 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=3852): Terminating (SIGTERM) 14:25:41.186323 run osmo-msc_10.42.42.6(pid=3853): Terminating (SIGTERM) 14:25:41.233078 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=3854): Terminating (SIGTERM) 14:25:41.276586 run osmo-mgw_10.42.42.3(pid=3855): Terminating (SIGTERM) 14:25:41.333537 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=3856): Terminating (SIGTERM) 14:25:41.374214 run osmo-mgw_10.42.42.4(pid=3857): Terminating (SIGTERM) 14:25:41.415044 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=3858): Terminating (SIGTERM) 14:25:41.455277 run osmo-bsc_10.42.42.7(pid=3859): Terminating (SIGTERM) 14:25:41.495748 run osmo-bts-sysmo(pid=3876): Terminating (SIGTERM) 14:25:41.505061 --- ParallelTerminationStrategy: PID 3848 died... 14:25:41.558807 run pcap-recorder_any(filters='host 10.42.42.2')(pid=3848): Terminated: ok {rc=0} 14:25:41.568962 --- ParallelTerminationStrategy: PID 3849 died... 14:25:41.625069 run osmo-hlr_10.42.42.2(pid=3849): Terminated: ok {rc=0} 14:25:41.635627 --- ParallelTerminationStrategy: PID 3850 died... 14:25:41.691942 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=3850): Terminated: ok {rc=0} 14:25:41.702667 --- ParallelTerminationStrategy: PID 3851 died... 14:25:41.756289 run osmo-stp_10.42.42.5(pid=3851): Terminated: ok {rc=0} 14:25:41.766437 --- ParallelTerminationStrategy: PID 3852 died... 14:25:41.819984 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=3852): Terminated: ok {rc=0} 14:25:41.830306 --- ParallelTerminationStrategy: PID 3853 died... 14:25:41.883718 run osmo-msc_10.42.42.6(pid=3853): Terminated: ok {rc=0} 14:25:41.894173 --- ParallelTerminationStrategy: PID 3854 died... 14:25:41.948330 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=3854): Terminated: ok {rc=0} 14:25:41.958262 --- ParallelTerminationStrategy: PID 3855 died... 14:25:42.011780 run osmo-mgw_10.42.42.3(pid=3855): Terminated {rc=15} 14:25:42.021611 --- ParallelTerminationStrategy: PID 3856 died... 14:25:42.075351 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=3856): Terminated: ok {rc=0} 14:25:42.085627 --- ParallelTerminationStrategy: PID 3857 died... 14:25:42.139075 run osmo-mgw_10.42.42.4(pid=3857): Terminated {rc=15} 14:25:42.148847 --- ParallelTerminationStrategy: PID 3858 died... 14:25:42.202578 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=3858): Terminated: ok {rc=0} 14:25:42.212964 --- ParallelTerminationStrategy: PID 3859 died... 14:25:42.266874 run osmo-bsc_10.42.42.7(pid=3859): Terminated: ok {rc=0} 14:25:42.277115 --- ParallelTerminationStrategy: PID 3876 died... 14:25:42.333734 run osmo-bts-sysmo(pid=3876): Terminated: ok {rc=0} 14:25:42.442801 bus /sierra_2: Setting Online False 14:25:43.527054 bus /sierra_2: Setting Powered False 14:25:48.958654 bus /sierra_2: Setting Powered False 14:25:50.064904 bus /sierra_2: Setting Powered False 14:25:51.192851 bus /sierra_1: Setting Online False 14:25:52.280409 bus /sierra_1: Setting Powered False 14:25:57.960758 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/mo_mt_call.py", line 8, in <module> test_mo_mt_call(False, False) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/lib/testlib.py", line 39, in test_mo_mt_call hlr.subscriber_add(ms_mt) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f64d5c30598>, timeout=10, timestep=1} [trial-809↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]