jenkins LTS upgrade
Failed
voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3.mo_mt_call.py (from voice_sysmo+mod-bts0-ts-tchh+cfg-codec-hr3)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a85b1048>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-816 voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3 mo_mt_call.py ---------------------------------------------- 00:26:18.034808 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x ip_address (candidates: 6) 00:26:18.046659 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x bts (candidates: 1) 00:26:18.196030 tst mo_mt_call.py: using LAC 8280 00:26:18.320261 tst mo_mt_call.py: using RAC 120 00:26:18.444536 tst mo_mt_call.py: using CellId 8280 00:26:18.568384 tst mo_mt_call.py: using BVCI 8276 00:26:18.592399 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x ip_address (candidates: 6) 00:26:18.603664 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x ip_address (candidates: 6) 00:26:18.614785 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x ip_address (candidates: 6) 00:26:18.626108 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x ip_address (candidates: 6) 00:26:18.637379 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x ip_address (candidates: 6) 00:26:18.648676 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x modem (candidates: 2) 00:26:18.756700 tst voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3: Using 1 x modem (candidates: 2) 00:26:18.955619 run osmo-hlr_10.42.42.2: Starting osmo-hlr 00:26:19.488483 run create_hlr_db(pid=28732): Launched 00:26:19.671434 bus /sierra_1: Setting Powered False 00:26:20.692112 run create_hlr_db(pid=28732): Terminated: ok {rc=0} 00:26:21.139596 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28734): Launched 00:26:21.477589 run osmo-hlr_10.42.42.2(pid=28735): Launched 00:26:21.602310 run osmo-stp_10.42.42.5: Starting osmo-stp 00:26:22.184936 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28736): Launched 00:26:22.526103 run osmo-stp_10.42.42.5(pid=28737): Launched 00:26:22.645425 run osmo-msc_10.42.42.6: Starting osmo-msc 00:26:23.266096 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28738): Launched 00:26:23.608970 run osmo-msc_10.42.42.6(pid=28739): Launched 00:26:26.780220 run osmo-mgw_10.42.42.3: Starting osmo-mgw 00:26:27.327934 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28740): Launched 00:26:27.647513 run osmo-mgw_10.42.42.3(pid=28741): Launched 00:26:27.760641 run osmo-mgw_10.42.42.4: Starting osmo-mgw 00:26:28.302153 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28742): Launched 00:26:28.618879 run osmo-mgw_10.42.42.4(pid=28743): Launched 00:26:28.731165 run osmo-bsc_10.42.42.7: Starting osmo-bsc 00:26:29.389342 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28744): Launched 00:26:29.708758 run osmo-bsc_10.42.42.7(pid=28745): Launched 00:26:35.978112 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 00:26:36.544303 run rm-remote-dir(pid=28746): Launched 00:26:38.739633 run rm-remote-dir(pid=28746): Terminated: ok {rc=0} 00:26:39.076363 run mk-remote-dir(pid=28747): Launched 00:26:41.270334 run mk-remote-dir(pid=28747): Terminated: ok {rc=0} 00:26:41.556805 run scp-inst-to-remote(pid=28748): Launched 00:26:54.780577 run scp-inst-to-remote(pid=28748): Terminated: ok {rc=0} 00:26:55.133836 run mk-remote-dir(pid=28751): Launched 00:26:57.336645 run mk-remote-dir(pid=28751): Terminated: ok {rc=0} 00:26:57.640604 run scp-cfg-to-remote(pid=28752): Launched 00:26:58.843060 run scp-cfg-to-remote(pid=28752): Terminated: ok {rc=0} 00:26:59.192818 run reload-dsp-firmware(pid=28754): Launched 00:27:02.394737 run reload-dsp-firmware(pid=28754): Terminated: ok {rc=0} 00:27:02.736436 run osmo-bts-sysmo(pid=28755): Launched 00:27:07.993258 tst mo_mt_call.py: using MSISDN 5368 00:27:08.169886 bus /sierra_1: Setting Powered True 00:27:11.649543 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5368', subscriber_id=1} 00:27:11.787665 tst mo_mt_call.py: using MSISDN 5369 00:27:11.959023 bus /sierra_2: Setting Powered True 00:27:24.305791 tst mo_mt_call.py:8: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a85b1048>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2] [trial-816↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3↪mo_mt_call.py:8] 00:27:24.313081 tst mo_mt_call.py:8: Test FAILED (66.3 sec) 00:27:24.348546 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28734): Terminating (SIGTERM) 00:27:24.388481 run osmo-hlr_10.42.42.2(pid=28735): Terminating (SIGTERM) 00:27:24.430281 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28736): Terminating (SIGTERM) 00:27:24.473724 run osmo-stp_10.42.42.5(pid=28737): Terminating (SIGTERM) 00:27:24.524022 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28738): Terminating (SIGTERM) 00:27:24.568266 run osmo-msc_10.42.42.6(pid=28739): Terminating (SIGTERM) 00:27:24.609660 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28740): Terminating (SIGTERM) 00:27:24.649677 run osmo-mgw_10.42.42.3(pid=28741): Terminating (SIGTERM) 00:27:24.690517 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28742): Terminating (SIGTERM) 00:27:24.731107 run osmo-mgw_10.42.42.4(pid=28743): Terminating (SIGTERM) 00:27:24.778262 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28744): Terminating (SIGTERM) 00:27:24.825139 run osmo-bsc_10.42.42.7(pid=28745): Terminating (SIGTERM) 00:27:24.870559 run osmo-bts-sysmo(pid=28755): Terminating (SIGTERM) 00:27:24.880951 --- ParallelTerminationStrategy: PID 28734 died... 00:27:24.953387 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28734): Terminated: ok {rc=0} 00:27:24.964168 --- ParallelTerminationStrategy: PID 28735 died... 00:27:25.023265 run osmo-hlr_10.42.42.2(pid=28735): Terminated: ok {rc=0} 00:27:25.034364 --- ParallelTerminationStrategy: PID 28736 died... 00:27:25.093201 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28736): Terminated: ok {rc=0} 00:27:25.105030 --- ParallelTerminationStrategy: PID 28737 died... 00:27:25.163756 run osmo-stp_10.42.42.5(pid=28737): Terminated: ok {rc=0} 00:27:25.174252 --- ParallelTerminationStrategy: PID 28738 died... 00:27:25.231620 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28738): Terminated: ok {rc=0} 00:27:25.242027 --- ParallelTerminationStrategy: PID 28739 died... 00:27:25.299583 run osmo-msc_10.42.42.6(pid=28739): Terminated: ok {rc=0} 00:27:25.310519 --- ParallelTerminationStrategy: PID 28740 died... 00:27:25.368213 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28740): Terminated: ok {rc=0} 00:27:25.378749 --- ParallelTerminationStrategy: PID 28741 died... 00:27:25.436411 run osmo-mgw_10.42.42.3(pid=28741): Terminated {rc=15} 00:27:25.446744 --- ParallelTerminationStrategy: PID 28742 died... 00:27:25.504381 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28742): Terminated: ok {rc=0} 00:27:25.514938 --- ParallelTerminationStrategy: PID 28743 died... 00:27:25.571660 run osmo-mgw_10.42.42.4(pid=28743): Terminated {rc=15} 00:27:25.582647 --- ParallelTerminationStrategy: PID 28744 died... 00:27:25.639648 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28744): Terminated: ok {rc=0} 00:27:25.650532 --- ParallelTerminationStrategy: PID 28745 died... 00:27:25.708062 run osmo-bsc_10.42.42.7(pid=28745): Terminated: ok {rc=0} 00:27:25.718473 --- ParallelTerminationStrategy: PID 28755 died... 00:27:25.776238 run osmo-bts-sysmo(pid=28755): Terminated: ok {rc=0} 00:27:25.883951 bus /sierra_2: Setting Online False 00:27:26.971476 bus /sierra_2: Setting Powered False 00:27:32.972280 bus /sierra_2: Setting Powered False 00:27:34.075208 bus /sierra_2: Setting Powered False 00:27:35.205327 bus /sierra_1: Setting Online False 00:27:36.291824 bus /sierra_1: Setting Powered False 00:27:41.970923 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/mo_mt_call.py", line 8, in <module> test_mo_mt_call(False, False) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/lib/testlib.py", line 39, in test_mo_mt_call hlr.subscriber_add(ms_mt) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a85b1048>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-ts-tchh+cfg-codec-hr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]