jenkins LTS upgrade
Failed
voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3.mo_mt_call.py (from voice_sysmo+mod-bts0-ts-tchf+cfg-codec-fr3)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a9835268>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-816 voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3 mo_mt_call.py ---------------------------------------------- 00:20:38.986246 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 00:20:38.997425 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x bts (candidates: 1) 00:20:39.151836 tst mo_mt_call.py: using LAC 8276 00:20:39.279984 tst mo_mt_call.py: using RAC 116 00:20:39.404196 tst mo_mt_call.py: using CellId 8276 00:20:39.531056 tst mo_mt_call.py: using BVCI 8272 00:20:39.555310 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 00:20:39.566614 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 00:20:39.577646 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 00:20:39.588668 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 00:20:39.599904 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x ip_address (candidates: 6) 00:20:39.611214 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x modem (candidates: 2) 00:20:39.716539 tst voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3: Using 1 x modem (candidates: 2) 00:20:39.916289 run osmo-hlr_10.42.42.2: Starting osmo-hlr 00:20:40.456181 run create_hlr_db(pid=28606): Launched 00:20:40.650543 bus /sierra_1: Setting Powered False 00:20:41.672180 run create_hlr_db(pid=28606): Terminated: ok {rc=0} 00:20:42.099895 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28608): Launched 00:20:42.420569 run osmo-hlr_10.42.42.2(pid=28609): Launched 00:20:42.534973 run osmo-stp_10.42.42.5: Starting osmo-stp 00:20:43.087849 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28610): Launched 00:20:43.406491 run osmo-stp_10.42.42.5(pid=28611): Launched 00:20:43.519436 run osmo-msc_10.42.42.6: Starting osmo-msc 00:20:44.100709 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28612): Launched 00:20:44.421052 run osmo-msc_10.42.42.6(pid=28613): Launched 00:20:47.590039 run osmo-mgw_10.42.42.3: Starting osmo-mgw 00:20:48.141530 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28614): Launched 00:20:48.461708 run osmo-mgw_10.42.42.3(pid=28615): Launched 00:20:48.574685 run osmo-mgw_10.42.42.4: Starting osmo-mgw 00:20:49.116042 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28616): Launched 00:20:49.435191 run osmo-mgw_10.42.42.4(pid=28617): Launched 00:20:49.548621 run osmo-bsc_10.42.42.7: Starting osmo-bsc 00:20:50.211606 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28618): Launched 00:20:50.536103 run osmo-bsc_10.42.42.7(pid=28619): Launched 00:20:56.806100 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 00:20:57.373879 run rm-remote-dir(pid=28620): Launched 00:20:59.570860 run rm-remote-dir(pid=28620): Terminated: ok {rc=0} 00:20:59.910915 run mk-remote-dir(pid=28621): Launched 00:21:02.106578 run mk-remote-dir(pid=28621): Terminated: ok {rc=0} 00:21:02.392433 run scp-inst-to-remote(pid=28622): Launched 00:21:15.605528 run scp-inst-to-remote(pid=28622): Terminated: ok {rc=0} 00:21:15.941057 run mk-remote-dir(pid=28624): Launched 00:21:18.132863 run mk-remote-dir(pid=28624): Terminated: ok {rc=0} 00:21:18.420227 run scp-cfg-to-remote(pid=28625): Launched 00:21:20.613743 run scp-cfg-to-remote(pid=28625): Terminated: ok {rc=0} 00:21:20.963168 run reload-dsp-firmware(pid=28627): Launched 00:21:23.152844 run reload-dsp-firmware(pid=28627): Terminated: ok {rc=0} 00:21:23.476519 run osmo-bts-sysmo(pid=28628): Launched 00:21:28.726718 tst mo_mt_call.py: using MSISDN 5360 00:21:28.902710 bus /sierra_1: Setting Powered True 00:21:32.397400 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5360', subscriber_id=1} 00:21:32.542077 tst mo_mt_call.py: using MSISDN 5361 00:21:32.724222 bus /sierra_2: Setting Powered True 00:21:45.072691 tst mo_mt_call.py:8: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a9835268>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2] [trial-816↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8] 00:21:45.080654 tst mo_mt_call.py:8: Test FAILED (66.1 sec) 00:21:45.117673 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28608): Terminating (SIGTERM) 00:21:45.160121 run osmo-hlr_10.42.42.2(pid=28609): Terminating (SIGTERM) 00:21:45.204413 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28610): Terminating (SIGTERM) 00:21:45.248613 run osmo-stp_10.42.42.5(pid=28611): Terminating (SIGTERM) 00:21:45.318856 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28612): Terminating (SIGTERM) 00:21:45.364413 run osmo-msc_10.42.42.6(pid=28613): Terminating (SIGTERM) 00:21:45.408475 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28614): Terminating (SIGTERM) 00:21:45.454115 run osmo-mgw_10.42.42.3(pid=28615): Terminating (SIGTERM) 00:21:45.509901 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28616): Terminating (SIGTERM) 00:21:45.555363 run osmo-mgw_10.42.42.4(pid=28617): Terminating (SIGTERM) 00:21:45.612042 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28618): Terminating (SIGTERM) 00:21:45.654647 run osmo-bsc_10.42.42.7(pid=28619): Terminating (SIGTERM) 00:21:45.696763 run osmo-bts-sysmo(pid=28628): Terminating (SIGTERM) 00:21:45.706487 --- ParallelTerminationStrategy: PID 28608 died... 00:21:45.763921 run pcap-recorder_any(filters='host 10.42.42.2')(pid=28608): Terminated: ok {rc=0} 00:21:45.774818 --- ParallelTerminationStrategy: PID 28609 died... 00:21:45.832762 run osmo-hlr_10.42.42.2(pid=28609): Terminated: ok {rc=0} 00:21:45.843728 --- ParallelTerminationStrategy: PID 28610 died... 00:21:45.902094 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=28610): Terminated: ok {rc=0} 00:21:45.913410 --- ParallelTerminationStrategy: PID 28611 died... 00:21:45.971139 run osmo-stp_10.42.42.5(pid=28611): Terminated: ok {rc=0} 00:21:45.981863 --- ParallelTerminationStrategy: PID 28612 died... 00:21:46.037969 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=28612): Terminated: ok {rc=0} 00:21:46.048155 --- ParallelTerminationStrategy: PID 28613 died... 00:21:46.104496 run osmo-msc_10.42.42.6(pid=28613): Terminated: ok {rc=0} 00:21:46.114644 --- ParallelTerminationStrategy: PID 28614 died... 00:21:46.171393 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=28614): Terminated: ok {rc=0} 00:21:46.181635 --- ParallelTerminationStrategy: PID 28615 died... 00:21:46.237567 run osmo-mgw_10.42.42.3(pid=28615): Terminated {rc=15} 00:21:46.247660 --- ParallelTerminationStrategy: PID 28616 died... 00:21:46.304083 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=28616): Terminated: ok {rc=0} 00:21:46.314269 --- ParallelTerminationStrategy: PID 28617 died... 00:21:46.370474 run osmo-mgw_10.42.42.4(pid=28617): Terminated {rc=15} 00:21:46.380618 --- ParallelTerminationStrategy: PID 28618 died... 00:21:46.436663 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=28618): Terminated: ok {rc=0} 00:21:46.446954 --- ParallelTerminationStrategy: PID 28619 died... 00:21:46.503097 run osmo-bsc_10.42.42.7(pid=28619): Terminated: ok {rc=0} 00:21:46.513402 --- ParallelTerminationStrategy: PID 28628 died... 00:21:46.569375 run osmo-bts-sysmo(pid=28628): Terminated: ok {rc=0} 00:21:46.677551 bus /sierra_2: Setting Online False 00:21:47.763472 bus /sierra_2: Setting Powered False 00:21:53.120731 bus /sierra_2: Setting Powered False 00:21:54.227569 bus /sierra_2: Setting Powered False 00:21:55.358458 bus /sierra_1: Setting Online False 00:21:56.446581 bus /sierra_1: Setting Powered False 00:22:01.966546 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/mo_mt_call.py", line 8, in <module> test_mo_mt_call(False, False) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/lib/testlib.py", line 39, in test_mo_mt_call hlr.subscriber_add(ms_mt) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f01a9835268>, timeout=10, timestep=1} [trial-816↪voice:sysmo+mod-bts0-ts-tchf+cfg-codec-fr3↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]