jenkins LTS upgrade
Failed
voice:sysmo+mod-bts0-dynts-osmo.mo_mt_call.py (from voice_sysmo+mod-bts0-dynts-osmo)
Stacktrace
/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f0055b4b0d0>, timeout=10, timestep=1} [trial-824↪voice:sysmo+mod-bts0-dynts-osmo↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]
Standard Output
---------------------------------------------- trial-824 voice:sysmo+mod-bts0-dynts-osmo mo_mt_call.py ---------------------------------------------- 09:47:29.980681 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x ip_address (candidates: 6) 09:47:29.992072 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x bts (candidates: 1) 09:47:30.151117 tst mo_mt_call.py: using LAC 8924 09:47:30.281261 tst mo_mt_call.py: using RAC 254 09:47:30.412698 tst mo_mt_call.py: using CellId 8924 09:47:30.541850 tst mo_mt_call.py: using BVCI 8920 09:47:30.567349 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x ip_address (candidates: 6) 09:47:30.578870 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x ip_address (candidates: 6) 09:47:30.590160 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x ip_address (candidates: 6) 09:47:30.601403 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x ip_address (candidates: 6) 09:47:30.613009 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x ip_address (candidates: 6) 09:47:30.624255 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x modem (candidates: 2) 09:47:30.729386 tst voice:sysmo+mod-bts0-dynts-osmo: Using 1 x modem (candidates: 2) 09:47:30.926821 run osmo-hlr_10.42.42.2: Starting osmo-hlr 09:47:31.454886 run create_hlr_db(pid=16685): Launched 09:47:31.640284 bus /sierra_1: Setting Powered False 09:47:32.659197 run create_hlr_db(pid=16685): Terminated: ok {rc=0} 09:47:33.104807 run pcap-recorder_any(filters='host 10.42.42.2')(pid=16687): Launched 09:47:33.441583 run osmo-hlr_10.42.42.2(pid=16688): Launched 09:47:33.562901 run osmo-stp_10.42.42.5: Starting osmo-stp 09:47:34.146453 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=16689): Launched 09:47:34.484231 run osmo-stp_10.42.42.5(pid=16690): Launched 09:47:34.603521 run osmo-msc_10.42.42.6: Starting osmo-msc 09:47:35.214725 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=16691): Launched 09:47:35.552884 run osmo-msc_10.42.42.6(pid=16692): Launched 09:47:38.732381 run osmo-mgw_10.42.42.3: Starting osmo-mgw 09:47:39.315558 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=16693): Launched 09:47:39.653242 run osmo-mgw_10.42.42.3(pid=16694): Launched 09:47:39.773138 run osmo-mgw_10.42.42.4: Starting osmo-mgw 09:47:40.345638 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=16695): Launched 09:47:40.683398 run osmo-mgw_10.42.42.4(pid=16696): Launched 09:47:40.804926 run osmo-bsc_10.42.42.7: Starting osmo-bsc 09:47:41.506469 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=16697): Launched 09:47:41.846174 run osmo-bsc_10.42.42.7(pid=16698): Launched 09:47:48.133548 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 09:47:48.737891 run rm-remote-dir(pid=16699): Launched 09:47:50.947348 run rm-remote-dir(pid=16699): Terminated: ok {rc=0} 09:47:51.309483 run mk-remote-dir(pid=16700): Launched 09:47:53.516240 run mk-remote-dir(pid=16700): Terminated: ok {rc=0} 09:47:53.822648 run scp-inst-to-remote(pid=16702): Launched 09:48:07.038129 run scp-inst-to-remote(pid=16702): Terminated: ok {rc=0} 09:48:07.377101 run mk-remote-dir(pid=16704): Launched 09:48:09.571264 run mk-remote-dir(pid=16704): Terminated: ok {rc=0} 09:48:09.862847 run scp-cfg-to-remote(pid=16705): Launched 09:48:12.058043 run scp-cfg-to-remote(pid=16705): Terminated: ok {rc=0} 09:48:12.391526 run reload-dsp-firmware(pid=16707): Launched 09:48:15.593049 run reload-dsp-firmware(pid=16707): Terminated: ok {rc=0} 09:48:15.936872 run osmo-bts-sysmo(pid=16708): Launched 09:48:21.194496 tst mo_mt_call.py: using MSISDN 5800 09:48:21.372373 bus /sierra_1: Setting Powered True 09:48:24.869848 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5800', subscriber_id=1} 09:48:25.006714 tst mo_mt_call.py: using MSISDN 5801 09:48:25.181264 bus /sierra_2: Setting Powered True 09:48:37.550050 tst mo_mt_call.py:8: ERR: Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f0055b4b0d0>, timeout=10, timestep=1} [trial-824↪voice:sysmo+mod-bts0-dynts-osmo↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2] [trial-824↪voice:sysmo+mod-bts0-dynts-osmo↪mo_mt_call.py:8] 09:48:37.557298 tst mo_mt_call.py:8: Test FAILED (67.6 sec) 09:48:37.592114 run pcap-recorder_any(filters='host 10.42.42.2')(pid=16687): Terminating (SIGTERM) 09:48:37.631966 run osmo-hlr_10.42.42.2(pid=16688): Terminating (SIGTERM) 09:48:37.672867 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=16689): Terminating (SIGTERM) 09:48:37.715306 run osmo-stp_10.42.42.5(pid=16690): Terminating (SIGTERM) 09:48:37.757934 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=16691): Terminating (SIGTERM) 09:48:37.801264 run osmo-msc_10.42.42.6(pid=16692): Terminating (SIGTERM) 09:48:37.842783 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=16693): Terminating (SIGTERM) 09:48:37.885344 run osmo-mgw_10.42.42.3(pid=16694): Terminating (SIGTERM) 09:48:37.927822 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=16695): Terminating (SIGTERM) 09:48:37.971398 run osmo-mgw_10.42.42.4(pid=16696): Terminating (SIGTERM) 09:48:38.012280 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=16697): Terminating (SIGTERM) 09:48:38.052026 run osmo-bsc_10.42.42.7(pid=16698): Terminating (SIGTERM) 09:48:38.092302 run osmo-bts-sysmo(pid=16708): Terminating (SIGTERM) 09:48:38.103478 --- ParallelTerminationStrategy: PID 16687 died... 09:48:38.158820 run pcap-recorder_any(filters='host 10.42.42.2')(pid=16687): Terminated: ok {rc=0} 09:48:38.168774 --- ParallelTerminationStrategy: PID 16688 died... 09:48:38.228442 run osmo-hlr_10.42.42.2(pid=16688): Terminated: ok {rc=0} 09:48:38.238500 --- ParallelTerminationStrategy: PID 16689 died... 09:48:38.291945 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=16689): Terminated: ok {rc=0} 09:48:38.301999 --- ParallelTerminationStrategy: PID 16690 died... 09:48:38.355800 run osmo-stp_10.42.42.5(pid=16690): Terminated: ok {rc=0} 09:48:38.365928 --- ParallelTerminationStrategy: PID 16691 died... 09:48:38.419535 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=16691): Terminated: ok {rc=0} 09:48:38.429138 --- ParallelTerminationStrategy: PID 16692 died... 09:48:38.482729 run osmo-msc_10.42.42.6(pid=16692): Terminated: ok {rc=0} 09:48:38.492277 --- ParallelTerminationStrategy: PID 16693 died... 09:48:38.546170 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=16693): Terminated: ok {rc=0} 09:48:38.556285 --- ParallelTerminationStrategy: PID 16694 died... 09:48:38.609835 run osmo-mgw_10.42.42.3(pid=16694): Terminated {rc=15} 09:48:38.619377 --- ParallelTerminationStrategy: PID 16695 died... 09:48:38.672899 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=16695): Terminated: ok {rc=0} 09:48:38.682579 --- ParallelTerminationStrategy: PID 16696 died... 09:48:38.736014 run osmo-mgw_10.42.42.4(pid=16696): Terminated {rc=15} 09:48:38.745829 --- ParallelTerminationStrategy: PID 16697 died... 09:48:38.799191 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=16697): Terminated: ok {rc=0} 09:48:38.808962 --- ParallelTerminationStrategy: PID 16698 died... 09:48:38.862723 run osmo-bsc_10.42.42.7(pid=16698): Terminated: ok {rc=0} 09:48:38.872594 --- ParallelTerminationStrategy: PID 16708 died... 09:48:38.925544 run osmo-bts-sysmo(pid=16708): Terminated: ok {rc=0} 09:48:39.034895 bus /sierra_2: Setting Online False 09:48:40.118976 bus /sierra_2: Setting Powered False 09:48:45.963227 bus /sierra_2: Setting Powered False 09:48:47.069054 bus /sierra_2: Setting Powered False 09:48:48.196246 bus /sierra_1: Setting Online False 09:48:49.282721 bus /sierra_1: Setting Powered False 09:48:54.964394 bus /sierra_1: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/mo_mt_call.py", line 8, in <module> test_mo_mt_call(False, False) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/voice/lib/testlib.py", line 39, in test_mo_mt_call hlr.subscriber_add(ms_mt) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id, File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10) File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7f0055b4b0d0>, timeout=10, timestep=1} [trial-824↪voice:sysmo+mod-bts0-dynts-osmo↪mo_mt_call.py:8↪mo_mt_call.py↪/sierra_2]