Failed

sms:sysmo.mo_mt_sms.py (from sms_sysmo)

Failing for the past 70 builds (Since #751 )

Took 1 min 19 sec.

Stacktrace

/sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb3401eca60>, timeout=10, timestep=1} [trial-820↪sms:sysmo↪mo_mt_sms.py:27↪mo_mt_sms.py↪/sierra_2]

Standard Output

----------------------------------------------
trial-820 sms:sysmo mo_mt_sms.py
----------------------------------------------
05:29:06.968548 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
05:29:06.979409 tst                        sms:sysmo: Using 1 x bts (candidates: 1)
05:29:07.104348 tst                     mo_mt_sms.py: using LAC 8589
05:29:07.226101 tst                     mo_mt_sms.py: using RAC 174
05:29:07.347959 tst                     mo_mt_sms.py: using CellId 8589
05:29:07.473386 tst                     mo_mt_sms.py: using BVCI 8585
05:29:07.498765 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
05:29:07.509880 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
05:29:07.520816 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
05:29:07.531786 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
05:29:07.542571 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
05:29:07.553364 tst                        sms:sysmo: Using 1 x modem (candidates: 2)
05:29:07.654300 tst                        sms:sysmo: Using 1 x modem (candidates: 2)
05:29:07.838667 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
05:29:09.454172 run          create_hlr_db(pid=7220): Launched
05:29:09.629849 bus                        /sierra_1: Setting Powered False
05:29:10.651527 run          create_hlr_db(pid=7220): Terminated: ok {rc=0}
05:29:11.059584 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7222): Launched
05:29:11.368073 run    osmo-hlr_10.42.42.2(pid=7223): Launched
05:29:11.478331 run              osmo-stp_10.42.42.5: Starting osmo-stp
05:29:13.922722 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7224): Launched
05:29:14.229623 run    osmo-stp_10.42.42.5(pid=7225): Launched
05:29:14.338782 run              osmo-msc_10.42.42.6: Starting osmo-msc
05:29:19.501726 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7226): Launched
05:29:19.807253 run    osmo-msc_10.42.42.6(pid=7227): Launched
05:29:22.970933 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
05:29:24.664084 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7228): Launched
05:29:24.971411 run    osmo-mgw_10.42.42.3(pid=7229): Launched
05:29:25.080647 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
05:29:25.603101 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7230): Launched
05:29:25.912906 run    osmo-mgw_10.42.42.4(pid=7231): Launched
05:29:26.022622 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
05:29:30.341707 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7232): Launched
05:29:30.652013 run    osmo-bsc_10.42.42.7(pid=7233): Launched
05:29:36.928141 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7
05:29:37.899813 run          rm-remote-dir(pid=7234): Launched
05:29:40.100880 run          rm-remote-dir(pid=7234): Terminated: ok {rc=0}
05:29:40.449487 run          mk-remote-dir(pid=7235): Launched
05:29:42.648398 run          mk-remote-dir(pid=7235): Terminated: ok {rc=0}
05:29:42.943986 run     scp-inst-to-remote(pid=7236): Launched
05:29:56.149994 run     scp-inst-to-remote(pid=7236): Terminated: ok {rc=0}
05:29:56.474327 run          mk-remote-dir(pid=7238): Launched
05:29:58.660515 run          mk-remote-dir(pid=7238): Terminated: ok {rc=0}
05:29:58.938059 run      scp-cfg-to-remote(pid=7239): Launched
05:30:01.123970 run      scp-cfg-to-remote(pid=7239): Terminated: ok {rc=0}
05:30:01.446028 run    reload-dsp-firmware(pid=7244): Launched
05:30:04.641488 run    reload-dsp-firmware(pid=7244): Terminated: ok {rc=0}
05:30:04.972253 run         osmo-bts-sysmo(pid=7245): Launched
05:30:09.227013 tst                     mo_mt_sms.py: using MSISDN 5561
05:30:09.397324 bus                        /sierra_1: Setting Powered True
05:30:12.886831 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000009031', msisdn='5561', subscriber_id=1}
05:30:13.030117 tst                     mo_mt_sms.py: using MSISDN 5562
05:30:13.199299 bus                        /sierra_2: Setting Powered True
05:30:25.575289 tst                  mo_mt_sms.py:27: ERR: Error: /sierra_2: Wait timeout {condition=&lt;function Modem.imsi.&lt;locals&gt;.&lt;lambda&gt; at 0x7fb3401eca60&gt;, timeout=10, timestep=1} [trial-820↪sms:sysmo↪mo_mt_sms.py:27↪mo_mt_sms.py↪/sierra_2]  [trial-820↪sms:sysmo↪mo_mt_sms.py:27]
05:30:25.582761 tst                  mo_mt_sms.py:27: Test FAILED (78.6 sec)
05:30:25.619388 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7222): Terminating (SIGTERM)
05:30:25.660928 run    osmo-hlr_10.42.42.2(pid=7223): Terminating (SIGTERM)
05:30:25.703883 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7224): Terminating (SIGTERM)
05:30:25.746746 run    osmo-stp_10.42.42.5(pid=7225): Terminating (SIGTERM)
05:30:25.816575 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7226): Terminating (SIGTERM)
05:30:25.861978 run    osmo-msc_10.42.42.6(pid=7227): Terminating (SIGTERM)
05:30:25.903275 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7228): Terminating (SIGTERM)
05:30:25.946691 run    osmo-mgw_10.42.42.3(pid=7229): Terminating (SIGTERM)
05:30:26.004865 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7230): Terminating (SIGTERM)
05:30:26.046984 run    osmo-mgw_10.42.42.4(pid=7231): Terminating (SIGTERM)
05:30:26.088228 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7232): Terminating (SIGTERM)
05:30:26.128591 run    osmo-bsc_10.42.42.7(pid=7233): Terminating (SIGTERM)
05:30:26.168426 run         osmo-bts-sysmo(pid=7245): Terminating (SIGTERM)
05:30:26.177716 ---      ParallelTerminationStrategy: PID 7222 died...
05:30:26.231731 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7222): Terminated: ok {rc=0}
05:30:26.241101 ---      ParallelTerminationStrategy: PID 7223 died...
05:30:26.294088 run    osmo-hlr_10.42.42.2(pid=7223): Terminated: ok {rc=0}
05:30:26.303665 ---      ParallelTerminationStrategy: PID 7224 died...
05:30:26.360463 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7224): Terminated: ok {rc=0}
05:30:26.369949 ---      ParallelTerminationStrategy: PID 7225 died...
05:30:26.423044 run    osmo-stp_10.42.42.5(pid=7225): Terminated: ok {rc=0}
05:30:26.432366 ---      ParallelTerminationStrategy: PID 7226 died...
05:30:26.485425 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7226): Terminated: ok {rc=0}
05:30:26.494903 ---      ParallelTerminationStrategy: PID 7227 died...
05:30:26.547682 run    osmo-msc_10.42.42.6(pid=7227): Terminated: ok {rc=0}
05:30:26.557366 ---      ParallelTerminationStrategy: PID 7228 died...
05:30:26.610256 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7228): Terminated: ok {rc=0}
05:30:26.619888 ---      ParallelTerminationStrategy: PID 7229 died...
05:30:26.672649 run    osmo-mgw_10.42.42.3(pid=7229): Terminated {rc=15}
05:30:26.682095 ---      ParallelTerminationStrategy: PID 7230 died...
05:30:26.735303 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7230): Terminated: ok {rc=0}
05:30:26.746877 ---      ParallelTerminationStrategy: PID 7231 died...
05:30:26.803580 run    osmo-mgw_10.42.42.4(pid=7231): Terminated {rc=15}
05:30:26.814035 ---      ParallelTerminationStrategy: PID 7232 died...
05:30:26.870914 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7232): Terminated: ok {rc=0}
05:30:26.880738 ---      ParallelTerminationStrategy: PID 7233 died...
05:30:26.934485 run    osmo-bsc_10.42.42.7(pid=7233): Terminated: ok {rc=0}
05:30:26.944213 ---      ParallelTerminationStrategy: PID 7245 died...
05:30:26.997862 run         osmo-bts-sysmo(pid=7245): Terminated: ok {rc=0}
05:30:27.100004 bus                        /sierra_2: Setting Online False
05:30:28.185014 bus                        /sierra_2: Setting Powered False
05:30:33.963782 bus                        /sierra_2: Setting Powered False
05:30:35.070532 bus                        /sierra_2: Setting Powered False
05:30:36.198403 bus                        /sierra_1: Setting Online False
05:30:37.284161 bus                        /sierra_1: Setting Powered False
05:30:42.962672 bus                        /sierra_1: Setting Powered False

Standard Error

Traceback (most recent call last):
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
    self.path)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
    spec.loader.exec_module( importlib.util.module_from_spec(spec) )
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/sms/mo_mt_sms.py", line 27, in <module>
    hlr.subscriber_add(ms_mt)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/hlr_osmo.py", line 121, in subscriber_add
    self.log('Add subscriber', msisdn=msisdn, imsi=modem.imsi(), subscriber_id=subscriber_id,
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 467, in imsi
    MainLoop.wait(lambda: simmgr.GetProperties().get('SubscriberIdentity', None) is not None, timeout=10)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait
    raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep)
osmo_gsm_tester.core.log.Error: /sierra_2: Wait timeout {condition=<function Modem.imsi.<locals>.<lambda> at 0x7fb3401eca60>, timeout=10, timestep=1} [trial-820↪sms:sysmo↪mo_mt_sms.py:27↪mo_mt_sms.py↪/sierra_2]