Stack Trace
osmo-bts-trx(pid=13066): Process ended prematurely: osmo-bts-trx(pid=13066) [trial-289↪sms:trx-b200↪mo_mt_sms.py:29↪mo_mt_sms.py↪osmo-bts-trx↪osmo-bts-trx(pid=13066)]Standard Output
----------------------------------------------
trial-289 sms:trx-b200 mo_mt_sms.py
----------------------------------------------
12:26:56.353190 tst sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:56.365881 tst sms:trx-b200: Using 1 x bts (candidates: 1)
12:26:56.927169 tst mo_mt_sms.py: using LAC 2829
12:26:57.040931 tst mo_mt_sms.py: using RAC 24
12:26:57.149532 tst mo_mt_sms.py: using CellId 2829
12:26:57.257991 tst mo_mt_sms.py: using BVCI 2830
12:26:57.279380 tst sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.290325 tst sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.301090 tst sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.312024 tst sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.322974 tst sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.333835 tst sms:trx-b200: Using 1 x modem (candidates: 2)
12:26:57.536233 tst sms:trx-b200: Using 1 x modem (candidates: 2)
12:26:57.754588 run osmo-hlr_10.42.42.2: Starting osmo-hlr
12:26:59.388062 run create_hlr_db(pid=13037): Launched
12:27:00.581782 run create_hlr_db(pid=13037): Terminated: ok {rc=0}
12:27:00.963935 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13039): Launched
12:27:01.331046 run osmo-hlr_10.42.42.2(pid=13040): Launched
12:27:01.435831 run osmo-stp_10.42.42.5: Starting osmo-stp
12:27:03.884661 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13041): Launched
12:27:04.256953 run osmo-stp_10.42.42.5(pid=13042): Launched
12:27:04.359525 run osmo-msc_10.42.42.6: Starting osmo-msc
12:27:09.105641 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13043): Launched
12:27:09.475018 run osmo-msc_10.42.42.6(pid=13044): Launched
12:27:12.652694 run osmo-mgw_10.42.42.3: Starting osmo-mgw
12:27:14.320020 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13045): Launched
12:27:14.690568 run osmo-mgw_10.42.42.3(pid=13046): Launched
12:27:14.794537 run osmo-mgw_10.42.42.4: Starting osmo-mgw
12:27:15.426586 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13047): Launched
12:27:15.794373 run osmo-mgw_10.42.42.4(pid=13048): Launched
12:27:15.897163 run osmo-bsc_10.42.42.7: Starting osmo-bsc
12:27:19.299333 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13049): Launched
12:27:19.672439 run osmo-bsc_10.42.42.7(pid=13050): Launched
12:27:26.002163 run osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
12:27:26.564134 run osmo-trx-uhd: OSMOTRX CONF: {'osmo_trx': {'clock_reference': 'external', 'launch_trx': 'True', 'type': 'uhd', 'bts_ip': '10.42.42.50', 'trx_ip': '10.42.42.116', 'egprs': 'disable', 'channels': [{}], 'dev_args': 'type=b200,serial=306BD11', 'multi_arfcn': 'True', 'remote_user': 'jenkins'}}
12:27:27.296218 run uname-m(pid=13051): Launched
12:27:28.517287 run uname-m(pid=13051): Terminated: ok {rc=0}
12:27:29.722557 run rm-remote-dir(pid=13052): Launched
12:27:30.947119 run rm-remote-dir(pid=13052): Terminated: ok {rc=0}
12:27:31.385992 run mk-remote-dir(pid=13053): Launched
12:27:32.609504 run mk-remote-dir(pid=13053): Terminated: ok {rc=0}
12:27:33.044498 run mk-remote-dir(pid=13054): Launched
12:27:34.265162 run mk-remote-dir(pid=13054): Terminated: ok {rc=0}
12:27:34.625214 run scp-inst-to-remote(pid=13055): Launched
12:27:36.833243 run scp-inst-to-remote(pid=13055): Terminated: ok {rc=0}
12:27:37.238570 run mk-remote-dir(pid=13057): Launched
12:27:38.444898 run mk-remote-dir(pid=13057): Terminated: ok {rc=0}
12:27:38.781336 run scp-cfg-to-remote(pid=13058): Launched
12:27:39.986709 run scp-cfg-to-remote(pid=13058): Terminated: ok {rc=0}
12:27:40.433256 run mk-remote-dir(pid=13062): Launched
12:27:41.646972 run mk-remote-dir(pid=13062): Terminated: ok {rc=0}
12:27:41.989546 run scp-wrapper-to-remote(pid=13063): Launched
12:27:43.198395 run scp-wrapper-to-remote(pid=13063): Terminated: ok {rc=0}
12:27:43.588269 run osmo-trx-uhd(pid=13065): Launched
12:27:43.691516 run osmo-bts-trx: Waiting for osmo-trx-uhd to start up...
12:27:47.364882 run osmo-bts-trx(pid=13066): Launched
12:27:48.755745 tst mo_mt_sms.py: using MSISDN 3278
12:27:48.960606 bus /sierra_2: Setting Powered True
12:27:52.628991 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='3278', subscriber_id=1}
12:27:52.817125 tst mo_mt_sms.py: using MSISDN 3279
12:27:53.019082 bus /sierra_3: Setting Powered True
12:27:57.037683 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='3279', subscriber_id=2}
12:27:57.402557 bus /sierra_2: Setting Online False
12:27:57.843963 run osmo-bts-trx(pid=13066): Terminated: ok {rc=0}
12:27:57.975673 run osmo-bts-trx(pid=13066): stdout:
| (launched: 2021-03-16_12:27:47.200955)
12:27:58.108160 run osmo-bts-trx(pid=13066): stderr:
| �[0;m�[1;36m20210316122757264 �[1;32mDOML�[0;m�[1;36m �[1;32mINFO�[0;m�[1;36m NM_CHAN_OP(bts0-trx0-ts7)[0x612000004420]{ENABLED}: state_chg to DISABLED_DEPENDENCY (nm_channel_fsm.c:141)
| �[0;m�[1;36m20210316122757264 �[1;32mDOML�[0;m�[1;36m �[1;32mINFO�[0;m�[1;36m OC=CHANNEL INST=(00,00,07) AVAIL STATE OK -> Dependency (oml.c:353)
| �[0;m�[1;36m20210316122757264 �[1;32mDOML�[0;m�[1;36m �[1;32mINFO�[0;m�[1;36m OC=CHANNEL INST=(00,00,07) OPER STATE Enabled -> Disabled (oml.c:360)
| �[0;m�[1;36m20210316122757264 �[1;34mDOML�[0;m�[1;36m �[1;34mDEBUG�[0;m�[1;36m OC=CHANNEL(03) INST=(00,00,07): Tx State Changed Event Report (oml.c:144)
| �[0;m�[1;36m20210316122757264 �[1;34mDOML�[0;m�[1;36m �[1;34mDEBUG�[0;m�[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: (bts=0,trx=0) Received TRX close cb rc=0 (bts_shutdown_fsm.c:242)
| �[0;m�[1;36m20210316122757264 �[1;32mDOML�[0;m�[1;36m �[1;32mINFO�[0;m�[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: Received Event BTS_SHUTDOWN_ST_WAIT_TRX_CLOSED (bts_shutdown_fsm.c:243)
| �[0;m�[1;36m20210316122757264 �[1;32mDOML�[0;m�[1;36m �[1;32mINFO�[0;m�[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: (bts=0,trx=0) TRX closed, 0 TRX remaining (bts_shutdown_fsm.c:142)
| �[0;m�[1;36m20210316122757264 �[1;32mDOML�[0;m�[1;36m �[1;32mINFO�[0;m�[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: state_chg to EXIT (bts_shutdown_fsm.c:144)
| �[0;m�[1;36m20210316122757264 �[1;33mDOML�[0;m�[1;36m �[1;33mNOTICE�[0;m�[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{EXIT}: Shutdown process completed successfuly, exiting process (bts_shutdown_fsm.c:151)
| �[0;m
12:27:58.165799 tst mo_mt_sms.py:29: ERR: Error: osmo-bts-trx(pid=13066): Process ended prematurely: osmo-bts-trx(pid=13066) [trial-289↪sms:trx-b200↪mo_mt_sms.py:29↪mo_mt_sms.py↪osmo-bts-trx↪osmo-bts-trx(pid=13066)] [trial-289↪sms:trx-b200↪mo_mt_sms.py:29]
12:27:58.176793 tst mo_mt_sms.py:29: Test FAILED (61.8 sec)
12:27:58.215445 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13039): Terminating (SIGTERM)
12:27:58.257168 run osmo-hlr_10.42.42.2(pid=13040): Terminating (SIGTERM)
12:27:58.300303 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13041): Terminating (SIGTERM)
12:27:58.344512 run osmo-stp_10.42.42.5(pid=13042): Terminating (SIGTERM)
12:27:58.395960 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13043): Terminating (SIGTERM)
12:27:58.439525 run osmo-msc_10.42.42.6(pid=13044): Terminating (SIGTERM)
12:27:58.484009 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13045): Terminating (SIGTERM)
12:27:58.530072 run osmo-mgw_10.42.42.3(pid=13046): Terminating (SIGTERM)
12:27:58.583150 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13047): Terminating (SIGTERM)
12:27:58.623518 run osmo-mgw_10.42.42.4(pid=13048): Terminating (SIGTERM)
12:27:58.678807 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13049): Terminating (SIGTERM)
12:27:58.721237 run osmo-bsc_10.42.42.7(pid=13050): Terminating (SIGTERM)
12:27:58.766579 run osmo-trx-uhd(pid=13065): Terminating (SIGTERM)
12:27:58.777983 --- ParallelTerminationStrategy: PID 13039 died...
12:27:58.847538 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13039): Terminated: ok {rc=0}
12:27:58.858414 --- ParallelTerminationStrategy: PID 13040 died...
12:27:58.927367 run osmo-hlr_10.42.42.2(pid=13040): Terminated: ok {rc=0}
12:27:58.938039 --- ParallelTerminationStrategy: PID 13041 died...
12:27:59.005712 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13041): Terminated: ok {rc=0}
12:27:59.017402 --- ParallelTerminationStrategy: PID 13043 died...
12:27:59.086058 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13043): Terminated: ok {rc=0}
12:27:59.096866 --- ParallelTerminationStrategy: PID 13045 died...
12:27:59.166105 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13045): Terminated: ok {rc=0}
12:27:59.176865 --- ParallelTerminationStrategy: PID 13046 died...
12:27:59.245809 run osmo-mgw_10.42.42.3(pid=13046): Terminated {rc=15}
12:27:59.256835 --- ParallelTerminationStrategy: PID 13047 died...
12:27:59.323138 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13047): Terminated: ok {rc=0}
12:27:59.333515 --- ParallelTerminationStrategy: PID 13042 died...
12:27:59.398658 run osmo-stp_10.42.42.5(pid=13042): Terminated {rc=256}
12:27:59.408713 --- ParallelTerminationStrategy: PID 13048 died...
12:27:59.474280 run osmo-mgw_10.42.42.4(pid=13048): Terminated {rc=15}
12:27:59.484296 --- ParallelTerminationStrategy: PID 13049 died...
12:27:59.549174 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13049): Terminated: ok {rc=0}
12:27:59.559100 --- ParallelTerminationStrategy: PID 13065 died...
12:27:59.623696 run osmo-trx-uhd(pid=13065): Terminated: ok {rc=0}
12:28:01.650299 --- ParallelTerminationStrategy: PID 13044 died...
12:28:01.716297 run osmo-msc_10.42.42.6(pid=13044): Terminated: ok {rc=0}
12:28:02.393216 --- ParallelTerminationStrategy: PID 13050 died...
12:28:02.459926 run osmo-bsc_10.42.42.7(pid=13050): Terminated: ok {rc=0}
12:28:02.589880 bus /sierra_3: Setting Online False
12:28:03.688778 bus /sierra_3: Setting Powered False
12:28:09.761083 bus /sierra_3: Setting Powered False
12:28:10.891695 bus /sierra_3: Setting Powered False
12:28:12.049353 bus /sierra_2: Setting Online False
12:28:13.149933 bus /sierra_2: Setting Powered False
12:28:18.765446 bus /sierra_2: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/sysmocom/suites/sms/mo_mt_sms.py", line 29, in <module>
ms_mo.connect(msc.mcc_mnc())
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 629, in connect
self.power_cycle()
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 616, in power_cycle
self.power_off()
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 605, in power_off
self.set_online(False)
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 450, in set_online
return self.dbus.set_online(online=online)
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 326, in set_online
self.set_bool('Online', online)
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 320, in set_bool
MainLoop.wait(self.property_is, name, bool_val)
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_manual-run/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-bts-trx(pid=13066): Process ended prematurely: osmo-bts-trx(pid=13066) [trial-289↪sms:trx-b200↪mo_mt_sms.py:29↪mo_mt_sms.py↪osmo-bts-trx↪osmo-bts-trx(pid=13066)]