
--------------------------------------------------------------------------------------------
trial-289
--------------------------------------------------------------------------------------------
12:26:54.435212 tst                        trial-289: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_manual-run/trial-289/run.2021-03-16_12-26-54/log

---------------------------------------------------------------------
trial-289 sms:trx-b200
---------------------------------------------------------------------
12:26:55.617229 tst                     sms:trx-b200: reserving resources in /var/tmp/osmo-gsm-tester/state ...
12:26:55.622466 tst                     sms:trx-b200: Verifying 1 x bts (candidates: 11)
12:26:55.658330 tst                     sms:trx-b200: Verifying 6 x ip_address (candidates: 9)
12:26:55.688724 tst                     sms:trx-b200: Verifying 2 x modem (candidates: 12)
12:26:55.746609 tst                     sms:trx-b200: Reserving 1 x bts (candidates: 11)
12:26:55.779079 tst                     sms:trx-b200: Reserving 6 x ip_address (candidates: 9)
12:26:55.805362 tst                     sms:trx-b200: Reserving 2 x modem (candidates: 12)
12:26:56.344870 tst                     sms:trx-b200: RESERVED RESOURCES for sms:
bts
  Ettus B200
ip_address
  10.42.42.2
  10.42.42.3
  10.42.42.4
  10.42.42.5
  10.42.42.6
  10.42.42.7
modem
  sierra_1st
  sierra_2nd

----------------------------------------------
trial-289 sms:trx-b200 mo_mt_sms.py
----------------------------------------------
12:26:56.352913 tst                     sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:56.365588 tst                     sms:trx-b200: Using 1 x bts (candidates: 1)
12:26:56.887440 tst                     mo_mt_sms.py: using LAC 2829
12:26:57.001190 tst                     mo_mt_sms.py: using RAC 24
12:26:57.109796 tst                     mo_mt_sms.py: using CellId 2829
12:26:57.218082 tst                     mo_mt_sms.py: using BVCI 2830
12:26:57.279102 tst                     sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.289957 tst                     sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.300827 tst                     sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.311758 tst                     sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.322710 tst                     sms:trx-b200: Using 1 x ip_address (candidates: 6)
12:26:57.333569 tst                     sms:trx-b200: Using 1 x modem (candidates: 2)
12:26:57.535957 tst                     sms:trx-b200: Using 1 x modem (candidates: 2)
12:26:57.714057 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
12:26:59.343715 run         create_hlr_db(pid=13037): Launched
12:27:00.534261 run         create_hlr_db(pid=13037): Terminated: ok {rc=0}
12:27:00.921562 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13039): Launched
12:27:01.288660 run   osmo-hlr_10.42.42.2(pid=13040): Launched
12:27:01.394988 run              osmo-stp_10.42.42.5: Starting osmo-stp
12:27:03.842564 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13041): Launched
12:27:04.214907 run   osmo-stp_10.42.42.5(pid=13042): Launched
12:27:04.319283 run              osmo-msc_10.42.42.6: Starting osmo-msc
12:27:09.063362 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13043): Launched
12:27:09.432302 run   osmo-msc_10.42.42.6(pid=13044): Launched
12:27:12.612661 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
12:27:14.277806 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13045): Launched
12:27:14.647816 run   osmo-mgw_10.42.42.3(pid=13046): Launched
12:27:14.754578 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
12:27:15.384083 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13047): Launched
12:27:15.751833 run   osmo-mgw_10.42.42.4(pid=13048): Launched
12:27:15.856789 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
12:27:19.256545 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13049): Launched
12:27:19.629802 run   osmo-bsc_10.42.42.7(pid=13050): Launched
12:27:25.958157 run                     osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
12:27:26.518282 run                     osmo-trx-uhd: OSMOTRX CONF: {'osmo_trx': {'clock_reference': 'external', 'launch_trx': 'True', 'type': 'uhd', 'bts_ip': '10.42.42.50', 'trx_ip': '10.42.42.116', 'egprs': 'disable', 'channels': [{}], 'dev_args': 'type=b200,serial=306BD11', 'multi_arfcn': 'True', 'remote_user': 'jenkins'}}
12:27:27.248268 run               uname-m(pid=13051): Launched
12:27:28.463135 run               uname-m(pid=13051): Terminated: ok {rc=0}
12:27:29.670529 run         rm-remote-dir(pid=13052): Launched
12:27:30.891884 run         rm-remote-dir(pid=13052): Terminated: ok {rc=0}
12:27:31.337072 run         mk-remote-dir(pid=13053): Launched
12:27:32.554520 run         mk-remote-dir(pid=13053): Terminated: ok {rc=0}
12:27:32.995889 run         mk-remote-dir(pid=13054): Launched
12:27:34.211093 run         mk-remote-dir(pid=13054): Terminated: ok {rc=0}
12:27:34.577197 run    scp-inst-to-remote(pid=13055): Launched
12:27:36.782489 run    scp-inst-to-remote(pid=13055): Terminated: ok {rc=0}
12:27:37.192627 run         mk-remote-dir(pid=13057): Launched
12:27:38.394037 run         mk-remote-dir(pid=13057): Terminated: ok {rc=0}
12:27:38.735626 run     scp-cfg-to-remote(pid=13058): Launched
12:27:39.936063 run     scp-cfg-to-remote(pid=13058): Terminated: ok {rc=0}
12:27:40.373777 run         mk-remote-dir(pid=13062): Launched
12:27:41.595551 run         mk-remote-dir(pid=13062): Terminated: ok {rc=0}
12:27:41.943452 run scp-wrapper-to-remote(pid=13063): Launched
12:27:43.146786 run scp-wrapper-to-remote(pid=13063): Terminated: ok {rc=0}
12:27:43.545016 run          osmo-trx-uhd(pid=13065): Launched
12:27:43.650636 run                     osmo-bts-trx: Waiting for osmo-trx-uhd to start up...
12:27:47.321622 run          osmo-bts-trx(pid=13066): Launched
12:27:48.708717 tst                     mo_mt_sms.py: using MSISDN 3278
12:27:48.911937 bus                        /sierra_2: Setting Powered True
12:27:52.583051 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='3278', subscriber_id=1}
12:27:52.769479 tst                     mo_mt_sms.py: using MSISDN 3279
12:27:52.969817 bus                        /sierra_3: Setting Powered True
12:27:56.992022 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='3279', subscriber_id=2}
12:27:57.352611 bus                        /sierra_2: Setting Online False
12:27:57.790125 run          osmo-bts-trx(pid=13066): Terminated: ok {rc=0}
12:27:57.923764 run          osmo-bts-trx(pid=13066): stdout: 
| (launched: 2021-03-16_12:27:47.200955) 
12:27:58.055980 run          osmo-bts-trx(pid=13066): stderr: 
| [0;m[1;36m20210316122757264 [1;32mDOML[0;m[1;36m [1;32mINFO[0;m[1;36m NM_CHAN_OP(bts0-trx0-ts7)[0x612000004420]{ENABLED}: state_chg to DISABLED_DEPENDENCY (nm_channel_fsm.c:141)
| [0;m[1;36m20210316122757264 [1;32mDOML[0;m[1;36m [1;32mINFO[0;m[1;36m OC=CHANNEL INST=(00,00,07) AVAIL STATE OK -> Dependency (oml.c:353)
| [0;m[1;36m20210316122757264 [1;32mDOML[0;m[1;36m [1;32mINFO[0;m[1;36m OC=CHANNEL INST=(00,00,07) OPER STATE Enabled -> Disabled (oml.c:360)
| [0;m[1;36m20210316122757264 [1;34mDOML[0;m[1;36m [1;34mDEBUG[0;m[1;36m OC=CHANNEL(03) INST=(00,00,07): Tx State Changed Event Report (oml.c:144)
| [0;m[1;36m20210316122757264 [1;34mDOML[0;m[1;36m [1;34mDEBUG[0;m[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: (bts=0,trx=0) Received TRX close cb rc=0 (bts_shutdown_fsm.c:242)
| [0;m[1;36m20210316122757264 [1;32mDOML[0;m[1;36m [1;32mINFO[0;m[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: Received Event BTS_SHUTDOWN_ST_WAIT_TRX_CLOSED (bts_shutdown_fsm.c:243)
| [0;m[1;36m20210316122757264 [1;32mDOML[0;m[1;36m [1;32mINFO[0;m[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: (bts=0,trx=0) TRX closed, 0 TRX remaining (bts_shutdown_fsm.c:142)
| [0;m[1;36m20210316122757264 [1;32mDOML[0;m[1;36m [1;32mINFO[0;m[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{WAIT_TRX_CLOSED}: state_chg to EXIT (bts_shutdown_fsm.c:144)
| [0;m[1;36m20210316122757264 [1;33mDOML[0;m[1;36m [1;33mNOTICE[0;m[1;36m BTS_SHUTDOWN(bts0)[0x612000003220]{EXIT}: Shutdown process completed successfuly, exiting process (bts_shutdown_fsm.c:151)
| [0;m 
12:27:58.163342 tst                  mo_mt_sms.py:29: ERR: Error: osmo-bts-trx(pid=13066): Process ended prematurely: osmo-bts-trx(pid=13066) [trial-289↪sms:trx-b200↪mo_mt_sms.py:29↪mo_mt_sms.py↪osmo-bts-trx↪osmo-bts-trx(pid=13066)]  [trial-289↪sms:trx-b200↪mo_mt_sms.py:29]
12:27:58.174423 tst                  mo_mt_sms.py:29: Test FAILED (61.8 sec)
12:27:58.198079 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13039): Terminating (SIGTERM)
12:27:58.240534 run   osmo-hlr_10.42.42.2(pid=13040): Terminating (SIGTERM)
12:27:58.283557 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13041): Terminating (SIGTERM)
12:27:58.327035 run   osmo-stp_10.42.42.5(pid=13042): Terminating (SIGTERM)
12:27:58.377974 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13043): Terminating (SIGTERM)
12:27:58.421828 run   osmo-msc_10.42.42.6(pid=13044): Terminating (SIGTERM)
12:27:58.466181 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13045): Terminating (SIGTERM)
12:27:58.511050 run   osmo-mgw_10.42.42.3(pid=13046): Terminating (SIGTERM)
12:27:58.567121 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13047): Terminating (SIGTERM)
12:27:58.607649 run   osmo-mgw_10.42.42.4(pid=13048): Terminating (SIGTERM)
12:27:58.660850 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13049): Terminating (SIGTERM)
12:27:58.704663 run   osmo-bsc_10.42.42.7(pid=13050): Terminating (SIGTERM)
12:27:58.748844 run          osmo-trx-uhd(pid=13065): Terminating (SIGTERM)
12:27:58.777727 ---      ParallelTerminationStrategy: PID 13039 died...
12:27:58.828252 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13039): Terminated: ok {rc=0}
12:27:58.858027 ---      ParallelTerminationStrategy: PID 13040 died...
12:27:58.908202 run   osmo-hlr_10.42.42.2(pid=13040): Terminated: ok {rc=0}
12:27:58.937793 ---      ParallelTerminationStrategy: PID 13041 died...
12:27:58.987862 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13041): Terminated: ok {rc=0}
12:27:59.017152 ---      ParallelTerminationStrategy: PID 13043 died...
12:27:59.066957 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=13043): Terminated: ok {rc=0}
12:27:59.096613 ---      ParallelTerminationStrategy: PID 13045 died...
12:27:59.146830 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13045): Terminated: ok {rc=0}
12:27:59.176616 ---      ParallelTerminationStrategy: PID 13046 died...
12:27:59.226698 run   osmo-mgw_10.42.42.3(pid=13046): Terminated {rc=15}
12:27:59.256585 ---      ParallelTerminationStrategy: PID 13047 died...
12:27:59.304715 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13047): Terminated: ok {rc=0}
12:27:59.333235 ---      ParallelTerminationStrategy: PID 13042 died...
12:27:59.380387 run   osmo-stp_10.42.42.5(pid=13042): Terminated {rc=256}
12:27:59.408325 ---      ParallelTerminationStrategy: PID 13048 died...
12:27:59.455916 run   osmo-mgw_10.42.42.4(pid=13048): Terminated {rc=15}
12:27:59.483929 ---      ParallelTerminationStrategy: PID 13049 died...
12:27:59.530978 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=13049): Terminated: ok {rc=0}
12:27:59.558858 ---      ParallelTerminationStrategy: PID 13065 died...
12:27:59.605349 run          osmo-trx-uhd(pid=13065): Terminated: ok {rc=0}
12:28:01.649896 ---      ParallelTerminationStrategy: PID 13044 died...
12:28:01.698481 run   osmo-msc_10.42.42.6(pid=13044): Terminated: ok {rc=0}
12:28:02.392907 ---      ParallelTerminationStrategy: PID 13050 died...
12:28:02.441568 run   osmo-bsc_10.42.42.7(pid=13050): Terminated: ok {rc=0}
12:28:02.571547 bus                        /sierra_3: Setting Online False
12:28:03.669518 bus                        /sierra_3: Setting Powered False
12:28:09.744124 bus                        /sierra_3: Setting Powered False
12:28:10.870250 bus                        /sierra_3: Setting Powered False
12:28:12.031244 bus                        /sierra_2: Setting Online False
12:28:13.130786 bus                        /sierra_2: Setting Powered False
12:28:18.748278 bus                        /sierra_2: Setting Powered False
---------------------------------------------------------------------
trial-289 sms:trx-b200 FAIL
---------------------------------------------------------------------
12:28:20.054147 tst                        trial-289: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_manual-run/trial-289/run.2021-03-16_12-26-54/trial-289.xml

--------------------------------------------------------------------------------------------
trial-289 FAIL
--------------------------------------------------------------------------------------------
12:28:20.169731 tst                        trial-289: trial-289: FAIL, 1 suites failed
FAIL: sms:trx-b200 (fail: 1)
    FAIL: mo_mt_sms.py (61.8 sec) Error: osmo-bts-trx(pid=13066): Process ended prematurely: osmo-bts-trx(pid=13066) [trial-289↪sms:trx-b200↪mo_mt_sms.py:29↪mo_mt_sms.py↪osmo-bts-trx↪osmo-bts-trx(pid=13066)]
