Failed

ussd:trx-b200.assert_extension.py (from ussd_trx-b200)

Failing for the past 67 builds (Since #749 )

Took 21 sec.

Stacktrace

uname-m(pid=22732): launch_sync(): local ssh process exited with status 255 [trial-815↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=22732)]

Standard Output

----------------------------------------------
trial-815 ussd:trx-b200 assert_extension.py
----------------------------------------------
23:14:45.380347 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
23:14:45.391448 tst                    ussd:trx-b200: Using 1 x bts (candidates: 1)
23:14:45.527023 tst              assert_extension.py: using LAC 8213
23:14:45.655321 tst              assert_extension.py: using RAC 53
23:14:45.783196 tst              assert_extension.py: using CellId 8213
23:14:45.907898 tst              assert_extension.py: using BVCI 8209
23:14:45.931633 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
23:14:45.942331 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
23:14:45.952829 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
23:14:45.963703 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
23:14:45.974606 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
23:14:45.985391 tst                    ussd:trx-b200: Using 1 x modem (candidates: 1)
23:14:46.182345 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
23:14:46.714059 run         create_hlr_db(pid=22715): Launched
23:14:46.907165 bus                        /sierra_1: Setting Powered False
23:14:47.926976 run         create_hlr_db(pid=22715): Terminated: ok {rc=0}
23:14:48.347769 run pcap-recorder_any(filters='host 10.42.42.2')(pid=22717): Launched
23:14:48.665944 run   osmo-hlr_10.42.42.2(pid=22718): Launched
23:14:48.777038 run              osmo-stp_10.42.42.5: Starting osmo-stp
23:14:49.318706 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=22719): Launched
23:14:49.634109 run   osmo-stp_10.42.42.5(pid=22720): Launched
23:14:49.745493 run              osmo-msc_10.42.42.6: Starting osmo-msc
23:14:50.314664 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=22721): Launched
23:14:50.630871 run   osmo-msc_10.42.42.6(pid=22722): Launched
23:14:53.805787 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
23:14:54.380811 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=22723): Launched
23:14:54.715809 run   osmo-mgw_10.42.42.3(pid=22724): Launched
23:14:54.833995 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
23:14:55.405071 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=22725): Launched
23:14:55.742563 run   osmo-mgw_10.42.42.4(pid=22726): Launched
23:14:55.862072 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
23:14:56.557073 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=22727): Launched
23:14:56.892453 run   osmo-bsc_10.42.42.7(pid=22728): Launched
23:15:03.168439 run                     osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
23:15:03.623444 run                     osmo-trx-uhd: OSMOTRX CONF: {'osmo_trx': {'clock_reference': 'external', 'launch_trx': 'True', 'type': 'uhd', 'bts_ip': '10.42.42.50', 'trx_ip': '127.0.0.1', 'egprs': 'disable', 'channels': [{}], 'dev_args': 'type=b200,serial=30A9FFB', 'multi_arfcn': 'True', 'remote_user': 'jenkins'}}
23:15:04.200641 run               uname-m(pid=22732): Launched
23:15:05.395305 run               uname-m(pid=22732): ERR: Terminated: ERROR {rc=255}  [trial-815↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=22732)]
23:15:05.534954 run               uname-m(pid=22732): stdout: 
| (launched: 2022-12-04_23:15:04.033658) 
23:15:05.673964 run               uname-m(pid=22732): stderr: 
| (launched: 2022-12-04_23:15:04.083799)
| Host key verification failed. 
23:15:05.731252 tst           assert_extension.py:24: ERR: Error: uname-m(pid=22732): launch_sync(): local ssh process exited with status 255 [trial-815↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=22732)]  [trial-815↪ussd:trx-b200↪assert_extension.py:24]
23:15:05.738617 tst           assert_extension.py:24: Test FAILED (20.3 sec)
23:15:05.774313 run pcap-recorder_any(filters='host 10.42.42.2')(pid=22717): Terminating (SIGTERM)
23:15:05.814471 run   osmo-hlr_10.42.42.2(pid=22718): Terminating (SIGTERM)
23:15:05.856813 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=22719): Terminating (SIGTERM)
23:15:05.901013 run   osmo-stp_10.42.42.5(pid=22720): Terminating (SIGTERM)
23:15:05.942571 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=22721): Terminating (SIGTERM)
23:15:05.987190 run   osmo-msc_10.42.42.6(pid=22722): Terminating (SIGTERM)
23:15:06.027680 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=22723): Terminating (SIGTERM)
23:15:06.072022 run   osmo-mgw_10.42.42.3(pid=22724): Terminating (SIGTERM)
23:15:06.113585 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=22725): Terminating (SIGTERM)
23:15:06.156069 run   osmo-mgw_10.42.42.4(pid=22726): Terminating (SIGTERM)
23:15:06.199116 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=22727): Terminating (SIGTERM)
23:15:06.238623 run   osmo-bsc_10.42.42.7(pid=22728): Terminating (SIGTERM)
23:15:06.247957 ---      ParallelTerminationStrategy: PID 22717 died...
23:15:06.300997 run pcap-recorder_any(filters='host 10.42.42.2')(pid=22717): Terminated: ok {rc=0}
23:15:06.310710 ---      ParallelTerminationStrategy: PID 22718 died...
23:15:06.363319 run   osmo-hlr_10.42.42.2(pid=22718): Terminated: ok {rc=0}
23:15:06.373314 ---      ParallelTerminationStrategy: PID 22719 died...
23:15:06.426497 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=22719): Terminated: ok {rc=0}
23:15:06.436018 ---      ParallelTerminationStrategy: PID 22720 died...
23:15:06.488739 run   osmo-stp_10.42.42.5(pid=22720): Terminated: ok {rc=0}
23:15:06.498196 ---      ParallelTerminationStrategy: PID 22721 died...
23:15:06.550909 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=22721): Terminated: ok {rc=0}
23:15:06.560585 ---      ParallelTerminationStrategy: PID 22722 died...
23:15:06.613275 run   osmo-msc_10.42.42.6(pid=22722): Terminated: ok {rc=0}
23:15:06.622983 ---      ParallelTerminationStrategy: PID 22723 died...
23:15:06.675637 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=22723): Terminated: ok {rc=0}
23:15:06.685382 ---      ParallelTerminationStrategy: PID 22724 died...
23:15:06.738752 run   osmo-mgw_10.42.42.3(pid=22724): Terminated {rc=15}
23:15:06.748309 ---      ParallelTerminationStrategy: PID 22725 died...
23:15:06.801025 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=22725): Terminated: ok {rc=0}
23:15:06.810747 ---      ParallelTerminationStrategy: PID 22726 died...
23:15:06.863983 run   osmo-mgw_10.42.42.4(pid=22726): Terminated {rc=15}
23:15:06.873622 ---      ParallelTerminationStrategy: PID 22727 died...
23:15:06.926687 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=22727): Terminated: ok {rc=0}
23:15:07.411466 ---      ParallelTerminationStrategy: PID 22728 died...
23:15:07.468914 run   osmo-bsc_10.42.42.7(pid=22728): Terminated: ok {rc=0}
23:15:07.556620 bus                        /sierra_1: Setting Powered False

Standard Error

Traceback (most recent call last):
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
    self.path)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
    spec.loader.exec_module( importlib.util.module_from_spec(spec) )
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/ussd/assert_extension.py", line 24, in <module>
    bts.start()
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_osmotrx.py", line 174, in start
    self.trx.start(keepalive)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_osmotrx.py", line 327, in start
    self.start_remotely(keepalive)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_osmotrx.py", line 293, in start_remotely
    have_inst = rem_host.inst_compatible_for_remote()
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 150, in inst_compatible_for_remote
    proc = self.run_remote_sync('uname-m', ('uname', '-m'))
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync
    proc.launch_sync()
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync
    raise self.RunError('launch_sync()')
osmo_gsm_tester.core.log.Error: uname-m(pid=22732): launch_sync(): local ssh process exited with status 255 [trial-815↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=22732)]