Failed

ussd:trx-b200.assert_extension.py (from ussd_trx-b200)

Failing for the past 96 builds (Since #749 )

Took 21 sec.

Stacktrace

uname-m(pid=4262): launch_sync(): local ssh process exited with status 255 [trial-844↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=4262)]

Standard Output

----------------------------------------------
trial-844 ussd:trx-b200 assert_extension.py
----------------------------------------------
09:57:11.247356 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
09:57:11.259088 tst                    ussd:trx-b200: Using 1 x bts (candidates: 1)
09:57:11.401234 tst              assert_extension.py: using LAC 10533
09:57:11.529957 tst              assert_extension.py: using RAC 78
09:57:11.655784 tst              assert_extension.py: using CellId 10533
09:57:11.782536 tst              assert_extension.py: using BVCI 10529
09:57:11.807195 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
09:57:11.818374 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
09:57:11.829453 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
09:57:11.840432 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
09:57:11.851445 tst                    ussd:trx-b200: Using 1 x ip_address (candidates: 6)
09:57:11.862435 tst                    ussd:trx-b200: Using 1 x modem (candidates: 1)
09:57:12.058717 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
09:57:12.577847 run          create_hlr_db(pid=4248): Launched
09:57:12.766558 bus                        /sierra_1: Setting Powered False
09:57:13.787272 run          create_hlr_db(pid=4248): Terminated: ok {rc=0}
09:57:14.197640 run pcap-recorder_any(filters='host 10.42.42.2')(pid=4250): Launched
09:57:14.505239 run    osmo-hlr_10.42.42.2(pid=4251): Launched
09:57:14.615741 run              osmo-stp_10.42.42.5: Starting osmo-stp
09:57:15.147050 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=4252): Launched
09:57:15.452605 run    osmo-stp_10.42.42.5(pid=4253): Launched
09:57:15.560674 run              osmo-msc_10.42.42.6: Starting osmo-msc
09:57:16.119215 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=4254): Launched
09:57:16.431947 run    osmo-msc_10.42.42.6(pid=4255): Launched
09:57:19.595267 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
09:57:20.126314 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=4256): Launched
09:57:20.434726 run    osmo-mgw_10.42.42.3(pid=4257): Launched
09:57:20.543683 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
09:57:21.064628 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=4258): Launched
09:57:21.374271 run    osmo-mgw_10.42.42.4(pid=4259): Launched
09:57:21.482944 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
09:57:22.125101 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=4260): Launched
09:57:22.433954 run    osmo-bsc_10.42.42.7(pid=4261): Launched
09:57:28.697583 run                     osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
09:57:29.139027 run                     osmo-trx-uhd: OSMOTRX CONF: {'osmo_trx': {'clock_reference': 'external', 'launch_trx': 'True', 'type': 'uhd', 'bts_ip': '10.42.42.50', 'trx_ip': '127.0.0.1', 'egprs': 'disable', 'channels': [{}], 'dev_args': 'type=b200,serial=30A9FFB', 'multi_arfcn': 'True', 'remote_user': 'jenkins'}}
09:57:29.720256 run                uname-m(pid=4262): Launched
09:57:30.923406 run                uname-m(pid=4262): ERR: Terminated: ERROR {rc=255}  [trial-844↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=4262)]
09:57:31.068513 run                uname-m(pid=4262): stdout: 
| (launched: 2022-12-06_09:57:29.539147) 
09:57:31.214946 run                uname-m(pid=4262): stderr: 
| (launched: 2022-12-06_09:57:29.588019)
| Host key verification failed. 
09:57:31.274979 tst           assert_extension.py:24: ERR: Error: uname-m(pid=4262): launch_sync(): local ssh process exited with status 255 [trial-844↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=4262)]  [trial-844↪ussd:trx-b200↪assert_extension.py:24]
09:57:31.283032 tst           assert_extension.py:24: Test FAILED (20.0 sec)
09:57:31.319580 run pcap-recorder_any(filters='host 10.42.42.2')(pid=4250): Terminating (SIGTERM)
09:57:31.361330 run    osmo-hlr_10.42.42.2(pid=4251): Terminating (SIGTERM)
09:57:31.404229 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=4252): Terminating (SIGTERM)
09:57:31.452197 run    osmo-stp_10.42.42.5(pid=4253): Terminating (SIGTERM)
09:57:31.502250 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=4254): Terminating (SIGTERM)
09:57:31.548202 run    osmo-msc_10.42.42.6(pid=4255): Terminating (SIGTERM)
09:57:31.592300 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=4256): Terminating (SIGTERM)
09:57:31.639079 run    osmo-mgw_10.42.42.3(pid=4257): Terminating (SIGTERM)
09:57:31.682728 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=4258): Terminating (SIGTERM)
09:57:31.725076 run    osmo-mgw_10.42.42.4(pid=4259): Terminating (SIGTERM)
09:57:31.768523 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=4260): Terminating (SIGTERM)
09:57:31.810775 run    osmo-bsc_10.42.42.7(pid=4261): Terminating (SIGTERM)
09:57:31.820054 ---      ParallelTerminationStrategy: PID 4250 died...
09:57:31.873793 run pcap-recorder_any(filters='host 10.42.42.2')(pid=4250): Terminated: ok {rc=0}
09:57:31.883484 ---      ParallelTerminationStrategy: PID 4251 died...
09:57:31.936653 run    osmo-hlr_10.42.42.2(pid=4251): Terminated: ok {rc=0}
09:57:31.946450 ---      ParallelTerminationStrategy: PID 4252 died...
09:57:31.999534 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=4252): Terminated: ok {rc=0}
09:57:32.009347 ---      ParallelTerminationStrategy: PID 4253 died...
09:57:32.062475 run    osmo-stp_10.42.42.5(pid=4253): Terminated: ok {rc=0}
09:57:32.072191 ---      ParallelTerminationStrategy: PID 4254 died...
09:57:32.125892 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=4254): Terminated: ok {rc=0}
09:57:32.135449 ---      ParallelTerminationStrategy: PID 4255 died...
09:57:32.188445 run    osmo-msc_10.42.42.6(pid=4255): Terminated: ok {rc=0}
09:57:32.198298 ---      ParallelTerminationStrategy: PID 4256 died...
09:57:32.251471 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=4256): Terminated: ok {rc=0}
09:57:32.261322 ---      ParallelTerminationStrategy: PID 4257 died...
09:57:32.315537 run    osmo-mgw_10.42.42.3(pid=4257): Terminated {rc=15}
09:57:32.325257 ---      ParallelTerminationStrategy: PID 4258 died...
09:57:32.378510 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=4258): Terminated: ok {rc=0}
09:57:32.388436 ---      ParallelTerminationStrategy: PID 4259 died...
09:57:32.441710 run    osmo-mgw_10.42.42.4(pid=4259): Terminated {rc=15}
09:57:32.451531 ---      ParallelTerminationStrategy: PID 4260 died...
09:57:32.506414 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=4260): Terminated: ok {rc=0}
09:57:32.684681 ---      ParallelTerminationStrategy: PID 4261 died...
09:57:32.739358 run    osmo-bsc_10.42.42.7(pid=4261): Terminated: ok {rc=0}
09:57:32.828253 bus                        /sierra_1: Setting Powered False

Standard Error

Traceback (most recent call last):
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
    self.path)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
    spec.loader.exec_module( importlib.util.module_from_spec(spec) )
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/sysmocom/suites/ussd/assert_extension.py", line 24, in <module>
    bts.start()
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_osmotrx.py", line 174, in start
    self.trx.start(keepalive)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_osmotrx.py", line 327, in start
    self.start_remotely(keepalive)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_osmotrx.py", line 293, in start_remotely
    have_inst = rem_host.inst_compatible_for_remote()
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 150, in inst_compatible_for_remote
    proc = self.run_remote_sync('uname-m', ('uname', '-m'))
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync
    proc.launch_sync()
  File "/home/jenkins/workspace/osmo-gsm-tester_run-rnd/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync
    raise self.RunError('launch_sync()')
osmo_gsm_tester.core.log.Error: uname-m(pid=4262): launch_sync(): local ssh process exited with status 255 [trial-844↪ussd:trx-b200↪assert_extension.py:24↪assert_extension.py↪osmo-bts-trx↪osmo-trx-uhd↪host-jenkins@127.0.0.1↪uname-m(pid=4262)]