rm-remote-dir(pid=9452): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)]

Standard Output

----------------------------------------------
trial-2069 gprs:sysmo cs_paging_gprs_active.py
----------------------------------------------
18:28:20.066730 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.079034 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
18:28:20.221742 tst         cs_paging_gprs_active.py: using LAC 41702
18:28:20.353850 tst         cs_paging_gprs_active.py: using RAC 137
18:28:20.484016 tst         cs_paging_gprs_active.py: using CellId 41702
18:28:20.618593 tst         cs_paging_gprs_active.py: using BVCI 41703
18:28:20.644751 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.659437 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.671765 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.685149 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.698529 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.715534 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.734351 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.747863 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:28:20.766355 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:28:20.936510 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:28:21.260998 tst      cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003...
18:28:21.432801 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:28:21.829316 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9407): Launched
18:28:22.123852 run iperf3-srv_10.42.42.10(pid=9408): Launched
18:28:22.318730 tst      cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004...
18:28:22.464680 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:28:22.859684 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9409): Launched
18:28:23.166595 run iperf3-srv_10.42.42.10(pid=9410): Launched
18:28:23.383616 tst      cs_paging_gprs_active.py:38: start network...
18:28:23.525994 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
18:28:24.087005 run          create_hlr_db(pid=9411): Launched
18:28:24.282370 bus                        /sierra_2: Setting Powered False
18:28:25.303491 run          create_hlr_db(pid=9411): Terminated: ok {rc=0}
18:28:25.758692 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9413): Launched
18:28:26.188031 run    osmo-hlr_10.42.42.2(pid=9414): Launched
18:28:26.351261 run              osmo-stp_10.42.42.5: Starting osmo-stp
18:28:26.939540 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9415): Launched
18:28:27.370358 run    osmo-stp_10.42.42.5(pid=9416): Launched
18:28:27.510124 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:28:29.118925 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9417): Launched
18:28:29.226569 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:28:29.489598 run               patchelf(pid=9420): Launched
18:28:30.670074 run               patchelf(pid=9420): Terminated: ok {rc=0}
18:28:30.780302 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:28:31.065712 run osmo-gsm-tester_setcap_net_admin.sh(pid=9421): Launched
18:28:32.258681 run osmo-gsm-tester_setcap_net_admin.sh(pid=9421): Terminated: ok {rc=0}
18:28:32.586807 run   osmo-ggsn_10.42.42.6(pid=9424): Launched
18:28:32.776076 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:28:37.770104 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9442): Launched
18:28:38.072694 run   osmo-sgsn_10.42.42.7(pid=9443): Launched
18:28:38.183176 run              osmo-msc_10.42.42.8: Starting osmo-msc
18:28:38.753227 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9444): Launched
18:28:39.056759 run    osmo-msc_10.42.42.8(pid=9445): Launched
18:28:42.218266 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
18:28:42.741327 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9446): Launched
18:28:43.042490 run    osmo-mgw_10.42.42.3(pid=9447): Launched
18:28:43.150433 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
18:28:43.672500 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9448): Launched
18:28:43.973660 run    osmo-mgw_10.42.42.4(pid=9449): Launched
18:28:44.080433 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
18:28:44.713702 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9450): Launched
18:28:45.016363 run    osmo-bsc_10.42.42.9(pid=9451): Launched
18:28:51.287778 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:28:51.895312 run          rm-remote-dir(pid=9452): Launched
18:28:55.099990 run          rm-remote-dir(pid=9452): ERR: Terminated: ERROR {rc=255}  [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)]
18:28:55.243348 run          rm-remote-dir(pid=9452): stdout: 
| (launched: 2022-12-20_18:28:51.707496) 
18:28:55.385776 run          rm-remote-dir(pid=9452): stderr: 
| (launched: 2022-12-20_18:28:51.760977)
| ssh: connect to host 10.42.42.114 port 22: No route to host 
18:28:55.445125 tst      cs_paging_gprs_active.py:38: ERR: Error: rm-remote-dir(pid=9452): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)]  [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38]
18:28:55.453395 tst      cs_paging_gprs_active.py:38: Test FAILED (35.4 sec)
18:28:55.490439 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9407): Terminating (SIGTERM)
18:28:55.532784 run iperf3-srv_10.42.42.10(pid=9408): Terminating (SIGTERM)
18:28:55.575287 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9409): Terminating (SIGTERM)
18:28:55.617401 run iperf3-srv_10.42.42.10(pid=9410): Terminating (SIGTERM)
18:28:55.659910 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9413): Terminating (SIGTERM)
18:28:55.701742 run    osmo-hlr_10.42.42.2(pid=9414): Terminating (SIGTERM)
18:28:55.745251 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9415): Terminating (SIGTERM)
18:28:55.789822 run    osmo-stp_10.42.42.5(pid=9416): Terminating (SIGTERM)
18:28:55.855518 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9417): Terminating (SIGTERM)
18:28:55.911383 run   osmo-ggsn_10.42.42.6(pid=9424): Terminating (SIGTERM)
18:28:55.965672 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9442): Terminating (SIGTERM)
18:28:56.032632 run   osmo-sgsn_10.42.42.7(pid=9443): Terminating (SIGTERM)
18:28:56.078230 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9444): Terminating (SIGTERM)
18:28:56.121824 run    osmo-msc_10.42.42.8(pid=9445): Terminating (SIGTERM)
18:28:56.165996 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9446): Terminating (SIGTERM)
18:28:56.209535 run    osmo-mgw_10.42.42.3(pid=9447): Terminating (SIGTERM)
18:28:56.251070 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9448): Terminating (SIGTERM)
18:28:56.291731 run    osmo-mgw_10.42.42.4(pid=9449): Terminating (SIGTERM)
18:28:56.338863 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9450): Terminating (SIGTERM)
18:28:56.380780 run    osmo-bsc_10.42.42.9(pid=9451): Terminating (SIGTERM)
18:28:56.390690 ---      ParallelTerminationStrategy: PID 9407 died...
18:28:56.446733 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9407): Terminated: ok {rc=0}
18:28:56.456827 ---      ParallelTerminationStrategy: PID 9408 died...
18:28:56.512377 run iperf3-srv_10.42.42.10(pid=9408): Terminated {rc=256}
18:28:56.522257 ---      ParallelTerminationStrategy: PID 9409 died...
18:28:56.578841 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9409): Terminated: ok {rc=0}
18:28:56.588964 ---      ParallelTerminationStrategy: PID 9410 died...
18:28:56.644892 run iperf3-srv_10.42.42.10(pid=9410): Terminated {rc=256}
18:28:56.654856 ---      ParallelTerminationStrategy: PID 9413 died...
18:28:56.710445 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9413): Terminated: ok {rc=0}
18:28:56.720393 ---      ParallelTerminationStrategy: PID 9414 died...
18:28:56.776400 run    osmo-hlr_10.42.42.2(pid=9414): Terminated: ok {rc=0}
18:28:56.786324 ---      ParallelTerminationStrategy: PID 9415 died...
18:28:56.842177 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9415): Terminated: ok {rc=0}
18:28:56.852469 ---      ParallelTerminationStrategy: PID 9416 died...
18:28:56.908864 run    osmo-stp_10.42.42.5(pid=9416): Terminated: ok {rc=0}
18:28:56.919773 ---      ParallelTerminationStrategy: PID 9417 died...
18:28:56.977170 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9417): Terminated: ok {rc=0}
18:28:56.987764 ---      ParallelTerminationStrategy: PID 9424 died...
18:28:57.044343 run   osmo-ggsn_10.42.42.6(pid=9424): Terminated: ok {rc=0}
18:28:57.055155 ---      ParallelTerminationStrategy: PID 9442 died...
18:28:57.129547 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9442): Terminated: ok {rc=0}
18:28:57.172938 ---      ParallelTerminationStrategy: PID 9444 died...
18:28:57.270360 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9444): Terminated: ok {rc=0}
18:28:57.288962 ---      ParallelTerminationStrategy: PID 9445 died...
18:28:57.380315 run    osmo-msc_10.42.42.8(pid=9445): Terminated: ok {rc=0}
18:28:57.392004 ---      ParallelTerminationStrategy: PID 9446 died...
18:28:57.456561 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9446): Terminated: ok {rc=0}
18:28:57.467563 ---      ParallelTerminationStrategy: PID 9447 died...
18:28:57.526443 run    osmo-mgw_10.42.42.3(pid=9447): Terminated {rc=15}
18:28:57.537140 ---      ParallelTerminationStrategy: PID 9448 died...
18:28:57.594032 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9448): Terminated: ok {rc=0}
18:28:57.607380 ---      ParallelTerminationStrategy: PID 9449 died...
18:28:57.663624 run    osmo-mgw_10.42.42.4(pid=9449): Terminated {rc=15}
18:28:57.673716 ---      ParallelTerminationStrategy: PID 9443 died...
18:28:57.729574 run   osmo-sgsn_10.42.42.7(pid=9443): Terminated: ok {rc=0}
18:28:57.739836 ---      ParallelTerminationStrategy: PID 9450 died...
18:28:57.795883 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9450): Terminated: ok {rc=0}
18:28:57.806245 ---      ParallelTerminationStrategy: PID 9451 died...
18:28:57.862215 run    osmo-bsc_10.42.42.9(pid=9451): Terminated: ok {rc=0}
18:28:57.950326 bus                          /gobi_0: Setting Powered False
18:28:59.053219 bus                          /gobi_0: Setting Powered False
18:29:00.160262 bus                          /gobi_6: Setting Powered False

Standard Error

Traceback (most recent call last):
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
    self.path)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
    spec.loader.exec_module( importlib.util.module_from_spec(spec) )
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/cs_paging_gprs_active.py", line 38, in <module>
    setup_run_iperf3_test_parallel(2, ready_cb=ready_cb_place_voicecall)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 74, in setup_run_iperf3_test_parallel
    bts.start()
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_sysmo.py", line 113, in start
    rem_host.recreate_remote_dir(self.remote_inst)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 146, in recreate_remote_dir
    self.rm_remote_dir(remote_dir)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 139, in rm_remote_dir
    self.run_remote_sync('rm-remote-dir', ('test', '!', '-d', remote_dir_str, '||', 'rm', '-rf', remote_dir_str))
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync
    proc.launch_sync()
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync
    raise self.RunError('launch_sync()')
osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=9452): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)]