Failed
gprs:sysmo.cs_paging_gprs_active.py (from gprs_sysmo)
Stacktrace
rm-remote-dir(pid=9452): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)]
Standard Output
---------------------------------------------- trial-2069 gprs:sysmo cs_paging_gprs_active.py ---------------------------------------------- 18:28:20.066730 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.079034 tst gprs:sysmo: Using 1 x bts (candidates: 1) 18:28:20.221742 tst cs_paging_gprs_active.py: using LAC 41702 18:28:20.353850 tst cs_paging_gprs_active.py: using RAC 137 18:28:20.484016 tst cs_paging_gprs_active.py: using CellId 41702 18:28:20.618593 tst cs_paging_gprs_active.py: using BVCI 41703 18:28:20.644751 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.659437 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.671765 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.685149 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.698529 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.715534 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.734351 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.747863 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:28:20.766355 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:28:20.936510 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:28:21.260998 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003... 18:28:21.432801 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:28:21.829316 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9407): Launched 18:28:22.123852 run iperf3-srv_10.42.42.10(pid=9408): Launched 18:28:22.318730 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004... 18:28:22.464680 run iperf3-srv_10.42.42.10: Starting iperf3-srv 18:28:22.859684 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9409): Launched 18:28:23.166595 run iperf3-srv_10.42.42.10(pid=9410): Launched 18:28:23.383616 tst cs_paging_gprs_active.py:38: start network... 18:28:23.525994 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:28:24.087005 run create_hlr_db(pid=9411): Launched 18:28:24.282370 bus /sierra_2: Setting Powered False 18:28:25.303491 run create_hlr_db(pid=9411): Terminated: ok {rc=0} 18:28:25.758692 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9413): Launched 18:28:26.188031 run osmo-hlr_10.42.42.2(pid=9414): Launched 18:28:26.351261 run osmo-stp_10.42.42.5: Starting osmo-stp 18:28:26.939540 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9415): Launched 18:28:27.370358 run osmo-stp_10.42.42.5(pid=9416): Launched 18:28:27.510124 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 18:28:29.118925 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9417): Launched 18:28:29.226569 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 18:28:29.489598 run patchelf(pid=9420): Launched 18:28:30.670074 run patchelf(pid=9420): Terminated: ok {rc=0} 18:28:30.780302 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 18:28:31.065712 run osmo-gsm-tester_setcap_net_admin.sh(pid=9421): Launched 18:28:32.258681 run osmo-gsm-tester_setcap_net_admin.sh(pid=9421): Terminated: ok {rc=0} 18:28:32.586807 run osmo-ggsn_10.42.42.6(pid=9424): Launched 18:28:32.776076 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 18:28:37.770104 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9442): Launched 18:28:38.072694 run osmo-sgsn_10.42.42.7(pid=9443): Launched 18:28:38.183176 run osmo-msc_10.42.42.8: Starting osmo-msc 18:28:38.753227 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9444): Launched 18:28:39.056759 run osmo-msc_10.42.42.8(pid=9445): Launched 18:28:42.218266 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:28:42.741327 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9446): Launched 18:28:43.042490 run osmo-mgw_10.42.42.3(pid=9447): Launched 18:28:43.150433 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:28:43.672500 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9448): Launched 18:28:43.973660 run osmo-mgw_10.42.42.4(pid=9449): Launched 18:28:44.080433 run osmo-bsc_10.42.42.9: Starting osmo-bsc 18:28:44.713702 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9450): Launched 18:28:45.016363 run osmo-bsc_10.42.42.9(pid=9451): Launched 18:28:51.287778 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 18:28:51.895312 run rm-remote-dir(pid=9452): Launched 18:28:55.099990 run rm-remote-dir(pid=9452): ERR: Terminated: ERROR {rc=255} [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)] 18:28:55.243348 run rm-remote-dir(pid=9452): stdout: | (launched: 2022-12-20_18:28:51.707496) 18:28:55.385776 run rm-remote-dir(pid=9452): stderr: | (launched: 2022-12-20_18:28:51.760977) | ssh: connect to host 10.42.42.114 port 22: No route to host 18:28:55.445125 tst cs_paging_gprs_active.py:38: ERR: Error: rm-remote-dir(pid=9452): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)] [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38] 18:28:55.453395 tst cs_paging_gprs_active.py:38: Test FAILED (35.4 sec) 18:28:55.490439 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9407): Terminating (SIGTERM) 18:28:55.532784 run iperf3-srv_10.42.42.10(pid=9408): Terminating (SIGTERM) 18:28:55.575287 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9409): Terminating (SIGTERM) 18:28:55.617401 run iperf3-srv_10.42.42.10(pid=9410): Terminating (SIGTERM) 18:28:55.659910 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9413): Terminating (SIGTERM) 18:28:55.701742 run osmo-hlr_10.42.42.2(pid=9414): Terminating (SIGTERM) 18:28:55.745251 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9415): Terminating (SIGTERM) 18:28:55.789822 run osmo-stp_10.42.42.5(pid=9416): Terminating (SIGTERM) 18:28:55.855518 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9417): Terminating (SIGTERM) 18:28:55.911383 run osmo-ggsn_10.42.42.6(pid=9424): Terminating (SIGTERM) 18:28:55.965672 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9442): Terminating (SIGTERM) 18:28:56.032632 run osmo-sgsn_10.42.42.7(pid=9443): Terminating (SIGTERM) 18:28:56.078230 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9444): Terminating (SIGTERM) 18:28:56.121824 run osmo-msc_10.42.42.8(pid=9445): Terminating (SIGTERM) 18:28:56.165996 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9446): Terminating (SIGTERM) 18:28:56.209535 run osmo-mgw_10.42.42.3(pid=9447): Terminating (SIGTERM) 18:28:56.251070 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9448): Terminating (SIGTERM) 18:28:56.291731 run osmo-mgw_10.42.42.4(pid=9449): Terminating (SIGTERM) 18:28:56.338863 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9450): Terminating (SIGTERM) 18:28:56.380780 run osmo-bsc_10.42.42.9(pid=9451): Terminating (SIGTERM) 18:28:56.390690 --- ParallelTerminationStrategy: PID 9407 died... 18:28:56.446733 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9407): Terminated: ok {rc=0} 18:28:56.456827 --- ParallelTerminationStrategy: PID 9408 died... 18:28:56.512377 run iperf3-srv_10.42.42.10(pid=9408): Terminated {rc=256} 18:28:56.522257 --- ParallelTerminationStrategy: PID 9409 died... 18:28:56.578841 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=9409): Terminated: ok {rc=0} 18:28:56.588964 --- ParallelTerminationStrategy: PID 9410 died... 18:28:56.644892 run iperf3-srv_10.42.42.10(pid=9410): Terminated {rc=256} 18:28:56.654856 --- ParallelTerminationStrategy: PID 9413 died... 18:28:56.710445 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9413): Terminated: ok {rc=0} 18:28:56.720393 --- ParallelTerminationStrategy: PID 9414 died... 18:28:56.776400 run osmo-hlr_10.42.42.2(pid=9414): Terminated: ok {rc=0} 18:28:56.786324 --- ParallelTerminationStrategy: PID 9415 died... 18:28:56.842177 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9415): Terminated: ok {rc=0} 18:28:56.852469 --- ParallelTerminationStrategy: PID 9416 died... 18:28:56.908864 run osmo-stp_10.42.42.5(pid=9416): Terminated: ok {rc=0} 18:28:56.919773 --- ParallelTerminationStrategy: PID 9417 died... 18:28:56.977170 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9417): Terminated: ok {rc=0} 18:28:56.987764 --- ParallelTerminationStrategy: PID 9424 died... 18:28:57.044343 run osmo-ggsn_10.42.42.6(pid=9424): Terminated: ok {rc=0} 18:28:57.055155 --- ParallelTerminationStrategy: PID 9442 died... 18:28:57.129547 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9442): Terminated: ok {rc=0} 18:28:57.172938 --- ParallelTerminationStrategy: PID 9444 died... 18:28:57.270360 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9444): Terminated: ok {rc=0} 18:28:57.288962 --- ParallelTerminationStrategy: PID 9445 died... 18:28:57.380315 run osmo-msc_10.42.42.8(pid=9445): Terminated: ok {rc=0} 18:28:57.392004 --- ParallelTerminationStrategy: PID 9446 died... 18:28:57.456561 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9446): Terminated: ok {rc=0} 18:28:57.467563 --- ParallelTerminationStrategy: PID 9447 died... 18:28:57.526443 run osmo-mgw_10.42.42.3(pid=9447): Terminated {rc=15} 18:28:57.537140 --- ParallelTerminationStrategy: PID 9448 died... 18:28:57.594032 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9448): Terminated: ok {rc=0} 18:28:57.607380 --- ParallelTerminationStrategy: PID 9449 died... 18:28:57.663624 run osmo-mgw_10.42.42.4(pid=9449): Terminated {rc=15} 18:28:57.673716 --- ParallelTerminationStrategy: PID 9443 died... 18:28:57.729574 run osmo-sgsn_10.42.42.7(pid=9443): Terminated: ok {rc=0} 18:28:57.739836 --- ParallelTerminationStrategy: PID 9450 died... 18:28:57.795883 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9450): Terminated: ok {rc=0} 18:28:57.806245 --- ParallelTerminationStrategy: PID 9451 died... 18:28:57.862215 run osmo-bsc_10.42.42.9(pid=9451): Terminated: ok {rc=0} 18:28:57.950326 bus /gobi_0: Setting Powered False 18:28:59.053219 bus /gobi_0: Setting Powered False 18:29:00.160262 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/cs_paging_gprs_active.py", line 38, in <module> setup_run_iperf3_test_parallel(2, ready_cb=ready_cb_place_voicecall) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 74, in setup_run_iperf3_test_parallel bts.start() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_sysmo.py", line 113, in start rem_host.recreate_remote_dir(self.remote_inst) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 146, in recreate_remote_dir self.rm_remote_dir(remote_dir) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 139, in rm_remote_dir self.run_remote_sync('rm-remote-dir', ('test', '!', '-d', remote_dir_str, '||', 'rm', '-rf', remote_dir_str)) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync proc.launch_sync() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync raise self.RunError('launch_sync()') osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=9452): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9452)]