Regression
gprs:sysmo.ping_idle_ping.py (from gprs_sysmo)
Failing for the past 1 build
(Since Failed )
Stacktrace
rm-remote-dir(pid=10217): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=10217)]
Standard Output
---------------------------------------------- trial-2069 gprs:sysmo ping_idle_ping.py ---------------------------------------------- 18:30:46.115617 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.127947 tst gprs:sysmo: Using 1 x bts (candidates: 1) 18:30:46.260489 tst ping_idle_ping.py: using LAC 41706 18:30:46.378166 tst ping_idle_ping.py: using RAC 141 18:30:46.499174 tst ping_idle_ping.py: using CellId 41706 18:30:46.615002 tst ping_idle_ping.py: using BVCI 41707 18:30:46.638082 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.648745 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.659270 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.669702 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.679958 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.690324 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.700857 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 18:30:46.711264 tst gprs:sysmo: Using 1 x modem (candidates: 4) 18:30:47.001283 tst ping_idle_ping.py:24: start network... 18:30:47.135185 run osmo-hlr_10.42.42.2: Starting osmo-hlr 18:30:47.641689 run create_hlr_db(pid=10171): Launched 18:30:47.828651 bus /gobi_6: Setting Powered False 18:30:48.849646 run create_hlr_db(pid=10171): Terminated: ok {rc=0} 18:30:49.251196 run pcap-recorder_any(filters='host 10.42.42.2')(pid=10173): Launched 18:30:49.550564 run osmo-hlr_10.42.42.2(pid=10174): Launched 18:30:49.659519 run osmo-stp_10.42.42.5: Starting osmo-stp 18:30:50.178297 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=10175): Launched 18:30:50.480426 run osmo-stp_10.42.42.5(pid=10176): Launched 18:30:50.587295 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 18:30:51.163612 run pcap-recorder_any(filters='host 10.42.42.6')(pid=10177): Launched 18:30:51.272002 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 18:30:51.543357 run patchelf(pid=10178): Launched 18:30:52.721753 run patchelf(pid=10178): Terminated: ok {rc=0} 18:30:52.831248 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 18:30:53.098072 run osmo-gsm-tester_setcap_net_admin.sh(pid=10179): Launched 18:30:54.276904 run osmo-gsm-tester_setcap_net_admin.sh(pid=10179): Terminated: ok {rc=0} 18:30:54.577246 run osmo-ggsn_10.42.42.6(pid=10182): Launched 18:30:54.852046 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 18:30:55.426080 run pcap-recorder_any(filters='host 10.42.42.7')(pid=10207): Launched 18:30:55.747975 run osmo-sgsn_10.42.42.7(pid=10208): Launched 18:30:55.866118 run osmo-msc_10.42.42.8: Starting osmo-msc 18:30:56.457392 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=10209): Launched 18:30:56.781307 run osmo-msc_10.42.42.8(pid=10210): Launched 18:30:59.943219 run osmo-mgw_10.42.42.3: Starting osmo-mgw 18:31:00.464573 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=10211): Launched 18:31:00.764561 run osmo-mgw_10.42.42.3(pid=10212): Launched 18:31:00.871022 run osmo-mgw_10.42.42.4: Starting osmo-mgw 18:31:01.379335 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=10213): Launched 18:31:01.678845 run osmo-mgw_10.42.42.4(pid=10214): Launched 18:31:01.785610 run osmo-bsc_10.42.42.9: Starting osmo-bsc 18:31:02.466195 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=10215): Launched 18:31:02.788577 run osmo-bsc_10.42.42.9(pid=10216): Launched 18:31:09.075177 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 18:31:09.647461 run rm-remote-dir(pid=10217): Launched 18:31:12.848725 run rm-remote-dir(pid=10217): ERR: Terminated: ERROR {rc=255} [trial-2069↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=10217)] 18:31:12.989332 run rm-remote-dir(pid=10217): stdout: | (launched: 2022-12-20_18:31:09.481778) 18:31:13.130175 run rm-remote-dir(pid=10217): stderr: | (launched: 2022-12-20_18:31:09.532407) | ssh: connect to host 10.42.42.114 port 22: No route to host 18:31:13.188369 tst ping_idle_ping.py:34: ERR: Error: rm-remote-dir(pid=10217): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=10217)] [trial-2069↪gprs:sysmo↪ping_idle_ping.py:34] 18:31:13.195984 tst ping_idle_ping.py:34: Test FAILED (27.1 sec) 18:31:13.232496 run pcap-recorder_any(filters='host 10.42.42.2')(pid=10173): Terminating (SIGTERM) 18:31:13.274115 run osmo-hlr_10.42.42.2(pid=10174): Terminating (SIGTERM) 18:31:13.316663 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=10175): Terminating (SIGTERM) 18:31:13.361006 run osmo-stp_10.42.42.5(pid=10176): Terminating (SIGTERM) 18:31:13.421954 run pcap-recorder_any(filters='host 10.42.42.6')(pid=10177): Terminating (SIGTERM) 18:31:13.479005 run osmo-ggsn_10.42.42.6(pid=10182): Terminating (SIGTERM) 18:31:13.554654 run pcap-recorder_any(filters='host 10.42.42.7')(pid=10207): Terminating (SIGTERM) 18:31:13.601365 run osmo-sgsn_10.42.42.7(pid=10208): Terminating (SIGTERM) 18:31:13.645472 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=10209): Terminating (SIGTERM) 18:31:13.690089 run osmo-msc_10.42.42.8(pid=10210): Terminating (SIGTERM) 18:31:13.734040 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=10211): Terminating (SIGTERM) 18:31:13.781532 run osmo-mgw_10.42.42.3(pid=10212): Terminating (SIGTERM) 18:31:13.822734 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=10213): Terminating (SIGTERM) 18:31:13.863018 run osmo-mgw_10.42.42.4(pid=10214): Terminating (SIGTERM) 18:31:13.908012 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=10215): Terminating (SIGTERM) 18:31:13.952092 run osmo-bsc_10.42.42.9(pid=10216): Terminating (SIGTERM) 18:31:13.963792 --- ParallelTerminationStrategy: PID 10173 died... 18:31:14.019739 run pcap-recorder_any(filters='host 10.42.42.2')(pid=10173): Terminated: ok {rc=0} 18:31:14.031074 --- ParallelTerminationStrategy: PID 10174 died... 18:31:14.086282 run osmo-hlr_10.42.42.2(pid=10174): Terminated: ok {rc=0} 18:31:14.097686 --- ParallelTerminationStrategy: PID 10175 died... 18:31:14.154614 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=10175): Terminated: ok {rc=0} 18:31:14.164976 --- ParallelTerminationStrategy: PID 10176 died... 18:31:14.219718 run osmo-stp_10.42.42.5(pid=10176): Terminated: ok {rc=0} 18:31:14.229728 --- ParallelTerminationStrategy: PID 10177 died... 18:31:14.283849 run pcap-recorder_any(filters='host 10.42.42.6')(pid=10177): Terminated: ok {rc=0} 18:31:14.294137 --- ParallelTerminationStrategy: PID 10182 died... 18:31:14.349039 run osmo-ggsn_10.42.42.6(pid=10182): Terminated: ok {rc=0} 18:31:14.358781 --- ParallelTerminationStrategy: PID 10207 died... 18:31:14.413235 run pcap-recorder_any(filters='host 10.42.42.7')(pid=10207): Terminated: ok {rc=0} 18:31:14.422938 --- ParallelTerminationStrategy: PID 10209 died... 18:31:14.477137 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=10209): Terminated: ok {rc=0} 18:31:14.487032 --- ParallelTerminationStrategy: PID 10210 died... 18:31:14.540682 run osmo-msc_10.42.42.8(pid=10210): Terminated: ok {rc=0} 18:31:14.551081 --- ParallelTerminationStrategy: PID 10211 died... 18:31:14.604952 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=10211): Terminated: ok {rc=0} 18:31:14.615584 --- ParallelTerminationStrategy: PID 10212 died... 18:31:14.671732 run osmo-mgw_10.42.42.3(pid=10212): Terminated {rc=15} 18:31:14.682969 --- ParallelTerminationStrategy: PID 10213 died... 18:31:14.739443 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=10213): Terminated: ok {rc=0} 18:31:14.750936 --- ParallelTerminationStrategy: PID 10214 died... 18:31:14.830184 run osmo-mgw_10.42.42.4(pid=10214): Terminated {rc=15} 18:31:14.840940 --- ParallelTerminationStrategy: PID 10215 died... 18:31:14.897679 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=10215): Terminated: ok {rc=0} 18:31:14.909582 --- ParallelTerminationStrategy: PID 10216 died... 18:31:14.963769 run osmo-bsc_10.42.42.9(pid=10216): Terminated: ok {rc=0} 18:31:14.974009 --- ParallelTerminationStrategy: PID 10208 died... 18:31:15.027720 run osmo-sgsn_10.42.42.7(pid=10208): Terminated: ok {rc=0} 18:31:15.112270 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/ping_idle_ping.py", line 34, in <module> bts.start() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_sysmo.py", line 113, in start rem_host.recreate_remote_dir(self.remote_inst) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 146, in recreate_remote_dir self.rm_remote_dir(remote_dir) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 139, in rm_remote_dir self.run_remote_sync('rm-remote-dir', ('test', '!', '-d', remote_dir_str, '||', 'rm', '-rf', remote_dir_str)) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync proc.launch_sync() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync raise self.RunError('launch_sync()') osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=10217): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2069↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=10217)]