Failed
gprs:sysmo.ping_idle_ping.py (from gprs_sysmo)
Failing for the past 4 builds
(Since Failed )
Stacktrace
rm-remote-dir(pid=13979): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2072↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=13979)]
Standard Output
---------------------------------------------- trial-2072 gprs:sysmo ping_idle_ping.py ---------------------------------------------- 16:57:03.248313 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.259848 tst gprs:sysmo: Using 1 x bts (candidates: 1) 16:57:03.394372 tst ping_idle_ping.py: using LAC 41727 16:57:03.518332 tst ping_idle_ping.py: using RAC 162 16:57:03.641612 tst ping_idle_ping.py: using CellId 41727 16:57:03.764580 tst ping_idle_ping.py: using BVCI 41728 16:57:03.789113 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.800011 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.810977 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.822216 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.833062 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.843969 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.855136 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 16:57:03.866263 tst gprs:sysmo: Using 1 x modem (candidates: 4) 16:57:04.156022 tst ping_idle_ping.py:24: start network... 16:57:04.289316 run osmo-hlr_10.42.42.2: Starting osmo-hlr 16:57:04.784467 run create_hlr_db(pid=13935): Launched 16:57:04.958633 bus /gobi_6: Setting Powered False 16:57:06.012552 run create_hlr_db(pid=13935): Terminated: ok {rc=0} 16:57:06.469747 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13937): Launched 16:57:06.995458 run osmo-hlr_10.42.42.2(pid=13938): Launched 16:57:07.104728 run osmo-stp_10.42.42.5: Starting osmo-stp 16:57:07.651473 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13939): Launched 16:57:07.970273 run osmo-stp_10.42.42.5(pid=13940): Launched 16:57:08.084206 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 16:57:08.645648 run pcap-recorder_any(filters='host 10.42.42.6')(pid=13941): Launched 16:57:08.759990 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 16:57:09.040926 run patchelf(pid=13942): Launched 16:57:10.232423 run patchelf(pid=13942): Terminated: ok {rc=0} 16:57:10.350411 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 16:57:10.637159 run osmo-gsm-tester_setcap_net_admin.sh(pid=13943): Launched 16:57:11.828091 run osmo-gsm-tester_setcap_net_admin.sh(pid=13943): Terminated: ok {rc=0} 16:57:12.161550 run osmo-ggsn_10.42.42.6(pid=13947): Launched 16:57:12.345184 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 16:57:12.949308 run pcap-recorder_any(filters='host 10.42.42.7')(pid=13969): Launched 16:57:13.261097 run osmo-sgsn_10.42.42.7(pid=13970): Launched 16:57:13.375854 run osmo-msc_10.42.42.8: Starting osmo-msc 16:57:13.951782 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=13971): Launched 16:57:14.267789 run osmo-msc_10.42.42.8(pid=13972): Launched 16:57:17.436914 run osmo-mgw_10.42.42.3: Starting osmo-mgw 16:57:17.985018 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13973): Launched 16:57:18.300992 run osmo-mgw_10.42.42.3(pid=13974): Launched 16:57:18.415498 run osmo-mgw_10.42.42.4: Starting osmo-mgw 16:57:18.956341 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13975): Launched 16:57:19.272086 run osmo-mgw_10.42.42.4(pid=13976): Launched 16:57:19.385985 run osmo-bsc_10.42.42.9: Starting osmo-bsc 16:57:20.050951 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=13977): Launched 16:57:20.369252 run osmo-bsc_10.42.42.9(pid=13978): Launched 16:57:26.639903 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 16:57:27.205958 run rm-remote-dir(pid=13979): Launched 16:57:30.406328 run rm-remote-dir(pid=13979): ERR: Terminated: ERROR {rc=255} [trial-2072↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=13979)] 16:57:30.548024 run rm-remote-dir(pid=13979): stdout: | (launched: 2023-03-07_16:57:27.042444) 16:57:30.689531 run rm-remote-dir(pid=13979): stderr: | (launched: 2023-03-07_16:57:27.092878) | ssh: connect to host 10.42.42.114 port 22: No route to host 16:57:30.747663 tst ping_idle_ping.py:34: ERR: Error: rm-remote-dir(pid=13979): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2072↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=13979)] [trial-2072↪gprs:sysmo↪ping_idle_ping.py:34] 16:57:30.755673 tst ping_idle_ping.py:34: Test FAILED (27.5 sec) 16:57:30.793184 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13937): Terminating (SIGTERM) 16:57:30.834966 run osmo-hlr_10.42.42.2(pid=13938): Terminating (SIGTERM) 16:57:30.878042 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13939): Terminating (SIGTERM) 16:57:30.922178 run osmo-stp_10.42.42.5(pid=13940): Terminating (SIGTERM) 16:57:30.966593 run pcap-recorder_any(filters='host 10.42.42.6')(pid=13941): Terminating (SIGTERM) 16:57:31.014248 run osmo-ggsn_10.42.42.6(pid=13947): Terminating (SIGTERM) 16:57:31.098397 run pcap-recorder_any(filters='host 10.42.42.7')(pid=13969): Terminating (SIGTERM) 16:57:31.153819 run osmo-sgsn_10.42.42.7(pid=13970): Terminating (SIGTERM) 16:57:31.211856 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=13971): Terminating (SIGTERM) 16:57:31.256489 run osmo-msc_10.42.42.8(pid=13972): Terminating (SIGTERM) 16:57:31.301179 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13973): Terminating (SIGTERM) 16:57:31.345774 run osmo-mgw_10.42.42.3(pid=13974): Terminating (SIGTERM) 16:57:31.390492 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13975): Terminating (SIGTERM) 16:57:31.435685 run osmo-mgw_10.42.42.4(pid=13976): Terminating (SIGTERM) 16:57:31.487276 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=13977): Terminating (SIGTERM) 16:57:31.532201 run osmo-bsc_10.42.42.9(pid=13978): Terminating (SIGTERM) 16:57:31.542474 --- ParallelTerminationStrategy: PID 13937 died... 16:57:31.602339 run pcap-recorder_any(filters='host 10.42.42.2')(pid=13937): Terminated: ok {rc=0} 16:57:31.613702 --- ParallelTerminationStrategy: PID 13938 died... 16:57:31.672777 run osmo-hlr_10.42.42.2(pid=13938): Terminated: ok {rc=0} 16:57:31.683921 --- ParallelTerminationStrategy: PID 13939 died... 16:57:31.744781 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=13939): Terminated: ok {rc=0} 16:57:31.755749 --- ParallelTerminationStrategy: PID 13940 died... 16:57:31.815089 run osmo-stp_10.42.42.5(pid=13940): Terminated: ok {rc=0} 16:57:31.826201 --- ParallelTerminationStrategy: PID 13941 died... 16:57:31.883328 run pcap-recorder_any(filters='host 10.42.42.6')(pid=13941): Terminated: ok {rc=0} 16:57:31.893941 --- ParallelTerminationStrategy: PID 13947 died... 16:57:31.950618 run osmo-ggsn_10.42.42.6(pid=13947): Terminated: ok {rc=0} 16:57:31.960892 --- ParallelTerminationStrategy: PID 13969 died... 16:57:32.017585 run pcap-recorder_any(filters='host 10.42.42.7')(pid=13969): Terminated: ok {rc=0} 16:57:32.027904 --- ParallelTerminationStrategy: PID 13971 died... 16:57:32.084727 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=13971): Terminated: ok {rc=0} 16:57:32.095009 --- ParallelTerminationStrategy: PID 13972 died... 16:57:32.151781 run osmo-msc_10.42.42.8(pid=13972): Terminated: ok {rc=0} 16:57:32.162128 --- ParallelTerminationStrategy: PID 13973 died... 16:57:32.219814 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=13973): Terminated: ok {rc=0} 16:57:32.230703 --- ParallelTerminationStrategy: PID 13974 died... 16:57:32.288537 run osmo-mgw_10.42.42.3(pid=13974): Terminated {rc=15} 16:57:32.299264 --- ParallelTerminationStrategy: PID 13975 died... 16:57:32.357124 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=13975): Terminated: ok {rc=0} 16:57:32.367951 --- ParallelTerminationStrategy: PID 13976 died... 16:57:32.425815 run osmo-mgw_10.42.42.4(pid=13976): Terminated {rc=15} 16:57:32.436645 --- ParallelTerminationStrategy: PID 13977 died... 16:57:32.494446 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=13977): Terminated: ok {rc=0} 16:57:32.505341 --- ParallelTerminationStrategy: PID 13978 died... 16:57:32.563580 run osmo-bsc_10.42.42.9(pid=13978): Terminated: ok {rc=0} 16:57:33.048892 --- ParallelTerminationStrategy: PID 13970 died... 16:57:33.103947 run osmo-sgsn_10.42.42.7(pid=13970): Terminated: ok {rc=0} 16:57:33.192332 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/ping_idle_ping.py", line 34, in <module> bts.start() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_sysmo.py", line 113, in start rem_host.recreate_remote_dir(self.remote_inst) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 146, in recreate_remote_dir self.rm_remote_dir(remote_dir) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 139, in rm_remote_dir self.run_remote_sync('rm-remote-dir', ('test', '!', '-d', remote_dir_str, '||', 'rm', '-rf', remote_dir_str)) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync proc.launch_sync() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync raise self.RunError('launch_sync()') osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=13979): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2072↪gprs:sysmo↪ping_idle_ping.py:34↪ping_idle_ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=13979)]