Failed
gprs:sysmo.ping.py (from gprs_sysmo)
Failing for the past 2 builds
(Since Failed )
Stacktrace
rm-remote-dir(pid=9965): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)]
Standard Output
---------------------------------------------- trial-2070 gprs:sysmo ping.py ---------------------------------------------- 15:44:22.579436 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:22.591124 tst gprs:sysmo: Using 1 x bts (candidates: 1) 15:44:22.720742 tst ping.py: using LAC 41712 15:44:22.847926 tst ping.py: using RAC 147 15:44:22.972998 tst ping.py: using CellId 41712 15:44:23.096374 tst ping.py: using BVCI 41713 15:44:23.120930 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:23.131854 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:23.142810 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:23.154376 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:23.165126 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:23.175928 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:23.187117 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 15:44:23.198181 tst gprs:sysmo: Using 1 x modem (candidates: 4) 15:44:23.472509 tst ping.py:19: start network... 15:44:23.596224 run osmo-hlr_10.42.42.2: Starting osmo-hlr 15:44:24.075180 run create_hlr_db(pid=9927): Launched 15:44:24.247407 bus /gobi_6: Setting Powered False 15:44:25.268583 run create_hlr_db(pid=9927): Terminated: ok {rc=0} 15:44:25.693742 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9929): Launched 15:44:26.012309 run osmo-hlr_10.42.42.2(pid=9930): Launched 15:44:26.127230 run osmo-stp_10.42.42.5: Starting osmo-stp 15:44:26.681363 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9931): Launched 15:44:26.999939 run osmo-stp_10.42.42.5(pid=9932): Launched 15:44:27.114326 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 15:44:27.668557 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9933): Launched 15:44:27.782239 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 15:44:28.059589 run patchelf(pid=9934): Launched 15:44:29.249647 run patchelf(pid=9934): Terminated: ok {rc=0} 15:44:29.366625 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 15:44:29.655057 run osmo-gsm-tester_setcap_net_admin.sh(pid=9935): Launched 15:44:30.844104 run osmo-gsm-tester_setcap_net_admin.sh(pid=9935): Terminated: ok {rc=0} 15:44:31.165843 run osmo-ggsn_10.42.42.6(pid=9938): Launched 15:44:31.434637 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 15:44:31.955338 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9955): Launched 15:44:32.254551 run osmo-sgsn_10.42.42.7(pid=9956): Launched 15:44:32.362879 run osmo-msc_10.42.42.8: Starting osmo-msc 15:44:32.907901 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9957): Launched 15:44:33.205346 run osmo-msc_10.42.42.8(pid=9958): Launched 15:44:36.377151 run osmo-mgw_10.42.42.3: Starting osmo-mgw 15:44:36.923808 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9959): Launched 15:44:37.267314 run osmo-mgw_10.42.42.3(pid=9960): Launched 15:44:37.372694 run osmo-mgw_10.42.42.4: Starting osmo-mgw 15:44:37.876117 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9961): Launched 15:44:38.170966 run osmo-mgw_10.42.42.4(pid=9962): Launched 15:44:38.275907 run osmo-bsc_10.42.42.9: Starting osmo-bsc 15:44:38.906252 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9963): Launched 15:44:39.202098 run osmo-bsc_10.42.42.9(pid=9964): Launched 15:44:45.454256 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 15:44:45.981469 run rm-remote-dir(pid=9965): Launched 15:44:49.167497 run rm-remote-dir(pid=9965): ERR: Terminated: ERROR {rc=255} [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)] 15:44:49.298285 run rm-remote-dir(pid=9965): stdout: | (launched: 2022-12-23_15:44:45.829596) 15:44:49.429302 run rm-remote-dir(pid=9965): stderr: | (launched: 2022-12-23_15:44:45.876684) | ssh: connect to host 10.42.42.114 port 22: No route to host 15:44:49.483780 tst ping.py:29: ERR: Error: rm-remote-dir(pid=9965): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)] [trial-2070↪gprs:sysmo↪ping.py:29] 15:44:49.490910 tst ping.py:29: Test FAILED (26.9 sec) 15:44:49.525553 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9929): Terminating (SIGTERM) 15:44:49.565160 run osmo-hlr_10.42.42.2(pid=9930): Terminating (SIGTERM) 15:44:49.605954 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9931): Terminating (SIGTERM) 15:44:49.649603 run osmo-stp_10.42.42.5(pid=9932): Terminating (SIGTERM) 15:44:49.690769 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9933): Terminating (SIGTERM) 15:44:49.733628 run osmo-ggsn_10.42.42.6(pid=9938): Terminating (SIGTERM) 15:44:49.792605 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9955): Terminating (SIGTERM) 15:44:49.852752 run osmo-sgsn_10.42.42.7(pid=9956): Terminating (SIGTERM) 15:44:49.894268 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9957): Terminating (SIGTERM) 15:44:49.935249 run osmo-msc_10.42.42.8(pid=9958): Terminating (SIGTERM) 15:44:49.975887 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9959): Terminating (SIGTERM) 15:44:50.015342 run osmo-mgw_10.42.42.3(pid=9960): Terminating (SIGTERM) 15:44:50.055472 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9961): Terminating (SIGTERM) 15:44:50.094922 run osmo-mgw_10.42.42.4(pid=9962): Terminating (SIGTERM) 15:44:50.135057 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9963): Terminating (SIGTERM) 15:44:50.174360 run osmo-bsc_10.42.42.9(pid=9964): Terminating (SIGTERM) 15:44:50.183671 --- ParallelTerminationStrategy: PID 9929 died... 15:44:50.237623 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9929): Terminated: ok {rc=0} 15:44:50.247233 --- ParallelTerminationStrategy: PID 9930 died... 15:44:50.303692 run osmo-hlr_10.42.42.2(pid=9930): Terminated: ok {rc=0} 15:44:50.313950 --- ParallelTerminationStrategy: PID 9931 died... 15:44:50.412795 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9931): Terminated: ok {rc=0} 15:44:50.435661 --- ParallelTerminationStrategy: PID 9932 died... 15:44:50.490394 run osmo-stp_10.42.42.5(pid=9932): Terminated: ok {rc=0} 15:44:50.500632 --- ParallelTerminationStrategy: PID 9933 died... 15:44:50.553633 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9933): Terminated: ok {rc=0} 15:44:50.563292 --- ParallelTerminationStrategy: PID 9938 died... 15:44:50.615862 run osmo-ggsn_10.42.42.6(pid=9938): Terminated: ok {rc=0} 15:44:50.625538 --- ParallelTerminationStrategy: PID 9955 died... 15:44:50.678409 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9955): Terminated: ok {rc=0} 15:44:50.688111 --- ParallelTerminationStrategy: PID 9957 died... 15:44:50.740821 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9957): Terminated: ok {rc=0} 15:44:50.750484 --- ParallelTerminationStrategy: PID 9958 died... 15:44:50.802976 run osmo-msc_10.42.42.8(pid=9958): Terminated: ok {rc=0} 15:44:50.812639 --- ParallelTerminationStrategy: PID 9959 died... 15:44:50.865604 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9959): Terminated: ok {rc=0} 15:44:50.875929 --- ParallelTerminationStrategy: PID 9960 died... 15:44:50.943351 run osmo-mgw_10.42.42.3(pid=9960): Terminated {rc=15} 15:44:50.953686 --- ParallelTerminationStrategy: PID 9961 died... 15:44:51.094121 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9961): Terminated: ok {rc=0} 15:44:51.118520 --- ParallelTerminationStrategy: PID 9962 died... 15:44:51.249882 run osmo-mgw_10.42.42.4(pid=9962): Terminated {rc=15} 15:44:51.281426 --- ParallelTerminationStrategy: PID 9956 died... 15:44:51.347798 run osmo-sgsn_10.42.42.7(pid=9956): Terminated: ok {rc=0} 15:44:51.359814 --- ParallelTerminationStrategy: PID 9963 died... 15:44:51.424705 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9963): Terminated: ok {rc=0} 15:44:51.437072 --- ParallelTerminationStrategy: PID 9964 died... 15:44:51.502740 run osmo-bsc_10.42.42.9(pid=9964): Terminated: ok {rc=0} 15:44:51.604369 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/ping.py", line 29, in <module> bts.start() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_sysmo.py", line 113, in start rem_host.recreate_remote_dir(self.remote_inst) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 146, in recreate_remote_dir self.rm_remote_dir(remote_dir) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 139, in rm_remote_dir self.run_remote_sync('rm-remote-dir', ('test', '!', '-d', remote_dir_str, '||', 'rm', '-rf', remote_dir_str)) File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync proc.launch_sync() File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync raise self.RunError('launch_sync()') osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=9965): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)]