Skip to content
[Jenkins]Jenkins
log in | sign up

Failed

gprs:sysmo.ping.py (from gprs_sysmo)

Failing for the past 2 builds (Since #2069 )
Took 27 sec.

Stacktrace

rm-remote-dir(pid=9965): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)]

Standard Output

----------------------------------------------
trial-2070 gprs:sysmo ping.py
----------------------------------------------
15:44:22.579436 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:22.591124 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
15:44:22.720742 tst                          ping.py: using LAC 41712
15:44:22.847926 tst                          ping.py: using RAC 147
15:44:22.972998 tst                          ping.py: using CellId 41712
15:44:23.096374 tst                          ping.py: using BVCI 41713
15:44:23.120930 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:23.131854 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:23.142810 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:23.154376 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:23.165126 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:23.175928 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:23.187117 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
15:44:23.198181 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
15:44:23.472509 tst                       ping.py:19: start network...
15:44:23.596224 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
15:44:24.075180 run          create_hlr_db(pid=9927): Launched
15:44:24.247407 bus                          /gobi_6: Setting Powered False
15:44:25.268583 run          create_hlr_db(pid=9927): Terminated: ok {rc=0}
15:44:25.693742 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9929): Launched
15:44:26.012309 run    osmo-hlr_10.42.42.2(pid=9930): Launched
15:44:26.127230 run              osmo-stp_10.42.42.5: Starting osmo-stp
15:44:26.681363 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9931): Launched
15:44:26.999939 run    osmo-stp_10.42.42.5(pid=9932): Launched
15:44:27.114326 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
15:44:27.668557 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9933): Launched
15:44:27.782239 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
15:44:28.059589 run               patchelf(pid=9934): Launched
15:44:29.249647 run               patchelf(pid=9934): Terminated: ok {rc=0}
15:44:29.366625 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
15:44:29.655057 run osmo-gsm-tester_setcap_net_admin.sh(pid=9935): Launched
15:44:30.844104 run osmo-gsm-tester_setcap_net_admin.sh(pid=9935): Terminated: ok {rc=0}
15:44:31.165843 run   osmo-ggsn_10.42.42.6(pid=9938): Launched
15:44:31.434637 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
15:44:31.955338 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9955): Launched
15:44:32.254551 run   osmo-sgsn_10.42.42.7(pid=9956): Launched
15:44:32.362879 run              osmo-msc_10.42.42.8: Starting osmo-msc
15:44:32.907901 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9957): Launched
15:44:33.205346 run    osmo-msc_10.42.42.8(pid=9958): Launched
15:44:36.377151 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
15:44:36.923808 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9959): Launched
15:44:37.267314 run    osmo-mgw_10.42.42.3(pid=9960): Launched
15:44:37.372694 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
15:44:37.876117 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9961): Launched
15:44:38.170966 run    osmo-mgw_10.42.42.4(pid=9962): Launched
15:44:38.275907 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
15:44:38.906252 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9963): Launched
15:44:39.202098 run    osmo-bsc_10.42.42.9(pid=9964): Launched
15:44:45.454256 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
15:44:45.981469 run          rm-remote-dir(pid=9965): Launched
15:44:49.167497 run          rm-remote-dir(pid=9965): ERR: Terminated: ERROR {rc=255}  [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)]
15:44:49.298285 run          rm-remote-dir(pid=9965): stdout: 
| (launched: 2022-12-23_15:44:45.829596) 
15:44:49.429302 run          rm-remote-dir(pid=9965): stderr: 
| (launched: 2022-12-23_15:44:45.876684)
| ssh: connect to host 10.42.42.114 port 22: No route to host 
15:44:49.483780 tst                       ping.py:29: ERR: Error: rm-remote-dir(pid=9965): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)]  [trial-2070↪gprs:sysmo↪ping.py:29]
15:44:49.490910 tst                       ping.py:29: Test FAILED (26.9 sec)
15:44:49.525553 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9929): Terminating (SIGTERM)
15:44:49.565160 run    osmo-hlr_10.42.42.2(pid=9930): Terminating (SIGTERM)
15:44:49.605954 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9931): Terminating (SIGTERM)
15:44:49.649603 run    osmo-stp_10.42.42.5(pid=9932): Terminating (SIGTERM)
15:44:49.690769 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9933): Terminating (SIGTERM)
15:44:49.733628 run   osmo-ggsn_10.42.42.6(pid=9938): Terminating (SIGTERM)
15:44:49.792605 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9955): Terminating (SIGTERM)
15:44:49.852752 run   osmo-sgsn_10.42.42.7(pid=9956): Terminating (SIGTERM)
15:44:49.894268 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9957): Terminating (SIGTERM)
15:44:49.935249 run    osmo-msc_10.42.42.8(pid=9958): Terminating (SIGTERM)
15:44:49.975887 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9959): Terminating (SIGTERM)
15:44:50.015342 run    osmo-mgw_10.42.42.3(pid=9960): Terminating (SIGTERM)
15:44:50.055472 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9961): Terminating (SIGTERM)
15:44:50.094922 run    osmo-mgw_10.42.42.4(pid=9962): Terminating (SIGTERM)
15:44:50.135057 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9963): Terminating (SIGTERM)
15:44:50.174360 run    osmo-bsc_10.42.42.9(pid=9964): Terminating (SIGTERM)
15:44:50.183671 ---      ParallelTerminationStrategy: PID 9929 died...
15:44:50.237623 run pcap-recorder_any(filters='host 10.42.42.2')(pid=9929): Terminated: ok {rc=0}
15:44:50.247233 ---      ParallelTerminationStrategy: PID 9930 died...
15:44:50.303692 run    osmo-hlr_10.42.42.2(pid=9930): Terminated: ok {rc=0}
15:44:50.313950 ---      ParallelTerminationStrategy: PID 9931 died...
15:44:50.412795 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=9931): Terminated: ok {rc=0}
15:44:50.435661 ---      ParallelTerminationStrategy: PID 9932 died...
15:44:50.490394 run    osmo-stp_10.42.42.5(pid=9932): Terminated: ok {rc=0}
15:44:50.500632 ---      ParallelTerminationStrategy: PID 9933 died...
15:44:50.553633 run pcap-recorder_any(filters='host 10.42.42.6')(pid=9933): Terminated: ok {rc=0}
15:44:50.563292 ---      ParallelTerminationStrategy: PID 9938 died...
15:44:50.615862 run   osmo-ggsn_10.42.42.6(pid=9938): Terminated: ok {rc=0}
15:44:50.625538 ---      ParallelTerminationStrategy: PID 9955 died...
15:44:50.678409 run pcap-recorder_any(filters='host 10.42.42.7')(pid=9955): Terminated: ok {rc=0}
15:44:50.688111 ---      ParallelTerminationStrategy: PID 9957 died...
15:44:50.740821 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=9957): Terminated: ok {rc=0}
15:44:50.750484 ---      ParallelTerminationStrategy: PID 9958 died...
15:44:50.802976 run    osmo-msc_10.42.42.8(pid=9958): Terminated: ok {rc=0}
15:44:50.812639 ---      ParallelTerminationStrategy: PID 9959 died...
15:44:50.865604 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=9959): Terminated: ok {rc=0}
15:44:50.875929 ---      ParallelTerminationStrategy: PID 9960 died...
15:44:50.943351 run    osmo-mgw_10.42.42.3(pid=9960): Terminated {rc=15}
15:44:50.953686 ---      ParallelTerminationStrategy: PID 9961 died...
15:44:51.094121 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=9961): Terminated: ok {rc=0}
15:44:51.118520 ---      ParallelTerminationStrategy: PID 9962 died...
15:44:51.249882 run    osmo-mgw_10.42.42.4(pid=9962): Terminated {rc=15}
15:44:51.281426 ---      ParallelTerminationStrategy: PID 9956 died...
15:44:51.347798 run   osmo-sgsn_10.42.42.7(pid=9956): Terminated: ok {rc=0}
15:44:51.359814 ---      ParallelTerminationStrategy: PID 9963 died...
15:44:51.424705 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=9963): Terminated: ok {rc=0}
15:44:51.437072 ---      ParallelTerminationStrategy: PID 9964 died...
15:44:51.502740 run    osmo-bsc_10.42.42.9(pid=9964): Terminated: ok {rc=0}
15:44:51.604369 bus                          /gobi_6: Setting Powered False

Standard Error

Traceback (most recent call last):
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
    self.path)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
    spec.loader.exec_module( importlib.util.module_from_spec(spec) )
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/ping.py", line 29, in <module>
    bts.start()
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_sysmo.py", line 113, in start
    rem_host.recreate_remote_dir(self.remote_inst)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 146, in recreate_remote_dir
    self.rm_remote_dir(remote_dir)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 139, in rm_remote_dir
    self.run_remote_sync('rm-remote-dir', ('test', '!', '-d', remote_dir_str, '||', 'rm', '-rf', remote_dir_str))
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync
    proc.launch_sync()
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync
    raise self.RunError('launch_sync()')
osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=9965): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2070↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=9965)]