Skip to content
[Jenkins]Jenkins
log in | sign up

Failed

gprs:sysmo.ping.py (from gprs_sysmo)

Failing for the past 3 builds (Since #2069 )
Took 27 sec.

Stacktrace

rm-remote-dir(pid=31679): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2071↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=31679)]

Standard Output

----------------------------------------------
trial-2071 gprs:sysmo ping.py
----------------------------------------------
22:00:40.450361 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:40.462143 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
22:00:40.591264 tst                          ping.py: using LAC 41719
22:00:40.718721 tst                          ping.py: using RAC 154
22:00:40.844508 tst                          ping.py: using CellId 41719
22:00:40.983170 tst                          ping.py: using BVCI 41720
22:00:41.007833 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:41.018897 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:41.029883 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:41.040793 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:41.051848 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:41.062742 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:41.074085 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
22:00:41.085066 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
22:00:41.363200 tst                       ping.py:19: start network...
22:00:41.488571 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
22:00:41.961771 run         create_hlr_db(pid=31639): Launched
22:00:42.137049 bus                          /gobi_6: Setting Powered False
22:00:43.157952 run         create_hlr_db(pid=31639): Terminated: ok {rc=0}
22:00:43.561886 run pcap-recorder_any(filters='host 10.42.42.2')(pid=31641): Launched
22:00:43.865510 run   osmo-hlr_10.42.42.2(pid=31642): Launched
22:00:43.974248 run              osmo-stp_10.42.42.5: Starting osmo-stp
22:00:44.500551 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=31643): Launched
22:00:44.802985 run   osmo-stp_10.42.42.5(pid=31644): Launched
22:00:44.910469 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
22:00:45.437219 run pcap-recorder_any(filters='host 10.42.42.6')(pid=31645): Launched
22:00:45.544007 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
22:00:45.806721 run              patchelf(pid=31646): Launched
22:00:46.987464 run              patchelf(pid=31646): Terminated: ok {rc=0}
22:00:47.098236 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
22:00:47.368989 run osmo-gsm-tester_setcap_net_admin.sh(pid=31647): Launched
22:00:48.549359 run osmo-gsm-tester_setcap_net_admin.sh(pid=31647): Terminated: ok {rc=0}
22:00:48.853069 run  osmo-ggsn_10.42.42.6(pid=31650): Launched
22:00:49.103193 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
22:00:49.646299 run pcap-recorder_any(filters='host 10.42.42.7')(pid=31667): Launched
22:00:49.947812 run  osmo-sgsn_10.42.42.7(pid=31668): Launched
22:00:50.059002 run              osmo-msc_10.42.42.8: Starting osmo-msc
22:00:50.611710 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=31669): Launched
22:00:51.049110 run   osmo-msc_10.42.42.8(pid=31670): Launched
22:00:54.237425 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
22:00:54.758619 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=31671): Launched
22:00:55.059534 run   osmo-mgw_10.42.42.3(pid=31672): Launched
22:00:55.167978 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
22:00:55.681160 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=31673): Launched
22:00:55.983446 run   osmo-mgw_10.42.42.4(pid=31674): Launched
22:00:56.090849 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
22:00:56.733027 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=31675): Launched
22:00:57.035979 run   osmo-bsc_10.42.42.9(pid=31676): Launched
22:01:03.294183 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
22:01:03.830260 run         rm-remote-dir(pid=31679): Launched
22:01:07.031464 run         rm-remote-dir(pid=31679): ERR: Terminated: ERROR {rc=255}  [trial-2071↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=31679)]
22:01:07.177611 run         rm-remote-dir(pid=31679): stdout: 
| (launched: 2023-01-03_22:01:03.675511) 
22:01:07.320074 run         rm-remote-dir(pid=31679): stderr: 
| (launched: 2023-01-03_22:01:03.723456)
| ssh: connect to host 10.42.42.114 port 22: No route to host 
22:01:07.378642 tst                       ping.py:29: ERR: Error: rm-remote-dir(pid=31679): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2071↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=31679)]  [trial-2071↪gprs:sysmo↪ping.py:29]
22:01:07.386320 tst                       ping.py:29: Test FAILED (26.9 sec)
22:01:07.423159 run pcap-recorder_any(filters='host 10.42.42.2')(pid=31641): Terminating (SIGTERM)
22:01:07.465706 run   osmo-hlr_10.42.42.2(pid=31642): Terminating (SIGTERM)
22:01:07.509236 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=31643): Terminating (SIGTERM)
22:01:07.554764 run   osmo-stp_10.42.42.5(pid=31644): Terminating (SIGTERM)
22:01:07.609707 run pcap-recorder_any(filters='host 10.42.42.6')(pid=31645): Terminating (SIGTERM)
22:01:07.665250 run  osmo-ggsn_10.42.42.6(pid=31650): Terminating (SIGTERM)
22:01:07.730246 run pcap-recorder_any(filters='host 10.42.42.7')(pid=31667): Terminating (SIGTERM)
22:01:07.796585 run  osmo-sgsn_10.42.42.7(pid=31668): Terminating (SIGTERM)
22:01:07.841442 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=31669): Terminating (SIGTERM)
22:01:07.887182 run   osmo-msc_10.42.42.8(pid=31670): Terminating (SIGTERM)
22:01:07.932132 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=31671): Terminating (SIGTERM)
22:01:07.977656 run   osmo-mgw_10.42.42.3(pid=31672): Terminating (SIGTERM)
22:01:08.021394 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=31673): Terminating (SIGTERM)
22:01:08.063507 run   osmo-mgw_10.42.42.4(pid=31674): Terminating (SIGTERM)
22:01:08.106695 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=31675): Terminating (SIGTERM)
22:01:08.149362 run   osmo-bsc_10.42.42.9(pid=31676): Terminating (SIGTERM)
22:01:08.160315 ---      ParallelTerminationStrategy: PID 31641 died...
22:01:08.218495 run pcap-recorder_any(filters='host 10.42.42.2')(pid=31641): Terminated: ok {rc=0}
22:01:08.229815 ---      ParallelTerminationStrategy: PID 31642 died...
22:01:08.305681 run   osmo-hlr_10.42.42.2(pid=31642): Terminated: ok {rc=0}
22:01:08.333110 ---      ParallelTerminationStrategy: PID 31643 died...
22:01:08.427877 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=31643): Terminated: ok {rc=0}
22:01:08.438635 ---      ParallelTerminationStrategy: PID 31644 died...
22:01:08.495847 run   osmo-stp_10.42.42.5(pid=31644): Terminated: ok {rc=0}
22:01:08.506610 ---      ParallelTerminationStrategy: PID 31645 died...
22:01:08.563171 run pcap-recorder_any(filters='host 10.42.42.6')(pid=31645): Terminated: ok {rc=0}
22:01:08.573579 ---      ParallelTerminationStrategy: PID 31650 died...
22:01:08.629725 run  osmo-ggsn_10.42.42.6(pid=31650): Terminated: ok {rc=0}
22:01:08.640054 ---      ParallelTerminationStrategy: PID 31667 died...
22:01:08.696226 run pcap-recorder_any(filters='host 10.42.42.7')(pid=31667): Terminated: ok {rc=0}
22:01:08.706604 ---      ParallelTerminationStrategy: PID 31669 died...
22:01:08.762653 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=31669): Terminated: ok {rc=0}
22:01:08.772998 ---      ParallelTerminationStrategy: PID 31670 died...
22:01:08.829365 run   osmo-msc_10.42.42.8(pid=31670): Terminated: ok {rc=0}
22:01:08.840092 ---      ParallelTerminationStrategy: PID 31671 died...
22:01:08.897649 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=31671): Terminated: ok {rc=0}
22:01:08.908562 ---      ParallelTerminationStrategy: PID 31672 died...
22:01:08.966020 run   osmo-mgw_10.42.42.3(pid=31672): Terminated {rc=15}
22:01:08.976885 ---      ParallelTerminationStrategy: PID 31673 died...
22:01:09.034744 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=31673): Terminated: ok {rc=0}
22:01:09.045772 ---      ParallelTerminationStrategy: PID 31674 died...
22:01:09.103777 run   osmo-mgw_10.42.42.4(pid=31674): Terminated {rc=15}
22:01:09.114589 ---      ParallelTerminationStrategy: PID 31675 died...
22:01:09.172308 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=31675): Terminated: ok {rc=0}
22:01:09.183111 ---      ParallelTerminationStrategy: PID 31676 died...
22:01:09.240550 run   osmo-bsc_10.42.42.9(pid=31676): Terminated: ok {rc=0}
22:01:09.726068 ---      ParallelTerminationStrategy: PID 31668 died...
22:01:09.784012 run  osmo-sgsn_10.42.42.7(pid=31668): Terminated: ok {rc=0}
22:01:09.871500 bus                          /gobi_6: Setting Powered False

Standard Error

Traceback (most recent call last):
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
    self.path)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
    spec.loader.exec_module( importlib.util.module_from_spec(spec) )
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/ping.py", line 29, in <module>
    bts.start()
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/bts_sysmo.py", line 113, in start
    rem_host.recreate_remote_dir(self.remote_inst)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 146, in recreate_remote_dir
    self.rm_remote_dir(remote_dir)
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 139, in rm_remote_dir
    self.run_remote_sync('rm-remote-dir', ('test', '!', '-d', remote_dir_str, '||', 'rm', '-rf', remote_dir_str))
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/remote.py", line 134, in run_remote_sync
    proc.launch_sync()
  File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync
    raise self.RunError('launch_sync()')
osmo_gsm_tester.core.log.Error: rm-remote-dir(pid=31679): launch_sync(): local ssh process exited with status 255 (ssh: connect to host 10.42.42.114 port 22: No route to host) [trial-2071↪gprs:sysmo↪ping.py:29↪ping.py↪osmo-bts-sysmo↪host-root@10.42.42.114↪rm-remote-dir(pid=31679)]