![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
Regression
gprs:sysmo.ping_idle_ping.py (from gprs_sysmo)
Failing for the past 1 build
(Since Failed )
Stacktrace
ping1(pid=21659): launch_sync(): local process exited with status 1 [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61↪ping_idle_ping.py↪/gobi_6↪ping1(pid=21659)]
Standard Output
----------------------------------------------
trial-2067 gprs:sysmo ping_idle_ping.py
----------------------------------------------
12:20:30.081796 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.093738 tst gprs:sysmo: Using 1 x bts (candidates: 1)
12:20:30.224222 tst ping_idle_ping.py: using LAC 3970
12:20:30.350047 tst ping_idle_ping.py: using RAC 145
12:20:30.479342 tst ping_idle_ping.py: using CellId 3970
12:20:30.603345 tst ping_idle_ping.py: using BVCI 3971
12:20:30.628030 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.639215 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.650216 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.661311 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.672282 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.683508 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.695322 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
12:20:30.706312 tst gprs:sysmo: Using 1 x modem (candidates: 4)
12:20:30.998841 tst ping_idle_ping.py:24: start network...
12:20:31.132930 run osmo-hlr_10.42.42.2: Starting osmo-hlr
12:20:31.648079 run create_hlr_db(pid=21574): Launched
12:20:31.834199 bus /gobi_6: Setting Powered False
12:20:32.855130 run create_hlr_db(pid=21574): Terminated: ok {rc=0}
12:20:33.265484 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21576): Launched
12:20:33.570023 run osmo-hlr_10.42.42.2(pid=21577): Launched
12:20:33.678245 run osmo-stp_10.42.42.5: Starting osmo-stp
12:20:34.203630 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21578): Launched
12:20:34.508282 run osmo-stp_10.42.42.5(pid=21579): Launched
12:20:34.614871 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
12:20:35.142177 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21580): Launched
12:20:35.249308 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
12:20:35.515597 run patchelf(pid=21581): Launched
12:20:36.695157 run patchelf(pid=21581): Terminated: ok {rc=0}
12:20:36.805028 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
12:20:37.076568 run osmo-gsm-tester_setcap_net_admin.sh(pid=21582): Launched
12:20:38.266521 run osmo-gsm-tester_setcap_net_admin.sh(pid=21582): Terminated: ok {rc=0}
12:20:38.593356 run osmo-ggsn_10.42.42.6(pid=21585): Launched
12:20:38.829248 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
12:20:39.401077 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21607): Launched
12:20:39.722163 run osmo-sgsn_10.42.42.7(pid=21608): Launched
12:20:39.835824 run osmo-msc_10.42.42.8: Starting osmo-msc
12:20:40.395235 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21609): Launched
12:20:40.702796 run osmo-msc_10.42.42.8(pid=21610): Launched
12:20:43.864705 run osmo-mgw_10.42.42.3: Starting osmo-mgw
12:20:44.392686 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21611): Launched
12:20:44.699853 run osmo-mgw_10.42.42.3(pid=21612): Launched
12:20:44.807625 run osmo-mgw_10.42.42.4: Starting osmo-mgw
12:20:45.325923 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21613): Launched
12:20:45.633601 run osmo-mgw_10.42.42.4(pid=21614): Launched
12:20:45.741647 run osmo-bsc_10.42.42.9: Starting osmo-bsc
12:20:46.380496 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21615): Launched
12:20:46.690954 run osmo-bsc_10.42.42.9(pid=21616): Launched
12:20:52.949756 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
12:20:53.494863 run rm-remote-dir(pid=21617): Launched
12:20:55.684836 run rm-remote-dir(pid=21617): Terminated: ok {rc=0}
12:20:56.012804 run mk-remote-dir(pid=21618): Launched
12:20:57.200402 run mk-remote-dir(pid=21618): Terminated: ok {rc=0}
12:20:57.476386 run scp-inst-to-remote(pid=21619): Launched
12:21:09.684733 run scp-inst-to-remote(pid=21619): Terminated: ok {rc=0}
12:21:10.027054 run mk-remote-dir(pid=21621): Launched
12:21:11.224531 run mk-remote-dir(pid=21621): Terminated: ok {rc=0}
12:21:11.517682 run scp-cfg-to-remote(pid=21622): Launched
12:21:12.712754 run scp-cfg-to-remote(pid=21622): Terminated: ok {rc=0}
12:21:13.051428 run reload-dsp-firmware(pid=21624): Launched
12:21:15.247274 run reload-dsp-firmware(pid=21624): Terminated: ok {rc=0}
12:21:15.576991 run osmo-bts-sysmo(pid=21625): Launched
12:21:19.889296 tst ping_idle_ping.py:36: Waiting for bts to be ready...
12:21:21.530687 run rm-remote-dir(pid=21626): Launched
12:21:23.722588 run rm-remote-dir(pid=21626): Terminated: ok {rc=0}
12:21:24.054907 run mk-remote-dir(pid=21627): Launched
12:21:26.237751 run mk-remote-dir(pid=21627): Terminated: ok {rc=0}
12:21:26.506057 run scp-inst-to-bts(pid=21628): Launched
12:21:39.697797 run scp-inst-to-bts(pid=21628): Terminated: ok {rc=0}
12:21:40.011612 run mk-remote-run-dir(pid=21631): Launched
12:21:42.192933 run mk-remote-run-dir(pid=21631): Terminated: ok {rc=0}
12:21:42.460275 run scp-cfg-to-sysmobts(pid=21632): Launched
12:21:44.639531 run scp-cfg-to-sysmobts(pid=21632): Terminated: ok {rc=0}
12:21:44.950382 run osmo-pcu(pid=21634): Launched
12:21:45.068816 tst ping_idle_ping.py: using MSISDN 3990
12:21:45.247220 bus /gobi_6: Setting Powered True
12:21:48.761924 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3990', subscriber_id=1}
12:21:49.035308 bus /gobi_6: Setting Online False
12:21:50.228343 bus /gobi_6: Setting Powered False
12:21:57.202252 bus /gobi_6: Setting Powered True
12:21:58.607077 bus /gobi_6: Setting Online True
12:22:01.011173 tst /gobi_6: Connect to ('901', '70')
12:22:01.288601 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
12:22:01.475058 tst ping_idle_ping.py:47: waiting for modems to attach...
12:22:08.048265 tst /gobi_6: Already registered with network ('901', '70')
12:22:19.916718 tst ping_idle_ping.py:51: waiting for modems to attach to data services...
12:22:43.918517 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
12:22:44.115798 tst ping_idle_ping.py:56: Setting up data plane for "'/gobi_6/context2'"
12:22:44.497949 run move_netns(pid=21637): Launched
12:22:45.692824 run move_netns(pid=21637): Terminated: ok {rc=0}
12:22:46.042547 run ifup(pid=21640): Launched
12:22:47.238063 run ifup(pid=21640): Terminated: ok {rc=0}
12:22:47.586950 run dhcp(pid=21643): Launched
12:22:48.770066 run dhcp(pid=21643): Terminated: ok {rc=0}
12:22:48.961262 tst ping_idle_ping.py:60: [1] Running 10 ping requests for "'/gobi_6/context2'"
12:22:49.306102 run ping1(pid=21659): Launched
12:23:09.530912 run ping1(pid=21659): ERR: Terminated: ERROR {rc=1} [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61↪ping_idle_ping.py↪/gobi_6↪ping1(pid=21659)]
12:23:09.668304 run ping1(pid=21659): stdout:
| (launched: 2022-09-08_12:22:49.140188)
| 21660: waiting for 21661
| PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
|
| --- 10.42.42.6 ping statistics ---
| 10 packets transmitted, 0 received, 100% packet loss, time 204ms
|
| child exited with 1
12:23:09.806443 run ping1(pid=21659): stderr:
| (launched: 2022-09-08_12:22:49.187567)
12:23:09.861444 tst ping_idle_ping.py:61: ERR: Error: ping1(pid=21659): launch_sync(): local process exited with status 1 [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61↪ping_idle_ping.py↪/gobi_6↪ping1(pid=21659)] [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61]
12:23:09.868406 tst ping_idle_ping.py:61: Test FAILED (159.8 sec)
12:23:09.905283 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21576): Terminating (SIGTERM)
12:23:09.947827 run osmo-hlr_10.42.42.2(pid=21577): Terminating (SIGTERM)
12:23:09.990852 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21578): Terminating (SIGTERM)
12:23:10.036640 run osmo-stp_10.42.42.5(pid=21579): Terminating (SIGTERM)
12:23:10.101770 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21580): Terminating (SIGTERM)
12:23:10.145882 run osmo-ggsn_10.42.42.6(pid=21585): Terminating (SIGTERM)
12:23:10.238645 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21607): Terminating (SIGTERM)
12:23:10.288469 run osmo-sgsn_10.42.42.7(pid=21608): Terminating (SIGTERM)
12:23:10.334358 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21609): Terminating (SIGTERM)
12:23:10.377543 run osmo-msc_10.42.42.8(pid=21610): Terminating (SIGTERM)
12:23:10.419807 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21611): Terminating (SIGTERM)
12:23:10.460589 run osmo-mgw_10.42.42.3(pid=21612): Terminating (SIGTERM)
12:23:10.501691 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21613): Terminating (SIGTERM)
12:23:10.542212 run osmo-mgw_10.42.42.4(pid=21614): Terminating (SIGTERM)
12:23:10.583362 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21615): Terminating (SIGTERM)
12:23:10.624246 run osmo-bsc_10.42.42.9(pid=21616): Terminating (SIGTERM)
12:23:10.664918 run osmo-bts-sysmo(pid=21625): Terminating (SIGTERM)
12:23:10.706050 run osmo-pcu(pid=21634): Terminating (SIGTERM)
12:23:10.715895 --- ParallelTerminationStrategy: PID 21576 died...
12:23:10.771244 run pcap-recorder_any(filters='host 10.42.42.2')(pid=21576): Terminated: ok {rc=0}
12:23:10.781773 --- ParallelTerminationStrategy: PID 21577 died...
12:23:10.839163 run osmo-hlr_10.42.42.2(pid=21577): Terminated: ok {rc=0}
12:23:10.849789 --- ParallelTerminationStrategy: PID 21578 died...
12:23:10.943508 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=21578): Terminated: ok {rc=0}
12:23:10.971556 --- ParallelTerminationStrategy: PID 21579 died...
12:23:11.046159 run osmo-stp_10.42.42.5(pid=21579): Terminated: ok {rc=0}
12:23:11.056810 --- ParallelTerminationStrategy: PID 21580 died...
12:23:11.114861 run pcap-recorder_any(filters='host 10.42.42.6')(pid=21580): Terminated: ok {rc=0}
12:23:11.125221 --- ParallelTerminationStrategy: PID 21585 died...
12:23:11.183076 run osmo-ggsn_10.42.42.6(pid=21585): Terminated: ok {rc=0}
12:23:11.193499 --- ParallelTerminationStrategy: PID 21607 died...
12:23:11.251481 run pcap-recorder_any(filters='host 10.42.42.7')(pid=21607): Terminated: ok {rc=0}
12:23:11.262064 --- ParallelTerminationStrategy: PID 21609 died...
12:23:11.320507 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=21609): Terminated: ok {rc=0}
12:23:11.331569 --- ParallelTerminationStrategy: PID 21610 died...
12:23:11.390381 run osmo-msc_10.42.42.8(pid=21610): Terminated: ok {rc=0}
12:23:11.401289 --- ParallelTerminationStrategy: PID 21611 died...
12:23:11.460390 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=21611): Terminated: ok {rc=0}
12:23:11.471938 --- ParallelTerminationStrategy: PID 21612 died...
12:23:11.531311 run osmo-mgw_10.42.42.3(pid=21612): Terminated {rc=15}
12:23:11.542189 --- ParallelTerminationStrategy: PID 21613 died...
12:23:11.601839 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=21613): Terminated: ok {rc=0}
12:23:11.612879 --- ParallelTerminationStrategy: PID 21614 died...
12:23:11.672178 run osmo-mgw_10.42.42.4(pid=21614): Terminated {rc=15}
12:23:11.683160 --- ParallelTerminationStrategy: PID 21615 died...
12:23:11.742887 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=21615): Terminated: ok {rc=0}
12:23:11.753910 --- ParallelTerminationStrategy: PID 21616 died...
12:23:11.813419 run osmo-bsc_10.42.42.9(pid=21616): Terminated: ok {rc=0}
12:23:11.824438 --- ParallelTerminationStrategy: PID 21625 died...
12:23:11.884079 run osmo-bts-sysmo(pid=21625): Terminated: ok {rc=0}
12:23:11.895189 --- ParallelTerminationStrategy: PID 21634 died...
12:23:11.954287 run osmo-pcu(pid=21634): Terminated: ok {rc=0}
12:23:12.133061 --- ParallelTerminationStrategy: PID 21608 died...
12:23:12.192455 run osmo-sgsn_10.42.42.7(pid=21608): Terminated: ok {rc=0}
12:23:12.418659 bus /gobi_6: Setting Online False
12:23:16.343470 bus /gobi_6: Setting Powered False
12:23:33.134040 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/ping_idle_ping.py", line 61, in <module>
proc = ms.run_netns_wait('ping1', ('ping', '-c', '10', ggsn.addr()))
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/ms_ofono.py", line 691, in run_netns_wait
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 216, in launch_sync
raise self.RunError('launch_sync()')
osmo_gsm_tester.core.log.Error: ping1(pid=21659): launch_sync(): local process exited with status 1 [trial-2067↪gprs:sysmo↪ping_idle_ping.py:61↪ping_idle_ping.py↪/gobi_6↪ping1(pid=21659)]