![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
Failed
gprs:sysmo.iperf3m4.py (from gprs_sysmo)
Failing for the past 1 build
(Since Failed )
Stacktrace
'sum_sent'
Standard Output
----------------------------------------------
trial-2063 gprs:sysmo iperf3m4.py
----------------------------------------------
14:57:05.142947 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.154084 tst gprs:sysmo: Using 1 x bts (candidates: 1)
14:57:05.285984 tst iperf3m4.py: using LAC 3940
14:57:05.408861 tst iperf3m4.py: using RAC 115
14:57:05.530496 tst iperf3m4.py: using CellId 3940
14:57:05.651203 tst iperf3m4.py: using BVCI 3941
14:57:05.674472 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.685058 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.695476 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.705761 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.715972 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.726283 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.736695 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.747414 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
14:57:05.758050 tst gprs:sysmo: Using 1 x modem (candidates: 4)
14:57:05.878210 tst gprs:sysmo: Using 1 x modem (candidates: 4)
14:57:06.000595 tst gprs:sysmo: Using 1 x modem (candidates: 4)
14:57:06.116631 tst gprs:sysmo: Using 1 x modem (candidates: 4)
14:57:06.412882 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
14:57:06.552176 run iperf3-srv_10.42.42.10: Starting iperf3-srv
14:57:06.893952 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7502): Launched
14:57:07.178491 run iperf3-srv_10.42.42.10(pid=7503): Launched
14:57:07.380256 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
14:57:07.518211 run iperf3-srv_10.42.42.10: Starting iperf3-srv
14:57:07.858523 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7504): Launched
14:57:08.142518 run iperf3-srv_10.42.42.10(pid=7505): Launched
14:57:08.344907 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
14:57:08.483184 run iperf3-srv_10.42.42.10: Starting iperf3-srv
14:57:08.825905 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7506): Launched
14:57:09.109532 run iperf3-srv_10.42.42.10(pid=7507): Launched
14:57:09.310775 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
14:57:09.447848 run iperf3-srv_10.42.42.10: Starting iperf3-srv
14:57:09.789556 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7508): Launched
14:57:10.073547 run iperf3-srv_10.42.42.10(pid=7509): Launched
14:57:10.275173 tst iperf3m4.py:8: start network...
14:57:10.412219 run osmo-hlr_10.42.42.2: Starting osmo-hlr
14:57:10.933006 run create_hlr_db(pid=7510): Launched
14:57:11.123595 bus /gobi_6: Setting Powered False
14:57:12.144779 run create_hlr_db(pid=7510): Terminated: ok {rc=0}
14:57:12.583491 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7512): Launched
14:57:12.915648 run osmo-hlr_10.42.42.2(pid=7513): Launched
14:57:13.034340 run osmo-stp_10.42.42.5: Starting osmo-stp
14:57:13.603600 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7514): Launched
14:57:13.933630 run osmo-stp_10.42.42.5(pid=7515): Launched
14:57:14.050391 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
14:57:14.622318 run pcap-recorder_any(filters='host 10.42.42.6')(pid=7516): Launched
14:57:14.739855 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
14:57:15.025241 run patchelf(pid=7517): Launched
14:57:16.220041 run patchelf(pid=7517): Terminated: ok {rc=0}
14:57:16.340841 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
14:57:16.632866 run osmo-gsm-tester_setcap_net_admin.sh(pid=7518): Launched
14:57:17.827931 run osmo-gsm-tester_setcap_net_admin.sh(pid=7518): Terminated: ok {rc=0}
14:57:18.162582 run osmo-ggsn_10.42.42.6(pid=7521): Launched
14:57:18.400450 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
14:57:18.964444 run pcap-recorder_any(filters='host 10.42.42.7')(pid=7541): Launched
14:57:19.271312 run osmo-sgsn_10.42.42.7(pid=7542): Launched
14:57:19.383735 run osmo-msc_10.42.42.8: Starting osmo-msc
14:57:19.987766 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=7543): Launched
14:57:20.328931 run osmo-msc_10.42.42.8(pid=7544): Launched
14:57:23.492761 run osmo-mgw_10.42.42.3: Starting osmo-mgw
14:57:24.024842 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7545): Launched
14:57:24.332342 run osmo-mgw_10.42.42.3(pid=7546): Launched
14:57:24.441405 run osmo-mgw_10.42.42.4: Starting osmo-mgw
14:57:24.964696 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7547): Launched
14:57:25.274198 run osmo-mgw_10.42.42.4(pid=7548): Launched
14:57:25.383238 run osmo-bsc_10.42.42.9: Starting osmo-bsc
14:57:26.028522 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=7549): Launched
14:57:26.340009 run osmo-bsc_10.42.42.9(pid=7550): Launched
14:57:32.603372 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
14:57:33.156303 run rm-remote-dir(pid=7551): Launched
14:57:35.348022 run rm-remote-dir(pid=7551): Terminated: ok {rc=0}
14:57:35.679170 run mk-remote-dir(pid=7552): Launched
14:57:36.868021 run mk-remote-dir(pid=7552): Terminated: ok {rc=0}
14:57:37.146814 run scp-inst-to-remote(pid=7553): Launched
14:57:49.350286 run scp-inst-to-remote(pid=7553): Terminated: ok {rc=0}
14:57:49.675327 run mk-remote-dir(pid=7566): Launched
14:57:50.876374 run mk-remote-dir(pid=7566): Terminated: ok {rc=0}
14:57:51.172903 run scp-cfg-to-remote(pid=7567): Launched
14:57:52.371586 run scp-cfg-to-remote(pid=7567): Terminated: ok {rc=0}
14:57:52.713967 run reload-dsp-firmware(pid=7569): Launched
14:57:54.913290 run reload-dsp-firmware(pid=7569): Terminated: ok {rc=0}
14:57:55.247933 run osmo-bts-sysmo(pid=7570): Launched
14:57:59.566608 tst iperf3m4.py:8: Waiting for bts to be ready...
14:58:01.215936 run rm-remote-dir(pid=7571): Launched
14:58:03.412090 run rm-remote-dir(pid=7571): Terminated: ok {rc=0}
14:58:03.750657 run mk-remote-dir(pid=7572): Launched
14:58:05.945494 run mk-remote-dir(pid=7572): Terminated: ok {rc=0}
14:58:06.235187 run scp-inst-to-bts(pid=7573): Launched
14:58:19.452976 run scp-inst-to-bts(pid=7573): Terminated: ok {rc=0}
14:58:19.792263 run mk-remote-run-dir(pid=7575): Launched
14:58:21.987269 run mk-remote-run-dir(pid=7575): Terminated: ok {rc=0}
14:58:22.276928 run scp-cfg-to-sysmobts(pid=7576): Launched
14:58:24.474262 run scp-cfg-to-sysmobts(pid=7576): Terminated: ok {rc=0}
14:58:24.812158 run osmo-pcu(pid=7578): Launched
14:58:24.943116 tst iperf3m4.py: using MSISDN 3941
14:58:25.114779 bus /gobi_6: Setting Powered True
14:58:28.649791 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3941', subscriber_id=1}
14:58:28.942153 bus /gobi_6: Setting Online False
14:58:30.129798 bus /gobi_6: Setting Powered False
14:58:37.233053 bus /gobi_6: Setting Powered True
14:58:38.609321 bus /gobi_6: Setting Online True
14:58:41.052330 tst /gobi_6: Connect to ('901', '70')
14:58:41.339881 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
14:58:41.471904 tst iperf3m4.py: using MSISDN 3942
14:58:41.753082 bus /gobi_0: Setting Powered True
14:58:45.254516 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3942', subscriber_id=2}
14:58:45.528846 bus /gobi_0: Setting Online False
14:58:46.724656 bus /gobi_0: Setting Powered False
14:58:48.226616 tst /gobi_6: Already registered with network ('901', '70')
14:58:54.212561 bus /gobi_0: Setting Powered True
14:58:55.584423 bus /gobi_0: Setting Online True
14:58:58.157401 tst /gobi_0: Connect to ('901', '70')
14:58:58.440418 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873'
14:58:58.565660 tst iperf3m4.py: using MSISDN 3943
14:58:58.856740 bus /gobi_1: Setting Powered True
14:59:02.670251 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3943', subscriber_id=3}
14:59:02.970137 bus /gobi_1: Setting Online False
14:59:04.156341 bus /gobi_1: Setting Powered False
14:59:11.225700 bus /gobi_1: Setting Powered True
14:59:12.625478 bus /gobi_1: Setting Online True
14:59:14.077529 tst /gobi_0: Already registered with network ('901', '70')
14:59:15.092529 tst /gobi_1: Connect to ('901', '70')
14:59:15.361081 tst /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024272103'
14:59:15.483904 tst iperf3m4.py: using MSISDN 3944
14:59:15.716849 bus /gobi_4: Setting Powered True
14:59:19.487131 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3944', subscriber_id=4}
14:59:19.824159 bus /gobi_4: Setting Online False
14:59:21.012633 bus /gobi_4: Setting Powered False
14:59:24.281798 tst /gobi_1: Already registered with network ('901', '70')
14:59:30.347354 bus /gobi_4: Setting Powered True
14:59:32.109426 bus /gobi_4: Setting Online True
14:59:34.516169 tst /gobi_4: Connect to ('901', '70')
14:59:34.779403 tst /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G 1 [Jul 21 2016 10:00:50]', Serial: '868323025161511'
14:59:34.965871 tst iperf3m4.py:8: waiting for modems to attach...
14:59:43.843043 tst /gobi_4: Already registered with network ('901', '70')
14:59:47.762384 tst iperf3m4.py:8: waiting for modems to attach to data services...
14:59:56.152488 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
14:59:56.342042 tst iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'"
14:59:56.703181 run move_netns(pid=7582): Launched
14:59:57.887630 run move_netns(pid=7582): Terminated: ok {rc=0}
14:59:58.221551 run ifup(pid=7585): Launched
14:59:59.417665 run ifup(pid=7585): Terminated: ok {rc=0}
14:59:59.761122 run dhcp(pid=7588): Launched
15:00:00.953640 run dhcp(pid=7588): Terminated: ok {rc=0}
15:00:09.275134 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
15:00:09.474446 tst iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'"
15:00:09.844945 run move_netns(pid=7639): Launched
15:00:11.040307 run move_netns(pid=7639): Terminated: ok {rc=0}
15:00:11.382551 run ifup(pid=7642): Launched
15:00:12.581356 run ifup(pid=7642): Terminated: ok {rc=0}
15:00:12.966618 run dhcp(pid=7758): Launched
15:00:14.171394 run dhcp(pid=7758): Terminated: ok {rc=0}
15:00:17.280235 tst /gobi_1: context activated {apn='inet46', path='/gobi_1/context8', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'}
15:00:17.489249 tst iperf3m4.py:8: Setting up data plan for "'/gobi_1/context8'"
15:00:17.901878 run move_netns(pid=8162): Launched
15:00:19.090335 run move_netns(pid=8162): Terminated: ok {rc=0}
15:00:19.475998 run ifup(pid=8167): Launched
15:00:20.661338 run ifup(pid=8167): Terminated: ok {rc=0}
15:00:20.992737 run dhcp(pid=8171): Launched
15:00:22.187445 run dhcp(pid=8171): Terminated: ok {rc=0}
15:00:25.280098 tst /gobi_4: context activated {apn='inet46', path='/gobi_4/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
15:00:25.480497 tst iperf3m4.py:8: Setting up data plan for "'/gobi_4/context3'"
15:00:25.861082 run move_netns(pid=8187): Launched
15:00:27.055013 run move_netns(pid=8187): Terminated: ok {rc=0}
15:00:27.396959 run ifup(pid=8192): Launched
15:00:28.592469 run ifup(pid=8192): Terminated: ok {rc=0}
15:00:28.935930 run dhcp(pid=8195): Launched
15:00:30.118339 run dhcp(pid=8195): Terminated: ok {rc=0}
15:00:30.311871 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'"
15:00:30.441692 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
15:00:30.832538 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211): Launched
15:00:31.087125 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'"
15:00:31.227368 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s)
15:00:31.629854 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214): Launched
15:00:31.867464 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context8'"
15:00:31.997979 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s)
15:00:32.378224 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217): Launched
15:00:32.615251 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context3'"
15:00:32.744548 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s)
15:00:33.121033 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220): Launched
15:00:33.436181 run iperf3-cli_10.42.42.10(pid=8223): Launched
15:00:33.709012 run iperf3-cli_10.42.42.10(pid=8226): Launched
15:00:33.982758 run iperf3-cli_10.42.42.10(pid=8229): Launched
15:00:34.254084 run iperf3-cli_10.42.42.10(pid=8232): Launched
15:01:48.580595 run iperf3-cli_10.42.42.10(pid=8223): ERR: Terminated: ERROR {rc=1} [trial-2063↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=8223)]
15:01:48.716453 run iperf3-cli_10.42.42.10(pid=8223): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
15:01:48.851243 run iperf3-cli_10.42.42.10(pid=8223): stderr:
| (launched: 2022-02-24_15:00:33.325873)
15:01:50.041884 run iperf3-cli_10.42.42.10(pid=8226): ERR: Terminated: ERROR {rc=1} [trial-2063↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=8226)]
15:01:50.177302 run iperf3-cli_10.42.42.10(pid=8226): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
15:01:50.312217 run iperf3-cli_10.42.42.10(pid=8226): stderr:
| (launched: 2022-02-24_15:00:33.595607)
15:01:51.501975 run iperf3-cli_10.42.42.10(pid=8229): ERR: Terminated: ERROR {rc=1} [trial-2063↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=8229)]
15:01:51.637844 run iperf3-cli_10.42.42.10(pid=8229): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
15:01:51.773410 run iperf3-cli_10.42.42.10(pid=8229): stderr:
| (launched: 2022-02-24_15:00:33.871204)
15:01:52.970974 run iperf3-cli_10.42.42.10(pid=8232): ERR: Terminated: ERROR {rc=1} [trial-2063↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=8232)]
15:01:53.107355 run iperf3-cli_10.42.42.10(pid=8232): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
15:01:53.242850 run iperf3-cli_10.42.42.10(pid=8232): stderr:
| (launched: 2022-02-24_15:00:34.144755)
15:01:53.367810 run iperf3-srv_10.42.42.10(pid=7503): Terminating (SIGINT)
15:01:53.544203 run iperf3-srv_10.42.42.10(pid=7503): Terminated {rc=1}
15:01:53.744508 tst iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'"
15:01:53.803366 tst iperf3m4.py:8: ERR: KeyError: 'sum_sent' [trial-2063↪gprs:sysmo↪iperf3m4.py:8]
15:01:53.810817 tst iperf3m4.py:8: Test FAILED (288.7 sec)
15:01:53.847724 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7502): Terminating (SIGTERM)
15:01:53.889851 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7504): Terminating (SIGTERM)
15:01:53.931496 run iperf3-srv_10.42.42.10(pid=7505): Terminating (SIGTERM)
15:01:53.973277 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7506): Terminating (SIGTERM)
15:01:54.014533 run iperf3-srv_10.42.42.10(pid=7507): Terminating (SIGTERM)
15:01:54.055655 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7508): Terminating (SIGTERM)
15:01:54.097200 run iperf3-srv_10.42.42.10(pid=7509): Terminating (SIGTERM)
15:01:54.138843 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7512): Terminating (SIGTERM)
15:01:54.180030 run osmo-hlr_10.42.42.2(pid=7513): Terminating (SIGTERM)
15:01:54.222674 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7514): Terminating (SIGTERM)
15:01:54.264952 run osmo-stp_10.42.42.5(pid=7515): Terminating (SIGTERM)
15:01:54.316643 run pcap-recorder_any(filters='host 10.42.42.6')(pid=7516): Terminating (SIGTERM)
15:01:54.359686 run osmo-ggsn_10.42.42.6(pid=7521): Terminating (SIGTERM)
15:01:54.414640 run pcap-recorder_any(filters='host 10.42.42.7')(pid=7541): Terminating (SIGTERM)
15:01:54.475353 run osmo-sgsn_10.42.42.7(pid=7542): Terminating (SIGTERM)
15:01:54.523364 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=7543): Terminating (SIGTERM)
15:01:54.571878 run osmo-msc_10.42.42.8(pid=7544): Terminating (SIGTERM)
15:01:54.623399 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7545): Terminating (SIGTERM)
15:01:54.672342 run osmo-mgw_10.42.42.3(pid=7546): Terminating (SIGTERM)
15:01:54.715647 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7547): Terminating (SIGTERM)
15:01:54.761157 run osmo-mgw_10.42.42.4(pid=7548): Terminating (SIGTERM)
15:01:54.806608 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=7549): Terminating (SIGTERM)
15:01:54.850424 run osmo-bsc_10.42.42.9(pid=7550): Terminating (SIGTERM)
15:01:54.893129 run osmo-bts-sysmo(pid=7570): Terminating (SIGTERM)
15:01:54.937285 run osmo-pcu(pid=7578): Terminating (SIGTERM)
15:01:54.980994 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211): Terminating (SIGTERM)
15:01:55.160509 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211)-killSignals.SIGTERM(pid=8307): Launched
15:01:56.255071 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211)-killSignals.SIGTERM(pid=8307): Terminated: ok {rc=0}
15:01:56.302661 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214): Terminating (SIGTERM)
15:01:56.455382 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214)-killSignals.SIGTERM(pid=8312): Launched
15:01:57.544049 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214)-killSignals.SIGTERM(pid=8312): Terminated: ok {rc=0}
15:01:57.589366 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217): Terminating (SIGTERM)
15:01:57.731844 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217)-killSignals.SIGTERM(pid=8315): Launched
15:01:58.822578 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217)-killSignals.SIGTERM(pid=8315): Terminated: ok {rc=0}
15:01:58.868077 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220): Terminating (SIGTERM)
15:01:59.018303 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220)-killSignals.SIGTERM(pid=8318): Launched
15:02:00.112528 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220)-killSignals.SIGTERM(pid=8318): Terminated: ok {rc=0}
15:02:00.126914 --- ParallelTerminationStrategy: PID 7502 died...
15:02:00.183652 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7502): Terminated: ok {rc=0}
15:02:00.193694 --- ParallelTerminationStrategy: PID 7504 died...
15:02:00.250002 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7504): Terminated: ok {rc=0}
15:02:00.260225 --- ParallelTerminationStrategy: PID 7505 died...
15:02:00.316252 run iperf3-srv_10.42.42.10(pid=7505): Terminated {rc=256}
15:02:00.326140 --- ParallelTerminationStrategy: PID 7506 died...
15:02:00.382130 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7506): Terminated: ok {rc=0}
15:02:00.392139 --- ParallelTerminationStrategy: PID 7507 died...
15:02:00.448230 run iperf3-srv_10.42.42.10(pid=7507): Terminated {rc=256}
15:02:00.458378 --- ParallelTerminationStrategy: PID 7508 died...
15:02:00.514493 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=7508): Terminated: ok {rc=0}
15:02:00.524454 --- ParallelTerminationStrategy: PID 7509 died...
15:02:00.580233 run iperf3-srv_10.42.42.10(pid=7509): Terminated {rc=256}
15:02:00.590440 --- ParallelTerminationStrategy: PID 7512 died...
15:02:00.647178 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7512): Terminated: ok {rc=0}
15:02:00.657159 --- ParallelTerminationStrategy: PID 7513 died...
15:02:00.713236 run osmo-hlr_10.42.42.2(pid=7513): Terminated: ok {rc=0}
15:02:00.723496 --- ParallelTerminationStrategy: PID 7514 died...
15:02:00.780010 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7514): Terminated: ok {rc=0}
15:02:00.790120 --- ParallelTerminationStrategy: PID 7515 died...
15:02:00.846618 run osmo-stp_10.42.42.5(pid=7515): Terminated: ok {rc=0}
15:02:00.856613 --- ParallelTerminationStrategy: PID 7516 died...
15:02:00.913025 run pcap-recorder_any(filters='host 10.42.42.6')(pid=7516): Terminated: ok {rc=0}
15:02:00.923093 --- ParallelTerminationStrategy: PID 7521 died...
15:02:00.979203 run osmo-ggsn_10.42.42.6(pid=7521): Terminated: ok {rc=0}
15:02:00.989157 --- ParallelTerminationStrategy: PID 7541 died...
15:02:01.045652 run pcap-recorder_any(filters='host 10.42.42.7')(pid=7541): Terminated: ok {rc=0}
15:02:01.055672 --- ParallelTerminationStrategy: PID 7542 died...
15:02:01.111930 run osmo-sgsn_10.42.42.7(pid=7542): Terminated: ok {rc=0}
15:02:01.121903 --- ParallelTerminationStrategy: PID 7543 died...
15:02:01.178233 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=7543): Terminated: ok {rc=0}
15:02:01.188318 --- ParallelTerminationStrategy: PID 7544 died...
15:02:01.244125 run osmo-msc_10.42.42.8(pid=7544): Terminated: ok {rc=0}
15:02:01.254258 --- ParallelTerminationStrategy: PID 7545 died...
15:02:01.310757 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7545): Terminated: ok {rc=0}
15:02:01.320720 --- ParallelTerminationStrategy: PID 7546 died...
15:02:01.376806 run osmo-mgw_10.42.42.3(pid=7546): Terminated {rc=15}
15:02:01.386681 --- ParallelTerminationStrategy: PID 7547 died...
15:02:01.443489 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7547): Terminated: ok {rc=0}
15:02:01.453438 --- ParallelTerminationStrategy: PID 7548 died...
15:02:01.509666 run osmo-mgw_10.42.42.4(pid=7548): Terminated {rc=15}
15:02:01.519680 --- ParallelTerminationStrategy: PID 7549 died...
15:02:01.575676 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=7549): Terminated: ok {rc=0}
15:02:01.586309 --- ParallelTerminationStrategy: PID 7550 died...
15:02:01.642032 run osmo-bsc_10.42.42.9(pid=7550): Terminated: ok {rc=0}
15:02:01.652119 --- ParallelTerminationStrategy: PID 7570 died...
15:02:01.708425 run osmo-bts-sysmo(pid=7570): Terminated: ok {rc=0}
15:02:01.718858 --- ParallelTerminationStrategy: PID 7578 died...
15:02:01.774874 run osmo-pcu(pid=7578): Terminated: ok {rc=0}
15:02:07.642330 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211): Terminating (SIGINT)
15:02:07.795985 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211)-killSignals.SIGINT(pid=8321): Launched
15:02:08.890247 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211)-killSignals.SIGINT(pid=8321): Terminated: ok {rc=0}
15:02:08.943398 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214): Terminating (SIGINT)
15:02:09.089401 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214)-killSignals.SIGINT(pid=8324): Launched
15:02:10.178380 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214)-killSignals.SIGINT(pid=8324): Terminated: ok {rc=0}
15:02:10.223362 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217): Terminating (SIGINT)
15:02:10.377120 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217)-killSignals.SIGINT(pid=8327): Launched
15:02:11.470589 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217)-killSignals.SIGINT(pid=8327): Terminated: ok {rc=0}
15:02:11.518248 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220): Terminating (SIGINT)
15:02:11.672212 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220)-killSignals.SIGINT(pid=8330): Launched
15:02:12.766813 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220)-killSignals.SIGINT(pid=8330): Terminated: ok {rc=0}
15:02:18.637862 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211): Terminating (SIGKILL)
15:02:18.790236 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211)-killSignals.SIGUSR1(pid=8333): Launched
15:02:19.884430 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8211)-killSignals.SIGUSR1(pid=8333): Terminated: ok {rc=0}
15:02:19.931754 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214): Terminating (SIGKILL)
15:02:20.083900 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214)-killSignals.SIGUSR1(pid=8336): Launched
15:02:21.178096 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8214)-killSignals.SIGUSR1(pid=8336): Terminated: ok {rc=0}
15:02:21.225458 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217): Terminating (SIGKILL)
15:02:21.380129 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217)-killSignals.SIGUSR1(pid=8339): Launched
15:02:22.469727 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8217)-killSignals.SIGUSR1(pid=8339): Terminated: ok {rc=0}
15:02:22.514633 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220): Terminating (SIGKILL)
15:02:22.661322 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220)-killSignals.SIGUSR1(pid=8342): Launched
15:02:23.751563 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=8220)-killSignals.SIGUSR1(pid=8342): Terminated: ok {rc=0}
15:02:23.985332 bus /gobi_4: Setting Online False
15:02:25.127145 bus /gobi_4: Setting Powered False
15:02:32.981372 bus /gobi_4: Setting Powered False
15:02:34.083212 bus /gobi_4: Setting Powered False
15:02:35.232574 bus /gobi_1: Setting Online False
15:02:36.358060 bus /gobi_1: Setting Powered False
15:02:44.068108 bus /gobi_1: Setting Powered False
15:02:45.170793 bus /gobi_1: Setting Powered False
15:02:46.328132 bus /gobi_0: Setting Online False
15:02:47.460572 bus /gobi_0: Setting Powered False
15:02:55.971201 bus /gobi_0: Setting Powered False
15:02:57.076926 bus /gobi_0: Setting Powered False
15:02:58.226847 bus /gobi_6: Setting Online False
15:02:59.366074 bus /gobi_6: Setting Powered False
15:03:07.982410 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/iperf3m4.py", line 8, in <module>
setup_run_iperf3_test_parallel(4)
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 105, in setup_run_iperf3_test_parallel
print_results(clients[i], servers[i])
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 5, in print_results
cli.print_results()
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/iperf3.py", line 342, in print_results
print_result_node_tcp(self.get_results(), 'client')
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/iperf3.py", line 72, in print_result_node_tcp
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/iperf3.py", line 65, in print_result_node_tcp
sent = result['end']['sum_sent']
KeyError: 'sum_sent'