![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
Failed
gprs:sysmo.iperf3m4.py (from gprs_sysmo)
Failing for the past 4 builds
(Since Failed )
Stacktrace
'sum_sent'
Standard Output
----------------------------------------------
trial-2066 gprs:sysmo iperf3m4.py
----------------------------------------------
11:13:41.270432 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.281570 tst gprs:sysmo: Using 1 x bts (candidates: 1)
11:13:41.404424 tst iperf3m4.py: using LAC 3961
11:13:41.526509 tst iperf3m4.py: using RAC 136
11:13:41.647075 tst iperf3m4.py: using CellId 3961
11:13:41.768981 tst iperf3m4.py: using BVCI 3962
11:13:41.792596 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.803157 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.813586 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.824218 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.834635 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.845095 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.855903 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.866488 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.877290 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:41.994032 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:42.111998 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:42.231442 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:42.513260 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
11:13:42.641683 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:42.964843 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Launched
11:13:43.233365 run iperf3-srv_10.42.42.10(pid=18291): Launched
11:13:43.422920 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
11:13:43.551724 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:43.873103 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Launched
11:13:44.140000 run iperf3-srv_10.42.42.10(pid=18293): Launched
11:13:44.327072 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
11:13:44.454962 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:44.775455 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Launched
11:13:45.042829 run iperf3-srv_10.42.42.10(pid=18295): Launched
11:13:45.229782 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
11:13:45.357796 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:45.678636 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Launched
11:13:45.944072 run iperf3-srv_10.42.42.10(pid=18297): Launched
11:13:46.131778 tst iperf3m4.py:8: start network...
11:13:46.259650 run osmo-hlr_10.42.42.2: Starting osmo-hlr
11:13:46.754106 run create_hlr_db(pid=18298): Launched
11:13:46.930662 bus /gobi_6: Setting Powered False
11:13:47.953736 run create_hlr_db(pid=18298): Terminated: ok {rc=0}
11:13:48.391947 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Launched
11:13:48.723369 run osmo-hlr_10.42.42.2(pid=18301): Launched
11:13:48.842874 run osmo-stp_10.42.42.5: Starting osmo-stp
11:13:49.417802 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Launched
11:13:49.749764 run osmo-stp_10.42.42.5(pid=18303): Launched
11:13:49.866773 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
11:13:50.440512 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Launched
11:13:50.557544 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
11:13:50.846437 run patchelf(pid=18305): Launched
11:13:52.043272 run patchelf(pid=18305): Terminated: ok {rc=0}
11:13:52.164353 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
11:13:52.459711 run osmo-gsm-tester_setcap_net_admin.sh(pid=18306): Launched
11:13:53.655390 run osmo-gsm-tester_setcap_net_admin.sh(pid=18306): Terminated: ok {rc=0}
11:13:53.991514 run osmo-ggsn_10.42.42.6(pid=18309): Launched
11:13:54.247217 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
11:13:54.861195 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Launched
11:13:55.193282 run osmo-sgsn_10.42.42.7(pid=18335): Launched
11:13:55.313936 run osmo-msc_10.42.42.8: Starting osmo-msc
11:13:55.933538 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Launched
11:13:56.247877 run osmo-msc_10.42.42.8(pid=18337): Launched
11:13:59.413401 run osmo-mgw_10.42.42.3: Starting osmo-mgw
11:13:59.951211 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Launched
11:14:00.261575 run osmo-mgw_10.42.42.3(pid=18339): Launched
11:14:00.372378 run osmo-mgw_10.42.42.4: Starting osmo-mgw
11:14:00.897929 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Launched
11:14:01.208094 run osmo-mgw_10.42.42.4(pid=18341): Launched
11:14:01.317746 run osmo-bsc_10.42.42.9: Starting osmo-bsc
11:14:01.967106 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Launched
11:14:02.278989 run osmo-bsc_10.42.42.9(pid=18343): Launched
11:14:08.543627 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
11:14:09.097788 run rm-remote-dir(pid=18344): Launched
11:14:11.291215 run rm-remote-dir(pid=18344): Terminated: ok {rc=0}
11:14:11.626471 run mk-remote-dir(pid=18345): Launched
11:14:12.817129 run mk-remote-dir(pid=18345): Terminated: ok {rc=0}
11:14:13.097593 run scp-inst-to-remote(pid=18346): Launched
11:14:25.302961 run scp-inst-to-remote(pid=18346): Terminated: ok {rc=0}
11:14:25.632334 run mk-remote-dir(pid=18348): Launched
11:14:26.819794 run mk-remote-dir(pid=18348): Terminated: ok {rc=0}
11:14:27.102289 run scp-cfg-to-remote(pid=18349): Launched
11:14:28.291140 run scp-cfg-to-remote(pid=18349): Terminated: ok {rc=0}
11:14:28.614416 run reload-dsp-firmware(pid=18351): Launched
11:14:30.802467 run reload-dsp-firmware(pid=18351): Terminated: ok {rc=0}
11:14:31.116572 run osmo-bts-sysmo(pid=18352): Launched
11:14:35.426080 tst iperf3m4.py:8: Waiting for bts to be ready...
11:14:37.076035 run rm-remote-dir(pid=18353): Launched
11:14:39.273502 run rm-remote-dir(pid=18353): Terminated: ok {rc=0}
11:14:39.614671 run mk-remote-dir(pid=18354): Launched
11:14:41.810805 run mk-remote-dir(pid=18354): Terminated: ok {rc=0}
11:14:42.102906 run scp-inst-to-bts(pid=18355): Launched
11:14:55.309456 run scp-inst-to-bts(pid=18355): Terminated: ok {rc=0}
11:14:55.633087 run mk-remote-run-dir(pid=18362): Launched
11:14:57.816443 run mk-remote-run-dir(pid=18362): Terminated: ok {rc=0}
11:14:58.087949 run scp-cfg-to-sysmobts(pid=18363): Launched
11:15:00.270833 run scp-cfg-to-sysmobts(pid=18363): Terminated: ok {rc=0}
11:15:00.591812 run osmo-pcu(pid=18365): Launched
11:15:00.721048 tst iperf3m4.py: using MSISDN 3974
11:15:00.894999 bus /gobi_6: Setting Powered True
11:15:04.440115 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3974', subscriber_id=1}
11:15:04.718588 bus /gobi_6: Setting Online False
11:15:05.917223 bus /gobi_6: Setting Powered False
11:15:13.221217 bus /gobi_6: Setting Powered True
11:15:14.646074 bus /gobi_6: Setting Online True
11:15:17.205657 tst /gobi_6: Connect to ('901', '70')
11:15:17.488459 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
11:15:17.617459 tst iperf3m4.py: using MSISDN 3975
11:15:17.851571 bus /gobi_0: Setting Powered True
11:15:21.357350 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3975', subscriber_id=2}
11:15:21.634105 bus /gobi_0: Setting Online False
11:15:22.823810 bus /gobi_0: Setting Powered False
11:15:30.207200 bus /gobi_0: Setting Powered True
11:15:31.566424 bus /gobi_0: Setting Online True
11:15:32.437298 tst /gobi_6: Already registered with network ('901', '70')
11:15:34.215227 tst /gobi_0: Connect to ('901', '70')
11:15:34.505310 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873'
11:15:34.634514 tst iperf3m4.py: using MSISDN 3976
11:15:34.873109 bus /gobi_1: Setting Powered True
11:15:38.477034 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3976', subscriber_id=3}
11:15:38.769444 bus /gobi_1: Setting Online False
11:15:39.958579 bus /gobi_1: Setting Powered False
11:15:47.232267 bus /gobi_1: Setting Powered True
11:15:48.634433 bus /gobi_1: Setting Online True
11:15:50.245581 tst /gobi_0: Already registered with network ('901', '70')
11:15:51.126457 tst /gobi_1: Connect to ('901', '70')
11:15:51.388694 tst /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024272103'
11:15:51.508304 tst iperf3m4.py: using MSISDN 3977
11:15:51.688502 bus /gobi_4: Setting Powered True
11:15:55.203861 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3977', subscriber_id=4}
11:15:55.505376 bus /gobi_4: Setting Online False
11:15:56.692708 bus /gobi_4: Setting Powered False
11:16:03.453280 bus /gobi_4: Setting Powered True
11:16:04.829794 bus /gobi_4: Setting Online True
11:16:07.271780 tst /gobi_4: Connect to ('901', '70')
11:16:07.543309 tst /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G 1 [Jul 21 2016 10:00:50]', Serial: '868323025161511'
11:16:07.757622 tst iperf3m4.py:8: waiting for modems to attach...
11:16:08.009116 tst /gobi_1: Already registered with network ('901', '70')
11:16:24.032212 tst /gobi_4: Already registered with network ('901', '70')
11:16:29.438159 tst iperf3m4.py:8: waiting for modems to attach to data services...
11:16:32.470261 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:16:32.656832 tst iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'"
11:16:33.010364 run move_netns(pid=18898): Launched
11:16:34.194610 run move_netns(pid=18898): Terminated: ok {rc=0}
11:16:34.527263 run ifup(pid=18901): Launched
11:16:35.712256 run ifup(pid=18901): Terminated: ok {rc=0}
11:16:36.046034 run dhcp(pid=18904): Launched
11:16:37.241544 run dhcp(pid=18904): Terminated: ok {rc=0}
11:16:45.629495 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:16:45.832892 tst iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'"
11:16:46.216989 run move_netns(pid=18922): Launched
11:16:47.413335 run move_netns(pid=18922): Terminated: ok {rc=0}
11:16:47.761162 run ifup(pid=18925): Launched
11:16:48.945720 run ifup(pid=18925): Terminated: ok {rc=0}
11:16:49.274821 run dhcp(pid=18928): Launched
11:16:50.459097 run dhcp(pid=18928): Terminated: ok {rc=0}
11:16:58.814582 tst /gobi_1: context activated {apn='inet46', path='/gobi_1/context11', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'}
11:16:59.017705 tst iperf3m4.py:8: Setting up data plan for "'/gobi_1/context11'"
11:16:59.392761 run move_netns(pid=18946): Launched
11:17:00.576526 run move_netns(pid=18946): Terminated: ok {rc=0}
11:17:00.901421 run ifup(pid=18949): Launched
11:17:02.086312 run ifup(pid=18949): Terminated: ok {rc=0}
11:17:02.421559 run dhcp(pid=18955): Launched
11:17:03.616651 run dhcp(pid=18955): Terminated: ok {rc=0}
11:17:11.990881 tst /gobi_4: context activated {apn='inet46', path='/gobi_4/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:17:12.177679 tst iperf3m4.py:8: Setting up data plan for "'/gobi_4/context2'"
11:17:12.535511 run move_netns(pid=18973): Launched
11:17:13.718550 run move_netns(pid=18973): Terminated: ok {rc=0}
11:17:14.052386 run ifup(pid=18976): Launched
11:17:15.249554 run ifup(pid=18976): Terminated: ok {rc=0}
11:17:15.599760 run dhcp(pid=18979): Launched
11:17:16.795873 run dhcp(pid=18979): Terminated: ok {rc=0}
11:17:17.004972 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'"
11:17:17.145835 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
11:17:17.550481 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Launched
11:17:17.788259 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'"
11:17:17.918451 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s)
11:17:18.299597 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Launched
11:17:18.537531 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context11'"
11:17:18.667800 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s)
11:17:19.045020 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Launched
11:17:19.282402 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context2'"
11:17:19.413055 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s)
11:17:19.792067 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Launched
11:17:20.107287 run iperf3-cli_10.42.42.10(pid=19007): Launched
11:17:20.376609 run iperf3-cli_10.42.42.10(pid=19010): Launched
11:17:20.646843 run iperf3-cli_10.42.42.10(pid=19013): Launched
11:17:20.915049 run iperf3-cli_10.42.42.10(pid=19016): Launched
11:18:29.234857 run iperf3-cli_10.42.42.10(pid=19007): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19007)]
11:18:29.382794 run iperf3-cli_10.42.42.10(pid=19007): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:29.526855 run iperf3-cli_10.42.42.10(pid=19007): stderr:
| (launched: 2022-09-07_11:17:19.996357)
11:18:35.720649 run iperf3-cli_10.42.42.10(pid=19010): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19010)]
11:18:35.851788 run iperf3-cli_10.42.42.10(pid=19010): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:35.979699 run iperf3-cli_10.42.42.10(pid=19010): stderr:
| (launched: 2022-09-07_11:17:20.264381)
11:18:37.162106 run iperf3-cli_10.42.42.10(pid=19013): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19013)]
11:18:37.291204 run iperf3-cli_10.42.42.10(pid=19013): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:37.419818 run iperf3-cli_10.42.42.10(pid=19013): stderr:
| (launched: 2022-09-07_11:17:20.534404)
11:18:38.607294 run iperf3-cli_10.42.42.10(pid=19016): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19016)]
11:18:38.737791 run iperf3-cli_10.42.42.10(pid=19016): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:38.868686 run iperf3-cli_10.42.42.10(pid=19016): stderr:
| (launched: 2022-09-07_11:17:20.804104)
11:18:38.987201 run iperf3-srv_10.42.42.10(pid=18291): Terminating (SIGINT)
11:18:39.341198 run iperf3-srv_10.42.42.10(pid=18291): Terminated {rc=1}
11:18:39.552386 tst iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'"
11:18:39.614165 tst iperf3m4.py:8: ERR: KeyError: 'sum_sent' [trial-2066↪gprs:sysmo↪iperf3m4.py:8]
11:18:39.622613 tst iperf3m4.py:8: Test FAILED (298.4 sec)
11:18:39.661212 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Terminating (SIGTERM)
11:18:39.705708 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Terminating (SIGTERM)
11:18:39.750490 run iperf3-srv_10.42.42.10(pid=18293): Terminating (SIGTERM)
11:18:39.795369 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Terminating (SIGTERM)
11:18:39.839100 run iperf3-srv_10.42.42.10(pid=18295): Terminating (SIGTERM)
11:18:39.883752 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Terminating (SIGTERM)
11:18:39.928412 run iperf3-srv_10.42.42.10(pid=18297): Terminating (SIGTERM)
11:18:39.972258 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Terminating (SIGTERM)
11:18:40.016802 run osmo-hlr_10.42.42.2(pid=18301): Terminating (SIGTERM)
11:18:40.061625 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Terminating (SIGTERM)
11:18:40.107513 run osmo-stp_10.42.42.5(pid=18303): Terminating (SIGTERM)
11:18:40.175402 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Terminating (SIGTERM)
11:18:40.220721 run osmo-ggsn_10.42.42.6(pid=18309): Terminating (SIGTERM)
11:18:40.321460 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Terminating (SIGTERM)
11:18:40.380068 run osmo-sgsn_10.42.42.7(pid=18335): Terminating (SIGTERM)
11:18:40.429862 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Terminating (SIGTERM)
11:18:40.484651 run osmo-msc_10.42.42.8(pid=18337): Terminating (SIGTERM)
11:18:40.536059 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Terminating (SIGTERM)
11:18:40.579308 run osmo-mgw_10.42.42.3(pid=18339): Terminating (SIGTERM)
11:18:40.621384 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Terminating (SIGTERM)
11:18:40.664572 run osmo-mgw_10.42.42.4(pid=18341): Terminating (SIGTERM)
11:18:40.705651 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Terminating (SIGTERM)
11:18:40.747107 run osmo-bsc_10.42.42.9(pid=18343): Terminating (SIGTERM)
11:18:40.788954 run osmo-bts-sysmo(pid=18352): Terminating (SIGTERM)
11:18:40.831012 run osmo-pcu(pid=18365): Terminating (SIGTERM)
11:18:40.874842 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGTERM)
11:18:41.024321 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGTERM(pid=19043): Launched
11:18:42.114506 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGTERM(pid=19043): Terminated: ok {rc=0}
11:18:42.160274 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGTERM)
11:18:42.312350 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGTERM(pid=19048): Launched
11:18:43.403477 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGTERM(pid=19048): Terminated: ok {rc=0}
11:18:43.449262 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGTERM)
11:18:43.597525 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGTERM(pid=19051): Launched
11:18:44.688563 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGTERM(pid=19051): Terminated: ok {rc=0}
11:18:44.734376 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGTERM)
11:18:44.882024 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGTERM(pid=19054): Launched
11:18:45.974138 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGTERM(pid=19054): Terminated: ok {rc=0}
11:18:45.988120 --- ParallelTerminationStrategy: PID 18290 died...
11:18:46.043019 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Terminated: ok {rc=0}
11:18:46.052846 --- ParallelTerminationStrategy: PID 18292 died...
11:18:46.107335 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Terminated: ok {rc=0}
11:18:46.117137 --- ParallelTerminationStrategy: PID 18293 died...
11:18:46.170990 run iperf3-srv_10.42.42.10(pid=18293): Terminated {rc=256}
11:18:46.180895 --- ParallelTerminationStrategy: PID 18294 died...
11:18:46.235868 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Terminated: ok {rc=0}
11:18:46.245712 --- ParallelTerminationStrategy: PID 18295 died...
11:18:46.301925 run iperf3-srv_10.42.42.10(pid=18295): Terminated {rc=256}
11:18:46.311600 --- ParallelTerminationStrategy: PID 18296 died...
11:18:46.365797 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Terminated: ok {rc=0}
11:18:46.375552 --- ParallelTerminationStrategy: PID 18297 died...
11:18:46.429390 run iperf3-srv_10.42.42.10(pid=18297): Terminated {rc=256}
11:18:46.439541 --- ParallelTerminationStrategy: PID 18300 died...
11:18:46.494619 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Terminated: ok {rc=0}
11:18:46.504383 --- ParallelTerminationStrategy: PID 18301 died...
11:18:46.559093 run osmo-hlr_10.42.42.2(pid=18301): Terminated: ok {rc=0}
11:18:46.568898 --- ParallelTerminationStrategy: PID 18302 died...
11:18:46.623050 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Terminated: ok {rc=0}
11:18:46.632859 --- ParallelTerminationStrategy: PID 18303 died...
11:18:46.686641 run osmo-stp_10.42.42.5(pid=18303): Terminated: ok {rc=0}
11:18:46.696466 --- ParallelTerminationStrategy: PID 18304 died...
11:18:46.750538 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Terminated: ok {rc=0}
11:18:46.760334 --- ParallelTerminationStrategy: PID 18309 died...
11:18:46.814218 run osmo-ggsn_10.42.42.6(pid=18309): Terminated: ok {rc=0}
11:18:46.824030 --- ParallelTerminationStrategy: PID 18334 died...
11:18:46.878097 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Terminated: ok {rc=0}
11:18:46.887899 --- ParallelTerminationStrategy: PID 18335 died...
11:18:46.941896 run osmo-sgsn_10.42.42.7(pid=18335): Terminated: ok {rc=0}
11:18:46.951724 --- ParallelTerminationStrategy: PID 18336 died...
11:18:47.005934 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Terminated: ok {rc=0}
11:18:47.015920 --- ParallelTerminationStrategy: PID 18337 died...
11:18:47.070303 run osmo-msc_10.42.42.8(pid=18337): Terminated: ok {rc=0}
11:18:47.080538 --- ParallelTerminationStrategy: PID 18338 died...
11:18:47.135209 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Terminated: ok {rc=0}
11:18:47.145444 --- ParallelTerminationStrategy: PID 18339 died...
11:18:47.200766 run osmo-mgw_10.42.42.3(pid=18339): Terminated {rc=15}
11:18:47.211137 --- ParallelTerminationStrategy: PID 18340 died...
11:18:47.265972 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Terminated: ok {rc=0}
11:18:47.276294 --- ParallelTerminationStrategy: PID 18341 died...
11:18:47.331076 run osmo-mgw_10.42.42.4(pid=18341): Terminated {rc=15}
11:18:47.341339 --- ParallelTerminationStrategy: PID 18342 died...
11:18:47.396064 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Terminated: ok {rc=0}
11:18:47.406224 --- ParallelTerminationStrategy: PID 18343 died...
11:18:47.461093 run osmo-bsc_10.42.42.9(pid=18343): Terminated: ok {rc=0}
11:18:47.471577 --- ParallelTerminationStrategy: PID 18352 died...
11:18:47.526400 run osmo-bts-sysmo(pid=18352): Terminated: ok {rc=0}
11:18:47.536794 --- ParallelTerminationStrategy: PID 18365 died...
11:18:47.591439 run osmo-pcu(pid=18365): Terminated: ok {rc=0}
11:18:53.457912 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGINT)
11:18:53.610339 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGINT(pid=19057): Launched
11:18:54.702652 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGINT(pid=19057): Terminated: ok {rc=0}
11:18:54.748746 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGINT)
11:18:54.903243 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGINT(pid=19060): Launched
11:18:55.999259 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGINT(pid=19060): Terminated: ok {rc=0}
11:18:56.047381 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGINT)
11:18:56.202254 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGINT(pid=19063): Launched
11:18:57.296337 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGINT(pid=19063): Terminated: ok {rc=0}
11:18:57.344711 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGINT)
11:18:57.498790 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGINT(pid=19070): Launched
11:18:58.594062 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGINT(pid=19070): Terminated: ok {rc=0}
11:19:04.466106 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGKILL)
11:19:04.621749 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGUSR1(pid=19073): Launched
11:19:05.716462 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGUSR1(pid=19073): Terminated: ok {rc=0}
11:19:05.764340 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGKILL)
11:19:05.917842 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGUSR1(pid=19076): Launched
11:19:07.011550 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGUSR1(pid=19076): Terminated: ok {rc=0}
11:19:07.059323 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGKILL)
11:19:07.212499 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGUSR1(pid=19079): Launched
11:19:08.306640 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGUSR1(pid=19079): Terminated: ok {rc=0}
11:19:08.354478 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGKILL)
11:19:08.511708 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGUSR1(pid=19082): Launched
11:19:09.606901 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGUSR1(pid=19082): Terminated: ok {rc=0}
11:19:09.839420 bus /gobi_4: Setting Online False
11:19:11.009890 bus /gobi_4: Setting Powered False
11:19:20.979513 bus /gobi_4: Setting Powered False
11:19:22.086087 bus /gobi_4: Setting Powered False
11:19:23.239535 bus /gobi_1: Setting Online False
11:19:24.352922 bus /gobi_1: Setting Powered False
11:19:32.122180 bus /gobi_1: Setting Powered False
11:19:33.225816 bus /gobi_1: Setting Powered False
11:19:34.384281 bus /gobi_0: Setting Online False
11:19:35.517399 bus /gobi_0: Setting Powered False
11:19:43.975893 bus /gobi_0: Setting Powered False
11:19:45.082850 bus /gobi_0: Setting Powered False
11:19:46.236210 bus /gobi_6: Setting Online False
11:19:47.354770 bus /gobi_6: Setting Powered False
11:19:55.131635 bus /gobi_6: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/iperf3m4.py", line 8, in <module>
setup_run_iperf3_test_parallel(4)
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 105, in setup_run_iperf3_test_parallel
print_results(clients[i], servers[i])
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 5, in print_results
cli.print_results()
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/iperf3.py", line 342, in print_results
print_result_node_tcp(self.get_results(), 'client')
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/iperf3.py", line 72, in print_result_node_tcp
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj/iperf3.py", line 65, in print_result_node_tcp
sent = result['end']['sum_sent']
KeyError: 'sum_sent'