Skipping 84 KB..
Full Log11:08:45.486538 tst cs_paging_gprs_active.py:38: ERR: KeyError: 'sum_sent' [trial-2066↪gprs:sysmo↪cs_paging_gprs_active.py:38]
11:08:45.493986 tst cs_paging_gprs_active.py:38: Test FAILED (247.5 sec)
11:08:45.501206 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=17999): Terminating (SIGTERM)
11:08:45.543318 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18001): Terminating (SIGTERM)
11:08:45.585532 run iperf3-srv_10.42.42.10(pid=18002): Terminating (SIGTERM)
11:08:45.627688 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18005): Terminating (SIGTERM)
11:08:45.669235 run osmo-hlr_10.42.42.2(pid=18006): Terminating (SIGTERM)
11:08:45.710791 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18007): Terminating (SIGTERM)
11:08:45.753678 run osmo-stp_10.42.42.5(pid=18008): Terminating (SIGTERM)
11:08:45.797648 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18009): Terminating (SIGTERM)
11:08:45.842587 run osmo-ggsn_10.42.42.6(pid=18014): Terminating (SIGTERM)
11:08:45.886959 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18035): Terminating (SIGTERM)
11:08:45.944252 run osmo-sgsn_10.42.42.7(pid=18036): Terminating (SIGTERM)
11:08:46.010876 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18037): Terminating (SIGTERM)
11:08:46.057130 run osmo-msc_10.42.42.8(pid=18038): Terminating (SIGTERM)
11:08:46.118641 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18039): Terminating (SIGTERM)
11:08:46.196233 run osmo-mgw_10.42.42.3(pid=18040): Terminating (SIGTERM)
11:08:46.252945 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18041): Terminating (SIGTERM)
11:08:46.302106 run osmo-mgw_10.42.42.4(pid=18042): Terminating (SIGTERM)
11:08:46.347934 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18043): Terminating (SIGTERM)
11:08:46.392001 run osmo-bsc_10.42.42.9(pid=18047): Terminating (SIGTERM)
11:08:46.433995 run osmo-bts-sysmo(pid=18056): Terminating (SIGTERM)
11:08:46.475983 run osmo-pcu(pid=18065): Terminating (SIGTERM)
11:08:46.517944 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116): Terminating (SIGTERM)
11:08:46.657266 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGTERM(pid=18146): Launched
11:08:47.762948 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGTERM(pid=18146): Terminated: ok {rc=0}
11:08:47.826373 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119): Terminating (SIGTERM)
11:08:47.962314 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGTERM(pid=18150): Launched
11:08:49.045145 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGTERM(pid=18150): Terminated: ok {rc=0}
11:08:49.108943 --- ParallelTerminationStrategy: PID 17999 died...
11:08:49.128115 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=17999): Terminated: ok {rc=0}
11:08:49.173816 --- ParallelTerminationStrategy: PID 18001 died...
11:08:49.192857 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18001): Terminated: ok {rc=0}
11:08:49.237797 --- ParallelTerminationStrategy: PID 18002 died...
11:08:49.256658 run iperf3-srv_10.42.42.10(pid=18002): Terminated {rc=256}
11:08:49.301369 --- ParallelTerminationStrategy: PID 18005 died...
11:08:49.320300 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18005): Terminated: ok {rc=0}
11:08:49.365158 --- ParallelTerminationStrategy: PID 18006 died...
11:08:49.384550 run osmo-hlr_10.42.42.2(pid=18006): Terminated: ok {rc=0}
11:08:49.429205 --- ParallelTerminationStrategy: PID 18007 died...
11:08:49.448291 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18007): Terminated: ok {rc=0}
11:08:49.493101 --- ParallelTerminationStrategy: PID 18008 died...
11:08:49.512080 run osmo-stp_10.42.42.5(pid=18008): Terminated: ok {rc=0}
11:08:49.557281 --- ParallelTerminationStrategy: PID 18009 died...
11:08:49.576194 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18009): Terminated: ok {rc=0}
11:08:49.621128 --- ParallelTerminationStrategy: PID 18014 died...
11:08:49.639947 run osmo-ggsn_10.42.42.6(pid=18014): Terminated: ok {rc=0}
11:08:49.685064 --- ParallelTerminationStrategy: PID 18035 died...
11:08:49.703923 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18035): Terminated: ok {rc=0}
11:08:49.748970 --- ParallelTerminationStrategy: PID 18036 died...
11:08:49.767810 run osmo-sgsn_10.42.42.7(pid=18036): Terminated: ok {rc=0}
11:08:49.813243 --- ParallelTerminationStrategy: PID 18037 died...
11:08:49.832329 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18037): Terminated: ok {rc=0}
11:08:49.877738 --- ParallelTerminationStrategy: PID 18038 died...
11:08:49.896733 run osmo-msc_10.42.42.8(pid=18038): Terminated: ok {rc=0}
11:08:49.943377 --- ParallelTerminationStrategy: PID 18039 died...
11:08:49.962339 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18039): Terminated: ok {rc=0}
11:08:50.007089 --- ParallelTerminationStrategy: PID 18040 died...
11:08:50.026058 run osmo-mgw_10.42.42.3(pid=18040): Terminated {rc=15}
11:08:50.070905 --- ParallelTerminationStrategy: PID 18041 died...
11:08:50.089723 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18041): Terminated: ok {rc=0}
11:08:50.135126 --- ParallelTerminationStrategy: PID 18042 died...
11:08:50.154173 run osmo-mgw_10.42.42.4(pid=18042): Terminated {rc=15}
11:08:50.199272 --- ParallelTerminationStrategy: PID 18043 died...
11:08:50.218365 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18043): Terminated: ok {rc=0}
11:08:50.263341 --- ParallelTerminationStrategy: PID 18047 died...
11:08:50.282202 run osmo-bsc_10.42.42.9(pid=18047): Terminated: ok {rc=0}
11:08:50.327474 --- ParallelTerminationStrategy: PID 18056 died...
11:08:50.346504 run osmo-bts-sysmo(pid=18056): Terminated: ok {rc=0}
11:08:50.391255 --- ParallelTerminationStrategy: PID 18065 died...
11:08:50.410210 run osmo-pcu(pid=18065): Terminated: ok {rc=0}
11:08:56.277465 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116): Terminating (SIGINT)
11:08:56.415314 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGINT(pid=18153): Launched
11:08:57.498514 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGINT(pid=18153): Terminated: ok {rc=0}
11:08:57.561904 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119): Terminating (SIGINT)
11:08:57.698886 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGINT(pid=18156): Launched
11:08:58.784713 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGINT(pid=18156): Terminated: ok {rc=0}
11:09:04.670806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116): Terminating (SIGKILL)
11:09:04.811831 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGUSR1(pid=18159): Launched
11:09:05.905827 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGUSR1(pid=18159): Terminated: ok {rc=0}
11:09:05.972272 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119): Terminating (SIGKILL)
11:09:06.115142 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGUSR1(pid=18162): Launched
11:09:07.203541 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGUSR1(pid=18162): Terminated: ok {rc=0}
11:09:07.459580 bus /gobi_0: Setting Online False
11:09:09.637248 bus /gobi_0: Setting Powered False
11:09:26.942156 bus /gobi_0: Setting Powered False
11:09:28.039852 bus /gobi_0: Setting Powered False
11:09:29.197698 bus /gobi_6: Setting Online False
11:09:30.307241 bus /gobi_6: Setting Powered False
11:09:38.939690 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2066 gprs:sysmo iperf3.py
----------------------------------------------
11:09:40.150263 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.162137 tst gprs:sysmo: Using 1 x bts (candidates: 1)
11:09:40.198786 tst iperf3.py: using LAC 3960
11:09:40.325090 tst iperf3.py: using RAC 135
11:09:40.445129 tst iperf3.py: using CellId 3960
11:09:40.577603 tst iperf3.py: using BVCI 3961
11:09:40.689770 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.700405 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.710923 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.721528 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.732799 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.743367 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.753959 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.764495 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:09:40.775277 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:09:40.894355 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003...
11:09:41.099276 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:09:41.409219 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18165): Launched
11:09:41.677856 run iperf3-srv_10.42.42.10(pid=18166): Launched
11:09:41.791899 tst iperf3.py:8: start network...
11:09:41.995786 run osmo-hlr_10.42.42.2: Starting osmo-hlr
11:09:42.469871 run create_hlr_db(pid=18167): Launched
11:09:42.587312 bus /gobi_6: Setting Powered False
11:09:43.648883 run create_hlr_db(pid=18167): Terminated: ok {rc=0}
11:09:44.097956 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18169): Launched
11:09:44.429676 run osmo-hlr_10.42.42.2(pid=18170): Launched
11:09:44.555773 run osmo-stp_10.42.42.5: Starting osmo-stp
11:09:45.115255 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18171): Launched
11:09:45.443709 run osmo-stp_10.42.42.5(pid=18172): Launched
11:09:45.569067 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
11:09:46.130517 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18173): Launched
11:09:46.255373 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
11:09:46.533597 run patchelf(pid=18174): Launched
11:09:47.720118 run patchelf(pid=18174): Terminated: ok {rc=0}
11:09:47.857962 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
11:09:48.138803 run osmo-gsm-tester_setcap_net_admin.sh(pid=18175): Launched
11:09:49.326246 run osmo-gsm-tester_setcap_net_admin.sh(pid=18175): Terminated: ok {rc=0}
11:09:49.667705 run osmo-ggsn_10.42.42.6(pid=18178): Launched
11:09:49.802946 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
11:09:50.472489 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18199): Launched
11:09:50.782603 run osmo-sgsn_10.42.42.7(pid=18200): Launched
11:09:50.900581 run osmo-msc_10.42.42.8: Starting osmo-msc
11:09:51.459751 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18201): Launched
11:09:51.768415 run osmo-msc_10.42.42.8(pid=18202): Launched
11:09:54.943511 run osmo-mgw_10.42.42.3: Starting osmo-mgw
11:09:55.467653 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18203): Launched
11:09:55.774002 run osmo-mgw_10.42.42.3(pid=18204): Launched
11:09:55.892207 run osmo-mgw_10.42.42.4: Starting osmo-mgw
11:09:56.408389 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18205): Launched
11:09:56.717272 run osmo-mgw_10.42.42.4(pid=18206): Launched
11:09:56.835298 run osmo-bsc_10.42.42.9: Starting osmo-bsc
11:09:57.471435 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18207): Launched
11:09:57.779481 run osmo-bsc_10.42.42.9(pid=18208): Launched
11:10:04.051774 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
11:10:04.585470 run rm-remote-dir(pid=18212): Launched
11:10:06.769746 run rm-remote-dir(pid=18212): Terminated: ok {rc=0}
11:10:07.107261 run mk-remote-dir(pid=18213): Launched
11:10:08.288846 run mk-remote-dir(pid=18213): Terminated: ok {rc=0}
11:10:08.576761 run scp-inst-to-remote(pid=18214): Launched
11:10:20.771981 run scp-inst-to-remote(pid=18214): Terminated: ok {rc=0}
11:10:21.130210 run mk-remote-dir(pid=18216): Launched
11:10:22.321153 run mk-remote-dir(pid=18216): Terminated: ok {rc=0}
11:10:22.626026 run scp-cfg-to-remote(pid=18217): Launched
11:10:23.815427 run scp-cfg-to-remote(pid=18217): Terminated: ok {rc=0}
11:10:24.166668 run reload-dsp-firmware(pid=18219): Launched
11:10:26.351014 run reload-dsp-firmware(pid=18219): Terminated: ok {rc=0}
11:10:26.673000 run osmo-bts-sysmo(pid=18220): Launched
11:10:30.903801 tst iperf3.py:8: Waiting for bts to be ready...
11:10:32.628316 run rm-remote-dir(pid=18221): Launched
11:10:34.812608 run rm-remote-dir(pid=18221): Terminated: ok {rc=0}
11:10:35.157787 run mk-remote-dir(pid=18222): Launched
11:10:37.342868 run mk-remote-dir(pid=18222): Terminated: ok {rc=0}
11:10:37.638953 run scp-inst-to-bts(pid=18223): Launched
11:10:50.840886 run scp-inst-to-bts(pid=18223): Terminated: ok {rc=0}
11:10:51.163706 run mk-remote-run-dir(pid=18226): Launched
11:10:53.337810 run mk-remote-run-dir(pid=18226): Terminated: ok {rc=0}
11:10:53.615926 run scp-cfg-to-sysmobts(pid=18227): Launched
11:10:55.793036 run scp-cfg-to-sysmobts(pid=18227): Terminated: ok {rc=0}
11:10:56.140404 run osmo-pcu(pid=18229): Launched
11:10:56.279487 tst iperf3.py: using MSISDN 3973
11:10:56.447044 bus /gobi_6: Setting Powered True
11:10:59.947882 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3973', subscriber_id=1}
11:11:00.233818 bus /gobi_6: Setting Online False
11:11:01.428516 bus /gobi_6: Setting Powered False
11:11:08.191590 bus /gobi_6: Setting Powered True
11:11:09.591885 bus /gobi_6: Setting Online True
11:11:12.068419 tst /gobi_6: Connect to ('901', '70')
11:11:12.348188 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
11:11:12.467150 tst iperf3.py:8: waiting for modems to attach...
11:11:19.647434 tst /gobi_6: Already registered with network ('901', '70')
11:11:30.999160 tst iperf3.py:8: waiting for modems to attach to data services...
11:11:48.997021 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:11:49.107767 tst iperf3.py:8: Setting up data plan for "'/gobi_6/context3'"
11:11:49.533391 run move_netns(pid=18232): Launched
11:11:50.727258 run move_netns(pid=18232): Terminated: ok {rc=0}
11:11:51.083914 run ifup(pid=18235): Launched
11:11:52.277175 run ifup(pid=18235): Terminated: ok {rc=0}
11:11:52.623536 run dhcp(pid=18238): Launched
11:11:53.813217 run dhcp(pid=18238): Terminated: ok {rc=0}
11:11:53.943650 tst iperf3.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context3'"
11:11:54.154345 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
11:11:54.523616 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Launched
11:11:54.846922 run iperf3-cli_10.42.42.10(pid=18257): Launched
11:13:03.157279 run iperf3-cli_10.42.42.10(pid=18257): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=18257)]
11:13:03.289119 run iperf3-cli_10.42.42.10(pid=18257): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:13:03.421569 run iperf3-cli_10.42.42.10(pid=18257): stderr:
| (launched: 2022-09-07_11:11:54.831390)
11:13:03.553575 run iperf3-srv_10.42.42.10(pid=18166): Terminating (SIGINT)
11:13:03.893630 run iperf3-srv_10.42.42.10(pid=18166): Terminated {rc=1}
11:13:04.018991 tst iperf3.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context3'"
Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 46758, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Wed, 07 Sep 2022 09:11:56 GMT', 'timesecs': 1662541916}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'uzoh4dtndr3cftrovt2xrbxlzmcpi6zwtirj', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0008819103240967, 'seconds': 1.0008819103240967, 'bytes': 101360, 'bits_per_second': 810165.5066754359, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 429537, 'rttvar': 172019, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0008819103240967, 'seconds': 1.0008819103240967, 'bytes': 101360, 'bits_per_second': 810165.5066754359, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0008819103240967, 'end': 2.0012218952178955, 'seconds': 1.0003399848937988, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 516133, 'rttvar': 302207, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0008819103240967, 'end': 2.0012218952178955, 'seconds': 1.0003399848937988, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0012218952178955, 'end': 3.001326084136963, 'seconds': 1.0001041889190674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 18824, 'rtt': 899837, 'rttvar': 858283, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0012218952178955, 'end': 3.001326084136963, 'seconds': 1.0001041889190674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.001326084136963, 'end': 4.001185894012451, 'seconds': 0.9998598098754883, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1184790, 'rttvar': 1213618, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.001326084136963, 'end': 4.001185894012451, 'seconds': 0.9998598098754883, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001185894012451, 'end': 5.001333951950073, 'seconds': 1.000148057937622, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 1903891, 'rttvar': 1952341, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001185894012451, 'end': 5.001333951950073, 'seconds': 1.000148057937622, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001333951950073, 'end': 6.001183032989502, 'seconds': 0.9998490810394287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 2325793, 'rttvar': 2308060, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001333951950073, 'end': 6.001183032989502, 'seconds': 0.9998490810394287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001183032989502, 'end': 7.001324892044067, 'seconds': 1.0001418590545654, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2782671, 'rttvar': 2644801, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001183032989502, 'end': 7.001324892044067, 'seconds': 1.0001418590545654, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001324892044067, 'end': 8.001322031021118, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 3783260, 'rttvar': 3245423, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001324892044067, 'end': 8.001322031021118, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001322031021118, 'end': 9.001244068145752, 'seconds': 0.9999220371246338, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 4211214, 'rttvar': 3289975, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001322031021118, 'end': 9.001244068145752, 'seconds': 0.9999220371246338, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001244068145752, 'end': 10.00119400024414, 'seconds': 0.9999499320983887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 4644648, 'rttvar': 3334349, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001244068145752, 'end': 10.00119400024414, 'seconds': 0.9999499320983887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.00119400024414, 'end': 11.001322984695435, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 5114064, 'rttvar': 3439594, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.00119400024414, 'end': 11.001322984695435, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001322984695435, 'end': 12.001194953918457, 'seconds': 0.9998719692230225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 6047951, 'rttvar': 3553665, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001322984695435, 'end': 12.001194953918457, 'seconds': 0.9998719692230225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001194953918457, 'end': 13.001245021820068, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 6517251, 'rttvar': 3603850, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001194953918457, 'end': 13.001245021820068, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001245021820068, 'end': 14.001223087310791, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 6930230, 'rttvar': 3528845, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001245021820068, 'end': 14.001223087310791, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001223087310791, 'end': 15.001184940338135, 'seconds': 0.9999618530273438, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 7766762, 'rttvar': 3434981, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001223087310791, 'end': 15.001184940338135, 'seconds': 0.9999618530273438, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001184940338135, 'end': 16.001183032989502, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 8195384, 'rttvar': 3433481, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001184940338135, 'end': 16.001183032989502, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001183032989502, 'end': 17.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 8991751, 'rttvar': 3335099, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001183032989502, 'end': 17.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001183032989502, 'end': 18.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9355035, 'rttvar': 3227893, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001183032989502, 'end': 18.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001183032989502, 'end': 19.001282930374146, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 9760525, 'rttvar': 3231900, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001183032989502, 'end': 19.001282930374146, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.001282930374146, 'end': 20.001282930374146, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 10514881, 'rttvar': 3148101, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.001282930374146, 'end': 20.001282930374146, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001282930374146, 'end': 21.00122094154358, 'seconds': 0.9999380111694336, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 10860707, 'rttvar': 3052726, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001282930374146, 'end': 21.00122094154358, 'seconds': 0.9999380111694336, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.00122094154358, 'end': 22.001190900802612, 'seconds': 0.9999699592590332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11599317, 'rttvar': 2998039, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.00122094154358, 'end': 22.001190900802612, 'seconds': 0.9999699592590332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001190900802612, 'end': 23.001322984695435, 'seconds': 1.0001320838928223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11994335, 'rttvar': 3038565, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001190900802612, 'end': 23.001322984695435, 'seconds': 1.0001320838928223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001322984695435, 'end': 24.001182079315186, 'seconds': 0.999859094619751, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 12349683, 'rttvar': 2989618, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001322984695435, 'end': 24.001182079315186, 'seconds': 0.999859094619751, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001182079315186, 'end': 25.00127100944519, 'seconds': 1.0000889301300049, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 12767896, 'rttvar': 3078642, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001182079315186, 'end': 25.00127100944519, 'seconds': 1.0000889301300049, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.00127100944519, 'end': 26.00040102005005, 'seconds': 0.9991300106048584, 'bytes': 256296, 'bits_per_second': 2052153.3516531426, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 13584206, 'rttvar': 3168691, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.00127100944519, 'end': 26.00040102005005, 'seconds': 0.9991300106048584, 'bytes': 256296, 'bits_per_second': 2052153.3516531426, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.00040102005005, 'end': 27.00122308731079, 'seconds': 1.0008220672607422, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 13870601, 'rttvar': 2949309, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.00040102005005, 'end': 27.00122308731079, 'seconds': 1.0008220672607422, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00122308731079, 'end': 28.001182079315186, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14510112, 'rttvar': 2766268, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00122308731079, 'end': 28.001182079315186, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001182079315186, 'end': 29.00128412246704, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14856161, 'rttvar': 2766799, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001182079315186, 'end': 29.00128412246704, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.00128412246704, 'end': 30.001219987869263, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 15510681, 'rttvar': 2712826, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.00128412246704, 'end': 30.001219987869263, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.001219987869263, 'end': 31.001322031021118, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 15816495, 'rttvar': 2646247, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.001219987869263, 'end': 31.001322031021118, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.001322031021118, 'end': 32.00118398666382, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 16171722, 'rttvar': 2695138, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.001322031021118, 'end': 32.00118398666382, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00118398666382, 'end': 33.00118398666382, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 16837129, 'rttvar': 2693977, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00118398666382, 'end': 33.00118398666382, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00118398666382, 'end': 34.000471115112305, 'seconds': 0.9992871284484863, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17409705, 'rttvar': 2529060, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00118398666382, 'end': 34.000471115112305, 'seconds': 0.9992871284484863, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.000471115112305, 'end': 35.001286029815674, 'seconds': 1.0008149147033691, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 17678190, 'rttvar': 2433765, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.000471115112305, 'end': 35.001286029815674, 'seconds': 1.0008149147033691, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.001286029815674, 'end': 36.00027894973755, 'seconds': 0.998992919921875, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18278383, 'rttvar': 2409482, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.001286029815674, 'end': 36.00027894973755, 'seconds': 0.998992919921875, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00027894973755, 'end': 37.00119090080261, 'seconds': 1.0009119510650635, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18613768, 'rttvar': 2477881, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00027894973755, 'end': 37.00119090080261, 'seconds': 1.0009119510650635, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00119090080261, 'end': 38.00023102760315, 'seconds': 0.9990401268005371, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19241917, 'rttvar': 2505975, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00119090080261, 'end': 38.00023102760315, 'seconds': 0.9990401268005371, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00023102760315, 'end': 39.0010929107666, 'seconds': 1.0008618831634521, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 19538757, 'rttvar': 2473162, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00023102760315, 'end': 39.0010929107666, 'seconds': 1.0008618831634521, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0010929107666, 'end': 40.00030994415283, 'seconds': 0.9992170333862305, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20033526, 'rttvar': 2249418, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0010929107666, 'end': 40.00030994415283, 'seconds': 0.9992170333862305, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00030994415283, 'end': 41.00133490562439, 'seconds': 1.0010249614715576, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20324344, 'rttvar': 2268699, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00030994415283, 'end': 41.00133490562439, 'seconds': 1.0010249614715576, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00133490562439, 'end': 42.00131607055664, 'seconds': 0.999981164932251, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 70952, 'rtt': 20575846, 'rttvar': 2204529, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00133490562439, 'end': 42.00131607055664, 'seconds': 0.999981164932251, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.00131607055664, 'end': 43.00129199028015, 'seconds': 0.9999759197235107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21152923, 'rttvar': 2240313, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.00131607055664, 'end': 43.00129199028015, 'seconds': 0.9999759197235107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00129199028015, 'end': 44.00128412246704, 'seconds': 0.9999921321868896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21390857, 'rttvar': 2156102, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00129199028015, 'end': 44.00128412246704, 'seconds': 0.9999921321868896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00128412246704, 'end': 45.00132393836975, 'seconds': 1.00003981590271, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 21944667, 'rttvar': 2171328, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00128412246704, 'end': 45.00132393836975, 'seconds': 1.00003981590271, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00132393836975, 'end': 46.00119400024414, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22166043, 'rttvar': 2071249, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00132393836975, 'end': 46.00119400024414, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00119400024414, 'end': 47.00120401382446, 'seconds': 1.0000100135803223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 22688497, 'rttvar': 2069277, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00119400024414, 'end': 47.00120401382446, 'seconds': 1.0000100135803223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.00120401382446, 'end': 48.001296043395996, 'seconds': 1.0000920295715332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 22892047, 'rttvar': 1959057, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.00120401382446, 'end': 48.001296043395996, 'seconds': 1.0000920295715332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.001296043395996, 'end': 49.001302003860474, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 78192, 'rtt': 23160095, 'rttvar': 2005389, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.001296043395996, 'end': 49.001302003860474, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.001302003860474, 'end': 50.00115704536438, 'seconds': 0.9998550415039062, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23523362, 'rttvar': 1749697, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.001302003860474, 'end': 50.00115704536438, 'seconds': 0.9998550415039062, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00115704536438, 'end': 51.0013370513916, 'seconds': 1.0001800060272217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23747679, 'rttvar': 1760909, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00115704536438, 'end': 51.0013370513916, 'seconds': 1.0001800060272217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.0013370513916, 'end': 52.001291036605835, 'seconds': 0.9999539852142334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24127079, 'rttvar': 1648911, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.0013370513916, 'end': 52.001291036605835, 'seconds': 0.9999539852142334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.001291036605835, 'end': 53.001343965530396, 'seconds': 1.0000529289245605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24371031, 'rttvar': 1724587, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.001291036605835, 'end': 53.001343965530396, 'seconds': 1.0000529289245605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.001343965530396, 'end': 54.00108790397644, 'seconds': 0.9997439384460449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 24761265, 'rttvar': 1647864, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.001343965530396, 'end': 54.00108790397644, 'seconds': 0.9997439384460449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00108790397644, 'end': 55.00133490562439, 'seconds': 1.0002470016479492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 25018324, 'rttvar': 1750017, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00108790397644, 'end': 55.00133490562439, 'seconds': 1.0002470016479492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.00133490562439, 'end': 56.001347064971924, 'seconds': 1.0000121593475342, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 85432, 'rtt': 25243236, 'rttvar': 1762336, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.00133490562439, 'end': 56.001347064971924, 'seconds': 1.0000121593475342, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.001347064971924, 'end': 57.00115609169006, 'seconds': 0.9998090267181396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25692832, 'rttvar': 1793209, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.001347064971924, 'end': 57.00115609169006, 'seconds': 0.9998090267181396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.00115609169006, 'end': 58.001091957092285, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25921216, 'rttvar': 1801676, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.00115609169006, 'end': 58.001091957092285, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001091957092285, 'end': 59.001336097717285, 'seconds': 1.000244140625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 88328, 'rtt': 26385834, 'rttvar': 1842781, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001091957092285, 'end': 59.001336097717285, 'seconds': 1.000244140625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}"
11:13:04.255320 tst iperf3.py:8: ERR: KeyError: 'sum_sent' [trial-2066↪gprs:sysmo↪iperf3.py:8]
11:13:04.263636 tst iperf3.py:8: Test FAILED (204.2 sec)
11:13:04.270845 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18165): Terminating (SIGTERM)
11:13:04.314557 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18169): Terminating (SIGTERM)
11:13:04.359563 run osmo-hlr_10.42.42.2(pid=18170): Terminating (SIGTERM)
11:13:04.402283 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18171): Terminating (SIGTERM)
11:13:04.447378 run osmo-stp_10.42.42.5(pid=18172): Terminating (SIGTERM)
11:13:04.497587 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18173): Terminating (SIGTERM)
11:13:04.548729 run osmo-ggsn_10.42.42.6(pid=18178): Terminating (SIGTERM)
11:13:04.595315 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18199): Terminating (SIGTERM)
11:13:04.688440 run osmo-sgsn_10.42.42.7(pid=18200): Terminating (SIGTERM)
11:13:04.758854 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18201): Terminating (SIGTERM)
11:13:04.805754 run osmo-msc_10.42.42.8(pid=18202): Terminating (SIGTERM)
11:13:04.857642 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18203): Terminating (SIGTERM)
11:13:04.905237 run osmo-mgw_10.42.42.3(pid=18204): Terminating (SIGTERM)
11:13:04.949056 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18205): Terminating (SIGTERM)
11:13:04.993520 run osmo-mgw_10.42.42.4(pid=18206): Terminating (SIGTERM)
11:13:05.040144 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18207): Terminating (SIGTERM)
11:13:05.085216 run osmo-bsc_10.42.42.9(pid=18208): Terminating (SIGTERM)
11:13:05.131132 run osmo-bts-sysmo(pid=18220): Terminating (SIGTERM)
11:13:05.177081 run osmo-pcu(pid=18229): Terminating (SIGTERM)
11:13:05.221727 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Terminating (SIGTERM)
11:13:05.368871 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGTERM(pid=18280): Launched
11:13:06.455494 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGTERM(pid=18280): Terminated: ok {rc=0}
11:13:06.520172 --- ParallelTerminationStrategy: PID 18165 died...
11:13:06.539697 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18165): Terminated: ok {rc=0}
11:13:06.585853 --- ParallelTerminationStrategy: PID 18169 died...
11:13:06.605227 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18169): Terminated: ok {rc=0}
11:13:06.650556 --- ParallelTerminationStrategy: PID 18170 died...
11:13:06.670075 run osmo-hlr_10.42.42.2(pid=18170): Terminated: ok {rc=0}
11:13:06.715461 --- ParallelTerminationStrategy: PID 18171 died...
11:13:06.734826 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18171): Terminated: ok {rc=0}
11:13:06.780183 --- ParallelTerminationStrategy: PID 18172 died...
11:13:06.799335 run osmo-stp_10.42.42.5(pid=18172): Terminated: ok {rc=0}
11:13:06.844754 --- ParallelTerminationStrategy: PID 18173 died...
11:13:06.863966 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18173): Terminated: ok {rc=0}
11:13:06.908893 --- ParallelTerminationStrategy: PID 18178 died...
11:13:06.928175 run osmo-ggsn_10.42.42.6(pid=18178): Terminated: ok {rc=0}
11:13:06.974025 --- ParallelTerminationStrategy: PID 18199 died...
11:13:06.993398 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18199): Terminated: ok {rc=0}
11:13:07.038535 --- ParallelTerminationStrategy: PID 18200 died...
11:13:07.058462 run osmo-sgsn_10.42.42.7(pid=18200): Terminated: ok {rc=0}
11:13:07.103952 --- ParallelTerminationStrategy: PID 18201 died...
11:13:07.123264 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18201): Terminated: ok {rc=0}
11:13:07.168677 --- ParallelTerminationStrategy: PID 18202 died...
11:13:07.188045 run osmo-msc_10.42.42.8(pid=18202): Terminated: ok {rc=0}
11:13:07.233656 --- ParallelTerminationStrategy: PID 18203 died...
11:13:07.253044 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18203): Terminated: ok {rc=0}
11:13:07.299257 --- ParallelTerminationStrategy: PID 18204 died...
11:13:07.318412 run osmo-mgw_10.42.42.3(pid=18204): Terminated {rc=15}
11:13:07.364248 --- ParallelTerminationStrategy: PID 18205 died...
11:13:07.383661 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18205): Terminated: ok {rc=0}
11:13:07.429020 --- ParallelTerminationStrategy: PID 18206 died...
11:13:07.448622 run osmo-mgw_10.42.42.4(pid=18206): Terminated {rc=15}
11:13:07.494048 --- ParallelTerminationStrategy: PID 18207 died...
11:13:07.513407 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18207): Terminated: ok {rc=0}
11:13:07.559731 --- ParallelTerminationStrategy: PID 18208 died...
11:13:07.579036 run osmo-bsc_10.42.42.9(pid=18208): Terminated: ok {rc=0}
11:13:07.624468 --- ParallelTerminationStrategy: PID 18220 died...
11:13:07.643914 run osmo-bts-sysmo(pid=18220): Terminated: ok {rc=0}
11:13:07.689997 --- ParallelTerminationStrategy: PID 18229 died...
11:13:07.709524 run osmo-pcu(pid=18229): Terminated: ok {rc=0}
11:13:13.577586 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Terminating (SIGINT)
11:13:13.716304 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGINT(pid=18284): Launched
11:13:14.804576 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGINT(pid=18284): Terminated: ok {rc=0}
11:13:20.690494 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Terminating (SIGKILL)
11:13:20.828779 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGUSR1(pid=18287): Launched
11:13:21.917433 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGUSR1(pid=18287): Terminated: ok {rc=0}
11:13:22.168393 bus /gobi_6: Setting Online False
11:13:25.214842 bus /gobi_6: Setting Powered False
11:13:40.058039 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2066 gprs:sysmo iperf3m4.py
----------------------------------------------
11:13:41.269794 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.280975 tst gprs:sysmo: Using 1 x bts (candidates: 1)
11:13:41.313658 tst iperf3m4.py: using LAC 3961
11:13:41.435938 tst iperf3m4.py: using RAC 136
11:13:41.558148 tst iperf3m4.py: using CellId 3961
11:13:41.678506 tst iperf3m4.py: using BVCI 3962
11:13:41.791980 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.802506 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.812992 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.823635 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.834050 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.844506 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.855266 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.865903 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:13:41.876707 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:41.993417 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:42.110854 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:42.230150 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:13:42.346439 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
11:13:42.554418 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:42.866051 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Launched
11:13:43.138871 run iperf3-srv_10.42.42.10(pid=18291): Launched
11:13:43.256201 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
11:13:43.464412 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:43.775164 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Launched
11:13:44.045960 run iperf3-srv_10.42.42.10(pid=18293): Launched
11:13:44.162390 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
11:13:44.368125 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:44.677734 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Launched
11:13:44.948537 run iperf3-srv_10.42.42.10(pid=18295): Launched
11:13:45.065182 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
11:13:45.270882 run iperf3-srv_10.42.42.10: Starting iperf3-srv
11:13:45.581080 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Launched
11:13:45.851581 run iperf3-srv_10.42.42.10(pid=18297): Launched
11:13:45.966390 tst iperf3m4.py:8: start network...
11:13:46.172982 run osmo-hlr_10.42.42.2: Starting osmo-hlr
11:13:46.651034 run create_hlr_db(pid=18298): Launched
11:13:46.777524 bus /gobi_6: Setting Powered False
11:13:47.839600 run create_hlr_db(pid=18298): Terminated: ok {rc=0}
11:13:48.289546 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Launched
11:13:48.621197 run osmo-hlr_10.42.42.2(pid=18301): Launched
11:13:48.747454 run osmo-stp_10.42.42.5: Starting osmo-stp
11:13:49.315037 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Launched
11:13:49.648006 run osmo-stp_10.42.42.5(pid=18303): Launched
11:13:49.773762 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
11:13:50.338749 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Launched
11:13:50.465028 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
11:13:50.744236 run patchelf(pid=18305): Launched
11:13:51.932732 run patchelf(pid=18305): Terminated: ok {rc=0}
11:13:52.071359 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
11:13:52.353850 run osmo-gsm-tester_setcap_net_admin.sh(pid=18306): Launched
11:13:53.544564 run osmo-gsm-tester_setcap_net_admin.sh(pid=18306): Terminated: ok {rc=0}
11:13:53.890554 run osmo-ggsn_10.42.42.6(pid=18309): Launched
11:13:54.015745 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
11:13:54.757988 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Launched
11:13:55.090009 run osmo-sgsn_10.42.42.7(pid=18335): Launched
11:13:55.217492 run osmo-msc_10.42.42.8: Starting osmo-msc
11:13:55.836133 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Launched
11:13:56.149683 run osmo-msc_10.42.42.8(pid=18337): Launched
11:13:59.325397 run osmo-mgw_10.42.42.3: Starting osmo-mgw
11:13:59.855951 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Launched
11:14:00.165240 run osmo-mgw_10.42.42.3(pid=18339): Launched
11:14:00.284323 run osmo-mgw_10.42.42.4: Starting osmo-mgw
11:14:00.803053 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Launched
11:14:01.112489 run osmo-mgw_10.42.42.4(pid=18341): Launched
11:14:01.230208 run osmo-bsc_10.42.42.9: Starting osmo-bsc
11:14:01.872074 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Launched
11:14:02.181958 run osmo-bsc_10.42.42.9(pid=18343): Launched
11:14:08.457450 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
11:14:08.996694 run rm-remote-dir(pid=18344): Launched
11:14:11.183910 run rm-remote-dir(pid=18344): Terminated: ok {rc=0}
11:14:11.525640 run mk-remote-dir(pid=18345): Launched
11:14:12.709840 run mk-remote-dir(pid=18345): Terminated: ok {rc=0}
11:14:13.000821 run scp-inst-to-remote(pid=18346): Launched
11:14:25.197056 run scp-inst-to-remote(pid=18346): Terminated: ok {rc=0}
11:14:25.533786 run mk-remote-dir(pid=18348): Launched
11:14:26.714036 run mk-remote-dir(pid=18348): Terminated: ok {rc=0}
11:14:27.003281 run scp-cfg-to-remote(pid=18349): Launched
11:14:28.185728 run scp-cfg-to-remote(pid=18349): Terminated: ok {rc=0}
11:14:28.519337 run reload-dsp-firmware(pid=18351): Launched
11:14:30.698882 run reload-dsp-firmware(pid=18351): Terminated: ok {rc=0}
11:14:31.024129 run osmo-bts-sysmo(pid=18352): Launched
11:14:35.256943 tst iperf3m4.py:8: Waiting for bts to be ready...
11:14:36.976024 run rm-remote-dir(pid=18353): Launched
11:14:39.163238 run rm-remote-dir(pid=18353): Terminated: ok {rc=0}
11:14:39.513317 run mk-remote-dir(pid=18354): Launched
11:14:41.700768 run mk-remote-dir(pid=18354): Terminated: ok {rc=0}
11:14:42.000967 run scp-inst-to-bts(pid=18355): Launched
11:14:55.207562 run scp-inst-to-bts(pid=18355): Terminated: ok {rc=0}
11:14:55.537645 run mk-remote-run-dir(pid=18362): Launched
11:14:57.714414 run mk-remote-run-dir(pid=18362): Terminated: ok {rc=0}
11:14:57.994792 run scp-cfg-to-sysmobts(pid=18363): Launched
11:15:00.168929 run scp-cfg-to-sysmobts(pid=18363): Terminated: ok {rc=0}
11:15:00.496674 run osmo-pcu(pid=18365): Launched
11:15:00.623957 tst iperf3m4.py: using MSISDN 3974
11:15:00.801526 bus /gobi_6: Setting Powered True
11:15:04.344215 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3974', subscriber_id=1}
11:15:04.623385 bus /gobi_6: Setting Online False
11:15:05.814386 bus /gobi_6: Setting Powered False
11:15:13.122631 bus /gobi_6: Setting Powered True
11:15:14.525849 bus /gobi_6: Setting Online True
11:15:17.116610 tst /gobi_6: Connect to ('901', '70')
11:15:17.393574 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
11:15:17.521137 tst iperf3m4.py: using MSISDN 3975
11:15:17.751259 bus /gobi_0: Setting Powered True
11:15:21.267712 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3975', subscriber_id=2}
11:15:21.537818 bus /gobi_0: Setting Online False
11:15:22.725546 bus /gobi_0: Setting Powered False
11:15:30.114189 bus /gobi_0: Setting Powered True
11:15:31.471794 bus /gobi_0: Setting Online True
11:15:32.323742 tst /gobi_6: Already registered with network ('901', '70')
11:15:34.125600 tst /gobi_0: Connect to ('901', '70')
11:15:34.409462 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873'
11:15:34.538319 tst iperf3m4.py: using MSISDN 3976
11:15:34.771169 bus /gobi_1: Setting Powered True
11:15:38.389429 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3976', subscriber_id=3}
11:15:38.666355 bus /gobi_1: Setting Online False
11:15:39.863655 bus /gobi_1: Setting Powered False
11:15:47.133763 bus /gobi_1: Setting Powered True
11:15:48.532390 bus /gobi_1: Setting Online True
11:15:50.141716 tst /gobi_0: Already registered with network ('901', '70')
11:15:51.039537 tst /gobi_1: Connect to ('901', '70')
11:15:51.301288 tst /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024272103'
11:15:51.418502 tst iperf3m4.py: using MSISDN 3977
11:15:51.588386 bus /gobi_4: Setting Powered True
11:15:55.116634 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3977', subscriber_id=4}
11:15:55.398665 bus /gobi_4: Setting Online False
11:15:56.598014 bus /gobi_4: Setting Powered False
11:16:03.353570 bus /gobi_4: Setting Powered True
11:16:04.726015 bus /gobi_4: Setting Online True
11:16:07.183735 tst /gobi_4: Connect to ('901', '70')
11:16:07.455859 tst /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G 1 [Jul 21 2016 10:00:50]', Serial: '868323025161511'
11:16:07.565009 tst iperf3m4.py:8: waiting for modems to attach...
11:16:07.909355 tst /gobi_1: Already registered with network ('901', '70')
11:16:23.932124 tst /gobi_4: Already registered with network ('901', '70')
11:16:29.272614 tst iperf3m4.py:8: waiting for modems to attach to data services...
11:16:32.381617 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:16:32.492159 tst iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'"
11:16:32.909860 run move_netns(pid=18898): Launched
11:16:34.090592 run move_netns(pid=18898): Terminated: ok {rc=0}
11:16:34.421967 run ifup(pid=18901): Launched
11:16:35.606995 run ifup(pid=18901): Terminated: ok {rc=0}
11:16:35.939582 run dhcp(pid=18904): Launched
11:16:37.130982 run dhcp(pid=18904): Terminated: ok {rc=0}
11:16:45.533429 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:16:45.652996 tst iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'"
11:16:46.105881 run move_netns(pid=18922): Launched
11:16:47.302340 run move_netns(pid=18922): Terminated: ok {rc=0}
11:16:47.658389 run ifup(pid=18925): Launched
11:16:48.842098 run ifup(pid=18925): Terminated: ok {rc=0}
11:16:49.172840 run dhcp(pid=18928): Launched
11:16:50.355050 run dhcp(pid=18928): Terminated: ok {rc=0}
11:16:58.718078 tst /gobi_1: context activated {apn='inet46', path='/gobi_1/context11', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'}
11:16:58.838153 tst iperf3m4.py:8: Setting up data plan for "'/gobi_1/context11'"
11:16:59.290042 run move_netns(pid=18946): Launched
11:17:00.473121 run move_netns(pid=18946): Terminated: ok {rc=0}
11:17:00.803190 run ifup(pid=18949): Launched
11:17:01.983352 run ifup(pid=18949): Terminated: ok {rc=0}
11:17:02.312716 run dhcp(pid=18955): Launched
11:17:03.506370 run dhcp(pid=18955): Terminated: ok {rc=0}
11:17:11.902197 tst /gobi_4: context activated {apn='inet46', path='/gobi_4/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:17:12.012778 tst iperf3m4.py:8: Setting up data plan for "'/gobi_4/context2'"
11:17:12.430409 run move_netns(pid=18973): Launched
11:17:13.615647 run move_netns(pid=18973): Terminated: ok {rc=0}
11:17:13.945764 run ifup(pid=18976): Launched
11:17:15.139486 run ifup(pid=18976): Terminated: ok {rc=0}
11:17:15.493344 run dhcp(pid=18979): Launched
11:17:16.685317 run dhcp(pid=18979): Terminated: ok {rc=0}
11:17:16.823960 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'"
11:17:17.050679 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
11:17:17.443272 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Launched
11:17:17.620309 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'"
11:17:17.830360 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s)
11:17:18.196285 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Launched
11:17:18.369176 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context11'"
11:17:18.579430 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s)
11:17:18.944731 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Launched
11:17:19.115343 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context2'"
11:17:19.324508 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s)
11:17:19.691083 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Launched
11:17:20.012037 run iperf3-cli_10.42.42.10(pid=19007): Launched
11:17:20.279961 run iperf3-cli_10.42.42.10(pid=19010): Launched
11:17:20.550378 run iperf3-cli_10.42.42.10(pid=19013): Launched
11:17:20.819758 run iperf3-cli_10.42.42.10(pid=19016): Launched
11:18:29.121289 run iperf3-cli_10.42.42.10(pid=19007): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19007)]
11:18:29.268554 run iperf3-cli_10.42.42.10(pid=19007): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:29.412519 run iperf3-cli_10.42.42.10(pid=19007): stderr:
| (launched: 2022-09-07_11:17:19.996357)
11:18:35.615531 run iperf3-cli_10.42.42.10(pid=19010): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19010)]
11:18:35.748293 run iperf3-cli_10.42.42.10(pid=19010): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:35.877600 run iperf3-cli_10.42.42.10(pid=19010): stderr:
| (launched: 2022-09-07_11:17:20.264381)
11:18:37.059998 run iperf3-cli_10.42.42.10(pid=19013): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19013)]
11:18:37.188907 run iperf3-cli_10.42.42.10(pid=19013): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:37.317018 run iperf3-cli_10.42.42.10(pid=19013): stderr:
| (launched: 2022-09-07_11:17:20.534404)
11:18:38.501973 run iperf3-cli_10.42.42.10(pid=19016): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19016)]
11:18:38.633936 run iperf3-cli_10.42.42.10(pid=19016): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
11:18:38.765409 run iperf3-cli_10.42.42.10(pid=19016): stderr:
| (launched: 2022-09-07_11:17:20.804104)
11:18:38.894613 run iperf3-srv_10.42.42.10(pid=18291): Terminating (SIGINT)
11:18:39.235425 run iperf3-srv_10.42.42.10(pid=18291): Terminated {rc=1}
11:18:39.366741 tst iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'"
Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 46808, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Wed, 07 Sep 2022 09:17:21 GMT', 'timesecs': 1662542241}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'gb3tn5uvljqx37b2fk4tcfcx2rwfvavcpihs', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0006179809570312, 'seconds': 1.0006179809570312, 'bytes': 101360, 'bits_per_second': 810379.2010857548, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 451757, 'rttvar': 184583, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0006179809570312, 'seconds': 1.0006179809570312, 'bytes': 101360, 'bits_per_second': 810379.2010857548, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0006179809570312, 'end': 2.0012199878692627, 'seconds': 1.0006020069122314, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 582836, 'rttvar': 400596, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0006179809570312, 'end': 2.0012199878692627, 'seconds': 1.0006020069122314, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0012199878692627, 'end': 3.0012929439544678, 'seconds': 1.000072956085205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 4344, 'rtt': 582836, 'rttvar': 400596, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0012199878692627, 'end': 3.0012929439544678, 'seconds': 1.000072956085205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0012929439544678, 'end': 4.001316070556641, 'seconds': 1.0000231266021729, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 869981, 'rttvar': 874738, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0012929439544678, 'end': 4.001316070556641, 'seconds': 1.0000231266021729, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001316070556641, 'end': 5.001352071762085, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1313938, 'rttvar': 1543967, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001316070556641, 'end': 5.001352071762085, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001352071762085, 'end': 6.001249074935913, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1313938, 'rttvar': 1543967, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001352071762085, 'end': 6.001249074935913, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001249074935913, 'end': 7.001249074935913, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1894370, 'rttvar': 2318839, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001249074935913, 'end': 7.001249074935913, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001249074935913, 'end': 8.001253128051758, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2589750, 'rttvar': 3129890, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001249074935913, 'end': 8.001253128051758, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001253128051758, 'end': 9.001287937164307, 'seconds': 1.0000348091125488, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2589750, 'rttvar': 3129890, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001253128051758, 'end': 9.001287937164307, 'seconds': 1.0000348091125488, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001287937164307, 'end': 10.001184940338135, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 3388632, 'rttvar': 3945180, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001287937164307, 'end': 10.001184940338135, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001184940338135, 'end': 11.001253128051758, 'seconds': 1.000068187713623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4269873, 'rttvar': 4721368, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001184940338135, 'end': 11.001253128051758, 'seconds': 1.000068187713623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001253128051758, 'end': 12.00125503540039, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4269873, 'rttvar': 4721368, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001253128051758, 'end': 12.00125503540039, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.00125503540039, 'end': 13.001213073730469, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 5228419, 'rttvar': 5458117, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.00125503540039, 'end': 13.001213073730469, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001213073730469, 'end': 14.001102924346924, 'seconds': 0.9998898506164551, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6269643, 'rttvar': 6176036, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001213073730469, 'end': 14.001102924346924, 'seconds': 0.9998898506164551, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001102924346924, 'end': 15.001307010650635, 'seconds': 1.000204086303711, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6269643, 'rttvar': 6176036, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001102924346924, 'end': 15.001307010650635, 'seconds': 1.000204086303711, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001307010650635, 'end': 16.00133991241455, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7370460, 'rttvar': 6833661, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001307010650635, 'end': 16.00133991241455, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.00133991241455, 'end': 17.001264095306396, 'seconds': 0.9999241828918457, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7370460, 'rttvar': 6833661, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.00133991241455, 'end': 17.001264095306396, 'seconds': 0.9999241828918457, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001264095306396, 'end': 18.001466035842896, 'seconds': 1.000201940536499, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8413339, 'rttvar': 7211003, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001264095306396, 'end': 18.001466035842896, 'seconds': 1.000201940536499, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001466035842896, 'end': 19.00122594833374, 'seconds': 0.9997599124908447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9441186, 'rttvar': 7463947, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001466035842896, 'end': 19.00122594833374, 'seconds': 0.9997599124908447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.00122594833374, 'end': 20.00122308731079, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9441186, 'rttvar': 7463947, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.00122594833374, 'end': 20.00122308731079, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.00122308731079, 'end': 21.00119709968567, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9441186, 'rttvar': 7463947, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.00122308731079, 'end': 21.00119709968567, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.00119709968567, 'end': 22.001327991485596, 'seconds': 1.0001308917999268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10548308, 'rttvar': 7812205, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.00119709968567, 'end': 22.001327991485596, 'seconds': 1.0001308917999268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001327991485596, 'end': 23.001250982284546, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10548308, 'rttvar': 7812205, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001327991485596, 'end': 23.001250982284546, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001250982284546, 'end': 24.00134801864624, 'seconds': 1.0000970363616943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 11709727, 'rttvar': 8181992, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001250982284546, 'end': 24.00134801864624, 'seconds': 1.0000970363616943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.00134801864624, 'end': 25.001326084136963, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12730721, 'rttvar': 8178482, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.00134801864624, 'end': 25.001326084136963, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001326084136963, 'end': 26.001183032989502, 'seconds': 0.9998569488525391, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12730721, 'rttvar': 8178482, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001326084136963, 'end': 26.001183032989502, 'seconds': 0.9998569488525391, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.001183032989502, 'end': 27.001212120056152, 'seconds': 1.0000290870666504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 13811324, 'rttvar': 8295066, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.001183032989502, 'end': 27.001212120056152, 'seconds': 1.0000290870666504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.001212120056152, 'end': 28.0013370513916, 'seconds': 1.0001249313354492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14754703, 'rttvar': 8108058, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.001212120056152, 'end': 28.0013370513916, 'seconds': 1.0001249313354492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.0013370513916, 'end': 29.001312017440796, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14754703, 'rttvar': 8108058, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.0013370513916, 'end': 29.001312017440796, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.001312017440796, 'end': 30.001383066177368, 'seconds': 1.0000710487365723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 15765410, 'rttvar': 8102458, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.001312017440796, 'end': 30.001383066177368, 'seconds': 1.0000710487365723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.001383066177368, 'end': 31.00122308731079, 'seconds': 0.9998400211334229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16651975, 'rttvar': 7849974, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.001383066177368, 'end': 31.00122308731079, 'seconds': 0.9998400211334229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00122308731079, 'end': 32.00126504898071, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16651975, 'rttvar': 7849974, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00122308731079, 'end': 32.00126504898071, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00126504898071, 'end': 33.00120496749878, 'seconds': 0.9999399185180664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 17620390, 'rttvar': 7824310, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00126504898071, 'end': 33.00120496749878, 'seconds': 0.9999399185180664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00120496749878, 'end': 34.00120210647583, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18470040, 'rttvar': 7567532, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00120496749878, 'end': 34.00120210647583, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00120210647583, 'end': 35.00121092796326, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18470040, 'rttvar': 7567532, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00120210647583, 'end': 35.00121092796326, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00121092796326, 'end': 36.0013530254364, 'seconds': 1.0001420974731445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 19398025, 'rttvar': 7531619, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00121092796326, 'end': 36.0013530254364, 'seconds': 1.0001420974731445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.0013530254364, 'end': 37.00132703781128, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20215248, 'rttvar': 7283161, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.0013530254364, 'end': 37.00132703781128, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00132703781128, 'end': 38.00142502784729, 'seconds': 1.0000979900360107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20215248, 'rttvar': 7283161, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00132703781128, 'end': 38.00142502784729, 'seconds': 1.0000979900360107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00142502784729, 'end': 39.00122904777527, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 21120726, 'rttvar': 7273326, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00142502784729, 'end': 39.00122904777527, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.00122904777527, 'end': 40.00126791000366, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21915414, 'rttvar': 7044371, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.00122904777527, 'end': 40.00126791000366, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00126791000366, 'end': 41.00125002861023, 'seconds': 0.9999821186065674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21915414, 'rttvar': 7044371, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00126791000366, 'end': 41.00125002861023, 'seconds': 0.9999821186065674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00125002861023, 'end': 42.00132608413696, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 22798230, 'rttvar': 7048910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00125002861023, 'end': 42.00132608413696, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.00132608413696, 'end': 43.00132513046265, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23560844, 'rttvar': 6811911, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.00132608413696, 'end': 43.00132513046265, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00132513046265, 'end': 44.00127196311951, 'seconds': 0.9999468326568604, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23560844, 'rttvar': 6811911, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00132513046265, 'end': 44.00127196311951, 'seconds': 0.9999468326568604, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00127196311951, 'end': 45.00125503540039, 'seconds': 0.9999830722808838, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 24430574, 'rttvar': 6848393, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00127196311951, 'end': 45.00125503540039, 'seconds': 0.9999830722808838, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00125503540039, 'end': 46.00121593475342, 'seconds': 0.9999608993530273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25181883, 'rttvar': 6638913, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00125503540039, 'end': 46.00121593475342, 'seconds': 0.9999608993530273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00121593475342, 'end': 47.00106906890869, 'seconds': 0.9998531341552734, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25181883, 'rttvar': 6638913, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00121593475342, 'end': 47.00106906890869, 'seconds': 0.9998531341552734, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.00106906890869, 'end': 48.001070976257324, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26036572, 'rttvar': 6688561, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.00106906890869, 'end': 48.001070976257324, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.001070976257324, 'end': 49.001359939575195, 'seconds': 1.000288963317871, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26036572, 'rttvar': 6688561, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.001070976257324, 'end': 49.001359939575195, 'seconds': 1.000288963317871, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.001359939575195, 'end': 50.00132393836975, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 26776847, 'rttvar': 6496971, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.001359939575195, 'end': 50.00132393836975, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00132393836975, 'end': 51.00134897232056, 'seconds': 1.0000250339508057, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27612180, 'rttvar': 6543396, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00132393836975, 'end': 51.00134897232056, 'seconds': 1.0000250339508057, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.00134897232056, 'end': 52.00131392478943, 'seconds': 0.9999649524688721, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27612180, 'rttvar': 6543396, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.00134897232056, 'end': 52.00131392478943, 'seconds': 0.9999649524688721, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00131392478943, 'end': 53.001079082489014, 'seconds': 0.999765157699585, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 28348183, 'rttvar': 6379552, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00131392478943, 'end': 53.001079082489014, 'seconds': 0.999765157699585, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.001079082489014, 'end': 54.0007209777832, 'seconds': 0.9996418952941895, 'bytes': 256296, 'bits_per_second': 2051102.5094607377, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29182041, 'rttvar': 6452381, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.001079082489014, 'end': 54.0007209777832, 'seconds': 0.9996418952941895, 'bytes': 256296, 'bits_per_second': 2051102.5094607377, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.0007209777832, 'end': 55.001338958740234, 'seconds': 1.0006179809570312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29182041, 'rttvar': 6452381, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.0007209777832, 'end': 55.001338958740234, 'seconds': 1.0006179809570312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.001338958740234, 'end': 56.00119709968567, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 29716673, 'rttvar': 5908549, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.001338958740234, 'end': 56.00119709968567, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.00119709968567, 'end': 57.001075983047485, 'seconds': 0.9998788833618164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30371991, 'rttvar': 5742048, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.00119709968567, 'end': 57.001075983047485, 'seconds': 0.9998788833618164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.001075983047485, 'end': 58.00132513046265, 'seconds': 1.0002491474151611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30371991, 'rttvar': 5742048, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.001075983047485, 'end': 58.00132513046265, 'seconds': 1.0002491474151611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.00132513046265, 'end': 59.00113892555237, 'seconds': 0.9998137950897217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 30947602, 'rttvar': 5457757, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.00132513046265, 'end': 59.00113892555237, 'seconds': 0.9998137950897217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}"
11:18:39.609822 tst iperf3m4.py:8: ERR: KeyError: 'sum_sent' [trial-2066↪gprs:sysmo↪iperf3m4.py:8]
11:18:39.617997 tst iperf3m4.py:8: Test FAILED (298.4 sec)
11:18:39.625962 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Terminating (SIGTERM)
11:18:39.670378 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Terminating (SIGTERM)
11:18:39.714503 run iperf3-srv_10.42.42.10(pid=18293): Terminating (SIGTERM)
11:18:39.759741 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Terminating (SIGTERM)
11:18:39.804539 run iperf3-srv_10.42.42.10(pid=18295): Terminating (SIGTERM)
11:18:39.848208 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Terminating (SIGTERM)
11:18:39.892508 run iperf3-srv_10.42.42.10(pid=18297): Terminating (SIGTERM)
11:18:39.937384 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Terminating (SIGTERM)
11:18:39.981943 run osmo-hlr_10.42.42.2(pid=18301): Terminating (SIGTERM)
11:18:40.025709 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Terminating (SIGTERM)
11:18:40.070897 run osmo-stp_10.42.42.5(pid=18303): Terminating (SIGTERM)
11:18:40.117832 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Terminating (SIGTERM)
11:18:40.184842 run osmo-ggsn_10.42.42.6(pid=18309): Terminating (SIGTERM)
11:18:40.232574 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Terminating (SIGTERM)
11:18:40.340723 run osmo-sgsn_10.42.42.7(pid=18335): Terminating (SIGTERM)
11:18:40.389425 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Terminating (SIGTERM)
11:18:40.440090 run osmo-msc_10.42.42.8(pid=18337): Terminating (SIGTERM)
11:18:40.494932 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Terminating (SIGTERM)
11:18:40.545425 run osmo-mgw_10.42.42.3(pid=18339): Terminating (SIGTERM)
11:18:40.588266 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Terminating (SIGTERM)
11:18:40.630286 run osmo-mgw_10.42.42.4(pid=18341): Terminating (SIGTERM)
11:18:40.673025 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Terminating (SIGTERM)
11:18:40.714612 run osmo-bsc_10.42.42.9(pid=18343): Terminating (SIGTERM)
11:18:40.755996 run osmo-bts-sysmo(pid=18352): Terminating (SIGTERM)
11:18:40.798049 run osmo-pcu(pid=18365): Terminating (SIGTERM)
11:18:40.840088 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGTERM)
11:18:40.980895 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGTERM(pid=19043): Launched
11:18:42.063449 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGTERM(pid=19043): Terminated: ok {rc=0}
11:18:42.127608 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGTERM)
11:18:42.265803 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGTERM(pid=19048): Launched
11:18:43.352481 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGTERM(pid=19048): Terminated: ok {rc=0}
11:18:43.416531 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGTERM)
11:18:43.554436 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGTERM(pid=19051): Launched
11:18:44.637433 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGTERM(pid=19051): Terminated: ok {rc=0}
11:18:44.701515 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGTERM)
11:18:44.839318 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGTERM(pid=19054): Launched
11:18:45.922859 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGTERM(pid=19054): Terminated: ok {rc=0}
11:18:45.987486 --- ParallelTerminationStrategy: PID 18290 died...
11:18:46.006582 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Terminated: ok {rc=0}
11:18:46.052270 --- ParallelTerminationStrategy: PID 18292 died...
11:18:46.071181 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Terminated: ok {rc=0}
11:18:46.116571 --- ParallelTerminationStrategy: PID 18293 died...
11:18:46.135393 run iperf3-srv_10.42.42.10(pid=18293): Terminated {rc=256}
11:18:46.180226 --- ParallelTerminationStrategy: PID 18294 died...
11:18:46.200058 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Terminated: ok {rc=0}
11:18:46.245144 --- ParallelTerminationStrategy: PID 18295 died...
11:18:46.263952 run iperf3-srv_10.42.42.10(pid=18295): Terminated {rc=256}
11:18:46.311037 --- ParallelTerminationStrategy: PID 18296 died...
11:18:46.330242 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Terminated: ok {rc=0}
11:18:46.374985 --- ParallelTerminationStrategy: PID 18297 died...
11:18:46.393650 run iperf3-srv_10.42.42.10(pid=18297): Terminated {rc=256}
11:18:46.438591 --- ParallelTerminationStrategy: PID 18300 died...
11:18:46.458022 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Terminated: ok {rc=0}
11:18:46.503810 --- ParallelTerminationStrategy: PID 18301 died...
11:18:46.522776 run osmo-hlr_10.42.42.2(pid=18301): Terminated: ok {rc=0}
11:18:46.568326 --- ParallelTerminationStrategy: PID 18302 died...
11:18:46.587163 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Terminated: ok {rc=0}
11:18:46.632287 --- ParallelTerminationStrategy: PID 18303 died...
11:18:46.651043 run osmo-stp_10.42.42.5(pid=18303): Terminated: ok {rc=0}
11:18:46.695847 --- ParallelTerminationStrategy: PID 18304 died...
11:18:46.714830 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Terminated: ok {rc=0}
11:18:46.759767 --- ParallelTerminationStrategy: PID 18309 died...
11:18:46.778532 run osmo-ggsn_10.42.42.6(pid=18309): Terminated: ok {rc=0}
11:18:46.823473 --- ParallelTerminationStrategy: PID 18334 died...
11:18:46.842349 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Terminated: ok {rc=0}
11:18:46.887334 --- ParallelTerminationStrategy: PID 18335 died...
11:18:46.906090 run osmo-sgsn_10.42.42.7(pid=18335): Terminated: ok {rc=0}
11:18:46.951172 --- ParallelTerminationStrategy: PID 18336 died...
11:18:46.969981 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Terminated: ok {rc=0}
11:18:47.015321 --- ParallelTerminationStrategy: PID 18337 died...
11:18:47.034366 run osmo-msc_10.42.42.8(pid=18337): Terminated: ok {rc=0}
11:18:47.079922 --- ParallelTerminationStrategy: PID 18338 died...
11:18:47.099126 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Terminated: ok {rc=0}
11:18:47.144769 --- ParallelTerminationStrategy: PID 18339 died...
11:18:47.164205 run osmo-mgw_10.42.42.3(pid=18339): Terminated {rc=15}
11:18:47.210439 --- ParallelTerminationStrategy: PID 18340 died...
11:18:47.229857 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Terminated: ok {rc=0}
11:18:47.275673 --- ParallelTerminationStrategy: PID 18341 died...
11:18:47.294902 run osmo-mgw_10.42.42.4(pid=18341): Terminated {rc=15}
11:18:47.340717 --- ParallelTerminationStrategy: PID 18342 died...
11:18:47.360054 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Terminated: ok {rc=0}
11:18:47.405599 --- ParallelTerminationStrategy: PID 18343 died...
11:18:47.424837 run osmo-bsc_10.42.42.9(pid=18343): Terminated: ok {rc=0}
11:18:47.470885 --- ParallelTerminationStrategy: PID 18352 died...
11:18:47.490334 run osmo-bts-sysmo(pid=18352): Terminated: ok {rc=0}
11:18:47.536164 --- ParallelTerminationStrategy: PID 18365 died...
11:18:47.555499 run osmo-pcu(pid=18365): Terminated: ok {rc=0}
11:18:53.423303 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGINT)
11:18:53.563664 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGINT(pid=19057): Launched
11:18:54.651478 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGINT(pid=19057): Terminated: ok {rc=0}
11:18:54.715835 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGINT)
11:18:54.853799 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGINT(pid=19060): Launched
11:18:55.946039 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGINT(pid=19060): Terminated: ok {rc=0}
11:18:56.012848 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGINT)
11:18:56.155859 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGINT(pid=19063): Launched
11:18:57.243305 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGINT(pid=19063): Terminated: ok {rc=0}
11:18:57.309916 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGINT)
11:18:57.453670 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGINT(pid=19070): Launched
11:18:58.541060 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGINT(pid=19070): Terminated: ok {rc=0}
11:19:04.430310 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGKILL)
11:19:04.574963 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGUSR1(pid=19073): Launched
11:19:05.663461 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGUSR1(pid=19073): Terminated: ok {rc=0}
11:19:05.729991 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGKILL)
11:19:05.873226 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGUSR1(pid=19076): Launched
11:19:06.958654 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGUSR1(pid=19076): Terminated: ok {rc=0}
11:19:07.025099 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGKILL)
11:19:07.168460 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGUSR1(pid=19079): Launched
11:19:08.253713 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGUSR1(pid=19079): Terminated: ok {rc=0}
11:19:08.320246 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGKILL)
11:19:08.463566 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGUSR1(pid=19082): Launched
11:19:09.554269 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGUSR1(pid=19082): Terminated: ok {rc=0}
11:19:09.801696 bus /gobi_4: Setting Online False
11:19:10.970235 bus /gobi_4: Setting Powered False
11:19:20.943960 bus /gobi_4: Setting Powered False
11:19:22.041570 bus /gobi_4: Setting Powered False
11:19:23.202581 bus /gobi_1: Setting Online False
11:19:24.313472 bus /gobi_1: Setting Powered False
11:19:32.084697 bus /gobi_1: Setting Powered False
11:19:33.184756 bus /gobi_1: Setting Powered False
11:19:34.345073 bus /gobi_0: Setting Online False
11:19:35.478508 bus /gobi_0: Setting Powered False
11:19:43.940610 bus /gobi_0: Setting Powered False
11:19:45.039619 bus /gobi_0: Setting Powered False
11:19:46.198600 bus /gobi_6: Setting Online False
11:19:47.317208 bus /gobi_6: Setting Powered False
11:19:55.094016 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2066 gprs:sysmo ping.py
----------------------------------------------
11:19:56.257163 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.283259 tst gprs:sysmo: Using 1 x bts (candidates: 1)
11:19:56.319563 tst ping.py: using LAC 3962
11:19:56.448492 tst ping.py: using RAC 137
11:19:56.575545 tst ping.py: using CellId 3962
11:19:56.703233 tst ping.py: using BVCI 3963
11:19:56.821932 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.833027 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.844200 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.855403 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.866318 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.877411 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.888745 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:19:56.899952 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:19:57.014636 tst ping.py:19: start network...
11:19:57.216427 run osmo-hlr_10.42.42.2: Starting osmo-hlr
11:19:57.685353 run create_hlr_db(pid=19087): Launched
11:19:57.808218 bus /gobi_6: Setting Powered False
11:19:58.868358 run create_hlr_db(pid=19087): Terminated: ok {rc=0}
11:19:59.290654 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19089): Launched
11:19:59.594575 run osmo-hlr_10.42.42.2(pid=19090): Launched
11:19:59.712925 run osmo-stp_10.42.42.5: Starting osmo-stp
11:20:00.233312 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19091): Launched
11:20:00.539676 run osmo-stp_10.42.42.5(pid=19092): Launched
11:20:00.655733 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
11:20:01.177314 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19093): Launched
11:20:01.294179 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
11:20:01.550636 run patchelf(pid=19094): Launched
11:20:02.724298 run patchelf(pid=19094): Terminated: ok {rc=0}
11:20:02.851039 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
11:20:03.110145 run osmo-gsm-tester_setcap_net_admin.sh(pid=19098): Launched
11:20:04.282313 run osmo-gsm-tester_setcap_net_admin.sh(pid=19098): Terminated: ok {rc=0}
11:20:04.597611 run osmo-ggsn_10.42.42.6(pid=19101): Launched
11:20:04.720280 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
11:20:05.416959 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19118): Launched
11:20:05.725684 run osmo-sgsn_10.42.42.7(pid=19119): Launched
11:20:05.843598 run osmo-msc_10.42.42.8: Starting osmo-msc
11:20:06.395358 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19120): Launched
11:20:06.700688 run osmo-msc_10.42.42.8(pid=19121): Launched
11:20:09.872905 run osmo-mgw_10.42.42.3: Starting osmo-mgw
11:20:10.390917 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19122): Launched
11:20:10.696194 run osmo-mgw_10.42.42.3(pid=19123): Launched
11:20:10.812078 run osmo-mgw_10.42.42.4: Starting osmo-mgw
11:20:11.320278 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19124): Launched
11:20:11.625331 run osmo-mgw_10.42.42.4(pid=19125): Launched
11:20:11.741911 run osmo-bsc_10.42.42.9: Starting osmo-bsc
11:20:12.370163 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19126): Launched
11:20:12.674856 run osmo-bsc_10.42.42.9(pid=19127): Launched
11:20:18.943494 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
11:20:19.470279 run rm-remote-dir(pid=19128): Launched
11:20:21.651641 run rm-remote-dir(pid=19128): Terminated: ok {rc=0}
11:20:21.984969 run mk-remote-dir(pid=19129): Launched
11:20:23.163504 run mk-remote-dir(pid=19129): Terminated: ok {rc=0}
11:20:23.447599 run scp-inst-to-remote(pid=19130): Launched
11:20:35.640572 run scp-inst-to-remote(pid=19130): Terminated: ok {rc=0}
11:20:35.991736 run mk-remote-dir(pid=19132): Launched
11:20:37.180640 run mk-remote-dir(pid=19132): Terminated: ok {rc=0}
11:20:37.482449 run scp-cfg-to-remote(pid=19133): Launched
11:20:38.670082 run scp-cfg-to-remote(pid=19133): Terminated: ok {rc=0}
11:20:39.018025 run reload-dsp-firmware(pid=19135): Launched
11:20:41.205451 run reload-dsp-firmware(pid=19135): Terminated: ok {rc=0}
11:20:41.546419 run osmo-bts-sysmo(pid=19136): Launched
11:20:45.775827 tst ping.py:31: Waiting for bts to be ready...
11:20:47.445366 run rm-remote-dir(pid=19137): Launched
11:20:49.618647 run rm-remote-dir(pid=19137): Terminated: ok {rc=0}
11:20:49.937919 run mk-remote-dir(pid=19138): Launched
11:20:52.114537 run mk-remote-dir(pid=19138): Terminated: ok {rc=0}
11:20:52.408842 run scp-inst-to-bts(pid=19139): Launched
11:21:05.605499 run scp-inst-to-bts(pid=19139): Terminated: ok {rc=0}
11:21:05.946758 run mk-remote-run-dir(pid=19141): Launched
11:21:08.129713 run mk-remote-run-dir(pid=19141): Terminated: ok {rc=0}
11:21:08.404971 run scp-cfg-to-sysmobts(pid=19142): Launched
11:21:10.578013 run scp-cfg-to-sysmobts(pid=19142): Terminated: ok {rc=0}
11:21:10.898545 run osmo-pcu(pid=19144): Launched
11:21:11.026008 tst ping.py: using MSISDN 3978
11:21:11.201236 bus /gobi_6: Setting Powered True
11:21:14.719849 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3978', subscriber_id=1}
11:21:14.990331 bus /gobi_6: Setting Online False
11:21:16.175369 bus /gobi_6: Setting Powered False
11:21:23.109816 bus /gobi_6: Setting Powered True
11:21:24.489313 bus /gobi_6: Setting Online True
11:21:26.982946 tst /gobi_6: Connect to ('901', '70')
11:21:27.244544 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
11:21:27.354534 tst ping.py:42: waiting for modems to attach...
11:21:41.329012 tst /gobi_6: Already registered with network ('901', '70')
11:21:46.800451 tst ping.py:46: waiting for modems to attach to data services...
11:21:49.873795 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:21:49.990804 tst ping.py:51: Setting up data plan for "'/gobi_6/context3'"
11:21:50.432512 run move_netns(pid=19148): Launched
11:21:51.624143 run move_netns(pid=19148): Terminated: ok {rc=0}
11:21:51.954152 run ifup(pid=19151): Launched
11:21:53.130211 run ifup(pid=19151): Terminated: ok {rc=0}
11:21:53.458426 run dhcp(pid=19154): Launched
11:21:54.634888 run dhcp(pid=19154): Terminated: ok {rc=0}
11:21:54.764748 tst ping.py:53: Running 10 ping requests for "'/gobi_6/context3'"
11:21:55.172159 run ping(pid=19170): Launched
11:22:05.378317 run ping(pid=19170): Terminated: ok {rc=0}
11:22:07.075266 tst /gobi_6: context deactivated {path='/gobi_6/context3'}
11:22:14.223126 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={Interface='r433306bbc2604b'}, Name='Internet', Password='', Protocol='ipv6', Settings={}, Type='internet', Username='ogt'}, user='ogt'}
11:22:21.535059 tst /gobi_6: context deactivated {path='/gobi_6/context3'}
11:22:21.644398 tst ping.py:70: (launched: 2022-09-07_11:21:55.109240)
19171: waiting for 19172
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=491 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=431 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=390 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=386 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=386 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=363 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=622 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=604 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=521 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=499 ms
--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 20ms
rtt min/avg/max/mdev = 363.029/469.397/621.852/88.624 ms
child exited with 0
11:22:21.849698 tst ping.py: Test passed (145.6 sec)
11:22:21.916060 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19089): Terminating (SIGTERM)
11:22:21.957380 run osmo-hlr_10.42.42.2(pid=19090): Terminating (SIGTERM)
11:22:21.998309 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19091): Terminating (SIGTERM)
11:22:22.041517 run osmo-stp_10.42.42.5(pid=19092): Terminating (SIGTERM)
11:22:22.086075 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19093): Terminating (SIGTERM)
11:22:22.129373 run osmo-ggsn_10.42.42.6(pid=19101): Terminating (SIGTERM)
11:22:22.175282 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19118): Terminating (SIGTERM)
11:22:22.221279 run osmo-sgsn_10.42.42.7(pid=19119): Terminating (SIGTERM)
11:22:22.265588 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19120): Terminating (SIGTERM)
11:22:22.309282 run osmo-msc_10.42.42.8(pid=19121): Terminating (SIGTERM)
11:22:22.352320 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19122): Terminating (SIGTERM)
11:22:22.402023 run osmo-mgw_10.42.42.3(pid=19123): Terminating (SIGTERM)
11:22:22.460466 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19124): Terminating (SIGTERM)
11:22:22.506244 run osmo-mgw_10.42.42.4(pid=19125): Terminating (SIGTERM)
11:22:22.549719 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19126): Terminating (SIGTERM)
11:22:22.593499 run osmo-bsc_10.42.42.9(pid=19127): Terminating (SIGTERM)
11:22:22.636286 run osmo-bts-sysmo(pid=19136): Terminating (SIGTERM)
11:22:22.679331 run osmo-pcu(pid=19144): Terminating (SIGTERM)
11:22:22.722658 --- ParallelTerminationStrategy: PID 19089 died...
11:22:22.743146 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19089): Terminated: ok {rc=0}
11:22:22.791463 --- ParallelTerminationStrategy: PID 19090 died...
11:22:22.811376 run osmo-hlr_10.42.42.2(pid=19090): Terminated: ok {rc=0}
11:22:22.860408 --- ParallelTerminationStrategy: PID 19091 died...
11:22:22.881030 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19091): Terminated: ok {rc=0}
11:22:22.960992 --- ParallelTerminationStrategy: PID 19092 died...
11:22:22.981721 run osmo-stp_10.42.42.5(pid=19092): Terminated: ok {rc=0}
11:22:23.031240 --- ParallelTerminationStrategy: PID 19093 died...
11:22:23.051803 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19093): Terminated: ok {rc=0}
11:22:23.114613 --- ParallelTerminationStrategy: PID 19101 died...
11:22:23.144609 run osmo-ggsn_10.42.42.6(pid=19101): Terminated: ok {rc=0}
11:22:23.194981 --- ParallelTerminationStrategy: PID 19118 died...
11:22:23.214114 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19118): Terminated: ok {rc=0}
11:22:23.259237 --- ParallelTerminationStrategy: PID 19120 died...
11:22:23.278628 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19120): Terminated: ok {rc=0}
11:22:23.326103 --- ParallelTerminationStrategy: PID 19121 died...
11:22:23.346010 run osmo-msc_10.42.42.8(pid=19121): Terminated: ok {rc=0}
11:22:23.394352 --- ParallelTerminationStrategy: PID 19122 died...
11:22:23.415084 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19122): Terminated: ok {rc=0}
11:22:23.463323 --- ParallelTerminationStrategy: PID 19123 died...
11:22:23.483768 run osmo-mgw_10.42.42.3(pid=19123): Terminated {rc=15}
11:22:23.532178 --- ParallelTerminationStrategy: PID 19124 died...
11:22:23.551978 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19124): Terminated: ok {rc=0}
11:22:23.598379 --- ParallelTerminationStrategy: PID 19119 died...
11:22:23.617902 run osmo-sgsn_10.42.42.7(pid=19119): Terminated: ok {rc=0}
11:22:23.663621 --- ParallelTerminationStrategy: PID 19125 died...
11:22:23.682665 run osmo-mgw_10.42.42.4(pid=19125): Terminated {rc=15}
11:22:23.728492 --- ParallelTerminationStrategy: PID 19126 died...
11:22:23.747725 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19126): Terminated: ok {rc=0}
11:22:23.793189 --- ParallelTerminationStrategy: PID 19127 died...
11:22:23.812317 run osmo-bsc_10.42.42.9(pid=19127): Terminated: ok {rc=0}
11:22:23.857961 --- ParallelTerminationStrategy: PID 19136 died...
11:22:23.877070 run osmo-bts-sysmo(pid=19136): Terminated: ok {rc=0}
11:22:23.922602 --- ParallelTerminationStrategy: PID 19144 died...
11:22:23.941946 run osmo-pcu(pid=19144): Terminated: ok {rc=0}
11:22:24.165335 bus /gobi_6: Setting Online False
11:22:28.240201 bus /gobi_6: Setting Powered False
11:22:46.060293 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2066 gprs:sysmo ping_idle_ping.py
----------------------------------------------
11:22:47.205379 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.217110 tst gprs:sysmo: Using 1 x bts (candidates: 1)
11:22:47.252188 tst ping_idle_ping.py: using LAC 3963
11:22:47.379621 tst ping_idle_ping.py: using RAC 138
11:22:47.507284 tst ping_idle_ping.py: using CellId 3963
11:22:47.633308 tst ping_idle_ping.py: using BVCI 3964
11:22:47.753226 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.764455 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.775558 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.786824 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.797991 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.808995 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.820462 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
11:22:47.831641 tst gprs:sysmo: Using 1 x modem (candidates: 4)
11:22:47.950945 tst ping_idle_ping.py:24: start network...
11:22:48.169217 run osmo-hlr_10.42.42.2: Starting osmo-hlr
11:22:48.674869 run create_hlr_db(pid=19195): Launched
11:22:48.807992 bus /gobi_6: Setting Powered False
11:22:49.872541 run create_hlr_db(pid=19195): Terminated: ok {rc=0}
11:22:50.308319 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19197): Launched
11:22:50.614663 run osmo-hlr_10.42.42.2(pid=19198): Launched
11:22:50.731360 run osmo-stp_10.42.42.5: Starting osmo-stp
11:22:51.252100 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19199): Launched
11:22:51.557975 run osmo-stp_10.42.42.5(pid=19200): Launched
11:22:51.675055 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
11:22:52.197743 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19201): Launched
11:22:52.314639 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
11:22:52.571678 run patchelf(pid=19202): Launched
11:22:53.746974 run patchelf(pid=19202): Terminated: ok {rc=0}
11:22:53.875105 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
11:22:54.135510 run osmo-gsm-tester_setcap_net_admin.sh(pid=19203): Launched
11:22:55.312881 run osmo-gsm-tester_setcap_net_admin.sh(pid=19203): Terminated: ok {rc=0}
11:22:55.629656 run osmo-ggsn_10.42.42.6(pid=19206): Launched
11:22:55.754350 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
11:22:56.464523 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19223): Launched
11:22:56.792380 run osmo-sgsn_10.42.42.7(pid=19224): Launched
11:22:56.919117 run osmo-msc_10.42.42.8: Starting osmo-msc
11:22:57.514142 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19225): Launched
11:22:57.843640 run osmo-msc_10.42.42.8(pid=19226): Launched
11:23:01.024850 run osmo-mgw_10.42.42.3: Starting osmo-mgw
11:23:01.583986 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19227): Launched
11:23:01.911672 run osmo-mgw_10.42.42.3(pid=19228): Launched
11:23:02.036155 run osmo-mgw_10.42.42.4: Starting osmo-mgw
11:23:02.585101 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19229): Launched
11:23:02.913543 run osmo-mgw_10.42.42.4(pid=19230): Launched
11:23:03.046300 run osmo-bsc_10.42.42.9: Starting osmo-bsc
11:23:03.722638 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19233): Launched
11:23:04.041414 run osmo-bsc_10.42.42.9(pid=19234): Launched
11:23:10.312966 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
11:23:10.843314 run rm-remote-dir(pid=19235): Launched
11:23:13.024745 run rm-remote-dir(pid=19235): Terminated: ok {rc=0}
11:23:13.359557 run mk-remote-dir(pid=19236): Launched
11:23:14.537819 run mk-remote-dir(pid=19236): Terminated: ok {rc=0}
11:23:14.824237 run scp-inst-to-remote(pid=19237): Launched
11:23:27.022020 run scp-inst-to-remote(pid=19237): Terminated: ok {rc=0}
11:23:27.374594 run mk-remote-dir(pid=19239): Launched
11:23:28.558129 run mk-remote-dir(pid=19239): Terminated: ok {rc=0}
11:23:28.841908 run scp-cfg-to-remote(pid=19240): Launched
11:23:30.018837 run scp-cfg-to-remote(pid=19240): Terminated: ok {rc=0}
11:23:30.345348 run reload-dsp-firmware(pid=19242): Launched
11:23:32.519675 run reload-dsp-firmware(pid=19242): Terminated: ok {rc=0}
11:23:32.839361 run osmo-bts-sysmo(pid=19243): Launched
11:23:37.062602 tst ping_idle_ping.py:36: Waiting for bts to be ready...
11:23:38.733647 run rm-remote-dir(pid=19244): Launched
11:23:40.907149 run rm-remote-dir(pid=19244): Terminated: ok {rc=0}
11:23:41.228405 run mk-remote-dir(pid=19245): Launched
11:23:43.405648 run mk-remote-dir(pid=19245): Terminated: ok {rc=0}
11:23:43.700953 run scp-inst-to-bts(pid=19246): Launched
11:23:56.894104 run scp-inst-to-bts(pid=19246): Terminated: ok {rc=0}
11:23:57.215829 run mk-remote-run-dir(pid=19248): Launched
11:23:59.387313 run mk-remote-run-dir(pid=19248): Terminated: ok {rc=0}
11:23:59.663390 run scp-cfg-to-sysmobts(pid=19249): Launched
11:24:01.839330 run scp-cfg-to-sysmobts(pid=19249): Terminated: ok {rc=0}
11:24:02.168751 run osmo-pcu(pid=19251): Launched
11:24:02.307345 tst ping_idle_ping.py: using MSISDN 3979
11:24:02.477201 bus /gobi_6: Setting Powered True
11:24:06.008935 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3979', subscriber_id=1}
11:24:06.291579 bus /gobi_6: Setting Online False
11:24:07.483862 bus /gobi_6: Setting Powered False
11:24:14.246844 bus /gobi_6: Setting Powered True
11:24:15.622207 bus /gobi_6: Setting Online True
11:24:18.104373 tst /gobi_6: Connect to ('901', '70')
11:24:18.375481 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
11:24:18.483656 tst ping_idle_ping.py:47: waiting for modems to attach...
11:24:25.484972 tst /gobi_6: Already registered with network ('901', '70')
11:24:36.936369 tst ping_idle_ping.py:51: waiting for modems to attach to data services...
11:24:54.190230 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
11:24:54.307532 tst ping_idle_ping.py:56: Setting up data plane for "'/gobi_6/context2'"
11:24:54.751779 run move_netns(pid=19254): Launched
11:24:55.944416 run move_netns(pid=19254): Terminated: ok {rc=0}
11:24:56.295667 run ifup(pid=19257): Launched
11:24:57.488444 run ifup(pid=19257): Terminated: ok {rc=0}
11:24:57.840210 run dhcp(pid=19260): Launched
11:24:59.028730 run dhcp(pid=19260): Terminated: ok {rc=0}
11:24:59.167414 tst ping_idle_ping.py:60: [1] Running 10 ping requests for "'/gobi_6/context2'"
11:24:59.597331 run ping1(pid=19287): Launched
11:25:09.799242 run ping1(pid=19287): Terminated: ok {rc=0}
11:25:09.928708 tst ping_idle_ping.py:64: (launched: 2022-09-07_11:24:59.532934)
19288: waiting for 19289
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=558 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=559 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=519 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=515 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=637 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=616 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=599 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=618 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=399 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=515 ms
--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 11ms
rtt min/avg/max/mdev = 398.572/553.507/637.368/67.349 ms
child exited with 0
11:25:10.135637 tst ping_idle_ping.py:68: Sleeping for 60 seconds
11:26:10.342590 tst ping_idle_ping.py:73: [2] Running 10 ping requests for "'/gobi_6/context2'"
11:26:10.775860 run ping2(pid=19295): Launched
11:26:20.981663 run ping2(pid=19295): Terminated: ok {rc=0}
11:26:21.111562 tst ping_idle_ping.py:77: (launched: 2022-09-07_11:26:10.710394)
19296: waiting for 19297
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=501 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=424 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=383 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=379 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=382 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=360 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=619 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=603 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=622 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=619 ms
--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 11ms
rtt min/avg/max/mdev = 360.123/489.363/622.350/109.597 ms
child exited with 0
11:26:23.034554 tst /gobi_6: context deactivated {path='/gobi_6/context2'}
11:26:23.146924 tst ping_idle_ping.py: Test passed (215.9 sec)
11:26:23.212314 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19197): Terminating (SIGTERM)
11:26:23.253370 run osmo-hlr_10.42.42.2(pid=19198): Terminating (SIGTERM)
11:26:23.295375 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19199): Terminating (SIGTERM)
11:26:23.338821 run osmo-stp_10.42.42.5(pid=19200): Terminating (SIGTERM)
11:26:23.388688 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19201): Terminating (SIGTERM)
11:26:23.435804 run osmo-ggsn_10.42.42.6(pid=19206): Terminating (SIGTERM)
11:26:23.482983 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19223): Terminating (SIGTERM)
11:26:23.560553 run osmo-sgsn_10.42.42.7(pid=19224): Terminating (SIGTERM)
11:26:23.613308 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19225): Terminating (SIGTERM)
11:26:23.660030 run osmo-msc_10.42.42.8(pid=19226): Terminating (SIGTERM)
11:26:23.707199 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19227): Terminating (SIGTERM)
11:26:23.752565 run osmo-mgw_10.42.42.3(pid=19228): Terminating (SIGTERM)
11:26:23.797286 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19229): Terminating (SIGTERM)
11:26:23.841381 run osmo-mgw_10.42.42.4(pid=19230): Terminating (SIGTERM)
11:26:23.885131 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19233): Terminating (SIGTERM)
11:26:23.930601 run osmo-bsc_10.42.42.9(pid=19234): Terminating (SIGTERM)
11:26:23.974126 run osmo-bts-sysmo(pid=19243): Terminating (SIGTERM)
11:26:24.017983 run osmo-pcu(pid=19251): Terminating (SIGTERM)
11:26:24.061712 --- ParallelTerminationStrategy: PID 19197 died...
11:26:24.082343 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19197): Terminated: ok {rc=0}
11:26:24.132621 --- ParallelTerminationStrategy: PID 19198 died...
11:26:24.153827 run osmo-hlr_10.42.42.2(pid=19198): Terminated: ok {rc=0}
11:26:24.204143 --- ParallelTerminationStrategy: PID 19199 died...
11:26:24.225242 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19199): Terminated: ok {rc=0}
11:26:24.307258 --- ParallelTerminationStrategy: PID 19200 died...
11:26:24.328295 run osmo-stp_10.42.42.5(pid=19200): Terminated: ok {rc=0}
11:26:24.387525 --- ParallelTerminationStrategy: PID 19201 died...
11:26:24.408547 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19201): Terminated: ok {rc=0}
11:26:24.459227 --- ParallelTerminationStrategy: PID 19206 died...
11:26:24.480092 run osmo-ggsn_10.42.42.6(pid=19206): Terminated: ok {rc=0}
11:26:24.528835 --- ParallelTerminationStrategy: PID 19223 died...
11:26:24.548866 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19223): Terminated: ok {rc=0}
11:26:24.596467 --- ParallelTerminationStrategy: PID 19225 died...
11:26:24.616680 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19225): Terminated: ok {rc=0}
11:26:24.666121 --- ParallelTerminationStrategy: PID 19226 died...
11:26:24.687163 run osmo-msc_10.42.42.8(pid=19226): Terminated: ok {rc=0}
11:26:24.736906 --- ParallelTerminationStrategy: PID 19227 died...
11:26:24.757866 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19227): Terminated: ok {rc=0}
11:26:24.807782 --- ParallelTerminationStrategy: PID 19228 died...
11:26:24.828971 run osmo-mgw_10.42.42.3(pid=19228): Terminated {rc=15}
11:26:24.878970 --- ParallelTerminationStrategy: PID 19229 died...
11:26:24.899954 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19229): Terminated: ok {rc=0}
11:26:24.949544 --- ParallelTerminationStrategy: PID 19230 died...
11:26:24.970537 run osmo-mgw_10.42.42.4(pid=19230): Terminated {rc=15}
11:26:25.020348 --- ParallelTerminationStrategy: PID 19233 died...
11:26:25.041695 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19233): Terminated: ok {rc=0}
11:26:25.091868 --- ParallelTerminationStrategy: PID 19234 died...
11:26:25.112985 run osmo-bsc_10.42.42.9(pid=19234): Terminated: ok {rc=0}
11:26:25.163089 --- ParallelTerminationStrategy: PID 19243 died...
11:26:25.184165 run osmo-bts-sysmo(pid=19243): Terminated: ok {rc=0}
11:26:25.234154 --- ParallelTerminationStrategy: PID 19251 died...
11:26:25.255335 run osmo-pcu(pid=19251): Terminated: ok {rc=0}
11:26:25.473386 --- ParallelTerminationStrategy: PID 19224 died...
11:26:25.495556 run osmo-sgsn_10.42.42.7(pid=19224): Terminated: ok {rc=0}
11:26:25.718598 bus /gobi_6: Setting Online False
11:26:28.780101 bus /gobi_6: Setting Powered False
11:26:45.945949 bus /gobi_6: Setting Powered False
---------------------------------------------------------------------
trial-2066 gprs:sysmo FAIL
---------------------------------------------------------------------
11:26:47.281387 tst trial-2066: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2066/run.2022-09-07_11-02-04/trial-2066.xml
--------------------------------------------------------------------------------------------
trial-2066 FAIL
--------------------------------------------------------------------------------------------
11:26:47.496099 tst trial-2066: trial-2066: FAIL, 2 suites failed, 1 suites passed
FAIL: nitb_sms:sysmo (fail: 1)
FAIL: mo_mt_sms.py (4.8 sec) Error: osmo-nitb_10.42.42.2(pid=17962): Process ended prematurely: osmo-nitb_10.42.42.2(pid=17962) [trial-2066↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=17962)]
FAIL: gprs:sysmo (fail: 3, pass: 2)
FAIL: cs_paging_gprs_active.py (247.5 sec) KeyError: 'sum_sent'
FAIL: iperf3.py (204.2 sec) KeyError: 'sum_sent'
FAIL: iperf3m4.py (298.4 sec) KeyError: 'sum_sent'
pass: ping.py (145.6 sec)
pass: ping_idle_ping.py (215.9 sec)
PASS: sms:sysmo (pass: 1)
pass: mo_mt_sms.py (115.6 sec)
+ rm -rf trial-2066/inst
+ cd trial-2066
+ command -v journalctl
+ [ ! -z /bin/journalctl ]
+ readlink last_run
+ journalctl -u ofono -o short-precise --since 2022-09-07 11:02:01
+ readlink last_run
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2066-run.tgz run.2022-09-07_11-02-04
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2066-bin.tgz checksums.md5 open5gs.build-8.tgz osmo-bsc.build-16888.tgz osmo-bts-oc2g.build-5192.tgz osmo-bts-sysmo.build-7696.tgz osmo-bts.build-7166.tgz osmo-ggsn.build-6364.tgz osmo-hlr.build-8262.tgz osmo-hnbgw.build-2020.tgz osmo-mgw.build-8470.tgz osmo-msc.build-28078.tgz osmo-nitb.build-12346.tgz osmo-pcu-oc2g.build-3690.tgz osmo-pcu-sysmo.build-4971.tgz osmo-pcu.build-4933.tgz osmo-sgsn.build-28683.tgz osmo-stp.build-8799.tgz osmo-trx.build-4360.tgz osmocom-bb.build-4442.tgz srslte.build-19.tgz
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
ERROR Gerrit response: ConflictFinished: FAILURE
![[Jenkins]](/jenkins/static/342c126a/images/svgs/logo.svg){kind=logo}
