Skipping 79 KB..
Full Log18:39:17.862752 tst cs_paging_gprs_active.py:38: ERR: KeyError: 'sum_sent' [trial-2064↪gprs:sysmo↪cs_paging_gprs_active.py:38]
18:39:17.875668 tst cs_paging_gprs_active.py:38: Test FAILED (228.9 sec)
18:39:17.883131 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Terminating (SIGTERM)
18:39:17.928872 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Terminating (SIGTERM)
18:39:17.972357 run iperf3-srv_10.42.42.10(pid=26684): Terminating (SIGTERM)
18:39:18.015494 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Terminating (SIGTERM)
18:39:18.058581 run osmo-hlr_10.42.42.2(pid=26688): Terminating (SIGTERM)
18:39:18.102081 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Terminating (SIGTERM)
18:39:18.147511 run osmo-stp_10.42.42.5(pid=26690): Terminating (SIGTERM)
18:39:18.198400 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Terminating (SIGTERM)
18:39:18.246865 run osmo-ggsn_10.42.42.6(pid=26696): Terminating (SIGTERM)
18:39:18.295120 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Terminating (SIGTERM)
18:39:18.348084 run osmo-sgsn_10.42.42.7(pid=26716): Terminating (SIGTERM)
18:39:18.400548 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Terminating (SIGTERM)
18:39:18.471460 run osmo-msc_10.42.42.8(pid=26718): Terminating (SIGTERM)
18:39:18.541263 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Terminating (SIGTERM)
18:39:18.586758 run osmo-mgw_10.42.42.3(pid=26720): Terminating (SIGTERM)
18:39:18.633718 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Terminating (SIGTERM)
18:39:18.678200 run osmo-mgw_10.42.42.4(pid=26722): Terminating (SIGTERM)
18:39:18.721873 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Terminating (SIGTERM)
18:39:18.765687 run osmo-bsc_10.42.42.9(pid=26724): Terminating (SIGTERM)
18:39:18.808811 run osmo-bts-sysmo(pid=26733): Terminating (SIGTERM)
18:39:18.852191 run osmo-pcu(pid=26742): Terminating (SIGTERM)
18:39:18.896577 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGTERM)
18:39:19.041243 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGTERM(pid=26820): Launched
18:39:20.147122 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGTERM(pid=26820): Terminated: ok {rc=0}
18:39:20.213485 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGTERM)
18:39:20.357004 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGTERM(pid=26825): Launched
18:39:21.450010 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGTERM(pid=26825): Terminated: ok {rc=0}
18:39:21.516329 --- ParallelTerminationStrategy: PID 26681 died...
18:39:21.536410 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Terminated: ok {rc=0}
18:39:21.583475 --- ParallelTerminationStrategy: PID 26683 died...
18:39:21.603319 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Terminated: ok {rc=0}
18:39:21.655678 --- ParallelTerminationStrategy: PID 26684 died...
18:39:21.675303 run iperf3-srv_10.42.42.10(pid=26684): Terminated {rc=256}
18:39:21.722589 --- ParallelTerminationStrategy: PID 26687 died...
18:39:21.743029 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Terminated: ok {rc=0}
18:39:21.789911 --- ParallelTerminationStrategy: PID 26688 died...
18:39:21.809718 run osmo-hlr_10.42.42.2(pid=26688): Terminated: ok {rc=0}
18:39:21.856930 --- ParallelTerminationStrategy: PID 26689 died...
18:39:21.876757 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Terminated: ok {rc=0}
18:39:21.923958 --- ParallelTerminationStrategy: PID 26690 died...
18:39:21.944181 run osmo-stp_10.42.42.5(pid=26690): Terminated: ok {rc=0}
18:39:21.990647 --- ParallelTerminationStrategy: PID 26691 died...
18:39:22.010317 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Terminated: ok {rc=0}
18:39:22.056924 --- ParallelTerminationStrategy: PID 26696 died...
18:39:22.077177 run osmo-ggsn_10.42.42.6(pid=26696): Terminated: ok {rc=0}
18:39:22.124009 --- ParallelTerminationStrategy: PID 26715 died...
18:39:22.143710 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Terminated: ok {rc=0}
18:39:22.190268 --- ParallelTerminationStrategy: PID 26716 died...
18:39:22.209909 run osmo-sgsn_10.42.42.7(pid=26716): Terminated: ok {rc=0}
18:39:22.256464 --- ParallelTerminationStrategy: PID 26717 died...
18:39:22.276078 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Terminated: ok {rc=0}
18:39:22.322820 --- ParallelTerminationStrategy: PID 26718 died...
18:39:22.342421 run osmo-msc_10.42.42.8(pid=26718): Terminated: ok {rc=0}
18:39:22.389583 --- ParallelTerminationStrategy: PID 26719 died...
18:39:22.409411 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Terminated: ok {rc=0}
18:39:22.456512 --- ParallelTerminationStrategy: PID 26720 died...
18:39:22.476280 run osmo-mgw_10.42.42.3(pid=26720): Terminated {rc=15}
18:39:22.523405 --- ParallelTerminationStrategy: PID 26721 died...
18:39:22.543318 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Terminated: ok {rc=0}
18:39:22.590518 --- ParallelTerminationStrategy: PID 26722 died...
18:39:22.610815 run osmo-mgw_10.42.42.4(pid=26722): Terminated {rc=15}
18:39:22.657754 --- ParallelTerminationStrategy: PID 26723 died...
18:39:22.677601 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Terminated: ok {rc=0}
18:39:22.724123 --- ParallelTerminationStrategy: PID 26724 died...
18:39:22.744086 run osmo-bsc_10.42.42.9(pid=26724): Terminated: ok {rc=0}
18:39:22.790849 --- ParallelTerminationStrategy: PID 26733 died...
18:39:22.810783 run osmo-bts-sysmo(pid=26733): Terminated: ok {rc=0}
18:39:22.857900 --- ParallelTerminationStrategy: PID 26742 died...
18:39:22.878159 run osmo-pcu(pid=26742): Terminated: ok {rc=0}
18:39:28.747801 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGINT)
18:39:28.895566 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGINT(pid=26828): Launched
18:39:29.981810 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGINT(pid=26828): Terminated: ok {rc=0}
18:39:30.047964 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGINT)
18:39:30.190153 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGINT(pid=26831): Launched
18:39:31.279904 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGINT(pid=26831): Terminated: ok {rc=0}
18:39:37.169022 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGKILL)
18:39:37.317978 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGUSR1(pid=26834): Launched
18:39:38.402893 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGUSR1(pid=26834): Terminated: ok {rc=0}
18:39:38.468708 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGKILL)
18:39:38.613806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGUSR1(pid=26837): Launched
18:39:39.699613 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGUSR1(pid=26837): Terminated: ok {rc=0}
18:39:39.952782 bus /gobi_0: Setting Online False
18:39:41.109901 bus /gobi_0: Setting Powered False
18:39:57.933861 bus /gobi_0: Setting Powered False
18:39:59.031683 bus /gobi_0: Setting Powered False
18:40:00.187625 bus /gobi_6: Setting Online False
18:40:01.324454 bus /gobi_6: Setting Powered False
18:40:09.937679 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2064 gprs:sysmo iperf3.py
----------------------------------------------
18:40:11.133475 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.144585 tst gprs:sysmo: Using 1 x bts (candidates: 1)
18:40:11.179606 tst iperf3.py: using LAC 3946
18:40:11.296486 tst iperf3.py: using RAC 121
18:40:11.417856 tst iperf3.py: using CellId 3946
18:40:11.534562 tst iperf3.py: using BVCI 3947
18:40:11.652136 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.663475 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.674472 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.685761 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.696994 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.708244 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.719346 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.730187 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.741327 tst gprs:sysmo: Using 1 x modem (candidates: 4)
18:40:11.859782 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003...
18:40:12.076074 run iperf3-srv_10.42.42.10: Starting iperf3-srv
18:40:12.401713 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Launched
18:40:12.684914 run iperf3-srv_10.42.42.10(pid=26844): Launched
18:40:12.809071 tst iperf3.py:8: start network...
18:40:13.028324 run osmo-hlr_10.42.42.2: Starting osmo-hlr
18:40:13.531050 run create_hlr_db(pid=26845): Launched
18:40:13.655300 bus /gobi_6: Setting Powered False
18:40:14.719347 run create_hlr_db(pid=26845): Terminated: ok {rc=0}
18:40:15.171589 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Launched
18:40:15.495601 run osmo-hlr_10.42.42.2(pid=26848): Launched
18:40:15.620203 run osmo-stp_10.42.42.5: Starting osmo-stp
18:40:16.173360 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Launched
18:40:16.499230 run osmo-stp_10.42.42.5(pid=26850): Launched
18:40:16.622825 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:40:17.179654 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Launched
18:40:17.304391 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:40:17.580170 run patchelf(pid=26852): Launched
18:40:18.763309 run patchelf(pid=26852): Terminated: ok {rc=0}
18:40:18.900791 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:40:19.182582 run osmo-gsm-tester_setcap_net_admin.sh(pid=26853): Launched
18:40:20.368115 run osmo-gsm-tester_setcap_net_admin.sh(pid=26853): Terminated: ok {rc=0}
18:40:20.708483 run osmo-ggsn_10.42.42.6(pid=26856): Launched
18:40:20.833195 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:40:21.546002 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Launched
18:40:21.871360 run osmo-sgsn_10.42.42.7(pid=26880): Launched
18:40:21.994607 run osmo-msc_10.42.42.8: Starting osmo-msc
18:40:22.580961 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Launched
18:40:22.905293 run osmo-msc_10.42.42.8(pid=26882): Launched
18:40:26.086848 run osmo-mgw_10.42.42.3: Starting osmo-mgw
18:40:26.639561 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Launched
18:40:26.965928 run osmo-mgw_10.42.42.3(pid=26884): Launched
18:40:27.091891 run osmo-mgw_10.42.42.4: Starting osmo-mgw
18:40:27.641697 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Launched
18:40:27.970361 run osmo-mgw_10.42.42.4(pid=26886): Launched
18:40:28.095899 run osmo-bsc_10.42.42.9: Starting osmo-bsc
18:40:28.771906 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Launched
18:40:29.102019 run osmo-bsc_10.42.42.9(pid=26888): Launched
18:40:35.392301 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:40:35.962825 run rm-remote-dir(pid=26889): Launched
18:40:38.159596 run rm-remote-dir(pid=26889): Terminated: ok {rc=0}
18:40:38.519994 run mk-remote-dir(pid=26890): Launched
18:40:39.711928 run mk-remote-dir(pid=26890): Terminated: ok {rc=0}
18:40:40.019352 run scp-inst-to-remote(pid=26891): Launched
18:40:52.225207 run scp-inst-to-remote(pid=26891): Terminated: ok {rc=0}
18:40:52.557665 run mk-remote-dir(pid=26893): Launched
18:40:53.735584 run mk-remote-dir(pid=26893): Terminated: ok {rc=0}
18:40:54.021235 run scp-cfg-to-remote(pid=26894): Launched
18:40:55.199682 run scp-cfg-to-remote(pid=26894): Terminated: ok {rc=0}
18:40:55.528849 run reload-dsp-firmware(pid=26896): Launched
18:40:57.706484 run reload-dsp-firmware(pid=26896): Terminated: ok {rc=0}
18:40:58.028650 run osmo-bts-sysmo(pid=26897): Launched
18:41:02.260450 tst iperf3.py:8: Waiting for bts to be ready...
18:41:03.982569 run rm-remote-dir(pid=26898): Launched
18:41:06.164315 run rm-remote-dir(pid=26898): Terminated: ok {rc=0}
18:41:06.487881 run mk-remote-dir(pid=26899): Launched
18:41:08.658824 run mk-remote-dir(pid=26899): Terminated: ok {rc=0}
18:41:08.936153 run scp-inst-to-bts(pid=26900): Launched
18:41:22.123036 run scp-inst-to-bts(pid=26900): Terminated: ok {rc=0}
18:41:22.446638 run mk-remote-run-dir(pid=26902): Launched
18:41:24.623632 run mk-remote-run-dir(pid=26902): Terminated: ok {rc=0}
18:41:24.920198 run scp-cfg-to-sysmobts(pid=26903): Launched
18:41:27.107023 run scp-cfg-to-sysmobts(pid=26903): Terminated: ok {rc=0}
18:41:27.452717 run osmo-pcu(pid=26905): Launched
18:41:27.588774 tst iperf3.py: using MSISDN 3951
18:41:27.764385 bus /gobi_6: Setting Powered True
18:41:31.283264 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3951', subscriber_id=1}
18:41:31.558641 bus /gobi_6: Setting Online False
18:41:32.751208 bus /gobi_6: Setting Powered False
18:41:40.133664 bus /gobi_6: Setting Powered True
18:41:41.535326 bus /gobi_6: Setting Online True
18:41:44.084922 tst /gobi_6: Connect to ('901', '70')
18:41:44.346316 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
18:41:44.455164 tst iperf3.py:8: waiting for modems to attach...
18:41:49.961316 tst /gobi_6: Already registered with network ('901', '70')
18:42:01.906359 tst iperf3.py:8: waiting for modems to attach to data services...
18:42:06.570252 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:42:06.688943 tst iperf3.py:8: Setting up data plan for "'/gobi_6/context3'"
18:42:07.139193 run move_netns(pid=26909): Launched
18:42:08.333756 run move_netns(pid=26909): Terminated: ok {rc=0}
18:42:08.696771 run ifup(pid=26912): Launched
18:42:09.889153 run ifup(pid=26912): Terminated: ok {rc=0}
18:42:10.245155 run dhcp(pid=26915): Launched
18:42:11.427619 run dhcp(pid=26915): Terminated: ok {rc=0}
18:42:11.556435 tst iperf3.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context3'"
18:42:11.766122 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
18:42:12.130786 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Launched
18:42:12.455587 run iperf3-cli_10.42.42.10(pid=26934): Launched
18:43:26.776963 run iperf3-cli_10.42.42.10(pid=26934): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=26934)]
18:43:26.915744 run iperf3-cli_10.42.42.10(pid=26934): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
18:43:27.057591 run iperf3-cli_10.42.42.10(pid=26934): stderr:
| (launched: 2022-03-29_18:42:12.439797)
18:43:27.196695 run iperf3-srv_10.42.42.10(pid=26844): Terminating (SIGINT)
18:43:27.541012 run iperf3-srv_10.42.42.10(pid=26844): Terminated {rc=1}
18:43:27.667529 tst iperf3.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context3'"
Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42866, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:42:19 GMT', 'timesecs': 1648572139}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'vfzcc6k7krnaudu2qhmwk43mkmmn3xd5enfe', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0008230209350586, 'seconds': 1.0008230209350586, 'bytes': 101360, 'bits_per_second': 810213.1775929807, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 416760, 'rttvar': 179188, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0008230209350586, 'seconds': 1.0008230209350586, 'bytes': 101360, 'bits_per_second': 810213.1775929807, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0008230209350586, 'end': 2.001343011856079, 'seconds': 1.0005199909210205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 504319, 'rttvar': 309509, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0008230209350586, 'end': 2.001343011856079, 'seconds': 1.0005199909210205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.001343011856079, 'end': 3.0013041496276855, 'seconds': 0.9999611377716064, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 18824, 'rtt': 887851, 'rttvar': 861683, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.001343011856079, 'end': 3.0013041496276855, 'seconds': 0.9999611377716064, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0013041496276855, 'end': 4.0011889934539795, 'seconds': 0.999884843826294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1169081, 'rttvar': 1208723, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0013041496276855, 'end': 4.0011889934539795, 'seconds': 0.999884843826294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.0011889934539795, 'end': 5.001459121704102, 'seconds': 1.000270128250122, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 1502871, 'rttvar': 1574122, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.0011889934539795, 'end': 5.001459121704102, 'seconds': 1.000270128250122, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001459121704102, 'end': 6.001329183578491, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 2303976, 'rttvar': 2296626, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001459121704102, 'end': 6.001329183578491, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001329183578491, 'end': 7.001317024230957, 'seconds': 0.9999878406524658, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2758356, 'rttvar': 2631231, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001329183578491, 'end': 7.001317024230957, 'seconds': 0.9999878406524658, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001317024230957, 'end': 8.001265048980713, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 3754779, 'rttvar': 3230560, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001317024230957, 'end': 8.001265048980713, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001265048980713, 'end': 9.001326084136963, 'seconds': 1.00006103515625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 4181887, 'rttvar': 3277135, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001265048980713, 'end': 9.001326084136963, 'seconds': 1.00006103515625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001326084136963, 'end': 10.001069068908691, 'seconds': 0.9997429847717285, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 4614394, 'rttvar': 3322867, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001326084136963, 'end': 10.001069068908691, 'seconds': 0.9997429847717285, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001069068908691, 'end': 11.001177072525024, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 5090389, 'rttvar': 3444140, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001069068908691, 'end': 11.001177072525024, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001177072525024, 'end': 12.001214981079102, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 5594052, 'rttvar': 3590431, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001177072525024, 'end': 12.001214981079102, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001214981079102, 'end': 13.001265048980713, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 6514479, 'rttvar': 3639030, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001214981079102, 'end': 13.001265048980713, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001265048980713, 'end': 14.001188039779663, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 6935277, 'rttvar': 3570868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001265048980713, 'end': 14.001188039779663, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001188039779663, 'end': 15.001317024230957, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 7784605, 'rttvar': 3480777, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001188039779663, 'end': 15.001317024230957, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001317024230957, 'end': 16.001317977905273, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 8221413, 'rttvar': 3484198, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001317024230957, 'end': 16.001317977905273, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001317977905273, 'end': 17.00117516517639, 'seconds': 0.9998571872711182, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9025140, 'rttvar': 3376202, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001317977905273, 'end': 17.00117516517639, 'seconds': 0.9998571872711182, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.00117516517639, 'end': 18.00126600265503, 'seconds': 1.0000908374786377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9394544, 'rttvar': 3270958, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.00117516517639, 'end': 18.00126600265503, 'seconds': 1.0000908374786377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.00126600265503, 'end': 19.00116515159607, 'seconds': 0.99989914894104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 9802550, 'rttvar': 3269232, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.00126600265503, 'end': 19.00116515159607, 'seconds': 0.99989914894104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.00116515159607, 'end': 20.001224040985107, 'seconds': 1.000058889389038, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 10159568, 'rttvar': 3165960, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.00116515159607, 'end': 20.001224040985107, 'seconds': 1.000058889389038, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001224040985107, 'end': 21.001187086105347, 'seconds': 0.9999630451202393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 10992522, 'rttvar': 3229428, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001224040985107, 'end': 21.001187086105347, 'seconds': 0.9999630451202393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001187086105347, 'end': 22.00130820274353, 'seconds': 1.0001211166381836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 11438503, 'rttvar': 3314034, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001187086105347, 'end': 22.00130820274353, 'seconds': 1.0001211166381836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.00130820274353, 'end': 23.001317024230957, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11833439, 'rttvar': 3275397, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.00130820274353, 'end': 23.001317024230957, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001317024230957, 'end': 24.001163005828857, 'seconds': 0.9998459815979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 12276390, 'rttvar': 3342449, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001317024230957, 'end': 24.001163005828857, 'seconds': 0.9998459815979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001163005828857, 'end': 25.001341104507446, 'seconds': 1.0001780986785889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 13150739, 'rttvar': 3424898, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001163005828857, 'end': 25.001341104507446, 'seconds': 1.0001780986785889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001341104507446, 'end': 26.001182079315186, 'seconds': 0.9998409748077393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 13558774, 'rttvar': 3384745, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001341104507446, 'end': 26.001182079315186, 'seconds': 0.9998409748077393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.001182079315186, 'end': 27.000904083251953, 'seconds': 0.9997220039367676, 'bytes': 256296, 'bits_per_second': 2050938.1527323928, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 14016223, 'rttvar': 3453455, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.001182079315186, 'end': 27.000904083251953, 'seconds': 0.9997220039367676, 'bytes': 256296, 'bits_per_second': 2050938.1527323928, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.000904083251953, 'end': 28.001161098480225, 'seconds': 1.0002570152282715, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 14676016, 'rttvar': 3109571, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.000904083251953, 'end': 28.001161098480225, 'seconds': 1.0002570152282715, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001161098480225, 'end': 29.00123620033264, 'seconds': 1.000075101852417, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14991541, 'rttvar': 2963228, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001161098480225, 'end': 29.00123620033264, 'seconds': 1.000075101852417, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.00123620033264, 'end': 30.000699996948242, 'seconds': 0.9994637966156006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 15673002, 'rttvar': 2847857, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.00123620033264, 'end': 30.000699996948242, 'seconds': 0.9994637966156006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.000699996948242, 'end': 31.00126814842224, 'seconds': 1.000568151473999, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 16031214, 'rttvar': 2852316, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.000699996948242, 'end': 31.00126814842224, 'seconds': 1.000568151473999, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00126814842224, 'end': 32.00126504898071, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 16699289, 'rttvar': 2785032, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00126814842224, 'end': 32.00126504898071, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00126504898071, 'end': 33.00132417678833, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 17019166, 'rttvar': 2728527, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00126504898071, 'end': 33.00132417678833, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00132417678833, 'end': 34.00120401382446, 'seconds': 0.9998798370361328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17650620, 'rttvar': 2615336, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00132417678833, 'end': 34.00120401382446, 'seconds': 0.9998798370361328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00120401382446, 'end': 35.00131702423096, 'seconds': 1.0001130104064941, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17964070, 'rttvar': 2588400, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00120401382446, 'end': 35.00131702423096, 'seconds': 1.0001130104064941, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00131702423096, 'end': 36.00126600265503, 'seconds': 0.9999489784240723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 18568273, 'rttvar': 2527256, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00131702423096, 'end': 36.00126600265503, 'seconds': 0.9999489784240723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00126600265503, 'end': 37.00130605697632, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18854070, 'rttvar': 2467036, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00126600265503, 'end': 37.00130605697632, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00130605697632, 'end': 38.00123310089111, 'seconds': 0.9999270439147949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19482133, 'rttvar': 2474958, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00130605697632, 'end': 38.00123310089111, 'seconds': 0.9999270439147949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00123310089111, 'end': 39.00126600265503, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19823876, 'rttvar': 2539705, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00123310089111, 'end': 39.00126600265503, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.00126600265503, 'end': 40.00121712684631, 'seconds': 0.9999511241912842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 20123559, 'rttvar': 2504145, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.00126600265503, 'end': 40.00121712684631, 'seconds': 0.9999511241912842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00121712684631, 'end': 41.00133299827576, 'seconds': 1.0001158714294434, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20649454, 'rttvar': 2320905, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00121712684631, 'end': 41.00133299827576, 'seconds': 1.0001158714294434, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00133299827576, 'end': 42.001216173172, 'seconds': 0.9998831748962402, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20953255, 'rttvar': 2348280, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00133299827576, 'end': 42.001216173172, 'seconds': 0.9998831748962402, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.001216173172, 'end': 43.00133299827576, 'seconds': 1.0001168251037598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 70952, 'rtt': 21538280, 'rttvar': 2358314, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.001216173172, 'end': 43.00133299827576, 'seconds': 1.0001168251037598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00133299827576, 'end': 44.00127410888672, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21783444, 'rttvar': 2259065, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00133299827576, 'end': 44.00127410888672, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00127410888672, 'end': 45.00116300582886, 'seconds': 0.9998888969421387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 22239630, 'rttvar': 2084817, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00127410888672, 'end': 45.00116300582886, 'seconds': 0.9998888969421387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00116300582886, 'end': 46.001182079315186, 'seconds': 1.0000190734863281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 22459443, 'rttvar': 2003239, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00116300582886, 'end': 46.001182079315186, 'seconds': 1.0000190734863281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.001182079315186, 'end': 47.0004141330719, 'seconds': 0.9992320537567139, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22644141, 'rttvar': 1871824, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.001182079315186, 'end': 47.0004141330719, 'seconds': 0.9992320537567139, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.0004141330719, 'end': 48.00126600265503, 'seconds': 1.0008518695831299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 23099025, 'rttvar': 1838000, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.0004141330719, 'end': 48.00126600265503, 'seconds': 1.0008518695831299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.00126600265503, 'end': 49.00107002258301, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 23258668, 'rttvar': 1697785, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.00126600265503, 'end': 49.00107002258301, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.00107002258301, 'end': 50.00130915641785, 'seconds': 1.0002391338348389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 78192, 'rtt': 23660133, 'rttvar': 1644268, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.00107002258301, 'end': 50.00130915641785, 'seconds': 1.0002391338348389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00130915641785, 'end': 51.001320123672485, 'seconds': 1.0000109672546387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23810270, 'rttvar': 1533477, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00130915641785, 'end': 51.001320123672485, 'seconds': 1.0000109672546387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.001320123672485, 'end': 52.00126504898071, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24213998, 'rttvar': 1558192, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.001320123672485, 'end': 52.00126504898071, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00126504898071, 'end': 53.00135803222656, 'seconds': 1.0000929832458496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24374374, 'rttvar': 1489395, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00126504898071, 'end': 53.00135803222656, 'seconds': 1.0000929832458496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00135803222656, 'end': 54.00108003616333, 'seconds': 0.9997220039367676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24799241, 'rttvar': 1573468, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00135803222656, 'end': 54.00108003616333, 'seconds': 0.9997220039367676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00108003616333, 'end': 55.0010781288147, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 24971236, 'rttvar': 1524092, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00108003616333, 'end': 55.0010781288147, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.0010781288147, 'end': 56.0014591217041, 'seconds': 1.0003809928894043, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 25207160, 'rttvar': 1614915, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.0010781288147, 'end': 56.0014591217041, 'seconds': 1.0003809928894043, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.0014591217041, 'end': 57.00111699104309, 'seconds': 0.9996578693389893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 85432, 'rtt': 25593805, 'rttvar': 1578728, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.0014591217041, 'end': 57.00111699104309, 'seconds': 0.9996578693389893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.00111699104309, 'end': 58.001320123672485, 'seconds': 1.0002031326293945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25836838, 'rttvar': 1670112, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.00111699104309, 'end': 58.001320123672485, 'seconds': 1.0002031326293945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001320123672485, 'end': 59.001305103302, 'seconds': 0.9999849796295166, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 88328, 'rtt': 26251788, 'rttvar': 1659292, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001320123672485, 'end': 59.001305103302, 'seconds': 0.9999849796295166, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}"
18:43:27.909347 tst iperf3.py:8: ERR: KeyError: 'sum_sent' [trial-2064↪gprs:sysmo↪iperf3.py:8]
18:43:27.917072 tst iperf3.py:8: Test FAILED (196.8 sec)
18:43:27.925109 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Terminating (SIGTERM)
18:43:27.968726 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Terminating (SIGTERM)
18:43:28.012435 run osmo-hlr_10.42.42.2(pid=26848): Terminating (SIGTERM)
18:43:28.055458 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Terminating (SIGTERM)
18:43:28.102051 run osmo-stp_10.42.42.5(pid=26850): Terminating (SIGTERM)
18:43:28.150418 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Terminating (SIGTERM)
18:43:28.214286 run osmo-ggsn_10.42.42.6(pid=26856): Terminating (SIGTERM)
18:43:28.262987 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Terminating (SIGTERM)
18:43:28.347167 run osmo-sgsn_10.42.42.7(pid=26880): Terminating (SIGTERM)
18:43:28.394825 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Terminating (SIGTERM)
18:43:28.448858 run osmo-msc_10.42.42.8(pid=26882): Terminating (SIGTERM)
18:43:28.496223 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Terminating (SIGTERM)
18:43:28.541767 run osmo-mgw_10.42.42.3(pid=26884): Terminating (SIGTERM)
18:43:28.588810 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Terminating (SIGTERM)
18:43:28.632853 run osmo-mgw_10.42.42.4(pid=26886): Terminating (SIGTERM)
18:43:28.676571 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Terminating (SIGTERM)
18:43:28.720875 run osmo-bsc_10.42.42.9(pid=26888): Terminating (SIGTERM)
18:43:28.763739 run osmo-bts-sysmo(pid=26897): Terminating (SIGTERM)
18:43:28.808091 run osmo-pcu(pid=26905): Terminating (SIGTERM)
18:43:28.852311 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGTERM)
18:43:28.996987 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGTERM(pid=26955): Launched
18:43:30.090195 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGTERM(pid=26955): Terminated: ok {rc=0}
18:43:30.156370 --- ParallelTerminationStrategy: PID 26843 died...
18:43:30.176371 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Terminated: ok {rc=0}
18:43:30.222871 --- ParallelTerminationStrategy: PID 26847 died...
18:43:30.242907 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Terminated: ok {rc=0}
18:43:30.289766 --- ParallelTerminationStrategy: PID 26848 died...
18:43:30.309613 run osmo-hlr_10.42.42.2(pid=26848): Terminated: ok {rc=0}
18:43:30.356237 --- ParallelTerminationStrategy: PID 26849 died...
18:43:30.375971 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Terminated: ok {rc=0}
18:43:30.422578 --- ParallelTerminationStrategy: PID 26850 died...
18:43:30.442912 run osmo-stp_10.42.42.5(pid=26850): Terminated: ok {rc=0}
18:43:30.489396 --- ParallelTerminationStrategy: PID 26851 died...
18:43:30.509185 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Terminated: ok {rc=0}
18:43:30.555822 --- ParallelTerminationStrategy: PID 26856 died...
18:43:30.575617 run osmo-ggsn_10.42.42.6(pid=26856): Terminated: ok {rc=0}
18:43:30.622206 --- ParallelTerminationStrategy: PID 26879 died...
18:43:30.642338 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Terminated: ok {rc=0}
18:43:30.688871 --- ParallelTerminationStrategy: PID 26880 died...
18:43:30.708755 run osmo-sgsn_10.42.42.7(pid=26880): Terminated: ok {rc=0}
18:43:30.755517 --- ParallelTerminationStrategy: PID 26881 died...
18:43:30.775344 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Terminated: ok {rc=0}
18:43:30.821866 --- ParallelTerminationStrategy: PID 26882 died...
18:43:30.841699 run osmo-msc_10.42.42.8(pid=26882): Terminated: ok {rc=0}
18:43:30.888599 --- ParallelTerminationStrategy: PID 26883 died...
18:43:30.908575 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Terminated: ok {rc=0}
18:43:30.955162 --- ParallelTerminationStrategy: PID 26884 died...
18:43:30.975000 run osmo-mgw_10.42.42.3(pid=26884): Terminated {rc=15}
18:43:31.021719 --- ParallelTerminationStrategy: PID 26885 died...
18:43:31.041588 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Terminated: ok {rc=0}
18:43:31.088246 --- ParallelTerminationStrategy: PID 26886 died...
18:43:31.108220 run osmo-mgw_10.42.42.4(pid=26886): Terminated {rc=15}
18:43:31.154903 --- ParallelTerminationStrategy: PID 26887 died...
18:43:31.174860 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Terminated: ok {rc=0}
18:43:31.221814 --- ParallelTerminationStrategy: PID 26888 died...
18:43:31.241582 run osmo-bsc_10.42.42.9(pid=26888): Terminated: ok {rc=0}
18:43:31.288191 --- ParallelTerminationStrategy: PID 26897 died...
18:43:31.308225 run osmo-bts-sysmo(pid=26897): Terminated: ok {rc=0}
18:43:31.355063 --- ParallelTerminationStrategy: PID 26905 died...
18:43:31.374911 run osmo-pcu(pid=26905): Terminated: ok {rc=0}
18:43:37.243809 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGINT)
18:43:37.382029 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGINT(pid=26960): Launched
18:43:38.472790 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGINT(pid=26960): Terminated: ok {rc=0}
18:43:44.358862 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGKILL)
18:43:44.502121 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGUSR1(pid=26963): Launched
18:43:45.589473 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGUSR1(pid=26963): Terminated: ok {rc=0}
18:43:45.841026 bus /gobi_6: Setting Online False
18:43:48.873285 bus /gobi_6: Setting Powered False
18:44:03.937866 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2064 gprs:sysmo iperf3m4.py
----------------------------------------------
18:44:05.136907 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.148186 tst gprs:sysmo: Using 1 x bts (candidates: 1)
18:44:05.183216 tst iperf3m4.py: using LAC 3947
18:44:05.311378 tst iperf3m4.py: using RAC 122
18:44:05.438236 tst iperf3m4.py: using CellId 3947
18:44:05.565537 tst iperf3m4.py: using BVCI 3948
18:44:05.685491 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.696587 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.707528 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.718354 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.729132 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.739899 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.750781 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.761644 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.772694 tst gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:05.889967 tst gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:06.007375 tst gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:06.126900 tst gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:06.243958 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
18:44:06.449284 run iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:06.759360 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Launched
18:44:07.028978 run iperf3-srv_10.42.42.10(pid=26967): Launched
18:44:07.145656 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
18:44:07.352490 run iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:07.670076 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Launched
18:44:07.957811 run iperf3-srv_10.42.42.10(pid=26971): Launched
18:44:08.080823 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
18:44:08.287392 run iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:08.595727 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Launched
18:44:08.864073 run iperf3-srv_10.42.42.10(pid=26973): Launched
18:44:08.977215 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
18:44:09.180898 run iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:09.489782 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Launched
18:44:09.759377 run iperf3-srv_10.42.42.10(pid=26975): Launched
18:44:09.872449 tst iperf3m4.py:8: start network...
18:44:10.075753 run osmo-hlr_10.42.42.2: Starting osmo-hlr
18:44:10.547693 run create_hlr_db(pid=26976): Launched
18:44:10.665746 bus /gobi_6: Setting Powered False
18:44:11.725965 run create_hlr_db(pid=26976): Terminated: ok {rc=0}
18:44:12.173453 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Launched
18:44:12.501801 run osmo-hlr_10.42.42.2(pid=26979): Launched
18:44:12.626842 run osmo-stp_10.42.42.5: Starting osmo-stp
18:44:13.188903 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Launched
18:44:13.515079 run osmo-stp_10.42.42.5(pid=26981): Launched
18:44:13.639823 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:44:14.200195 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Launched
18:44:14.323817 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:44:14.600687 run patchelf(pid=26983): Launched
18:44:15.785745 run patchelf(pid=26983): Terminated: ok {rc=0}
18:44:15.914425 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:44:16.175801 run osmo-gsm-tester_setcap_net_admin.sh(pid=26984): Launched
18:44:17.363964 run osmo-gsm-tester_setcap_net_admin.sh(pid=26984): Terminated: ok {rc=0}
18:44:17.707289 run osmo-ggsn_10.42.42.6(pid=26987): Launched
18:44:17.830611 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:44:18.532486 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Launched
18:44:18.859651 run osmo-sgsn_10.42.42.7(pid=27009): Launched
18:44:18.984720 run osmo-msc_10.42.42.8: Starting osmo-msc
18:44:19.579104 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Launched
18:44:19.908672 run osmo-msc_10.42.42.8(pid=27011): Launched
18:44:23.091202 run osmo-mgw_10.42.42.3: Starting osmo-mgw
18:44:23.664655 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Launched
18:44:23.993712 run osmo-mgw_10.42.42.3(pid=27013): Launched
18:44:24.119230 run osmo-mgw_10.42.42.4: Starting osmo-mgw
18:44:24.668716 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Launched
18:44:24.996797 run osmo-mgw_10.42.42.4(pid=27015): Launched
18:44:25.121290 run osmo-bsc_10.42.42.9: Starting osmo-bsc
18:44:25.796587 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Launched
18:44:26.125430 run osmo-bsc_10.42.42.9(pid=27017): Launched
18:44:32.415553 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:44:32.987097 run rm-remote-dir(pid=27018): Launched
18:44:35.185020 run rm-remote-dir(pid=27018): Terminated: ok {rc=0}
18:44:35.546097 run mk-remote-dir(pid=27019): Launched
18:44:36.737476 run mk-remote-dir(pid=27019): Terminated: ok {rc=0}
18:44:37.045347 run scp-inst-to-remote(pid=27020): Launched
18:44:49.256754 run scp-inst-to-remote(pid=27020): Terminated: ok {rc=0}
18:44:49.613206 run mk-remote-dir(pid=27022): Launched
18:44:50.802045 run mk-remote-dir(pid=27022): Terminated: ok {rc=0}
18:44:51.108424 run scp-cfg-to-remote(pid=27023): Launched
18:44:52.298931 run scp-cfg-to-remote(pid=27023): Terminated: ok {rc=0}
18:44:52.652745 run reload-dsp-firmware(pid=27025): Launched
18:44:54.843227 run reload-dsp-firmware(pid=27025): Terminated: ok {rc=0}
18:44:55.189647 run osmo-bts-sysmo(pid=27026): Launched
18:44:59.429304 tst iperf3m4.py:8: Waiting for bts to be ready...
18:45:01.154806 run rm-remote-dir(pid=27027): Launched
18:45:03.341684 run rm-remote-dir(pid=27027): Terminated: ok {rc=0}
18:45:03.691280 run mk-remote-dir(pid=27031): Launched
18:45:05.880125 run mk-remote-dir(pid=27031): Terminated: ok {rc=0}
18:45:06.177538 run scp-inst-to-bts(pid=27032): Launched
18:45:19.377424 run scp-inst-to-bts(pid=27032): Terminated: ok {rc=0}
18:45:19.724486 run mk-remote-run-dir(pid=27034): Launched
18:45:21.912452 run mk-remote-run-dir(pid=27034): Terminated: ok {rc=0}
18:45:22.209247 run scp-cfg-to-sysmobts(pid=27035): Launched
18:45:24.394149 run scp-cfg-to-sysmobts(pid=27035): Terminated: ok {rc=0}
18:45:24.740605 run osmo-pcu(pid=27037): Launched
18:45:24.874454 tst iperf3m4.py: using MSISDN 3952
18:45:25.045636 bus /gobi_6: Setting Powered True
18:45:28.564255 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3952', subscriber_id=1}
18:45:28.837192 bus /gobi_6: Setting Online False
18:45:30.025793 bus /gobi_6: Setting Powered False
18:45:37.109329 bus /gobi_6: Setting Powered True
18:45:38.498647 bus /gobi_6: Setting Online True
18:45:40.959901 tst /gobi_6: Connect to ('901', '70')
18:45:41.236070 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
18:45:41.354649 tst iperf3m4.py: using MSISDN 3953
18:45:41.629393 bus /gobi_0: Setting Powered True
18:45:45.141049 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3953', subscriber_id=2}
18:45:45.417127 bus /gobi_0: Setting Online False
18:45:46.608611 bus /gobi_0: Setting Powered False
18:45:54.109834 bus /gobi_0: Setting Powered True
18:45:55.375490 tst /gobi_6: Already registered with network ('901', '70')
18:45:55.596071 bus /gobi_0: Setting Online True
18:45:58.080877 tst /gobi_0: Connect to ('901', '70')
18:45:58.345558 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873'
18:45:58.470839 tst iperf3m4.py: using MSISDN 3954
18:45:59.030117 bus /gobi_1: Setting Powered True
18:46:02.522178 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3954', subscriber_id=3}
18:46:02.793149 bus /gobi_1: Setting Online False
18:46:03.987042 bus /gobi_1: Setting Powered False
18:46:11.135406 bus /gobi_1: Setting Powered True
18:46:12.516607 bus /gobi_1: Setting Online True
18:46:13.430167 tst /gobi_0: Already registered with network ('901', '70')
18:46:15.240244 tst /gobi_1: Connect to ('901', '70')
18:46:15.517543 tst /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024272103'
18:46:15.643109 tst iperf3m4.py: using MSISDN 3955
18:46:15.820467 bus /gobi_4: Setting Powered True
18:46:19.346144 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3955', subscriber_id=4}
18:46:19.608722 bus /gobi_4: Setting Online False
18:46:20.796686 bus /gobi_4: Setting Powered False
18:46:28.115681 bus /gobi_4: Setting Powered True
18:46:29.508779 bus /gobi_4: Setting Online True
18:46:30.931992 tst /gobi_1: Already registered with network ('901', '70')
18:46:32.176918 tst /gobi_4: Connect to ('901', '70')
18:46:32.464749 tst /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G 1 [Jul 21 2016 10:00:50]', Serial: '868323025161511'
18:46:32.573732 tst iperf3m4.py:8: waiting for modems to attach...
18:46:48.650917 tst /gobi_4: Already registered with network ('901', '70')
18:46:54.201032 tst iperf3m4.py:8: waiting for modems to attach to data services...
18:47:02.637656 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:02.756476 tst iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'"
18:47:03.210169 run move_netns(pid=27042): Launched
18:47:04.401302 run move_netns(pid=27042): Terminated: ok {rc=0}
18:47:04.757128 run ifup(pid=27045): Launched
18:47:05.946933 run ifup(pid=27045): Terminated: ok {rc=0}
18:47:06.300682 run dhcp(pid=27048): Launched
18:47:07.487692 run dhcp(pid=27048): Terminated: ok {rc=0}
18:47:15.821796 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:15.932128 tst iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'"
18:47:16.350000 run move_netns(pid=27066): Launched
18:47:17.536607 run move_netns(pid=27066): Terminated: ok {rc=0}
18:47:17.867113 run ifup(pid=27069): Launched
18:47:19.051424 run ifup(pid=27069): Terminated: ok {rc=0}
18:47:19.380586 run dhcp(pid=27072): Launched
18:47:20.570818 run dhcp(pid=27072): Terminated: ok {rc=0}
18:47:23.596883 tst /gobi_1: context activated {apn='inet46', path='/gobi_1/context9', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:23.706843 tst iperf3m4.py:8: Setting up data plan for "'/gobi_1/context9'"
18:47:24.123762 run move_netns(pid=27090): Launched
18:47:25.310363 run move_netns(pid=27090): Terminated: ok {rc=0}
18:47:25.641387 run ifup(pid=27093): Launched
18:47:26.832699 run ifup(pid=27093): Terminated: ok {rc=0}
18:47:27.187196 run dhcp(pid=27096): Launched
18:47:28.378114 run dhcp(pid=27096): Terminated: ok {rc=0}
18:47:36.783631 tst /gobi_4: context activated {apn='inet46', path='/gobi_4/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:36.894066 tst iperf3m4.py:8: Setting up data plan for "'/gobi_4/context2'"
18:47:37.311648 run move_netns(pid=27114): Launched
18:47:38.506943 run move_netns(pid=27114): Terminated: ok {rc=0}
18:47:38.862571 run ifup(pid=27117): Launched
18:47:40.054669 run ifup(pid=27117): Terminated: ok {rc=0}
18:47:40.408583 run dhcp(pid=27120): Launched
18:47:41.597956 run dhcp(pid=27120): Terminated: ok {rc=0}
18:47:41.737207 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'"
18:47:41.964359 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
18:47:42.355348 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Launched
18:47:42.538390 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'"
18:47:42.765153 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s)
18:47:43.157076 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Launched
18:47:43.338744 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context9'"
18:47:43.563135 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s)
18:47:43.952340 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Launched
18:47:44.132025 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context2'"
18:47:44.356819 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s)
18:47:44.745979 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Launched
18:47:45.105657 run iperf3-cli_10.42.42.10(pid=27148): Launched
18:47:45.382355 run iperf3-cli_10.42.42.10(pid=27151): Launched
18:47:45.659249 run iperf3-cli_10.42.42.10(pid=27154): Launched
18:47:45.929122 run iperf3-cli_10.42.42.10(pid=27157): Launched
18:48:54.251448 run iperf3-cli_10.42.42.10(pid=27148): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27148)]
18:48:54.518913 run iperf3-cli_10.42.42.10(pid=27148): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
18:48:54.656399 run iperf3-cli_10.42.42.10(pid=27148): stderr:
| (launched: 2022-03-29_18:47:45.089716)
18:48:55.843780 run iperf3-cli_10.42.42.10(pid=27151): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27151)]
18:48:55.980978 run iperf3-cli_10.42.42.10(pid=27151): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
18:48:56.124272 run iperf3-cli_10.42.42.10(pid=27151): stderr:
| (launched: 2022-03-29_18:47:45.365519)
18:48:57.329256 run iperf3-cli_10.42.42.10(pid=27154): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27154)]
18:48:57.471555 run iperf3-cli_10.42.42.10(pid=27154): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
18:48:57.614442 run iperf3-cli_10.42.42.10(pid=27154): stderr:
| (launched: 2022-03-29_18:47:45.642079)
18:48:58.810636 run iperf3-cli_10.42.42.10(pid=27157): ERR: Terminated: ERROR {rc=1} [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27157)]
18:48:58.939380 run iperf3-cli_10.42.42.10(pid=27157): stdout:
| "bits_per_second": 0,
| "retransmits": 0,
| "omitted": false
| }
| }],
| "end": {
| },
| "error": "error - control socket has closed unexpectedly"
| }
| child exited with 1
18:48:59.067024 run iperf3-cli_10.42.42.10(pid=27157): stderr:
| (launched: 2022-03-29_18:47:45.912502)
18:48:59.194618 run iperf3-srv_10.42.42.10(pid=26967): Terminating (SIGINT)
18:48:59.525727 run iperf3-srv_10.42.42.10(pid=26967): Terminated {rc=1}
18:48:59.640844 tst iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'"
Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42908, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:47:46 GMT', 'timesecs': 1648572466}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'bbwlr3tnbptbxm2dq6maiza55z7ufvcy5did', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0006639957427979, 'seconds': 1.0006639957427979, 'bytes': 101360, 'bits_per_second': 810341.9364040171, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 379730, 'rttvar': 175208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0006639957427979, 'seconds': 1.0006639957427979, 'bytes': 101360, 'bits_per_second': 810341.9364040171, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0006639957427979, 'end': 2.0005221366882324, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 521979, 'rttvar': 415904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0006639957427979, 'end': 2.0005221366882324, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0005221366882324, 'end': 3.001317024230957, 'seconds': 1.0007948875427246, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 4344, 'rtt': 521979, 'rttvar': 415904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0005221366882324, 'end': 3.001317024230957, 'seconds': 1.0007948875427246, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.001317024230957, 'end': 4.001322984695435, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 809107, 'rttvar': 886184, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.001317024230957, 'end': 4.001322984695435, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001322984695435, 'end': 5.001317977905273, 'seconds': 0.9999949932098389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1250143, 'rttvar': 1546710, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001322984695435, 'end': 5.001317977905273, 'seconds': 0.9999949932098389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001317977905273, 'end': 6.000808000564575, 'seconds': 0.9994900226593018, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1250143, 'rttvar': 1546710, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001317977905273, 'end': 6.000808000564575, 'seconds': 0.9994900226593018, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.000808000564575, 'end': 7.001322984695435, 'seconds': 1.0005149841308594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1828734, 'rttvar': 2317215, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.000808000564575, 'end': 7.001322984695435, 'seconds': 1.0005149841308594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001322984695435, 'end': 8.000629186630249, 'seconds': 0.9993062019348145, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2527605, 'rttvar': 3135652, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001322984695435, 'end': 8.000629186630249, 'seconds': 0.9993062019348145, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.000629186630249, 'end': 9.00132417678833, 'seconds': 1.000694990158081, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2527605, 'rttvar': 3135652, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.000629186630249, 'end': 9.00132417678833, 'seconds': 1.000694990158081, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.00132417678833, 'end': 10.001171112060547, 'seconds': 0.9998469352722168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 3326609, 'rttvar': 3949748, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.00132417678833, 'end': 10.001171112060547, 'seconds': 0.9998469352722168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001171112060547, 'end': 11.000813007354736, 'seconds': 0.9996418952941895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4210285, 'rttvar': 4729664, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001171112060547, 'end': 11.000813007354736, 'seconds': 0.9996418952941895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.000813007354736, 'end': 12.001137971878052, 'seconds': 1.0003249645233154, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4210285, 'rttvar': 4729664, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.000813007354736, 'end': 12.001137971878052, 'seconds': 1.0003249645233154, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001137971878052, 'end': 13.001238107681274, 'seconds': 1.0001001358032227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 5170983, 'rttvar': 5468642, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001137971878052, 'end': 13.001238107681274, 'seconds': 1.0001001358032227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001238107681274, 'end': 14.000625133514404, 'seconds': 0.9993870258331299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6209445, 'rttvar': 6178407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001238107681274, 'end': 14.000625133514404, 'seconds': 0.9993870258331299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.000625133514404, 'end': 15.00132417678833, 'seconds': 1.0006990432739258, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6209445, 'rttvar': 6178407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.000625133514404, 'end': 15.00132417678833, 'seconds': 1.0006990432739258, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.00132417678833, 'end': 16.001312971115112, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7307874, 'rttvar': 6830663, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.00132417678833, 'end': 16.001312971115112, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001312971115112, 'end': 17.001317024230957, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7307874, 'rttvar': 6830663, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001312971115112, 'end': 17.001317024230957, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001317024230957, 'end': 18.000580072402954, 'seconds': 0.9992630481719971, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8366236, 'rttvar': 7239721, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001317024230957, 'end': 18.000580072402954, 'seconds': 0.9992630481719971, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.000580072402954, 'end': 19.000797033309937, 'seconds': 1.0002169609069824, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.000580072402954, 'end': 19.000797033309937, 'seconds': 1.0002169609069824, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.000797033309937, 'end': 20.001474142074585, 'seconds': 1.0006771087646484, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.000797033309937, 'end': 20.001474142074585, 'seconds': 1.0006771087646484, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001474142074585, 'end': 21.000576972961426, 'seconds': 0.9991028308868408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001474142074585, 'end': 21.000576972961426, 'seconds': 0.9991028308868408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.000576972961426, 'end': 22.001118183135986, 'seconds': 1.0005412101745605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10496162, 'rttvar': 7820208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.000576972961426, 'end': 22.001118183135986, 'seconds': 1.0005412101745605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001118183135986, 'end': 23.001479148864746, 'seconds': 1.0003609657287598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10496162, 'rttvar': 7820208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001118183135986, 'end': 23.001479148864746, 'seconds': 1.0003609657287598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001479148864746, 'end': 24.001168966293335, 'seconds': 0.9996898174285889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 11651412, 'rttvar': 8175655, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001479148864746, 'end': 24.001168966293335, 'seconds': 0.9996898174285889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001168966293335, 'end': 25.001140117645264, 'seconds': 0.9999711513519287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12672600, 'rttvar': 8174116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001168966293335, 'end': 25.001140117645264, 'seconds': 0.9999711513519287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001140117645264, 'end': 26.000615119934082, 'seconds': 0.9994750022888184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12672600, 'rttvar': 8174116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001140117645264, 'end': 26.000615119934082, 'seconds': 0.9994750022888184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.000615119934082, 'end': 27.00078797340393, 'seconds': 1.0001728534698486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 13753602, 'rttvar': 8292592, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.000615119934082, 'end': 27.00078797340393, 'seconds': 1.0001728534698486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00078797340393, 'end': 28.00065517425537, 'seconds': 0.9998672008514404, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14696598, 'rttvar': 8105436, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00078797340393, 'end': 28.00065517425537, 'seconds': 0.9998672008514404, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.00065517425537, 'end': 29.0013210773468, 'seconds': 1.0006659030914307, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14696598, 'rttvar': 8105436, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.00065517425537, 'end': 29.0013210773468, 'seconds': 1.0006659030914307, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.0013210773468, 'end': 30.000684022903442, 'seconds': 0.9993629455566406, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 15721861, 'rttvar': 8129602, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.0013210773468, 'end': 30.000684022903442, 'seconds': 0.9993629455566406, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.000684022903442, 'end': 31.00068497657776, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16616168, 'rttvar': 7885816, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.000684022903442, 'end': 31.00068497657776, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00068497657776, 'end': 32.00132203102112, 'seconds': 1.0006370544433594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16616168, 'rttvar': 7885816, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00068497657776, 'end': 32.00132203102112, 'seconds': 1.0006370544433594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00132203102112, 'end': 33.00074911117554, 'seconds': 0.999427080154419, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 17588449, 'rttvar': 7858924, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00132203102112, 'end': 33.00074911117554, 'seconds': 0.999427080154419, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00074911117554, 'end': 34.00058197975159, 'seconds': 0.9998328685760498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18444463, 'rttvar': 7606221, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00074911117554, 'end': 34.00058197975159, 'seconds': 0.9998328685760498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00058197975159, 'end': 35.0013210773468, 'seconds': 1.0007390975952148, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18444463, 'rttvar': 7606221, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00058197975159, 'end': 35.0013210773468, 'seconds': 1.0007390975952148, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.0013210773468, 'end': 36.00113916397095, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 19375780, 'rttvar': 7567300, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.0013210773468, 'end': 36.00113916397095, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00113916397095, 'end': 37.00057411193848, 'seconds': 0.9994349479675293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20193508, 'rttvar': 7310931, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00113916397095, 'end': 37.00057411193848, 'seconds': 0.9994349479675293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00057411193848, 'end': 38.00039196014404, 'seconds': 0.9998178482055664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20193508, 'rttvar': 7310931, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00057411193848, 'end': 38.00039196014404, 'seconds': 0.9998178482055664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00039196014404, 'end': 39.0006890296936, 'seconds': 1.0002970695495605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 21094187, 'rttvar': 7284557, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00039196014404, 'end': 39.0006890296936, 'seconds': 1.0002970695495605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0006890296936, 'end': 40.00058197975159, 'seconds': 0.9998929500579834, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21887500, 'rttvar': 7050044, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0006890296936, 'end': 40.00058197975159, 'seconds': 0.9998929500579834, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00058197975159, 'end': 41.00132203102112, 'seconds': 1.0007400512695312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21887500, 'rttvar': 7050044, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00058197975159, 'end': 41.00132203102112, 'seconds': 1.0007400512695312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00132203102112, 'end': 42.000596046447754, 'seconds': 0.9992740154266357, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 22769210, 'rttvar': 7050952, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00132203102112, 'end': 42.000596046447754, 'seconds': 0.9992740154266357, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.000596046447754, 'end': 43.00114297866821, 'seconds': 1.000546932220459, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23532537, 'rttvar': 6814868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.000596046447754, 'end': 43.00114297866821, 'seconds': 1.000546932220459, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00114297866821, 'end': 44.00148797035217, 'seconds': 1.00034499168396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23532537, 'rttvar': 6814868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00114297866821, 'end': 44.00148797035217, 'seconds': 1.00034499168396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00148797035217, 'end': 45.000646114349365, 'seconds': 0.9991581439971924, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 24400576, 'rttvar': 6847229, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00148797035217, 'end': 45.000646114349365, 'seconds': 0.9991581439971924, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.000646114349365, 'end': 46.00114417076111, 'seconds': 1.0004980564117432, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25155044, 'rttvar': 6644358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.000646114349365, 'end': 46.00114417076111, 'seconds': 1.0004980564117432, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00114417076111, 'end': 47.0008499622345, 'seconds': 0.9997057914733887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25155044, 'rttvar': 6644358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00114417076111, 'end': 47.0008499622345, 'seconds': 0.9997057914733887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.0008499622345, 'end': 48.00132203102112, 'seconds': 1.000472068786621, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26022865, 'rttvar': 6718910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.0008499622345, 'end': 48.00132203102112, 'seconds': 1.000472068786621, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.00132203102112, 'end': 49.0011351108551, 'seconds': 0.9998130798339844, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26022865, 'rttvar': 6718910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.00132203102112, 'end': 49.0011351108551, 'seconds': 0.9998130798339844, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.0011351108551, 'end': 50.00085496902466, 'seconds': 0.9997198581695557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 26794956, 'rttvar': 6583365, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.0011351108551, 'end': 50.00085496902466, 'seconds': 0.9997198581695557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00085496902466, 'end': 51.000646114349365, 'seconds': 0.999791145324707, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27670031, 'rttvar': 6687674, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00085496902466, 'end': 51.000646114349365, 'seconds': 0.999791145324707, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.000646114349365, 'end': 52.00117516517639, 'seconds': 1.0005290508270264, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27670031, 'rttvar': 6687674, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.000646114349365, 'end': 52.00117516517639, 'seconds': 1.0005290508270264, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00117516517639, 'end': 53.00131797790527, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 28446091, 'rttvar': 6567875, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00117516517639, 'end': 53.00131797790527, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00131797790527, 'end': 54.00036907196045, 'seconds': 0.9990510940551758, 'bytes': 256296, 'bits_per_second': 2052315.4543352732, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29302817, 'rttvar': 6639359, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00131797790527, 'end': 54.00036907196045, 'seconds': 0.9990510940551758, 'bytes': 256296, 'bits_per_second': 2052315.4543352732, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00036907196045, 'end': 55.00075912475586, 'seconds': 1.0003900527954102, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29302817, 'rttvar': 6639359, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00036907196045, 'end': 55.00075912475586, 'seconds': 1.0003900527954102, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.00075912475586, 'end': 56.00075316429138, 'seconds': 0.9999940395355225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 29869581, 'rttvar': 6113047, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.00075912475586, 'end': 56.00075316429138, 'seconds': 0.9999940395355225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.00075316429138, 'end': 57.000683069229126, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30555344, 'rttvar': 5956311, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.00075316429138, 'end': 57.000683069229126, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.000683069229126, 'end': 58.001219034194946, 'seconds': 1.0005359649658203, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30555344, 'rttvar': 5956311, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.000683069229126, 'end': 58.001219034194946, 'seconds': 1.0005359649658203, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001219034194946, 'end': 59.00129508972168, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 31175458, 'rttvar': 5707460, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001219034194946, 'end': 59.00129508972168, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}"
18:48:59.859128 tst iperf3m4.py:8: ERR: KeyError: 'sum_sent' [trial-2064↪gprs:sysmo↪iperf3m4.py:8]
18:48:59.866575 tst iperf3m4.py:8: Test FAILED (294.8 sec)
18:48:59.873349 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Terminating (SIGTERM)
18:48:59.916435 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Terminating (SIGTERM)
18:48:59.958484 run iperf3-srv_10.42.42.10(pid=26971): Terminating (SIGTERM)
18:48:59.999372 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Terminating (SIGTERM)
18:49:00.040518 run iperf3-srv_10.42.42.10(pid=26973): Terminating (SIGTERM)
18:49:00.081176 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Terminating (SIGTERM)
18:49:00.122120 run iperf3-srv_10.42.42.10(pid=26975): Terminating (SIGTERM)
18:49:00.162967 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Terminating (SIGTERM)
18:49:00.203950 run osmo-hlr_10.42.42.2(pid=26979): Terminating (SIGTERM)
18:49:00.244856 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Terminating (SIGTERM)
18:49:00.287028 run osmo-stp_10.42.42.5(pid=26981): Terminating (SIGTERM)
18:49:00.328772 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Terminating (SIGTERM)
18:49:00.382622 run osmo-ggsn_10.42.42.6(pid=26987): Terminating (SIGTERM)
18:49:00.424467 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Terminating (SIGTERM)
18:49:00.473819 run osmo-sgsn_10.42.42.7(pid=27009): Terminating (SIGTERM)
18:49:00.529295 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Terminating (SIGTERM)
18:49:00.580227 run osmo-msc_10.42.42.8(pid=27011): Terminating (SIGTERM)
18:49:00.633554 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Terminating (SIGTERM)
18:49:00.686829 run osmo-mgw_10.42.42.3(pid=27013): Terminating (SIGTERM)
18:49:00.750948 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Terminating (SIGTERM)
18:49:00.799371 run osmo-mgw_10.42.42.4(pid=27015): Terminating (SIGTERM)
18:49:00.844649 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Terminating (SIGTERM)
18:49:00.890072 run osmo-bsc_10.42.42.9(pid=27017): Terminating (SIGTERM)
18:49:00.935822 run osmo-bts-sysmo(pid=27026): Terminating (SIGTERM)
18:49:00.983304 run osmo-pcu(pid=27037): Terminating (SIGTERM)
18:49:01.029399 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGTERM)
18:49:01.176449 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGTERM(pid=27181): Launched
18:49:02.277299 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGTERM(pid=27181): Terminated: ok {rc=0}
18:49:02.344389 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGTERM)
18:49:02.488995 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGTERM(pid=27186): Launched
18:49:03.577455 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGTERM(pid=27186): Terminated: ok {rc=0}
18:49:03.643834 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGTERM)
18:49:03.787356 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGTERM(pid=27189): Launched
18:49:04.880339 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGTERM(pid=27189): Terminated: ok {rc=0}
18:49:04.946489 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGTERM)
18:49:05.090521 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGTERM(pid=27192): Launched
18:49:06.177574 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGTERM(pid=27192): Terminated: ok {rc=0}
18:49:06.244170 --- ParallelTerminationStrategy: PID 26966 died...
18:49:06.264257 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Terminated: ok {rc=0}
18:49:06.311419 --- ParallelTerminationStrategy: PID 26969 died...
18:49:06.331213 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Terminated: ok {rc=0}
18:49:06.377416 --- ParallelTerminationStrategy: PID 26971 died...
18:49:06.397009 run iperf3-srv_10.42.42.10(pid=26971): Terminated {rc=256}
18:49:06.443413 --- ParallelTerminationStrategy: PID 26972 died...
18:49:06.463037 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Terminated: ok {rc=0}
18:49:06.509326 --- ParallelTerminationStrategy: PID 26973 died...
18:49:06.528920 run iperf3-srv_10.42.42.10(pid=26973): Terminated {rc=256}
18:49:06.577483 --- ParallelTerminationStrategy: PID 26974 died...
18:49:06.596438 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Terminated: ok {rc=0}
18:49:06.642684 --- ParallelTerminationStrategy: PID 26975 died...
18:49:06.661640 run iperf3-srv_10.42.42.10(pid=26975): Terminated {rc=256}
18:49:06.707945 --- ParallelTerminationStrategy: PID 26978 died...
18:49:06.727174 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Terminated: ok {rc=0}
18:49:06.773390 --- ParallelTerminationStrategy: PID 26979 died...
18:49:06.792396 run osmo-hlr_10.42.42.2(pid=26979): Terminated: ok {rc=0}
18:49:06.838635 --- ParallelTerminationStrategy: PID 26980 died...
18:49:06.857660 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Terminated: ok {rc=0}
18:49:06.903935 --- ParallelTerminationStrategy: PID 26981 died...
18:49:06.922961 run osmo-stp_10.42.42.5(pid=26981): Terminated: ok {rc=0}
18:49:06.968966 --- ParallelTerminationStrategy: PID 26982 died...
18:49:06.988234 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Terminated: ok {rc=0}
18:49:07.034375 --- ParallelTerminationStrategy: PID 26987 died...
18:49:07.053316 run osmo-ggsn_10.42.42.6(pid=26987): Terminated: ok {rc=0}
18:49:07.099491 --- ParallelTerminationStrategy: PID 27008 died...
18:49:07.118433 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Terminated: ok {rc=0}
18:49:07.164729 --- ParallelTerminationStrategy: PID 27009 died...
18:49:07.183676 run osmo-sgsn_10.42.42.7(pid=27009): Terminated: ok {rc=0}
18:49:07.229597 --- ParallelTerminationStrategy: PID 27010 died...
18:49:07.248614 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Terminated: ok {rc=0}
18:49:07.294555 --- ParallelTerminationStrategy: PID 27011 died...
18:49:07.313817 run osmo-msc_10.42.42.8(pid=27011): Terminated: ok {rc=0}
18:49:07.359845 --- ParallelTerminationStrategy: PID 27012 died...
18:49:07.378951 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Terminated: ok {rc=0}
18:49:07.425626 --- ParallelTerminationStrategy: PID 27013 died...
18:49:07.444927 run osmo-mgw_10.42.42.3(pid=27013): Terminated {rc=15}
18:49:07.491404 --- ParallelTerminationStrategy: PID 27014 died...
18:49:07.510826 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Terminated: ok {rc=0}
18:49:07.557304 --- ParallelTerminationStrategy: PID 27015 died...
18:49:07.576806 run osmo-mgw_10.42.42.4(pid=27015): Terminated {rc=15}
18:49:07.627753 --- ParallelTerminationStrategy: PID 27016 died...
18:49:07.647875 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Terminated: ok {rc=0}
18:49:07.694884 --- ParallelTerminationStrategy: PID 27017 died...
18:49:07.714878 run osmo-bsc_10.42.42.9(pid=27017): Terminated: ok {rc=0}
18:49:07.762099 --- ParallelTerminationStrategy: PID 27026 died...
18:49:07.782164 run osmo-bts-sysmo(pid=27026): Terminated: ok {rc=0}
18:49:07.829070 --- ParallelTerminationStrategy: PID 27037 died...
18:49:07.849277 run osmo-pcu(pid=27037): Terminated: ok {rc=0}
18:49:13.719053 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGINT)
18:49:13.862564 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGINT(pid=27199): Launched
18:49:14.951823 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGINT(pid=27199): Terminated: ok {rc=0}
18:49:15.017876 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGINT)
18:49:15.161781 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGINT(pid=27202): Launched
18:49:16.246800 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGINT(pid=27202): Terminated: ok {rc=0}
18:49:16.313204 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGINT)
18:49:16.455282 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGINT(pid=27205): Launched
18:49:17.539859 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGINT(pid=27205): Terminated: ok {rc=0}
18:49:17.606118 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGINT)
18:49:17.749720 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGINT(pid=27208): Launched
18:49:18.842561 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGINT(pid=27208): Terminated: ok {rc=0}
18:49:24.731262 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGKILL)
18:49:24.875297 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGUSR1(pid=27211): Launched
18:49:25.965709 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGUSR1(pid=27211): Terminated: ok {rc=0}
18:49:26.031968 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGKILL)
18:49:26.174568 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGUSR1(pid=27214): Launched
18:49:27.265545 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGUSR1(pid=27214): Terminated: ok {rc=0}
18:49:27.331672 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGKILL)
18:49:27.474564 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGUSR1(pid=27217): Launched
18:49:28.565339 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGUSR1(pid=27217): Terminated: ok {rc=0}
18:49:28.631806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGKILL)
18:49:28.774252 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGUSR1(pid=27220): Launched
18:49:29.859536 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGUSR1(pid=27220): Terminated: ok {rc=0}
18:49:30.105783 bus /gobi_4: Setting Online False
18:49:31.244861 bus /gobi_4: Setting Powered False
18:49:45.944600 bus /gobi_4: Setting Powered False
18:49:47.044501 bus /gobi_4: Setting Powered False
18:49:48.202038 bus /gobi_1: Setting Online False
18:49:49.324698 bus /gobi_1: Setting Powered False
18:49:57.098064 bus /gobi_1: Setting Powered False
18:49:58.197022 bus /gobi_1: Setting Powered False
18:49:59.354516 bus /gobi_0: Setting Online False
18:50:00.491432 bus /gobi_0: Setting Powered False
18:50:08.937999 bus /gobi_0: Setting Powered False
18:50:10.037687 bus /gobi_0: Setting Powered False
18:50:11.196010 bus /gobi_6: Setting Online False
18:50:12.331026 bus /gobi_6: Setting Powered False
18:50:20.944844 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2064 gprs:sysmo ping.py
----------------------------------------------
18:50:22.103660 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.116667 tst gprs:sysmo: Using 1 x bts (candidates: 1)
18:50:22.150394 tst ping.py: using LAC 3948
18:50:22.271535 tst ping.py: using RAC 123
18:50:22.389731 tst ping.py: using CellId 3948
18:50:22.508184 tst ping.py: using BVCI 3949
18:50:22.619861 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.630352 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.640895 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.651947 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.662828 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.673696 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.684731 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.695704 tst gprs:sysmo: Using 1 x modem (candidates: 4)
18:50:22.813045 tst ping.py:19: start network...
18:50:23.015211 run osmo-hlr_10.42.42.2: Starting osmo-hlr
18:50:23.481826 run create_hlr_db(pid=27227): Launched
18:50:23.603372 bus /gobi_6: Setting Powered False
18:50:24.664060 run create_hlr_db(pid=27227): Terminated: ok {rc=0}
18:50:25.105335 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Launched
18:50:25.428549 run osmo-hlr_10.42.42.2(pid=27230): Launched
18:50:25.551650 run osmo-stp_10.42.42.5: Starting osmo-stp
18:50:26.101626 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Launched
18:50:26.426917 run osmo-stp_10.42.42.5(pid=27232): Launched
18:50:26.550984 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:50:27.104949 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Launched
18:50:27.228518 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:50:27.502204 run patchelf(pid=27234): Launched
18:50:28.685851 run patchelf(pid=27234): Terminated: ok {rc=0}
18:50:28.821713 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:50:29.098946 run osmo-gsm-tester_setcap_net_admin.sh(pid=27235): Launched
18:50:30.287309 run osmo-gsm-tester_setcap_net_admin.sh(pid=27235): Terminated: ok {rc=0}
18:50:30.624080 run osmo-ggsn_10.42.42.6(pid=27238): Launched
18:50:30.747704 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:50:31.422082 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Launched
18:50:31.725937 run osmo-sgsn_10.42.42.7(pid=27261): Launched
18:50:31.841137 run osmo-msc_10.42.42.8: Starting osmo-msc
18:50:32.403724 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Launched
18:50:32.705078 run osmo-msc_10.42.42.8(pid=27263): Launched
18:50:35.875207 run osmo-mgw_10.42.42.3: Starting osmo-mgw
18:50:36.389750 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Launched
18:50:36.692742 run osmo-mgw_10.42.42.3(pid=27265): Launched
18:50:36.808625 run osmo-mgw_10.42.42.4: Starting osmo-mgw
18:50:37.312282 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Launched
18:50:37.615382 run osmo-mgw_10.42.42.4(pid=27267): Launched
18:50:37.731142 run osmo-bsc_10.42.42.9: Starting osmo-bsc
18:50:38.355954 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Launched
18:50:38.658582 run osmo-bsc_10.42.42.9(pid=27269): Launched
18:50:44.924840 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:50:45.450442 run rm-remote-dir(pid=27270): Launched
18:50:47.630057 run rm-remote-dir(pid=27270): Terminated: ok {rc=0}
18:50:47.962609 run mk-remote-dir(pid=27271): Launched
18:50:49.139843 run mk-remote-dir(pid=27271): Terminated: ok {rc=0}
18:50:49.423231 run scp-inst-to-remote(pid=27272): Launched
18:51:01.614934 run scp-inst-to-remote(pid=27272): Terminated: ok {rc=0}
18:51:01.965540 run mk-remote-dir(pid=27274): Launched
18:51:03.153689 run mk-remote-dir(pid=27274): Terminated: ok {rc=0}
18:51:03.454827 run scp-cfg-to-remote(pid=27275): Launched
18:51:04.643143 run scp-cfg-to-remote(pid=27275): Terminated: ok {rc=0}
18:51:04.990265 run reload-dsp-firmware(pid=27277): Launched
18:51:07.177513 run reload-dsp-firmware(pid=27277): Terminated: ok {rc=0}
18:51:07.517467 run osmo-bts-sysmo(pid=27278): Launched
18:51:11.754838 tst ping.py:31: Waiting for bts to be ready...
18:51:13.470836 run rm-remote-dir(pid=27279): Launched
18:51:15.651499 run rm-remote-dir(pid=27279): Terminated: ok {rc=0}
18:51:15.970610 run mk-remote-dir(pid=27280): Launched
18:51:18.141366 run mk-remote-dir(pid=27280): Terminated: ok {rc=0}
18:51:18.414515 run scp-inst-to-bts(pid=27281): Launched
18:51:31.606348 run scp-inst-to-bts(pid=27281): Terminated: ok {rc=0}
18:51:31.949063 run mk-remote-run-dir(pid=27283): Launched
18:51:34.134158 run mk-remote-run-dir(pid=27283): Terminated: ok {rc=0}
18:51:34.426639 run scp-cfg-to-sysmobts(pid=27284): Launched
18:51:36.603879 run scp-cfg-to-sysmobts(pid=27284): Terminated: ok {rc=0}
18:51:36.922588 run osmo-pcu(pid=27286): Launched
18:51:37.045170 tst ping.py: using MSISDN 3956
18:51:37.217912 bus /gobi_6: Setting Powered True
18:51:40.726402 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3956', subscriber_id=1}
18:51:40.987713 bus /gobi_6: Setting Online False
18:51:42.173276 bus /gobi_6: Setting Powered False
18:51:49.105245 bus /gobi_6: Setting Powered True
18:51:50.463321 bus /gobi_6: Setting Online True
18:51:52.875153 tst /gobi_6: Connect to ('901', '70')
18:51:53.151325 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
18:51:53.268187 tst ping.py:42: waiting for modems to attach...
18:52:07.461869 tst /gobi_6: Already registered with network ('901', '70')
18:52:12.803006 tst ping.py:46: waiting for modems to attach to data services...
18:52:19.982260 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:52:20.089653 tst ping.py:51: Setting up data plan for "'/gobi_6/context3'"
18:52:20.497538 run move_netns(pid=27290): Launched
18:52:21.673417 run move_netns(pid=27290): Terminated: ok {rc=0}
18:52:21.995974 run ifup(pid=27293): Launched
18:52:23.170671 run ifup(pid=27293): Terminated: ok {rc=0}
18:52:23.516847 run dhcp(pid=27296): Launched
18:52:24.701866 run dhcp(pid=27296): Terminated: ok {rc=0}
18:52:24.829168 tst ping.py:53: Running 10 ping requests for "'/gobi_6/context3'"
18:52:25.221334 run ping(pid=27312): Launched
18:52:35.416106 run ping(pid=27312): Terminated: ok {rc=0}
18:52:37.180118 tst /gobi_6: context deactivated {path='/gobi_6/context3'}
18:52:44.456774 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={Interface='r433306bbc2604b'}, Name='Internet', Password='', Protocol='ipv6', Settings={}, Type='internet', Username='ogt'}, user='ogt'}
18:52:52.219378 tst /gobi_6: context deactivated {path='/gobi_6/context3'}
18:52:52.337523 tst ping.py:70: (launched: 2022-03-29_18:52:25.160958)
27313: waiting for 27314
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=943 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=584 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=585 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=581 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=563 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=581 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=577 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=560 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=578 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=574 ms
--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 18ms
rtt min/avg/max/mdev = 559.738/612.790/943.318/110.469 ms
child exited with 0
18:52:52.558240 tst ping.py: Test passed (150.5 sec)
18:52:52.626225 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Terminating (SIGTERM)
18:52:52.669513 run osmo-hlr_10.42.42.2(pid=27230): Terminating (SIGTERM)
18:52:52.713004 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Terminating (SIGTERM)
18:52:52.758484 run osmo-stp_10.42.42.5(pid=27232): Terminating (SIGTERM)
18:52:52.805607 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Terminating (SIGTERM)
18:52:52.871714 run osmo-ggsn_10.42.42.6(pid=27238): Terminating (SIGTERM)
18:52:52.918210 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Terminating (SIGTERM)
18:52:52.992420 run osmo-sgsn_10.42.42.7(pid=27261): Terminating (SIGTERM)
18:52:53.037946 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Terminating (SIGTERM)
18:52:53.083257 run osmo-msc_10.42.42.8(pid=27263): Terminating (SIGTERM)
18:52:53.127270 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Terminating (SIGTERM)
18:52:53.171173 run osmo-mgw_10.42.42.3(pid=27265): Terminating (SIGTERM)
18:52:53.215766 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Terminating (SIGTERM)
18:52:53.259448 run osmo-mgw_10.42.42.4(pid=27267): Terminating (SIGTERM)
18:52:53.301765 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Terminating (SIGTERM)
18:52:53.344995 run osmo-bsc_10.42.42.9(pid=27269): Terminating (SIGTERM)
18:52:53.387124 run osmo-bts-sysmo(pid=27278): Terminating (SIGTERM)
18:52:53.429318 run osmo-pcu(pid=27286): Terminating (SIGTERM)
18:52:53.472420 --- ParallelTerminationStrategy: PID 27229 died...
18:52:53.492815 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Terminated: ok {rc=0}
18:52:53.539802 --- ParallelTerminationStrategy: PID 27230 died...
18:52:53.559815 run osmo-hlr_10.42.42.2(pid=27230): Terminated: ok {rc=0}
18:52:53.606344 --- ParallelTerminationStrategy: PID 27231 died...
18:52:53.626146 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Terminated: ok {rc=0}
18:52:53.672982 --- ParallelTerminationStrategy: PID 27232 died...
18:52:53.692773 run osmo-stp_10.42.42.5(pid=27232): Terminated: ok {rc=0}
18:52:53.739317 --- ParallelTerminationStrategy: PID 27233 died...
18:52:53.759222 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Terminated: ok {rc=0}
18:52:53.806385 --- ParallelTerminationStrategy: PID 27238 died...
18:52:53.827039 run osmo-ggsn_10.42.42.6(pid=27238): Terminated: ok {rc=0}
18:52:53.875159 --- ParallelTerminationStrategy: PID 27260 died...
18:52:53.895859 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Terminated: ok {rc=0}
18:52:53.945206 --- ParallelTerminationStrategy: PID 27262 died...
18:52:53.966123 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Terminated: ok {rc=0}
18:52:54.027165 --- ParallelTerminationStrategy: PID 27264 died...
18:52:54.080419 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Terminated: ok {rc=0}
18:52:54.171259 --- ParallelTerminationStrategy: PID 27265 died...
18:52:54.228114 run osmo-mgw_10.42.42.3(pid=27265): Terminated {rc=15}
18:52:54.276430 --- ParallelTerminationStrategy: PID 27263 died...
18:52:54.297171 run osmo-msc_10.42.42.8(pid=27263): Terminated: ok {rc=0}
18:52:54.345432 --- ParallelTerminationStrategy: PID 27266 died...
18:52:54.366278 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Terminated: ok {rc=0}
18:52:54.414423 --- ParallelTerminationStrategy: PID 27267 died...
18:52:54.435201 run osmo-mgw_10.42.42.4(pid=27267): Terminated {rc=15}
18:52:54.483808 --- ParallelTerminationStrategy: PID 27268 died...
18:52:54.504445 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Terminated: ok {rc=0}
18:52:54.552566 --- ParallelTerminationStrategy: PID 27269 died...
18:52:54.573226 run osmo-bsc_10.42.42.9(pid=27269): Terminated: ok {rc=0}
18:52:54.621660 --- ParallelTerminationStrategy: PID 27278 died...
18:52:54.642676 run osmo-bts-sysmo(pid=27278): Terminated: ok {rc=0}
18:52:54.691239 --- ParallelTerminationStrategy: PID 27286 died...
18:52:54.712056 run osmo-pcu(pid=27286): Terminated: ok {rc=0}
18:52:54.928817 --- ParallelTerminationStrategy: PID 27261 died...
18:52:54.950844 run osmo-sgsn_10.42.42.7(pid=27261): Terminated: ok {rc=0}
18:52:55.176451 bus /gobi_6: Setting Online False
18:52:58.227734 bus /gobi_6: Setting Powered False
18:53:14.935274 bus /gobi_6: Setting Powered False
----------------------------------------------
trial-2064 gprs:sysmo ping_idle_ping.py
----------------------------------------------
18:53:16.079068 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.090959 tst gprs:sysmo: Using 1 x bts (candidates: 1)
18:53:16.126507 tst ping_idle_ping.py: using LAC 3949
18:53:16.254656 tst ping_idle_ping.py: using RAC 124
18:53:16.384775 tst ping_idle_ping.py: using CellId 3949
18:53:16.505449 tst ping_idle_ping.py: using BVCI 3950
18:53:16.616094 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.626804 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.637471 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.648230 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.658742 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.669406 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.680100 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.690661 tst gprs:sysmo: Using 1 x modem (candidates: 4)
18:53:16.808349 tst ping_idle_ping.py:24: start network...
18:53:17.028600 run osmo-hlr_10.42.42.2: Starting osmo-hlr
18:53:17.533122 run create_hlr_db(pid=27333): Launched
18:53:17.658259 bus /gobi_6: Setting Powered False
18:53:18.723381 run create_hlr_db(pid=27333): Terminated: ok {rc=0}
18:53:19.159982 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Launched
18:53:19.466114 run osmo-hlr_10.42.42.2(pid=27336): Launched
18:53:19.584068 run osmo-stp_10.42.42.5: Starting osmo-stp
18:53:20.105723 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Launched
18:53:20.411899 run osmo-stp_10.42.42.5(pid=27338): Launched
18:53:20.530549 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:53:21.050559 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Launched
18:53:21.167295 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:53:21.424726 run patchelf(pid=27340): Launched
18:53:22.597045 run patchelf(pid=27340): Terminated: ok {rc=0}
18:53:22.724490 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:53:23.004661 run osmo-gsm-tester_setcap_net_admin.sh(pid=27341): Launched
18:53:24.182487 run osmo-gsm-tester_setcap_net_admin.sh(pid=27341): Terminated: ok {rc=0}
18:53:24.497812 run osmo-ggsn_10.42.42.6(pid=27344): Launched
18:53:24.622209 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:53:25.283124 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Launched
18:53:25.587362 run osmo-sgsn_10.42.42.7(pid=27362): Launched
18:53:25.703244 run osmo-msc_10.42.42.8: Starting osmo-msc
18:53:26.254298 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Launched
18:53:26.559736 run osmo-msc_10.42.42.8(pid=27364): Launched
18:53:29.731189 run osmo-mgw_10.42.42.3: Starting osmo-mgw
18:53:30.249747 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Launched
18:53:30.553884 run osmo-mgw_10.42.42.3(pid=27366): Launched
18:53:30.670312 run osmo-mgw_10.42.42.4: Starting osmo-mgw
18:53:31.176762 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Launched
18:53:31.480988 run osmo-mgw_10.42.42.4(pid=27368): Launched
18:53:31.597684 run osmo-bsc_10.42.42.9: Starting osmo-bsc
18:53:32.224502 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Launched
18:53:32.528231 run osmo-bsc_10.42.42.9(pid=27370): Launched
18:53:38.797577 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:53:39.327861 run rm-remote-dir(pid=27371): Launched
18:53:41.511474 run rm-remote-dir(pid=27371): Terminated: ok {rc=0}
18:53:41.872872 run mk-remote-dir(pid=27372): Launched
18:53:43.063839 run mk-remote-dir(pid=27372): Terminated: ok {rc=0}
18:53:43.371396 run scp-inst-to-remote(pid=27373): Launched
18:53:55.581549 run scp-inst-to-remote(pid=27373): Terminated: ok {rc=0}
18:53:55.938539 run mk-remote-dir(pid=27375): Launched
18:53:57.126607 run mk-remote-dir(pid=27375): Terminated: ok {rc=0}
18:53:57.431550 run scp-cfg-to-remote(pid=27376): Launched
18:53:58.619835 run scp-cfg-to-remote(pid=27376): Terminated: ok {rc=0}
18:53:58.970296 run reload-dsp-firmware(pid=27378): Launched
18:54:01.158887 run reload-dsp-firmware(pid=27378): Terminated: ok {rc=0}
18:54:01.502246 run osmo-bts-sysmo(pid=27379): Launched
18:54:05.742859 tst ping_idle_ping.py:36: Waiting for bts to be ready...
18:54:07.463034 run rm-remote-dir(pid=27380): Launched
18:54:09.647121 run rm-remote-dir(pid=27380): Terminated: ok {rc=0}
18:54:09.990246 run mk-remote-dir(pid=27383): Launched
18:54:12.172710 run mk-remote-dir(pid=27383): Terminated: ok {rc=0}
18:54:12.446443 run scp-inst-to-bts(pid=27384): Launched
18:54:25.631617 run scp-inst-to-bts(pid=27384): Terminated: ok {rc=0}
18:54:25.953028 run mk-remote-run-dir(pid=27386): Launched
18:54:28.124103 run mk-remote-run-dir(pid=27386): Terminated: ok {rc=0}
18:54:28.397430 run scp-cfg-to-sysmobts(pid=27387): Launched
18:54:30.569468 run scp-cfg-to-sysmobts(pid=27387): Terminated: ok {rc=0}
18:54:30.912395 run osmo-pcu(pid=27389): Launched
18:54:31.049751 tst ping_idle_ping.py: using MSISDN 3957
18:54:31.223360 bus /gobi_6: Setting Powered True
18:54:34.734287 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3957', subscriber_id=1}
18:54:34.997006 bus /gobi_6: Setting Online False
18:54:36.183822 bus /gobi_6: Setting Powered False
18:54:43.108849 bus /gobi_6: Setting Powered True
18:54:44.475734 bus /gobi_6: Setting Online True
18:54:46.966328 tst /gobi_6: Connect to ('901', '70')
18:54:47.251806 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648'
18:54:47.368116 tst ping_idle_ping.py:47: waiting for modems to attach...
18:54:54.342296 tst /gobi_6: Already registered with network ('901', '70')
18:55:05.838172 tst ping_idle_ping.py:51: waiting for modems to attach to data services...
18:55:58.827522 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:55:58.946065 tst ping_idle_ping.py:56: Setting up data plane for "'/gobi_6/context2'"
18:55:59.392822 run move_netns(pid=27396): Launched
18:56:00.587265 run move_netns(pid=27396): Terminated: ok {rc=0}
18:56:00.941440 run ifup(pid=27399): Launched
18:56:02.123449 run ifup(pid=27399): Terminated: ok {rc=0}
18:56:02.450444 run dhcp(pid=27402): Launched
18:56:03.626231 run dhcp(pid=27402): Terminated: ok {rc=0}
18:56:03.755621 tst ping_idle_ping.py:60: [1] Running 10 ping requests for "'/gobi_6/context2'"
18:56:04.155091 run ping1(pid=27418): Launched
18:56:14.359359 run ping1(pid=27418): Terminated: ok {rc=0}
18:56:14.495867 tst ping_idle_ping.py:64: (launched: 2022-03-29_18:56:04.093106)
27419: waiting for 27420
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=679 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=638 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=615 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=596 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=592 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=532 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=533 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=492 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=488 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=487 ms
--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 21ms
rtt min/avg/max/mdev = 486.902/565.056/678.590/64.750 ms
child exited with 0
18:56:14.718200 tst ping_idle_ping.py:68: Sleeping for 60 seconds
18:57:14.941764 tst ping_idle_ping.py:73: [2] Running 10 ping requests for "'/gobi_6/context2'"
18:57:15.372936 run ping2(pid=27424): Launched
18:57:25.578233 run ping2(pid=27424): Terminated: ok {rc=0}
18:57:25.714790 tst ping_idle_ping.py:77: (launched: 2022-03-29_18:57:15.307744)
27425: waiting for 27426
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=536 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=454 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=431 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=413 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=431 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=427 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=408 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=405 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=405 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=381 ms
--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 21ms
rtt min/avg/max/mdev = 381.454/429.068/536.304/40.329 ms
child exited with 0
18:57:27.707993 tst /gobi_6: context deactivated {path='/gobi_6/context2'}
18:57:27.817824 tst ping_idle_ping.py: Test passed (251.7 sec)
18:57:27.883815 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Terminating (SIGTERM)
18:57:27.925168 run osmo-hlr_10.42.42.2(pid=27336): Terminating (SIGTERM)
18:57:27.966164 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Terminating (SIGTERM)
18:57:28.009810 run osmo-stp_10.42.42.5(pid=27338): Terminating (SIGTERM)
18:57:28.059656 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Terminating (SIGTERM)
18:57:28.105538 run osmo-ggsn_10.42.42.6(pid=27344): Terminating (SIGTERM)
18:57:28.150157 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Terminating (SIGTERM)
18:57:28.210933 run osmo-sgsn_10.42.42.7(pid=27362): Terminating (SIGTERM)
18:57:28.278431 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Terminating (SIGTERM)
18:57:28.325669 run osmo-msc_10.42.42.8(pid=27364): Terminating (SIGTERM)
18:57:28.371429 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Terminating (SIGTERM)
18:57:28.416376 run osmo-mgw_10.42.42.3(pid=27366): Terminating (SIGTERM)
18:57:28.461094 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Terminating (SIGTERM)
18:57:28.507834 run osmo-mgw_10.42.42.4(pid=27368): Terminating (SIGTERM)
18:57:28.551819 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Terminating (SIGTERM)
18:57:28.596373 run osmo-bsc_10.42.42.9(pid=27370): Terminating (SIGTERM)
18:57:28.639641 run osmo-bts-sysmo(pid=27379): Terminating (SIGTERM)
18:57:28.683368 run osmo-pcu(pid=27389): Terminating (SIGTERM)
18:57:28.727835 --- ParallelTerminationStrategy: PID 27335 died...
18:57:28.748547 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Terminated: ok {rc=0}
18:57:28.797671 --- ParallelTerminationStrategy: PID 27336 died...
18:57:28.818327 run osmo-hlr_10.42.42.2(pid=27336): Terminated: ok {rc=0}
18:57:28.867859 --- ParallelTerminationStrategy: PID 27337 died...
18:57:28.888557 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Terminated: ok {rc=0}
18:57:28.938065 --- ParallelTerminationStrategy: PID 27338 died...
18:57:28.958857 run osmo-stp_10.42.42.5(pid=27338): Terminated: ok {rc=0}
18:57:29.008339 --- ParallelTerminationStrategy: PID 27339 died...
18:57:29.028524 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Terminated: ok {rc=0}
18:57:29.076642 --- ParallelTerminationStrategy: PID 27344 died...
18:57:29.097617 run osmo-ggsn_10.42.42.6(pid=27344): Terminated: ok {rc=0}
18:57:29.145956 --- ParallelTerminationStrategy: PID 27361 died...
18:57:29.166800 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Terminated: ok {rc=0}
18:57:29.215559 --- ParallelTerminationStrategy: PID 27363 died...
18:57:29.236338 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Terminated: ok {rc=0}
18:57:29.285039 --- ParallelTerminationStrategy: PID 27364 died...
18:57:29.305812 run osmo-msc_10.42.42.8(pid=27364): Terminated: ok {rc=0}
18:57:29.354191 --- ParallelTerminationStrategy: PID 27365 died...
18:57:29.375437 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Terminated: ok {rc=0}
18:57:29.424266 --- ParallelTerminationStrategy: PID 27366 died...
18:57:29.445247 run osmo-mgw_10.42.42.3(pid=27366): Terminated {rc=15}
18:57:29.493765 --- ParallelTerminationStrategy: PID 27367 died...
18:57:29.514624 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Terminated: ok {rc=0}
18:57:29.563536 --- ParallelTerminationStrategy: PID 27368 died...
18:57:29.584396 run osmo-mgw_10.42.42.4(pid=27368): Terminated {rc=15}
18:57:29.632948 --- ParallelTerminationStrategy: PID 27369 died...
18:57:29.653818 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Terminated: ok {rc=0}
18:57:29.702845 --- ParallelTerminationStrategy: PID 27370 died...
18:57:29.723571 run osmo-bsc_10.42.42.9(pid=27370): Terminated: ok {rc=0}
18:57:29.772222 --- ParallelTerminationStrategy: PID 27379 died...
18:57:29.793199 run osmo-bts-sysmo(pid=27379): Terminated: ok {rc=0}
18:57:29.842106 --- ParallelTerminationStrategy: PID 27389 died...
18:57:29.863251 run osmo-pcu(pid=27389): Terminated: ok {rc=0}
18:57:30.080476 --- ParallelTerminationStrategy: PID 27362 died...
18:57:30.102519 run osmo-sgsn_10.42.42.7(pid=27362): Terminated: ok {rc=0}
18:57:30.325893 bus /gobi_6: Setting Online False
18:57:33.386328 bus /gobi_6: Setting Powered False
18:57:49.001216 bus /gobi_6: Setting Powered False
---------------------------------------------------------------------
trial-2064 gprs:sysmo FAIL
---------------------------------------------------------------------
18:57:50.341374 tst trial-2064: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064/run.2022-03-29_18-32-52/trial-2064.xml
--------------------------------------------------------------------------------------------
trial-2064 FAIL
--------------------------------------------------------------------------------------------
18:57:50.571122 tst trial-2064: trial-2064: FAIL, 2 suites failed, 1 suites passed
FAIL: nitb_sms:sysmo (fail: 1)
FAIL: mo_mt_sms.py (5.1 sec) Error: osmo-nitb_10.42.42.2(pid=26640): Process ended prematurely: osmo-nitb_10.42.42.2(pid=26640) [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=26640)]
FAIL: gprs:sysmo (fail: 3, pass: 2)
FAIL: cs_paging_gprs_active.py (228.9 sec) KeyError: 'sum_sent'
FAIL: iperf3.py (196.8 sec) KeyError: 'sum_sent'
FAIL: iperf3m4.py (294.8 sec) KeyError: 'sum_sent'
pass: ping.py (150.5 sec)
pass: ping_idle_ping.py (251.7 sec)
PASS: sms:sysmo (pass: 1)
pass: mo_mt_sms.py (116.8 sec)
+ rm -rf trial-2064/inst
+ cd trial-2064
+ command -v journalctl
+ [ ! -z /bin/journalctl ]
+ readlink last_run
+ journalctl -u ofono -o short-precise --since 2022-03-29 18:32:50
+ readlink last_run
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064-run.tgz run.2022-03-29_18-32-52
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064-bin.tgz checksums.md5 open5gs.build-8.tgz osmo-bsc.build-15570.tgz osmo-bts-oc2g.build-4604.tgz osmo-bts-sysmo.build-7108.tgz osmo-bts.build-6578.tgz osmo-ggsn.build-5814.tgz osmo-hlr.build-7536.tgz osmo-hnbgw.build-592.tgz osmo-mgw.build-7731.tgz osmo-msc.build-25979.tgz osmo-nitb.build-11307.tgz osmo-pcu-oc2g.build-3308.tgz osmo-pcu-sysmo.build-4589.tgz osmo-pcu.build-4550.tgz osmo-sgsn.build-26313.tgz osmo-stp.build-8062.tgz osmo-trx.build-3840.tgz osmocom-bb.build-4035.tgz srslte.build-19.tgz
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
ERROR Gerrit response: ConflictFinished: FAILURE
![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
