Skip to content
Failed

Console Output

Triggered by Gerrit: https://gerrit.osmocom.org/c/osmo-gsm-tester/+/27601
Running as SYSTEM
Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_gerrit
The recommended git tool is: NONE
using credential d5eda5e9-b59d-44ba-88d2-43473cb6e42d
 > git rev-parse --resolve-git-dir /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
Fetching upstream changes from ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester
 > git --version # timeout=10
 > git --version # 'git version 2.20.1'
using GIT_SSH to set credentials 
 > git fetch --tags --force --progress -- ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/01/27601/1 # timeout=10
 > git rev-parse 5a80d769ec542c1b578a2924f372d41e2757b3c4^{commit} # timeout=10
JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://plugins.jenkins.io/git/#remove-git-plugin-buildsbybranch-builddata-script
Checking out Revision 5a80d769ec542c1b578a2924f372d41e2757b3c4 (refs/changes/01/27601/1)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5a80d769ec542c1b578a2924f372d41e2757b3c4 # timeout=10
Commit message: "contrib: Fix osmocom-bb-fimware no longer available in order prebuilt pkg repo"
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
 > git rev-list --no-walk ae8aa17301137f70e586478f9db5e19ab5380ea9 # timeout=10
[osmo-gsm-tester_gerrit] $ /bin/sh -xe /tmp/jenkins3140653776722019423.sh
+ rm -f trial-2063-bin.tgz trial-2063-run.tgz *.md5
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bsc" build number 15570
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts" build number 6578
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-ggsn" build number 5814
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hlr" build number 7536
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-mgw" build number 7731
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hnbgw" build number 592
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-msc" build number 25979
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu" build number 4550
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-sgsn" build number 26313
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-trx" build number 3840
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-nitb" build number 11307
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-stp" build number 8062
Copied 2 artifacts from "osmo-gsm-tester_build-osmocom-bb" build number 4035
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-sysmo" build number 7108
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-sysmo" build number 4589
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-oc2g" build number 4604
Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-oc2g" build number 3308
Copied 2 artifacts from "osmo-gsm-tester_build-open5gs" build number 8
Copied 2 artifacts from "osmo-gsm-tester_build-srslte" build number 19
[osmo-gsm-tester_gerrit] $ /bin/sh /tmp/jenkins8606377750804127806.sh
+ unlink osmo-gsm-tester/sysmocom/resources.conf
unlink: cannot unlink 'osmo-gsm-tester/sysmocom/resources.conf': No such file or directory
+ true
+ ln -s resources.conf.prod osmo-gsm-tester/sysmocom/resources.conf
+ export OSMO_GSM_TESTER_OPTS=-s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo
+ ./osmo-gsm-tester/contrib/jenkins-make-check-and-run.sh
+ base=/home/jenkins/workspace/osmo-gsm-tester_gerrit
+ run_script=osmo-gsm-tester/contrib/jenkins-run.sh
+ test -x osmo-gsm-tester/contrib/jenkins-run.sh
+ cd osmo-gsm-tester
+ make deps
./contrib/check_dependencies.py
Skip checking modules: []
Changing workdir dir to /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester
importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_ms_driver
importing src.osmo_ms_driver.location_update_test
importing src.osmo_ms_driver.event_server
importing src.osmo_ms_driver.cdf
importing src.osmo_ms_driver.__main__
importing src.osmo_ms_driver.__init__
importing src.osmo_ms_driver.test_support
importing src.osmo_ms_driver.starter
importing src.osmo_ms_driver.simple_loop
importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core
importing src.osmo_gsm_tester.core.util
importing src.osmo_gsm_tester.core.schema
importing src.osmo_gsm_tester.core.trial
importing src.osmo_gsm_tester.core.template
importing src.osmo_gsm_tester.core.config
importing src.osmo_gsm_tester.core.event_loop
importing src.osmo_gsm_tester.core.log
importing src.osmo_gsm_tester.core.scenario
importing src.osmo_gsm_tester.core.remote
importing src.osmo_gsm_tester.core.__init__
importing src.osmo_gsm_tester.core.process
importing src.osmo_gsm_tester.core.report
importing src.osmo_gsm_tester.core.test
importing src.osmo_gsm_tester.core.resource
importing src.osmo_gsm_tester.core.suite
importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj
importing src.osmo_gsm_tester.obj.android_apn
importing src.osmo_gsm_tester.obj.bts
importing src.osmo_gsm_tester.obj.mgcpgw_osmo
importing src.osmo_gsm_tester.obj.sgsn_osmo
importing src.osmo_gsm_tester.obj.epc_srs
importing src.osmo_gsm_tester.obj.bts_nanobts
importing src.osmo_gsm_tester.obj.enb_srs
importing src.osmo_gsm_tester.obj.esme
importing src.osmo_gsm_tester.obj.qc_diag
importing src.osmo_gsm_tester.obj.stp_osmo
importing src.osmo_gsm_tester.obj.ms_ofono
importing src.osmo_gsm_tester.obj.ms_android
importing src.osmo_gsm_tester.obj.osmo_vty
importing src.osmo_gsm_tester.obj.hlr_osmo
importing src.osmo_gsm_tester.obj.enb_amarisoft
importing src.osmo_gsm_tester.obj.android_host
importing src.osmo_gsm_tester.obj.sgwu_open5gs
importing src.osmo_gsm_tester.obj.powersupply
importing src.osmo_gsm_tester.obj.iperf3
importing src.osmo_gsm_tester.obj.epc
importing src.osmo_gsm_tester.obj.rfemu_minicircuits
importing src.osmo_gsm_tester.obj.bts_osmo
importing src.osmo_gsm_tester.obj.upf_open5gs
importing src.osmo_gsm_tester.obj.nitb_osmo
importing src.osmo_gsm_tester.obj.android_bitrate_monitor
importing src.osmo_gsm_tester.obj.pcu_oc2g
importing src.osmo_gsm_tester.obj.ms_srs
importing src.osmo_gsm_tester.obj.ms_driver
importing src.osmo_gsm_tester.obj.bts_sysmo
importing src.osmo_gsm_tester.obj.rfemu_srsenb_stdin
importing src.osmo_gsm_tester.obj.pcu
importing src.osmo_gsm_tester.obj.osmocon
importing src.osmo_gsm_tester.obj.epc_amarisoft
importing src.osmo_gsm_tester.obj.ggsn_osmo
importing src.osmo_gsm_tester.obj.enb
importing src.osmo_gsm_tester.obj.rfemu
importing src.osmo_gsm_tester.obj.bts_octphy
importing src.osmo_gsm_tester.obj.pcap_recorder
importing src.osmo_gsm_tester.obj.sms
importing src.osmo_gsm_tester.obj.bts_oc2g
importing src.osmo_gsm_tester.obj.stress
importing src.osmo_gsm_tester.obj.smsc
importing src.osmo_gsm_tester.obj.msc_osmo
importing src.osmo_gsm_tester.obj.sgwc_open5gs
importing src.osmo_gsm_tester.obj.osmo_ctrl
importing src.osmo_gsm_tester.obj.bts_osmotrx
importing src.osmo_gsm_tester.obj.rfemu_amarisoftctrl
importing src.osmo_gsm_tester.obj.ms
importing src.osmo_gsm_tester.obj.powersupply_intellinet
importing src.osmo_gsm_tester.obj.ms_amarisoft
importing src.osmo_gsm_tester.obj.srslte_common
importing src.osmo_gsm_tester.obj.powersupply_sispm
importing src.osmo_gsm_tester.obj.gnuradio_zmq_broker
importing src.osmo_gsm_tester.obj.pcu_sysmo
importing src.osmo_gsm_tester.obj.bts_osmovirtual
importing src.osmo_gsm_tester.obj.hss_open5gs
importing src.osmo_gsm_tester.obj.mgw_osmo
importing src.osmo_gsm_tester.obj.smf_open5gs
importing src.osmo_gsm_tester.obj.pcrf_open5gs
importing src.osmo_gsm_tester.obj.mme_open5gs
importing src.osmo_gsm_tester.obj.bsc_osmo
importing src.osmo_gsm_tester.obj.epc_open5gs
importing src.osmo_gsm_tester.obj.run_node
importing src.osmo_gsm_tester.obj.pcu_osmo
importing src.osmo_gsm_tester.obj.rfemu_gnuradio_zmq
importing src.osmo_gsm_tester.obj.ms_osmo_mobile
importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester
importing src.osmo_gsm_tester.testenv
importing src.osmo_gsm_tester.__init__
Importing dependencies ok, all installed
Retreiving list of imported modules...
+ make check
make -C selftest check
make[1]: Entering directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest'
echo "export PYTHONPATH=\"/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/../src\"" > set_pythonpath
./all_tests.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/cdf_test/cdf_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/config_test/config_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/lock_test/lock_test.sh
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/log_test/log_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/process_test/process_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/rate_ctrs_test/rate_ctrs_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/report_test/report_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/resource_test/resource_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/scenario_test/scenario_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/schema_test/schema_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/sms_test/sms_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/suite_test/suite_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/template_test/template_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/trial_test/trial_test.py
/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/util_test/util_test.py
15 tests ok
make[1]: Leaving directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest'
make check: success
+ ./contrib/jenkins-build-manuals.sh
+ base=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester
+ export OSMO_GSM_MANUALS_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals
+ [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals ]
+ [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals ]
+ git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals pull
From https://git.osmocom.org/osmo-gsm-manuals
   b1318e3..5cba08b  master     -> origin/master
Updating b1318e3..5cba08b
Fast-forward
 common/chapters/gb-ns2.adoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
+ git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals checkout -f HEAD
Your branch is up to date with 'origin/master'.
+ temp=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp
+ [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp ]
+ cp -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp
+ cd /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp
+ make
OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env)
ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build build
ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common common
# a2x can't use a different output file. To support out-of-tree builds,
# we create a symlink at /srcfile.adoc pointing at
# /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc. osmo-gsm-tester-manual.adoc is the /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc,
#  is like basename from coreutils, and  is WD.
if ! [ -f osmo-gsm-tester-manual.adoc ]; then \
	ln -s osmo-gsm-tester-manual.adoc osmo-gsm-tester-manual.adoc; \
fi
# TEXINPUTS: find LaTeX includes like \includegraphics{./common/images/sysmocom.pdf}
# in /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals.
TEXINPUTS="/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals" \
	a2x -vv -L --asciidoc-opts="-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'" --dblatex-opts="-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0" -a docinfo -a revnumber="DRAFT 0.1-1214-g5a80" -a revdate="2022-Mar-29" osmo-gsm-tester-manual.adoc
a2x: args: ['-vv', '-L', "--asciidoc-opts=-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'", '--dblatex-opts=-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0', '-a', 'docinfo', '-a', 'revnumber=DRAFT 0.1-1214-g5a80', '-a', 'revdate=2022-Mar-29', 'osmo-gsm-tester-manual.adoc']
a2x: resource files: []
a2x: resource directories: ['/etc/asciidoc/images', '/etc/asciidoc/stylesheets']
a2x: executing: "/usr/bin/asciidoc" --backend docbook -a "a2x-format=pdf" -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common' --attribute "docinfo" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "revdate=2022-Mar-29" --verbose  --out-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml" "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc"

asciidoc: reading: /etc/asciidoc/asciidoc.conf
asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf
asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf
asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf
asciidoc: reading: /etc/asciidoc/asciidoc.conf
asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc
asciidoc: reading: /etc/asciidoc/docbook45.conf
asciidoc: reading: /etc/asciidoc/filters/code/code-filter.conf
asciidoc: reading: /etc/asciidoc/filters/graphviz/graphviz-filter.conf
asciidoc: reading: /etc/asciidoc/filters/music/music-filter.conf
asciidoc: reading: /etc/asciidoc/filters/source/source-highlight-filter.conf
asciidoc: reading: /etc/asciidoc/filters/latex/latex-filter.conf
asciidoc: reading: /etc/asciidoc/lang-en.conf
asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf
asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf
asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf
asciidoc: writing: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml
asciidoc: osmo-gsm-tester-manual.adoc: line 5: evaluating: {include:/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual-docinfo.xml}
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 13: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc
asciidoc: intro.adoc: line 55: evaluating: {counter2:target-number}
asciidoc: intro.adoc: line 55: evaluating: {set2:target:osmo-gsm-tester-manual__1.png}
asciidoc: intro.adoc: line 103: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png" -L dot -F png -
Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png"
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 15: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc
asciidoc: trial.adoc: line 14: evaluating: {counter2:target-number}
asciidoc: trial.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__2.png}
asciidoc: trial.adoc: line 25: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png" -L dot -F png -
Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png"
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 17: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 19: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc
asciidoc: resource_pool.adoc: line 14: evaluating: {counter2:target-number}
asciidoc: resource_pool.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__3.png}
asciidoc: resource_pool.adoc: line 79: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png" -L dot -F png -
Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png"
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc
asciidoc: resource_pool.adoc: line 111: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc
asciidoc: config_mangling.adoc: line 165: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45  --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Mar-29" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s -
asciidoc: config_mangling.adoc: line 255: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45  --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Mar-29" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s -
asciidoc: config_mangling.adoc: line 330: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45  --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1214-g5a80" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Mar-29" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s -
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 21: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 23: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 25: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 27: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 29: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 31: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc
asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc
asciidoc: osmo-gsm-tester-manual.adoc: line 33: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc

a2x: executing: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0 -V  "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml"

Build the book set list...
xsltproc -o doclist.txt --xinclude --xincludestyle doclist.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml
Build the listings...
xsltproc -o listings.xml --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' /usr/share/dblatex/xsl/common/mklistings.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml
xsltproc -o osmo-gsm-tester-manual.rtex --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' --param listings.xml '/tmp/tmpnYNw_7/listings.xml' /tmp/tmpnYNw_7/custom.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml
XSLT stylesheets DocBook - LaTeX 2e (0.3.10)
===================================================
WARNING: xref to <simpara id="scenario_suite_params"> has no generated text. Trying its ancestor elements.
Image 'dblatex' not found
Cannot convert 'osmo-gsm-tester-manual__1.png' to pdf
Cannot convert 'osmo-gsm-tester-manual__2.png' to pdf
Cannot convert 'osmo-gsm-tester-manual__3.png' to pdf
Build osmo-gsm-tester-manual.pdf
built-in module pdftex registered
no support found for ifthen
no support found for ifxetex
no support found for fontspec
no support found for xltxtra
no support found for fontenc
no support found for inputenc
no support found for fancybox
built-in module makeidx registered
no support found for custom-dblatex
building additional files...
checking if compiling is necessary...
the output file doesn't exist
pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex
running post-compilation scripts...
[index] the index file /tmp/tmpnYNw_7/osmo-gsm-tester-manual.idx is empty
osmo-gsm-tester-manual.toc MD5 checksum changed
osmo-gsm-tester-manual.aux MD5 checksum changed
the /tmp/tmpnYNw_7/osmo-gsm-tester-manual.toc file has changed
pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex
running post-compilation scripts...
[index] the index file /tmp/tmpnYNw_7/osmo-gsm-tester-manual.idx is empty
no new compilation is needed
running last-compilation scripts...
'osmo-gsm-tester-manual.pdf' successfully built

a2x: deleting /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml
+ make check
OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env)
if [ -n "$ASCIIDOC_WARNINGS_CHECK" ]; then \
	make osmo-gsm-tester-manual.check; \
fi
+ [  = 1 ]
+ rm -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp
+ cd /home/jenkins/workspace/osmo-gsm-tester_gerrit
+ PATH=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games osmo-gsm-tester/contrib/jenkins-run.sh
+ base=/home/jenkins/workspace/osmo-gsm-tester_gerrit
+ dirname -- osmo-gsm-tester/contrib/jenkins-run.sh
+ CDPATH= cd -- osmo-gsm-tester/contrib
+ pwd -P
+ SCRIPT_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib
+ OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf
+ date +%F %T
+ time_start=2022-03-29 18:32:50
+ trial_dir_prefix=trial-
+ rm -rf trial-2063
+ trial_dir=trial-2064
+ mkdir -p trial-2064
+ mv open5gs.build-8.tgz osmo-bsc.build-15570.tgz osmo-bts-oc2g.build-4604.tgz osmo-bts-sysmo.build-7108.tgz osmo-bts.build-6578.tgz osmo-ggsn.build-5814.tgz osmo-hlr.build-7536.tgz osmo-hnbgw.build-592.tgz osmo-mgw.build-7731.tgz osmo-msc.build-25979.tgz osmo-nitb.build-11307.tgz osmo-pcu-oc2g.build-3308.tgz osmo-pcu-sysmo.build-4589.tgz osmo-pcu.build-4550.tgz osmo-sgsn.build-26313.tgz osmo-stp.build-8062.tgz osmo-trx.build-3840.tgz osmocom-bb.build-4035.tgz srslte.build-19.tgz trial-2064
+ cat open5gs.build-8.md5 osmo-bsc.build-15570.md5 osmo-bts-oc2g.build-4604.md5 osmo-bts-sysmo.build-7108.md5 osmo-bts.build-6578.md5 osmo-ggsn.build-5814.md5 osmo-hlr.build-7536.md5 osmo-hnbgw.build-592.md5 osmo-mgw.build-7731.md5 osmo-msc.build-25979.md5 osmo-nitb.build-11307.md5 osmo-pcu-oc2g.build-3308.md5 osmo-pcu-sysmo.build-4589.md5 osmo-pcu.build-4550.md5 osmo-sgsn.build-26313.md5 osmo-stp.build-8062.md5 osmo-trx.build-3840.md5 osmocom-bb.build-4035.md5 srslte.build-19.md5
+ rm open5gs.build-8.md5 osmo-bsc.build-15570.md5 osmo-bts-oc2g.build-4604.md5 osmo-bts-sysmo.build-7108.md5 osmo-bts.build-6578.md5 osmo-ggsn.build-5814.md5 osmo-hlr.build-7536.md5 osmo-hnbgw.build-592.md5 osmo-mgw.build-7731.md5 osmo-msc.build-25979.md5 osmo-nitb.build-11307.md5 osmo-pcu-oc2g.build-3308.md5 osmo-pcu-sysmo.build-4589.md5 osmo-pcu.build-4550.md5 osmo-sgsn.build-26313.md5 osmo-stp.build-8062.md5 osmo-trx.build-3840.md5 osmocom-bb.build-4035.md5 srslte.build-19.md5
+ exit_code=1
+ which osmo-gsm-tester.py
+ python3 -u /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo-gsm-tester.py -c /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf trial-2064 -s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo
combinations: ['nitb_sms:sysmo', 'sms:sysmo', 'gprs:sysmo']
trial: 'trial-2064'
tests: None

--------------------------------------------------------------------------------------------
trial-2064
--------------------------------------------------------------------------------------------
18:32:52.346499 tst                       trial-2064: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064/run.2022-03-29_18-32-52/log

---------------------------------------------------------------------
trial-2064 nitb_sms:sysmo
---------------------------------------------------------------------
18:32:55.173840 tst                   nitb_sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ...
18:32:55.177601 tst                   nitb_sms:sysmo: Verifying 1 x bts (candidates: 11)
18:32:55.198970 tst                   nitb_sms:sysmo: Verifying 1 x ip_address (candidates: 9)
18:32:55.206105 tst                   nitb_sms:sysmo: Verifying 2 x modem (candidates: 12)
18:32:55.263287 tst                   nitb_sms:sysmo: Reserving 1 x bts (candidates: 11)
18:32:55.285008 tst                   nitb_sms:sysmo: Reserving 1 x ip_address (candidates: 9)
18:32:55.291918 tst                   nitb_sms:sysmo: Reserving 2 x modem (candidates: 12)
18:32:55.831054 tst                   nitb_sms:sysmo: RESERVED RESOURCES for nitb_sms:
bts
  sysmoBTS 1002
ip_address
  10.42.42.2
modem
  sierra_1st
  sierra_2nd

----------------------------------------------
trial-2064 nitb_sms:sysmo mo_mt_sms.py
----------------------------------------------
18:32:55.838323 tst                   nitb_sms:sysmo: Using 1 x ip_address (candidates: 1)
18:32:55.849461 tst                   nitb_sms:sysmo: Using 1 x bts (candidates: 1)
18:32:56.226482 tst                     mo_mt_sms.py: using LAC 3943
18:32:56.472628 tst                     mo_mt_sms.py: using RAC 118
18:32:56.594157 tst                     mo_mt_sms.py: using CellId 3943
18:32:56.714254 tst                     mo_mt_sms.py: using BVCI 3944
18:32:56.826162 tst                   nitb_sms:sysmo: Using 1 x modem (candidates: 2)
18:32:57.032268 tst                   nitb_sms:sysmo: Using 1 x modem (candidates: 2)
18:32:57.148887 tst                   mo_mt_sms.py:9: start nitb and bts...
18:32:57.363011 run             osmo-nitb_10.42.42.2: Starting osmo-nitb
18:32:59.729286 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=26639): Launched
18:33:00.052345 run  osmo-nitb_10.42.42.2(pid=26640): Launched
18:33:00.228664 run  osmo-nitb_10.42.42.2(pid=26640): ERR: Terminated: ERROR {rc=2}  [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=26640)]
18:33:00.359508 run  osmo-nitb_10.42.42.2(pid=26640): stdout: 
| (launched: 2022-03-29_18:32:59.992947) 
18:33:00.492150 run  osmo-nitb_10.42.42.2(pid=26640): stderr: 
| (launched: 2022-03-29_18:33:00.040289)
| ======================================================================
| ERROR: You should not run osmo-nitb. It is ancient, unmaintained and
| obsolete for several years.  If you really want to ignore years of hard work on
| the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp),
| you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line
| argument.  DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!!
| Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box
| ====================================================================== 
18:33:00.624403 run  osmo-nitb_10.42.42.2(pid=26640): stdout: 
| (launched: 2022-03-29_18:32:59.992947) 
18:33:00.751165 run  osmo-nitb_10.42.42.2(pid=26640): stderr: 
| (launched: 2022-03-29_18:33:00.040289)
| ======================================================================
| ERROR: You should not run osmo-nitb. It is ancient, unmaintained and
| obsolete for several years.  If you really want to ignore years of hard work on
| the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp),
| you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line
| argument.  DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!!
| Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box
| ====================================================================== 
18:33:00.902948 tst                  mo_mt_sms.py:11: ERR: Error: osmo-nitb_10.42.42.2(pid=26640): Process ended prematurely: osmo-nitb_10.42.42.2(pid=26640) [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=26640)]  [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11]
18:33:00.911748 tst                  mo_mt_sms.py:11: Test FAILED (5.1 sec)
18:33:00.918510 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=26639): Terminating (SIGTERM)
18:33:01.129201 ---      ParallelTerminationStrategy: PID 26639 died...
18:33:01.150897 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=26639): Terminated: ok {rc=0}
18:33:01.239280 bus                        /sierra_3: Setting Powered False
18:33:02.339581 bus                        /sierra_3: Setting Powered False
18:33:03.459342 bus                        /sierra_2: Setting Powered False
---------------------------------------------------------------------
trial-2064 nitb_sms:sysmo FAIL
---------------------------------------------------------------------

---------------------------------------------------------------------
trial-2064 sms:sysmo
---------------------------------------------------------------------
18:33:04.637858 tst                        sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ...
18:33:04.641828 tst                        sms:sysmo: Verifying 1 x bts (candidates: 11)
18:33:04.663298 tst                        sms:sysmo: Verifying 6 x ip_address (candidates: 9)
18:33:04.698672 tst                        sms:sysmo: Verifying 2 x modem (candidates: 12)
18:33:04.753423 tst                        sms:sysmo: Reserving 1 x bts (candidates: 11)
18:33:04.775146 tst                        sms:sysmo: Reserving 6 x ip_address (candidates: 9)
18:33:04.802770 tst                        sms:sysmo: Reserving 2 x modem (candidates: 12)
18:33:05.351204 tst                        sms:sysmo: RESERVED RESOURCES for sms:
bts
  sysmoBTS 1002
ip_address
  10.42.42.2
  10.42.42.3
  10.42.42.4
  10.42.42.5
  10.42.42.6
  10.42.42.7
modem
  sierra_1st
  sierra_2nd

----------------------------------------------
trial-2064 sms:sysmo mo_mt_sms.py
----------------------------------------------
18:33:05.359402 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
18:33:05.370437 tst                        sms:sysmo: Using 1 x bts (candidates: 1)
18:33:05.405248 tst                     mo_mt_sms.py: using LAC 3944
18:33:05.529739 tst                     mo_mt_sms.py: using RAC 119
18:33:05.655003 tst                     mo_mt_sms.py: using CellId 3944
18:33:05.779286 tst                     mo_mt_sms.py: using BVCI 3945
18:33:05.887805 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
18:33:05.898529 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
18:33:05.909255 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
18:33:05.919664 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
18:33:05.930257 tst                        sms:sysmo: Using 1 x ip_address (candidates: 6)
18:33:05.941084 tst                        sms:sysmo: Using 1 x modem (candidates: 2)
18:33:06.059606 tst                        sms:sysmo: Using 1 x modem (candidates: 2)
18:33:06.174614 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
18:33:07.790785 run         create_hlr_db(pid=26644): Launched
18:33:07.904797 bus                        /sierra_2: Setting Powered False
18:33:08.967750 run         create_hlr_db(pid=26644): Terminated: ok {rc=0}
18:33:09.396732 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26646): Launched
18:33:09.720587 run   osmo-hlr_10.42.42.2(pid=26647): Launched
18:33:09.845902 run              osmo-stp_10.42.42.5: Starting osmo-stp
18:33:12.420857 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26648): Launched
18:33:12.741689 run   osmo-stp_10.42.42.5(pid=26649): Launched
18:33:12.862089 run              osmo-msc_10.42.42.6: Starting osmo-msc
18:33:18.319175 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=26650): Launched
18:33:18.642224 run   osmo-msc_10.42.42.6(pid=26651): Launched
18:33:21.820694 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
18:33:23.546630 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26652): Launched
18:33:23.868579 run   osmo-mgw_10.42.42.3(pid=26653): Launched
18:33:23.993165 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
18:33:24.529749 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26654): Launched
18:33:24.851313 run   osmo-mgw_10.42.42.4(pid=26655): Launched
18:33:24.972029 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
18:33:29.475994 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=26656): Launched
18:33:29.798586 run   osmo-bsc_10.42.42.7(pid=26657): Launched
18:33:36.082359 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7
18:33:37.027646 run         rm-remote-dir(pid=26658): Launched
18:33:39.219942 run         rm-remote-dir(pid=26658): Terminated: ok {rc=0}
18:33:39.572658 run         mk-remote-dir(pid=26659): Launched
18:33:40.761198 run         mk-remote-dir(pid=26659): Terminated: ok {rc=0}
18:33:41.066341 run    scp-inst-to-remote(pid=26660): Launched
18:33:53.272603 run    scp-inst-to-remote(pid=26660): Terminated: ok {rc=0}
18:33:53.619887 run         mk-remote-dir(pid=26662): Launched
18:33:54.803456 run         mk-remote-dir(pid=26662): Terminated: ok {rc=0}
18:33:55.102300 run     scp-cfg-to-remote(pid=26663): Launched
18:33:56.285855 run     scp-cfg-to-remote(pid=26663): Terminated: ok {rc=0}
18:33:56.607179 run   reload-dsp-firmware(pid=26665): Launched
18:33:58.779333 run   reload-dsp-firmware(pid=26665): Terminated: ok {rc=0}
18:33:59.094218 run        osmo-bts-sysmo(pid=26666): Launched
18:34:03.324933 tst                     mo_mt_sms.py: using MSISDN 3947
18:34:03.488533 bus                        /sierra_2: Setting Powered True
18:34:06.957432 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='3947', subscriber_id=1}
18:34:07.088860 tst                     mo_mt_sms.py: using MSISDN 3948
18:34:07.260917 bus                        /sierra_3: Setting Powered True
18:34:10.731625 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='3948', subscriber_id=2}
18:34:10.997049 bus                        /sierra_2: Setting Online False
18:34:12.185982 bus                        /sierra_2: Setting Powered False
18:34:18.052828 bus                        /sierra_2: Setting Powered True
18:34:19.386505 bus                        /sierra_2: Setting Online True
18:34:21.903790 tst                        /sierra_2: Connect to ('901', '70')
18:34:22.195195 bus                        /sierra_3: Setting Online False
18:34:23.380896 bus                        /sierra_3: Setting Powered False
18:34:29.052113 bus                        /sierra_3: Setting Powered True
18:34:30.397256 bus                        /sierra_3: Setting Online True
18:34:32.940422 tst                        /sierra_3: Connect to ('901', '70')
18:34:33.112734 tst                        /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053'
18:34:33.233944 tst                        /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028'
18:34:33.347198 tst                  mo_mt_sms.py:35: waiting for modems to attach...
18:34:36.680742 tst                        /sierra_2: Already registered with network ('901', '70')
18:34:49.476641 tst                        /sierra_3: Already registered with network ('901', '70')
18:34:54.852412 tst                        /sierra_2: sending sms to MSISDN 3948 {sms='message nr. 1, from /sierra_2, to /sierra_3, from 3947, to 3948'}
18:35:01.327379 tst                        /sierra_3: Incoming SMS: 'message nr. 1, from /sierra_2, to /sierra_3, from 3947, to 3948'
18:35:01.971975 tst                        /sierra_3: SMS received as expected: 'message nr. 1, from /sierra_2, to /sierra_3, from 3947, to 3948'
18:35:02.137193 tst                     mo_mt_sms.py: Test passed (116.8 sec)
18:35:02.201277 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26646): Terminating (SIGTERM)
18:35:02.241347 run   osmo-hlr_10.42.42.2(pid=26647): Terminating (SIGTERM)
18:35:02.281198 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26648): Terminating (SIGTERM)
18:35:02.322983 run   osmo-stp_10.42.42.5(pid=26649): Terminating (SIGTERM)
18:35:02.371438 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=26650): Terminating (SIGTERM)
18:35:02.419434 run   osmo-msc_10.42.42.6(pid=26651): Terminating (SIGTERM)
18:35:02.465743 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26652): Terminating (SIGTERM)
18:35:02.511231 run   osmo-mgw_10.42.42.3(pid=26653): Terminating (SIGTERM)
18:35:02.562968 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26654): Terminating (SIGTERM)
18:35:02.613458 run   osmo-mgw_10.42.42.4(pid=26655): Terminating (SIGTERM)
18:35:02.667012 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=26656): Terminating (SIGTERM)
18:35:02.717952 run   osmo-bsc_10.42.42.7(pid=26657): Terminating (SIGTERM)
18:35:02.762008 run        osmo-bts-sysmo(pid=26666): Terminating (SIGTERM)
18:35:02.806993 ---      ParallelTerminationStrategy: PID 26646 died...
18:35:02.828899 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26646): Terminated: ok {rc=0}
18:35:02.876312 ---      ParallelTerminationStrategy: PID 26647 died...
18:35:02.896064 run   osmo-hlr_10.42.42.2(pid=26647): Terminated: ok {rc=0}
18:35:02.943192 ---      ParallelTerminationStrategy: PID 26648 died...
18:35:02.962919 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26648): Terminated: ok {rc=0}
18:35:03.007740 ---      ParallelTerminationStrategy: PID 26649 died...
18:35:03.026792 run   osmo-stp_10.42.42.5(pid=26649): Terminated: ok {rc=0}
18:35:03.071827 ---      ParallelTerminationStrategy: PID 26650 died...
18:35:03.090596 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=26650): Terminated: ok {rc=0}
18:35:03.135583 ---      ParallelTerminationStrategy: PID 26651 died...
18:35:03.154335 run   osmo-msc_10.42.42.6(pid=26651): Terminated: ok {rc=0}
18:35:03.198582 ---      ParallelTerminationStrategy: PID 26652 died...
18:35:03.217824 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26652): Terminated: ok {rc=0}
18:35:03.262429 ---      ParallelTerminationStrategy: PID 26653 died...
18:35:03.281149 run   osmo-mgw_10.42.42.3(pid=26653): Terminated {rc=15}
18:35:03.325432 ---      ParallelTerminationStrategy: PID 26654 died...
18:35:03.344203 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26654): Terminated: ok {rc=0}
18:35:03.388567 ---      ParallelTerminationStrategy: PID 26655 died...
18:35:03.407445 run   osmo-mgw_10.42.42.4(pid=26655): Terminated {rc=15}
18:35:03.451788 ---      ParallelTerminationStrategy: PID 26656 died...
18:35:03.470585 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=26656): Terminated: ok {rc=0}
18:35:03.515318 ---      ParallelTerminationStrategy: PID 26657 died...
18:35:03.534154 run   osmo-bsc_10.42.42.7(pid=26657): Terminated: ok {rc=0}
18:35:03.578489 ---      ParallelTerminationStrategy: PID 26666 died...
18:35:03.597386 run        osmo-bts-sysmo(pid=26666): Terminated: ok {rc=0}
18:35:03.724493 bus                        /sierra_3: Setting Online False
18:35:04.839909 bus                        /sierra_3: Setting Powered False
18:35:15.011295 bus                        /sierra_3: Setting Powered False
18:35:16.108714 bus                        /sierra_3: Setting Powered False
18:35:17.268015 bus                        /sierra_2: Setting Online False
18:35:18.419827 bus                        /sierra_2: Setting Powered False
18:35:26.915106 bus                        /sierra_2: Setting Powered False
---------------------------------------------------------------------
trial-2064 sms:sysmo PASS
---------------------------------------------------------------------

---------------------------------------------------------------------
trial-2064 gprs:sysmo
---------------------------------------------------------------------
18:35:28.182892 tst                       gprs:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ...
18:35:28.187551 tst                       gprs:sysmo: Verifying 1 x bts (candidates: 11)
18:35:28.209858 tst                       gprs:sysmo: Verifying 9 x ip_address (candidates: 9)
18:35:28.249976 tst                       gprs:sysmo: Verifying 4 x modem (candidates: 12)
18:35:28.337631 tst                       gprs:sysmo: Reserving 1 x bts (candidates: 11)
18:35:28.359258 tst                       gprs:sysmo: Reserving 9 x ip_address (candidates: 9)
18:35:28.399139 tst                       gprs:sysmo: Reserving 4 x modem (candidates: 12)
18:35:28.990568 tst                       gprs:sysmo: RESERVED RESOURCES for gprs:
bts
  sysmoBTS 1002
ip_address
  10.42.42.2
  10.42.42.3
  10.42.42.4
  10.42.42.5
  10.42.42.6
  10.42.42.7
  10.42.42.8
  10.42.42.9
  10.42.42.10
modem
  ec20_1
  ec20_2
  ec20_3
  ec20_4

----------------------------------------------
trial-2064 gprs:sysmo cs_paging_gprs_active.py
----------------------------------------------
18:35:29.053199 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.064283 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
18:35:29.100023 tst         cs_paging_gprs_active.py: using LAC 3945
18:35:29.228744 tst         cs_paging_gprs_active.py: using RAC 120
18:35:29.355187 tst         cs_paging_gprs_active.py: using CellId 3945
18:35:29.477550 tst         cs_paging_gprs_active.py: using BVCI 3946
18:35:29.588388 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.599040 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.609540 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.619984 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.630211 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.640539 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.651547 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.661876 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:35:29.672477 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:35:29.791298 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:35:29.911059 tst      cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003...
18:35:30.133666 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:35:30.465363 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Launched
18:35:30.756878 run iperf3-srv_10.42.42.10(pid=26682): Launched
18:35:30.880977 tst      cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004...
18:35:31.102852 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:35:31.432791 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Launched
18:35:31.718373 run iperf3-srv_10.42.42.10(pid=26684): Launched
18:35:31.842361 tst      cs_paging_gprs_active.py:38: start network...
18:35:32.063125 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
18:35:32.570667 run         create_hlr_db(pid=26685): Launched
18:35:32.696672 bus                        /sierra_2: Setting Powered False
18:35:33.761210 run         create_hlr_db(pid=26685): Terminated: ok {rc=0}
18:35:34.199900 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Launched
18:35:34.507072 run   osmo-hlr_10.42.42.2(pid=26688): Launched
18:35:34.623423 run              osmo-stp_10.42.42.5: Starting osmo-stp
18:35:35.147529 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Launched
18:35:35.453834 run   osmo-stp_10.42.42.5(pid=26690): Launched
18:35:35.570249 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:35:37.089080 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Launched
18:35:37.206677 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:35:37.470046 run              patchelf(pid=26692): Launched
18:35:38.645784 run              patchelf(pid=26692): Terminated: ok {rc=0}
18:35:38.774375 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:35:39.039678 run osmo-gsm-tester_setcap_net_admin.sh(pid=26693): Launched
18:35:40.220718 run osmo-gsm-tester_setcap_net_admin.sh(pid=26693): Terminated: ok {rc=0}
18:35:40.539550 run  osmo-ggsn_10.42.42.6(pid=26696): Launched
18:35:40.665713 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:35:45.549008 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Launched
18:35:45.855774 run  osmo-sgsn_10.42.42.7(pid=26716): Launched
18:35:45.973605 run              osmo-msc_10.42.42.8: Starting osmo-msc
18:35:46.576368 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Launched
18:35:46.906062 run   osmo-msc_10.42.42.8(pid=26718): Launched
18:35:50.078466 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
18:35:50.651513 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Launched
18:35:50.990474 run   osmo-mgw_10.42.42.3(pid=26720): Launched
18:35:51.117211 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
18:35:51.674573 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Launched
18:35:52.009110 run   osmo-mgw_10.42.42.4(pid=26722): Launched
18:35:52.135863 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
18:35:52.815800 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Launched
18:35:53.164683 run   osmo-bsc_10.42.42.9(pid=26724): Launched
18:35:59.457431 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:36:00.030105 run         rm-remote-dir(pid=26725): Launched
18:36:02.225781 run         rm-remote-dir(pid=26725): Terminated: ok {rc=0}
18:36:02.587660 run         mk-remote-dir(pid=26726): Launched
18:36:03.782132 run         mk-remote-dir(pid=26726): Terminated: ok {rc=0}
18:36:04.091239 run    scp-inst-to-remote(pid=26727): Launched
18:36:16.299433 run    scp-inst-to-remote(pid=26727): Terminated: ok {rc=0}
18:36:16.632145 run         mk-remote-dir(pid=26729): Launched
18:36:17.811202 run         mk-remote-dir(pid=26729): Terminated: ok {rc=0}
18:36:18.099043 run     scp-cfg-to-remote(pid=26730): Launched
18:36:19.278230 run     scp-cfg-to-remote(pid=26730): Terminated: ok {rc=0}
18:36:19.607837 run   reload-dsp-firmware(pid=26732): Launched
18:36:21.785104 run   reload-dsp-firmware(pid=26732): Terminated: ok {rc=0}
18:36:22.107676 run        osmo-bts-sysmo(pid=26733): Launched
18:36:26.335161 tst      cs_paging_gprs_active.py:38: Waiting for bts to be ready...
18:36:28.358345 run         rm-remote-dir(pid=26735): Launched
18:36:30.544612 run         rm-remote-dir(pid=26735): Terminated: ok {rc=0}
18:36:30.892086 run         mk-remote-dir(pid=26736): Launched
18:36:33.077977 run         mk-remote-dir(pid=26736): Terminated: ok {rc=0}
18:36:33.378680 run       scp-inst-to-bts(pid=26737): Launched
18:36:46.586554 run       scp-inst-to-bts(pid=26737): Terminated: ok {rc=0}
18:36:46.933894 run     mk-remote-run-dir(pid=26739): Launched
18:36:49.117396 run     mk-remote-run-dir(pid=26739): Terminated: ok {rc=0}
18:36:49.396757 run   scp-cfg-to-sysmobts(pid=26740): Launched
18:36:51.572576 run   scp-cfg-to-sysmobts(pid=26740): Terminated: ok {rc=0}
18:36:51.898339 run              osmo-pcu(pid=26742): Launched
18:36:52.026297 tst         cs_paging_gprs_active.py: using MSISDN 3949
18:36:52.202811 bus                          /gobi_6: Setting Powered True
18:36:55.699584 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3949', subscriber_id=1}
18:36:55.975599 bus                          /gobi_6: Setting Online False
18:36:57.163713 bus                          /gobi_6: Setting Powered False
18:37:04.112610 bus                          /gobi_6: Setting Powered True
18:37:05.505700 bus                          /gobi_6: Setting Online True
18:37:07.931860 tst                          /gobi_6: Connect to ('901', '70')
18:37:08.219441 tst                          /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G  1  [Jul  5 2017 10:49:17]', Serial: '868323024273648'
18:37:08.345572 tst         cs_paging_gprs_active.py: using MSISDN 3950
18:37:08.619084 bus                          /gobi_0: Setting Powered True
18:37:12.112074 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3950', subscriber_id=2}
18:37:12.385508 bus                          /gobi_0: Setting Online False
18:37:13.579550 bus                          /gobi_0: Setting Powered False
18:37:20.323005 bus                          /gobi_0: Setting Powered True
18:37:21.694558 bus                          /gobi_0: Setting Online True
18:37:23.400941 tst                          /gobi_6: Already registered with network ('901', '70')
18:37:24.203602 tst                          /gobi_0: Connect to ('901', '70')
18:37:24.480896 tst                          /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G  1  [Mar 14 2016 19:33:01]', Serial: '868323024271873'
18:37:24.596348 tst      cs_paging_gprs_active.py:38: waiting for modems to attach...
18:37:38.011549 tst                          /gobi_0: Already registered with network ('901', '70')
18:37:43.128601 tst      cs_paging_gprs_active.py:38: waiting for modems to attach to data services...
18:37:46.281134 tst                          /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:37:46.398467 tst      cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_6/context2'"
18:37:46.842011 run            move_netns(pid=26746): Launched
18:37:48.040448 run            move_netns(pid=26746): Terminated: ok {rc=0}
18:37:48.370869 run                  ifup(pid=26749): Launched
18:37:49.548570 run                  ifup(pid=26749): Terminated: ok {rc=0}
18:37:49.879871 run                  dhcp(pid=26752): Launched
18:37:51.056449 run                  dhcp(pid=26752): Terminated: ok {rc=0}
18:37:59.436372 tst                          /gobi_0: context activated {apn='inet46', path='/gobi_0/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:37:59.553538 tst      cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_0/context3'"
18:37:59.998060 run            move_netns(pid=26770): Launched
18:38:01.184894 run            move_netns(pid=26770): Terminated: ok {rc=0}
18:38:01.535692 run                  ifup(pid=26773): Launched
18:38:02.723339 run                  ifup(pid=26773): Terminated: ok {rc=0}
18:38:03.073258 run                  dhcp(pid=26776): Launched
18:38:04.257763 run                  dhcp(pid=26776): Terminated: ok {rc=0}
18:38:04.388039 tst      cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'"
18:38:04.602998 run           iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
18:38:04.972753 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Launched
18:38:05.168986 tst      cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context3'"
18:38:05.393125 run           iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s)
18:38:05.781897 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Launched
18:38:06.120090 run iperf3-cli_10.42.42.10(pid=26798): Launched
18:38:06.405457 run iperf3-cli_10.42.42.10(pid=26801): Launched
18:38:06.534218 tst      cs_paging_gprs_active.py:13: waiting a few secs to make sure iperf3 test is running
18:38:08.773866 tst      cs_paging_gprs_active.py:16: iperf3 running, let's place a call
18:38:13.491195 tst      cs_paging_gprs_active.py:22: dial success
18:38:16.313230 tst      cs_paging_gprs_active.py:28: answer success, call established and ongoing
18:38:22.970525 tst      cs_paging_gprs_active.py:35: hangup success
18:39:15.327878 run iperf3-cli_10.42.42.10(pid=26798): ERR: Terminated: ERROR {rc=1}  [trial-2064↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=26798)]
18:39:15.459964 run iperf3-cli_10.42.42.10(pid=26798): stdout: 
| 				"bits_per_second":	0,
| 				"retransmits":	0,
| 				"omitted":	false
| 			}
| 		}],
| 	"end":	{
| 	},
| 	"error":	"error - control socket has closed unexpectedly"
| }
| child exited with 1 
18:39:15.592854 run iperf3-cli_10.42.42.10(pid=26798): stderr: 
| (launched: 2022-03-29_18:38:06.104023) 
18:39:16.781776 run iperf3-cli_10.42.42.10(pid=26801): ERR: Terminated: ERROR {rc=1}  [trial-2064↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=26801)]
18:39:16.916973 run iperf3-cli_10.42.42.10(pid=26801): stdout: 
| 				"bits_per_second":	0,
| 				"retransmits":	0,
| 				"omitted":	false
| 			}
| 		}],
| 	"end":	{
| 	},
| 	"error":	"error - control socket has closed unexpectedly"
| }
| child exited with 1 
18:39:17.047789 run iperf3-cli_10.42.42.10(pid=26801): stderr: 
| (launched: 2022-03-29_18:38:06.389480) 
18:39:17.180135 run iperf3-srv_10.42.42.10(pid=26682): Terminating (SIGINT)
18:39:17.516776 run iperf3-srv_10.42.42.10(pid=26682): Terminated {rc=1}
18:39:17.638473 tst      cs_paging_gprs_active.py:38: Results for 10.42.42.10:5003 through "'/gobi_6/context2'"
Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42828, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:38:07 GMT', 'timesecs': 1648571887}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'eqam3drdr7ify2k6fvilb6zyzmko7t7hyhnl', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0011892318725586, 'seconds': 1.0011892318725586, 'bytes': 101360, 'bits_per_second': 809916.8211022239, 'retransmits': 4, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0011892318725586, 'seconds': 1.0011892318725586, 'bytes': 101360, 'bits_per_second': 809916.8211022239, 'retransmits': 4, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0011892318725586, 'end': 2.001164197921753, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0011892318725586, 'end': 2.001164197921753, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.001164197921753, 'end': 3.001214027404785, 'seconds': 1.0000498294830322, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.001164197921753, 'end': 3.001214027404785, 'seconds': 1.0000498294830322, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.001214027404785, 'end': 4.00121808052063, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.001214027404785, 'end': 4.00121808052063, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.00121808052063, 'end': 5.001165151596069, 'seconds': 0.9999470710754395, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.00121808052063, 'end': 5.001165151596069, 'seconds': 0.9999470710754395, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001165151596069, 'end': 6.001265048980713, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001165151596069, 'end': 6.001265048980713, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001265048980713, 'end': 7.001306056976318, 'seconds': 1.0000410079956055, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001265048980713, 'end': 7.001306056976318, 'seconds': 1.0000410079956055, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001306056976318, 'end': 8.001264095306396, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001306056976318, 'end': 8.001264095306396, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001264095306396, 'end': 9.001306056976318, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001264095306396, 'end': 9.001306056976318, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001306056976318, 'end': 10.001192092895508, 'seconds': 0.9998860359191895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 2176035, 'rttvar': 3694344, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001306056976318, 'end': 10.001192092895508, 'seconds': 0.9998860359191895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001192092895508, 'end': 11.001074075698853, 'seconds': 0.9998819828033447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 14480, 'rtt': 3171415, 'rttvar': 4761518, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001192092895508, 'end': 11.001074075698853, 'seconds': 0.9998819828033447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001074075698853, 'end': 12.00110912322998, 'seconds': 1.000035047531128, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 4160072, 'rttvar': 5548452, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001074075698853, 'end': 12.00110912322998, 'seconds': 1.000035047531128, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.00110912322998, 'end': 13.001227140426636, 'seconds': 1.0001180171966553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 5177411, 'rttvar': 6196018, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.00110912322998, 'end': 13.001227140426636, 'seconds': 1.0001180171966553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001227140426636, 'end': 14.001306056976318, 'seconds': 1.0000789165496826, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 13032, 'rtt': 7130959, 'rttvar': 6897406, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001227140426636, 'end': 14.001306056976318, 'seconds': 1.0000789165496826, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001306056976318, 'end': 15.001320123672485, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 13032, 'rtt': 8059624, 'rttvar': 7030385, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001306056976318, 'end': 15.001320123672485, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001320123672485, 'end': 16.001228094100952, 'seconds': 0.9999079704284668, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 11584, 'rtt': 8959227, 'rttvar': 7071995, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001320123672485, 'end': 16.001228094100952, 'seconds': 0.9999079704284668, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001228094100952, 'end': 17.001160144805908, 'seconds': 0.999932050704956, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 10136, 'rtt': 10696389, 'rttvar': 7012272, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001228094100952, 'end': 17.001160144805908, 'seconds': 0.999932050704956, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001160144805908, 'end': 18.001296043395996, 'seconds': 1.000135898590088, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 10136, 'rtt': 10967200, 'rttvar': 5800825, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001160144805908, 'end': 18.001296043395996, 'seconds': 1.000135898590088, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001296043395996, 'end': 19.001232147216797, 'seconds': 0.9999361038208008, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 11584, 'rtt': 10210576, 'rttvar': 4576076, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001296043395996, 'end': 19.001232147216797, 'seconds': 0.9999361038208008, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.001232147216797, 'end': 20.00131916999817, 'seconds': 1.000087022781372, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 11584, 'rtt': 9834182, 'rttvar': 4184844, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.001232147216797, 'end': 20.00131916999817, 'seconds': 1.000087022781372, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.00131916999817, 'end': 21.001143217086792, 'seconds': 0.999824047088623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 13032, 'rtt': 9021949, 'rttvar': 3761266, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.00131916999817, 'end': 21.001143217086792, 'seconds': 0.999824047088623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001143217086792, 'end': 22.001182079315186, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 8512041, 'rttvar': 3771203, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001143217086792, 'end': 22.001182079315186, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001182079315186, 'end': 23.00033211708069, 'seconds': 0.9991500377655029, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 8065902, 'rttvar': 3720679, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001182079315186, 'end': 23.00033211708069, 'seconds': 0.9991500377655029, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.00033211708069, 'end': 24.00107502937317, 'seconds': 1.0007429122924805, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 7328549, 'rttvar': 3370981, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.00033211708069, 'end': 24.00107502937317, 'seconds': 1.0007429122924805, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.00107502937317, 'end': 25.001262187957764, 'seconds': 1.0001871585845947, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 7120323, 'rttvar': 2944687, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.00107502937317, 'end': 25.001262187957764, 'seconds': 1.0001871585845947, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001262187957764, 'end': 26.001070022583008, 'seconds': 0.9998078346252441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 6777745, 'rttvar': 2251565, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001262187957764, 'end': 26.001070022583008, 'seconds': 0.9998078346252441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.001070022583008, 'end': 27.00107502937317, 'seconds': 1.0000050067901611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 6723145, 'rttvar': 1797874, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.001070022583008, 'end': 27.00107502937317, 'seconds': 1.0000050067901611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00107502937317, 'end': 28.00122618675232, 'seconds': 1.0001511573791504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 6700290, 'rttvar': 1394116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00107502937317, 'end': 28.00122618675232, 'seconds': 1.0001511573791504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.00122618675232, 'end': 29.00107717514038, 'seconds': 0.9998509883880615, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 6699805, 'rttvar': 1046556, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.00122618675232, 'end': 29.00107717514038, 'seconds': 0.9998509883880615, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.00107717514038, 'end': 30.00109314918518, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 6875469, 'rttvar': 893771, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.00107717514038, 'end': 30.00109314918518, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.00109314918518, 'end': 31.001081228256226, 'seconds': 0.9999880790710449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 6951140, 'rttvar': 821669, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.00109314918518, 'end': 31.001081228256226, 'seconds': 0.9999880790710449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.001081228256226, 'end': 32.00107407569885, 'seconds': 0.999992847442627, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 7249288, 'rttvar': 980299, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.001081228256226, 'end': 32.00107407569885, 'seconds': 0.999992847442627, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00107407569885, 'end': 33.0010941028595, 'seconds': 1.0000200271606445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 7367546, 'rttvar': 971741, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00107407569885, 'end': 33.0010941028595, 'seconds': 1.0000200271606445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.0010941028595, 'end': 34.0011100769043, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 7556482, 'rttvar': 1106678, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.0010941028595, 'end': 34.0011100769043, 'seconds': 1.0000159740447998, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.0011100769043, 'end': 35.00132417678833, 'seconds': 1.0002140998840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 7861612, 'rttvar': 1151562, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.0011100769043, 'end': 35.00132417678833, 'seconds': 1.0002140998840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00132417678833, 'end': 36.00116300582886, 'seconds': 0.9998388290405273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 8074202, 'rttvar': 1288852, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00132417678833, 'end': 36.00116300582886, 'seconds': 0.9998388290405273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00116300582886, 'end': 37.00122618675232, 'seconds': 1.000063180923462, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 8397018, 'rttvar': 1277904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00116300582886, 'end': 37.00122618675232, 'seconds': 1.000063180923462, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00122618675232, 'end': 38.00110912322998, 'seconds': 0.9998829364776611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8604892, 'rttvar': 1374175, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00122618675232, 'end': 38.00110912322998, 'seconds': 0.9998829364776611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00110912322998, 'end': 39.0010712146759, 'seconds': 0.9999620914459229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 8903011, 'rttvar': 1288297, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00110912322998, 'end': 39.0010712146759, 'seconds': 0.9999620914459229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0010712146759, 'end': 40.001259088516235, 'seconds': 1.000187873840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9122079, 'rttvar': 1404358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0010712146759, 'end': 40.001259088516235, 'seconds': 1.000187873840332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.001259088516235, 'end': 41.001121044158936, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 9311519, 'rttvar': 1432149, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.001259088516235, 'end': 41.001121044158936, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.001121044158936, 'end': 42.0011100769043, 'seconds': 0.9999890327453613, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 9700066, 'rttvar': 1502442, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.001121044158936, 'end': 42.0011100769043, 'seconds': 0.9999890327453613, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.0011100769043, 'end': 43.000755071640015, 'seconds': 0.9996449947357178, 'bytes': 63712, 'bits_per_second': 509877.0090223394, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 9899537, 'rttvar': 1525774, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.0011100769043, 'end': 43.000755071640015, 'seconds': 0.9996449947357178, 'bytes': 63712, 'bits_per_second': 509877.0090223394, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.000755071640015, 'end': 44.00115919113159, 'seconds': 1.0004041194915771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 10307435, 'rttvar': 1589286, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.000755071640015, 'end': 44.00115919113159, 'seconds': 1.0004041194915771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00115919113159, 'end': 45.00115919113159, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 10514235, 'rttvar': 1605563, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00115919113159, 'end': 45.00115919113159, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00115919113159, 'end': 46.00112318992615, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 10696819, 'rttvar': 1569342, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00115919113159, 'end': 46.00112318992615, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00112318992615, 'end': 47.00115919113159, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 11161544, 'rttvar': 1688485, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00112318992615, 'end': 47.00115919113159, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}"
18:39:17.862752 tst      cs_paging_gprs_active.py:38: ERR: KeyError: 'sum_sent'  [trial-2064↪gprs:sysmo↪cs_paging_gprs_active.py:38]
18:39:17.875668 tst      cs_paging_gprs_active.py:38: Test FAILED (228.9 sec)
18:39:17.883131 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Terminating (SIGTERM)
18:39:17.928872 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Terminating (SIGTERM)
18:39:17.972357 run iperf3-srv_10.42.42.10(pid=26684): Terminating (SIGTERM)
18:39:18.015494 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Terminating (SIGTERM)
18:39:18.058581 run   osmo-hlr_10.42.42.2(pid=26688): Terminating (SIGTERM)
18:39:18.102081 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Terminating (SIGTERM)
18:39:18.147511 run   osmo-stp_10.42.42.5(pid=26690): Terminating (SIGTERM)
18:39:18.198400 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Terminating (SIGTERM)
18:39:18.246865 run  osmo-ggsn_10.42.42.6(pid=26696): Terminating (SIGTERM)
18:39:18.295120 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Terminating (SIGTERM)
18:39:18.348084 run  osmo-sgsn_10.42.42.7(pid=26716): Terminating (SIGTERM)
18:39:18.400548 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Terminating (SIGTERM)
18:39:18.471460 run   osmo-msc_10.42.42.8(pid=26718): Terminating (SIGTERM)
18:39:18.541263 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Terminating (SIGTERM)
18:39:18.586758 run   osmo-mgw_10.42.42.3(pid=26720): Terminating (SIGTERM)
18:39:18.633718 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Terminating (SIGTERM)
18:39:18.678200 run   osmo-mgw_10.42.42.4(pid=26722): Terminating (SIGTERM)
18:39:18.721873 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Terminating (SIGTERM)
18:39:18.765687 run   osmo-bsc_10.42.42.9(pid=26724): Terminating (SIGTERM)
18:39:18.808811 run        osmo-bts-sysmo(pid=26733): Terminating (SIGTERM)
18:39:18.852191 run              osmo-pcu(pid=26742): Terminating (SIGTERM)
18:39:18.896577 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGTERM)
18:39:19.041243 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGTERM(pid=26820): Launched
18:39:20.147122 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGTERM(pid=26820): Terminated: ok {rc=0}
18:39:20.213485 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGTERM)
18:39:20.357004 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGTERM(pid=26825): Launched
18:39:21.450010 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGTERM(pid=26825): Terminated: ok {rc=0}
18:39:21.516329 ---      ParallelTerminationStrategy: PID 26681 died...
18:39:21.536410 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26681): Terminated: ok {rc=0}
18:39:21.583475 ---      ParallelTerminationStrategy: PID 26683 died...
18:39:21.603319 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26683): Terminated: ok {rc=0}
18:39:21.655678 ---      ParallelTerminationStrategy: PID 26684 died...
18:39:21.675303 run iperf3-srv_10.42.42.10(pid=26684): Terminated {rc=256}
18:39:21.722589 ---      ParallelTerminationStrategy: PID 26687 died...
18:39:21.743029 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26687): Terminated: ok {rc=0}
18:39:21.789911 ---      ParallelTerminationStrategy: PID 26688 died...
18:39:21.809718 run   osmo-hlr_10.42.42.2(pid=26688): Terminated: ok {rc=0}
18:39:21.856930 ---      ParallelTerminationStrategy: PID 26689 died...
18:39:21.876757 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26689): Terminated: ok {rc=0}
18:39:21.923958 ---      ParallelTerminationStrategy: PID 26690 died...
18:39:21.944181 run   osmo-stp_10.42.42.5(pid=26690): Terminated: ok {rc=0}
18:39:21.990647 ---      ParallelTerminationStrategy: PID 26691 died...
18:39:22.010317 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26691): Terminated: ok {rc=0}
18:39:22.056924 ---      ParallelTerminationStrategy: PID 26696 died...
18:39:22.077177 run  osmo-ggsn_10.42.42.6(pid=26696): Terminated: ok {rc=0}
18:39:22.124009 ---      ParallelTerminationStrategy: PID 26715 died...
18:39:22.143710 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26715): Terminated: ok {rc=0}
18:39:22.190268 ---      ParallelTerminationStrategy: PID 26716 died...
18:39:22.209909 run  osmo-sgsn_10.42.42.7(pid=26716): Terminated: ok {rc=0}
18:39:22.256464 ---      ParallelTerminationStrategy: PID 26717 died...
18:39:22.276078 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26717): Terminated: ok {rc=0}
18:39:22.322820 ---      ParallelTerminationStrategy: PID 26718 died...
18:39:22.342421 run   osmo-msc_10.42.42.8(pid=26718): Terminated: ok {rc=0}
18:39:22.389583 ---      ParallelTerminationStrategy: PID 26719 died...
18:39:22.409411 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26719): Terminated: ok {rc=0}
18:39:22.456512 ---      ParallelTerminationStrategy: PID 26720 died...
18:39:22.476280 run   osmo-mgw_10.42.42.3(pid=26720): Terminated {rc=15}
18:39:22.523405 ---      ParallelTerminationStrategy: PID 26721 died...
18:39:22.543318 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26721): Terminated: ok {rc=0}
18:39:22.590518 ---      ParallelTerminationStrategy: PID 26722 died...
18:39:22.610815 run   osmo-mgw_10.42.42.4(pid=26722): Terminated {rc=15}
18:39:22.657754 ---      ParallelTerminationStrategy: PID 26723 died...
18:39:22.677601 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26723): Terminated: ok {rc=0}
18:39:22.724123 ---      ParallelTerminationStrategy: PID 26724 died...
18:39:22.744086 run   osmo-bsc_10.42.42.9(pid=26724): Terminated: ok {rc=0}
18:39:22.790849 ---      ParallelTerminationStrategy: PID 26733 died...
18:39:22.810783 run        osmo-bts-sysmo(pid=26733): Terminated: ok {rc=0}
18:39:22.857900 ---      ParallelTerminationStrategy: PID 26742 died...
18:39:22.878159 run              osmo-pcu(pid=26742): Terminated: ok {rc=0}
18:39:28.747801 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGINT)
18:39:28.895566 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGINT(pid=26828): Launched
18:39:29.981810 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGINT(pid=26828): Terminated: ok {rc=0}
18:39:30.047964 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGINT)
18:39:30.190153 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGINT(pid=26831): Launched
18:39:31.279904 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGINT(pid=26831): Terminated: ok {rc=0}
18:39:37.169022 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792): Terminating (SIGKILL)
18:39:37.317978 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGUSR1(pid=26834): Launched
18:39:38.402893 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26792)-killSignals.SIGUSR1(pid=26834): Terminated: ok {rc=0}
18:39:38.468708 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795): Terminating (SIGKILL)
18:39:38.613806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGUSR1(pid=26837): Launched
18:39:39.699613 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26795)-killSignals.SIGUSR1(pid=26837): Terminated: ok {rc=0}
18:39:39.952782 bus                          /gobi_0: Setting Online False
18:39:41.109901 bus                          /gobi_0: Setting Powered False
18:39:57.933861 bus                          /gobi_0: Setting Powered False
18:39:59.031683 bus                          /gobi_0: Setting Powered False
18:40:00.187625 bus                          /gobi_6: Setting Online False
18:40:01.324454 bus                          /gobi_6: Setting Powered False
18:40:09.937679 bus                          /gobi_6: Setting Powered False

----------------------------------------------
trial-2064 gprs:sysmo iperf3.py
----------------------------------------------
18:40:11.133475 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.144585 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
18:40:11.179606 tst                        iperf3.py: using LAC 3946
18:40:11.296486 tst                        iperf3.py: using RAC 121
18:40:11.417856 tst                        iperf3.py: using CellId 3946
18:40:11.534562 tst                        iperf3.py: using BVCI 3947
18:40:11.652136 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.663475 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.674472 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.685761 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.696994 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.708244 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.719346 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.730187 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:40:11.741327 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:40:11.859782 tst                      iperf3.py:8: start iperfv3 server 10.42.42.10:5003...
18:40:12.076074 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:40:12.401713 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Launched
18:40:12.684914 run iperf3-srv_10.42.42.10(pid=26844): Launched
18:40:12.809071 tst                      iperf3.py:8: start network...
18:40:13.028324 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
18:40:13.531050 run         create_hlr_db(pid=26845): Launched
18:40:13.655300 bus                          /gobi_6: Setting Powered False
18:40:14.719347 run         create_hlr_db(pid=26845): Terminated: ok {rc=0}
18:40:15.171589 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Launched
18:40:15.495601 run   osmo-hlr_10.42.42.2(pid=26848): Launched
18:40:15.620203 run              osmo-stp_10.42.42.5: Starting osmo-stp
18:40:16.173360 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Launched
18:40:16.499230 run   osmo-stp_10.42.42.5(pid=26850): Launched
18:40:16.622825 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:40:17.179654 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Launched
18:40:17.304391 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:40:17.580170 run              patchelf(pid=26852): Launched
18:40:18.763309 run              patchelf(pid=26852): Terminated: ok {rc=0}
18:40:18.900791 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:40:19.182582 run osmo-gsm-tester_setcap_net_admin.sh(pid=26853): Launched
18:40:20.368115 run osmo-gsm-tester_setcap_net_admin.sh(pid=26853): Terminated: ok {rc=0}
18:40:20.708483 run  osmo-ggsn_10.42.42.6(pid=26856): Launched
18:40:20.833195 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:40:21.546002 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Launched
18:40:21.871360 run  osmo-sgsn_10.42.42.7(pid=26880): Launched
18:40:21.994607 run              osmo-msc_10.42.42.8: Starting osmo-msc
18:40:22.580961 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Launched
18:40:22.905293 run   osmo-msc_10.42.42.8(pid=26882): Launched
18:40:26.086848 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
18:40:26.639561 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Launched
18:40:26.965928 run   osmo-mgw_10.42.42.3(pid=26884): Launched
18:40:27.091891 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
18:40:27.641697 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Launched
18:40:27.970361 run   osmo-mgw_10.42.42.4(pid=26886): Launched
18:40:28.095899 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
18:40:28.771906 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Launched
18:40:29.102019 run   osmo-bsc_10.42.42.9(pid=26888): Launched
18:40:35.392301 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:40:35.962825 run         rm-remote-dir(pid=26889): Launched
18:40:38.159596 run         rm-remote-dir(pid=26889): Terminated: ok {rc=0}
18:40:38.519994 run         mk-remote-dir(pid=26890): Launched
18:40:39.711928 run         mk-remote-dir(pid=26890): Terminated: ok {rc=0}
18:40:40.019352 run    scp-inst-to-remote(pid=26891): Launched
18:40:52.225207 run    scp-inst-to-remote(pid=26891): Terminated: ok {rc=0}
18:40:52.557665 run         mk-remote-dir(pid=26893): Launched
18:40:53.735584 run         mk-remote-dir(pid=26893): Terminated: ok {rc=0}
18:40:54.021235 run     scp-cfg-to-remote(pid=26894): Launched
18:40:55.199682 run     scp-cfg-to-remote(pid=26894): Terminated: ok {rc=0}
18:40:55.528849 run   reload-dsp-firmware(pid=26896): Launched
18:40:57.706484 run   reload-dsp-firmware(pid=26896): Terminated: ok {rc=0}
18:40:58.028650 run        osmo-bts-sysmo(pid=26897): Launched
18:41:02.260450 tst                      iperf3.py:8: Waiting for bts to be ready...
18:41:03.982569 run         rm-remote-dir(pid=26898): Launched
18:41:06.164315 run         rm-remote-dir(pid=26898): Terminated: ok {rc=0}
18:41:06.487881 run         mk-remote-dir(pid=26899): Launched
18:41:08.658824 run         mk-remote-dir(pid=26899): Terminated: ok {rc=0}
18:41:08.936153 run       scp-inst-to-bts(pid=26900): Launched
18:41:22.123036 run       scp-inst-to-bts(pid=26900): Terminated: ok {rc=0}
18:41:22.446638 run     mk-remote-run-dir(pid=26902): Launched
18:41:24.623632 run     mk-remote-run-dir(pid=26902): Terminated: ok {rc=0}
18:41:24.920198 run   scp-cfg-to-sysmobts(pid=26903): Launched
18:41:27.107023 run   scp-cfg-to-sysmobts(pid=26903): Terminated: ok {rc=0}
18:41:27.452717 run              osmo-pcu(pid=26905): Launched
18:41:27.588774 tst                        iperf3.py: using MSISDN 3951
18:41:27.764385 bus                          /gobi_6: Setting Powered True
18:41:31.283264 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3951', subscriber_id=1}
18:41:31.558641 bus                          /gobi_6: Setting Online False
18:41:32.751208 bus                          /gobi_6: Setting Powered False
18:41:40.133664 bus                          /gobi_6: Setting Powered True
18:41:41.535326 bus                          /gobi_6: Setting Online True
18:41:44.084922 tst                          /gobi_6: Connect to ('901', '70')
18:41:44.346316 tst                          /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G  1  [Jul  5 2017 10:49:17]', Serial: '868323024273648'
18:41:44.455164 tst                      iperf3.py:8: waiting for modems to attach...
18:41:49.961316 tst                          /gobi_6: Already registered with network ('901', '70')
18:42:01.906359 tst                      iperf3.py:8: waiting for modems to attach to data services...
18:42:06.570252 tst                          /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:42:06.688943 tst                      iperf3.py:8: Setting up data plan for "'/gobi_6/context3'"
18:42:07.139193 run            move_netns(pid=26909): Launched
18:42:08.333756 run            move_netns(pid=26909): Terminated: ok {rc=0}
18:42:08.696771 run                  ifup(pid=26912): Launched
18:42:09.889153 run                  ifup(pid=26912): Terminated: ok {rc=0}
18:42:10.245155 run                  dhcp(pid=26915): Launched
18:42:11.427619 run                  dhcp(pid=26915): Terminated: ok {rc=0}
18:42:11.556435 tst                      iperf3.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context3'"
18:42:11.766122 run           iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
18:42:12.130786 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Launched
18:42:12.455587 run iperf3-cli_10.42.42.10(pid=26934): Launched
18:43:26.776963 run iperf3-cli_10.42.42.10(pid=26934): ERR: Terminated: ERROR {rc=1}  [trial-2064↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=26934)]
18:43:26.915744 run iperf3-cli_10.42.42.10(pid=26934): stdout: 
| 				"bits_per_second":	0,
| 				"retransmits":	0,
| 				"omitted":	false
| 			}
| 		}],
| 	"end":	{
| 	},
| 	"error":	"error - control socket has closed unexpectedly"
| }
| child exited with 1 
18:43:27.057591 run iperf3-cli_10.42.42.10(pid=26934): stderr: 
| (launched: 2022-03-29_18:42:12.439797) 
18:43:27.196695 run iperf3-srv_10.42.42.10(pid=26844): Terminating (SIGINT)
18:43:27.541012 run iperf3-srv_10.42.42.10(pid=26844): Terminated {rc=1}
18:43:27.667529 tst                      iperf3.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context3'"
Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42866, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:42:19 GMT', 'timesecs': 1648572139}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'vfzcc6k7krnaudu2qhmwk43mkmmn3xd5enfe', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0008230209350586, 'seconds': 1.0008230209350586, 'bytes': 101360, 'bits_per_second': 810213.1775929807, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 416760, 'rttvar': 179188, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0008230209350586, 'seconds': 1.0008230209350586, 'bytes': 101360, 'bits_per_second': 810213.1775929807, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0008230209350586, 'end': 2.001343011856079, 'seconds': 1.0005199909210205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 504319, 'rttvar': 309509, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0008230209350586, 'end': 2.001343011856079, 'seconds': 1.0005199909210205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.001343011856079, 'end': 3.0013041496276855, 'seconds': 0.9999611377716064, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 18824, 'rtt': 887851, 'rttvar': 861683, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.001343011856079, 'end': 3.0013041496276855, 'seconds': 0.9999611377716064, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0013041496276855, 'end': 4.0011889934539795, 'seconds': 0.999884843826294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1169081, 'rttvar': 1208723, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0013041496276855, 'end': 4.0011889934539795, 'seconds': 0.999884843826294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.0011889934539795, 'end': 5.001459121704102, 'seconds': 1.000270128250122, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 1502871, 'rttvar': 1574122, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.0011889934539795, 'end': 5.001459121704102, 'seconds': 1.000270128250122, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001459121704102, 'end': 6.001329183578491, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 2303976, 'rttvar': 2296626, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001459121704102, 'end': 6.001329183578491, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001329183578491, 'end': 7.001317024230957, 'seconds': 0.9999878406524658, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2758356, 'rttvar': 2631231, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001329183578491, 'end': 7.001317024230957, 'seconds': 0.9999878406524658, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001317024230957, 'end': 8.001265048980713, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 3754779, 'rttvar': 3230560, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001317024230957, 'end': 8.001265048980713, 'seconds': 0.9999480247497559, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001265048980713, 'end': 9.001326084136963, 'seconds': 1.00006103515625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 4181887, 'rttvar': 3277135, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001265048980713, 'end': 9.001326084136963, 'seconds': 1.00006103515625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001326084136963, 'end': 10.001069068908691, 'seconds': 0.9997429847717285, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 4614394, 'rttvar': 3322867, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001326084136963, 'end': 10.001069068908691, 'seconds': 0.9997429847717285, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001069068908691, 'end': 11.001177072525024, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 5090389, 'rttvar': 3444140, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001069068908691, 'end': 11.001177072525024, 'seconds': 1.000108003616333, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001177072525024, 'end': 12.001214981079102, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 5594052, 'rttvar': 3590431, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001177072525024, 'end': 12.001214981079102, 'seconds': 1.0000379085540771, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001214981079102, 'end': 13.001265048980713, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 6514479, 'rttvar': 3639030, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001214981079102, 'end': 13.001265048980713, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001265048980713, 'end': 14.001188039779663, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 6935277, 'rttvar': 3570868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001265048980713, 'end': 14.001188039779663, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001188039779663, 'end': 15.001317024230957, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 7784605, 'rttvar': 3480777, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001188039779663, 'end': 15.001317024230957, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001317024230957, 'end': 16.001317977905273, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 8221413, 'rttvar': 3484198, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001317024230957, 'end': 16.001317977905273, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001317977905273, 'end': 17.00117516517639, 'seconds': 0.9998571872711182, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9025140, 'rttvar': 3376202, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001317977905273, 'end': 17.00117516517639, 'seconds': 0.9998571872711182, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.00117516517639, 'end': 18.00126600265503, 'seconds': 1.0000908374786377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9394544, 'rttvar': 3270958, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.00117516517639, 'end': 18.00126600265503, 'seconds': 1.0000908374786377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.00126600265503, 'end': 19.00116515159607, 'seconds': 0.99989914894104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 9802550, 'rttvar': 3269232, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.00126600265503, 'end': 19.00116515159607, 'seconds': 0.99989914894104, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.00116515159607, 'end': 20.001224040985107, 'seconds': 1.000058889389038, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 10159568, 'rttvar': 3165960, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.00116515159607, 'end': 20.001224040985107, 'seconds': 1.000058889389038, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001224040985107, 'end': 21.001187086105347, 'seconds': 0.9999630451202393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 10992522, 'rttvar': 3229428, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001224040985107, 'end': 21.001187086105347, 'seconds': 0.9999630451202393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.001187086105347, 'end': 22.00130820274353, 'seconds': 1.0001211166381836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 11438503, 'rttvar': 3314034, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.001187086105347, 'end': 22.00130820274353, 'seconds': 1.0001211166381836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.00130820274353, 'end': 23.001317024230957, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11833439, 'rttvar': 3275397, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.00130820274353, 'end': 23.001317024230957, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001317024230957, 'end': 24.001163005828857, 'seconds': 0.9998459815979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 12276390, 'rttvar': 3342449, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001317024230957, 'end': 24.001163005828857, 'seconds': 0.9998459815979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001163005828857, 'end': 25.001341104507446, 'seconds': 1.0001780986785889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 13150739, 'rttvar': 3424898, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001163005828857, 'end': 25.001341104507446, 'seconds': 1.0001780986785889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001341104507446, 'end': 26.001182079315186, 'seconds': 0.9998409748077393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 13558774, 'rttvar': 3384745, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001341104507446, 'end': 26.001182079315186, 'seconds': 0.9998409748077393, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.001182079315186, 'end': 27.000904083251953, 'seconds': 0.9997220039367676, 'bytes': 256296, 'bits_per_second': 2050938.1527323928, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 14016223, 'rttvar': 3453455, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.001182079315186, 'end': 27.000904083251953, 'seconds': 0.9997220039367676, 'bytes': 256296, 'bits_per_second': 2050938.1527323928, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.000904083251953, 'end': 28.001161098480225, 'seconds': 1.0002570152282715, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 14676016, 'rttvar': 3109571, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.000904083251953, 'end': 28.001161098480225, 'seconds': 1.0002570152282715, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001161098480225, 'end': 29.00123620033264, 'seconds': 1.000075101852417, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14991541, 'rttvar': 2963228, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001161098480225, 'end': 29.00123620033264, 'seconds': 1.000075101852417, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.00123620033264, 'end': 30.000699996948242, 'seconds': 0.9994637966156006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 15673002, 'rttvar': 2847857, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.00123620033264, 'end': 30.000699996948242, 'seconds': 0.9994637966156006, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.000699996948242, 'end': 31.00126814842224, 'seconds': 1.000568151473999, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 16031214, 'rttvar': 2852316, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.000699996948242, 'end': 31.00126814842224, 'seconds': 1.000568151473999, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00126814842224, 'end': 32.00126504898071, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 16699289, 'rttvar': 2785032, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00126814842224, 'end': 32.00126504898071, 'seconds': 0.9999969005584717, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00126504898071, 'end': 33.00132417678833, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 17019166, 'rttvar': 2728527, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00126504898071, 'end': 33.00132417678833, 'seconds': 1.0000591278076172, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00132417678833, 'end': 34.00120401382446, 'seconds': 0.9998798370361328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17650620, 'rttvar': 2615336, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00132417678833, 'end': 34.00120401382446, 'seconds': 0.9998798370361328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00120401382446, 'end': 35.00131702423096, 'seconds': 1.0001130104064941, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17964070, 'rttvar': 2588400, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00120401382446, 'end': 35.00131702423096, 'seconds': 1.0001130104064941, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00131702423096, 'end': 36.00126600265503, 'seconds': 0.9999489784240723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 18568273, 'rttvar': 2527256, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00131702423096, 'end': 36.00126600265503, 'seconds': 0.9999489784240723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00126600265503, 'end': 37.00130605697632, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18854070, 'rttvar': 2467036, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00126600265503, 'end': 37.00130605697632, 'seconds': 1.000040054321289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00130605697632, 'end': 38.00123310089111, 'seconds': 0.9999270439147949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19482133, 'rttvar': 2474958, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00130605697632, 'end': 38.00123310089111, 'seconds': 0.9999270439147949, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00123310089111, 'end': 39.00126600265503, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19823876, 'rttvar': 2539705, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00123310089111, 'end': 39.00126600265503, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.00126600265503, 'end': 40.00121712684631, 'seconds': 0.9999511241912842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 20123559, 'rttvar': 2504145, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.00126600265503, 'end': 40.00121712684631, 'seconds': 0.9999511241912842, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00121712684631, 'end': 41.00133299827576, 'seconds': 1.0001158714294434, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20649454, 'rttvar': 2320905, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00121712684631, 'end': 41.00133299827576, 'seconds': 1.0001158714294434, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00133299827576, 'end': 42.001216173172, 'seconds': 0.9998831748962402, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20953255, 'rttvar': 2348280, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00133299827576, 'end': 42.001216173172, 'seconds': 0.9998831748962402, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.001216173172, 'end': 43.00133299827576, 'seconds': 1.0001168251037598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 70952, 'rtt': 21538280, 'rttvar': 2358314, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.001216173172, 'end': 43.00133299827576, 'seconds': 1.0001168251037598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00133299827576, 'end': 44.00127410888672, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21783444, 'rttvar': 2259065, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00133299827576, 'end': 44.00127410888672, 'seconds': 0.9999411106109619, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00127410888672, 'end': 45.00116300582886, 'seconds': 0.9998888969421387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 22239630, 'rttvar': 2084817, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00127410888672, 'end': 45.00116300582886, 'seconds': 0.9998888969421387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00116300582886, 'end': 46.001182079315186, 'seconds': 1.0000190734863281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 22459443, 'rttvar': 2003239, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00116300582886, 'end': 46.001182079315186, 'seconds': 1.0000190734863281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.001182079315186, 'end': 47.0004141330719, 'seconds': 0.9992320537567139, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22644141, 'rttvar': 1871824, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.001182079315186, 'end': 47.0004141330719, 'seconds': 0.9992320537567139, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.0004141330719, 'end': 48.00126600265503, 'seconds': 1.0008518695831299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 23099025, 'rttvar': 1838000, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.0004141330719, 'end': 48.00126600265503, 'seconds': 1.0008518695831299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.00126600265503, 'end': 49.00107002258301, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 23258668, 'rttvar': 1697785, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.00126600265503, 'end': 49.00107002258301, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.00107002258301, 'end': 50.00130915641785, 'seconds': 1.0002391338348389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 78192, 'rtt': 23660133, 'rttvar': 1644268, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.00107002258301, 'end': 50.00130915641785, 'seconds': 1.0002391338348389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00130915641785, 'end': 51.001320123672485, 'seconds': 1.0000109672546387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23810270, 'rttvar': 1533477, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00130915641785, 'end': 51.001320123672485, 'seconds': 1.0000109672546387, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.001320123672485, 'end': 52.00126504898071, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24213998, 'rttvar': 1558192, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.001320123672485, 'end': 52.00126504898071, 'seconds': 0.9999449253082275, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00126504898071, 'end': 53.00135803222656, 'seconds': 1.0000929832458496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24374374, 'rttvar': 1489395, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00126504898071, 'end': 53.00135803222656, 'seconds': 1.0000929832458496, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00135803222656, 'end': 54.00108003616333, 'seconds': 0.9997220039367676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24799241, 'rttvar': 1573468, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00135803222656, 'end': 54.00108003616333, 'seconds': 0.9997220039367676, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00108003616333, 'end': 55.0010781288147, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 24971236, 'rttvar': 1524092, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00108003616333, 'end': 55.0010781288147, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.0010781288147, 'end': 56.0014591217041, 'seconds': 1.0003809928894043, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 25207160, 'rttvar': 1614915, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.0010781288147, 'end': 56.0014591217041, 'seconds': 1.0003809928894043, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.0014591217041, 'end': 57.00111699104309, 'seconds': 0.9996578693389893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 85432, 'rtt': 25593805, 'rttvar': 1578728, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.0014591217041, 'end': 57.00111699104309, 'seconds': 0.9996578693389893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.00111699104309, 'end': 58.001320123672485, 'seconds': 1.0002031326293945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25836838, 'rttvar': 1670112, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.00111699104309, 'end': 58.001320123672485, 'seconds': 1.0002031326293945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001320123672485, 'end': 59.001305103302, 'seconds': 0.9999849796295166, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 88328, 'rtt': 26251788, 'rttvar': 1659292, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001320123672485, 'end': 59.001305103302, 'seconds': 0.9999849796295166, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}"
18:43:27.909347 tst                      iperf3.py:8: ERR: KeyError: 'sum_sent'  [trial-2064↪gprs:sysmo↪iperf3.py:8]
18:43:27.917072 tst                      iperf3.py:8: Test FAILED (196.8 sec)
18:43:27.925109 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Terminating (SIGTERM)
18:43:27.968726 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Terminating (SIGTERM)
18:43:28.012435 run   osmo-hlr_10.42.42.2(pid=26848): Terminating (SIGTERM)
18:43:28.055458 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Terminating (SIGTERM)
18:43:28.102051 run   osmo-stp_10.42.42.5(pid=26850): Terminating (SIGTERM)
18:43:28.150418 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Terminating (SIGTERM)
18:43:28.214286 run  osmo-ggsn_10.42.42.6(pid=26856): Terminating (SIGTERM)
18:43:28.262987 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Terminating (SIGTERM)
18:43:28.347167 run  osmo-sgsn_10.42.42.7(pid=26880): Terminating (SIGTERM)
18:43:28.394825 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Terminating (SIGTERM)
18:43:28.448858 run   osmo-msc_10.42.42.8(pid=26882): Terminating (SIGTERM)
18:43:28.496223 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Terminating (SIGTERM)
18:43:28.541767 run   osmo-mgw_10.42.42.3(pid=26884): Terminating (SIGTERM)
18:43:28.588810 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Terminating (SIGTERM)
18:43:28.632853 run   osmo-mgw_10.42.42.4(pid=26886): Terminating (SIGTERM)
18:43:28.676571 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Terminating (SIGTERM)
18:43:28.720875 run   osmo-bsc_10.42.42.9(pid=26888): Terminating (SIGTERM)
18:43:28.763739 run        osmo-bts-sysmo(pid=26897): Terminating (SIGTERM)
18:43:28.808091 run              osmo-pcu(pid=26905): Terminating (SIGTERM)
18:43:28.852311 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGTERM)
18:43:28.996987 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGTERM(pid=26955): Launched
18:43:30.090195 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGTERM(pid=26955): Terminated: ok {rc=0}
18:43:30.156370 ---      ParallelTerminationStrategy: PID 26843 died...
18:43:30.176371 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26843): Terminated: ok {rc=0}
18:43:30.222871 ---      ParallelTerminationStrategy: PID 26847 died...
18:43:30.242907 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26847): Terminated: ok {rc=0}
18:43:30.289766 ---      ParallelTerminationStrategy: PID 26848 died...
18:43:30.309613 run   osmo-hlr_10.42.42.2(pid=26848): Terminated: ok {rc=0}
18:43:30.356237 ---      ParallelTerminationStrategy: PID 26849 died...
18:43:30.375971 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26849): Terminated: ok {rc=0}
18:43:30.422578 ---      ParallelTerminationStrategy: PID 26850 died...
18:43:30.442912 run   osmo-stp_10.42.42.5(pid=26850): Terminated: ok {rc=0}
18:43:30.489396 ---      ParallelTerminationStrategy: PID 26851 died...
18:43:30.509185 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26851): Terminated: ok {rc=0}
18:43:30.555822 ---      ParallelTerminationStrategy: PID 26856 died...
18:43:30.575617 run  osmo-ggsn_10.42.42.6(pid=26856): Terminated: ok {rc=0}
18:43:30.622206 ---      ParallelTerminationStrategy: PID 26879 died...
18:43:30.642338 run pcap-recorder_any(filters='host 10.42.42.7')(pid=26879): Terminated: ok {rc=0}
18:43:30.688871 ---      ParallelTerminationStrategy: PID 26880 died...
18:43:30.708755 run  osmo-sgsn_10.42.42.7(pid=26880): Terminated: ok {rc=0}
18:43:30.755517 ---      ParallelTerminationStrategy: PID 26881 died...
18:43:30.775344 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=26881): Terminated: ok {rc=0}
18:43:30.821866 ---      ParallelTerminationStrategy: PID 26882 died...
18:43:30.841699 run   osmo-msc_10.42.42.8(pid=26882): Terminated: ok {rc=0}
18:43:30.888599 ---      ParallelTerminationStrategy: PID 26883 died...
18:43:30.908575 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=26883): Terminated: ok {rc=0}
18:43:30.955162 ---      ParallelTerminationStrategy: PID 26884 died...
18:43:30.975000 run   osmo-mgw_10.42.42.3(pid=26884): Terminated {rc=15}
18:43:31.021719 ---      ParallelTerminationStrategy: PID 26885 died...
18:43:31.041588 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=26885): Terminated: ok {rc=0}
18:43:31.088246 ---      ParallelTerminationStrategy: PID 26886 died...
18:43:31.108220 run   osmo-mgw_10.42.42.4(pid=26886): Terminated {rc=15}
18:43:31.154903 ---      ParallelTerminationStrategy: PID 26887 died...
18:43:31.174860 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=26887): Terminated: ok {rc=0}
18:43:31.221814 ---      ParallelTerminationStrategy: PID 26888 died...
18:43:31.241582 run   osmo-bsc_10.42.42.9(pid=26888): Terminated: ok {rc=0}
18:43:31.288191 ---      ParallelTerminationStrategy: PID 26897 died...
18:43:31.308225 run        osmo-bts-sysmo(pid=26897): Terminated: ok {rc=0}
18:43:31.355063 ---      ParallelTerminationStrategy: PID 26905 died...
18:43:31.374911 run              osmo-pcu(pid=26905): Terminated: ok {rc=0}
18:43:37.243809 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGINT)
18:43:37.382029 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGINT(pid=26960): Launched
18:43:38.472790 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGINT(pid=26960): Terminated: ok {rc=0}
18:43:44.358862 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931): Terminating (SIGKILL)
18:43:44.502121 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGUSR1(pid=26963): Launched
18:43:45.589473 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26931)-killSignals.SIGUSR1(pid=26963): Terminated: ok {rc=0}
18:43:45.841026 bus                          /gobi_6: Setting Online False
18:43:48.873285 bus                          /gobi_6: Setting Powered False
18:44:03.937866 bus                          /gobi_6: Setting Powered False

----------------------------------------------
trial-2064 gprs:sysmo iperf3m4.py
----------------------------------------------
18:44:05.136907 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.148186 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
18:44:05.183216 tst                      iperf3m4.py: using LAC 3947
18:44:05.311378 tst                      iperf3m4.py: using RAC 122
18:44:05.438236 tst                      iperf3m4.py: using CellId 3947
18:44:05.565537 tst                      iperf3m4.py: using BVCI 3948
18:44:05.685491 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.696587 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.707528 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.718354 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.729132 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.739899 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.750781 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.761644 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:44:05.772694 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:05.889967 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:06.007375 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:06.126900 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:44:06.243958 tst                    iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
18:44:06.449284 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:06.759360 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Launched
18:44:07.028978 run iperf3-srv_10.42.42.10(pid=26967): Launched
18:44:07.145656 tst                    iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
18:44:07.352490 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:07.670076 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Launched
18:44:07.957811 run iperf3-srv_10.42.42.10(pid=26971): Launched
18:44:08.080823 tst                    iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
18:44:08.287392 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:08.595727 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Launched
18:44:08.864073 run iperf3-srv_10.42.42.10(pid=26973): Launched
18:44:08.977215 tst                    iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
18:44:09.180898 run           iperf3-srv_10.42.42.10: Starting iperf3-srv
18:44:09.489782 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Launched
18:44:09.759377 run iperf3-srv_10.42.42.10(pid=26975): Launched
18:44:09.872449 tst                    iperf3m4.py:8: start network...
18:44:10.075753 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
18:44:10.547693 run         create_hlr_db(pid=26976): Launched
18:44:10.665746 bus                          /gobi_6: Setting Powered False
18:44:11.725965 run         create_hlr_db(pid=26976): Terminated: ok {rc=0}
18:44:12.173453 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Launched
18:44:12.501801 run   osmo-hlr_10.42.42.2(pid=26979): Launched
18:44:12.626842 run              osmo-stp_10.42.42.5: Starting osmo-stp
18:44:13.188903 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Launched
18:44:13.515079 run   osmo-stp_10.42.42.5(pid=26981): Launched
18:44:13.639823 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:44:14.200195 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Launched
18:44:14.323817 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:44:14.600687 run              patchelf(pid=26983): Launched
18:44:15.785745 run              patchelf(pid=26983): Terminated: ok {rc=0}
18:44:15.914425 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:44:16.175801 run osmo-gsm-tester_setcap_net_admin.sh(pid=26984): Launched
18:44:17.363964 run osmo-gsm-tester_setcap_net_admin.sh(pid=26984): Terminated: ok {rc=0}
18:44:17.707289 run  osmo-ggsn_10.42.42.6(pid=26987): Launched
18:44:17.830611 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:44:18.532486 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Launched
18:44:18.859651 run  osmo-sgsn_10.42.42.7(pid=27009): Launched
18:44:18.984720 run              osmo-msc_10.42.42.8: Starting osmo-msc
18:44:19.579104 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Launched
18:44:19.908672 run   osmo-msc_10.42.42.8(pid=27011): Launched
18:44:23.091202 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
18:44:23.664655 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Launched
18:44:23.993712 run   osmo-mgw_10.42.42.3(pid=27013): Launched
18:44:24.119230 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
18:44:24.668716 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Launched
18:44:24.996797 run   osmo-mgw_10.42.42.4(pid=27015): Launched
18:44:25.121290 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
18:44:25.796587 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Launched
18:44:26.125430 run   osmo-bsc_10.42.42.9(pid=27017): Launched
18:44:32.415553 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:44:32.987097 run         rm-remote-dir(pid=27018): Launched
18:44:35.185020 run         rm-remote-dir(pid=27018): Terminated: ok {rc=0}
18:44:35.546097 run         mk-remote-dir(pid=27019): Launched
18:44:36.737476 run         mk-remote-dir(pid=27019): Terminated: ok {rc=0}
18:44:37.045347 run    scp-inst-to-remote(pid=27020): Launched
18:44:49.256754 run    scp-inst-to-remote(pid=27020): Terminated: ok {rc=0}
18:44:49.613206 run         mk-remote-dir(pid=27022): Launched
18:44:50.802045 run         mk-remote-dir(pid=27022): Terminated: ok {rc=0}
18:44:51.108424 run     scp-cfg-to-remote(pid=27023): Launched
18:44:52.298931 run     scp-cfg-to-remote(pid=27023): Terminated: ok {rc=0}
18:44:52.652745 run   reload-dsp-firmware(pid=27025): Launched
18:44:54.843227 run   reload-dsp-firmware(pid=27025): Terminated: ok {rc=0}
18:44:55.189647 run        osmo-bts-sysmo(pid=27026): Launched
18:44:59.429304 tst                    iperf3m4.py:8: Waiting for bts to be ready...
18:45:01.154806 run         rm-remote-dir(pid=27027): Launched
18:45:03.341684 run         rm-remote-dir(pid=27027): Terminated: ok {rc=0}
18:45:03.691280 run         mk-remote-dir(pid=27031): Launched
18:45:05.880125 run         mk-remote-dir(pid=27031): Terminated: ok {rc=0}
18:45:06.177538 run       scp-inst-to-bts(pid=27032): Launched
18:45:19.377424 run       scp-inst-to-bts(pid=27032): Terminated: ok {rc=0}
18:45:19.724486 run     mk-remote-run-dir(pid=27034): Launched
18:45:21.912452 run     mk-remote-run-dir(pid=27034): Terminated: ok {rc=0}
18:45:22.209247 run   scp-cfg-to-sysmobts(pid=27035): Launched
18:45:24.394149 run   scp-cfg-to-sysmobts(pid=27035): Terminated: ok {rc=0}
18:45:24.740605 run              osmo-pcu(pid=27037): Launched
18:45:24.874454 tst                      iperf3m4.py: using MSISDN 3952
18:45:25.045636 bus                          /gobi_6: Setting Powered True
18:45:28.564255 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3952', subscriber_id=1}
18:45:28.837192 bus                          /gobi_6: Setting Online False
18:45:30.025793 bus                          /gobi_6: Setting Powered False
18:45:37.109329 bus                          /gobi_6: Setting Powered True
18:45:38.498647 bus                          /gobi_6: Setting Online True
18:45:40.959901 tst                          /gobi_6: Connect to ('901', '70')
18:45:41.236070 tst                          /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G  1  [Jul  5 2017 10:49:17]', Serial: '868323024273648'
18:45:41.354649 tst                      iperf3m4.py: using MSISDN 3953
18:45:41.629393 bus                          /gobi_0: Setting Powered True
18:45:45.141049 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3953', subscriber_id=2}
18:45:45.417127 bus                          /gobi_0: Setting Online False
18:45:46.608611 bus                          /gobi_0: Setting Powered False
18:45:54.109834 bus                          /gobi_0: Setting Powered True
18:45:55.375490 tst                          /gobi_6: Already registered with network ('901', '70')
18:45:55.596071 bus                          /gobi_0: Setting Online True
18:45:58.080877 tst                          /gobi_0: Connect to ('901', '70')
18:45:58.345558 tst                          /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G  1  [Mar 14 2016 19:33:01]', Serial: '868323024271873'
18:45:58.470839 tst                      iperf3m4.py: using MSISDN 3954
18:45:59.030117 bus                          /gobi_1: Setting Powered True
18:46:02.522178 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3954', subscriber_id=3}
18:46:02.793149 bus                          /gobi_1: Setting Online False
18:46:03.987042 bus                          /gobi_1: Setting Powered False
18:46:11.135406 bus                          /gobi_1: Setting Powered True
18:46:12.516607 bus                          /gobi_1: Setting Online True
18:46:13.430167 tst                          /gobi_0: Already registered with network ('901', '70')
18:46:15.240244 tst                          /gobi_1: Connect to ('901', '70')
18:46:15.517543 tst                          /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G  1  [Mar 14 2016 19:33:01]', Serial: '868323024272103'
18:46:15.643109 tst                      iperf3m4.py: using MSISDN 3955
18:46:15.820467 bus                          /gobi_4: Setting Powered True
18:46:19.346144 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3955', subscriber_id=4}
18:46:19.608722 bus                          /gobi_4: Setting Online False
18:46:20.796686 bus                          /gobi_4: Setting Powered False
18:46:28.115681 bus                          /gobi_4: Setting Powered True
18:46:29.508779 bus                          /gobi_4: Setting Online True
18:46:30.931992 tst                          /gobi_1: Already registered with network ('901', '70')
18:46:32.176918 tst                          /gobi_4: Connect to ('901', '70')
18:46:32.464749 tst                          /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G  1  [Jul 21 2016 10:00:50]', Serial: '868323025161511'
18:46:32.573732 tst                    iperf3m4.py:8: waiting for modems to attach...
18:46:48.650917 tst                          /gobi_4: Already registered with network ('901', '70')
18:46:54.201032 tst                    iperf3m4.py:8: waiting for modems to attach to data services...
18:47:02.637656 tst                          /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:02.756476 tst                    iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'"
18:47:03.210169 run            move_netns(pid=27042): Launched
18:47:04.401302 run            move_netns(pid=27042): Terminated: ok {rc=0}
18:47:04.757128 run                  ifup(pid=27045): Launched
18:47:05.946933 run                  ifup(pid=27045): Terminated: ok {rc=0}
18:47:06.300682 run                  dhcp(pid=27048): Launched
18:47:07.487692 run                  dhcp(pid=27048): Terminated: ok {rc=0}
18:47:15.821796 tst                          /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:15.932128 tst                    iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'"
18:47:16.350000 run            move_netns(pid=27066): Launched
18:47:17.536607 run            move_netns(pid=27066): Terminated: ok {rc=0}
18:47:17.867113 run                  ifup(pid=27069): Launched
18:47:19.051424 run                  ifup(pid=27069): Terminated: ok {rc=0}
18:47:19.380586 run                  dhcp(pid=27072): Launched
18:47:20.570818 run                  dhcp(pid=27072): Terminated: ok {rc=0}
18:47:23.596883 tst                          /gobi_1: context activated {apn='inet46', path='/gobi_1/context9', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:23.706843 tst                    iperf3m4.py:8: Setting up data plan for "'/gobi_1/context9'"
18:47:24.123762 run            move_netns(pid=27090): Launched
18:47:25.310363 run            move_netns(pid=27090): Terminated: ok {rc=0}
18:47:25.641387 run                  ifup(pid=27093): Launched
18:47:26.832699 run                  ifup(pid=27093): Terminated: ok {rc=0}
18:47:27.187196 run                  dhcp(pid=27096): Launched
18:47:28.378114 run                  dhcp(pid=27096): Terminated: ok {rc=0}
18:47:36.783631 tst                          /gobi_4: context activated {apn='inet46', path='/gobi_4/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:47:36.894066 tst                    iperf3m4.py:8: Setting up data plan for "'/gobi_4/context2'"
18:47:37.311648 run            move_netns(pid=27114): Launched
18:47:38.506943 run            move_netns(pid=27114): Terminated: ok {rc=0}
18:47:38.862571 run                  ifup(pid=27117): Launched
18:47:40.054669 run                  ifup(pid=27117): Terminated: ok {rc=0}
18:47:40.408583 run                  dhcp(pid=27120): Launched
18:47:41.597956 run                  dhcp(pid=27120): Terminated: ok {rc=0}
18:47:41.737207 tst                    iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'"
18:47:41.964359 run           iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s)
18:47:42.355348 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Launched
18:47:42.538390 tst                    iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'"
18:47:42.765153 run           iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s)
18:47:43.157076 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Launched
18:47:43.338744 tst                    iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context9'"
18:47:43.563135 run           iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s)
18:47:43.952340 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Launched
18:47:44.132025 tst                    iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context2'"
18:47:44.356819 run           iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s)
18:47:44.745979 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Launched
18:47:45.105657 run iperf3-cli_10.42.42.10(pid=27148): Launched
18:47:45.382355 run iperf3-cli_10.42.42.10(pid=27151): Launched
18:47:45.659249 run iperf3-cli_10.42.42.10(pid=27154): Launched
18:47:45.929122 run iperf3-cli_10.42.42.10(pid=27157): Launched
18:48:54.251448 run iperf3-cli_10.42.42.10(pid=27148): ERR: Terminated: ERROR {rc=1}  [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27148)]
18:48:54.518913 run iperf3-cli_10.42.42.10(pid=27148): stdout: 
| 				"bits_per_second":	0,
| 				"retransmits":	0,
| 				"omitted":	false
| 			}
| 		}],
| 	"end":	{
| 	},
| 	"error":	"error - control socket has closed unexpectedly"
| }
| child exited with 1 
18:48:54.656399 run iperf3-cli_10.42.42.10(pid=27148): stderr: 
| (launched: 2022-03-29_18:47:45.089716) 
18:48:55.843780 run iperf3-cli_10.42.42.10(pid=27151): ERR: Terminated: ERROR {rc=1}  [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27151)]
18:48:55.980978 run iperf3-cli_10.42.42.10(pid=27151): stdout: 
| 				"bits_per_second":	0,
| 				"retransmits":	0,
| 				"omitted":	false
| 			}
| 		}],
| 	"end":	{
| 	},
| 	"error":	"error - control socket has closed unexpectedly"
| }
| child exited with 1 
18:48:56.124272 run iperf3-cli_10.42.42.10(pid=27151): stderr: 
| (launched: 2022-03-29_18:47:45.365519) 
18:48:57.329256 run iperf3-cli_10.42.42.10(pid=27154): ERR: Terminated: ERROR {rc=1}  [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27154)]
18:48:57.471555 run iperf3-cli_10.42.42.10(pid=27154): stdout: 
| 				"bits_per_second":	0,
| 				"retransmits":	0,
| 				"omitted":	false
| 			}
| 		}],
| 	"end":	{
| 	},
| 	"error":	"error - control socket has closed unexpectedly"
| }
| child exited with 1 
18:48:57.614442 run iperf3-cli_10.42.42.10(pid=27154): stderr: 
| (launched: 2022-03-29_18:47:45.642079) 
18:48:58.810636 run iperf3-cli_10.42.42.10(pid=27157): ERR: Terminated: ERROR {rc=1}  [trial-2064↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=27157)]
18:48:58.939380 run iperf3-cli_10.42.42.10(pid=27157): stdout: 
| 				"bits_per_second":	0,
| 				"retransmits":	0,
| 				"omitted":	false
| 			}
| 		}],
| 	"end":	{
| 	},
| 	"error":	"error - control socket has closed unexpectedly"
| }
| child exited with 1 
18:48:59.067024 run iperf3-cli_10.42.42.10(pid=27157): stderr: 
| (launched: 2022-03-29_18:47:45.912502) 
18:48:59.194618 run iperf3-srv_10.42.42.10(pid=26967): Terminating (SIGINT)
18:48:59.525727 run iperf3-srv_10.42.42.10(pid=26967): Terminated {rc=1}
18:48:59.640844 tst                    iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'"
Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 42908, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Tue, 29 Mar 2022 16:47:46 GMT', 'timesecs': 1648572466}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'bbwlr3tnbptbxm2dq6maiza55z7ufvcy5did', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0006639957427979, 'seconds': 1.0006639957427979, 'bytes': 101360, 'bits_per_second': 810341.9364040171, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 379730, 'rttvar': 175208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0006639957427979, 'seconds': 1.0006639957427979, 'bytes': 101360, 'bits_per_second': 810341.9364040171, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0006639957427979, 'end': 2.0005221366882324, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 521979, 'rttvar': 415904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0006639957427979, 'end': 2.0005221366882324, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0005221366882324, 'end': 3.001317024230957, 'seconds': 1.0007948875427246, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 4344, 'rtt': 521979, 'rttvar': 415904, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0005221366882324, 'end': 3.001317024230957, 'seconds': 1.0007948875427246, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.001317024230957, 'end': 4.001322984695435, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 809107, 'rttvar': 886184, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.001317024230957, 'end': 4.001322984695435, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001322984695435, 'end': 5.001317977905273, 'seconds': 0.9999949932098389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1250143, 'rttvar': 1546710, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001322984695435, 'end': 5.001317977905273, 'seconds': 0.9999949932098389, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001317977905273, 'end': 6.000808000564575, 'seconds': 0.9994900226593018, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1250143, 'rttvar': 1546710, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001317977905273, 'end': 6.000808000564575, 'seconds': 0.9994900226593018, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.000808000564575, 'end': 7.001322984695435, 'seconds': 1.0005149841308594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1828734, 'rttvar': 2317215, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.000808000564575, 'end': 7.001322984695435, 'seconds': 1.0005149841308594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001322984695435, 'end': 8.000629186630249, 'seconds': 0.9993062019348145, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2527605, 'rttvar': 3135652, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001322984695435, 'end': 8.000629186630249, 'seconds': 0.9993062019348145, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.000629186630249, 'end': 9.00132417678833, 'seconds': 1.000694990158081, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2527605, 'rttvar': 3135652, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.000629186630249, 'end': 9.00132417678833, 'seconds': 1.000694990158081, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.00132417678833, 'end': 10.001171112060547, 'seconds': 0.9998469352722168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 3326609, 'rttvar': 3949748, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.00132417678833, 'end': 10.001171112060547, 'seconds': 0.9998469352722168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001171112060547, 'end': 11.000813007354736, 'seconds': 0.9996418952941895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4210285, 'rttvar': 4729664, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001171112060547, 'end': 11.000813007354736, 'seconds': 0.9996418952941895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.000813007354736, 'end': 12.001137971878052, 'seconds': 1.0003249645233154, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4210285, 'rttvar': 4729664, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.000813007354736, 'end': 12.001137971878052, 'seconds': 1.0003249645233154, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001137971878052, 'end': 13.001238107681274, 'seconds': 1.0001001358032227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 5170983, 'rttvar': 5468642, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001137971878052, 'end': 13.001238107681274, 'seconds': 1.0001001358032227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001238107681274, 'end': 14.000625133514404, 'seconds': 0.9993870258331299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6209445, 'rttvar': 6178407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001238107681274, 'end': 14.000625133514404, 'seconds': 0.9993870258331299, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.000625133514404, 'end': 15.00132417678833, 'seconds': 1.0006990432739258, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6209445, 'rttvar': 6178407, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.000625133514404, 'end': 15.00132417678833, 'seconds': 1.0006990432739258, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.00132417678833, 'end': 16.001312971115112, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7307874, 'rttvar': 6830663, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.00132417678833, 'end': 16.001312971115112, 'seconds': 0.9999887943267822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001312971115112, 'end': 17.001317024230957, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7307874, 'rttvar': 6830663, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001312971115112, 'end': 17.001317024230957, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001317024230957, 'end': 18.000580072402954, 'seconds': 0.9992630481719971, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8366236, 'rttvar': 7239721, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001317024230957, 'end': 18.000580072402954, 'seconds': 0.9992630481719971, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.000580072402954, 'end': 19.000797033309937, 'seconds': 1.0002169609069824, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.000580072402954, 'end': 19.000797033309937, 'seconds': 1.0002169609069824, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.000797033309937, 'end': 20.001474142074585, 'seconds': 1.0006771087646484, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.000797033309937, 'end': 20.001474142074585, 'seconds': 1.0006771087646484, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001474142074585, 'end': 21.000576972961426, 'seconds': 0.9991028308868408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9390212, 'rttvar': 7477743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001474142074585, 'end': 21.000576972961426, 'seconds': 0.9991028308868408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.000576972961426, 'end': 22.001118183135986, 'seconds': 1.0005412101745605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10496162, 'rttvar': 7820208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.000576972961426, 'end': 22.001118183135986, 'seconds': 1.0005412101745605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001118183135986, 'end': 23.001479148864746, 'seconds': 1.0003609657287598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10496162, 'rttvar': 7820208, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001118183135986, 'end': 23.001479148864746, 'seconds': 1.0003609657287598, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001479148864746, 'end': 24.001168966293335, 'seconds': 0.9996898174285889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 11651412, 'rttvar': 8175655, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001479148864746, 'end': 24.001168966293335, 'seconds': 0.9996898174285889, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001168966293335, 'end': 25.001140117645264, 'seconds': 0.9999711513519287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12672600, 'rttvar': 8174116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001168966293335, 'end': 25.001140117645264, 'seconds': 0.9999711513519287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001140117645264, 'end': 26.000615119934082, 'seconds': 0.9994750022888184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12672600, 'rttvar': 8174116, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001140117645264, 'end': 26.000615119934082, 'seconds': 0.9994750022888184, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.000615119934082, 'end': 27.00078797340393, 'seconds': 1.0001728534698486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 13753602, 'rttvar': 8292592, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.000615119934082, 'end': 27.00078797340393, 'seconds': 1.0001728534698486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00078797340393, 'end': 28.00065517425537, 'seconds': 0.9998672008514404, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14696598, 'rttvar': 8105436, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00078797340393, 'end': 28.00065517425537, 'seconds': 0.9998672008514404, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.00065517425537, 'end': 29.0013210773468, 'seconds': 1.0006659030914307, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14696598, 'rttvar': 8105436, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.00065517425537, 'end': 29.0013210773468, 'seconds': 1.0006659030914307, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.0013210773468, 'end': 30.000684022903442, 'seconds': 0.9993629455566406, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 15721861, 'rttvar': 8129602, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.0013210773468, 'end': 30.000684022903442, 'seconds': 0.9993629455566406, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.000684022903442, 'end': 31.00068497657776, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16616168, 'rttvar': 7885816, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.000684022903442, 'end': 31.00068497657776, 'seconds': 1.0000009536743164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00068497657776, 'end': 32.00132203102112, 'seconds': 1.0006370544433594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16616168, 'rttvar': 7885816, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00068497657776, 'end': 32.00132203102112, 'seconds': 1.0006370544433594, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00132203102112, 'end': 33.00074911117554, 'seconds': 0.999427080154419, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 17588449, 'rttvar': 7858924, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00132203102112, 'end': 33.00074911117554, 'seconds': 0.999427080154419, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00074911117554, 'end': 34.00058197975159, 'seconds': 0.9998328685760498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18444463, 'rttvar': 7606221, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00074911117554, 'end': 34.00058197975159, 'seconds': 0.9998328685760498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00058197975159, 'end': 35.0013210773468, 'seconds': 1.0007390975952148, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18444463, 'rttvar': 7606221, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00058197975159, 'end': 35.0013210773468, 'seconds': 1.0007390975952148, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.0013210773468, 'end': 36.00113916397095, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 19375780, 'rttvar': 7567300, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.0013210773468, 'end': 36.00113916397095, 'seconds': 0.9998180866241455, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00113916397095, 'end': 37.00057411193848, 'seconds': 0.9994349479675293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20193508, 'rttvar': 7310931, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00113916397095, 'end': 37.00057411193848, 'seconds': 0.9994349479675293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00057411193848, 'end': 38.00039196014404, 'seconds': 0.9998178482055664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20193508, 'rttvar': 7310931, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00057411193848, 'end': 38.00039196014404, 'seconds': 0.9998178482055664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00039196014404, 'end': 39.0006890296936, 'seconds': 1.0002970695495605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 21094187, 'rttvar': 7284557, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00039196014404, 'end': 39.0006890296936, 'seconds': 1.0002970695495605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0006890296936, 'end': 40.00058197975159, 'seconds': 0.9998929500579834, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21887500, 'rttvar': 7050044, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0006890296936, 'end': 40.00058197975159, 'seconds': 0.9998929500579834, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00058197975159, 'end': 41.00132203102112, 'seconds': 1.0007400512695312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21887500, 'rttvar': 7050044, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00058197975159, 'end': 41.00132203102112, 'seconds': 1.0007400512695312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00132203102112, 'end': 42.000596046447754, 'seconds': 0.9992740154266357, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 22769210, 'rttvar': 7050952, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00132203102112, 'end': 42.000596046447754, 'seconds': 0.9992740154266357, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.000596046447754, 'end': 43.00114297866821, 'seconds': 1.000546932220459, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23532537, 'rttvar': 6814868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.000596046447754, 'end': 43.00114297866821, 'seconds': 1.000546932220459, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00114297866821, 'end': 44.00148797035217, 'seconds': 1.00034499168396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23532537, 'rttvar': 6814868, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00114297866821, 'end': 44.00148797035217, 'seconds': 1.00034499168396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00148797035217, 'end': 45.000646114349365, 'seconds': 0.9991581439971924, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 24400576, 'rttvar': 6847229, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00148797035217, 'end': 45.000646114349365, 'seconds': 0.9991581439971924, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.000646114349365, 'end': 46.00114417076111, 'seconds': 1.0004980564117432, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25155044, 'rttvar': 6644358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.000646114349365, 'end': 46.00114417076111, 'seconds': 1.0004980564117432, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00114417076111, 'end': 47.0008499622345, 'seconds': 0.9997057914733887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25155044, 'rttvar': 6644358, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00114417076111, 'end': 47.0008499622345, 'seconds': 0.9997057914733887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.0008499622345, 'end': 48.00132203102112, 'seconds': 1.000472068786621, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26022865, 'rttvar': 6718910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.0008499622345, 'end': 48.00132203102112, 'seconds': 1.000472068786621, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.00132203102112, 'end': 49.0011351108551, 'seconds': 0.9998130798339844, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26022865, 'rttvar': 6718910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.00132203102112, 'end': 49.0011351108551, 'seconds': 0.9998130798339844, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.0011351108551, 'end': 50.00085496902466, 'seconds': 0.9997198581695557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 26794956, 'rttvar': 6583365, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.0011351108551, 'end': 50.00085496902466, 'seconds': 0.9997198581695557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00085496902466, 'end': 51.000646114349365, 'seconds': 0.999791145324707, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27670031, 'rttvar': 6687674, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00085496902466, 'end': 51.000646114349365, 'seconds': 0.999791145324707, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.000646114349365, 'end': 52.00117516517639, 'seconds': 1.0005290508270264, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27670031, 'rttvar': 6687674, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.000646114349365, 'end': 52.00117516517639, 'seconds': 1.0005290508270264, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00117516517639, 'end': 53.00131797790527, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 28446091, 'rttvar': 6567875, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00117516517639, 'end': 53.00131797790527, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00131797790527, 'end': 54.00036907196045, 'seconds': 0.9990510940551758, 'bytes': 256296, 'bits_per_second': 2052315.4543352732, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29302817, 'rttvar': 6639359, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00131797790527, 'end': 54.00036907196045, 'seconds': 0.9990510940551758, 'bytes': 256296, 'bits_per_second': 2052315.4543352732, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00036907196045, 'end': 55.00075912475586, 'seconds': 1.0003900527954102, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29302817, 'rttvar': 6639359, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00036907196045, 'end': 55.00075912475586, 'seconds': 1.0003900527954102, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.00075912475586, 'end': 56.00075316429138, 'seconds': 0.9999940395355225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 29869581, 'rttvar': 6113047, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.00075912475586, 'end': 56.00075316429138, 'seconds': 0.9999940395355225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.00075316429138, 'end': 57.000683069229126, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30555344, 'rttvar': 5956311, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.00075316429138, 'end': 57.000683069229126, 'seconds': 0.9999299049377441, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.000683069229126, 'end': 58.001219034194946, 'seconds': 1.0005359649658203, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30555344, 'rttvar': 5956311, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.000683069229126, 'end': 58.001219034194946, 'seconds': 1.0005359649658203, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001219034194946, 'end': 59.00129508972168, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 31175458, 'rttvar': 5707460, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001219034194946, 'end': 59.00129508972168, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}"
18:48:59.859128 tst                    iperf3m4.py:8: ERR: KeyError: 'sum_sent'  [trial-2064↪gprs:sysmo↪iperf3m4.py:8]
18:48:59.866575 tst                    iperf3m4.py:8: Test FAILED (294.8 sec)
18:48:59.873349 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Terminating (SIGTERM)
18:48:59.916435 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Terminating (SIGTERM)
18:48:59.958484 run iperf3-srv_10.42.42.10(pid=26971): Terminating (SIGTERM)
18:48:59.999372 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Terminating (SIGTERM)
18:49:00.040518 run iperf3-srv_10.42.42.10(pid=26973): Terminating (SIGTERM)
18:49:00.081176 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Terminating (SIGTERM)
18:49:00.122120 run iperf3-srv_10.42.42.10(pid=26975): Terminating (SIGTERM)
18:49:00.162967 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Terminating (SIGTERM)
18:49:00.203950 run   osmo-hlr_10.42.42.2(pid=26979): Terminating (SIGTERM)
18:49:00.244856 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Terminating (SIGTERM)
18:49:00.287028 run   osmo-stp_10.42.42.5(pid=26981): Terminating (SIGTERM)
18:49:00.328772 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Terminating (SIGTERM)
18:49:00.382622 run  osmo-ggsn_10.42.42.6(pid=26987): Terminating (SIGTERM)
18:49:00.424467 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Terminating (SIGTERM)
18:49:00.473819 run  osmo-sgsn_10.42.42.7(pid=27009): Terminating (SIGTERM)
18:49:00.529295 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Terminating (SIGTERM)
18:49:00.580227 run   osmo-msc_10.42.42.8(pid=27011): Terminating (SIGTERM)
18:49:00.633554 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Terminating (SIGTERM)
18:49:00.686829 run   osmo-mgw_10.42.42.3(pid=27013): Terminating (SIGTERM)
18:49:00.750948 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Terminating (SIGTERM)
18:49:00.799371 run   osmo-mgw_10.42.42.4(pid=27015): Terminating (SIGTERM)
18:49:00.844649 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Terminating (SIGTERM)
18:49:00.890072 run   osmo-bsc_10.42.42.9(pid=27017): Terminating (SIGTERM)
18:49:00.935822 run        osmo-bts-sysmo(pid=27026): Terminating (SIGTERM)
18:49:00.983304 run              osmo-pcu(pid=27037): Terminating (SIGTERM)
18:49:01.029399 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGTERM)
18:49:01.176449 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGTERM(pid=27181): Launched
18:49:02.277299 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGTERM(pid=27181): Terminated: ok {rc=0}
18:49:02.344389 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGTERM)
18:49:02.488995 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGTERM(pid=27186): Launched
18:49:03.577455 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGTERM(pid=27186): Terminated: ok {rc=0}
18:49:03.643834 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGTERM)
18:49:03.787356 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGTERM(pid=27189): Launched
18:49:04.880339 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGTERM(pid=27189): Terminated: ok {rc=0}
18:49:04.946489 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGTERM)
18:49:05.090521 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGTERM(pid=27192): Launched
18:49:06.177574 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGTERM(pid=27192): Terminated: ok {rc=0}
18:49:06.244170 ---      ParallelTerminationStrategy: PID 26966 died...
18:49:06.264257 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26966): Terminated: ok {rc=0}
18:49:06.311419 ---      ParallelTerminationStrategy: PID 26969 died...
18:49:06.331213 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26969): Terminated: ok {rc=0}
18:49:06.377416 ---      ParallelTerminationStrategy: PID 26971 died...
18:49:06.397009 run iperf3-srv_10.42.42.10(pid=26971): Terminated {rc=256}
18:49:06.443413 ---      ParallelTerminationStrategy: PID 26972 died...
18:49:06.463037 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26972): Terminated: ok {rc=0}
18:49:06.509326 ---      ParallelTerminationStrategy: PID 26973 died...
18:49:06.528920 run iperf3-srv_10.42.42.10(pid=26973): Terminated {rc=256}
18:49:06.577483 ---      ParallelTerminationStrategy: PID 26974 died...
18:49:06.596438 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26974): Terminated: ok {rc=0}
18:49:06.642684 ---      ParallelTerminationStrategy: PID 26975 died...
18:49:06.661640 run iperf3-srv_10.42.42.10(pid=26975): Terminated {rc=256}
18:49:06.707945 ---      ParallelTerminationStrategy: PID 26978 died...
18:49:06.727174 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26978): Terminated: ok {rc=0}
18:49:06.773390 ---      ParallelTerminationStrategy: PID 26979 died...
18:49:06.792396 run   osmo-hlr_10.42.42.2(pid=26979): Terminated: ok {rc=0}
18:49:06.838635 ---      ParallelTerminationStrategy: PID 26980 died...
18:49:06.857660 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26980): Terminated: ok {rc=0}
18:49:06.903935 ---      ParallelTerminationStrategy: PID 26981 died...
18:49:06.922961 run   osmo-stp_10.42.42.5(pid=26981): Terminated: ok {rc=0}
18:49:06.968966 ---      ParallelTerminationStrategy: PID 26982 died...
18:49:06.988234 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26982): Terminated: ok {rc=0}
18:49:07.034375 ---      ParallelTerminationStrategy: PID 26987 died...
18:49:07.053316 run  osmo-ggsn_10.42.42.6(pid=26987): Terminated: ok {rc=0}
18:49:07.099491 ---      ParallelTerminationStrategy: PID 27008 died...
18:49:07.118433 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27008): Terminated: ok {rc=0}
18:49:07.164729 ---      ParallelTerminationStrategy: PID 27009 died...
18:49:07.183676 run  osmo-sgsn_10.42.42.7(pid=27009): Terminated: ok {rc=0}
18:49:07.229597 ---      ParallelTerminationStrategy: PID 27010 died...
18:49:07.248614 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27010): Terminated: ok {rc=0}
18:49:07.294555 ---      ParallelTerminationStrategy: PID 27011 died...
18:49:07.313817 run   osmo-msc_10.42.42.8(pid=27011): Terminated: ok {rc=0}
18:49:07.359845 ---      ParallelTerminationStrategy: PID 27012 died...
18:49:07.378951 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27012): Terminated: ok {rc=0}
18:49:07.425626 ---      ParallelTerminationStrategy: PID 27013 died...
18:49:07.444927 run   osmo-mgw_10.42.42.3(pid=27013): Terminated {rc=15}
18:49:07.491404 ---      ParallelTerminationStrategy: PID 27014 died...
18:49:07.510826 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27014): Terminated: ok {rc=0}
18:49:07.557304 ---      ParallelTerminationStrategy: PID 27015 died...
18:49:07.576806 run   osmo-mgw_10.42.42.4(pid=27015): Terminated {rc=15}
18:49:07.627753 ---      ParallelTerminationStrategy: PID 27016 died...
18:49:07.647875 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27016): Terminated: ok {rc=0}
18:49:07.694884 ---      ParallelTerminationStrategy: PID 27017 died...
18:49:07.714878 run   osmo-bsc_10.42.42.9(pid=27017): Terminated: ok {rc=0}
18:49:07.762099 ---      ParallelTerminationStrategy: PID 27026 died...
18:49:07.782164 run        osmo-bts-sysmo(pid=27026): Terminated: ok {rc=0}
18:49:07.829070 ---      ParallelTerminationStrategy: PID 27037 died...
18:49:07.849277 run              osmo-pcu(pid=27037): Terminated: ok {rc=0}
18:49:13.719053 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGINT)
18:49:13.862564 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGINT(pid=27199): Launched
18:49:14.951823 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGINT(pid=27199): Terminated: ok {rc=0}
18:49:15.017876 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGINT)
18:49:15.161781 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGINT(pid=27202): Launched
18:49:16.246800 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGINT(pid=27202): Terminated: ok {rc=0}
18:49:16.313204 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGINT)
18:49:16.455282 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGINT(pid=27205): Launched
18:49:17.539859 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGINT(pid=27205): Terminated: ok {rc=0}
18:49:17.606118 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGINT)
18:49:17.749720 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGINT(pid=27208): Launched
18:49:18.842561 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGINT(pid=27208): Terminated: ok {rc=0}
18:49:24.731262 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136): Terminating (SIGKILL)
18:49:24.875297 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGUSR1(pid=27211): Launched
18:49:25.965709 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27136)-killSignals.SIGUSR1(pid=27211): Terminated: ok {rc=0}
18:49:26.031968 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139): Terminating (SIGKILL)
18:49:26.174568 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGUSR1(pid=27214): Launched
18:49:27.265545 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27139)-killSignals.SIGUSR1(pid=27214): Terminated: ok {rc=0}
18:49:27.331672 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142): Terminating (SIGKILL)
18:49:27.474564 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGUSR1(pid=27217): Launched
18:49:28.565339 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27142)-killSignals.SIGUSR1(pid=27217): Terminated: ok {rc=0}
18:49:28.631806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145): Terminating (SIGKILL)
18:49:28.774252 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGUSR1(pid=27220): Launched
18:49:29.859536 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=27145)-killSignals.SIGUSR1(pid=27220): Terminated: ok {rc=0}
18:49:30.105783 bus                          /gobi_4: Setting Online False
18:49:31.244861 bus                          /gobi_4: Setting Powered False
18:49:45.944600 bus                          /gobi_4: Setting Powered False
18:49:47.044501 bus                          /gobi_4: Setting Powered False
18:49:48.202038 bus                          /gobi_1: Setting Online False
18:49:49.324698 bus                          /gobi_1: Setting Powered False
18:49:57.098064 bus                          /gobi_1: Setting Powered False
18:49:58.197022 bus                          /gobi_1: Setting Powered False
18:49:59.354516 bus                          /gobi_0: Setting Online False
18:50:00.491432 bus                          /gobi_0: Setting Powered False
18:50:08.937999 bus                          /gobi_0: Setting Powered False
18:50:10.037687 bus                          /gobi_0: Setting Powered False
18:50:11.196010 bus                          /gobi_6: Setting Online False
18:50:12.331026 bus                          /gobi_6: Setting Powered False
18:50:20.944844 bus                          /gobi_6: Setting Powered False

----------------------------------------------
trial-2064 gprs:sysmo ping.py
----------------------------------------------
18:50:22.103660 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.116667 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
18:50:22.150394 tst                          ping.py: using LAC 3948
18:50:22.271535 tst                          ping.py: using RAC 123
18:50:22.389731 tst                          ping.py: using CellId 3948
18:50:22.508184 tst                          ping.py: using BVCI 3949
18:50:22.619861 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.630352 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.640895 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.651947 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.662828 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.673696 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.684731 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:50:22.695704 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:50:22.813045 tst                       ping.py:19: start network...
18:50:23.015211 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
18:50:23.481826 run         create_hlr_db(pid=27227): Launched
18:50:23.603372 bus                          /gobi_6: Setting Powered False
18:50:24.664060 run         create_hlr_db(pid=27227): Terminated: ok {rc=0}
18:50:25.105335 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Launched
18:50:25.428549 run   osmo-hlr_10.42.42.2(pid=27230): Launched
18:50:25.551650 run              osmo-stp_10.42.42.5: Starting osmo-stp
18:50:26.101626 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Launched
18:50:26.426917 run   osmo-stp_10.42.42.5(pid=27232): Launched
18:50:26.550984 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:50:27.104949 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Launched
18:50:27.228518 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:50:27.502204 run              patchelf(pid=27234): Launched
18:50:28.685851 run              patchelf(pid=27234): Terminated: ok {rc=0}
18:50:28.821713 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:50:29.098946 run osmo-gsm-tester_setcap_net_admin.sh(pid=27235): Launched
18:50:30.287309 run osmo-gsm-tester_setcap_net_admin.sh(pid=27235): Terminated: ok {rc=0}
18:50:30.624080 run  osmo-ggsn_10.42.42.6(pid=27238): Launched
18:50:30.747704 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:50:31.422082 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Launched
18:50:31.725937 run  osmo-sgsn_10.42.42.7(pid=27261): Launched
18:50:31.841137 run              osmo-msc_10.42.42.8: Starting osmo-msc
18:50:32.403724 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Launched
18:50:32.705078 run   osmo-msc_10.42.42.8(pid=27263): Launched
18:50:35.875207 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
18:50:36.389750 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Launched
18:50:36.692742 run   osmo-mgw_10.42.42.3(pid=27265): Launched
18:50:36.808625 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
18:50:37.312282 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Launched
18:50:37.615382 run   osmo-mgw_10.42.42.4(pid=27267): Launched
18:50:37.731142 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
18:50:38.355954 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Launched
18:50:38.658582 run   osmo-bsc_10.42.42.9(pid=27269): Launched
18:50:44.924840 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:50:45.450442 run         rm-remote-dir(pid=27270): Launched
18:50:47.630057 run         rm-remote-dir(pid=27270): Terminated: ok {rc=0}
18:50:47.962609 run         mk-remote-dir(pid=27271): Launched
18:50:49.139843 run         mk-remote-dir(pid=27271): Terminated: ok {rc=0}
18:50:49.423231 run    scp-inst-to-remote(pid=27272): Launched
18:51:01.614934 run    scp-inst-to-remote(pid=27272): Terminated: ok {rc=0}
18:51:01.965540 run         mk-remote-dir(pid=27274): Launched
18:51:03.153689 run         mk-remote-dir(pid=27274): Terminated: ok {rc=0}
18:51:03.454827 run     scp-cfg-to-remote(pid=27275): Launched
18:51:04.643143 run     scp-cfg-to-remote(pid=27275): Terminated: ok {rc=0}
18:51:04.990265 run   reload-dsp-firmware(pid=27277): Launched
18:51:07.177513 run   reload-dsp-firmware(pid=27277): Terminated: ok {rc=0}
18:51:07.517467 run        osmo-bts-sysmo(pid=27278): Launched
18:51:11.754838 tst                       ping.py:31: Waiting for bts to be ready...
18:51:13.470836 run         rm-remote-dir(pid=27279): Launched
18:51:15.651499 run         rm-remote-dir(pid=27279): Terminated: ok {rc=0}
18:51:15.970610 run         mk-remote-dir(pid=27280): Launched
18:51:18.141366 run         mk-remote-dir(pid=27280): Terminated: ok {rc=0}
18:51:18.414515 run       scp-inst-to-bts(pid=27281): Launched
18:51:31.606348 run       scp-inst-to-bts(pid=27281): Terminated: ok {rc=0}
18:51:31.949063 run     mk-remote-run-dir(pid=27283): Launched
18:51:34.134158 run     mk-remote-run-dir(pid=27283): Terminated: ok {rc=0}
18:51:34.426639 run   scp-cfg-to-sysmobts(pid=27284): Launched
18:51:36.603879 run   scp-cfg-to-sysmobts(pid=27284): Terminated: ok {rc=0}
18:51:36.922588 run              osmo-pcu(pid=27286): Launched
18:51:37.045170 tst                          ping.py: using MSISDN 3956
18:51:37.217912 bus                          /gobi_6: Setting Powered True
18:51:40.726402 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3956', subscriber_id=1}
18:51:40.987713 bus                          /gobi_6: Setting Online False
18:51:42.173276 bus                          /gobi_6: Setting Powered False
18:51:49.105245 bus                          /gobi_6: Setting Powered True
18:51:50.463321 bus                          /gobi_6: Setting Online True
18:51:52.875153 tst                          /gobi_6: Connect to ('901', '70')
18:51:53.151325 tst                          /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G  1  [Jul  5 2017 10:49:17]', Serial: '868323024273648'
18:51:53.268187 tst                       ping.py:42: waiting for modems to attach...
18:52:07.461869 tst                          /gobi_6: Already registered with network ('901', '70')
18:52:12.803006 tst                       ping.py:46: waiting for modems to attach to data services...
18:52:19.982260 tst                          /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:52:20.089653 tst                       ping.py:51: Setting up data plan for "'/gobi_6/context3'"
18:52:20.497538 run            move_netns(pid=27290): Launched
18:52:21.673417 run            move_netns(pid=27290): Terminated: ok {rc=0}
18:52:21.995974 run                  ifup(pid=27293): Launched
18:52:23.170671 run                  ifup(pid=27293): Terminated: ok {rc=0}
18:52:23.516847 run                  dhcp(pid=27296): Launched
18:52:24.701866 run                  dhcp(pid=27296): Terminated: ok {rc=0}
18:52:24.829168 tst                       ping.py:53: Running 10 ping requests for "'/gobi_6/context3'"
18:52:25.221334 run                  ping(pid=27312): Launched
18:52:35.416106 run                  ping(pid=27312): Terminated: ok {rc=0}
18:52:37.180118 tst                          /gobi_6: context deactivated {path='/gobi_6/context3'}
18:52:44.456774 tst                          /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={Interface='r433306bbc2604b'}, Name='Internet', Password='', Protocol='ipv6', Settings={}, Type='internet', Username='ogt'}, user='ogt'}
18:52:52.219378 tst                          /gobi_6: context deactivated {path='/gobi_6/context3'}
18:52:52.337523 tst                       ping.py:70: (launched: 2022-03-29_18:52:25.160958)
27313: waiting for 27314
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=943 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=584 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=585 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=581 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=563 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=581 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=577 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=560 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=578 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=574 ms

--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 18ms
rtt min/avg/max/mdev = 559.738/612.790/943.318/110.469 ms
child exited with 0
18:52:52.558240 tst                          ping.py: Test passed (150.5 sec)
18:52:52.626225 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Terminating (SIGTERM)
18:52:52.669513 run   osmo-hlr_10.42.42.2(pid=27230): Terminating (SIGTERM)
18:52:52.713004 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Terminating (SIGTERM)
18:52:52.758484 run   osmo-stp_10.42.42.5(pid=27232): Terminating (SIGTERM)
18:52:52.805607 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Terminating (SIGTERM)
18:52:52.871714 run  osmo-ggsn_10.42.42.6(pid=27238): Terminating (SIGTERM)
18:52:52.918210 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Terminating (SIGTERM)
18:52:52.992420 run  osmo-sgsn_10.42.42.7(pid=27261): Terminating (SIGTERM)
18:52:53.037946 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Terminating (SIGTERM)
18:52:53.083257 run   osmo-msc_10.42.42.8(pid=27263): Terminating (SIGTERM)
18:52:53.127270 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Terminating (SIGTERM)
18:52:53.171173 run   osmo-mgw_10.42.42.3(pid=27265): Terminating (SIGTERM)
18:52:53.215766 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Terminating (SIGTERM)
18:52:53.259448 run   osmo-mgw_10.42.42.4(pid=27267): Terminating (SIGTERM)
18:52:53.301765 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Terminating (SIGTERM)
18:52:53.344995 run   osmo-bsc_10.42.42.9(pid=27269): Terminating (SIGTERM)
18:52:53.387124 run        osmo-bts-sysmo(pid=27278): Terminating (SIGTERM)
18:52:53.429318 run              osmo-pcu(pid=27286): Terminating (SIGTERM)
18:52:53.472420 ---      ParallelTerminationStrategy: PID 27229 died...
18:52:53.492815 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27229): Terminated: ok {rc=0}
18:52:53.539802 ---      ParallelTerminationStrategy: PID 27230 died...
18:52:53.559815 run   osmo-hlr_10.42.42.2(pid=27230): Terminated: ok {rc=0}
18:52:53.606344 ---      ParallelTerminationStrategy: PID 27231 died...
18:52:53.626146 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27231): Terminated: ok {rc=0}
18:52:53.672982 ---      ParallelTerminationStrategy: PID 27232 died...
18:52:53.692773 run   osmo-stp_10.42.42.5(pid=27232): Terminated: ok {rc=0}
18:52:53.739317 ---      ParallelTerminationStrategy: PID 27233 died...
18:52:53.759222 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27233): Terminated: ok {rc=0}
18:52:53.806385 ---      ParallelTerminationStrategy: PID 27238 died...
18:52:53.827039 run  osmo-ggsn_10.42.42.6(pid=27238): Terminated: ok {rc=0}
18:52:53.875159 ---      ParallelTerminationStrategy: PID 27260 died...
18:52:53.895859 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27260): Terminated: ok {rc=0}
18:52:53.945206 ---      ParallelTerminationStrategy: PID 27262 died...
18:52:53.966123 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27262): Terminated: ok {rc=0}
18:52:54.027165 ---      ParallelTerminationStrategy: PID 27264 died...
18:52:54.080419 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27264): Terminated: ok {rc=0}
18:52:54.171259 ---      ParallelTerminationStrategy: PID 27265 died...
18:52:54.228114 run   osmo-mgw_10.42.42.3(pid=27265): Terminated {rc=15}
18:52:54.276430 ---      ParallelTerminationStrategy: PID 27263 died...
18:52:54.297171 run   osmo-msc_10.42.42.8(pid=27263): Terminated: ok {rc=0}
18:52:54.345432 ---      ParallelTerminationStrategy: PID 27266 died...
18:52:54.366278 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27266): Terminated: ok {rc=0}
18:52:54.414423 ---      ParallelTerminationStrategy: PID 27267 died...
18:52:54.435201 run   osmo-mgw_10.42.42.4(pid=27267): Terminated {rc=15}
18:52:54.483808 ---      ParallelTerminationStrategy: PID 27268 died...
18:52:54.504445 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27268): Terminated: ok {rc=0}
18:52:54.552566 ---      ParallelTerminationStrategy: PID 27269 died...
18:52:54.573226 run   osmo-bsc_10.42.42.9(pid=27269): Terminated: ok {rc=0}
18:52:54.621660 ---      ParallelTerminationStrategy: PID 27278 died...
18:52:54.642676 run        osmo-bts-sysmo(pid=27278): Terminated: ok {rc=0}
18:52:54.691239 ---      ParallelTerminationStrategy: PID 27286 died...
18:52:54.712056 run              osmo-pcu(pid=27286): Terminated: ok {rc=0}
18:52:54.928817 ---      ParallelTerminationStrategy: PID 27261 died...
18:52:54.950844 run  osmo-sgsn_10.42.42.7(pid=27261): Terminated: ok {rc=0}
18:52:55.176451 bus                          /gobi_6: Setting Online False
18:52:58.227734 bus                          /gobi_6: Setting Powered False
18:53:14.935274 bus                          /gobi_6: Setting Powered False

----------------------------------------------
trial-2064 gprs:sysmo ping_idle_ping.py
----------------------------------------------
18:53:16.079068 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.090959 tst                       gprs:sysmo: Using 1 x bts (candidates: 1)
18:53:16.126507 tst                ping_idle_ping.py: using LAC 3949
18:53:16.254656 tst                ping_idle_ping.py: using RAC 124
18:53:16.384775 tst                ping_idle_ping.py: using CellId 3949
18:53:16.505449 tst                ping_idle_ping.py: using BVCI 3950
18:53:16.616094 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.626804 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.637471 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.648230 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.658742 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.669406 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.680100 tst                       gprs:sysmo: Using 1 x ip_address (candidates: 9)
18:53:16.690661 tst                       gprs:sysmo: Using 1 x modem (candidates: 4)
18:53:16.808349 tst             ping_idle_ping.py:24: start network...
18:53:17.028600 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
18:53:17.533122 run         create_hlr_db(pid=27333): Launched
18:53:17.658259 bus                          /gobi_6: Setting Powered False
18:53:18.723381 run         create_hlr_db(pid=27333): Terminated: ok {rc=0}
18:53:19.159982 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Launched
18:53:19.466114 run   osmo-hlr_10.42.42.2(pid=27336): Launched
18:53:19.584068 run              osmo-stp_10.42.42.5: Starting osmo-stp
18:53:20.105723 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Launched
18:53:20.411899 run   osmo-stp_10.42.42.5(pid=27338): Launched
18:53:20.530549 run             osmo-ggsn_10.42.42.6: Starting osmo-ggsn
18:53:21.050559 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Launched
18:53:21.167295 run             osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
18:53:21.424726 run              patchelf(pid=27340): Launched
18:53:22.597045 run              patchelf(pid=27340): Terminated: ok {rc=0}
18:53:22.724490 run             osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn
18:53:23.004661 run osmo-gsm-tester_setcap_net_admin.sh(pid=27341): Launched
18:53:24.182487 run osmo-gsm-tester_setcap_net_admin.sh(pid=27341): Terminated: ok {rc=0}
18:53:24.497812 run  osmo-ggsn_10.42.42.6(pid=27344): Launched
18:53:24.622209 run             osmo-sgsn_10.42.42.7: Starting osmo-sgsn
18:53:25.283124 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Launched
18:53:25.587362 run  osmo-sgsn_10.42.42.7(pid=27362): Launched
18:53:25.703244 run              osmo-msc_10.42.42.8: Starting osmo-msc
18:53:26.254298 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Launched
18:53:26.559736 run   osmo-msc_10.42.42.8(pid=27364): Launched
18:53:29.731189 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
18:53:30.249747 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Launched
18:53:30.553884 run   osmo-mgw_10.42.42.3(pid=27366): Launched
18:53:30.670312 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
18:53:31.176762 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Launched
18:53:31.480988 run   osmo-mgw_10.42.42.4(pid=27368): Launched
18:53:31.597684 run              osmo-bsc_10.42.42.9: Starting osmo-bsc
18:53:32.224502 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Launched
18:53:32.528231 run   osmo-bsc_10.42.42.9(pid=27370): Launched
18:53:38.797577 run                   osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9
18:53:39.327861 run         rm-remote-dir(pid=27371): Launched
18:53:41.511474 run         rm-remote-dir(pid=27371): Terminated: ok {rc=0}
18:53:41.872872 run         mk-remote-dir(pid=27372): Launched
18:53:43.063839 run         mk-remote-dir(pid=27372): Terminated: ok {rc=0}
18:53:43.371396 run    scp-inst-to-remote(pid=27373): Launched
18:53:55.581549 run    scp-inst-to-remote(pid=27373): Terminated: ok {rc=0}
18:53:55.938539 run         mk-remote-dir(pid=27375): Launched
18:53:57.126607 run         mk-remote-dir(pid=27375): Terminated: ok {rc=0}
18:53:57.431550 run     scp-cfg-to-remote(pid=27376): Launched
18:53:58.619835 run     scp-cfg-to-remote(pid=27376): Terminated: ok {rc=0}
18:53:58.970296 run   reload-dsp-firmware(pid=27378): Launched
18:54:01.158887 run   reload-dsp-firmware(pid=27378): Terminated: ok {rc=0}
18:54:01.502246 run        osmo-bts-sysmo(pid=27379): Launched
18:54:05.742859 tst             ping_idle_ping.py:36: Waiting for bts to be ready...
18:54:07.463034 run         rm-remote-dir(pid=27380): Launched
18:54:09.647121 run         rm-remote-dir(pid=27380): Terminated: ok {rc=0}
18:54:09.990246 run         mk-remote-dir(pid=27383): Launched
18:54:12.172710 run         mk-remote-dir(pid=27383): Terminated: ok {rc=0}
18:54:12.446443 run       scp-inst-to-bts(pid=27384): Launched
18:54:25.631617 run       scp-inst-to-bts(pid=27384): Terminated: ok {rc=0}
18:54:25.953028 run     mk-remote-run-dir(pid=27386): Launched
18:54:28.124103 run     mk-remote-run-dir(pid=27386): Terminated: ok {rc=0}
18:54:28.397430 run   scp-cfg-to-sysmobts(pid=27387): Launched
18:54:30.569468 run   scp-cfg-to-sysmobts(pid=27387): Terminated: ok {rc=0}
18:54:30.912395 run              osmo-pcu(pid=27389): Launched
18:54:31.049751 tst                ping_idle_ping.py: using MSISDN 3957
18:54:31.223360 bus                          /gobi_6: Setting Powered True
18:54:34.734287 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3957', subscriber_id=1}
18:54:34.997006 bus                          /gobi_6: Setting Online False
18:54:36.183822 bus                          /gobi_6: Setting Powered False
18:54:43.108849 bus                          /gobi_6: Setting Powered True
18:54:44.475734 bus                          /gobi_6: Setting Online True
18:54:46.966328 tst                          /gobi_6: Connect to ('901', '70')
18:54:47.251806 tst                          /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G  1  [Jul  5 2017 10:49:17]', Serial: '868323024273648'
18:54:47.368116 tst             ping_idle_ping.py:47: waiting for modems to attach...
18:54:54.342296 tst                          /gobi_6: Already registered with network ('901', '70')
18:55:05.838172 tst             ping_idle_ping.py:51: waiting for modems to attach to data services...
18:55:58.827522 tst                          /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'}
18:55:58.946065 tst             ping_idle_ping.py:56: Setting up data plane for "'/gobi_6/context2'"
18:55:59.392822 run            move_netns(pid=27396): Launched
18:56:00.587265 run            move_netns(pid=27396): Terminated: ok {rc=0}
18:56:00.941440 run                  ifup(pid=27399): Launched
18:56:02.123449 run                  ifup(pid=27399): Terminated: ok {rc=0}
18:56:02.450444 run                  dhcp(pid=27402): Launched
18:56:03.626231 run                  dhcp(pid=27402): Terminated: ok {rc=0}
18:56:03.755621 tst             ping_idle_ping.py:60: [1] Running 10 ping requests for "'/gobi_6/context2'"
18:56:04.155091 run                 ping1(pid=27418): Launched
18:56:14.359359 run                 ping1(pid=27418): Terminated: ok {rc=0}
18:56:14.495867 tst             ping_idle_ping.py:64: (launched: 2022-03-29_18:56:04.093106)
27419: waiting for 27420
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=679 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=638 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=615 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=596 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=592 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=532 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=533 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=492 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=488 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=487 ms

--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 21ms
rtt min/avg/max/mdev = 486.902/565.056/678.590/64.750 ms
child exited with 0
18:56:14.718200 tst             ping_idle_ping.py:68: Sleeping for 60 seconds
18:57:14.941764 tst             ping_idle_ping.py:73: [2] Running 10 ping requests for "'/gobi_6/context2'"
18:57:15.372936 run                 ping2(pid=27424): Launched
18:57:25.578233 run                 ping2(pid=27424): Terminated: ok {rc=0}
18:57:25.714790 tst             ping_idle_ping.py:77: (launched: 2022-03-29_18:57:15.307744)
27425: waiting for 27426
PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data.
64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=536 ms
64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=454 ms
64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=431 ms
64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=413 ms
64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=431 ms
64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=427 ms
64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=408 ms
64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=405 ms
64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=405 ms
64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=381 ms

--- 10.42.42.6 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 21ms
rtt min/avg/max/mdev = 381.454/429.068/536.304/40.329 ms
child exited with 0
18:57:27.707993 tst                          /gobi_6: context deactivated {path='/gobi_6/context2'}
18:57:27.817824 tst                ping_idle_ping.py: Test passed (251.7 sec)
18:57:27.883815 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Terminating (SIGTERM)
18:57:27.925168 run   osmo-hlr_10.42.42.2(pid=27336): Terminating (SIGTERM)
18:57:27.966164 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Terminating (SIGTERM)
18:57:28.009810 run   osmo-stp_10.42.42.5(pid=27338): Terminating (SIGTERM)
18:57:28.059656 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Terminating (SIGTERM)
18:57:28.105538 run  osmo-ggsn_10.42.42.6(pid=27344): Terminating (SIGTERM)
18:57:28.150157 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Terminating (SIGTERM)
18:57:28.210933 run  osmo-sgsn_10.42.42.7(pid=27362): Terminating (SIGTERM)
18:57:28.278431 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Terminating (SIGTERM)
18:57:28.325669 run   osmo-msc_10.42.42.8(pid=27364): Terminating (SIGTERM)
18:57:28.371429 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Terminating (SIGTERM)
18:57:28.416376 run   osmo-mgw_10.42.42.3(pid=27366): Terminating (SIGTERM)
18:57:28.461094 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Terminating (SIGTERM)
18:57:28.507834 run   osmo-mgw_10.42.42.4(pid=27368): Terminating (SIGTERM)
18:57:28.551819 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Terminating (SIGTERM)
18:57:28.596373 run   osmo-bsc_10.42.42.9(pid=27370): Terminating (SIGTERM)
18:57:28.639641 run        osmo-bts-sysmo(pid=27379): Terminating (SIGTERM)
18:57:28.683368 run              osmo-pcu(pid=27389): Terminating (SIGTERM)
18:57:28.727835 ---      ParallelTerminationStrategy: PID 27335 died...
18:57:28.748547 run pcap-recorder_any(filters='host 10.42.42.2')(pid=27335): Terminated: ok {rc=0}
18:57:28.797671 ---      ParallelTerminationStrategy: PID 27336 died...
18:57:28.818327 run   osmo-hlr_10.42.42.2(pid=27336): Terminated: ok {rc=0}
18:57:28.867859 ---      ParallelTerminationStrategy: PID 27337 died...
18:57:28.888557 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=27337): Terminated: ok {rc=0}
18:57:28.938065 ---      ParallelTerminationStrategy: PID 27338 died...
18:57:28.958857 run   osmo-stp_10.42.42.5(pid=27338): Terminated: ok {rc=0}
18:57:29.008339 ---      ParallelTerminationStrategy: PID 27339 died...
18:57:29.028524 run pcap-recorder_any(filters='host 10.42.42.6')(pid=27339): Terminated: ok {rc=0}
18:57:29.076642 ---      ParallelTerminationStrategy: PID 27344 died...
18:57:29.097617 run  osmo-ggsn_10.42.42.6(pid=27344): Terminated: ok {rc=0}
18:57:29.145956 ---      ParallelTerminationStrategy: PID 27361 died...
18:57:29.166800 run pcap-recorder_any(filters='host 10.42.42.7')(pid=27361): Terminated: ok {rc=0}
18:57:29.215559 ---      ParallelTerminationStrategy: PID 27363 died...
18:57:29.236338 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=27363): Terminated: ok {rc=0}
18:57:29.285039 ---      ParallelTerminationStrategy: PID 27364 died...
18:57:29.305812 run   osmo-msc_10.42.42.8(pid=27364): Terminated: ok {rc=0}
18:57:29.354191 ---      ParallelTerminationStrategy: PID 27365 died...
18:57:29.375437 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=27365): Terminated: ok {rc=0}
18:57:29.424266 ---      ParallelTerminationStrategy: PID 27366 died...
18:57:29.445247 run   osmo-mgw_10.42.42.3(pid=27366): Terminated {rc=15}
18:57:29.493765 ---      ParallelTerminationStrategy: PID 27367 died...
18:57:29.514624 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=27367): Terminated: ok {rc=0}
18:57:29.563536 ---      ParallelTerminationStrategy: PID 27368 died...
18:57:29.584396 run   osmo-mgw_10.42.42.4(pid=27368): Terminated {rc=15}
18:57:29.632948 ---      ParallelTerminationStrategy: PID 27369 died...
18:57:29.653818 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=27369): Terminated: ok {rc=0}
18:57:29.702845 ---      ParallelTerminationStrategy: PID 27370 died...
18:57:29.723571 run   osmo-bsc_10.42.42.9(pid=27370): Terminated: ok {rc=0}
18:57:29.772222 ---      ParallelTerminationStrategy: PID 27379 died...
18:57:29.793199 run        osmo-bts-sysmo(pid=27379): Terminated: ok {rc=0}
18:57:29.842106 ---      ParallelTerminationStrategy: PID 27389 died...
18:57:29.863251 run              osmo-pcu(pid=27389): Terminated: ok {rc=0}
18:57:30.080476 ---      ParallelTerminationStrategy: PID 27362 died...
18:57:30.102519 run  osmo-sgsn_10.42.42.7(pid=27362): Terminated: ok {rc=0}
18:57:30.325893 bus                          /gobi_6: Setting Online False
18:57:33.386328 bus                          /gobi_6: Setting Powered False
18:57:49.001216 bus                          /gobi_6: Setting Powered False
---------------------------------------------------------------------
trial-2064 gprs:sysmo FAIL
---------------------------------------------------------------------
18:57:50.341374 tst                       trial-2064: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064/run.2022-03-29_18-32-52/trial-2064.xml

--------------------------------------------------------------------------------------------
trial-2064 FAIL
--------------------------------------------------------------------------------------------
18:57:50.571122 tst                       trial-2064: trial-2064: FAIL, 2 suites failed, 1 suites passed
FAIL: nitb_sms:sysmo (fail: 1)
    FAIL: mo_mt_sms.py (5.1 sec) Error: osmo-nitb_10.42.42.2(pid=26640): Process ended prematurely: osmo-nitb_10.42.42.2(pid=26640) [trial-2064↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=26640)]
FAIL: gprs:sysmo (fail: 3, pass: 2)
    FAIL: cs_paging_gprs_active.py (228.9 sec) KeyError: 'sum_sent'
    FAIL: iperf3.py (196.8 sec) KeyError: 'sum_sent'
    FAIL: iperf3m4.py (294.8 sec) KeyError: 'sum_sent'
    pass: ping.py (150.5 sec)
    pass: ping_idle_ping.py (251.7 sec)
PASS: sms:sysmo (pass: 1)
    pass: mo_mt_sms.py (116.8 sec)
+ rm -rf trial-2064/inst
+ cd trial-2064
+ command -v journalctl
+ [ ! -z /bin/journalctl ]
+ readlink last_run
+ journalctl -u ofono -o short-precise --since 2022-03-29 18:32:50
+ readlink last_run
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064-run.tgz run.2022-03-29_18-32-52
+ tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2064-bin.tgz checksums.md5 open5gs.build-8.tgz osmo-bsc.build-15570.tgz osmo-bts-oc2g.build-4604.tgz osmo-bts-sysmo.build-7108.tgz osmo-bts.build-6578.tgz osmo-ggsn.build-5814.tgz osmo-hlr.build-7536.tgz osmo-hnbgw.build-592.tgz osmo-mgw.build-7731.tgz osmo-msc.build-25979.tgz osmo-nitb.build-11307.tgz osmo-pcu-oc2g.build-3308.tgz osmo-pcu-sysmo.build-4589.tgz osmo-pcu.build-4550.tgz osmo-sgsn.build-26313.tgz osmo-stp.build-8062.tgz osmo-trx.build-3840.tgz osmocom-bb.build-4035.tgz srslte.build-19.tgz
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
ERROR Gerrit response: ConflictFinished: FAILURE