![[Jenkins]](/jenkins/static/b68c8964/images/svgs/logo.svg){kind=logo}
Triggered by Gerrit: https://gerrit.osmocom.org/c/osmo-gsm-tester/+/29282 Running as SYSTEM Building remotely on osmo-gsm-tester-prod (osmo-gsm-tester) in workspace /home/jenkins/workspace/osmo-gsm-tester_gerrit The recommended git tool is: NONE using credential d5eda5e9-b59d-44ba-88d2-43473cb6e42d > git rev-parse --resolve-git-dir /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/.git # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Fetching upstream changes from ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester > git --version # timeout=10 > git --version # 'git version 2.20.1' using GIT_SSH to set credentials > git fetch --tags --force --progress -- ssh://jenkins@gerrit.osmocom.org:29418/osmo-gsm-tester refs/changes/82/29282/1 # timeout=10 > git rev-parse d577d68365d0ec0275926b0aaa4a34ea5fc623a7^{commit} # timeout=10 JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://plugins.jenkins.io/git/#remove-git-plugin-buildsbybranch-builddata-script Checking out Revision d577d68365d0ec0275926b0aaa4a34ea5fc623a7 (refs/changes/82/29282/1) > git config core.sparsecheckout # timeout=10 > git checkout -f d577d68365d0ec0275926b0aaa4a34ea5fc623a7 # timeout=10 Commit message: "contrib: osmo-bts now depends on libosmo-netif" > git rev-parse FETCH_HEAD^{commit} # timeout=10 > git rev-list --no-walk 882b8aee01a79c8505d6028e881e7101c319c704 # timeout=10 [osmo-gsm-tester_gerrit] $ /bin/sh -xe /tmp/jenkins666047743048590055.sh + rm -f trial-2065-bin.tgz trial-2065-run.tgz *.md5 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bsc" build number 16888 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts" build number 7166 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-ggsn" build number 6364 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hlr" build number 8262 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-mgw" build number 8470 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-hnbgw" build number 2020 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-msc" build number 28078 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu" build number 4933 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-sgsn" build number 28683 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-trx" build number 4360 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-nitb" build number 12346 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-stp" build number 8799 Copied 2 artifacts from "osmo-gsm-tester_build-osmocom-bb" build number 4442 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-sysmo" build number 7696 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-sysmo" build number 4971 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-bts-oc2g" build number 5192 Copied 2 artifacts from "osmo-gsm-tester_build-osmo-pcu-oc2g" build number 3690 Copied 2 artifacts from "osmo-gsm-tester_build-open5gs" build number 8 Copied 2 artifacts from "osmo-gsm-tester_build-srslte" build number 19 [osmo-gsm-tester_gerrit] $ /bin/sh /tmp/jenkins7842538596273757768.sh + unlink osmo-gsm-tester/sysmocom/resources.conf unlink: cannot unlink 'osmo-gsm-tester/sysmocom/resources.conf': No such file or directory + true + ln -s resources.conf.prod osmo-gsm-tester/sysmocom/resources.conf + export OSMO_GSM_TESTER_OPTS=-s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo + ./osmo-gsm-tester/contrib/jenkins-make-check-and-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit + run_script=osmo-gsm-tester/contrib/jenkins-run.sh + test -x osmo-gsm-tester/contrib/jenkins-run.sh + cd osmo-gsm-tester + make deps ./contrib/check_dependencies.py Skip checking modules: [] Changing workdir dir to /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_ms_driver importing src.osmo_ms_driver.location_update_test importing src.osmo_ms_driver.event_server importing src.osmo_ms_driver.cdf importing src.osmo_ms_driver.__main__ importing src.osmo_ms_driver.__init__ importing src.osmo_ms_driver.test_support importing src.osmo_ms_driver.starter importing src.osmo_ms_driver.simple_loop importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/core importing src.osmo_gsm_tester.core.util importing src.osmo_gsm_tester.core.schema importing src.osmo_gsm_tester.core.trial importing src.osmo_gsm_tester.core.template importing src.osmo_gsm_tester.core.config importing src.osmo_gsm_tester.core.event_loop importing src.osmo_gsm_tester.core.log importing src.osmo_gsm_tester.core.scenario importing src.osmo_gsm_tester.core.remote importing src.osmo_gsm_tester.core.__init__ importing src.osmo_gsm_tester.core.process importing src.osmo_gsm_tester.core.report importing src.osmo_gsm_tester.core.test importing src.osmo_gsm_tester.core.resource importing src.osmo_gsm_tester.core.suite importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester/obj importing src.osmo_gsm_tester.obj.android_apn importing src.osmo_gsm_tester.obj.bts importing src.osmo_gsm_tester.obj.mgcpgw_osmo importing src.osmo_gsm_tester.obj.sgsn_osmo importing src.osmo_gsm_tester.obj.epc_srs importing src.osmo_gsm_tester.obj.bts_nanobts importing src.osmo_gsm_tester.obj.enb_srs importing src.osmo_gsm_tester.obj.esme importing src.osmo_gsm_tester.obj.qc_diag importing src.osmo_gsm_tester.obj.stp_osmo importing src.osmo_gsm_tester.obj.ms_ofono importing src.osmo_gsm_tester.obj.ms_android importing src.osmo_gsm_tester.obj.osmo_vty importing src.osmo_gsm_tester.obj.hlr_osmo importing src.osmo_gsm_tester.obj.enb_amarisoft importing src.osmo_gsm_tester.obj.android_host importing src.osmo_gsm_tester.obj.sgwu_open5gs importing src.osmo_gsm_tester.obj.powersupply importing src.osmo_gsm_tester.obj.iperf3 importing src.osmo_gsm_tester.obj.epc importing src.osmo_gsm_tester.obj.rfemu_minicircuits importing src.osmo_gsm_tester.obj.bts_osmo importing src.osmo_gsm_tester.obj.upf_open5gs importing src.osmo_gsm_tester.obj.nitb_osmo importing src.osmo_gsm_tester.obj.android_bitrate_monitor importing src.osmo_gsm_tester.obj.pcu_oc2g importing src.osmo_gsm_tester.obj.ms_srs importing src.osmo_gsm_tester.obj.ms_driver importing src.osmo_gsm_tester.obj.bts_sysmo importing src.osmo_gsm_tester.obj.rfemu_srsenb_stdin importing src.osmo_gsm_tester.obj.pcu importing src.osmo_gsm_tester.obj.osmocon importing src.osmo_gsm_tester.obj.epc_amarisoft importing src.osmo_gsm_tester.obj.ggsn_osmo importing src.osmo_gsm_tester.obj.enb importing src.osmo_gsm_tester.obj.rfemu importing src.osmo_gsm_tester.obj.bts_octphy importing src.osmo_gsm_tester.obj.pcap_recorder importing src.osmo_gsm_tester.obj.sms importing src.osmo_gsm_tester.obj.bts_oc2g importing src.osmo_gsm_tester.obj.stress importing src.osmo_gsm_tester.obj.smsc importing src.osmo_gsm_tester.obj.msc_osmo importing src.osmo_gsm_tester.obj.sgwc_open5gs importing src.osmo_gsm_tester.obj.osmo_ctrl importing src.osmo_gsm_tester.obj.bts_osmotrx importing src.osmo_gsm_tester.obj.rfemu_amarisoftctrl importing src.osmo_gsm_tester.obj.ms importing src.osmo_gsm_tester.obj.powersupply_intellinet importing src.osmo_gsm_tester.obj.ms_amarisoft importing src.osmo_gsm_tester.obj.srslte_common importing src.osmo_gsm_tester.obj.powersupply_sispm importing src.osmo_gsm_tester.obj.gnuradio_zmq_broker importing src.osmo_gsm_tester.obj.pcu_sysmo importing src.osmo_gsm_tester.obj.bts_osmovirtual importing src.osmo_gsm_tester.obj.hss_open5gs importing src.osmo_gsm_tester.obj.mgw_osmo importing src.osmo_gsm_tester.obj.smf_open5gs importing src.osmo_gsm_tester.obj.pcrf_open5gs importing src.osmo_gsm_tester.obj.mme_open5gs importing src.osmo_gsm_tester.obj.bsc_osmo importing src.osmo_gsm_tester.obj.epc_open5gs importing src.osmo_gsm_tester.obj.run_node importing src.osmo_gsm_tester.obj.pcu_osmo importing src.osmo_gsm_tester.obj.rfemu_gnuradio_zmq importing src.osmo_gsm_tester.obj.ms_osmo_mobile importing files in directory /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo_gsm_tester importing src.osmo_gsm_tester.testenv importing src.osmo_gsm_tester.__init__ Importing dependencies ok, all installed Retreiving list of imported modules... + make check make -C selftest check make[1]: Entering directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest' echo "export PYTHONPATH=\"/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/../src\"" > set_pythonpath ./all_tests.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/cdf_test/cdf_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/config_test/config_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/lock_test/lock_test.sh /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/log_test/log_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/process_test/process_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/rate_ctrs_test/rate_ctrs_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/report_test/report_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/resource_test/resource_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/scenario_test/scenario_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/schema_test/schema_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/sms_test/sms_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/suite_test/suite_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/template_test/template_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/trial_test/trial_test.py /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest/util_test/util_test.py 15 tests ok make[1]: Leaving directory '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/selftest' make check: success + ./contrib/jenkins-build-manuals.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester + export OSMO_GSM_MANUALS_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals ] + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals ] + git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals pull warning: redirecting to https://gitea.osmocom.org/cellular-infrastructure/osmo-gsm-manuals/ From https://git.osmocom.org/osmo-gsm-manuals 9ad5777..35b5e27 master -> origin/master * [new tag] 1.3.0 -> 1.3.0 Updating 9ad5777..35b5e27 Fast-forward common/chapters/port_numbers.adoc | 1 + common/chapters/sigtran.adoc | 6 +++--- common/chapters/trx_if.adoc | 28 ++++++++++++++-------------- debian/changelog | 24 ++++++++++++++++++++++++ 4 files changed, 42 insertions(+), 17 deletions(-) + git -C /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals checkout -f HEAD Your branch is up to date with 'origin/master'. + temp=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + [ -d /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp ] + cp -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/doc/manuals /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + cd /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + make OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env) ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build build ln -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common common # a2x can't use a different output file. To support out-of-tree builds, # we create a symlink at /srcfile.adoc pointing at # /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc. osmo-gsm-tester-manual.adoc is the /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/srcfile.adoc, # is like basename from coreutils, and is WD. if ! [ -f osmo-gsm-tester-manual.adoc ]; then \ ln -s osmo-gsm-tester-manual.adoc osmo-gsm-tester-manual.adoc; \ fi # TEXINPUTS: find LaTeX includes like \includegraphics{./common/images/sysmocom.pdf} # in /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals. TEXINPUTS="/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals" \ a2x -vv -L --asciidoc-opts="-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'" --dblatex-opts="-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0" -a docinfo -a revnumber="DRAFT 0.1-1216-gd577" -a revdate="2022-Sep-07" osmo-gsm-tester-manual.adoc a2x: args: ['-vv', '-L', "--asciidoc-opts=-f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common'", '--dblatex-opts=-s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0', '-a', 'docinfo', '-a', 'revnumber=DRAFT 0.1-1216-gd577', '-a', 'revdate=2022-Sep-07', 'osmo-gsm-tester-manual.adoc'] a2x: resource files: [] a2x: resource directories: ['/etc/asciidoc/images', '/etc/asciidoc/stylesheets'] a2x: executing: "/usr/bin/asciidoc" --backend docbook -a "a2x-format=pdf" -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf -f /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf -a srcdir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' -a commondir='/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common' --attribute "docinfo" --attribute "revnumber=DRAFT 0.1-1216-gd577" --attribute "revdate=2022-Sep-07" --verbose --out-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml" "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc" asciidoc: reading: /etc/asciidoc/asciidoc.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf asciidoc: reading: /etc/asciidoc/asciidoc.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.adoc asciidoc: reading: /etc/asciidoc/docbook45.conf asciidoc: reading: /etc/asciidoc/filters/code/code-filter.conf asciidoc: reading: /etc/asciidoc/filters/graphviz/graphviz-filter.conf asciidoc: reading: /etc/asciidoc/filters/music/music-filter.conf asciidoc: reading: /etc/asciidoc/filters/source/source-highlight-filter.conf asciidoc: reading: /etc/asciidoc/filters/latex/latex-filter.conf asciidoc: reading: /etc/asciidoc/lang-en.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf asciidoc: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf asciidoc: writing: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml asciidoc: osmo-gsm-tester-manual.adoc: line 5: evaluating: {include:/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual-docinfo.xml} asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 13: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/intro.adoc asciidoc: intro.adoc: line 55: evaluating: {counter2:target-number} asciidoc: intro.adoc: line 55: evaluating: {set2:target:osmo-gsm-tester-manual__1.png} asciidoc: intro.adoc: line 103: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__1.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 15: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/trial.adoc asciidoc: trial.adoc: line 14: evaluating: {counter2:target-number} asciidoc: trial.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__2.png} asciidoc: trial.adoc: line 25: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__2.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 17: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 19: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/resource_pool.adoc asciidoc: resource_pool.adoc: line 14: evaluating: {counter2:target-number} asciidoc: resource_pool.adoc: line 14: evaluating: {set2:target:osmo-gsm-tester-manual__3.png} asciidoc: resource_pool.adoc: line 79: filtering: "/usr/bin/python2" "/etc/asciidoc/filters/graphviz/graphviz2png.py" -v -o "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png" -L dot -F png - Execute: dot -Tpng "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.txt" > "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual__3.png" asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc asciidoc: resource_pool.adoc: line 111: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/config_mangling.adoc asciidoc: config_mangling.adoc: line 165: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1216-gd577" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Sep-07" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: config_mangling.adoc: line 255: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1216-gd577" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Sep-07" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: config_mangling.adoc: line 330: filtering: "/usr/bin/python2" "/usr/bin/asciidoc" -b docbook45 --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/mscgen-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/diag-filter.conf" --conf-file "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/docinfo-releaseinfo.conf" --attribute "revnumber=DRAFT 0.1-1216-gd577" --attribute "commondir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/common" --attribute "srcdir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp" --attribute "app-name=Osmo-GSM-Tester" --attribute "revdate=2022-Sep-07" --attribute "a2x-format=pdf" --attribute "docinfo" -a "indir=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters" -a "blockname=table" -s - asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 21: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/test_api.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 23: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 25: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/install_device.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 27: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/ansible.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 29: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/docker.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 31: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/debugging.adoc asciidoc: include: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc asciidoc: osmo-gsm-tester-manual.adoc: line 33: reading: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/chapters/troubleshooting.adoc a2x: executing: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -s /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals/build/custom-dblatex.sty -P draft.mode=yes -P draft.watermark=0 -V "/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml" Build the book set list... xsltproc -o doclist.txt --xinclude --xincludestyle doclist.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml Build the listings... xsltproc -o listings.xml --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' /usr/share/dblatex/xsl/common/mklistings.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml xsltproc -o osmo-gsm-tester-manual.rtex --xinclude --xincludestyle --param current.dir '/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp' --param listings.xml '/tmp/tmpyQMDcW/listings.xml' /tmp/tmpyQMDcW/custom.xsl /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml XSLT stylesheets DocBook - LaTeX 2e (0.3.10) =================================================== WARNING: xref to <simpara id="scenario_suite_params"> has no generated text. Trying its ancestor elements. Image 'dblatex' not found Cannot convert 'osmo-gsm-tester-manual__1.png' to pdf Cannot convert 'osmo-gsm-tester-manual__2.png' to pdf Cannot convert 'osmo-gsm-tester-manual__3.png' to pdf Build osmo-gsm-tester-manual.pdf built-in module pdftex registered no support found for ifthen no support found for ifxetex no support found for fontspec no support found for xltxtra no support found for fontenc no support found for inputenc no support found for fancybox built-in module makeidx registered no support found for custom-dblatex building additional files... checking if compiling is necessary... the output file doesn't exist pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex running post-compilation scripts... [index] the index file /tmp/tmpyQMDcW/osmo-gsm-tester-manual.idx is empty osmo-gsm-tester-manual.aux MD5 checksum changed osmo-gsm-tester-manual.toc MD5 checksum changed the /tmp/tmpyQMDcW/osmo-gsm-tester-manual.aux file has changed pdflatex -interaction=batchmode osmo-gsm-tester-manual.tex running post-compilation scripts... [index] the index file /tmp/tmpyQMDcW/osmo-gsm-tester-manual.idx is empty no new compilation is needed running last-compilation scripts... 'osmo-gsm-tester-manual.pdf' successfully built a2x: deleting /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp/osmo-gsm-tester-manual.xml + make check OSMO_GSM_MANUALS_DIR: /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/osmo-gsm-manuals (from env) if [ -n "$ASCIIDOC_WARNINGS_CHECK" ]; then \ make osmo-gsm-tester-manual.check; \ fi + [ = 1 ] + rm -r /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/_manuals_temp + cd /home/jenkins/workspace/osmo-gsm-tester_gerrit + PATH=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src:/usr/local/bin:/usr/bin:/bin:/usr/games osmo-gsm-tester/contrib/jenkins-run.sh + base=/home/jenkins/workspace/osmo-gsm-tester_gerrit + dirname -- osmo-gsm-tester/contrib/jenkins-run.sh + CDPATH= cd -- osmo-gsm-tester/contrib + pwd -P + SCRIPT_DIR=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib + OSMO_GSM_TESTER_CONF=/home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf + date +%F %T + time_start=2022-09-07 11:02:01 + trial_dir_prefix=trial- + rm -rf trial-2065 + trial_dir=trial-2066 + mkdir -p trial-2066 + mv open5gs.build-8.tgz osmo-bsc.build-16888.tgz osmo-bts-oc2g.build-5192.tgz osmo-bts-sysmo.build-7696.tgz osmo-bts.build-7166.tgz osmo-ggsn.build-6364.tgz osmo-hlr.build-8262.tgz osmo-hnbgw.build-2020.tgz osmo-mgw.build-8470.tgz osmo-msc.build-28078.tgz osmo-nitb.build-12346.tgz osmo-pcu-oc2g.build-3690.tgz osmo-pcu-sysmo.build-4971.tgz osmo-pcu.build-4933.tgz osmo-sgsn.build-28683.tgz osmo-stp.build-8799.tgz osmo-trx.build-4360.tgz osmocom-bb.build-4442.tgz srslte.build-19.tgz trial-2066 + cat open5gs.build-8.md5 osmo-bsc.build-16888.md5 osmo-bts-oc2g.build-5192.md5 osmo-bts-sysmo.build-7696.md5 osmo-bts.build-7166.md5 osmo-ggsn.build-6364.md5 osmo-hlr.build-8262.md5 osmo-hnbgw.build-2020.md5 osmo-mgw.build-8470.md5 osmo-msc.build-28078.md5 osmo-nitb.build-12346.md5 osmo-pcu-oc2g.build-3690.md5 osmo-pcu-sysmo.build-4971.md5 osmo-pcu.build-4933.md5 osmo-sgsn.build-28683.md5 osmo-stp.build-8799.md5 osmo-trx.build-4360.md5 osmocom-bb.build-4442.md5 srslte.build-19.md5 + rm open5gs.build-8.md5 osmo-bsc.build-16888.md5 osmo-bts-oc2g.build-5192.md5 osmo-bts-sysmo.build-7696.md5 osmo-bts.build-7166.md5 osmo-ggsn.build-6364.md5 osmo-hlr.build-8262.md5 osmo-hnbgw.build-2020.md5 osmo-mgw.build-8470.md5 osmo-msc.build-28078.md5 osmo-nitb.build-12346.md5 osmo-pcu-oc2g.build-3690.md5 osmo-pcu-sysmo.build-4971.md5 osmo-pcu.build-4933.md5 osmo-sgsn.build-28683.md5 osmo-stp.build-8799.md5 osmo-trx.build-4360.md5 osmocom-bb.build-4442.md5 srslte.build-19.md5 + exit_code=1 + which osmo-gsm-tester.py + python3 -u /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/src/osmo-gsm-tester.py -c /home/jenkins/workspace/osmo-gsm-tester_gerrit/osmo-gsm-tester/contrib/../sysmocom/main.conf trial-2066 -s nitb_sms:sysmo -s sms:sysmo -s gprs:sysmo combinations: ['nitb_sms:sysmo', 'sms:sysmo', 'gprs:sysmo'] trial: 'trial-2066' tests: None -------------------------------------------------------------------------------------------- trial-2066 -------------------------------------------------------------------------------------------- 11:02:04.069422 tst trial-2066: Detailed log at /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2066/run.2022-09-07_11-02-04/log --------------------------------------------------------------------- trial-2066 nitb_sms:sysmo --------------------------------------------------------------------- 11:02:06.873998 tst nitb_sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 11:02:06.877754 tst nitb_sms:sysmo: Verifying 1 x bts (candidates: 11) 11:02:06.899072 tst nitb_sms:sysmo: Verifying 1 x ip_address (candidates: 9) 11:02:06.906042 tst nitb_sms:sysmo: Verifying 2 x modem (candidates: 12) 11:02:06.960709 tst nitb_sms:sysmo: Reserving 1 x bts (candidates: 11) 11:02:06.982017 tst nitb_sms:sysmo: Reserving 1 x ip_address (candidates: 9) 11:02:06.988934 tst nitb_sms:sysmo: Reserving 2 x modem (candidates: 12) 11:02:07.528631 tst nitb_sms:sysmo: RESERVED RESOURCES for nitb_sms: bts sysmoBTS 1002 ip_address 10.42.42.2 modem sierra_1st sierra_2nd ---------------------------------------------- trial-2066 nitb_sms:sysmo mo_mt_sms.py ---------------------------------------------- 11:02:07.536626 tst nitb_sms:sysmo: Using 1 x ip_address (candidates: 1) 11:02:07.548565 tst nitb_sms:sysmo: Using 1 x bts (candidates: 1) 11:02:07.914559 tst mo_mt_sms.py: using LAC 3957 11:02:08.143395 tst mo_mt_sms.py: using RAC 132 11:02:08.255376 tst mo_mt_sms.py: using CellId 3957 11:02:08.369032 tst mo_mt_sms.py: using BVCI 3958 11:02:08.473318 tst nitb_sms:sysmo: Using 1 x modem (candidates: 2) 11:02:08.679036 tst nitb_sms:sysmo: Using 1 x modem (candidates: 2) 11:02:08.794036 tst mo_mt_sms.py:9: start nitb and bts... 11:02:08.994477 run osmo-nitb_10.42.42.2: Starting osmo-nitb 11:02:11.219949 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=17961): Launched 11:02:11.520482 run osmo-nitb_10.42.42.2(pid=17962): Launched 11:02:11.685122 run osmo-nitb_10.42.42.2(pid=17962): ERR: Terminated: ERROR {rc=2} [trial-2066↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=17962)] 11:02:11.808796 run osmo-nitb_10.42.42.2(pid=17962): stdout: | (launched: 2022-09-07_11:02:11.464993) 11:02:11.934437 run osmo-nitb_10.42.42.2(pid=17962): stderr: | (launched: 2022-09-07_11:02:11.509433) | ====================================================================== | ERROR: You should not run osmo-nitb. It is ancient, unmaintained and | obsolete for several years. If you really want to ignore years of hard work on | the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp), | you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line | argument. DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!! | Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box | ====================================================================== 11:02:12.059949 run osmo-nitb_10.42.42.2(pid=17962): stdout: | (launched: 2022-09-07_11:02:11.464993) 11:02:12.179504 run osmo-nitb_10.42.42.2(pid=17962): stderr: | (launched: 2022-09-07_11:02:11.509433) | ====================================================================== | ERROR: You should not run osmo-nitb. It is ancient, unmaintained and | obsolete for several years. If you really want to ignore years of hard work on | the new post-NITB stack (osmo-bsc, osmo-mgw, osmo-msc, osmo-hlr, osmo-stp), | you may ues the '--yes-i-really-want-to-run-prehistoric-software' command line | argument. DON'T EVER BOTHER THE DEVELOPERS IF YOU ENCOUNTER PROBLEMS WITH IT!!! | Use https://osmocom.org/projects/cellular-infrastructure/wiki/Osmocom_Network_In_The_Box | ====================================================================== 11:02:12.322831 tst mo_mt_sms.py:11: ERR: Error: osmo-nitb_10.42.42.2(pid=17962): Process ended prematurely: osmo-nitb_10.42.42.2(pid=17962) [trial-2066↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=17962)] [trial-2066↪nitb_sms:sysmo↪mo_mt_sms.py:11] 11:02:12.331053 tst mo_mt_sms.py:11: Test FAILED (4.8 sec) 11:02:12.337566 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=17961): Terminating (SIGTERM) 11:02:12.547427 --- ParallelTerminationStrategy: PID 17961 died... 11:02:12.568747 run pcap-recorder_any(filters='host 10.42.42.2 and port not 22')(pid=17961): Terminated: ok {rc=0} 11:02:12.656812 bus /sierra_3: Setting Powered False 11:02:13.758377 bus /sierra_3: Setting Powered False 11:02:14.878112 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-2066 nitb_sms:sysmo FAIL --------------------------------------------------------------------- --------------------------------------------------------------------- trial-2066 sms:sysmo --------------------------------------------------------------------- 11:02:16.058247 tst sms:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 11:02:16.062328 tst sms:sysmo: Verifying 1 x bts (candidates: 11) 11:02:16.083273 tst sms:sysmo: Verifying 6 x ip_address (candidates: 9) 11:02:16.111519 tst sms:sysmo: Verifying 2 x modem (candidates: 12) 11:02:16.164268 tst sms:sysmo: Reserving 1 x bts (candidates: 11) 11:02:16.185224 tst sms:sysmo: Reserving 6 x ip_address (candidates: 9) 11:02:16.211752 tst sms:sysmo: Reserving 2 x modem (candidates: 12) 11:02:16.756637 tst sms:sysmo: RESERVED RESOURCES for sms: bts sysmoBTS 1002 ip_address 10.42.42.2 10.42.42.3 10.42.42.4 10.42.42.5 10.42.42.6 10.42.42.7 modem sierra_1st sierra_2nd ---------------------------------------------- trial-2066 sms:sysmo mo_mt_sms.py ---------------------------------------------- 11:02:16.764950 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 11:02:16.775663 tst sms:sysmo: Using 1 x bts (candidates: 1) 11:02:16.812788 tst mo_mt_sms.py: using LAC 3958 11:02:16.936661 tst mo_mt_sms.py: using RAC 133 11:02:17.054076 tst mo_mt_sms.py: using CellId 3958 11:02:17.167988 tst mo_mt_sms.py: using BVCI 3959 11:02:17.275194 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 11:02:17.285625 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 11:02:17.296083 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 11:02:17.306762 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 11:02:17.317354 tst sms:sysmo: Using 1 x ip_address (candidates: 6) 11:02:17.327865 tst sms:sysmo: Using 1 x modem (candidates: 2) 11:02:17.442357 tst sms:sysmo: Using 1 x modem (candidates: 2) 11:02:17.561108 run osmo-hlr_10.42.42.2: Starting osmo-hlr 11:02:19.269574 run create_hlr_db(pid=17967): Launched 11:02:19.402546 bus /sierra_2: Setting Powered False 11:02:20.466358 run create_hlr_db(pid=17967): Terminated: ok {rc=0} 11:02:20.912925 run pcap-recorder_any(filters='host 10.42.42.2')(pid=17969): Launched 11:02:21.235075 run osmo-hlr_10.42.42.2(pid=17970): Launched 11:02:21.360814 run osmo-stp_10.42.42.5: Starting osmo-stp 11:02:23.960531 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=17971): Launched 11:02:24.282885 run osmo-stp_10.42.42.5(pid=17972): Launched 11:02:24.406071 run osmo-msc_10.42.42.6: Starting osmo-msc 11:02:29.862500 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=17973): Launched 11:02:30.185038 run osmo-msc_10.42.42.6(pid=17974): Launched 11:02:33.365838 run osmo-mgw_10.42.42.3: Starting osmo-mgw 11:02:35.124008 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=17975): Launched 11:02:35.449867 run osmo-mgw_10.42.42.3(pid=17976): Launched 11:02:35.578039 run osmo-mgw_10.42.42.4: Starting osmo-mgw 11:02:36.116591 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=17977): Launched 11:02:36.437908 run osmo-mgw_10.42.42.4(pid=17978): Launched 11:02:36.559769 run osmo-bsc_10.42.42.7: Starting osmo-bsc 11:02:40.970873 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=17980): Launched 11:02:41.292004 run osmo-bsc_10.42.42.7(pid=17981): Launched 11:02:47.635312 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.7 11:02:48.576695 run rm-remote-dir(pid=17982): Launched 11:02:50.767404 run rm-remote-dir(pid=17982): Terminated: ok {rc=0} 11:02:51.119335 run mk-remote-dir(pid=17983): Launched 11:02:52.305708 run mk-remote-dir(pid=17983): Terminated: ok {rc=0} 11:02:52.609179 run scp-inst-to-remote(pid=17984): Launched 11:03:04.805210 run scp-inst-to-remote(pid=17984): Terminated: ok {rc=0} 11:03:05.151998 run mk-remote-dir(pid=17986): Launched 11:03:06.333175 run mk-remote-dir(pid=17986): Terminated: ok {rc=0} 11:03:06.612590 run scp-cfg-to-remote(pid=17987): Launched 11:03:07.787146 run scp-cfg-to-remote(pid=17987): Terminated: ok {rc=0} 11:03:08.108359 run reload-dsp-firmware(pid=17989): Launched 11:03:10.282190 run reload-dsp-firmware(pid=17989): Terminated: ok {rc=0} 11:03:10.596288 run osmo-bts-sysmo(pid=17990): Launched 11:03:14.858798 tst mo_mt_sms.py: using MSISDN 3969 11:03:15.024888 bus /sierra_2: Setting Powered True 11:03:18.515667 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='3969', subscriber_id=1} 11:03:18.647032 tst mo_mt_sms.py: using MSISDN 3970 11:03:18.820863 bus /sierra_3: Setting Powered True 11:03:22.285692 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='3970', subscriber_id=2} 11:03:22.543626 bus /sierra_2: Setting Online False 11:03:23.726944 bus /sierra_2: Setting Powered False 11:03:30.052312 bus /sierra_2: Setting Powered True 11:03:31.389428 bus /sierra_2: Setting Online True 11:03:33.888225 tst /sierra_2: Connect to ('901', '70') 11:03:34.183530 bus /sierra_3: Setting Online False 11:03:35.365362 bus /sierra_3: Setting Powered False 11:03:41.073307 bus /sierra_3: Setting Powered True 11:03:42.431140 bus /sierra_3: Setting Online True 11:03:44.858285 tst /sierra_3: Connect to ('901', '70') 11:03:45.030481 tst /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053' 11:03:45.147412 tst /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028' 11:03:45.262055 tst mo_mt_sms.py:35: waiting for modems to attach... 11:03:48.903871 tst /sierra_2: Already registered with network ('901', '70') 11:04:01.223635 tst /sierra_3: Already registered with network ('901', '70') 11:04:06.066273 tst /sierra_2: sending sms to MSISDN 3970 {sms='message nr. 1, from /sierra_2, to /sierra_3, from 3969, to 3970'} 11:04:11.664840 tst /sierra_3: Incoming SMS: 'message nr. 1, from /sierra_2, to /sierra_3, from 3969, to 3970' 11:04:12.185352 tst /sierra_3: SMS received as expected: 'message nr. 1, from /sierra_2, to /sierra_3, from 3969, to 3970' 11:04:12.360268 tst mo_mt_sms.py: Test passed (115.6 sec) 11:04:12.428525 run pcap-recorder_any(filters='host 10.42.42.2')(pid=17969): Terminating (SIGTERM) 11:04:12.471300 run osmo-hlr_10.42.42.2(pid=17970): Terminating (SIGTERM) 11:04:12.513811 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=17971): Terminating (SIGTERM) 11:04:12.558904 run osmo-stp_10.42.42.5(pid=17972): Terminating (SIGTERM) 11:04:12.604594 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=17973): Terminating (SIGTERM) 11:04:12.663170 run osmo-msc_10.42.42.6(pid=17974): Terminating (SIGTERM) 11:04:12.708619 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=17975): Terminating (SIGTERM) 11:04:12.754502 run osmo-mgw_10.42.42.3(pid=17976): Terminating (SIGTERM) 11:04:12.811153 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=17977): Terminating (SIGTERM) 11:04:12.856354 run osmo-mgw_10.42.42.4(pid=17978): Terminating (SIGTERM) 11:04:12.901267 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=17980): Terminating (SIGTERM) 11:04:12.946348 run osmo-bsc_10.42.42.7(pid=17981): Terminating (SIGTERM) 11:04:12.990305 run osmo-bts-sysmo(pid=17990): Terminating (SIGTERM) 11:04:13.033141 --- ParallelTerminationStrategy: PID 17969 died... 11:04:13.053350 run pcap-recorder_any(filters='host 10.42.42.2')(pid=17969): Terminated: ok {rc=0} 11:04:13.101199 --- ParallelTerminationStrategy: PID 17970 died... 11:04:13.121312 run osmo-hlr_10.42.42.2(pid=17970): Terminated: ok {rc=0} 11:04:13.168957 --- ParallelTerminationStrategy: PID 17971 died... 11:04:13.188950 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=17971): Terminated: ok {rc=0} 11:04:13.236510 --- ParallelTerminationStrategy: PID 17972 died... 11:04:13.256348 run osmo-stp_10.42.42.5(pid=17972): Terminated: ok {rc=0} 11:04:13.303851 --- ParallelTerminationStrategy: PID 17973 died... 11:04:13.324088 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=17973): Terminated: ok {rc=0} 11:04:13.372750 --- ParallelTerminationStrategy: PID 17974 died... 11:04:13.393285 run osmo-msc_10.42.42.6(pid=17974): Terminated: ok {rc=0} 11:04:13.441846 --- ParallelTerminationStrategy: PID 17975 died... 11:04:13.462309 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=17975): Terminated: ok {rc=0} 11:04:13.510897 --- ParallelTerminationStrategy: PID 17976 died... 11:04:13.531352 run osmo-mgw_10.42.42.3(pid=17976): Terminated {rc=15} 11:04:13.579849 --- ParallelTerminationStrategy: PID 17977 died... 11:04:13.599808 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=17977): Terminated: ok {rc=0} 11:04:13.647228 --- ParallelTerminationStrategy: PID 17978 died... 11:04:13.667175 run osmo-mgw_10.42.42.4(pid=17978): Terminated {rc=15} 11:04:13.714866 --- ParallelTerminationStrategy: PID 17980 died... 11:04:13.734625 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=17980): Terminated: ok {rc=0} 11:04:13.784809 --- ParallelTerminationStrategy: PID 17981 died... 11:04:13.804946 run osmo-bsc_10.42.42.7(pid=17981): Terminated: ok {rc=0} 11:04:13.852167 --- ParallelTerminationStrategy: PID 17990 died... 11:04:13.871948 run osmo-bts-sysmo(pid=17990): Terminated: ok {rc=0} 11:04:14.003887 bus /sierra_3: Setting Online False 11:04:15.144210 bus /sierra_3: Setting Powered False 11:04:25.917174 bus /sierra_3: Setting Powered False 11:04:27.015994 bus /sierra_3: Setting Powered False 11:04:28.176144 bus /sierra_2: Setting Online False 11:04:29.321957 bus /sierra_2: Setting Powered False 11:04:35.918562 bus /sierra_2: Setting Powered False --------------------------------------------------------------------- trial-2066 sms:sysmo PASS --------------------------------------------------------------------- --------------------------------------------------------------------- trial-2066 gprs:sysmo --------------------------------------------------------------------- 11:04:37.195011 tst gprs:sysmo: reserving resources in /var/tmp/osmo-gsm-tester/state ... 11:04:37.199436 tst gprs:sysmo: Verifying 1 x bts (candidates: 11) 11:04:37.219958 tst gprs:sysmo: Verifying 9 x ip_address (candidates: 9) 11:04:37.259109 tst gprs:sysmo: Verifying 4 x modem (candidates: 12) 11:04:37.345029 tst gprs:sysmo: Reserving 1 x bts (candidates: 11) 11:04:37.365492 tst gprs:sysmo: Reserving 9 x ip_address (candidates: 9) 11:04:37.403553 tst gprs:sysmo: Reserving 4 x modem (candidates: 12) 11:04:37.992604 tst gprs:sysmo: RESERVED RESOURCES for gprs: bts sysmoBTS 1002 ip_address 10.42.42.2 10.42.42.3 10.42.42.4 10.42.42.5 10.42.42.6 10.42.42.7 10.42.42.8 10.42.42.9 10.42.42.10 modem ec20_1 ec20_2 ec20_3 ec20_4 ---------------------------------------------- trial-2066 gprs:sysmo cs_paging_gprs_active.py ---------------------------------------------- 11:04:38.052041 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.062905 tst gprs:sysmo: Using 1 x bts (candidates: 1) 11:04:38.097569 tst cs_paging_gprs_active.py: using LAC 3959 11:04:38.220613 tst cs_paging_gprs_active.py: using RAC 134 11:04:38.347888 tst cs_paging_gprs_active.py: using CellId 3959 11:04:38.483681 tst cs_paging_gprs_active.py: using BVCI 3960 11:04:38.604039 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.615151 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.626021 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.637029 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.647868 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.658620 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.670020 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.680864 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:04:38.691930 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:04:38.808236 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:04:38.926259 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003... 11:04:39.135437 run iperf3-srv_10.42.42.10: Starting iperf3-srv 11:04:39.446135 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=17999): Launched 11:04:39.717165 run iperf3-srv_10.42.42.10(pid=18000): Launched 11:04:39.835041 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004... 11:04:40.040692 run iperf3-srv_10.42.42.10: Starting iperf3-srv 11:04:40.350521 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18001): Launched 11:04:40.618673 run iperf3-srv_10.42.42.10(pid=18002): Launched 11:04:40.734108 tst cs_paging_gprs_active.py:38: start network... 11:04:40.939489 run osmo-hlr_10.42.42.2: Starting osmo-hlr 11:04:41.416891 run create_hlr_db(pid=18003): Launched 11:04:41.545843 bus /sierra_2: Setting Powered False 11:04:42.607488 run create_hlr_db(pid=18003): Terminated: ok {rc=0} 11:04:43.058173 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18005): Launched 11:04:43.389278 run osmo-hlr_10.42.42.2(pid=18006): Launched 11:04:43.515473 run osmo-stp_10.42.42.5: Starting osmo-stp 11:04:44.078904 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18007): Launched 11:04:44.409642 run osmo-stp_10.42.42.5(pid=18008): Launched 11:04:44.535826 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 11:04:46.077518 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18009): Launched 11:04:46.195562 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 11:04:46.458662 run patchelf(pid=18010): Launched 11:04:47.635835 run patchelf(pid=18010): Terminated: ok {rc=0} 11:04:47.765790 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 11:04:48.032948 run osmo-gsm-tester_setcap_net_admin.sh(pid=18011): Launched 11:04:49.212427 run osmo-gsm-tester_setcap_net_admin.sh(pid=18011): Terminated: ok {rc=0} 11:04:49.533919 run osmo-ggsn_10.42.42.6(pid=18014): Launched 11:04:49.651387 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 11:04:54.540671 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18035): Launched 11:04:54.851569 run osmo-sgsn_10.42.42.7(pid=18036): Launched 11:04:54.971440 run osmo-msc_10.42.42.8: Starting osmo-msc 11:04:55.548110 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18037): Launched 11:04:55.860030 run osmo-msc_10.42.42.8(pid=18038): Launched 11:04:59.035038 run osmo-mgw_10.42.42.3: Starting osmo-mgw 11:04:59.563024 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18039): Launched 11:04:59.873544 run osmo-mgw_10.42.42.3(pid=18040): Launched 11:04:59.990997 run osmo-mgw_10.42.42.4: Starting osmo-mgw 11:05:00.508972 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18041): Launched 11:05:00.822242 run osmo-mgw_10.42.42.4(pid=18042): Launched 11:05:00.941052 run osmo-bsc_10.42.42.9: Starting osmo-bsc 11:05:01.580748 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18043): Launched 11:05:01.910210 run osmo-bsc_10.42.42.9(pid=18047): Launched 11:05:08.185070 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 11:05:08.722512 run rm-remote-dir(pid=18048): Launched 11:05:10.909900 run rm-remote-dir(pid=18048): Terminated: ok {rc=0} 11:05:11.251091 run mk-remote-dir(pid=18049): Launched 11:05:12.435225 run mk-remote-dir(pid=18049): Terminated: ok {rc=0} 11:05:12.724930 run scp-inst-to-remote(pid=18050): Launched 11:05:24.929166 run scp-inst-to-remote(pid=18050): Terminated: ok {rc=0} 11:05:25.286437 run mk-remote-dir(pid=18052): Launched 11:05:26.477230 run mk-remote-dir(pid=18052): Terminated: ok {rc=0} 11:05:26.784823 run scp-cfg-to-remote(pid=18053): Launched 11:05:27.975152 run scp-cfg-to-remote(pid=18053): Terminated: ok {rc=0} 11:05:28.329563 run reload-dsp-firmware(pid=18055): Launched 11:05:30.519200 run reload-dsp-firmware(pid=18055): Terminated: ok {rc=0} 11:05:30.865782 run osmo-bts-sysmo(pid=18056): Launched 11:05:35.102007 tst cs_paging_gprs_active.py:38: Waiting for bts to be ready... 11:05:37.139024 run rm-remote-dir(pid=18057): Launched 11:05:39.316463 run rm-remote-dir(pid=18057): Terminated: ok {rc=0} 11:05:39.642489 run mk-remote-dir(pid=18059): Launched 11:05:41.817340 run mk-remote-dir(pid=18059): Terminated: ok {rc=0} 11:05:42.096486 run scp-inst-to-bts(pid=18060): Launched 11:05:55.290559 run scp-inst-to-bts(pid=18060): Terminated: ok {rc=0} 11:05:55.638821 run mk-remote-run-dir(pid=18062): Launched 11:05:57.826144 run mk-remote-run-dir(pid=18062): Terminated: ok {rc=0} 11:05:58.124954 run scp-cfg-to-sysmobts(pid=18063): Launched 11:06:00.310425 run scp-cfg-to-sysmobts(pid=18063): Terminated: ok {rc=0} 11:06:00.659072 run osmo-pcu(pid=18065): Launched 11:06:00.795027 tst cs_paging_gprs_active.py: using MSISDN 3971 11:06:00.971459 bus /gobi_6: Setting Powered True 11:06:04.468649 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3971', subscriber_id=1} 11:06:04.743297 bus /gobi_6: Setting Online False 11:06:05.927347 bus /gobi_6: Setting Powered False 11:06:13.122016 bus /gobi_6: Setting Powered True 11:06:14.489903 bus /gobi_6: Setting Online True 11:06:16.960121 tst /gobi_6: Connect to ('901', '70') 11:06:17.235588 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 11:06:17.355948 tst cs_paging_gprs_active.py: using MSISDN 3972 11:06:17.583591 bus /gobi_0: Setting Powered True 11:06:21.104233 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3972', subscriber_id=2} 11:06:21.368041 bus /gobi_0: Setting Online False 11:06:22.556694 bus /gobi_0: Setting Powered False 11:06:29.297560 bus /gobi_0: Setting Powered True 11:06:30.688126 bus /gobi_0: Setting Online True 11:06:32.327131 tst /gobi_6: Already registered with network ('901', '70') 11:06:33.147527 tst /gobi_0: Connect to ('901', '70') 11:06:33.424328 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873' 11:06:33.540001 tst cs_paging_gprs_active.py:38: waiting for modems to attach... 11:06:48.709680 tst /gobi_0: Already registered with network ('901', '70') 11:06:54.057423 tst cs_paging_gprs_active.py:38: waiting for modems to attach to data services... 11:07:02.534499 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:07:02.651544 tst cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_6/context2'" 11:07:03.096151 run move_netns(pid=18070): Launched 11:07:04.292857 run move_netns(pid=18070): Terminated: ok {rc=0} 11:07:04.644297 run ifup(pid=18073): Launched 11:07:05.832433 run ifup(pid=18073): Terminated: ok {rc=0} 11:07:06.183149 run dhcp(pid=18076): Launched 11:07:07.372127 run dhcp(pid=18076): Terminated: ok {rc=0} 11:07:10.507459 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:07:10.619003 tst cs_paging_gprs_active.py:38: Setting up data plan for "'/gobi_0/context3'" 11:07:11.038173 run move_netns(pid=18094): Launched 11:07:12.229193 run move_netns(pid=18094): Terminated: ok {rc=0} 11:07:12.560191 run ifup(pid=18097): Launched 11:07:13.744089 run ifup(pid=18097): Terminated: ok {rc=0} 11:07:14.071279 run dhcp(pid=18100): Launched 11:07:15.261771 run dhcp(pid=18100): Terminated: ok {rc=0} 11:07:15.399197 tst cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'" 11:07:15.621914 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 11:07:16.007196 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116): Launched 11:07:16.179847 tst cs_paging_gprs_active.py:38: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context3'" 11:07:16.386618 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s) 11:07:16.749818 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119): Launched 11:07:17.067705 run iperf3-cli_10.42.42.10(pid=18122): Launched 11:07:17.333263 run iperf3-cli_10.42.42.10(pid=18125): Launched 11:07:17.451543 tst cs_paging_gprs_active.py:13: waiting a few secs to make sure iperf3 test is running 11:07:19.682060 tst cs_paging_gprs_active.py:16: iperf3 running, let's place a call 11:07:24.268946 tst cs_paging_gprs_active.py:22: dial success 11:07:26.971506 tst cs_paging_gprs_active.py:28: answer success, call established and ongoing 11:07:33.579132 tst cs_paging_gprs_active.py:35: hangup success 11:08:42.954976 run iperf3-cli_10.42.42.10(pid=18122): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=18122)] 11:08:43.086450 run iperf3-cli_10.42.42.10(pid=18122): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 11:08:43.218973 run iperf3-cli_10.42.42.10(pid=18122): stderr: | (launched: 2022-09-07_11:07:17.052275) 11:08:44.405784 run iperf3-cli_10.42.42.10(pid=18125): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=18125)] 11:08:44.536852 run iperf3-cli_10.42.42.10(pid=18125): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 11:08:44.669333 run iperf3-cli_10.42.42.10(pid=18125): stderr: | (launched: 2022-09-07_11:07:17.318037) 11:08:44.800980 run iperf3-srv_10.42.42.10(pid=18000): Terminating (SIGINT) 11:08:45.138327 run iperf3-srv_10.42.42.10(pid=18000): Terminated {rc=1} 11:08:45.257904 tst cs_paging_gprs_active.py:38: Results for 10.42.42.10:5003 through "'/gobi_6/context2'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 46720, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Wed, 07 Sep 2022 09:07:34 GMT', 'timesecs': 1662541654}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'bdb5uclecydqftbf4r5yfky6vun4avdtge3m', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0008049011230469, 'seconds': 1.0008049011230469, 'bytes': 101360, 'bits_per_second': 810227.8466962703, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 706466, 'rttvar': 349626, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0008049011230469, 'seconds': 1.0008049011230469, 'bytes': 101360, 'bits_per_second': 810227.8466962703, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0008049011230469, 'end': 2.001323938369751, 'seconds': 1.000519037246704, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 755350, 'rttvar': 359988, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0008049011230469, 'end': 2.001323938369751, 'seconds': 1.000519037246704, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.001323938369751, 'end': 3.0009329319000244, 'seconds': 0.9996089935302734, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1075670, 'rttvar': 780458, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.001323938369751, 'end': 3.0009329319000244, 'seconds': 0.9996089935302734, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0009329319000244, 'end': 4.001227855682373, 'seconds': 1.0002949237823486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1331057, 'rttvar': 1096119, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0009329319000244, 'end': 4.001227855682373, 'seconds': 1.0002949237823486, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001227855682373, 'end': 5.00131893157959, 'seconds': 1.0000910758972168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 2006446, 'rttvar': 1810621, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001227855682373, 'end': 5.00131893157959, 'seconds': 1.0000910758972168, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.00131893157959, 'end': 6.000869035720825, 'seconds': 0.9995501041412354, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 2410217, 'rttvar': 2165509, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.00131893157959, 'end': 6.000869035720825, 'seconds': 0.9995501041412354, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.000869035720825, 'end': 7.0007710456848145, 'seconds': 0.9999020099639893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2849021, 'rttvar': 2501740, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.000869035720825, 'end': 7.0007710456848145, 'seconds': 0.9999020099639893, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.0007710456848145, 'end': 8.00132703781128, 'seconds': 1.0005559921264648, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 3819722, 'rttvar': 3113187, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.0007710456848145, 'end': 8.00132703781128, 'seconds': 1.0005559921264648, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.00132703781128, 'end': 9.001224994659424, 'seconds': 0.9998979568481445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 4217351, 'rttvar': 3130149, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.00132703781128, 'end': 9.001224994659424, 'seconds': 0.9998979568481445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001224994659424, 'end': 10.001227855682373, 'seconds': 1.0000028610229492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 5024367, 'rttvar': 3159515, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001224994659424, 'end': 10.001227855682373, 'seconds': 1.0000028610229492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001227855682373, 'end': 11.000874042510986, 'seconds': 0.9996461868286133, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 5438631, 'rttvar': 3198163, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001227855682373, 'end': 11.000874042510986, 'seconds': 0.9996461868286133, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.000874042510986, 'end': 12.000244855880737, 'seconds': 0.999370813369751, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 6212797, 'rttvar': 3164832, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.000874042510986, 'end': 12.000244855880737, 'seconds': 0.999370813369751, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.000244855880737, 'end': 13.000736951828003, 'seconds': 1.0004920959472656, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 6578404, 'rttvar': 3104837, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.000244855880737, 'end': 13.000736951828003, 'seconds': 1.0004920959472656, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.000736951828003, 'end': 14.000695943832397, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 6991209, 'rttvar': 3154238, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.000736951828003, 'end': 14.000695943832397, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.000695943832397, 'end': 15.000710010528564, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 7356969, 'rttvar': 3097199, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.000695943832397, 'end': 15.000710010528564, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.000710010528564, 'end': 16.00118899345398, 'seconds': 1.000478982925415, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 8176806, 'rttvar': 3167861, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.000710010528564, 'end': 16.00118899345398, 'seconds': 1.000478982925415, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.00118899345398, 'end': 17.001147031784058, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 8606662, 'rttvar': 3235609, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.00118899345398, 'end': 17.001147031784058, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001147031784058, 'end': 18.001174926757812, 'seconds': 1.0000278949737549, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 9391278, 'rttvar': 3202043, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001147031784058, 'end': 18.001174926757812, 'seconds': 1.0000278949737549, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001174926757812, 'end': 19.000690937042236, 'seconds': 0.9995160102844238, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9756955, 'rttvar': 3132886, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001174926757812, 'end': 19.000690937042236, 'seconds': 0.9995160102844238, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.000690937042236, 'end': 20.00127601623535, 'seconds': 1.0005850791931152, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 10164588, 'rttvar': 3164930, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.000690937042236, 'end': 20.00127601623535, 'seconds': 1.0005850791931152, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.00127601623535, 'end': 21.000741958618164, 'seconds': 0.9994659423828125, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 11197648, 'rttvar': 2894740, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.00127601623535, 'end': 21.000741958618164, 'seconds': 0.9994659423828125, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.000741958618164, 'end': 22.001484870910645, 'seconds': 1.0007429122924805, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 11532499, 'rttvar': 2840757, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.000741958618164, 'end': 22.001484870910645, 'seconds': 1.0007429122924805, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001484870910645, 'end': 23.001327991485596, 'seconds': 0.9998431205749512, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 12146487, 'rttvar': 2685387, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001484870910645, 'end': 23.001327991485596, 'seconds': 0.9998431205749512, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001327991485596, 'end': 24.000727891921997, 'seconds': 0.9993999004364014, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 12445375, 'rttvar': 2611816, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001327991485596, 'end': 24.000727891921997, 'seconds': 0.9993999004364014, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.000727891921997, 'end': 25.001147985458374, 'seconds': 1.000420093536377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 12794531, 'rttvar': 2657176, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.000727891921997, 'end': 25.001147985458374, 'seconds': 1.000420093536377, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001147985458374, 'end': 26.00118899345398, 'seconds': 1.0000410079956055, 'bytes': 244712, 'bits_per_second': 1957615.7221030707, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 13473804, 'rttvar': 2696659, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001147985458374, 'end': 26.00118899345398, 'seconds': 1.0000410079956055, 'bytes': 244712, 'bits_per_second': 1957615.7221030707, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.00118899345398, 'end': 27.00065302848816, 'seconds': 0.9994640350341797, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 13806914, 'rttvar': 2688714, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.00118899345398, 'end': 27.00065302848816, 'seconds': 0.9994640350341797, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00065302848816, 'end': 28.001178979873657, 'seconds': 1.000525951385498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 14210888, 'rttvar': 2824483, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00065302848816, 'end': 28.001178979873657, 'seconds': 1.000525951385498, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001178979873657, 'end': 29.000942945480347, 'seconds': 0.9997639656066895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 14576965, 'rttvar': 2850517, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001178979873657, 'end': 29.000942945480347, 'seconds': 0.9997639656066895, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.000942945480347, 'end': 30.00082492828369, 'seconds': 0.9998819828033447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 15343847, 'rttvar': 2933478, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.000942945480347, 'end': 30.00082492828369, 'seconds': 0.9998819828033447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.00082492828369, 'end': 31.00132989883423, 'seconds': 1.000504970550537, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 15745536, 'rttvar': 3003487, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.00082492828369, 'end': 31.00132989883423, 'seconds': 1.000504970550537, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00132989883423, 'end': 32.00117492675781, 'seconds': 0.999845027923584, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 16483554, 'rttvar': 2992105, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00132989883423, 'end': 32.00117492675781, 'seconds': 0.999845027923584, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00117492675781, 'end': 33.001261949539185, 'seconds': 1.000087022781372, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 16822879, 'rttvar': 2922730, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00117492675781, 'end': 33.001261949539185, 'seconds': 1.000087022781372, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.001261949539185, 'end': 34.000937938690186, 'seconds': 0.999675989151001, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 17530869, 'rttvar': 2868001, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.001261949539185, 'end': 34.000937938690186, 'seconds': 0.999675989151001, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.000937938690186, 'end': 35.00117087364197, 'seconds': 1.0002329349517822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 17904380, 'rttvar': 2898024, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.000937938690186, 'end': 35.00117087364197, 'seconds': 1.0002329349517822, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00117087364197, 'end': 36.000648021698, 'seconds': 0.9994771480560303, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 18205691, 'rttvar': 2776139, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00117087364197, 'end': 36.000648021698, 'seconds': 0.9994771480560303, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.000648021698, 'end': 37.00076603889465, 'seconds': 1.0001180171966553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 18668162, 'rttvar': 2363387, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.000648021698, 'end': 37.00076603889465, 'seconds': 1.0001180171966553, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00076603889465, 'end': 38.000908851623535, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 18854438, 'rttvar': 2145092, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00076603889465, 'end': 38.000908851623535, 'seconds': 1.0001428127288818, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.000908851623535, 'end': 39.000771045684814, 'seconds': 0.9998621940612793, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 19262133, 'rttvar': 1899274, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.000908851623535, 'end': 39.000771045684814, 'seconds': 0.9998621940612793, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.000771045684814, 'end': 40.001179933547974, 'seconds': 1.0004088878631592, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19606354, 'rttvar': 1688393, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.000771045684814, 'end': 40.001179933547974, 'seconds': 1.0004088878631592, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.001179933547974, 'end': 41.000771045684814, 'seconds': 0.9995911121368408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 19787843, 'rttvar': 1629273, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.001179933547974, 'end': 41.000771045684814, 'seconds': 0.9995911121368408, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.000771045684814, 'end': 42.00073790550232, 'seconds': 0.9999668598175049, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20177553, 'rttvar': 1615354, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.000771045684814, 'end': 42.00073790550232, 'seconds': 0.9999668598175049, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.00073790550232, 'end': 43.00074791908264, 'seconds': 1.0000100135803223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20382726, 'rttvar': 1621861, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.00073790550232, 'end': 43.00074791908264, 'seconds': 1.0000100135803223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00074791908264, 'end': 44.00077486038208, 'seconds': 1.0000269412994385, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 70952, 'rtt': 20914247, 'rttvar': 1856976, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00074791908264, 'end': 44.00077486038208, 'seconds': 1.0000269412994385, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00077486038208, 'end': 45.00122785568237, 'seconds': 1.000452995300293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21172231, 'rttvar': 1908701, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00077486038208, 'end': 45.00122785568237, 'seconds': 1.000452995300293, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00122785568237, 'end': 46.001184940338135, 'seconds': 0.9999570846557617, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21400211, 'rttvar': 1887485, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00122785568237, 'end': 46.001184940338135, 'seconds': 0.9999570846557617, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.001184940338135, 'end': 47.00070786476135, 'seconds': 0.9995229244232178, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 21941633, 'rttvar': 2000963, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.001184940338135, 'end': 47.00070786476135, 'seconds': 0.9995229244232178, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.00070786476135, 'end': 48.000715017318726, 'seconds': 1.000007152557373, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22161180, 'rttvar': 1939815, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.00070786476135, 'end': 48.000715017318726, 'seconds': 1.000007152557373, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.000715017318726, 'end': 49.00069785118103, 'seconds': 0.9999828338623047, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 22684914, 'rttvar': 1997608, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.000715017318726, 'end': 49.00069785118103, 'seconds': 0.9999828338623047, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.00069785118103, 'end': 50.00065302848816, 'seconds': 0.9999551773071289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 22891650, 'rttvar': 1911677, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.00069785118103, 'end': 50.00065302848816, 'seconds': 0.9999551773071289, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00065302848816, 'end': 51.00086688995361, 'seconds': 1.000213861465454, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 78192, 'rtt': 23394481, 'rttvar': 1945550, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00065302848816, 'end': 51.00086688995361, 'seconds': 1.000213861465454, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.00086688995361, 'end': 52.00068497657776, 'seconds': 0.9998180866241455, 'bytes': 220096, 'bits_per_second': 1761088.3655297516, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23587569, 'rttvar': 1845340, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.00086688995361, 'end': 52.00068497657776, 'seconds': 0.9998180866241455, 'bytes': 220096, 'bits_per_second': 1761088.3655297516, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00068497657776, 'end': 53.00066089630127, 'seconds': 0.9999759197235107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24041096, 'rttvar': 1816731, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00068497657776, 'end': 53.00066089630127, 'seconds': 0.9999759197235107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.00066089630127, 'end': 54.000673055648804, 'seconds': 1.0000121593475342, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24200677, 'rttvar': 1681710, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.00066089630127, 'end': 54.000673055648804, 'seconds': 1.0000121593475342, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.000673055648804, 'end': 55.000757932662964, 'seconds': 1.0000848770141602, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24430407, 'rttvar': 1720743, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.000673055648804, 'end': 55.000757932662964, 'seconds': 1.0000848770141602, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.000757932662964, 'end': 56.00077199935913, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 24801913, 'rttvar': 1610459, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.000757932662964, 'end': 56.00077199935913, 'seconds': 1.000014066696167, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.00077199935913, 'end': 57.00080895423889, 'seconds': 1.0000369548797607, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 25035986, 'rttvar': 1675990, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.00077199935913, 'end': 57.00080895423889, 'seconds': 1.0000369548797607, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.00080895423889, 'end': 58.001221895217896, 'seconds': 1.000412940979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 85432, 'rtt': 25423377, 'rttvar': 1614862, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.00080895423889, 'end': 58.001221895217896, 'seconds': 1.000412940979004, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001221895217896, 'end': 59.00071597099304, 'seconds': 0.9994940757751465, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25672772, 'rttvar': 1709936, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001221895217896, 'end': 59.00071597099304, 'seconds': 0.9994940757751465, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 11:08:45.486538 tst cs_paging_gprs_active.py:38: ERR: KeyError: 'sum_sent' [trial-2066↪gprs:sysmo↪cs_paging_gprs_active.py:38] 11:08:45.493986 tst cs_paging_gprs_active.py:38: Test FAILED (247.5 sec) 11:08:45.501206 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=17999): Terminating (SIGTERM) 11:08:45.543318 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18001): Terminating (SIGTERM) 11:08:45.585532 run iperf3-srv_10.42.42.10(pid=18002): Terminating (SIGTERM) 11:08:45.627688 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18005): Terminating (SIGTERM) 11:08:45.669235 run osmo-hlr_10.42.42.2(pid=18006): Terminating (SIGTERM) 11:08:45.710791 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18007): Terminating (SIGTERM) 11:08:45.753678 run osmo-stp_10.42.42.5(pid=18008): Terminating (SIGTERM) 11:08:45.797648 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18009): Terminating (SIGTERM) 11:08:45.842587 run osmo-ggsn_10.42.42.6(pid=18014): Terminating (SIGTERM) 11:08:45.886959 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18035): Terminating (SIGTERM) 11:08:45.944252 run osmo-sgsn_10.42.42.7(pid=18036): Terminating (SIGTERM) 11:08:46.010876 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18037): Terminating (SIGTERM) 11:08:46.057130 run osmo-msc_10.42.42.8(pid=18038): Terminating (SIGTERM) 11:08:46.118641 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18039): Terminating (SIGTERM) 11:08:46.196233 run osmo-mgw_10.42.42.3(pid=18040): Terminating (SIGTERM) 11:08:46.252945 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18041): Terminating (SIGTERM) 11:08:46.302106 run osmo-mgw_10.42.42.4(pid=18042): Terminating (SIGTERM) 11:08:46.347934 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18043): Terminating (SIGTERM) 11:08:46.392001 run osmo-bsc_10.42.42.9(pid=18047): Terminating (SIGTERM) 11:08:46.433995 run osmo-bts-sysmo(pid=18056): Terminating (SIGTERM) 11:08:46.475983 run osmo-pcu(pid=18065): Terminating (SIGTERM) 11:08:46.517944 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116): Terminating (SIGTERM) 11:08:46.657266 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGTERM(pid=18146): Launched 11:08:47.762948 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGTERM(pid=18146): Terminated: ok {rc=0} 11:08:47.826373 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119): Terminating (SIGTERM) 11:08:47.962314 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGTERM(pid=18150): Launched 11:08:49.045145 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGTERM(pid=18150): Terminated: ok {rc=0} 11:08:49.108943 --- ParallelTerminationStrategy: PID 17999 died... 11:08:49.128115 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=17999): Terminated: ok {rc=0} 11:08:49.173816 --- ParallelTerminationStrategy: PID 18001 died... 11:08:49.192857 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18001): Terminated: ok {rc=0} 11:08:49.237797 --- ParallelTerminationStrategy: PID 18002 died... 11:08:49.256658 run iperf3-srv_10.42.42.10(pid=18002): Terminated {rc=256} 11:08:49.301369 --- ParallelTerminationStrategy: PID 18005 died... 11:08:49.320300 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18005): Terminated: ok {rc=0} 11:08:49.365158 --- ParallelTerminationStrategy: PID 18006 died... 11:08:49.384550 run osmo-hlr_10.42.42.2(pid=18006): Terminated: ok {rc=0} 11:08:49.429205 --- ParallelTerminationStrategy: PID 18007 died... 11:08:49.448291 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18007): Terminated: ok {rc=0} 11:08:49.493101 --- ParallelTerminationStrategy: PID 18008 died... 11:08:49.512080 run osmo-stp_10.42.42.5(pid=18008): Terminated: ok {rc=0} 11:08:49.557281 --- ParallelTerminationStrategy: PID 18009 died... 11:08:49.576194 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18009): Terminated: ok {rc=0} 11:08:49.621128 --- ParallelTerminationStrategy: PID 18014 died... 11:08:49.639947 run osmo-ggsn_10.42.42.6(pid=18014): Terminated: ok {rc=0} 11:08:49.685064 --- ParallelTerminationStrategy: PID 18035 died... 11:08:49.703923 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18035): Terminated: ok {rc=0} 11:08:49.748970 --- ParallelTerminationStrategy: PID 18036 died... 11:08:49.767810 run osmo-sgsn_10.42.42.7(pid=18036): Terminated: ok {rc=0} 11:08:49.813243 --- ParallelTerminationStrategy: PID 18037 died... 11:08:49.832329 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18037): Terminated: ok {rc=0} 11:08:49.877738 --- ParallelTerminationStrategy: PID 18038 died... 11:08:49.896733 run osmo-msc_10.42.42.8(pid=18038): Terminated: ok {rc=0} 11:08:49.943377 --- ParallelTerminationStrategy: PID 18039 died... 11:08:49.962339 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18039): Terminated: ok {rc=0} 11:08:50.007089 --- ParallelTerminationStrategy: PID 18040 died... 11:08:50.026058 run osmo-mgw_10.42.42.3(pid=18040): Terminated {rc=15} 11:08:50.070905 --- ParallelTerminationStrategy: PID 18041 died... 11:08:50.089723 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18041): Terminated: ok {rc=0} 11:08:50.135126 --- ParallelTerminationStrategy: PID 18042 died... 11:08:50.154173 run osmo-mgw_10.42.42.4(pid=18042): Terminated {rc=15} 11:08:50.199272 --- ParallelTerminationStrategy: PID 18043 died... 11:08:50.218365 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18043): Terminated: ok {rc=0} 11:08:50.263341 --- ParallelTerminationStrategy: PID 18047 died... 11:08:50.282202 run osmo-bsc_10.42.42.9(pid=18047): Terminated: ok {rc=0} 11:08:50.327474 --- ParallelTerminationStrategy: PID 18056 died... 11:08:50.346504 run osmo-bts-sysmo(pid=18056): Terminated: ok {rc=0} 11:08:50.391255 --- ParallelTerminationStrategy: PID 18065 died... 11:08:50.410210 run osmo-pcu(pid=18065): Terminated: ok {rc=0} 11:08:56.277465 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116): Terminating (SIGINT) 11:08:56.415314 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGINT(pid=18153): Launched 11:08:57.498514 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGINT(pid=18153): Terminated: ok {rc=0} 11:08:57.561904 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119): Terminating (SIGINT) 11:08:57.698886 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGINT(pid=18156): Launched 11:08:58.784713 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGINT(pid=18156): Terminated: ok {rc=0} 11:09:04.670806 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116): Terminating (SIGKILL) 11:09:04.811831 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGUSR1(pid=18159): Launched 11:09:05.905827 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18116)-killSignals.SIGUSR1(pid=18159): Terminated: ok {rc=0} 11:09:05.972272 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119): Terminating (SIGKILL) 11:09:06.115142 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGUSR1(pid=18162): Launched 11:09:07.203541 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18119)-killSignals.SIGUSR1(pid=18162): Terminated: ok {rc=0} 11:09:07.459580 bus /gobi_0: Setting Online False 11:09:09.637248 bus /gobi_0: Setting Powered False 11:09:26.942156 bus /gobi_0: Setting Powered False 11:09:28.039852 bus /gobi_0: Setting Powered False 11:09:29.197698 bus /gobi_6: Setting Online False 11:09:30.307241 bus /gobi_6: Setting Powered False 11:09:38.939690 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2066 gprs:sysmo iperf3.py ---------------------------------------------- 11:09:40.150263 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.162137 tst gprs:sysmo: Using 1 x bts (candidates: 1) 11:09:40.198786 tst iperf3.py: using LAC 3960 11:09:40.325090 tst iperf3.py: using RAC 135 11:09:40.445129 tst iperf3.py: using CellId 3960 11:09:40.577603 tst iperf3.py: using BVCI 3961 11:09:40.689770 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.700405 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.710923 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.721528 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.732799 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.743367 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.753959 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.764495 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:09:40.775277 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:09:40.894355 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003... 11:09:41.099276 run iperf3-srv_10.42.42.10: Starting iperf3-srv 11:09:41.409219 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18165): Launched 11:09:41.677856 run iperf3-srv_10.42.42.10(pid=18166): Launched 11:09:41.791899 tst iperf3.py:8: start network... 11:09:41.995786 run osmo-hlr_10.42.42.2: Starting osmo-hlr 11:09:42.469871 run create_hlr_db(pid=18167): Launched 11:09:42.587312 bus /gobi_6: Setting Powered False 11:09:43.648883 run create_hlr_db(pid=18167): Terminated: ok {rc=0} 11:09:44.097956 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18169): Launched 11:09:44.429676 run osmo-hlr_10.42.42.2(pid=18170): Launched 11:09:44.555773 run osmo-stp_10.42.42.5: Starting osmo-stp 11:09:45.115255 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18171): Launched 11:09:45.443709 run osmo-stp_10.42.42.5(pid=18172): Launched 11:09:45.569067 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 11:09:46.130517 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18173): Launched 11:09:46.255373 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 11:09:46.533597 run patchelf(pid=18174): Launched 11:09:47.720118 run patchelf(pid=18174): Terminated: ok {rc=0} 11:09:47.857962 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 11:09:48.138803 run osmo-gsm-tester_setcap_net_admin.sh(pid=18175): Launched 11:09:49.326246 run osmo-gsm-tester_setcap_net_admin.sh(pid=18175): Terminated: ok {rc=0} 11:09:49.667705 run osmo-ggsn_10.42.42.6(pid=18178): Launched 11:09:49.802946 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 11:09:50.472489 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18199): Launched 11:09:50.782603 run osmo-sgsn_10.42.42.7(pid=18200): Launched 11:09:50.900581 run osmo-msc_10.42.42.8: Starting osmo-msc 11:09:51.459751 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18201): Launched 11:09:51.768415 run osmo-msc_10.42.42.8(pid=18202): Launched 11:09:54.943511 run osmo-mgw_10.42.42.3: Starting osmo-mgw 11:09:55.467653 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18203): Launched 11:09:55.774002 run osmo-mgw_10.42.42.3(pid=18204): Launched 11:09:55.892207 run osmo-mgw_10.42.42.4: Starting osmo-mgw 11:09:56.408389 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18205): Launched 11:09:56.717272 run osmo-mgw_10.42.42.4(pid=18206): Launched 11:09:56.835298 run osmo-bsc_10.42.42.9: Starting osmo-bsc 11:09:57.471435 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18207): Launched 11:09:57.779481 run osmo-bsc_10.42.42.9(pid=18208): Launched 11:10:04.051774 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 11:10:04.585470 run rm-remote-dir(pid=18212): Launched 11:10:06.769746 run rm-remote-dir(pid=18212): Terminated: ok {rc=0} 11:10:07.107261 run mk-remote-dir(pid=18213): Launched 11:10:08.288846 run mk-remote-dir(pid=18213): Terminated: ok {rc=0} 11:10:08.576761 run scp-inst-to-remote(pid=18214): Launched 11:10:20.771981 run scp-inst-to-remote(pid=18214): Terminated: ok {rc=0} 11:10:21.130210 run mk-remote-dir(pid=18216): Launched 11:10:22.321153 run mk-remote-dir(pid=18216): Terminated: ok {rc=0} 11:10:22.626026 run scp-cfg-to-remote(pid=18217): Launched 11:10:23.815427 run scp-cfg-to-remote(pid=18217): Terminated: ok {rc=0} 11:10:24.166668 run reload-dsp-firmware(pid=18219): Launched 11:10:26.351014 run reload-dsp-firmware(pid=18219): Terminated: ok {rc=0} 11:10:26.673000 run osmo-bts-sysmo(pid=18220): Launched 11:10:30.903801 tst iperf3.py:8: Waiting for bts to be ready... 11:10:32.628316 run rm-remote-dir(pid=18221): Launched 11:10:34.812608 run rm-remote-dir(pid=18221): Terminated: ok {rc=0} 11:10:35.157787 run mk-remote-dir(pid=18222): Launched 11:10:37.342868 run mk-remote-dir(pid=18222): Terminated: ok {rc=0} 11:10:37.638953 run scp-inst-to-bts(pid=18223): Launched 11:10:50.840886 run scp-inst-to-bts(pid=18223): Terminated: ok {rc=0} 11:10:51.163706 run mk-remote-run-dir(pid=18226): Launched 11:10:53.337810 run mk-remote-run-dir(pid=18226): Terminated: ok {rc=0} 11:10:53.615926 run scp-cfg-to-sysmobts(pid=18227): Launched 11:10:55.793036 run scp-cfg-to-sysmobts(pid=18227): Terminated: ok {rc=0} 11:10:56.140404 run osmo-pcu(pid=18229): Launched 11:10:56.279487 tst iperf3.py: using MSISDN 3973 11:10:56.447044 bus /gobi_6: Setting Powered True 11:10:59.947882 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3973', subscriber_id=1} 11:11:00.233818 bus /gobi_6: Setting Online False 11:11:01.428516 bus /gobi_6: Setting Powered False 11:11:08.191590 bus /gobi_6: Setting Powered True 11:11:09.591885 bus /gobi_6: Setting Online True 11:11:12.068419 tst /gobi_6: Connect to ('901', '70') 11:11:12.348188 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 11:11:12.467150 tst iperf3.py:8: waiting for modems to attach... 11:11:19.647434 tst /gobi_6: Already registered with network ('901', '70') 11:11:30.999160 tst iperf3.py:8: waiting for modems to attach to data services... 11:11:48.997021 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:11:49.107767 tst iperf3.py:8: Setting up data plan for "'/gobi_6/context3'" 11:11:49.533391 run move_netns(pid=18232): Launched 11:11:50.727258 run move_netns(pid=18232): Terminated: ok {rc=0} 11:11:51.083914 run ifup(pid=18235): Launched 11:11:52.277175 run ifup(pid=18235): Terminated: ok {rc=0} 11:11:52.623536 run dhcp(pid=18238): Launched 11:11:53.813217 run dhcp(pid=18238): Terminated: ok {rc=0} 11:11:53.943650 tst iperf3.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context3'" 11:11:54.154345 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 11:11:54.523616 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Launched 11:11:54.846922 run iperf3-cli_10.42.42.10(pid=18257): Launched 11:13:03.157279 run iperf3-cli_10.42.42.10(pid=18257): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=18257)] 11:13:03.289119 run iperf3-cli_10.42.42.10(pid=18257): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 11:13:03.421569 run iperf3-cli_10.42.42.10(pid=18257): stderr: | (launched: 2022-09-07_11:11:54.831390) 11:13:03.553575 run iperf3-srv_10.42.42.10(pid=18166): Terminating (SIGINT) 11:13:03.893630 run iperf3-srv_10.42.42.10(pid=18166): Terminated {rc=1} 11:13:04.018991 tst iperf3.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context3'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 46758, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Wed, 07 Sep 2022 09:11:56 GMT', 'timesecs': 1662541916}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'uzoh4dtndr3cftrovt2xrbxlzmcpi6zwtirj', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0008819103240967, 'seconds': 1.0008819103240967, 'bytes': 101360, 'bits_per_second': 810165.5066754359, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 429537, 'rttvar': 172019, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0008819103240967, 'seconds': 1.0008819103240967, 'bytes': 101360, 'bits_per_second': 810165.5066754359, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0008819103240967, 'end': 2.0012218952178955, 'seconds': 1.0003399848937988, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 516133, 'rttvar': 302207, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0008819103240967, 'end': 2.0012218952178955, 'seconds': 1.0003399848937988, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0012218952178955, 'end': 3.001326084136963, 'seconds': 1.0001041889190674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 18824, 'rtt': 899837, 'rttvar': 858283, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0012218952178955, 'end': 3.001326084136963, 'seconds': 1.0001041889190674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.001326084136963, 'end': 4.001185894012451, 'seconds': 0.9998598098754883, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1184790, 'rttvar': 1213618, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.001326084136963, 'end': 4.001185894012451, 'seconds': 0.9998598098754883, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001185894012451, 'end': 5.001333951950073, 'seconds': 1.000148057937622, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 1903891, 'rttvar': 1952341, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001185894012451, 'end': 5.001333951950073, 'seconds': 1.000148057937622, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001333951950073, 'end': 6.001183032989502, 'seconds': 0.9998490810394287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 2325793, 'rttvar': 2308060, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001333951950073, 'end': 6.001183032989502, 'seconds': 0.9998490810394287, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001183032989502, 'end': 7.001324892044067, 'seconds': 1.0001418590545654, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 2782671, 'rttvar': 2644801, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001183032989502, 'end': 7.001324892044067, 'seconds': 1.0001418590545654, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001324892044067, 'end': 8.001322031021118, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 3783260, 'rttvar': 3245423, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001324892044067, 'end': 8.001322031021118, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001322031021118, 'end': 9.001244068145752, 'seconds': 0.9999220371246338, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 4211214, 'rttvar': 3289975, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001322031021118, 'end': 9.001244068145752, 'seconds': 0.9999220371246338, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001244068145752, 'end': 10.00119400024414, 'seconds': 0.9999499320983887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 4644648, 'rttvar': 3334349, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001244068145752, 'end': 10.00119400024414, 'seconds': 0.9999499320983887, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.00119400024414, 'end': 11.001322984695435, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 5114064, 'rttvar': 3439594, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.00119400024414, 'end': 11.001322984695435, 'seconds': 1.000128984451294, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001322984695435, 'end': 12.001194953918457, 'seconds': 0.9998719692230225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 6047951, 'rttvar': 3553665, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001322984695435, 'end': 12.001194953918457, 'seconds': 0.9998719692230225, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.001194953918457, 'end': 13.001245021820068, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 6517251, 'rttvar': 3603850, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.001194953918457, 'end': 13.001245021820068, 'seconds': 1.0000500679016113, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001245021820068, 'end': 14.001223087310791, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 6930230, 'rttvar': 3528845, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001245021820068, 'end': 14.001223087310791, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001223087310791, 'end': 15.001184940338135, 'seconds': 0.9999618530273438, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 7766762, 'rttvar': 3434981, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001223087310791, 'end': 15.001184940338135, 'seconds': 0.9999618530273438, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001184940338135, 'end': 16.001183032989502, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 8195384, 'rttvar': 3433481, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001184940338135, 'end': 16.001183032989502, 'seconds': 0.9999980926513672, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.001183032989502, 'end': 17.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 8991751, 'rttvar': 3335099, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.001183032989502, 'end': 17.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001183032989502, 'end': 18.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 9355035, 'rttvar': 3227893, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001183032989502, 'end': 18.001183032989502, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001183032989502, 'end': 19.001282930374146, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 9760525, 'rttvar': 3231900, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001183032989502, 'end': 19.001282930374146, 'seconds': 1.0000998973846436, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.001282930374146, 'end': 20.001282930374146, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 10514881, 'rttvar': 3148101, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.001282930374146, 'end': 20.001282930374146, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.001282930374146, 'end': 21.00122094154358, 'seconds': 0.9999380111694336, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 10860707, 'rttvar': 3052726, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.001282930374146, 'end': 21.00122094154358, 'seconds': 0.9999380111694336, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.00122094154358, 'end': 22.001190900802612, 'seconds': 0.9999699592590332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11599317, 'rttvar': 2998039, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.00122094154358, 'end': 22.001190900802612, 'seconds': 0.9999699592590332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001190900802612, 'end': 23.001322984695435, 'seconds': 1.0001320838928223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 11994335, 'rttvar': 3038565, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001190900802612, 'end': 23.001322984695435, 'seconds': 1.0001320838928223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001322984695435, 'end': 24.001182079315186, 'seconds': 0.999859094619751, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 12349683, 'rttvar': 2989618, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001322984695435, 'end': 24.001182079315186, 'seconds': 0.999859094619751, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.001182079315186, 'end': 25.00127100944519, 'seconds': 1.0000889301300049, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 12767896, 'rttvar': 3078642, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.001182079315186, 'end': 25.00127100944519, 'seconds': 1.0000889301300049, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.00127100944519, 'end': 26.00040102005005, 'seconds': 0.9991300106048584, 'bytes': 256296, 'bits_per_second': 2052153.3516531426, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 13584206, 'rttvar': 3168691, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.00127100944519, 'end': 26.00040102005005, 'seconds': 0.9991300106048584, 'bytes': 256296, 'bits_per_second': 2052153.3516531426, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.00040102005005, 'end': 27.00122308731079, 'seconds': 1.0008220672607422, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 13870601, 'rttvar': 2949309, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.00040102005005, 'end': 27.00122308731079, 'seconds': 1.0008220672607422, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.00122308731079, 'end': 28.001182079315186, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14510112, 'rttvar': 2766268, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.00122308731079, 'end': 28.001182079315186, 'seconds': 0.9999589920043945, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.001182079315186, 'end': 29.00128412246704, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 14856161, 'rttvar': 2766799, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.001182079315186, 'end': 29.00128412246704, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.00128412246704, 'end': 30.001219987869263, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 57920, 'rtt': 15510681, 'rttvar': 2712826, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.00128412246704, 'end': 30.001219987869263, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.001219987869263, 'end': 31.001322031021118, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 15816495, 'rttvar': 2646247, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.001219987869263, 'end': 31.001322031021118, 'seconds': 1.0001020431518555, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.001322031021118, 'end': 32.00118398666382, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 59368, 'rtt': 16171722, 'rttvar': 2695138, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.001322031021118, 'end': 32.00118398666382, 'seconds': 0.9998619556427002, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00118398666382, 'end': 33.00118398666382, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 60816, 'rtt': 16837129, 'rttvar': 2693977, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00118398666382, 'end': 33.00118398666382, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00118398666382, 'end': 34.000471115112305, 'seconds': 0.9992871284484863, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 62264, 'rtt': 17409705, 'rttvar': 2529060, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00118398666382, 'end': 34.000471115112305, 'seconds': 0.9992871284484863, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.000471115112305, 'end': 35.001286029815674, 'seconds': 1.0008149147033691, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 63712, 'rtt': 17678190, 'rttvar': 2433765, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.000471115112305, 'end': 35.001286029815674, 'seconds': 1.0008149147033691, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.001286029815674, 'end': 36.00027894973755, 'seconds': 0.998992919921875, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18278383, 'rttvar': 2409482, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.001286029815674, 'end': 36.00027894973755, 'seconds': 0.998992919921875, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.00027894973755, 'end': 37.00119090080261, 'seconds': 1.0009119510650635, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 65160, 'rtt': 18613768, 'rttvar': 2477881, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.00027894973755, 'end': 37.00119090080261, 'seconds': 1.0009119510650635, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00119090080261, 'end': 38.00023102760315, 'seconds': 0.9990401268005371, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 66608, 'rtt': 19241917, 'rttvar': 2505975, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00119090080261, 'end': 38.00023102760315, 'seconds': 0.9990401268005371, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00023102760315, 'end': 39.0010929107666, 'seconds': 1.0008618831634521, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 68056, 'rtt': 19538757, 'rttvar': 2473162, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00023102760315, 'end': 39.0010929107666, 'seconds': 1.0008618831634521, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.0010929107666, 'end': 40.00030994415283, 'seconds': 0.9992170333862305, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20033526, 'rttvar': 2249418, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.0010929107666, 'end': 40.00030994415283, 'seconds': 0.9992170333862305, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00030994415283, 'end': 41.00133490562439, 'seconds': 1.0010249614715576, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 69504, 'rtt': 20324344, 'rttvar': 2268699, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00030994415283, 'end': 41.00133490562439, 'seconds': 1.0010249614715576, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00133490562439, 'end': 42.00131607055664, 'seconds': 0.999981164932251, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 70952, 'rtt': 20575846, 'rttvar': 2204529, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00133490562439, 'end': 42.00131607055664, 'seconds': 0.999981164932251, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.00131607055664, 'end': 43.00129199028015, 'seconds': 0.9999759197235107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21152923, 'rttvar': 2240313, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.00131607055664, 'end': 43.00129199028015, 'seconds': 0.9999759197235107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00129199028015, 'end': 44.00128412246704, 'seconds': 0.9999921321868896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 72400, 'rtt': 21390857, 'rttvar': 2156102, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00129199028015, 'end': 44.00128412246704, 'seconds': 0.9999921321868896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00128412246704, 'end': 45.00132393836975, 'seconds': 1.00003981590271, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 73848, 'rtt': 21944667, 'rttvar': 2171328, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00128412246704, 'end': 45.00132393836975, 'seconds': 1.00003981590271, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00132393836975, 'end': 46.00119400024414, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 75296, 'rtt': 22166043, 'rttvar': 2071249, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00132393836975, 'end': 46.00119400024414, 'seconds': 0.9998700618743896, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00119400024414, 'end': 47.00120401382446, 'seconds': 1.0000100135803223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 22688497, 'rttvar': 2069277, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00119400024414, 'end': 47.00120401382446, 'seconds': 1.0000100135803223, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.00120401382446, 'end': 48.001296043395996, 'seconds': 1.0000920295715332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 76744, 'rtt': 22892047, 'rttvar': 1959057, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.00120401382446, 'end': 48.001296043395996, 'seconds': 1.0000920295715332, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.001296043395996, 'end': 49.001302003860474, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 78192, 'rtt': 23160095, 'rttvar': 2005389, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.001296043395996, 'end': 49.001302003860474, 'seconds': 1.0000059604644775, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.001302003860474, 'end': 50.00115704536438, 'seconds': 0.9998550415039062, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23523362, 'rttvar': 1749697, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.001302003860474, 'end': 50.00115704536438, 'seconds': 0.9998550415039062, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00115704536438, 'end': 51.0013370513916, 'seconds': 1.0001800060272217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 79640, 'rtt': 23747679, 'rttvar': 1760909, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00115704536438, 'end': 51.0013370513916, 'seconds': 1.0001800060272217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.0013370513916, 'end': 52.001291036605835, 'seconds': 0.9999539852142334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 81088, 'rtt': 24127079, 'rttvar': 1648911, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.0013370513916, 'end': 52.001291036605835, 'seconds': 0.9999539852142334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.001291036605835, 'end': 53.001343965530396, 'seconds': 1.0000529289245605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 82536, 'rtt': 24371031, 'rttvar': 1724587, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.001291036605835, 'end': 53.001343965530396, 'seconds': 1.0000529289245605, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.001343965530396, 'end': 54.00108790397644, 'seconds': 0.9997439384460449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 24761265, 'rttvar': 1647864, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.001343965530396, 'end': 54.00108790397644, 'seconds': 0.9997439384460449, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.00108790397644, 'end': 55.00133490562439, 'seconds': 1.0002470016479492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 83984, 'rtt': 25018324, 'rttvar': 1750017, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.00108790397644, 'end': 55.00133490562439, 'seconds': 1.0002470016479492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.00133490562439, 'end': 56.001347064971924, 'seconds': 1.0000121593475342, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 85432, 'rtt': 25243236, 'rttvar': 1762336, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.00133490562439, 'end': 56.001347064971924, 'seconds': 1.0000121593475342, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.001347064971924, 'end': 57.00115609169006, 'seconds': 0.9998090267181396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25692832, 'rttvar': 1793209, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.001347064971924, 'end': 57.00115609169006, 'seconds': 0.9998090267181396, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.00115609169006, 'end': 58.001091957092285, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 86880, 'rtt': 25921216, 'rttvar': 1801676, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.00115609169006, 'end': 58.001091957092285, 'seconds': 0.9999358654022217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.001091957092285, 'end': 59.001336097717285, 'seconds': 1.000244140625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 88328, 'rtt': 26385834, 'rttvar': 1842781, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.001091957092285, 'end': 59.001336097717285, 'seconds': 1.000244140625, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 11:13:04.255320 tst iperf3.py:8: ERR: KeyError: 'sum_sent' [trial-2066↪gprs:sysmo↪iperf3.py:8] 11:13:04.263636 tst iperf3.py:8: Test FAILED (204.2 sec) 11:13:04.270845 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18165): Terminating (SIGTERM) 11:13:04.314557 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18169): Terminating (SIGTERM) 11:13:04.359563 run osmo-hlr_10.42.42.2(pid=18170): Terminating (SIGTERM) 11:13:04.402283 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18171): Terminating (SIGTERM) 11:13:04.447378 run osmo-stp_10.42.42.5(pid=18172): Terminating (SIGTERM) 11:13:04.497587 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18173): Terminating (SIGTERM) 11:13:04.548729 run osmo-ggsn_10.42.42.6(pid=18178): Terminating (SIGTERM) 11:13:04.595315 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18199): Terminating (SIGTERM) 11:13:04.688440 run osmo-sgsn_10.42.42.7(pid=18200): Terminating (SIGTERM) 11:13:04.758854 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18201): Terminating (SIGTERM) 11:13:04.805754 run osmo-msc_10.42.42.8(pid=18202): Terminating (SIGTERM) 11:13:04.857642 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18203): Terminating (SIGTERM) 11:13:04.905237 run osmo-mgw_10.42.42.3(pid=18204): Terminating (SIGTERM) 11:13:04.949056 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18205): Terminating (SIGTERM) 11:13:04.993520 run osmo-mgw_10.42.42.4(pid=18206): Terminating (SIGTERM) 11:13:05.040144 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18207): Terminating (SIGTERM) 11:13:05.085216 run osmo-bsc_10.42.42.9(pid=18208): Terminating (SIGTERM) 11:13:05.131132 run osmo-bts-sysmo(pid=18220): Terminating (SIGTERM) 11:13:05.177081 run osmo-pcu(pid=18229): Terminating (SIGTERM) 11:13:05.221727 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Terminating (SIGTERM) 11:13:05.368871 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGTERM(pid=18280): Launched 11:13:06.455494 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGTERM(pid=18280): Terminated: ok {rc=0} 11:13:06.520172 --- ParallelTerminationStrategy: PID 18165 died... 11:13:06.539697 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18165): Terminated: ok {rc=0} 11:13:06.585853 --- ParallelTerminationStrategy: PID 18169 died... 11:13:06.605227 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18169): Terminated: ok {rc=0} 11:13:06.650556 --- ParallelTerminationStrategy: PID 18170 died... 11:13:06.670075 run osmo-hlr_10.42.42.2(pid=18170): Terminated: ok {rc=0} 11:13:06.715461 --- ParallelTerminationStrategy: PID 18171 died... 11:13:06.734826 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18171): Terminated: ok {rc=0} 11:13:06.780183 --- ParallelTerminationStrategy: PID 18172 died... 11:13:06.799335 run osmo-stp_10.42.42.5(pid=18172): Terminated: ok {rc=0} 11:13:06.844754 --- ParallelTerminationStrategy: PID 18173 died... 11:13:06.863966 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18173): Terminated: ok {rc=0} 11:13:06.908893 --- ParallelTerminationStrategy: PID 18178 died... 11:13:06.928175 run osmo-ggsn_10.42.42.6(pid=18178): Terminated: ok {rc=0} 11:13:06.974025 --- ParallelTerminationStrategy: PID 18199 died... 11:13:06.993398 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18199): Terminated: ok {rc=0} 11:13:07.038535 --- ParallelTerminationStrategy: PID 18200 died... 11:13:07.058462 run osmo-sgsn_10.42.42.7(pid=18200): Terminated: ok {rc=0} 11:13:07.103952 --- ParallelTerminationStrategy: PID 18201 died... 11:13:07.123264 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18201): Terminated: ok {rc=0} 11:13:07.168677 --- ParallelTerminationStrategy: PID 18202 died... 11:13:07.188045 run osmo-msc_10.42.42.8(pid=18202): Terminated: ok {rc=0} 11:13:07.233656 --- ParallelTerminationStrategy: PID 18203 died... 11:13:07.253044 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18203): Terminated: ok {rc=0} 11:13:07.299257 --- ParallelTerminationStrategy: PID 18204 died... 11:13:07.318412 run osmo-mgw_10.42.42.3(pid=18204): Terminated {rc=15} 11:13:07.364248 --- ParallelTerminationStrategy: PID 18205 died... 11:13:07.383661 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18205): Terminated: ok {rc=0} 11:13:07.429020 --- ParallelTerminationStrategy: PID 18206 died... 11:13:07.448622 run osmo-mgw_10.42.42.4(pid=18206): Terminated {rc=15} 11:13:07.494048 --- ParallelTerminationStrategy: PID 18207 died... 11:13:07.513407 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18207): Terminated: ok {rc=0} 11:13:07.559731 --- ParallelTerminationStrategy: PID 18208 died... 11:13:07.579036 run osmo-bsc_10.42.42.9(pid=18208): Terminated: ok {rc=0} 11:13:07.624468 --- ParallelTerminationStrategy: PID 18220 died... 11:13:07.643914 run osmo-bts-sysmo(pid=18220): Terminated: ok {rc=0} 11:13:07.689997 --- ParallelTerminationStrategy: PID 18229 died... 11:13:07.709524 run osmo-pcu(pid=18229): Terminated: ok {rc=0} 11:13:13.577586 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Terminating (SIGINT) 11:13:13.716304 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGINT(pid=18284): Launched 11:13:14.804576 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGINT(pid=18284): Terminated: ok {rc=0} 11:13:20.690494 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254): Terminating (SIGKILL) 11:13:20.828779 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGUSR1(pid=18287): Launched 11:13:21.917433 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18254)-killSignals.SIGUSR1(pid=18287): Terminated: ok {rc=0} 11:13:22.168393 bus /gobi_6: Setting Online False 11:13:25.214842 bus /gobi_6: Setting Powered False 11:13:40.058039 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2066 gprs:sysmo iperf3m4.py ---------------------------------------------- 11:13:41.269794 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.280975 tst gprs:sysmo: Using 1 x bts (candidates: 1) 11:13:41.313658 tst iperf3m4.py: using LAC 3961 11:13:41.435938 tst iperf3m4.py: using RAC 136 11:13:41.558148 tst iperf3m4.py: using CellId 3961 11:13:41.678506 tst iperf3m4.py: using BVCI 3962 11:13:41.791980 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.802506 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.812992 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.823635 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.834050 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.844506 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.855266 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.865903 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:13:41.876707 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:13:41.993417 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:13:42.110854 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:13:42.230150 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:13:42.346439 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003... 11:13:42.554418 run iperf3-srv_10.42.42.10: Starting iperf3-srv 11:13:42.866051 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Launched 11:13:43.138871 run iperf3-srv_10.42.42.10(pid=18291): Launched 11:13:43.256201 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004... 11:13:43.464412 run iperf3-srv_10.42.42.10: Starting iperf3-srv 11:13:43.775164 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Launched 11:13:44.045960 run iperf3-srv_10.42.42.10(pid=18293): Launched 11:13:44.162390 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005... 11:13:44.368125 run iperf3-srv_10.42.42.10: Starting iperf3-srv 11:13:44.677734 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Launched 11:13:44.948537 run iperf3-srv_10.42.42.10(pid=18295): Launched 11:13:45.065182 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006... 11:13:45.270882 run iperf3-srv_10.42.42.10: Starting iperf3-srv 11:13:45.581080 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Launched 11:13:45.851581 run iperf3-srv_10.42.42.10(pid=18297): Launched 11:13:45.966390 tst iperf3m4.py:8: start network... 11:13:46.172982 run osmo-hlr_10.42.42.2: Starting osmo-hlr 11:13:46.651034 run create_hlr_db(pid=18298): Launched 11:13:46.777524 bus /gobi_6: Setting Powered False 11:13:47.839600 run create_hlr_db(pid=18298): Terminated: ok {rc=0} 11:13:48.289546 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Launched 11:13:48.621197 run osmo-hlr_10.42.42.2(pid=18301): Launched 11:13:48.747454 run osmo-stp_10.42.42.5: Starting osmo-stp 11:13:49.315037 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Launched 11:13:49.648006 run osmo-stp_10.42.42.5(pid=18303): Launched 11:13:49.773762 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 11:13:50.338749 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Launched 11:13:50.465028 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 11:13:50.744236 run patchelf(pid=18305): Launched 11:13:51.932732 run patchelf(pid=18305): Terminated: ok {rc=0} 11:13:52.071359 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 11:13:52.353850 run osmo-gsm-tester_setcap_net_admin.sh(pid=18306): Launched 11:13:53.544564 run osmo-gsm-tester_setcap_net_admin.sh(pid=18306): Terminated: ok {rc=0} 11:13:53.890554 run osmo-ggsn_10.42.42.6(pid=18309): Launched 11:13:54.015745 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 11:13:54.757988 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Launched 11:13:55.090009 run osmo-sgsn_10.42.42.7(pid=18335): Launched 11:13:55.217492 run osmo-msc_10.42.42.8: Starting osmo-msc 11:13:55.836133 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Launched 11:13:56.149683 run osmo-msc_10.42.42.8(pid=18337): Launched 11:13:59.325397 run osmo-mgw_10.42.42.3: Starting osmo-mgw 11:13:59.855951 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Launched 11:14:00.165240 run osmo-mgw_10.42.42.3(pid=18339): Launched 11:14:00.284323 run osmo-mgw_10.42.42.4: Starting osmo-mgw 11:14:00.803053 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Launched 11:14:01.112489 run osmo-mgw_10.42.42.4(pid=18341): Launched 11:14:01.230208 run osmo-bsc_10.42.42.9: Starting osmo-bsc 11:14:01.872074 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Launched 11:14:02.181958 run osmo-bsc_10.42.42.9(pid=18343): Launched 11:14:08.457450 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 11:14:08.996694 run rm-remote-dir(pid=18344): Launched 11:14:11.183910 run rm-remote-dir(pid=18344): Terminated: ok {rc=0} 11:14:11.525640 run mk-remote-dir(pid=18345): Launched 11:14:12.709840 run mk-remote-dir(pid=18345): Terminated: ok {rc=0} 11:14:13.000821 run scp-inst-to-remote(pid=18346): Launched 11:14:25.197056 run scp-inst-to-remote(pid=18346): Terminated: ok {rc=0} 11:14:25.533786 run mk-remote-dir(pid=18348): Launched 11:14:26.714036 run mk-remote-dir(pid=18348): Terminated: ok {rc=0} 11:14:27.003281 run scp-cfg-to-remote(pid=18349): Launched 11:14:28.185728 run scp-cfg-to-remote(pid=18349): Terminated: ok {rc=0} 11:14:28.519337 run reload-dsp-firmware(pid=18351): Launched 11:14:30.698882 run reload-dsp-firmware(pid=18351): Terminated: ok {rc=0} 11:14:31.024129 run osmo-bts-sysmo(pid=18352): Launched 11:14:35.256943 tst iperf3m4.py:8: Waiting for bts to be ready... 11:14:36.976024 run rm-remote-dir(pid=18353): Launched 11:14:39.163238 run rm-remote-dir(pid=18353): Terminated: ok {rc=0} 11:14:39.513317 run mk-remote-dir(pid=18354): Launched 11:14:41.700768 run mk-remote-dir(pid=18354): Terminated: ok {rc=0} 11:14:42.000967 run scp-inst-to-bts(pid=18355): Launched 11:14:55.207562 run scp-inst-to-bts(pid=18355): Terminated: ok {rc=0} 11:14:55.537645 run mk-remote-run-dir(pid=18362): Launched 11:14:57.714414 run mk-remote-run-dir(pid=18362): Terminated: ok {rc=0} 11:14:57.994792 run scp-cfg-to-sysmobts(pid=18363): Launched 11:15:00.168929 run scp-cfg-to-sysmobts(pid=18363): Terminated: ok {rc=0} 11:15:00.496674 run osmo-pcu(pid=18365): Launched 11:15:00.623957 tst iperf3m4.py: using MSISDN 3974 11:15:00.801526 bus /gobi_6: Setting Powered True 11:15:04.344215 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3974', subscriber_id=1} 11:15:04.623385 bus /gobi_6: Setting Online False 11:15:05.814386 bus /gobi_6: Setting Powered False 11:15:13.122631 bus /gobi_6: Setting Powered True 11:15:14.525849 bus /gobi_6: Setting Online True 11:15:17.116610 tst /gobi_6: Connect to ('901', '70') 11:15:17.393574 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 11:15:17.521137 tst iperf3m4.py: using MSISDN 3975 11:15:17.751259 bus /gobi_0: Setting Powered True 11:15:21.267712 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023847', msisdn='3975', subscriber_id=2} 11:15:21.537818 bus /gobi_0: Setting Online False 11:15:22.725546 bus /gobi_0: Setting Powered False 11:15:30.114189 bus /gobi_0: Setting Powered True 11:15:31.471794 bus /gobi_0: Setting Online True 11:15:32.323742 tst /gobi_6: Already registered with network ('901', '70') 11:15:34.125600 tst /gobi_0: Connect to ('901', '70') 11:15:34.409462 tst /gobi_0: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024271873' 11:15:34.538319 tst iperf3m4.py: using MSISDN 3976 11:15:34.771169 bus /gobi_1: Setting Powered True 11:15:38.389429 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023848', msisdn='3976', subscriber_id=3} 11:15:38.666355 bus /gobi_1: Setting Online False 11:15:39.863655 bus /gobi_1: Setting Powered False 11:15:47.133763 bus /gobi_1: Setting Powered True 11:15:48.532390 bus /gobi_1: Setting Online True 11:15:50.141716 tst /gobi_0: Already registered with network ('901', '70') 11:15:51.039537 tst /gobi_1: Connect to ('901', '70') 11:15:51.301288 tst /gobi_1: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A05E2G 1 [Mar 14 2016 19:33:01]', Serial: '868323024272103' 11:15:51.418502 tst iperf3m4.py: using MSISDN 3977 11:15:51.588386 bus /gobi_4: Setting Powered True 11:15:55.116634 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000023849', msisdn='3977', subscriber_id=4} 11:15:55.398665 bus /gobi_4: Setting Online False 11:15:56.598014 bus /gobi_4: Setting Powered False 11:16:03.353570 bus /gobi_4: Setting Powered True 11:16:04.726015 bus /gobi_4: Setting Online True 11:16:07.183735 tst /gobi_4: Connect to ('901', '70') 11:16:07.455859 tst /gobi_4: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A08E2G 1 [Jul 21 2016 10:00:50]', Serial: '868323025161511' 11:16:07.565009 tst iperf3m4.py:8: waiting for modems to attach... 11:16:07.909355 tst /gobi_1: Already registered with network ('901', '70') 11:16:23.932124 tst /gobi_4: Already registered with network ('901', '70') 11:16:29.272614 tst iperf3m4.py:8: waiting for modems to attach to data services... 11:16:32.381617 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:16:32.492159 tst iperf3m4.py:8: Setting up data plan for "'/gobi_6/context2'" 11:16:32.909860 run move_netns(pid=18898): Launched 11:16:34.090592 run move_netns(pid=18898): Terminated: ok {rc=0} 11:16:34.421967 run ifup(pid=18901): Launched 11:16:35.606995 run ifup(pid=18901): Terminated: ok {rc=0} 11:16:35.939582 run dhcp(pid=18904): Launched 11:16:37.130982 run dhcp(pid=18904): Terminated: ok {rc=0} 11:16:45.533429 tst /gobi_0: context activated {apn='inet46', path='/gobi_0/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.2', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='r82dcf628584c1c', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:16:45.652996 tst iperf3m4.py:8: Setting up data plan for "'/gobi_0/context2'" 11:16:46.105881 run move_netns(pid=18922): Launched 11:16:47.302340 run move_netns(pid=18922): Terminated: ok {rc=0} 11:16:47.658389 run ifup(pid=18925): Launched 11:16:48.842098 run ifup(pid=18925): Terminated: ok {rc=0} 11:16:49.172840 run dhcp(pid=18928): Launched 11:16:50.355050 run dhcp(pid=18928): Terminated: ok {rc=0} 11:16:58.718078 tst /gobi_1: context activated {apn='inet46', path='/gobi_1/context11', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.3', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.1', Interface='rf303c5eb721dad', Method='static', Netmask='255.255.255.248'}, Type='internet', Username='ogt'}, user='ogt'} 11:16:58.838153 tst iperf3m4.py:8: Setting up data plan for "'/gobi_1/context11'" 11:16:59.290042 run move_netns(pid=18946): Launched 11:17:00.473121 run move_netns(pid=18946): Terminated: ok {rc=0} 11:17:00.803190 run ifup(pid=18949): Launched 11:17:01.983352 run ifup(pid=18949): Terminated: ok {rc=0} 11:17:02.312716 run dhcp(pid=18955): Launched 11:17:03.506370 run dhcp(pid=18955): Terminated: ok {rc=0} 11:17:11.902197 tst /gobi_4: context activated {apn='inet46', path='/gobi_4/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.4', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.5', Interface='r9d788e55295d1b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:17:12.012778 tst iperf3m4.py:8: Setting up data plan for "'/gobi_4/context2'" 11:17:12.430409 run move_netns(pid=18973): Launched 11:17:13.615647 run move_netns(pid=18973): Terminated: ok {rc=0} 11:17:13.945764 run ifup(pid=18976): Launched 11:17:15.139486 run ifup(pid=18976): Terminated: ok {rc=0} 11:17:15.493344 run dhcp(pid=18979): Launched 11:17:16.685317 run dhcp(pid=18979): Terminated: ok {rc=0} 11:17:16.823960 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5003 through "'/gobi_6/context2'" 11:17:17.050679 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5003 (proto=tcp,time=60s) 11:17:17.443272 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Launched 11:17:17.620309 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5004 through "'/gobi_0/context2'" 11:17:17.830360 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5004 (proto=tcp,time=60s) 11:17:18.196285 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Launched 11:17:18.369176 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5005 through "'/gobi_1/context11'" 11:17:18.579430 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5005 (proto=tcp,time=60s) 11:17:18.944731 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Launched 11:17:19.115343 tst iperf3m4.py:8: Running iperf3 client to 10.42.42.10:5006 through "'/gobi_4/context2'" 11:17:19.324508 run iperf3-cli_10.42.42.10: Preparing iperf3-client connecting to 10.42.42.10:5006 (proto=tcp,time=60s) 11:17:19.691083 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Launched 11:17:20.012037 run iperf3-cli_10.42.42.10(pid=19007): Launched 11:17:20.279961 run iperf3-cli_10.42.42.10(pid=19010): Launched 11:17:20.550378 run iperf3-cli_10.42.42.10(pid=19013): Launched 11:17:20.819758 run iperf3-cli_10.42.42.10(pid=19016): Launched 11:18:29.121289 run iperf3-cli_10.42.42.10(pid=19007): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19007)] 11:18:29.268554 run iperf3-cli_10.42.42.10(pid=19007): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 11:18:29.412519 run iperf3-cli_10.42.42.10(pid=19007): stderr: | (launched: 2022-09-07_11:17:19.996357) 11:18:35.615531 run iperf3-cli_10.42.42.10(pid=19010): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19010)] 11:18:35.748293 run iperf3-cli_10.42.42.10(pid=19010): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 11:18:35.877600 run iperf3-cli_10.42.42.10(pid=19010): stderr: | (launched: 2022-09-07_11:17:20.264381) 11:18:37.059998 run iperf3-cli_10.42.42.10(pid=19013): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19013)] 11:18:37.188907 run iperf3-cli_10.42.42.10(pid=19013): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 11:18:37.317018 run iperf3-cli_10.42.42.10(pid=19013): stderr: | (launched: 2022-09-07_11:17:20.534404) 11:18:38.501973 run iperf3-cli_10.42.42.10(pid=19016): ERR: Terminated: ERROR {rc=1} [trial-2066↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪iperf3-srv_10.42.42.10↪iperf3-cli_10.42.42.10↪iperf3-cli_10.42.42.10(pid=19016)] 11:18:38.633936 run iperf3-cli_10.42.42.10(pid=19016): stdout: | "bits_per_second": 0, | "retransmits": 0, | "omitted": false | } | }], | "end": { | }, | "error": "error - control socket has closed unexpectedly" | } | child exited with 1 11:18:38.765409 run iperf3-cli_10.42.42.10(pid=19016): stderr: | (launched: 2022-09-07_11:17:20.804104) 11:18:38.894613 run iperf3-srv_10.42.42.10(pid=18291): Terminating (SIGINT) 11:18:39.235425 run iperf3-srv_10.42.42.10(pid=18291): Terminated {rc=1} 11:18:39.366741 tst iperf3m4.py:8: Results for 10.42.42.10:5003 through "'/gobi_6/context2'" Exception while using iperf3 client results: "{'start': {'connected': [{'socket': 5, 'local_host': '176.16.46.1', 'local_port': 46808, 'remote_host': '10.42.42.10', 'remote_port': 5003}], 'version': 'iperf 3.6', 'system_info': 'Linux osmo-gsm-tester-prod 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64', 'timestamp': {'time': 'Wed, 07 Sep 2022 09:17:21 GMT', 'timesecs': 1662542241}, 'connecting_to': {'host': '10.42.42.10', 'port': 5003}, 'cookie': 'gb3tn5uvljqx37b2fk4tcfcx2rwfvavcpihs', 'tcp_mss_default': 1448, 'sock_bufsize': 0, 'sndbuf_actual': 16384, 'rcvbuf_actual': 131072, 'test_start': {'protocol': 'TCP', 'num_streams': 1, 'blksize': 131072, 'omit': 0, 'duration': 60, 'bytes': 0, 'blocks': 0, 'reverse': 0, 'tos': 0}}, 'intervals': [{'streams': [{'socket': 5, 'start': 0, 'end': 1.0006179809570312, 'seconds': 1.0006179809570312, 'bytes': 101360, 'bits_per_second': 810379.2010857548, 'retransmits': 0, 'snd_cwnd': 14480, 'rtt': 451757, 'rttvar': 184583, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 0, 'end': 1.0006179809570312, 'seconds': 1.0006179809570312, 'bytes': 101360, 'bits_per_second': 810379.2010857548, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 1.0006179809570312, 'end': 2.0012199878692627, 'seconds': 1.0006020069122314, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 15928, 'rtt': 582836, 'rttvar': 400596, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 1.0006179809570312, 'end': 2.0012199878692627, 'seconds': 1.0006020069122314, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 2.0012199878692627, 'end': 3.0012929439544678, 'seconds': 1.000072956085205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'snd_cwnd': 4344, 'rtt': 582836, 'rttvar': 400596, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 2.0012199878692627, 'end': 3.0012929439544678, 'seconds': 1.000072956085205, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 1, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 3.0012929439544678, 'end': 4.001316070556641, 'seconds': 1.0000231266021729, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 17376, 'rtt': 869981, 'rttvar': 874738, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 3.0012929439544678, 'end': 4.001316070556641, 'seconds': 1.0000231266021729, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 4.001316070556641, 'end': 5.001352071762085, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1313938, 'rttvar': 1543967, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 4.001316070556641, 'end': 5.001352071762085, 'seconds': 1.0000360012054443, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 5.001352071762085, 'end': 6.001249074935913, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 18824, 'rtt': 1313938, 'rttvar': 1543967, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 5.001352071762085, 'end': 6.001249074935913, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 6.001249074935913, 'end': 7.001249074935913, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 20272, 'rtt': 1894370, 'rttvar': 2318839, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 6.001249074935913, 'end': 7.001249074935913, 'seconds': 1, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 7.001249074935913, 'end': 8.001253128051758, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2589750, 'rttvar': 3129890, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 7.001249074935913, 'end': 8.001253128051758, 'seconds': 1.0000040531158447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 8.001253128051758, 'end': 9.001287937164307, 'seconds': 1.0000348091125488, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 21720, 'rtt': 2589750, 'rttvar': 3129890, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 8.001253128051758, 'end': 9.001287937164307, 'seconds': 1.0000348091125488, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 9.001287937164307, 'end': 10.001184940338135, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 23168, 'rtt': 3388632, 'rttvar': 3945180, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 9.001287937164307, 'end': 10.001184940338135, 'seconds': 0.9998970031738281, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 10.001184940338135, 'end': 11.001253128051758, 'seconds': 1.000068187713623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4269873, 'rttvar': 4721368, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 10.001184940338135, 'end': 11.001253128051758, 'seconds': 1.000068187713623, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 11.001253128051758, 'end': 12.00125503540039, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 24616, 'rtt': 4269873, 'rttvar': 4721368, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 11.001253128051758, 'end': 12.00125503540039, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 12.00125503540039, 'end': 13.001213073730469, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 26064, 'rtt': 5228419, 'rttvar': 5458117, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 12.00125503540039, 'end': 13.001213073730469, 'seconds': 0.9999580383300781, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 13.001213073730469, 'end': 14.001102924346924, 'seconds': 0.9998898506164551, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6269643, 'rttvar': 6176036, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 13.001213073730469, 'end': 14.001102924346924, 'seconds': 0.9998898506164551, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 14.001102924346924, 'end': 15.001307010650635, 'seconds': 1.000204086303711, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 27512, 'rtt': 6269643, 'rttvar': 6176036, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 14.001102924346924, 'end': 15.001307010650635, 'seconds': 1.000204086303711, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 15.001307010650635, 'end': 16.00133991241455, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7370460, 'rttvar': 6833661, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 15.001307010650635, 'end': 16.00133991241455, 'seconds': 1.000032901763916, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 16.00133991241455, 'end': 17.001264095306396, 'seconds': 0.9999241828918457, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 28960, 'rtt': 7370460, 'rttvar': 6833661, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 16.00133991241455, 'end': 17.001264095306396, 'seconds': 0.9999241828918457, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 17.001264095306396, 'end': 18.001466035842896, 'seconds': 1.000201940536499, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 30408, 'rtt': 8413339, 'rttvar': 7211003, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 17.001264095306396, 'end': 18.001466035842896, 'seconds': 1.000201940536499, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 18.001466035842896, 'end': 19.00122594833374, 'seconds': 0.9997599124908447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9441186, 'rttvar': 7463947, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 18.001466035842896, 'end': 19.00122594833374, 'seconds': 0.9997599124908447, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 19.00122594833374, 'end': 20.00122308731079, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9441186, 'rttvar': 7463947, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 19.00122594833374, 'end': 20.00122308731079, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 20.00122308731079, 'end': 21.00119709968567, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 31856, 'rtt': 9441186, 'rttvar': 7463947, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 20.00122308731079, 'end': 21.00119709968567, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 21.00119709968567, 'end': 22.001327991485596, 'seconds': 1.0001308917999268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10548308, 'rttvar': 7812205, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 21.00119709968567, 'end': 22.001327991485596, 'seconds': 1.0001308917999268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 22.001327991485596, 'end': 23.001250982284546, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 33304, 'rtt': 10548308, 'rttvar': 7812205, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 22.001327991485596, 'end': 23.001250982284546, 'seconds': 0.9999229907989502, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 23.001250982284546, 'end': 24.00134801864624, 'seconds': 1.0000970363616943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 34752, 'rtt': 11709727, 'rttvar': 8181992, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 23.001250982284546, 'end': 24.00134801864624, 'seconds': 1.0000970363616943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 24.00134801864624, 'end': 25.001326084136963, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12730721, 'rttvar': 8178482, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 24.00134801864624, 'end': 25.001326084136963, 'seconds': 0.9999780654907227, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 25.001326084136963, 'end': 26.001183032989502, 'seconds': 0.9998569488525391, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 36200, 'rtt': 12730721, 'rttvar': 8178482, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 25.001326084136963, 'end': 26.001183032989502, 'seconds': 0.9998569488525391, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 26.001183032989502, 'end': 27.001212120056152, 'seconds': 1.0000290870666504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 37648, 'rtt': 13811324, 'rttvar': 8295066, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 26.001183032989502, 'end': 27.001212120056152, 'seconds': 1.0000290870666504, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 27.001212120056152, 'end': 28.0013370513916, 'seconds': 1.0001249313354492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14754703, 'rttvar': 8108058, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 27.001212120056152, 'end': 28.0013370513916, 'seconds': 1.0001249313354492, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 28.0013370513916, 'end': 29.001312017440796, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 39096, 'rtt': 14754703, 'rttvar': 8108058, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 28.0013370513916, 'end': 29.001312017440796, 'seconds': 0.9999749660491943, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 29.001312017440796, 'end': 30.001383066177368, 'seconds': 1.0000710487365723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 40544, 'rtt': 15765410, 'rttvar': 8102458, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 29.001312017440796, 'end': 30.001383066177368, 'seconds': 1.0000710487365723, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 30.001383066177368, 'end': 31.00122308731079, 'seconds': 0.9998400211334229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16651975, 'rttvar': 7849974, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 30.001383066177368, 'end': 31.00122308731079, 'seconds': 0.9998400211334229, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 31.00122308731079, 'end': 32.00126504898071, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 41992, 'rtt': 16651975, 'rttvar': 7849974, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 31.00122308731079, 'end': 32.00126504898071, 'seconds': 1.0000419616699219, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 32.00126504898071, 'end': 33.00120496749878, 'seconds': 0.9999399185180664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 43440, 'rtt': 17620390, 'rttvar': 7824310, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 32.00126504898071, 'end': 33.00120496749878, 'seconds': 0.9999399185180664, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 33.00120496749878, 'end': 34.00120210647583, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18470040, 'rttvar': 7567532, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 33.00120496749878, 'end': 34.00120210647583, 'seconds': 0.9999971389770508, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 34.00120210647583, 'end': 35.00121092796326, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 18470040, 'rttvar': 7567532, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 34.00120210647583, 'end': 35.00121092796326, 'seconds': 1.0000088214874268, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 35.00121092796326, 'end': 36.0013530254364, 'seconds': 1.0001420974731445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 44888, 'rtt': 19398025, 'rttvar': 7531619, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 35.00121092796326, 'end': 36.0013530254364, 'seconds': 1.0001420974731445, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 36.0013530254364, 'end': 37.00132703781128, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20215248, 'rttvar': 7283161, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 36.0013530254364, 'end': 37.00132703781128, 'seconds': 0.9999740123748779, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 37.00132703781128, 'end': 38.00142502784729, 'seconds': 1.0000979900360107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 20215248, 'rttvar': 7283161, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 37.00132703781128, 'end': 38.00142502784729, 'seconds': 1.0000979900360107, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 38.00142502784729, 'end': 39.00122904777527, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 46336, 'rtt': 21120726, 'rttvar': 7273326, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 38.00142502784729, 'end': 39.00122904777527, 'seconds': 0.9998040199279785, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 39.00122904777527, 'end': 40.00126791000366, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21915414, 'rttvar': 7044371, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 39.00122904777527, 'end': 40.00126791000366, 'seconds': 1.0000388622283936, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 40.00126791000366, 'end': 41.00125002861023, 'seconds': 0.9999821186065674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 21915414, 'rttvar': 7044371, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 40.00126791000366, 'end': 41.00125002861023, 'seconds': 0.9999821186065674, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 41.00125002861023, 'end': 42.00132608413696, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 47784, 'rtt': 22798230, 'rttvar': 7048910, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 41.00125002861023, 'end': 42.00132608413696, 'seconds': 1.0000760555267334, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 42.00132608413696, 'end': 43.00132513046265, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23560844, 'rttvar': 6811911, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 42.00132608413696, 'end': 43.00132513046265, 'seconds': 0.9999990463256836, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 43.00132513046265, 'end': 44.00127196311951, 'seconds': 0.9999468326568604, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 23560844, 'rttvar': 6811911, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 43.00132513046265, 'end': 44.00127196311951, 'seconds': 0.9999468326568604, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 44.00127196311951, 'end': 45.00125503540039, 'seconds': 0.9999830722808838, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 49232, 'rtt': 24430574, 'rttvar': 6848393, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 44.00127196311951, 'end': 45.00125503540039, 'seconds': 0.9999830722808838, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 45.00125503540039, 'end': 46.00121593475342, 'seconds': 0.9999608993530273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25181883, 'rttvar': 6638913, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 45.00125503540039, 'end': 46.00121593475342, 'seconds': 0.9999608993530273, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 46.00121593475342, 'end': 47.00106906890869, 'seconds': 0.9998531341552734, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 25181883, 'rttvar': 6638913, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 46.00121593475342, 'end': 47.00106906890869, 'seconds': 0.9998531341552734, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 47.00106906890869, 'end': 48.001070976257324, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26036572, 'rttvar': 6688561, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 47.00106906890869, 'end': 48.001070976257324, 'seconds': 1.0000019073486328, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 48.001070976257324, 'end': 49.001359939575195, 'seconds': 1.000288963317871, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 50680, 'rtt': 26036572, 'rttvar': 6688561, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 48.001070976257324, 'end': 49.001359939575195, 'seconds': 1.000288963317871, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 49.001359939575195, 'end': 50.00132393836975, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 26776847, 'rttvar': 6496971, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 49.001359939575195, 'end': 50.00132393836975, 'seconds': 0.9999639987945557, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 50.00132393836975, 'end': 51.00134897232056, 'seconds': 1.0000250339508057, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27612180, 'rttvar': 6543396, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 50.00132393836975, 'end': 51.00134897232056, 'seconds': 1.0000250339508057, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 51.00134897232056, 'end': 52.00131392478943, 'seconds': 0.9999649524688721, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 52128, 'rtt': 27612180, 'rttvar': 6543396, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 51.00134897232056, 'end': 52.00131392478943, 'seconds': 0.9999649524688721, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 52.00131392478943, 'end': 53.001079082489014, 'seconds': 0.999765157699585, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 28348183, 'rttvar': 6379552, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 52.00131392478943, 'end': 53.001079082489014, 'seconds': 0.999765157699585, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 53.001079082489014, 'end': 54.0007209777832, 'seconds': 0.9996418952941895, 'bytes': 256296, 'bits_per_second': 2051102.5094607377, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29182041, 'rttvar': 6452381, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 53.001079082489014, 'end': 54.0007209777832, 'seconds': 0.9996418952941895, 'bytes': 256296, 'bits_per_second': 2051102.5094607377, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 54.0007209777832, 'end': 55.001338958740234, 'seconds': 1.0006179809570312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 53576, 'rtt': 29182041, 'rttvar': 6452381, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 54.0007209777832, 'end': 55.001338958740234, 'seconds': 1.0006179809570312, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 55.001338958740234, 'end': 56.00119709968567, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 29716673, 'rttvar': 5908549, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 55.001338958740234, 'end': 56.00119709968567, 'seconds': 0.9998581409454346, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 56.00119709968567, 'end': 57.001075983047485, 'seconds': 0.9998788833618164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30371991, 'rttvar': 5742048, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 56.00119709968567, 'end': 57.001075983047485, 'seconds': 0.9998788833618164, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 57.001075983047485, 'end': 58.00132513046265, 'seconds': 1.0002491474151611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 55024, 'rtt': 30371991, 'rttvar': 5742048, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 57.001075983047485, 'end': 58.00132513046265, 'seconds': 1.0002491474151611, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}, {'streams': [{'socket': 5, 'start': 58.00132513046265, 'end': 59.00113892555237, 'seconds': 0.9998137950897217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'snd_cwnd': 56472, 'rtt': 30947602, 'rttvar': 5457757, 'pmtu': 1500, 'omitted': False}], 'sum': {'start': 58.00132513046265, 'end': 59.00113892555237, 'seconds': 0.9998137950897217, 'bytes': 0, 'bits_per_second': 0, 'retransmits': 0, 'omitted': False}}], 'end': {}, 'error': 'error - control socket has closed unexpectedly'}" 11:18:39.609822 tst iperf3m4.py:8: ERR: KeyError: 'sum_sent' [trial-2066↪gprs:sysmo↪iperf3m4.py:8] 11:18:39.617997 tst iperf3m4.py:8: Test FAILED (298.4 sec) 11:18:39.625962 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Terminating (SIGTERM) 11:18:39.670378 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Terminating (SIGTERM) 11:18:39.714503 run iperf3-srv_10.42.42.10(pid=18293): Terminating (SIGTERM) 11:18:39.759741 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Terminating (SIGTERM) 11:18:39.804539 run iperf3-srv_10.42.42.10(pid=18295): Terminating (SIGTERM) 11:18:39.848208 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Terminating (SIGTERM) 11:18:39.892508 run iperf3-srv_10.42.42.10(pid=18297): Terminating (SIGTERM) 11:18:39.937384 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Terminating (SIGTERM) 11:18:39.981943 run osmo-hlr_10.42.42.2(pid=18301): Terminating (SIGTERM) 11:18:40.025709 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Terminating (SIGTERM) 11:18:40.070897 run osmo-stp_10.42.42.5(pid=18303): Terminating (SIGTERM) 11:18:40.117832 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Terminating (SIGTERM) 11:18:40.184842 run osmo-ggsn_10.42.42.6(pid=18309): Terminating (SIGTERM) 11:18:40.232574 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Terminating (SIGTERM) 11:18:40.340723 run osmo-sgsn_10.42.42.7(pid=18335): Terminating (SIGTERM) 11:18:40.389425 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Terminating (SIGTERM) 11:18:40.440090 run osmo-msc_10.42.42.8(pid=18337): Terminating (SIGTERM) 11:18:40.494932 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Terminating (SIGTERM) 11:18:40.545425 run osmo-mgw_10.42.42.3(pid=18339): Terminating (SIGTERM) 11:18:40.588266 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Terminating (SIGTERM) 11:18:40.630286 run osmo-mgw_10.42.42.4(pid=18341): Terminating (SIGTERM) 11:18:40.673025 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Terminating (SIGTERM) 11:18:40.714612 run osmo-bsc_10.42.42.9(pid=18343): Terminating (SIGTERM) 11:18:40.755996 run osmo-bts-sysmo(pid=18352): Terminating (SIGTERM) 11:18:40.798049 run osmo-pcu(pid=18365): Terminating (SIGTERM) 11:18:40.840088 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGTERM) 11:18:40.980895 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGTERM(pid=19043): Launched 11:18:42.063449 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGTERM(pid=19043): Terminated: ok {rc=0} 11:18:42.127608 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGTERM) 11:18:42.265803 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGTERM(pid=19048): Launched 11:18:43.352481 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGTERM(pid=19048): Terminated: ok {rc=0} 11:18:43.416531 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGTERM) 11:18:43.554436 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGTERM(pid=19051): Launched 11:18:44.637433 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGTERM(pid=19051): Terminated: ok {rc=0} 11:18:44.701515 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGTERM) 11:18:44.839318 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGTERM(pid=19054): Launched 11:18:45.922859 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGTERM(pid=19054): Terminated: ok {rc=0} 11:18:45.987486 --- ParallelTerminationStrategy: PID 18290 died... 11:18:46.006582 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18290): Terminated: ok {rc=0} 11:18:46.052270 --- ParallelTerminationStrategy: PID 18292 died... 11:18:46.071181 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18292): Terminated: ok {rc=0} 11:18:46.116571 --- ParallelTerminationStrategy: PID 18293 died... 11:18:46.135393 run iperf3-srv_10.42.42.10(pid=18293): Terminated {rc=256} 11:18:46.180226 --- ParallelTerminationStrategy: PID 18294 died... 11:18:46.200058 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18294): Terminated: ok {rc=0} 11:18:46.245144 --- ParallelTerminationStrategy: PID 18295 died... 11:18:46.263952 run iperf3-srv_10.42.42.10(pid=18295): Terminated {rc=256} 11:18:46.311037 --- ParallelTerminationStrategy: PID 18296 died... 11:18:46.330242 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18296): Terminated: ok {rc=0} 11:18:46.374985 --- ParallelTerminationStrategy: PID 18297 died... 11:18:46.393650 run iperf3-srv_10.42.42.10(pid=18297): Terminated {rc=256} 11:18:46.438591 --- ParallelTerminationStrategy: PID 18300 died... 11:18:46.458022 run pcap-recorder_any(filters='host 10.42.42.2')(pid=18300): Terminated: ok {rc=0} 11:18:46.503810 --- ParallelTerminationStrategy: PID 18301 died... 11:18:46.522776 run osmo-hlr_10.42.42.2(pid=18301): Terminated: ok {rc=0} 11:18:46.568326 --- ParallelTerminationStrategy: PID 18302 died... 11:18:46.587163 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=18302): Terminated: ok {rc=0} 11:18:46.632287 --- ParallelTerminationStrategy: PID 18303 died... 11:18:46.651043 run osmo-stp_10.42.42.5(pid=18303): Terminated: ok {rc=0} 11:18:46.695847 --- ParallelTerminationStrategy: PID 18304 died... 11:18:46.714830 run pcap-recorder_any(filters='host 10.42.42.6')(pid=18304): Terminated: ok {rc=0} 11:18:46.759767 --- ParallelTerminationStrategy: PID 18309 died... 11:18:46.778532 run osmo-ggsn_10.42.42.6(pid=18309): Terminated: ok {rc=0} 11:18:46.823473 --- ParallelTerminationStrategy: PID 18334 died... 11:18:46.842349 run pcap-recorder_any(filters='host 10.42.42.7')(pid=18334): Terminated: ok {rc=0} 11:18:46.887334 --- ParallelTerminationStrategy: PID 18335 died... 11:18:46.906090 run osmo-sgsn_10.42.42.7(pid=18335): Terminated: ok {rc=0} 11:18:46.951172 --- ParallelTerminationStrategy: PID 18336 died... 11:18:46.969981 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=18336): Terminated: ok {rc=0} 11:18:47.015321 --- ParallelTerminationStrategy: PID 18337 died... 11:18:47.034366 run osmo-msc_10.42.42.8(pid=18337): Terminated: ok {rc=0} 11:18:47.079922 --- ParallelTerminationStrategy: PID 18338 died... 11:18:47.099126 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=18338): Terminated: ok {rc=0} 11:18:47.144769 --- ParallelTerminationStrategy: PID 18339 died... 11:18:47.164205 run osmo-mgw_10.42.42.3(pid=18339): Terminated {rc=15} 11:18:47.210439 --- ParallelTerminationStrategy: PID 18340 died... 11:18:47.229857 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=18340): Terminated: ok {rc=0} 11:18:47.275673 --- ParallelTerminationStrategy: PID 18341 died... 11:18:47.294902 run osmo-mgw_10.42.42.4(pid=18341): Terminated {rc=15} 11:18:47.340717 --- ParallelTerminationStrategy: PID 18342 died... 11:18:47.360054 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=18342): Terminated: ok {rc=0} 11:18:47.405599 --- ParallelTerminationStrategy: PID 18343 died... 11:18:47.424837 run osmo-bsc_10.42.42.9(pid=18343): Terminated: ok {rc=0} 11:18:47.470885 --- ParallelTerminationStrategy: PID 18352 died... 11:18:47.490334 run osmo-bts-sysmo(pid=18352): Terminated: ok {rc=0} 11:18:47.536164 --- ParallelTerminationStrategy: PID 18365 died... 11:18:47.555499 run osmo-pcu(pid=18365): Terminated: ok {rc=0} 11:18:53.423303 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGINT) 11:18:53.563664 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGINT(pid=19057): Launched 11:18:54.651478 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGINT(pid=19057): Terminated: ok {rc=0} 11:18:54.715835 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGINT) 11:18:54.853799 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGINT(pid=19060): Launched 11:18:55.946039 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGINT(pid=19060): Terminated: ok {rc=0} 11:18:56.012848 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGINT) 11:18:56.155859 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGINT(pid=19063): Launched 11:18:57.243305 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGINT(pid=19063): Terminated: ok {rc=0} 11:18:57.309916 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGINT) 11:18:57.453670 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGINT(pid=19070): Launched 11:18:58.541060 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGINT(pid=19070): Terminated: ok {rc=0} 11:19:04.430310 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995): Terminating (SIGKILL) 11:19:04.574963 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGUSR1(pid=19073): Launched 11:19:05.663461 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18995)-killSignals.SIGUSR1(pid=19073): Terminated: ok {rc=0} 11:19:05.729991 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998): Terminating (SIGKILL) 11:19:05.873226 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGUSR1(pid=19076): Launched 11:19:06.958654 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=18998)-killSignals.SIGUSR1(pid=19076): Terminated: ok {rc=0} 11:19:07.025099 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001): Terminating (SIGKILL) 11:19:07.168460 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGUSR1(pid=19079): Launched 11:19:08.253713 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19001)-killSignals.SIGUSR1(pid=19079): Terminated: ok {rc=0} 11:19:08.320246 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004): Terminating (SIGKILL) 11:19:08.463566 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGUSR1(pid=19082): Launched 11:19:09.554269 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=19004)-killSignals.SIGUSR1(pid=19082): Terminated: ok {rc=0} 11:19:09.801696 bus /gobi_4: Setting Online False 11:19:10.970235 bus /gobi_4: Setting Powered False 11:19:20.943960 bus /gobi_4: Setting Powered False 11:19:22.041570 bus /gobi_4: Setting Powered False 11:19:23.202581 bus /gobi_1: Setting Online False 11:19:24.313472 bus /gobi_1: Setting Powered False 11:19:32.084697 bus /gobi_1: Setting Powered False 11:19:33.184756 bus /gobi_1: Setting Powered False 11:19:34.345073 bus /gobi_0: Setting Online False 11:19:35.478508 bus /gobi_0: Setting Powered False 11:19:43.940610 bus /gobi_0: Setting Powered False 11:19:45.039619 bus /gobi_0: Setting Powered False 11:19:46.198600 bus /gobi_6: Setting Online False 11:19:47.317208 bus /gobi_6: Setting Powered False 11:19:55.094016 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2066 gprs:sysmo ping.py ---------------------------------------------- 11:19:56.257163 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.283259 tst gprs:sysmo: Using 1 x bts (candidates: 1) 11:19:56.319563 tst ping.py: using LAC 3962 11:19:56.448492 tst ping.py: using RAC 137 11:19:56.575545 tst ping.py: using CellId 3962 11:19:56.703233 tst ping.py: using BVCI 3963 11:19:56.821932 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.833027 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.844200 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.855403 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.866318 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.877411 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.888745 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:19:56.899952 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:19:57.014636 tst ping.py:19: start network... 11:19:57.216427 run osmo-hlr_10.42.42.2: Starting osmo-hlr 11:19:57.685353 run create_hlr_db(pid=19087): Launched 11:19:57.808218 bus /gobi_6: Setting Powered False 11:19:58.868358 run create_hlr_db(pid=19087): Terminated: ok {rc=0} 11:19:59.290654 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19089): Launched 11:19:59.594575 run osmo-hlr_10.42.42.2(pid=19090): Launched 11:19:59.712925 run osmo-stp_10.42.42.5: Starting osmo-stp 11:20:00.233312 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19091): Launched 11:20:00.539676 run osmo-stp_10.42.42.5(pid=19092): Launched 11:20:00.655733 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 11:20:01.177314 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19093): Launched 11:20:01.294179 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 11:20:01.550636 run patchelf(pid=19094): Launched 11:20:02.724298 run patchelf(pid=19094): Terminated: ok {rc=0} 11:20:02.851039 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 11:20:03.110145 run osmo-gsm-tester_setcap_net_admin.sh(pid=19098): Launched 11:20:04.282313 run osmo-gsm-tester_setcap_net_admin.sh(pid=19098): Terminated: ok {rc=0} 11:20:04.597611 run osmo-ggsn_10.42.42.6(pid=19101): Launched 11:20:04.720280 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 11:20:05.416959 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19118): Launched 11:20:05.725684 run osmo-sgsn_10.42.42.7(pid=19119): Launched 11:20:05.843598 run osmo-msc_10.42.42.8: Starting osmo-msc 11:20:06.395358 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19120): Launched 11:20:06.700688 run osmo-msc_10.42.42.8(pid=19121): Launched 11:20:09.872905 run osmo-mgw_10.42.42.3: Starting osmo-mgw 11:20:10.390917 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19122): Launched 11:20:10.696194 run osmo-mgw_10.42.42.3(pid=19123): Launched 11:20:10.812078 run osmo-mgw_10.42.42.4: Starting osmo-mgw 11:20:11.320278 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19124): Launched 11:20:11.625331 run osmo-mgw_10.42.42.4(pid=19125): Launched 11:20:11.741911 run osmo-bsc_10.42.42.9: Starting osmo-bsc 11:20:12.370163 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19126): Launched 11:20:12.674856 run osmo-bsc_10.42.42.9(pid=19127): Launched 11:20:18.943494 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 11:20:19.470279 run rm-remote-dir(pid=19128): Launched 11:20:21.651641 run rm-remote-dir(pid=19128): Terminated: ok {rc=0} 11:20:21.984969 run mk-remote-dir(pid=19129): Launched 11:20:23.163504 run mk-remote-dir(pid=19129): Terminated: ok {rc=0} 11:20:23.447599 run scp-inst-to-remote(pid=19130): Launched 11:20:35.640572 run scp-inst-to-remote(pid=19130): Terminated: ok {rc=0} 11:20:35.991736 run mk-remote-dir(pid=19132): Launched 11:20:37.180640 run mk-remote-dir(pid=19132): Terminated: ok {rc=0} 11:20:37.482449 run scp-cfg-to-remote(pid=19133): Launched 11:20:38.670082 run scp-cfg-to-remote(pid=19133): Terminated: ok {rc=0} 11:20:39.018025 run reload-dsp-firmware(pid=19135): Launched 11:20:41.205451 run reload-dsp-firmware(pid=19135): Terminated: ok {rc=0} 11:20:41.546419 run osmo-bts-sysmo(pid=19136): Launched 11:20:45.775827 tst ping.py:31: Waiting for bts to be ready... 11:20:47.445366 run rm-remote-dir(pid=19137): Launched 11:20:49.618647 run rm-remote-dir(pid=19137): Terminated: ok {rc=0} 11:20:49.937919 run mk-remote-dir(pid=19138): Launched 11:20:52.114537 run mk-remote-dir(pid=19138): Terminated: ok {rc=0} 11:20:52.408842 run scp-inst-to-bts(pid=19139): Launched 11:21:05.605499 run scp-inst-to-bts(pid=19139): Terminated: ok {rc=0} 11:21:05.946758 run mk-remote-run-dir(pid=19141): Launched 11:21:08.129713 run mk-remote-run-dir(pid=19141): Terminated: ok {rc=0} 11:21:08.404971 run scp-cfg-to-sysmobts(pid=19142): Launched 11:21:10.578013 run scp-cfg-to-sysmobts(pid=19142): Terminated: ok {rc=0} 11:21:10.898545 run osmo-pcu(pid=19144): Launched 11:21:11.026008 tst ping.py: using MSISDN 3978 11:21:11.201236 bus /gobi_6: Setting Powered True 11:21:14.719849 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3978', subscriber_id=1} 11:21:14.990331 bus /gobi_6: Setting Online False 11:21:16.175369 bus /gobi_6: Setting Powered False 11:21:23.109816 bus /gobi_6: Setting Powered True 11:21:24.489313 bus /gobi_6: Setting Online True 11:21:26.982946 tst /gobi_6: Connect to ('901', '70') 11:21:27.244544 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 11:21:27.354534 tst ping.py:42: waiting for modems to attach... 11:21:41.329012 tst /gobi_6: Already registered with network ('901', '70') 11:21:46.800451 tst ping.py:46: waiting for modems to attach to data services... 11:21:49.873795 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:21:49.990804 tst ping.py:51: Setting up data plan for "'/gobi_6/context3'" 11:21:50.432512 run move_netns(pid=19148): Launched 11:21:51.624143 run move_netns(pid=19148): Terminated: ok {rc=0} 11:21:51.954152 run ifup(pid=19151): Launched 11:21:53.130211 run ifup(pid=19151): Terminated: ok {rc=0} 11:21:53.458426 run dhcp(pid=19154): Launched 11:21:54.634888 run dhcp(pid=19154): Terminated: ok {rc=0} 11:21:54.764748 tst ping.py:53: Running 10 ping requests for "'/gobi_6/context3'" 11:21:55.172159 run ping(pid=19170): Launched 11:22:05.378317 run ping(pid=19170): Terminated: ok {rc=0} 11:22:07.075266 tst /gobi_6: context deactivated {path='/gobi_6/context3'} 11:22:14.223126 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context3', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={Interface='r433306bbc2604b'}, Name='Internet', Password='', Protocol='ipv6', Settings={}, Type='internet', Username='ogt'}, user='ogt'} 11:22:21.535059 tst /gobi_6: context deactivated {path='/gobi_6/context3'} 11:22:21.644398 tst ping.py:70: (launched: 2022-09-07_11:21:55.109240) 19171: waiting for 19172 PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. 64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=491 ms 64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=431 ms 64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=390 ms 64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=386 ms 64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=386 ms 64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=363 ms 64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=622 ms 64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=604 ms 64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=521 ms 64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=499 ms --- 10.42.42.6 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 20ms rtt min/avg/max/mdev = 363.029/469.397/621.852/88.624 ms child exited with 0 11:22:21.849698 tst ping.py: Test passed (145.6 sec) 11:22:21.916060 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19089): Terminating (SIGTERM) 11:22:21.957380 run osmo-hlr_10.42.42.2(pid=19090): Terminating (SIGTERM) 11:22:21.998309 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19091): Terminating (SIGTERM) 11:22:22.041517 run osmo-stp_10.42.42.5(pid=19092): Terminating (SIGTERM) 11:22:22.086075 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19093): Terminating (SIGTERM) 11:22:22.129373 run osmo-ggsn_10.42.42.6(pid=19101): Terminating (SIGTERM) 11:22:22.175282 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19118): Terminating (SIGTERM) 11:22:22.221279 run osmo-sgsn_10.42.42.7(pid=19119): Terminating (SIGTERM) 11:22:22.265588 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19120): Terminating (SIGTERM) 11:22:22.309282 run osmo-msc_10.42.42.8(pid=19121): Terminating (SIGTERM) 11:22:22.352320 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19122): Terminating (SIGTERM) 11:22:22.402023 run osmo-mgw_10.42.42.3(pid=19123): Terminating (SIGTERM) 11:22:22.460466 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19124): Terminating (SIGTERM) 11:22:22.506244 run osmo-mgw_10.42.42.4(pid=19125): Terminating (SIGTERM) 11:22:22.549719 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19126): Terminating (SIGTERM) 11:22:22.593499 run osmo-bsc_10.42.42.9(pid=19127): Terminating (SIGTERM) 11:22:22.636286 run osmo-bts-sysmo(pid=19136): Terminating (SIGTERM) 11:22:22.679331 run osmo-pcu(pid=19144): Terminating (SIGTERM) 11:22:22.722658 --- ParallelTerminationStrategy: PID 19089 died... 11:22:22.743146 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19089): Terminated: ok {rc=0} 11:22:22.791463 --- ParallelTerminationStrategy: PID 19090 died... 11:22:22.811376 run osmo-hlr_10.42.42.2(pid=19090): Terminated: ok {rc=0} 11:22:22.860408 --- ParallelTerminationStrategy: PID 19091 died... 11:22:22.881030 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19091): Terminated: ok {rc=0} 11:22:22.960992 --- ParallelTerminationStrategy: PID 19092 died... 11:22:22.981721 run osmo-stp_10.42.42.5(pid=19092): Terminated: ok {rc=0} 11:22:23.031240 --- ParallelTerminationStrategy: PID 19093 died... 11:22:23.051803 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19093): Terminated: ok {rc=0} 11:22:23.114613 --- ParallelTerminationStrategy: PID 19101 died... 11:22:23.144609 run osmo-ggsn_10.42.42.6(pid=19101): Terminated: ok {rc=0} 11:22:23.194981 --- ParallelTerminationStrategy: PID 19118 died... 11:22:23.214114 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19118): Terminated: ok {rc=0} 11:22:23.259237 --- ParallelTerminationStrategy: PID 19120 died... 11:22:23.278628 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19120): Terminated: ok {rc=0} 11:22:23.326103 --- ParallelTerminationStrategy: PID 19121 died... 11:22:23.346010 run osmo-msc_10.42.42.8(pid=19121): Terminated: ok {rc=0} 11:22:23.394352 --- ParallelTerminationStrategy: PID 19122 died... 11:22:23.415084 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19122): Terminated: ok {rc=0} 11:22:23.463323 --- ParallelTerminationStrategy: PID 19123 died... 11:22:23.483768 run osmo-mgw_10.42.42.3(pid=19123): Terminated {rc=15} 11:22:23.532178 --- ParallelTerminationStrategy: PID 19124 died... 11:22:23.551978 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19124): Terminated: ok {rc=0} 11:22:23.598379 --- ParallelTerminationStrategy: PID 19119 died... 11:22:23.617902 run osmo-sgsn_10.42.42.7(pid=19119): Terminated: ok {rc=0} 11:22:23.663621 --- ParallelTerminationStrategy: PID 19125 died... 11:22:23.682665 run osmo-mgw_10.42.42.4(pid=19125): Terminated {rc=15} 11:22:23.728492 --- ParallelTerminationStrategy: PID 19126 died... 11:22:23.747725 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19126): Terminated: ok {rc=0} 11:22:23.793189 --- ParallelTerminationStrategy: PID 19127 died... 11:22:23.812317 run osmo-bsc_10.42.42.9(pid=19127): Terminated: ok {rc=0} 11:22:23.857961 --- ParallelTerminationStrategy: PID 19136 died... 11:22:23.877070 run osmo-bts-sysmo(pid=19136): Terminated: ok {rc=0} 11:22:23.922602 --- ParallelTerminationStrategy: PID 19144 died... 11:22:23.941946 run osmo-pcu(pid=19144): Terminated: ok {rc=0} 11:22:24.165335 bus /gobi_6: Setting Online False 11:22:28.240201 bus /gobi_6: Setting Powered False 11:22:46.060293 bus /gobi_6: Setting Powered False ---------------------------------------------- trial-2066 gprs:sysmo ping_idle_ping.py ---------------------------------------------- 11:22:47.205379 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.217110 tst gprs:sysmo: Using 1 x bts (candidates: 1) 11:22:47.252188 tst ping_idle_ping.py: using LAC 3963 11:22:47.379621 tst ping_idle_ping.py: using RAC 138 11:22:47.507284 tst ping_idle_ping.py: using CellId 3963 11:22:47.633308 tst ping_idle_ping.py: using BVCI 3964 11:22:47.753226 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.764455 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.775558 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.786824 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.797991 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.808995 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.820462 tst gprs:sysmo: Using 1 x ip_address (candidates: 9) 11:22:47.831641 tst gprs:sysmo: Using 1 x modem (candidates: 4) 11:22:47.950945 tst ping_idle_ping.py:24: start network... 11:22:48.169217 run osmo-hlr_10.42.42.2: Starting osmo-hlr 11:22:48.674869 run create_hlr_db(pid=19195): Launched 11:22:48.807992 bus /gobi_6: Setting Powered False 11:22:49.872541 run create_hlr_db(pid=19195): Terminated: ok {rc=0} 11:22:50.308319 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19197): Launched 11:22:50.614663 run osmo-hlr_10.42.42.2(pid=19198): Launched 11:22:50.731360 run osmo-stp_10.42.42.5: Starting osmo-stp 11:22:51.252100 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19199): Launched 11:22:51.557975 run osmo-stp_10.42.42.5(pid=19200): Launched 11:22:51.675055 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn 11:22:52.197743 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19201): Launched 11:22:52.314639 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn 11:22:52.571678 run patchelf(pid=19202): Launched 11:22:53.746974 run patchelf(pid=19202): Terminated: ok {rc=0} 11:22:53.875105 run osmo-ggsn_10.42.42.6: Applying CAP_NET_ADMIN capability to osmo-ggsn 11:22:54.135510 run osmo-gsm-tester_setcap_net_admin.sh(pid=19203): Launched 11:22:55.312881 run osmo-gsm-tester_setcap_net_admin.sh(pid=19203): Terminated: ok {rc=0} 11:22:55.629656 run osmo-ggsn_10.42.42.6(pid=19206): Launched 11:22:55.754350 run osmo-sgsn_10.42.42.7: Starting osmo-sgsn 11:22:56.464523 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19223): Launched 11:22:56.792380 run osmo-sgsn_10.42.42.7(pid=19224): Launched 11:22:56.919117 run osmo-msc_10.42.42.8: Starting osmo-msc 11:22:57.514142 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19225): Launched 11:22:57.843640 run osmo-msc_10.42.42.8(pid=19226): Launched 11:23:01.024850 run osmo-mgw_10.42.42.3: Starting osmo-mgw 11:23:01.583986 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19227): Launched 11:23:01.911672 run osmo-mgw_10.42.42.3(pid=19228): Launched 11:23:02.036155 run osmo-mgw_10.42.42.4: Starting osmo-mgw 11:23:02.585101 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19229): Launched 11:23:02.913543 run osmo-mgw_10.42.42.4(pid=19230): Launched 11:23:03.046300 run osmo-bsc_10.42.42.9: Starting osmo-bsc 11:23:03.722638 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19233): Launched 11:23:04.041414 run osmo-bsc_10.42.42.9(pid=19234): Launched 11:23:10.312966 run osmo-bts-sysmo: Starting sysmoBTS to connect to osmo-bsc_10.42.42.9 11:23:10.843314 run rm-remote-dir(pid=19235): Launched 11:23:13.024745 run rm-remote-dir(pid=19235): Terminated: ok {rc=0} 11:23:13.359557 run mk-remote-dir(pid=19236): Launched 11:23:14.537819 run mk-remote-dir(pid=19236): Terminated: ok {rc=0} 11:23:14.824237 run scp-inst-to-remote(pid=19237): Launched 11:23:27.022020 run scp-inst-to-remote(pid=19237): Terminated: ok {rc=0} 11:23:27.374594 run mk-remote-dir(pid=19239): Launched 11:23:28.558129 run mk-remote-dir(pid=19239): Terminated: ok {rc=0} 11:23:28.841908 run scp-cfg-to-remote(pid=19240): Launched 11:23:30.018837 run scp-cfg-to-remote(pid=19240): Terminated: ok {rc=0} 11:23:30.345348 run reload-dsp-firmware(pid=19242): Launched 11:23:32.519675 run reload-dsp-firmware(pid=19242): Terminated: ok {rc=0} 11:23:32.839361 run osmo-bts-sysmo(pid=19243): Launched 11:23:37.062602 tst ping_idle_ping.py:36: Waiting for bts to be ready... 11:23:38.733647 run rm-remote-dir(pid=19244): Launched 11:23:40.907149 run rm-remote-dir(pid=19244): Terminated: ok {rc=0} 11:23:41.228405 run mk-remote-dir(pid=19245): Launched 11:23:43.405648 run mk-remote-dir(pid=19245): Terminated: ok {rc=0} 11:23:43.700953 run scp-inst-to-bts(pid=19246): Launched 11:23:56.894104 run scp-inst-to-bts(pid=19246): Terminated: ok {rc=0} 11:23:57.215829 run mk-remote-run-dir(pid=19248): Launched 11:23:59.387313 run mk-remote-run-dir(pid=19248): Terminated: ok {rc=0} 11:23:59.663390 run scp-cfg-to-sysmobts(pid=19249): Launched 11:24:01.839330 run scp-cfg-to-sysmobts(pid=19249): Terminated: ok {rc=0} 11:24:02.168751 run osmo-pcu(pid=19251): Launched 11:24:02.307345 tst ping_idle_ping.py: using MSISDN 3979 11:24:02.477201 bus /gobi_6: Setting Powered True 11:24:06.008935 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000010978', msisdn='3979', subscriber_id=1} 11:24:06.291579 bus /gobi_6: Setting Online False 11:24:07.483862 bus /gobi_6: Setting Powered False 11:24:14.246844 bus /gobi_6: Setting Powered True 11:24:15.622207 bus /gobi_6: Setting Online True 11:24:18.104373 tst /gobi_6: Connect to ('901', '70') 11:24:18.375481 tst /gobi_6: Manufacturer: 'Quectel', Model: 'EC20', Revision: 'EC20EQAR02A13E2G 1 [Jul 5 2017 10:49:17]', Serial: '868323024273648' 11:24:18.483656 tst ping_idle_ping.py:47: waiting for modems to attach... 11:24:25.484972 tst /gobi_6: Already registered with network ('901', '70') 11:24:36.936369 tst ping_idle_ping.py:51: waiting for modems to attach to data services... 11:24:54.190230 tst /gobi_6: context activated {apn='inet46', path='/gobi_6/context2', properties={AccessPointName='inet46', Active=True, AuthenticationMethod='chap', IPv6.Settings={}, Name='Internet', Password='', Protocol='ip', Settings={Address='176.16.46.1', DomainNameServers=['192.168.100.1', '8.8.8.8'], Gateway='176.16.46.2', Interface='r433306bbc2604b', Method='static', Netmask='255.255.255.252'}, Type='internet', Username='ogt'}, user='ogt'} 11:24:54.307532 tst ping_idle_ping.py:56: Setting up data plane for "'/gobi_6/context2'" 11:24:54.751779 run move_netns(pid=19254): Launched 11:24:55.944416 run move_netns(pid=19254): Terminated: ok {rc=0} 11:24:56.295667 run ifup(pid=19257): Launched 11:24:57.488444 run ifup(pid=19257): Terminated: ok {rc=0} 11:24:57.840210 run dhcp(pid=19260): Launched 11:24:59.028730 run dhcp(pid=19260): Terminated: ok {rc=0} 11:24:59.167414 tst ping_idle_ping.py:60: [1] Running 10 ping requests for "'/gobi_6/context2'" 11:24:59.597331 run ping1(pid=19287): Launched 11:25:09.799242 run ping1(pid=19287): Terminated: ok {rc=0} 11:25:09.928708 tst ping_idle_ping.py:64: (launched: 2022-09-07_11:24:59.532934) 19288: waiting for 19289 PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. 64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=558 ms 64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=559 ms 64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=519 ms 64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=515 ms 64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=637 ms 64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=616 ms 64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=599 ms 64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=618 ms 64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=399 ms 64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=515 ms --- 10.42.42.6 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 11ms rtt min/avg/max/mdev = 398.572/553.507/637.368/67.349 ms child exited with 0 11:25:10.135637 tst ping_idle_ping.py:68: Sleeping for 60 seconds 11:26:10.342590 tst ping_idle_ping.py:73: [2] Running 10 ping requests for "'/gobi_6/context2'" 11:26:10.775860 run ping2(pid=19295): Launched 11:26:20.981663 run ping2(pid=19295): Terminated: ok {rc=0} 11:26:21.111562 tst ping_idle_ping.py:77: (launched: 2022-09-07_11:26:10.710394) 19296: waiting for 19297 PING 10.42.42.6 (10.42.42.6) 56(84) bytes of data. 64 bytes from 10.42.42.6: icmp_seq=1 ttl=64 time=501 ms 64 bytes from 10.42.42.6: icmp_seq=2 ttl=64 time=424 ms 64 bytes from 10.42.42.6: icmp_seq=3 ttl=64 time=383 ms 64 bytes from 10.42.42.6: icmp_seq=4 ttl=64 time=379 ms 64 bytes from 10.42.42.6: icmp_seq=5 ttl=64 time=382 ms 64 bytes from 10.42.42.6: icmp_seq=6 ttl=64 time=360 ms 64 bytes from 10.42.42.6: icmp_seq=7 ttl=64 time=619 ms 64 bytes from 10.42.42.6: icmp_seq=8 ttl=64 time=603 ms 64 bytes from 10.42.42.6: icmp_seq=9 ttl=64 time=622 ms 64 bytes from 10.42.42.6: icmp_seq=10 ttl=64 time=619 ms --- 10.42.42.6 ping statistics --- 10 packets transmitted, 10 received, 0% packet loss, time 11ms rtt min/avg/max/mdev = 360.123/489.363/622.350/109.597 ms child exited with 0 11:26:23.034554 tst /gobi_6: context deactivated {path='/gobi_6/context2'} 11:26:23.146924 tst ping_idle_ping.py: Test passed (215.9 sec) 11:26:23.212314 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19197): Terminating (SIGTERM) 11:26:23.253370 run osmo-hlr_10.42.42.2(pid=19198): Terminating (SIGTERM) 11:26:23.295375 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19199): Terminating (SIGTERM) 11:26:23.338821 run osmo-stp_10.42.42.5(pid=19200): Terminating (SIGTERM) 11:26:23.388688 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19201): Terminating (SIGTERM) 11:26:23.435804 run osmo-ggsn_10.42.42.6(pid=19206): Terminating (SIGTERM) 11:26:23.482983 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19223): Terminating (SIGTERM) 11:26:23.560553 run osmo-sgsn_10.42.42.7(pid=19224): Terminating (SIGTERM) 11:26:23.613308 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19225): Terminating (SIGTERM) 11:26:23.660030 run osmo-msc_10.42.42.8(pid=19226): Terminating (SIGTERM) 11:26:23.707199 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19227): Terminating (SIGTERM) 11:26:23.752565 run osmo-mgw_10.42.42.3(pid=19228): Terminating (SIGTERM) 11:26:23.797286 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19229): Terminating (SIGTERM) 11:26:23.841381 run osmo-mgw_10.42.42.4(pid=19230): Terminating (SIGTERM) 11:26:23.885131 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19233): Terminating (SIGTERM) 11:26:23.930601 run osmo-bsc_10.42.42.9(pid=19234): Terminating (SIGTERM) 11:26:23.974126 run osmo-bts-sysmo(pid=19243): Terminating (SIGTERM) 11:26:24.017983 run osmo-pcu(pid=19251): Terminating (SIGTERM) 11:26:24.061712 --- ParallelTerminationStrategy: PID 19197 died... 11:26:24.082343 run pcap-recorder_any(filters='host 10.42.42.2')(pid=19197): Terminated: ok {rc=0} 11:26:24.132621 --- ParallelTerminationStrategy: PID 19198 died... 11:26:24.153827 run osmo-hlr_10.42.42.2(pid=19198): Terminated: ok {rc=0} 11:26:24.204143 --- ParallelTerminationStrategy: PID 19199 died... 11:26:24.225242 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=19199): Terminated: ok {rc=0} 11:26:24.307258 --- ParallelTerminationStrategy: PID 19200 died... 11:26:24.328295 run osmo-stp_10.42.42.5(pid=19200): Terminated: ok {rc=0} 11:26:24.387525 --- ParallelTerminationStrategy: PID 19201 died... 11:26:24.408547 run pcap-recorder_any(filters='host 10.42.42.6')(pid=19201): Terminated: ok {rc=0} 11:26:24.459227 --- ParallelTerminationStrategy: PID 19206 died... 11:26:24.480092 run osmo-ggsn_10.42.42.6(pid=19206): Terminated: ok {rc=0} 11:26:24.528835 --- ParallelTerminationStrategy: PID 19223 died... 11:26:24.548866 run pcap-recorder_any(filters='host 10.42.42.7')(pid=19223): Terminated: ok {rc=0} 11:26:24.596467 --- ParallelTerminationStrategy: PID 19225 died... 11:26:24.616680 run pcap-recorder_any(filters='host 10.42.42.8 and port not 22')(pid=19225): Terminated: ok {rc=0} 11:26:24.666121 --- ParallelTerminationStrategy: PID 19226 died... 11:26:24.687163 run osmo-msc_10.42.42.8(pid=19226): Terminated: ok {rc=0} 11:26:24.736906 --- ParallelTerminationStrategy: PID 19227 died... 11:26:24.757866 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=19227): Terminated: ok {rc=0} 11:26:24.807782 --- ParallelTerminationStrategy: PID 19228 died... 11:26:24.828971 run osmo-mgw_10.42.42.3(pid=19228): Terminated {rc=15} 11:26:24.878970 --- ParallelTerminationStrategy: PID 19229 died... 11:26:24.899954 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=19229): Terminated: ok {rc=0} 11:26:24.949544 --- ParallelTerminationStrategy: PID 19230 died... 11:26:24.970537 run osmo-mgw_10.42.42.4(pid=19230): Terminated {rc=15} 11:26:25.020348 --- ParallelTerminationStrategy: PID 19233 died... 11:26:25.041695 run pcap-recorder_any(filters='host 10.42.42.9 and port not 22')(pid=19233): Terminated: ok {rc=0} 11:26:25.091868 --- ParallelTerminationStrategy: PID 19234 died... 11:26:25.112985 run osmo-bsc_10.42.42.9(pid=19234): Terminated: ok {rc=0} 11:26:25.163089 --- ParallelTerminationStrategy: PID 19243 died... 11:26:25.184165 run osmo-bts-sysmo(pid=19243): Terminated: ok {rc=0} 11:26:25.234154 --- ParallelTerminationStrategy: PID 19251 died... 11:26:25.255335 run osmo-pcu(pid=19251): Terminated: ok {rc=0} 11:26:25.473386 --- ParallelTerminationStrategy: PID 19224 died... 11:26:25.495556 run osmo-sgsn_10.42.42.7(pid=19224): Terminated: ok {rc=0} 11:26:25.718598 bus /gobi_6: Setting Online False 11:26:28.780101 bus /gobi_6: Setting Powered False 11:26:45.945949 bus /gobi_6: Setting Powered False --------------------------------------------------------------------- trial-2066 gprs:sysmo FAIL --------------------------------------------------------------------- 11:26:47.281387 tst trial-2066: Storing JUnit report in /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2066/run.2022-09-07_11-02-04/trial-2066.xml -------------------------------------------------------------------------------------------- trial-2066 FAIL -------------------------------------------------------------------------------------------- 11:26:47.496099 tst trial-2066: trial-2066: FAIL, 2 suites failed, 1 suites passed FAIL: nitb_sms:sysmo (fail: 1) FAIL: mo_mt_sms.py (4.8 sec) Error: osmo-nitb_10.42.42.2(pid=17962): Process ended prematurely: osmo-nitb_10.42.42.2(pid=17962) [trial-2066↪nitb_sms:sysmo↪mo_mt_sms.py:11↪mo_mt_sms.py↪osmo-nitb_10.42.42.2↪osmo-nitb_10.42.42.2(pid=17962)] FAIL: gprs:sysmo (fail: 3, pass: 2) FAIL: cs_paging_gprs_active.py (247.5 sec) KeyError: 'sum_sent' FAIL: iperf3.py (204.2 sec) KeyError: 'sum_sent' FAIL: iperf3m4.py (298.4 sec) KeyError: 'sum_sent' pass: ping.py (145.6 sec) pass: ping_idle_ping.py (215.9 sec) PASS: sms:sysmo (pass: 1) pass: mo_mt_sms.py (115.6 sec) + rm -rf trial-2066/inst + cd trial-2066 + command -v journalctl + [ ! -z /bin/journalctl ] + readlink last_run + journalctl -u ofono -o short-precise --since 2022-09-07 11:02:01 + readlink last_run + tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2066-run.tgz run.2022-09-07_11-02-04 + tar czf /home/jenkins/workspace/osmo-gsm-tester_gerrit/trial-2066-bin.tgz checksums.md5 open5gs.build-8.tgz osmo-bsc.build-16888.tgz osmo-bts-oc2g.build-5192.tgz osmo-bts-sysmo.build-7696.tgz osmo-bts.build-7166.tgz osmo-ggsn.build-6364.tgz osmo-hlr.build-8262.tgz osmo-hnbgw.build-2020.tgz osmo-mgw.build-8470.tgz osmo-msc.build-28078.tgz osmo-nitb.build-12346.tgz osmo-pcu-oc2g.build-3690.tgz osmo-pcu-sysmo.build-4971.tgz osmo-pcu.build-4933.tgz osmo-sgsn.build-28683.tgz osmo-stp.build-8799.tgz osmo-trx.build-4360.tgz osmocom-bb.build-4442.tgz srslte.build-19.tgz + exit 1 Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results [Checks API] No suitable checks publisher found. ERROR Gerrit response: ConflictFinished: FAILURE