55gprs:trx-umtrx9.0osmo-stp_10.42.42.5(pid=20415): Process ended prematurely: osmo-stp_10.42.42.5(pid=20415) [trial-2372↪gprs:trx-umtrx↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20415)]2318cs_paging_gprs_active.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/cs_paging_gprs_active.py", line 38, in <module>
setup_run_iperf3_test_parallel(2, ready_cb=ready_cb_place_voicecall)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=20415): Process ended prematurely: osmo-stp_10.42.42.5(pid=20415) [trial-2372↪gprs:trx-umtrx↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20415)]
----------------------------------------------
trial-2372 gprs:trx-umtrx cs_paging_gprs_active.py
----------------------------------------------
09:00:08.669999 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:08.680556 tst gprs:trx-umtrx: Using 1 x bts (candidates: 1)
09:00:08.819496 tst cs_paging_gprs_active.py: using LAC 27295
09:00:08.938496 tst cs_paging_gprs_active.py: using RAC 10
09:00:09.058438 tst cs_paging_gprs_active.py: using CellId 27295
09:00:09.179794 tst cs_paging_gprs_active.py: using BVCI 27296
09:00:09.203237 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.213853 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.224238 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.235166 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.245810 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.256513 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.267498 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.278183 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:09.288858 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:09.409722 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:09.705044 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003...
09:00:09.841577 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:00:10.180300 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20404): Launched
09:00:10.462965 run iperf3-srv_10.42.42.10(pid=20405): Launched
09:00:10.663159 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004...
09:00:10.798980 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:00:11.136780 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20406): Launched
09:00:11.418670 run iperf3-srv_10.42.42.10(pid=20407): Launched
09:00:11.618375 tst cs_paging_gprs_active.py:38: start network...
09:00:11.754371 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:00:12.271173 run create_hlr_db(pid=20408): Launched
09:00:12.458935 bus /sierra_2: Setting Powered False
09:00:13.479828 run create_hlr_db(pid=20408): Terminated: ok {rc=0}
09:00:13.894980 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20411): Launched
09:00:14.205295 run osmo-hlr_10.42.42.2(pid=20412): Launched
09:00:14.319195 run osmo-stp_10.42.42.5: Starting osmo-stp
09:00:14.851403 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20414): Launched
09:00:15.160314 run osmo-stp_10.42.42.5(pid=20415): Launched
09:00:15.272057 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:00:15.806539 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20417): Launched
09:00:15.925594 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:00:16.245871 run patchelf(pid=20418): Launched
09:00:16.440347 run osmo-stp_10.42.42.5(pid=20415): ERR: Terminated: ERROR {rc=1} [trial-2372↪gprs:trx-umtrx↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20415)]
09:00:16.582545 run osmo-stp_10.42.42.5(pid=20415): stdout:
| (launched: 2022-12-10_09:00:15.006614)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:16.726594 run osmo-stp_10.42.42.5(pid=20415): stderr:
| �[0;m�[38;5;43m20221210090015180 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090015180 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090015180 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090015181 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090015181 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090015181 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090015181 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:16.865562 run osmo-stp_10.42.42.5(pid=20415): stdout:
| (launched: 2022-12-10_09:00:15.006614)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:17.003004 run osmo-stp_10.42.42.5(pid=20415): stderr:
| �[0;m�[38;5;43m20221210090015180 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090015180 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090015180 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090015181 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090015181 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090015181 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090015181 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:17.162344 run patchelf(pid=20418): Terminating (SIGINT)
09:00:17.346511 run patchelf(pid=20418): Terminated: ok {rc=0}
09:00:17.379889 tst cs_paging_gprs_active.py:38: ERR: Error: osmo-stp_10.42.42.5(pid=20415): Process ended prematurely: osmo-stp_10.42.42.5(pid=20415) [trial-2372↪gprs:trx-umtrx↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20415)] [trial-2372↪gprs:trx-umtrx↪cs_paging_gprs_active.py:38]
09:00:17.389736 tst cs_paging_gprs_active.py:38: Test FAILED (8.8 sec)
09:00:17.452171 run osmo-hlr_10.42.42.2(pid=20412): ERR: Terminated: ERROR {rc=237} [trial-2372↪gprs:trx-umtrx↪cs_paging_gprs_active.py↪cs_paging_gprs_active.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=20412)]
09:00:17.503433 run osmo-hlr_10.42.42.2(pid=20412): stdout:
| (launched: 2022-12-10_09:00:14.050997)
09:00:17.555023 run osmo-hlr_10.42.42.2(pid=20412): stderr:
| �[0;m�[1;31m20221210090014231 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210090014231 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210090014231 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210090014231 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210090014231 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210090014231 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210090014244 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2372/run.2022-12-10_08-20-36/gprs:trx-umtrx/cs_paging_gprs_active.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210090014253 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210090014253 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210090014253 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:00:17.599118 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20404): Terminating (SIGTERM)
09:00:17.640877 run iperf3-srv_10.42.42.10(pid=20405): Terminating (SIGTERM)
09:00:17.682465 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20406): Terminating (SIGTERM)
09:00:17.723701 run iperf3-srv_10.42.42.10(pid=20407): Terminating (SIGTERM)
09:00:17.765062 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20411): Terminating (SIGTERM)
09:00:17.806287 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20414): Terminating (SIGTERM)
09:00:17.848174 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20417): Terminating (SIGTERM)
09:00:17.857902 --- ParallelTerminationStrategy: PID 20404 died...
09:00:17.913535 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20404): Terminated: ok {rc=0}
09:00:17.923697 --- ParallelTerminationStrategy: PID 20405 died...
09:00:17.978810 run iperf3-srv_10.42.42.10(pid=20405): Terminated {rc=256}
09:00:17.988706 --- ParallelTerminationStrategy: PID 20406 died...
09:00:18.044043 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20406): Terminated: ok {rc=0}
09:00:18.054236 --- ParallelTerminationStrategy: PID 20407 died...
09:00:18.109744 run iperf3-srv_10.42.42.10(pid=20407): Terminated {rc=256}
09:00:18.119837 --- ParallelTerminationStrategy: PID 20411 died...
09:00:18.174917 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20411): Terminated: ok {rc=0}
09:00:18.184832 --- ParallelTerminationStrategy: PID 20414 died...
09:00:18.240432 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20414): Terminated: ok {rc=0}
09:00:18.250645 --- ParallelTerminationStrategy: PID 20417 died...
09:00:18.306206 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20417): Terminated: ok {rc=0}
09:00:18.393147 bus /gobi_0: Setting Powered False
09:00:19.495610 bus /gobi_0: Setting Powered False
09:00:20.604616 bus /gobi_6: Setting Powered False
55gprs:trx-umtrx8.0osmo-stp_10.42.42.5(pid=20427): Process ended prematurely: osmo-stp_10.42.42.5(pid=20427) [trial-2372↪gprs:trx-umtrx↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20427)]2318iperf3.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/iperf3.py", line 8, in <module>
setup_run_iperf3_test_parallel(1)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=20427): Process ended prematurely: osmo-stp_10.42.42.5(pid=20427) [trial-2372↪gprs:trx-umtrx↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20427)]
----------------------------------------------
trial-2372 gprs:trx-umtrx iperf3.py
----------------------------------------------
09:00:21.719950 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:21.731471 tst gprs:trx-umtrx: Using 1 x bts (candidates: 1)
09:00:21.879598 tst iperf3.py: using LAC 27296
09:00:22.009493 tst iperf3.py: using RAC 11
09:00:22.140126 tst iperf3.py: using CellId 27296
09:00:22.265575 tst iperf3.py: using BVCI 27297
09:00:22.289213 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.299991 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.310522 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.322105 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.332982 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.344435 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.356015 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.367081 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:22.378590 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:22.659987 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003...
09:00:22.788916 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:00:23.109102 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20419): Launched
09:00:23.376086 run iperf3-srv_10.42.42.10(pid=20420): Launched
09:00:23.564252 tst iperf3.py:8: start network...
09:00:23.693322 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:00:24.178269 run create_hlr_db(pid=20421): Launched
09:00:24.354761 bus /gobi_6: Setting Powered False
09:00:25.375283 run create_hlr_db(pid=20421): Terminated: ok {rc=0}
09:00:25.810906 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20423): Launched
09:00:26.140934 run osmo-hlr_10.42.42.2(pid=20424): Launched
09:00:26.260115 run osmo-stp_10.42.42.5: Starting osmo-stp
09:00:26.828035 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20426): Launched
09:00:27.161868 run osmo-stp_10.42.42.5(pid=20427): Launched
09:00:27.279311 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:00:27.849435 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20429): Launched
09:00:27.965711 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:00:28.253534 run patchelf(pid=20430): Launched
09:00:28.443408 run osmo-stp_10.42.42.5(pid=20427): ERR: Terminated: ERROR {rc=1} [trial-2372↪gprs:trx-umtrx↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20427)]
09:00:28.582555 run osmo-stp_10.42.42.5(pid=20427): stdout:
| (launched: 2022-12-10_09:00:26.993141)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:28.722285 run osmo-stp_10.42.42.5(pid=20427): stderr:
| �[0;m�[38;5;43m20221210090027189 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090027190 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090027190 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090027190 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090027191 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090027191 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090027191 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:28.857107 run osmo-stp_10.42.42.5(pid=20427): stdout:
| (launched: 2022-12-10_09:00:26.993141)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:28.990440 run osmo-stp_10.42.42.5(pid=20427): stderr:
| �[0;m�[38;5;43m20221210090027189 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090027190 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090027190 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090027190 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090027191 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090027191 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090027191 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:29.145366 run patchelf(pid=20430): Terminating (SIGINT)
09:00:29.327083 run patchelf(pid=20430): Terminated: ok {rc=0}
09:00:29.360149 tst iperf3.py:8: ERR: Error: osmo-stp_10.42.42.5(pid=20427): Process ended prematurely: osmo-stp_10.42.42.5(pid=20427) [trial-2372↪gprs:trx-umtrx↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20427)] [trial-2372↪gprs:trx-umtrx↪iperf3.py:8]
09:00:29.369811 tst iperf3.py:8: Test FAILED (7.7 sec)
09:00:29.431546 run osmo-hlr_10.42.42.2(pid=20424): ERR: Terminated: ERROR {rc=237} [trial-2372↪gprs:trx-umtrx↪iperf3.py↪iperf3.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=20424)]
09:00:29.481964 run osmo-hlr_10.42.42.2(pid=20424): stdout:
| (launched: 2022-12-10_09:00:25.976065)
09:00:29.533062 run osmo-hlr_10.42.42.2(pid=20424): stderr:
| �[0;m�[1;31m20221210090026179 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210090026179 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210090026179 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210090026179 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210090026179 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210090026179 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210090026191 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2372/run.2022-12-10_08-20-36/gprs:trx-umtrx/iperf3.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210090026202 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210090026202 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210090026202 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:00:29.577080 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20419): Terminating (SIGTERM)
09:00:29.618969 run iperf3-srv_10.42.42.10(pid=20420): Terminating (SIGTERM)
09:00:29.660439 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20423): Terminating (SIGTERM)
09:00:29.702135 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20426): Terminating (SIGTERM)
09:00:29.743607 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20429): Terminating (SIGTERM)
09:00:29.753417 --- ParallelTerminationStrategy: PID 20419 died...
09:00:29.808979 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20419): Terminated: ok {rc=0}
09:00:29.819105 --- ParallelTerminationStrategy: PID 20420 died...
09:00:29.874261 run iperf3-srv_10.42.42.10(pid=20420): Terminated {rc=256}
09:00:29.884151 --- ParallelTerminationStrategy: PID 20423 died...
09:00:29.939301 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20423): Terminated: ok {rc=0}
09:00:29.949538 --- ParallelTerminationStrategy: PID 20426 died...
09:00:30.004892 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20426): Terminated: ok {rc=0}
09:00:30.015097 --- ParallelTerminationStrategy: PID 20429 died...
09:00:30.070232 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20429): Terminated: ok {rc=0}
09:00:30.156734 bus /gobi_6: Setting Powered False
198gprs:trx-umtrx11.0osmo-stp_10.42.42.5(pid=20445): Process ended prematurely: osmo-stp_10.42.42.5(pid=20445) [trial-2372↪gprs:trx-umtrx↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20445)]2175iperf3m4.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/iperf3m4.py", line 8, in <module>
setup_run_iperf3_test_parallel(4)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=20445): Process ended prematurely: osmo-stp_10.42.42.5(pid=20445) [trial-2372↪gprs:trx-umtrx↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20445)]
----------------------------------------------
trial-2372 gprs:trx-umtrx iperf3m4.py
----------------------------------------------
09:00:31.290632 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.302868 tst gprs:trx-umtrx: Using 1 x bts (candidates: 1)
09:00:31.444930 tst iperf3m4.py: using LAC 27297
09:00:31.576462 tst iperf3m4.py: using RAC 12
09:00:31.707015 tst iperf3m4.py: using CellId 27297
09:00:31.835537 tst iperf3m4.py: using BVCI 27298
09:00:31.861450 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.873416 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.885034 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.896351 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.907661 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.919036 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.930683 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.941930 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:31.953443 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:32.096894 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:32.223616 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:32.342724 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:32.638912 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
09:00:32.775203 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:00:33.115398 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20431): Launched
09:00:33.400036 run iperf3-srv_10.42.42.10(pid=20432): Launched
09:00:33.599335 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
09:00:33.735865 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:00:34.074375 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20433): Launched
09:00:34.356927 run iperf3-srv_10.42.42.10(pid=20434): Launched
09:00:34.556302 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
09:00:34.692807 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:00:35.031991 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20435): Launched
09:00:35.316415 run iperf3-srv_10.42.42.10(pid=20436): Launched
09:00:35.515560 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
09:00:35.652003 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:00:35.992337 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20437): Launched
09:00:36.275439 run iperf3-srv_10.42.42.10(pid=20438): Launched
09:00:36.474537 tst iperf3m4.py:8: start network...
09:00:36.610508 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:00:37.133620 run create_hlr_db(pid=20439): Launched
09:00:37.321250 bus /gobi_6: Setting Powered False
09:00:38.342707 run create_hlr_db(pid=20439): Terminated: ok {rc=0}
09:00:38.756366 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20441): Launched
09:00:39.064805 run osmo-hlr_10.42.42.2(pid=20442): Launched
09:00:39.188201 run osmo-stp_10.42.42.5: Starting osmo-stp
09:00:39.735279 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20444): Launched
09:00:40.045828 run osmo-stp_10.42.42.5(pid=20445): Launched
09:00:40.158843 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:00:40.695334 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20447): Launched
09:00:40.805301 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:00:41.074646 run patchelf(pid=20448): Launched
09:00:41.253928 run osmo-stp_10.42.42.5(pid=20445): ERR: Terminated: ERROR {rc=1} [trial-2372↪gprs:trx-umtrx↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20445)]
09:00:41.384835 run osmo-stp_10.42.42.5(pid=20445): stdout:
| (launched: 2022-12-10_09:00:39.890991)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:41.516216 run osmo-stp_10.42.42.5(pid=20445): stderr:
| �[0;m�[38;5;43m20221210090040065 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090040065 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090040065 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090040066 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090040066 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090040066 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090040066 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:41.643557 run osmo-stp_10.42.42.5(pid=20445): stdout:
| (launched: 2022-12-10_09:00:39.890991)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:41.769877 run osmo-stp_10.42.42.5(pid=20445): stderr:
| �[0;m�[38;5;43m20221210090040065 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090040065 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090040065 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090040066 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090040066 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090040066 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090040066 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:41.916489 run patchelf(pid=20448): Terminating (SIGINT)
09:00:42.088189 run patchelf(pid=20448): Terminated: ok {rc=0}
09:00:42.119771 tst iperf3m4.py:8: ERR: Error: osmo-stp_10.42.42.5(pid=20445): Process ended prematurely: osmo-stp_10.42.42.5(pid=20445) [trial-2372↪gprs:trx-umtrx↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20445)] [trial-2372↪gprs:trx-umtrx↪iperf3m4.py:8]
09:00:42.128833 tst iperf3m4.py:8: Test FAILED (10.9 sec)
09:00:42.189480 run osmo-hlr_10.42.42.2(pid=20442): ERR: Terminated: ERROR {rc=237} [trial-2372↪gprs:trx-umtrx↪iperf3m4.py↪iperf3m4.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=20442)]
09:00:42.238218 run osmo-hlr_10.42.42.2(pid=20442): stdout:
| (launched: 2022-12-10_09:00:38.911612)
09:00:42.287780 run osmo-hlr_10.42.42.2(pid=20442): stderr:
| �[0;m�[1;31m20221210090039092 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210090039092 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210090039092 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210090039092 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210090039092 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210090039092 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210090039105 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2372/run.2022-12-10_08-20-36/gprs:trx-umtrx/iperf3m4.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210090039114 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210090039114 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210090039114 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:00:42.329200 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20431): Terminating (SIGTERM)
09:00:42.368657 run iperf3-srv_10.42.42.10(pid=20432): Terminating (SIGTERM)
09:00:42.408585 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20433): Terminating (SIGTERM)
09:00:42.448050 run iperf3-srv_10.42.42.10(pid=20434): Terminating (SIGTERM)
09:00:42.487801 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20435): Terminating (SIGTERM)
09:00:42.527312 run iperf3-srv_10.42.42.10(pid=20436): Terminating (SIGTERM)
09:00:42.566710 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20437): Terminating (SIGTERM)
09:00:42.606385 run iperf3-srv_10.42.42.10(pid=20438): Terminating (SIGTERM)
09:00:42.645875 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20441): Terminating (SIGTERM)
09:00:42.685375 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20444): Terminating (SIGTERM)
09:00:42.724847 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20447): Terminating (SIGTERM)
09:00:42.734480 --- ParallelTerminationStrategy: PID 20431 died...
09:00:42.787478 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20431): Terminated: ok {rc=0}
09:00:42.797392 --- ParallelTerminationStrategy: PID 20432 died...
09:00:42.849882 run iperf3-srv_10.42.42.10(pid=20432): Terminated {rc=256}
09:00:42.859336 --- ParallelTerminationStrategy: PID 20433 died...
09:00:42.911624 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20433): Terminated: ok {rc=0}
09:00:42.921445 --- ParallelTerminationStrategy: PID 20434 died...
09:00:42.973261 run iperf3-srv_10.42.42.10(pid=20434): Terminated {rc=256}
09:00:42.982947 --- ParallelTerminationStrategy: PID 20435 died...
09:00:43.035353 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20435): Terminated: ok {rc=0}
09:00:43.044855 --- ParallelTerminationStrategy: PID 20436 died...
09:00:43.097093 run iperf3-srv_10.42.42.10(pid=20436): Terminated {rc=256}
09:00:43.106844 --- ParallelTerminationStrategy: PID 20437 died...
09:00:43.159342 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=20437): Terminated: ok {rc=0}
09:00:43.169163 --- ParallelTerminationStrategy: PID 20438 died...
09:00:43.221334 run iperf3-srv_10.42.42.10(pid=20438): Terminated {rc=256}
09:00:43.231017 --- ParallelTerminationStrategy: PID 20441 died...
09:00:43.283670 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20441): Terminated: ok {rc=0}
09:00:43.293236 --- ParallelTerminationStrategy: PID 20444 died...
09:00:43.345790 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20444): Terminated: ok {rc=0}
09:00:43.355358 --- ParallelTerminationStrategy: PID 20447 died...
09:00:43.407859 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20447): Terminated: ok {rc=0}
09:00:43.489966 bus /gobi_4: Setting Powered False
09:00:44.596788 bus /gobi_4: Setting Powered False
09:00:45.703400 bus /gobi_1: Setting Powered False
09:00:46.818011 bus /gobi_1: Setting Powered False
09:00:47.926958 bus /gobi_0: Setting Powered False
09:00:49.030837 bus /gobi_0: Setting Powered False
09:00:50.138910 bus /gobi_6: Setting Powered False
173gprs:trx-umtrx7.0osmo-stp_10.42.42.5(pid=20455): Process ended prematurely: osmo-stp_10.42.42.5(pid=20455) [trial-2372↪gprs:trx-umtrx↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20455)]2200ping.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/ping.py", line 22, in <module>
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=20455): Process ended prematurely: osmo-stp_10.42.42.5(pid=20455) [trial-2372↪gprs:trx-umtrx↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20455)]
----------------------------------------------
trial-2372 gprs:trx-umtrx ping.py
----------------------------------------------
09:00:51.205309 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.216377 tst gprs:trx-umtrx: Using 1 x bts (candidates: 1)
09:00:51.353726 tst ping.py: using LAC 27298
09:00:51.478498 tst ping.py: using RAC 13
09:00:51.604529 tst ping.py: using CellId 27298
09:00:51.732492 tst ping.py: using BVCI 27299
09:00:51.756900 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.768271 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.779407 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.790607 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.801664 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.812676 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.823743 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:51.834801 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:00:52.110117 tst ping.py:19: start network...
09:00:52.235264 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:00:52.711207 run create_hlr_db(pid=20449): Launched
09:00:52.884396 bus /gobi_6: Setting Powered False
09:00:53.905438 run create_hlr_db(pid=20449): Terminated: ok {rc=0}
09:00:54.308069 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20451): Launched
09:00:54.611135 run osmo-hlr_10.42.42.2(pid=20452): Launched
09:00:54.727105 run osmo-stp_10.42.42.5: Starting osmo-stp
09:00:55.261761 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20454): Launched
09:00:55.566585 run osmo-stp_10.42.42.5(pid=20455): Launched
09:00:55.676691 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:00:56.204488 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20457): Launched
09:00:56.311686 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:00:56.578556 run patchelf(pid=20458): Launched
09:00:56.755574 run osmo-stp_10.42.42.5(pid=20455): ERR: Terminated: ERROR {rc=1} [trial-2372↪gprs:trx-umtrx↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20455)]
09:00:56.884521 run osmo-stp_10.42.42.5(pid=20455): stdout:
| (launched: 2022-12-10_09:00:55.414036)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:57.014074 run osmo-stp_10.42.42.5(pid=20455): stderr:
| �[0;m�[38;5;43m20221210090055587 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090055587 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090055588 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090055588 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090055588 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090055589 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090055589 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:57.139321 run osmo-stp_10.42.42.5(pid=20455): stdout:
| (launched: 2022-12-10_09:00:55.414036)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:00:57.263518 run osmo-stp_10.42.42.5(pid=20455): stderr:
| �[0;m�[38;5;43m20221210090055587 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090055587 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090055588 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090055588 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090055588 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090055589 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090055589 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:00:57.407340 run patchelf(pid=20458): Terminating (SIGINT)
09:00:57.576676 run patchelf(pid=20458): Terminated: ok {rc=0}
09:00:57.607533 tst ping.py:22: ERR: Error: osmo-stp_10.42.42.5(pid=20455): Process ended prematurely: osmo-stp_10.42.42.5(pid=20455) [trial-2372↪gprs:trx-umtrx↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20455)] [trial-2372↪gprs:trx-umtrx↪ping.py:22]
09:00:57.616038 tst ping.py:22: Test FAILED (6.4 sec)
09:00:57.675626 run osmo-hlr_10.42.42.2(pid=20452): ERR: Terminated: ERROR {rc=237} [trial-2372↪gprs:trx-umtrx↪ping.py↪ping.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=20452)]
09:00:57.724356 run osmo-hlr_10.42.42.2(pid=20452): stdout:
| (launched: 2022-12-10_09:00:54.460335)
09:00:57.773656 run osmo-hlr_10.42.42.2(pid=20452): stderr:
| �[0;m�[1;31m20221210090054639 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210090054639 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210090054639 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210090054639 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210090054639 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210090054639 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210090054650 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2372/run.2022-12-10_08-20-36/gprs:trx-umtrx/ping.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210090054658 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210090054658 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210090054659 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:00:57.815095 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20451): Terminating (SIGTERM)
09:00:57.854485 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20454): Terminating (SIGTERM)
09:00:57.893874 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20457): Terminating (SIGTERM)
09:00:57.903316 --- ParallelTerminationStrategy: PID 20451 died...
09:00:57.956282 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20451): Terminated: ok {rc=0}
09:00:57.966248 --- ParallelTerminationStrategy: PID 20454 died...
09:00:58.018966 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20454): Terminated: ok {rc=0}
09:00:58.028746 --- ParallelTerminationStrategy: PID 20457 died...
09:00:58.081372 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20457): Terminated: ok {rc=0}
09:00:58.169366 bus /gobi_6: Setting Powered False
234gprs:trx-umtrx7.0osmo-stp_10.42.42.5(pid=20465): Process ended prematurely: osmo-stp_10.42.42.5(pid=20465) [trial-2372↪gprs:trx-umtrx↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20465)]2139ping_idle_ping.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/ping_idle_ping.py", line 27, in <module>
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=20465): Process ended prematurely: osmo-stp_10.42.42.5(pid=20465) [trial-2372↪gprs:trx-umtrx↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20465)]
----------------------------------------------
trial-2372 gprs:trx-umtrx ping_idle_ping.py
----------------------------------------------
09:00:59.219230 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.230501 tst gprs:trx-umtrx: Using 1 x bts (candidates: 1)
09:00:59.369143 tst ping_idle_ping.py: using LAC 27299
09:00:59.494804 tst ping_idle_ping.py: using RAC 14
09:00:59.622956 tst ping_idle_ping.py: using CellId 27299
09:00:59.751193 tst ping_idle_ping.py: using BVCI 27300
09:00:59.776020 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.787341 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.798274 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.809702 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.820733 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.831729 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.842889 tst gprs:trx-umtrx: Using 1 x ip_address (candidates: 9)
09:00:59.854031 tst gprs:trx-umtrx: Using 1 x modem (candidates: 4)
09:01:00.130550 tst ping_idle_ping.py:24: start network...
09:01:00.255895 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:01:00.736930 run create_hlr_db(pid=20459): Launched
09:01:00.937415 bus /gobi_6: Setting Powered False
09:01:01.942306 run create_hlr_db(pid=20459): Terminated: ok {rc=0}
09:01:02.386298 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20461): Launched
09:01:02.694607 run osmo-hlr_10.42.42.2(pid=20462): Launched
09:01:02.814351 run osmo-stp_10.42.42.5: Starting osmo-stp
09:01:03.347633 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20464): Launched
09:01:03.651338 run osmo-stp_10.42.42.5(pid=20465): Launched
09:01:03.761845 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:01:04.289395 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20467): Launched
09:01:04.396692 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:01:04.662848 run patchelf(pid=20468): Launched
09:01:04.839224 run osmo-stp_10.42.42.5(pid=20465): ERR: Terminated: ERROR {rc=1} [trial-2372↪gprs:trx-umtrx↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20465)]
09:01:04.967636 run osmo-stp_10.42.42.5(pid=20465): stdout:
| (launched: 2022-12-10_09:01:03.500232)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:01:05.096922 run osmo-stp_10.42.42.5(pid=20465): stderr:
| �[0;m�[38;5;43m20221210090103673 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090103673 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090103673 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090103674 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090103676 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090103676 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090103676 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:01:05.222089 run osmo-stp_10.42.42.5(pid=20465): stdout:
| (launched: 2022-12-10_09:01:03.500232)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:01:05.346090 run osmo-stp_10.42.42.5(pid=20465): stderr:
| �[0;m�[38;5;43m20221210090103673 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210090103673 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210090103673 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210090103674 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210090103676 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210090103676 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210090103676 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:01:05.490361 run patchelf(pid=20468): Terminating (SIGINT)
09:01:05.659474 run patchelf(pid=20468): Terminated: ok {rc=0}
09:01:05.690075 tst ping_idle_ping.py:27: ERR: Error: osmo-stp_10.42.42.5(pid=20465): Process ended prematurely: osmo-stp_10.42.42.5(pid=20465) [trial-2372↪gprs:trx-umtrx↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=20465)] [trial-2372↪gprs:trx-umtrx↪ping_idle_ping.py:27]
09:01:05.698599 tst ping_idle_ping.py:27: Test FAILED (6.5 sec)
09:01:05.758302 run osmo-hlr_10.42.42.2(pid=20462): ERR: Terminated: ERROR {rc=237} [trial-2372↪gprs:trx-umtrx↪ping_idle_ping.py↪ping_idle_ping.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=20462)]
09:01:05.806836 run osmo-hlr_10.42.42.2(pid=20462): stdout:
| (launched: 2022-12-10_09:01:02.540102)
09:01:05.856050 run osmo-hlr_10.42.42.2(pid=20462): stderr:
| �[0;m�[1;31m20221210090102718 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210090102718 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210090102718 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210090102718 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210090102718 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210090102718 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210090102730 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2372/run.2022-12-10_08-20-36/gprs:trx-umtrx/ping_idle_ping.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210090102739 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210090102739 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210090102739 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:01:05.897689 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20461): Terminating (SIGTERM)
09:01:05.937244 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20464): Terminating (SIGTERM)
09:01:05.976841 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20467): Terminating (SIGTERM)
09:01:05.986266 --- ParallelTerminationStrategy: PID 20461 died...
09:01:06.038976 run pcap-recorder_any(filters='host 10.42.42.2')(pid=20461): Terminated: ok {rc=0}
09:01:06.048677 --- ParallelTerminationStrategy: PID 20464 died...
09:01:06.101127 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=20464): Terminated: ok {rc=0}
09:01:06.110840 --- ParallelTerminationStrategy: PID 20467 died...
09:01:06.164059 run pcap-recorder_any(filters='host 10.42.42.6')(pid=20467): Terminated: ok {rc=0}
09:01:06.251549 bus /gobi_6: Setting Powered False
5gprs:trx-umtrx00