60gprs:sysmo10.0osmo-stp_10.42.42.5(pid=26071): Process ended prematurely: osmo-stp_10.42.42.5(pid=26071) [trial-2373↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26071)]2314cs_paging_gprs_active.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/cs_paging_gprs_active.py", line 38, in <module>
setup_run_iperf3_test_parallel(2, ready_cb=ready_cb_place_voicecall)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26071): Process ended prematurely: osmo-stp_10.42.42.5(pid=26071) [trial-2373↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26071)]
----------------------------------------------
trial-2373 gprs:sysmo cs_paging_gprs_active.py
----------------------------------------------
09:53:58.137757 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.148529 tst gprs:sysmo: Using 1 x bts (candidates: 1)
09:53:58.279277 tst cs_paging_gprs_active.py: using LAC 27577
09:53:58.405931 tst cs_paging_gprs_active.py: using RAC 37
09:53:58.532416 tst cs_paging_gprs_active.py: using CellId 27577
09:53:58.658712 tst cs_paging_gprs_active.py: using BVCI 27578
09:53:58.683447 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.694421 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.705295 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.716308 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.727341 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.738341 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.749522 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.760414 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:53:58.771655 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:53:58.888332 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:53:59.166021 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5003...
09:53:59.293511 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:53:59.606742 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26061): Launched
09:53:59.866505 run iperf3-srv_10.42.42.10(pid=26062): Launched
09:54:00.050777 tst cs_paging_gprs_active.py:38: start iperfv3 server 10.42.42.10:5004...
09:54:00.176430 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:54:00.489354 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26063): Launched
09:54:00.747259 run iperf3-srv_10.42.42.10(pid=26064): Launched
09:54:00.942051 tst cs_paging_gprs_active.py:38: start network...
09:54:01.069566 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:54:01.599783 run create_hlr_db(pid=26065): Launched
09:54:01.778407 bus /sierra_2: Setting Powered False
09:54:02.795657 run create_hlr_db(pid=26065): Terminated: ok {rc=0}
09:54:03.199196 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26067): Launched
09:54:03.502056 run osmo-hlr_10.42.42.2(pid=26068): Launched
09:54:03.613363 run osmo-stp_10.42.42.5: Starting osmo-stp
09:54:04.139126 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26070): Launched
09:54:04.441678 run osmo-stp_10.42.42.5(pid=26071): Launched
09:54:04.560638 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:54:06.066798 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26073): Launched
09:54:06.174397 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:54:06.439332 run patchelf(pid=26074): Launched
09:54:06.617152 run osmo-stp_10.42.42.5(pid=26071): ERR: Terminated: ERROR {rc=1} [trial-2373↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26071)]
09:54:06.747086 run osmo-stp_10.42.42.5(pid=26071): stdout:
| (launched: 2022-12-10_09:54:04.293217)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:06.877178 run osmo-stp_10.42.42.5(pid=26071): stderr:
| �[0;m�[38;5;43m20221210095404466 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095404467 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095404467 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095404467 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095404468 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095404468 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095404468 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:07.002483 run osmo-stp_10.42.42.5(pid=26071): stdout:
| (launched: 2022-12-10_09:54:04.293217)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:07.126781 run osmo-stp_10.42.42.5(pid=26071): stderr:
| �[0;m�[38;5;43m20221210095404466 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095404467 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095404467 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095404467 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095404468 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095404468 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095404468 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:07.271735 run patchelf(pid=26074): Terminating (SIGINT)
09:54:07.442874 run patchelf(pid=26074): Terminated: ok {rc=0}
09:54:07.474102 tst cs_paging_gprs_active.py:38: ERR: Error: osmo-stp_10.42.42.5(pid=26071): Process ended prematurely: osmo-stp_10.42.42.5(pid=26071) [trial-2373↪gprs:sysmo↪cs_paging_gprs_active.py:38↪cs_paging_gprs_active.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26071)] [trial-2373↪gprs:sysmo↪cs_paging_gprs_active.py:38]
09:54:07.483121 tst cs_paging_gprs_active.py:38: Test FAILED (9.4 sec)
09:54:07.543087 run osmo-hlr_10.42.42.2(pid=26068): ERR: Terminated: ERROR {rc=237} [trial-2373↪gprs:sysmo↪cs_paging_gprs_active.py↪cs_paging_gprs_active.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26068)]
09:54:07.591837 run osmo-hlr_10.42.42.2(pid=26068): stdout:
| (launched: 2022-12-10_09:54:03.353767)
09:54:07.641234 run osmo-hlr_10.42.42.2(pid=26068): stderr:
| �[0;m�[1;31m20221210095403531 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210095403531 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210095403532 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210095403532 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210095403532 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210095403532 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210095403547 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2373/run.2022-12-10_09-48-07/gprs:sysmo/cs_paging_gprs_active.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210095403557 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210095403557 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210095403558 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:54:07.683322 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26061): Terminating (SIGTERM)
09:54:07.723646 run iperf3-srv_10.42.42.10(pid=26062): Terminating (SIGTERM)
09:54:07.763383 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26063): Terminating (SIGTERM)
09:54:07.802970 run iperf3-srv_10.42.42.10(pid=26064): Terminating (SIGTERM)
09:54:07.842840 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26067): Terminating (SIGTERM)
09:54:07.882434 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26070): Terminating (SIGTERM)
09:54:07.922048 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26073): Terminating (SIGTERM)
09:54:07.931132 --- ParallelTerminationStrategy: PID 26061 died...
09:54:07.984535 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26061): Terminated: ok {rc=0}
09:54:07.994189 --- ParallelTerminationStrategy: PID 26062 died...
09:54:08.046967 run iperf3-srv_10.42.42.10(pid=26062): Terminated {rc=256}
09:54:08.056454 --- ParallelTerminationStrategy: PID 26063 died...
09:54:08.109646 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26063): Terminated: ok {rc=0}
09:54:08.119162 --- ParallelTerminationStrategy: PID 26064 died...
09:54:08.172240 run iperf3-srv_10.42.42.10(pid=26064): Terminated {rc=256}
09:54:08.181904 --- ParallelTerminationStrategy: PID 26067 died...
09:54:08.235197 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26067): Terminated: ok {rc=0}
09:54:08.244829 --- ParallelTerminationStrategy: PID 26070 died...
09:54:08.297693 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26070): Terminated: ok {rc=0}
09:54:08.307282 --- ParallelTerminationStrategy: PID 26073 died...
09:54:08.360162 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26073): Terminated: ok {rc=0}
09:54:08.449753 bus /gobi_0: Setting Powered False
09:54:09.553238 bus /gobi_0: Setting Powered False
09:54:10.664595 bus /gobi_6: Setting Powered False
60gprs:sysmo8.0osmo-stp_10.42.42.5(pid=26083): Process ended prematurely: osmo-stp_10.42.42.5(pid=26083) [trial-2373↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26083)]2314iperf3.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/iperf3.py", line 8, in <module>
setup_run_iperf3_test_parallel(1)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26083): Process ended prematurely: osmo-stp_10.42.42.5(pid=26083) [trial-2373↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26083)]
----------------------------------------------
trial-2373 gprs:sysmo iperf3.py
----------------------------------------------
09:54:11.776495 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:11.787668 tst gprs:sysmo: Using 1 x bts (candidates: 1)
09:54:11.916492 tst iperf3.py: using LAC 27578
09:54:12.044006 tst iperf3.py: using RAC 38
09:54:12.172046 tst iperf3.py: using CellId 27578
09:54:12.298358 tst iperf3.py: using BVCI 27579
09:54:12.323013 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.333999 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.344855 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.355899 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.366861 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.377934 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.389022 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.399943 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:12.411083 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:54:12.706151 tst iperf3.py:8: start iperfv3 server 10.42.42.10:5003...
09:54:12.841129 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:54:13.175192 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26075): Launched
09:54:13.453736 run iperf3-srv_10.42.42.10(pid=26076): Launched
09:54:13.652189 tst iperf3.py:8: start network...
09:54:13.787824 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:54:14.296227 run create_hlr_db(pid=26077): Launched
09:54:14.484399 bus /gobi_6: Setting Powered False
09:54:15.504497 run create_hlr_db(pid=26077): Terminated: ok {rc=0}
09:54:15.944274 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26079): Launched
09:54:16.314085 run osmo-hlr_10.42.42.2(pid=26080): Launched
09:54:16.547098 run osmo-stp_10.42.42.5: Starting osmo-stp
09:54:17.190706 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26082): Launched
09:54:17.516322 run osmo-stp_10.42.42.5(pid=26083): Launched
09:54:17.634680 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:54:18.197872 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26085): Launched
09:54:18.314523 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:54:18.595610 run patchelf(pid=26086): Launched
09:54:18.785411 run osmo-stp_10.42.42.5(pid=26083): ERR: Terminated: ERROR {rc=1} [trial-2373↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26083)]
09:54:18.924374 run osmo-stp_10.42.42.5(pid=26083): stdout:
| (launched: 2022-12-10_09:54:17.355781)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:19.063321 run osmo-stp_10.42.42.5(pid=26083): stderr:
| �[0;m�[38;5;43m20221210095417544 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095417544 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095417545 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095417545 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095417547 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095417547 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095417548 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:19.197390 run osmo-stp_10.42.42.5(pid=26083): stdout:
| (launched: 2022-12-10_09:54:17.355781)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:19.330963 run osmo-stp_10.42.42.5(pid=26083): stderr:
| �[0;m�[38;5;43m20221210095417544 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095417544 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095417545 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095417545 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095417547 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095417547 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095417548 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:19.485687 run patchelf(pid=26086): Terminating (SIGINT)
09:54:19.667249 run patchelf(pid=26086): Terminated: ok {rc=0}
09:54:19.700445 tst iperf3.py:8: ERR: Error: osmo-stp_10.42.42.5(pid=26083): Process ended prematurely: osmo-stp_10.42.42.5(pid=26083) [trial-2373↪gprs:sysmo↪iperf3.py:8↪iperf3.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26083)] [trial-2373↪gprs:sysmo↪iperf3.py:8]
09:54:19.710257 tst iperf3.py:8: Test FAILED (8.0 sec)
09:54:19.774013 run osmo-hlr_10.42.42.2(pid=26080): ERR: Terminated: ERROR {rc=237} [trial-2373↪gprs:sysmo↪iperf3.py↪iperf3.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26080)]
09:54:19.826073 run osmo-hlr_10.42.42.2(pid=26080): stdout:
| (launched: 2022-12-10_09:54:16.114429)
09:54:19.878508 run osmo-hlr_10.42.42.2(pid=26080): stderr:
| �[0;m�[1;31m20221210095416467 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210095416467 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210095416467 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210095416467 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210095416467 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210095416467 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210095416486 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2373/run.2022-12-10_09-48-07/gprs:sysmo/iperf3.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210095416494 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210095416494 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210095416494 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:54:19.922874 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26075): Terminating (SIGTERM)
09:54:19.965142 run iperf3-srv_10.42.42.10(pid=26076): Terminating (SIGTERM)
09:54:20.007168 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26079): Terminating (SIGTERM)
09:54:20.049503 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26082): Terminating (SIGTERM)
09:54:20.091477 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26085): Terminating (SIGTERM)
09:54:20.101190 --- ParallelTerminationStrategy: PID 26075 died...
09:54:20.157476 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26075): Terminated: ok {rc=0}
09:54:20.167514 --- ParallelTerminationStrategy: PID 26076 died...
09:54:20.224032 run iperf3-srv_10.42.42.10(pid=26076): Terminated {rc=256}
09:54:20.234007 --- ParallelTerminationStrategy: PID 26079 died...
09:54:20.289862 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26079): Terminated: ok {rc=0}
09:54:20.300168 --- ParallelTerminationStrategy: PID 26082 died...
09:54:20.356257 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26082): Terminated: ok {rc=0}
09:54:20.366622 --- ParallelTerminationStrategy: PID 26085 died...
09:54:20.422665 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26085): Terminated: ok {rc=0}
09:54:20.510359 bus /gobi_6: Setting Powered False
61gprs:sysmo12.0osmo-stp_10.42.42.5(pid=26101): Process ended prematurely: osmo-stp_10.42.42.5(pid=26101) [trial-2373↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26101)]2313iperf3m4.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/iperf3m4.py", line 8, in <module>
setup_run_iperf3_test_parallel(4)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/lib/testlib.py", line 67, in setup_run_iperf3_test_parallel
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26101): Process ended prematurely: osmo-stp_10.42.42.5(pid=26101) [trial-2373↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26101)]
----------------------------------------------
trial-2373 gprs:sysmo iperf3m4.py
----------------------------------------------
09:54:21.626297 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:21.637863 tst gprs:sysmo: Using 1 x bts (candidates: 1)
09:54:21.769501 tst iperf3m4.py: using LAC 27579
09:54:21.898210 tst iperf3m4.py: using RAC 39
09:54:22.024417 tst iperf3m4.py: using CellId 27579
09:54:22.152897 tst iperf3m4.py: using BVCI 27580
09:54:22.177723 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.188731 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.199643 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.211146 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.222259 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.233089 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.244145 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.255338 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:22.266515 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:54:22.384791 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:54:22.503919 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:54:22.619975 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:54:22.914269 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5003...
09:54:23.049954 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:54:23.385177 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26087): Launched
09:54:23.663145 run iperf3-srv_10.42.42.10(pid=26088): Launched
09:54:23.862610 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5004...
09:54:23.998465 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:54:24.333768 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26089): Launched
09:54:24.610866 run iperf3-srv_10.42.42.10(pid=26090): Launched
09:54:24.810018 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5005...
09:54:24.945624 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:54:25.280228 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26091): Launched
09:54:25.556571 run iperf3-srv_10.42.42.10(pid=26092): Launched
09:54:25.753946 tst iperf3m4.py:8: start iperfv3 server 10.42.42.10:5006...
09:54:25.888665 run iperf3-srv_10.42.42.10: Starting iperf3-srv
09:54:26.222916 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26093): Launched
09:54:26.513646 run iperf3-srv_10.42.42.10(pid=26094): Launched
09:54:26.711561 tst iperf3m4.py:8: start network...
09:54:26.846406 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:54:27.354669 run create_hlr_db(pid=26095): Launched
09:54:27.542908 bus /gobi_6: Setting Powered False
09:54:28.562822 run create_hlr_db(pid=26095): Terminated: ok {rc=0}
09:54:28.992069 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26097): Launched
09:54:29.316110 run osmo-hlr_10.42.42.2(pid=26098): Launched
09:54:29.436697 run osmo-stp_10.42.42.5: Starting osmo-stp
09:54:30.000342 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26100): Launched
09:54:30.323936 run osmo-stp_10.42.42.5(pid=26101): Launched
09:54:30.442516 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:54:31.039151 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26103): Launched
09:54:31.162472 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:54:31.467640 run patchelf(pid=26104): Launched
09:54:31.661573 run osmo-stp_10.42.42.5(pid=26101): ERR: Terminated: ERROR {rc=1} [trial-2373↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26101)]
09:54:31.804110 run osmo-stp_10.42.42.5(pid=26101): stdout:
| (launched: 2022-12-10_09:54:30.164643)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:31.947056 run osmo-stp_10.42.42.5(pid=26101): stderr:
| �[0;m�[38;5;43m20221210095430358 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095430358 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095430358 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095430359 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095430359 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095430359 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095430359 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:32.084509 run osmo-stp_10.42.42.5(pid=26101): stdout:
| (launched: 2022-12-10_09:54:30.164643)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:32.221145 run osmo-stp_10.42.42.5(pid=26101): stderr:
| �[0;m�[38;5;43m20221210095430358 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095430358 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095430358 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095430359 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095430359 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095430359 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095430359 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:32.376240 run patchelf(pid=26104): Terminating (SIGINT)
09:54:32.558869 run patchelf(pid=26104): Terminated: ok {rc=0}
09:54:32.592326 tst iperf3m4.py:8: ERR: Error: osmo-stp_10.42.42.5(pid=26101): Process ended prematurely: osmo-stp_10.42.42.5(pid=26101) [trial-2373↪gprs:sysmo↪iperf3m4.py:8↪iperf3m4.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26101)] [trial-2373↪gprs:sysmo↪iperf3m4.py:8]
09:54:32.602379 tst iperf3m4.py:8: Test FAILED (11.0 sec)
09:54:32.667834 run osmo-hlr_10.42.42.2(pid=26098): ERR: Terminated: ERROR {rc=237} [trial-2373↪gprs:sysmo↪iperf3m4.py↪iperf3m4.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26098)]
09:54:32.720444 run osmo-hlr_10.42.42.2(pid=26098): stdout:
| (launched: 2022-12-10_09:54:29.156760)
09:54:32.773462 run osmo-hlr_10.42.42.2(pid=26098): stderr:
| �[0;m�[1;31m20221210095429354 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210095429354 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210095429354 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210095429354 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210095429354 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210095429354 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210095429367 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2373/run.2022-12-10_09-48-07/gprs:sysmo/iperf3m4.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210095429375 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210095429375 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210095429376 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:54:32.818468 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26087): Terminating (SIGTERM)
09:54:32.861325 run iperf3-srv_10.42.42.10(pid=26088): Terminating (SIGTERM)
09:54:32.904286 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26089): Terminating (SIGTERM)
09:54:32.947030 run iperf3-srv_10.42.42.10(pid=26090): Terminating (SIGTERM)
09:54:32.990065 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26091): Terminating (SIGTERM)
09:54:33.032864 run iperf3-srv_10.42.42.10(pid=26092): Terminating (SIGTERM)
09:54:33.075610 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26093): Terminating (SIGTERM)
09:54:33.118371 run iperf3-srv_10.42.42.10(pid=26094): Terminating (SIGTERM)
09:54:33.160887 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26097): Terminating (SIGTERM)
09:54:33.203613 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26100): Terminating (SIGTERM)
09:54:33.247005 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26103): Terminating (SIGTERM)
09:54:33.256837 --- ParallelTerminationStrategy: PID 26087 died...
09:54:33.314236 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26087): Terminated: ok {rc=0}
09:54:33.324598 --- ParallelTerminationStrategy: PID 26088 died...
09:54:33.381591 run iperf3-srv_10.42.42.10(pid=26088): Terminated {rc=256}
09:54:33.391912 --- ParallelTerminationStrategy: PID 26089 died...
09:54:33.448874 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26089): Terminated: ok {rc=0}
09:54:33.459090 --- ParallelTerminationStrategy: PID 26090 died...
09:54:33.516283 run iperf3-srv_10.42.42.10(pid=26090): Terminated {rc=256}
09:54:33.526556 --- ParallelTerminationStrategy: PID 26091 died...
09:54:33.583708 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26091): Terminated: ok {rc=0}
09:54:33.594012 --- ParallelTerminationStrategy: PID 26092 died...
09:54:33.650989 run iperf3-srv_10.42.42.10(pid=26092): Terminated {rc=256}
09:54:33.661213 --- ParallelTerminationStrategy: PID 26093 died...
09:54:33.717871 run pcap-recorder_any(filters='host 10.42.42.10 and port not 22')(pid=26093): Terminated: ok {rc=0}
09:54:33.728411 --- ParallelTerminationStrategy: PID 26094 died...
09:54:33.785329 run iperf3-srv_10.42.42.10(pid=26094): Terminated {rc=256}
09:54:33.795713 --- ParallelTerminationStrategy: PID 26097 died...
09:54:33.852646 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26097): Terminated: ok {rc=0}
09:54:33.862856 --- ParallelTerminationStrategy: PID 26100 died...
09:54:33.920031 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26100): Terminated: ok {rc=0}
09:54:33.930354 --- ParallelTerminationStrategy: PID 26103 died...
09:54:33.987316 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26103): Terminated: ok {rc=0}
09:54:34.075531 bus /gobi_4: Setting Powered False
09:54:35.179083 bus /gobi_4: Setting Powered False
09:54:36.286403 bus /gobi_1: Setting Powered False
09:54:37.387758 bus /gobi_1: Setting Powered False
09:54:38.500432 bus /gobi_0: Setting Powered False
09:54:39.602795 bus /gobi_0: Setting Powered False
09:54:40.715628 bus /gobi_6: Setting Powered False
43gprs:sysmo7.0osmo-stp_10.42.42.5(pid=26111): Process ended prematurely: osmo-stp_10.42.42.5(pid=26111) [trial-2373↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26111)]2331ping.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/ping.py", line 22, in <module>
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26111): Process ended prematurely: osmo-stp_10.42.42.5(pid=26111) [trial-2373↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26111)]
----------------------------------------------
trial-2373 gprs:sysmo ping.py
----------------------------------------------
09:54:41.783078 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:41.794147 tst gprs:sysmo: Using 1 x bts (candidates: 1)
09:54:41.922293 tst ping.py: using LAC 27580
09:54:42.046022 tst ping.py: using RAC 40
09:54:42.169553 tst ping.py: using CellId 27580
09:54:42.293303 tst ping.py: using BVCI 27581
09:54:42.317920 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:42.329107 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:42.340048 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:42.351604 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:42.362518 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:42.373491 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:42.384592 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:42.395796 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:54:42.671163 tst ping.py:19: start network...
09:54:42.794588 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:54:43.270207 run create_hlr_db(pid=26105): Launched
09:54:43.441852 bus /gobi_6: Setting Powered False
09:54:44.463299 run create_hlr_db(pid=26105): Terminated: ok {rc=0}
09:54:44.884106 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26107): Launched
09:54:45.199795 run osmo-hlr_10.42.42.2(pid=26108): Launched
09:54:45.317076 run osmo-stp_10.42.42.5: Starting osmo-stp
09:54:45.871860 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26110): Launched
09:54:46.305569 run osmo-stp_10.42.42.5(pid=26111): Launched
09:54:46.522017 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:54:47.092159 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26113): Launched
09:54:47.200324 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:54:47.467862 run patchelf(pid=26114): Launched
09:54:47.643475 run osmo-stp_10.42.42.5(pid=26111): ERR: Terminated: ERROR {rc=1} [trial-2373↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26111)]
09:54:47.771960 run osmo-stp_10.42.42.5(pid=26111): stdout:
| (launched: 2022-12-10_09:54:46.044050)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:47.900547 run osmo-stp_10.42.42.5(pid=26111): stderr:
| �[0;m�[38;5;43m20221210095446332 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095446332 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095446333 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095446333 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095446334 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095446334 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095446334 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:48.024785 run osmo-stp_10.42.42.5(pid=26111): stdout:
| (launched: 2022-12-10_09:54:46.044050)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:48.147490 run osmo-stp_10.42.42.5(pid=26111): stderr:
| �[0;m�[38;5;43m20221210095446332 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095446332 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095446333 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095446333 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095446334 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095446334 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095446334 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:48.290604 run patchelf(pid=26114): Terminating (SIGINT)
09:54:48.458536 run patchelf(pid=26114): Terminated: ok {rc=0}
09:54:48.489126 tst ping.py:22: ERR: Error: osmo-stp_10.42.42.5(pid=26111): Process ended prematurely: osmo-stp_10.42.42.5(pid=26111) [trial-2373↪gprs:sysmo↪ping.py:22↪ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26111)] [trial-2373↪gprs:sysmo↪ping.py:22]
09:54:48.497630 tst ping.py:22: Test FAILED (6.7 sec)
09:54:48.557569 run osmo-hlr_10.42.42.2(pid=26108): ERR: Terminated: ERROR {rc=237} [trial-2373↪gprs:sysmo↪ping.py↪ping.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26108)]
09:54:48.607020 run osmo-hlr_10.42.42.2(pid=26108): stdout:
| (launched: 2022-12-10_09:54:45.045462)
09:54:48.657083 run osmo-hlr_10.42.42.2(pid=26108): stderr:
| �[0;m�[1;31m20221210095445245 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210095445245 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210095445245 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210095445245 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210095445246 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210095445246 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210095445270 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2373/run.2022-12-10_09-48-07/gprs:sysmo/ping.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210095445281 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210095445281 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210095445281 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:54:48.699413 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26107): Terminating (SIGTERM)
09:54:48.739625 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26110): Terminating (SIGTERM)
09:54:48.779871 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26113): Terminating (SIGTERM)
09:54:48.789376 --- ParallelTerminationStrategy: PID 26107 died...
09:54:48.843418 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26107): Terminated: ok {rc=0}
09:54:48.853096 --- ParallelTerminationStrategy: PID 26110 died...
09:54:48.906245 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26110): Terminated: ok {rc=0}
09:54:48.915778 --- ParallelTerminationStrategy: PID 26113 died...
09:54:48.968988 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26113): Terminated: ok {rc=0}
09:54:49.057046 bus /gobi_6: Setting Powered False
43gprs:sysmo7.0osmo-stp_10.42.42.5(pid=26121): Process ended prematurely: osmo-stp_10.42.42.5(pid=26121) [trial-2373↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26121)]2331ping_idle_ping.pyfalseFAILEDTraceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/sysmocom/suites/gprs/ping_idle_ping.py", line 27, in <module>
ggsn.start()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/obj/ggsn_osmo.py", line 57, in start
util.change_elf_rpath(binary, util.prepend_library_path(lib), self.run_dir.new_dir('patchelf'))
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 57, in change_elf_rpath
proc.launch_sync()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 214, in launch_sync
raise e
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 211, in launch_sync
self.wait()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 421, in wait
MainLoop.wait(self.terminated, timeout=timeout)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 108, in wait
if not self.wait_no_raise(condition, condition_args, condition_kwargs, timeout, timestep):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 98, in wait_no_raise
self.poll(may_block=True)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 86, in poll
self.deferred_handling.handle_queue()
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 33, in handle_queue
handler(*args, **kwargs)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod/osmo-gsm-tester/src/osmo_gsm_tester/testenv.py", line 142, in poll
raise log_module.Error('Process ended prematurely: %s' % proc.name())
osmo_gsm_tester.core.log.Error: osmo-stp_10.42.42.5(pid=26121): Process ended prematurely: osmo-stp_10.42.42.5(pid=26121) [trial-2373↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26121)]
----------------------------------------------
trial-2373 gprs:sysmo ping_idle_ping.py
----------------------------------------------
09:54:50.109647 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.121277 tst gprs:sysmo: Using 1 x bts (candidates: 1)
09:54:50.253664 tst ping_idle_ping.py: using LAC 27581
09:54:50.379253 tst ping_idle_ping.py: using RAC 41
09:54:50.506686 tst ping_idle_ping.py: using CellId 27581
09:54:50.629419 tst ping_idle_ping.py: using BVCI 27582
09:54:50.652253 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.663233 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.673964 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.684525 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.695033 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.705491 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.715970 tst gprs:sysmo: Using 1 x ip_address (candidates: 9)
09:54:50.726512 tst gprs:sysmo: Using 1 x modem (candidates: 4)
09:54:51.023719 tst ping_idle_ping.py:24: start network...
09:54:51.157446 run osmo-hlr_10.42.42.2: Starting osmo-hlr
09:54:51.656446 run create_hlr_db(pid=26115): Launched
09:54:51.842184 bus /gobi_6: Setting Powered False
09:54:52.862092 run create_hlr_db(pid=26115): Terminated: ok {rc=0}
09:54:53.285214 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26117): Launched
09:54:53.602549 run osmo-hlr_10.42.42.2(pid=26118): Launched
09:54:53.721186 run osmo-stp_10.42.42.5: Starting osmo-stp
09:54:54.275433 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26120): Launched
09:54:54.592239 run osmo-stp_10.42.42.5(pid=26121): Launched
09:54:54.707876 run osmo-ggsn_10.42.42.6: Starting osmo-ggsn
09:54:55.263449 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26123): Launched
09:54:55.378308 run osmo-ggsn_10.42.42.6: Setting RPATH for osmo-ggsn
09:54:55.654712 run patchelf(pid=26124): Launched
09:54:55.841489 run osmo-stp_10.42.42.5(pid=26121): ERR: Terminated: ERROR {rc=1} [trial-2373↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26121)]
09:54:55.978287 run osmo-stp_10.42.42.5(pid=26121): stdout:
| (launched: 2022-12-10_09:54:54.436753)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:56.115508 run osmo-stp_10.42.42.5(pid=26121): stderr:
| �[0;m�[38;5;43m20221210095454627 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095454627 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095454627 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095454628 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095454628 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095454628 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095454628 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:56.248311 run osmo-stp_10.42.42.5(pid=26121): stdout:
| (launched: 2022-12-10_09:54:54.436753)
| Copyright (C) 2015-2020 by Harald Welte <laforge@gnumonks.org>
| Contributions by Holger Freyther, Neels Hofmeyr, Pau Espin, Vadim Yanitskiy
| License GPLv2+: GNU GPL Version 2 or later <http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Free Software lives by contribution. If you use this, please contribute!
|
09:54:56.378920 run osmo-stp_10.42.42.5(pid=26121): stderr:
| �[0;m�[38;5;43m20221210095454627 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Creating m3ua Server (null):2905 (osmo_ss7.c:2189)
| �[0;m�[38;5;43m20221210095454627 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m Created m3ua server on (null):2905 (osmo_ss7.c:2207)
| �[0;m�[38;5;43m20221210095454627 �[1;32mDLSS7�[0;m�[38;5;43m �[1;32mINFO�[0;m�[38;5;43m (Re)binding m3ua Server to 10.42.42.5:2905 (osmo_ss7.c:2235)
| �[0;m20221210095454628 �[1;33mDLGLOBAL�[0;m �[1;33mNOTICE�[0;m unable to bind socket: 10.42.42.5:2905: Address already in use (socket.c:822)
| % Unable to bind xUA server to IP(s)
| 20221210095454628 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.5:4239: Address already in use (socket.c:935)
| 20221210095454628 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.5:4239 (socket.c:946)
| 20221210095454628 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.5 4239 (telnet_interface.c:99)
| Error binding VTY port
| : Address already in use
09:54:56.531242 run patchelf(pid=26124): Terminating (SIGINT)
09:54:56.709980 run patchelf(pid=26124): Terminated: ok {rc=0}
09:54:56.742286 tst ping_idle_ping.py:27: ERR: Error: osmo-stp_10.42.42.5(pid=26121): Process ended prematurely: osmo-stp_10.42.42.5(pid=26121) [trial-2373↪gprs:sysmo↪ping_idle_ping.py:27↪ping_idle_ping.py↪osmo-stp_10.42.42.5↪osmo-stp_10.42.42.5(pid=26121)] [trial-2373↪gprs:sysmo↪ping_idle_ping.py:27]
09:54:56.751249 tst ping_idle_ping.py:27: Test FAILED (6.6 sec)
09:54:56.814043 run osmo-hlr_10.42.42.2(pid=26118): ERR: Terminated: ERROR {rc=237} [trial-2373↪gprs:sysmo↪ping_idle_ping.py↪ping_idle_ping.py↪osmo-hlr_10.42.42.2↪osmo-hlr_10.42.42.2(pid=26118)]
09:54:56.865346 run osmo-hlr_10.42.42.2(pid=26118): stdout:
| (launched: 2022-12-10_09:54:53.446984)
09:54:56.917115 run osmo-hlr_10.42.42.2(pid=26118): stderr:
| �[0;m�[1;31m20221210095453646 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SECURE_DELETE' (db.c:579)
| �[0;m�[1;31m20221210095453646 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'SOUNDEX' (db.c:579)
| �[0;m�[1;31m20221210095453646 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'TEMP_STORE=1' (db.c:579)
| �[0;m�[1;31m20221210095453646 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'THREADSAFE=1' (db.c:579)
| �[0;m�[1;31m20221210095453647 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m SQLite3 compiled with 'USE_URI' (db.c:579)
| �[0;m�[1;31m20221210095453647 �[1;34mDDB�[0;m�[1;31m �[1;34mDEBUG�[0;m�[1;31m Not setting SQL log callback: SQLite3 compiled without support for it (db.c:596)
| �[0;m�[1;31m20221210095453659 �[1;33mDDB�[0;m�[1;31m �[1;33mNOTICE�[0;m�[1;31m Database '/home/jenkins/workspace/osmo-gsm-tester_run-prod/trial-2373/run.2022-12-10_09-48-07/gprs:sysmo/ping_idle_ping.py/osmo-hlr_10.42.42.2/hlr.db' has HLR DB schema version 6 (db.c:636)
| �[0;m20221210095453667 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m unable to bind socket:10.42.42.2:4258: Address already in use (socket.c:935)
| 20221210095453668 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m no suitable addr found for: 10.42.42.2:4258 (socket.c:946)
| 20221210095453668 �[1;31mDLGLOBAL�[0;m �[1;31mERROR�[0;m Cannot bind telnet at 10.42.42.2 4258 (telnet_interface.c:99)
09:54:56.961266 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26117): Terminating (SIGTERM)
09:54:57.002674 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26120): Terminating (SIGTERM)
09:54:57.044394 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26123): Terminating (SIGTERM)
09:54:57.054258 --- ParallelTerminationStrategy: PID 26117 died...
09:54:57.110157 run pcap-recorder_any(filters='host 10.42.42.2')(pid=26117): Terminated: ok {rc=0}
09:54:57.120071 --- ParallelTerminationStrategy: PID 26120 died...
09:54:57.175605 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=26120): Terminated: ok {rc=0}
09:54:57.185960 --- ParallelTerminationStrategy: PID 26123 died...
09:54:57.241996 run pcap-recorder_any(filters='host 10.42.42.6')(pid=26123): Terminated: ok {rc=0}
09:54:57.328903 bus /gobi_6: Setting Powered False
5gprs:sysmo00