Regression
ttcn3_bts_tests:trx.ttcn3_bts_tests.py (from ttcn3_bts_tests_trx)
Failing for the past 1 build
(Since Failed )
Stacktrace
ttcn3(pid=12129): Wait timeout {condition=<bound method Process.terminated of ttcn3(pid=12129)>, timeout=5400, timestep=1} [trial-2754↪ttcn3_bts_tests:trx↪ttcn3_bts_tests.py:53↪ttcn3(pid=12129)]
Standard Output
---------------------------------------------- trial-2754 ttcn3_bts_tests:trx ttcn3_bts_tests.py ---------------------------------------------- 00:21:12.514523 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:12.528654 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:12.542722 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:12.557440 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:12.571748 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:12.586357 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:12.600452 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:12.614728 tst ttcn3_bts_tests:trx: Using 1 x bts (candidates: 1) 00:21:13.178554 tst ttcn3_bts_tests.py: using LAC 9849 00:21:13.310219 tst ttcn3_bts_tests.py: using RAC 159 00:21:13.438678 tst ttcn3_bts_tests.py: using CellId 9848 00:21:13.567747 tst ttcn3_bts_tests.py: using BVCI 2131 00:21:13.593477 tst ttcn3_bts_tests:trx: Using 1 x osmocon_phone (candidates: 1) 00:21:13.782487 tst ttcn3_bts_tests.py:35: Starting CNI 00:21:13.921001 run osmo-hlr_10.42.42.2: Starting osmo-hlr 00:21:15.713521 run create_hlr_db(pid=12099): Launched 00:21:16.902889 run create_hlr_db(pid=12099): Terminated: ok {rc=0} 00:21:17.233386 run pcap-recorder_any(filters='host 10.42.42.2')(pid=12101): Launched 00:21:17.545926 run osmo-hlr_10.42.42.2(pid=12102): Launched 00:21:17.658608 run osmo-stp_10.42.42.4: Starting osmo-stp 00:21:20.384930 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=12103): Launched 00:21:20.698559 run osmo-stp_10.42.42.4(pid=12104): Launched 00:21:20.811053 run osmo-msc_10.42.42.5: Starting osmo-msc 00:21:25.782145 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=12105): Launched 00:21:26.095079 run osmo-msc_10.42.42.5(pid=12106): Launched 00:21:26.209097 run osmo-mgw_10.42.42.3: Starting osmo-mgw 00:21:28.030350 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=12107): Launched 00:21:28.344375 run osmo-mgw_10.42.42.3(pid=12108): Launched 00:21:28.368694 tst ttcn3_bts_tests:trx: Using 1 x ip_address (candidates: 8) 00:21:28.471127 run osmo-bsc_10.42.42.8: Starting osmo-bsc 00:21:32.336479 run pcap-recorder_any(filters='host 10.42.42.8 or host 10.42.42.9 and port not 22')(pid=12109): Launched 00:21:32.670278 run osmo-bsc_10.42.42.8(pid=12110): Launched 00:21:32.794658 run osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.8 00:21:33.258270 run osmo-trx-uhd: OSMOTRX CONF: {'osmo_trx': {'trx_ip': '10.42.42.116', 'bts_ip': '10.42.42.50', 'channels': [{}], 'egprs': 'disable', 'type': 'uhd', 'dev_args': 'type=b200,serial=306BD11', 'clock_reference': 'external', 'remote_user': 'jenkins', 'multi_arfcn': 'True', 'launch_trx': 'True'}} 00:21:33.847761 run uname-m(pid=12111): Launched 00:21:35.066873 run uname-m(pid=12111): Terminated: ok {rc=0} 00:21:36.410367 run rm-remote-dir(pid=12112): Launched 00:21:37.616595 run rm-remote-dir(pid=12112): Terminated: ok {rc=0} 00:21:37.979990 run mk-remote-dir(pid=12113): Launched 00:21:39.184110 run mk-remote-dir(pid=12113): Terminated: ok {rc=0} 00:21:39.531299 run mk-remote-dir(pid=12114): Launched 00:21:40.733269 run mk-remote-dir(pid=12114): Terminated: ok {rc=0} 00:21:41.038350 run scp-inst-to-remote(pid=12115): Launched 00:21:43.253372 run scp-inst-to-remote(pid=12115): Terminated: ok {rc=0} 00:21:43.615106 run mk-remote-dir(pid=12117): Launched 00:21:44.827231 run mk-remote-dir(pid=12117): Terminated: ok {rc=0} 00:21:45.145330 run scp-cfg-to-remote(pid=12118): Launched 00:21:46.359562 run scp-cfg-to-remote(pid=12118): Terminated: ok {rc=0} 00:21:46.734669 run mk-remote-dir(pid=12120): Launched 00:21:47.952587 run mk-remote-dir(pid=12120): Terminated: ok {rc=0} 00:21:48.266391 run scp-wrapper-to-remote(pid=12121): Launched 00:21:49.481421 run scp-wrapper-to-remote(pid=12121): Terminated: ok {rc=0} 00:21:49.835654 run osmo-trx-uhd(pid=12123): Launched 00:21:49.958272 run osmo-bts-trx: Waiting for osmo-trx-uhd to start up... 00:21:54.105500 run osmo-bts-trx(pid=12124): Launched 00:21:54.310132 tst ttcn3_bts_tests.py:49: Starting osmocon 00:21:54.451897 run osmocon_ttyUSB4: Resetting the phone 00:21:54.576296 run osmocon_ttyUSB4: Starting osmocon 00:21:56.232308 run osmocon_ttyUSB4(pid=12125): Launched 00:21:56.363513 run osmocon_ttyUSB4: Waiting for osmocon to be up and running 00:21:57.564012 run osmo-bts-trx(pid=12124): Terminated: ok {rc=0} 00:21:57.932779 run osmo-bts-trx(pid=12127): Launched 00:21:58.174834 run osmo-trx-uhd(pid=12123): Terminated: ok {rc=0} 00:21:58.546715 run osmo-trx-uhd(pid=12128): Launched 00:21:58.761961 tst ttcn3_bts_tests.py:53: Creating template 00:21:59.011135 tst ttcn3_bts_tests.py:53: Starting TTCN3 test suite 00:21:59.320554 run ttcn3(pid=12129): Launched 00:21:59.534466 tst ttcn3_bts_tests.py:53: TTCN3 test suite launched, waiting until it finishes 00:22:01.829802 run osmo-bts-trx(pid=12127): Terminated: ok {rc=0} 00:22:02.409272 run osmo-bts-trx(pid=12530): Launched 00:22:05.757987 run osmo-bts-trx(pid=12530): Terminated: ok {rc=0} 00:22:06.123412 run osmo-bts-trx(pid=12544): Launched 00:22:06.367956 run osmo-trx-uhd(pid=12128): Terminated: ok {rc=0} 00:22:06.721657 run osmo-trx-uhd(pid=12545): Launched 00:22:33.965710 run osmo-bts-trx(pid=12544): Terminated: ok {rc=0} 00:22:34.342247 run osmo-bts-trx(pid=12579): Launched 00:22:47.992043 run osmo-bts-trx(pid=12579): Terminated: ok {rc=0} 00:22:48.367437 run osmo-bts-trx(pid=12609): Launched 00:23:02.008736 run osmo-bts-trx(pid=12609): Terminated: ok {rc=0} 00:23:02.374444 run osmo-bts-trx(pid=12641): Launched 00:23:18.035588 run osmo-bts-trx(pid=12641): Terminated: ok {rc=0} 00:23:18.400090 run osmo-bts-trx(pid=12685): Launched 00:24:07.134035 run osmo-bts-trx(pid=12685): Terminated: ok {rc=0} 00:24:07.506737 run osmo-bts-trx(pid=12723): Launched 00:24:32.173794 run osmo-bts-trx(pid=12723): Terminated: ok {rc=0} 00:24:32.546183 run osmo-bts-trx(pid=12761): Launched 00:25:03.218775 run osmo-bts-trx(pid=12761): Terminated: ok {rc=0} 00:25:03.584152 run osmo-bts-trx(pid=12802): Launched 00:25:25.255820 run osmo-bts-trx(pid=12802): Terminated: ok {rc=0} 00:25:25.610931 run osmo-bts-trx(pid=12835): Launched 00:26:18.355640 run osmo-bts-trx(pid=12835): Terminated: ok {rc=0} 00:26:18.728879 run osmo-bts-trx(pid=12871): Launched 00:26:48.395021 run osmo-bts-trx(pid=12871): Terminated: ok {rc=0} 00:26:48.749982 run osmo-bts-trx(pid=12909): Launched 00:27:03.429260 run osmo-bts-trx(pid=12909): Terminated: ok {rc=0} 00:27:03.809438 run osmo-bts-trx(pid=12941): Launched 00:27:18.445598 run osmo-bts-trx(pid=12941): Terminated: ok {rc=0} 00:27:18.810422 run osmo-bts-trx(pid=12974): Launched 00:28:30.572996 run osmo-bts-trx(pid=12974): Terminated: ok {rc=0} 00:28:30.947086 run osmo-bts-trx(pid=13009): Launched 01:52:00.089849 run ttcn3(pid=12129): Terminating (SIGINT) 01:52:01.719193 run ttcn3(pid=12129): Terminated {rc=130} 01:52:01.754850 tst ttcn3_bts_tests.py:53: ERR: Error: ttcn3(pid=12129): Wait timeout {condition=<bound method Process.terminated of ttcn3(pid=12129)>, timeout=5400, timestep=1} [trial-2754↪ttcn3_bts_tests:trx↪ttcn3_bts_tests.py:53↪ttcn3(pid=12129)] [trial-2754↪ttcn3_bts_tests:trx↪ttcn3_bts_tests.py:53] 01:52:01.766653 tst ttcn3_bts_tests.py:53: Test FAILED (5449.3 sec) 01:52:01.814521 run pcap-recorder_any(filters='host 10.42.42.2')(pid=12101): Terminating (SIGTERM) 01:52:01.869407 run osmo-hlr_10.42.42.2(pid=12102): Terminating (SIGTERM) 01:52:01.925880 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=12103): Terminating (SIGTERM) 01:52:01.984202 run osmo-stp_10.42.42.4(pid=12104): Terminating (SIGTERM) 01:52:02.042110 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=12105): Terminating (SIGTERM) 01:52:02.100656 run osmo-msc_10.42.42.5(pid=12106): Terminating (SIGTERM) 01:52:02.158205 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=12107): Terminating (SIGTERM) 01:52:02.212178 run osmo-mgw_10.42.42.3(pid=12108): Terminating (SIGTERM) 01:52:02.266675 run pcap-recorder_any(filters='host 10.42.42.8 or host 10.42.42.9 and port not 22')(pid=12109): Terminating (SIGTERM) 01:52:02.320548 run osmo-bsc_10.42.42.8(pid=12110): Terminating (SIGTERM) 01:52:02.374434 run osmo-trx-uhd(pid=12545): Terminating (SIGTERM) 01:52:02.426917 run osmo-bts-trx(pid=13009): Terminating (SIGTERM) 01:52:02.477935 run osmocon_ttyUSB4(pid=12125): Terminating (SIGTERM) 01:52:02.489564 --- ParallelTerminationStrategy: PID 12101 died... 01:52:02.558256 run pcap-recorder_any(filters='host 10.42.42.2')(pid=12101): Terminated: ok {rc=0} 01:52:02.570951 --- ParallelTerminationStrategy: PID 12102 died... 01:52:02.639998 run osmo-hlr_10.42.42.2(pid=12102): Terminated: ok {rc=0} 01:52:02.652005 --- ParallelTerminationStrategy: PID 12103 died... 01:52:02.719633 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=12103): Terminated: ok {rc=0} 01:52:02.731714 --- ParallelTerminationStrategy: PID 12104 died... 01:52:02.798732 run osmo-stp_10.42.42.4(pid=12104): Terminated: ok {rc=0} 01:52:02.810811 --- ParallelTerminationStrategy: PID 12105 died... 01:52:02.878232 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=12105): Terminated: ok {rc=0} 01:52:02.890321 --- ParallelTerminationStrategy: PID 12107 died... 01:52:02.962880 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=12107): Terminated: ok {rc=0} 01:52:02.974965 --- ParallelTerminationStrategy: PID 12108 died... 01:52:03.042310 run osmo-mgw_10.42.42.3(pid=12108): Terminated {rc=15} 01:52:03.054446 --- ParallelTerminationStrategy: PID 12109 died... 01:52:03.122109 run pcap-recorder_any(filters='host 10.42.42.8 or host 10.42.42.9 and port not 22')(pid=12109): Terminated: ok {rc=0} 01:52:03.134322 --- ParallelTerminationStrategy: PID 12125 died... 01:52:03.202100 run osmocon_ttyUSB4(pid=12125): Terminated {rc=15} 01:52:03.214305 --- ParallelTerminationStrategy: PID 12545 died... 01:52:03.282467 run osmo-trx-uhd(pid=12545): Terminated {rc=36608} 01:52:05.311485 --- ParallelTerminationStrategy: PID 12106 died... 01:52:05.380859 run osmo-msc_10.42.42.5(pid=12106): Terminated: ok {rc=0} 01:52:06.059940 --- ParallelTerminationStrategy: PID 12110 died... 01:52:06.128275 run osmo-bsc_10.42.42.8(pid=12110): Terminated: ok {rc=0} 01:52:09.321024 run osmo-bts-trx(pid=13009): Terminating (SIGINT) 01:52:09.500593 --- ParallelTerminationStrategy: PID 13009 died... 01:52:09.568969 run osmo-bts-trx(pid=13009): Terminated: ok {rc=0}
Standard Error
Traceback (most recent call last): File "/home/jenkins/workspace/osmo-gsm-tester_ttcn3/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run self.path) File "/home/jenkins/workspace/osmo-gsm-tester_ttcn3/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file spec.loader.exec_module( importlib.util.module_from_spec(spec) ) File "<frozen importlib._bootstrap_external>", line 673, in exec_module File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed File "/home/jenkins/workspace/osmo-gsm-tester_ttcn3/osmo-gsm-tester/sysmocom/ttcn3/suites/ttcn3_bts_tests/ttcn3_bts_tests.py", line 53, in <module> run_ttcn3(tenv, testdir, bts, osmocon, nat_rsl_ip, ttcn3_test_groups) File "/home/jenkins/workspace/osmo-gsm-tester_ttcn3/osmo-gsm-tester/sysmocom/ttcn3/suites/ttcn3_bts_tests/lib/testlib.py", line 42, in run_ttcn3 raise e File "/home/jenkins/workspace/osmo-gsm-tester_ttcn3/osmo-gsm-tester/sysmocom/ttcn3/suites/ttcn3_bts_tests/lib/testlib.py", line 39, in run_ttcn3 proc.wait(timeout=5400) # 1.5h File "/home/jenkins/workspace/osmo-gsm-tester_ttcn3/osmo-gsm-tester/src/osmo_gsm_tester/core/process.py", line 367, in wait MainLoop.wait(self.terminated, timeout=timeout) File "/home/jenkins/workspace/osmo-gsm-tester_ttcn3/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep) osmo_gsm_tester.core.log.Error: ttcn3(pid=12129): Wait timeout {condition=<bound method Process.terminated of ttcn3(pid=12129)>, timeout=5400, timestep=1} [trial-2754↪ttcn3_bts_tests:trx↪ttcn3_bts_tests.py:53↪ttcn3(pid=12129)]