![[Jenkins]](/jenkins/static/342c126a/images/svgs/logo.svg){kind=logo}
Failed
handover_2G:nanobts+bts1-trx-sysmocell5000.handover.py (from handover_2G_nanobts+bts1-trx-sysmocell5000)
Stacktrace
handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-164↪handover_2G:nanobts+bts1-trx-sysmocell5000↪handover.py:84]Standard Output
----------------------------------------------
trial-164 handover_2G:nanobts+bts1-trx-sysmocell5000 handover.py
----------------------------------------------
14:38:36.042193 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
14:38:36.055870 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x bts (candidates: 2)
14:38:36.197352 tst handover.py: using LAC 16386
14:38:36.335396 tst handover.py: using RAC 66
14:38:36.470501 tst handover.py: using CellId 16385
14:38:36.613628 tst handover.py: using BVCI 3064
14:38:36.642190 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x bts (candidates: 2)
14:38:36.803502 tst handover.py: using LAC 16387
14:38:36.940987 tst handover.py: using RAC 67
14:38:37.075507 tst handover.py: using CellId 16386
14:38:37.210889 tst handover.py: using BVCI 3065
14:38:37.237573 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
14:38:37.251176 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
14:38:37.264886 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
14:38:37.278486 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
14:38:37.292054 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x ip_address (candidates: 6)
14:38:37.305531 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x modem (candidates: 2)
14:38:37.434224 tst handover_2G:nanobts+bts1-trx-sysmocell5000: Using 1 x modem (candidates: 2)
14:38:37.671699 run osmo-hlr_10.42.42.2: Starting osmo-hlr
14:38:38.235292 run create_hlr_db(pid=11594): Launched
14:38:38.390864 bus /sierra_2: Setting Powered False
14:38:39.480649 run create_hlr_db(pid=11594): Terminated: ok {rc=0}
14:38:39.927602 run pcap-recorder_any(filters='host 10.42.42.2')(pid=11596): Launched
14:38:40.300440 run osmo-hlr_10.42.42.2(pid=11597): Launched
14:38:40.435168 run osmo-stp_10.42.42.5: Starting osmo-stp
14:38:41.048209 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=11598): Launched
14:38:41.421380 run osmo-stp_10.42.42.5(pid=11599): Launched
14:38:41.554502 run osmo-msc_10.42.42.6: Starting osmo-msc
14:38:42.201159 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=11600): Launched
14:38:42.574668 run osmo-msc_10.42.42.6(pid=11601): Launched
14:38:42.715115 run osmo-mgw_10.42.42.3: Starting osmo-mgw
14:38:43.345805 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=11602): Launched
14:38:43.715748 run osmo-mgw_10.42.42.3(pid=11603): Launched
14:38:43.848021 run osmo-mgw_10.42.42.4: Starting osmo-mgw
14:38:44.443632 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=11604): Launched
14:38:44.790623 run osmo-mgw_10.42.42.4(pid=11605): Launched
14:38:44.914545 run osmo-bsc_10.42.42.7: Starting osmo-bsc
14:38:45.685851 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=11606): Launched
14:38:46.045890 run osmo-bsc_10.42.42.7(pid=11607): Launched
14:38:51.691672 run pcap-recorder_any(filters='(host 10.42.42.120) and port not 22')(pid=11608): Launched
14:38:51.836238 run nanobts_NanoBTS-ONW-1900: Finding nanobts 10.42.42.120, binding on 10.42.42.1...
14:38:52.149580 run abisip-find-preconf(pid=11610): Launched
14:41:01.518743 run nanobts_NanoBTS-ONW-1900: Found nanobts 10.42.42.120 with unit_id 9 trx 0
14:41:01.658715 run abisip-find-preconf(pid=11610): Terminating (SIGINT)
14:41:02.028947 run abisip-find-preconf(pid=11610): Terminated {rc=-2}
14:41:02.365519 run ipaccess-config(pid=12004): Launched
14:41:03.580961 run ipaccess-config(pid=12004): Terminated: ok {rc=0}
14:41:55.105852 run nanobts_NanoBTS-ONW-1900: nanoBTS connected to BSC
14:41:56.569334 tst handover.py:43: ----- Report fragment: handover_2G:nanobts+bts1-trx-sysmocell5000/handover.py/01_bts0_started: pass (186.7s)
14:41:56.731019 tst handover.py: using MSISDN 13224
14:41:56.921556 bus /sierra_2: Setting Powered True
14:42:00.452783 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='13224', subscriber_id=1}
14:42:00.610648 tst handover.py: using MSISDN 13225
14:42:00.813347 bus /sierra_3: Setting Powered True
14:42:04.322872 run osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='13225', subscriber_id=2}
14:42:04.644420 bus /sierra_2: Setting Online False
14:42:05.861906 bus /sierra_2: Setting Powered False
14:42:11.878158 bus /sierra_2: Setting Powered True
14:42:13.350702 bus /sierra_2: Setting Online True
14:42:15.913220 tst /sierra_2: Connect to ('901', '70')
14:42:16.278496 bus /sierra_3: Setting Online False
14:42:17.500451 bus /sierra_3: Setting Powered False
14:42:23.876300 bus /sierra_3: Setting Powered True
14:42:25.253949 bus /sierra_3: Setting Online True
14:42:27.836286 tst /sierra_3: Connect to ('901', '70')
14:42:28.031237 tst /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053'
14:42:28.164826 tst /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028'
14:42:28.378978 tst handover.py:54: waiting for modems to attach...
14:42:30.483707 tst /sierra_2: Already registered with network ('901', '70')
14:42:34.793989 tst handover.py:57: ----- Report fragment: handover_2G:nanobts+bts1-trx-sysmocell5000/handover.py/02.1_ms0_attach: pass (6.2s)
14:42:43.409733 tst /sierra_3: Already registered with network ('901', '70')
14:42:47.116668 tst handover.py:60: ----- Report fragment: handover_2G:nanobts+bts1-trx-sysmocell5000/handover.py/02.2_ms1_attach: pass (12.1s)
14:42:48.660098 tst handover.py:63: ----- Report fragment: handover_2G:nanobts+bts1-trx-sysmocell5000/handover.py/02.3_subscribed_in_msc: pass (1.3s)
14:42:53.254442 tst handover.py:68: dial success
14:42:55.984752 tst handover.py:75: answer success, call established and ongoing
14:42:56.315974 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
14:42:56.552799 tst handover.py:79: ----- Report fragment: handover_2G:nanobts+bts1-trx-sysmocell5000/handover.py/03_call_established: pass (3.0s)
14:42:56.711428 run osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
14:42:59.101689 run ssh_sc5_ccli(pid=12007): Launched
14:43:18.350349 run ssh_sc5_ccli(pid=12007): ERR: Terminated: ERROR {rc=255} [trial-164↪handover_2G:nanobts+bts1-trx-sysmocell5000↪handover.py:83↪handover.py↪osmo-bts-trx↪sc5-trx↪ssh_sc5_ccli(pid=12007)]
14:43:18.505697 run ssh_sc5_ccli(pid=12007): stdout:
| (launched: 2020-12-07_14:42:58.914893)
14:43:18.657577 run ssh_sc5_ccli(pid=12007): stderr:
| (launched: 2020-12-07_14:42:58.978879)
| ssh: connect to host 10.42.42.112 port 22: No route to host
14:43:18.787443 run sc5-trx: Configuring SC5 TRX failed, retrying 9 more times
14:43:24.174942 run ssh_sc5_ccli(pid=12008): Launched
14:43:27.408230 run ssh_sc5_ccli(pid=12008): Terminated: ok {rc=0}
14:43:27.548969 run osmo-bts-trx: Waiting for sc5-trx to start up...
14:43:28.880936 run osmo-bts-trx(pid=12009): Launched
14:48:29.639532 tst handover.py:84: ----- Report fragment: handover_2G:nanobts+bts1-trx-sysmocell5000/handover.py/04.1_bts1_started: FAIL (333.0s)
14:48:29.647734 tst handover.py:84: ERR: Error: handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-164↪handover_2G:nanobts+bts1-trx-sysmocell5000↪handover.py:84] [trial-164↪handover_2G:nanobts+bts1-trx-sysmocell5000↪handover.py:84]
14:48:29.656110 tst handover.py:84: Test FAILED (593.6 sec)
14:48:29.701549 run pcap-recorder_any(filters='host 10.42.42.2')(pid=11596): Terminating (SIGTERM)
14:48:29.753998 run osmo-hlr_10.42.42.2(pid=11597): Terminating (SIGTERM)
14:48:29.810049 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=11598): Terminating (SIGTERM)
14:48:29.866418 run osmo-stp_10.42.42.5(pid=11599): Terminating (SIGTERM)
14:48:29.923372 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=11600): Terminating (SIGTERM)
14:48:29.977470 run osmo-msc_10.42.42.6(pid=11601): Terminating (SIGTERM)
14:48:30.031220 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=11602): Terminating (SIGTERM)
14:48:30.083792 run osmo-mgw_10.42.42.3(pid=11603): Terminating (SIGTERM)
14:48:30.137199 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=11604): Terminating (SIGTERM)
14:48:30.192980 run osmo-mgw_10.42.42.4(pid=11605): Terminating (SIGTERM)
14:48:30.247807 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=11606): Terminating (SIGTERM)
14:48:30.301371 run osmo-bsc_10.42.42.7(pid=11607): Terminating (SIGTERM)
14:48:30.353830 run pcap-recorder_any(filters='(host 10.42.42.120) and port not 22')(pid=11608): Terminating (SIGTERM)
14:48:30.406305 run osmo-bts-trx(pid=12009): Terminating (SIGTERM)
14:48:30.419602 --- ParallelTerminationStrategy: PID 11596 died...
14:48:30.488419 run pcap-recorder_any(filters='host 10.42.42.2')(pid=11596): Terminated: ok {rc=0}
14:48:30.501101 --- ParallelTerminationStrategy: PID 11597 died...
14:48:30.569544 run osmo-hlr_10.42.42.2(pid=11597): Terminated: ok {rc=0}
14:48:30.582050 --- ParallelTerminationStrategy: PID 11598 died...
14:48:30.650768 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=11598): Terminated: ok {rc=0}
14:48:30.663323 --- ParallelTerminationStrategy: PID 11599 died...
14:48:30.732112 run osmo-stp_10.42.42.5(pid=11599): Terminated: ok {rc=0}
14:48:30.744786 --- ParallelTerminationStrategy: PID 11600 died...
14:48:30.813823 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=11600): Terminated: ok {rc=0}
14:48:30.826258 --- ParallelTerminationStrategy: PID 11602 died...
14:48:30.895445 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=11602): Terminated: ok {rc=0}
14:48:30.907944 --- ParallelTerminationStrategy: PID 11603 died...
14:48:30.978115 run osmo-mgw_10.42.42.3(pid=11603): Terminated {rc=15}
14:48:30.990481 --- ParallelTerminationStrategy: PID 11604 died...
14:48:31.059289 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=11604): Terminated: ok {rc=0}
14:48:31.071717 --- ParallelTerminationStrategy: PID 11605 died...
14:48:31.140299 run osmo-mgw_10.42.42.4(pid=11605): Terminated {rc=15}
14:48:31.153120 --- ParallelTerminationStrategy: PID 11606 died...
14:48:31.221681 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=11606): Terminated: ok {rc=0}
14:48:31.234292 --- ParallelTerminationStrategy: PID 11608 died...
14:48:31.302688 run pcap-recorder_any(filters='(host 10.42.42.120) and port not 22')(pid=11608): Terminated: ok {rc=0}
14:48:33.331171 --- ParallelTerminationStrategy: PID 11601 died...
14:48:33.400228 run osmo-msc_10.42.42.6(pid=11601): Terminated: ok {rc=0}
14:48:34.079506 --- ParallelTerminationStrategy: PID 11607 died...
14:48:34.146927 run osmo-bsc_10.42.42.7(pid=11607): Terminated: ok {rc=0}
14:48:34.158859 --- ParallelTerminationStrategy: PID 12009 died...
14:48:34.224441 run osmo-bts-trx(pid=12009): Terminated: ok {rc=0}
14:48:34.376137 bus /sierra_3: Setting Online False
14:48:35.532612 bus /sierra_3: Setting Powered False
14:48:59.635981 bus /sierra_3: Setting Powered False
14:49:00.763603 bus /sierra_3: Setting Powered False
14:49:01.948033 bus /sierra_2: Setting Online False
14:49:03.081086 bus /sierra_2: Setting Powered False
14:49:08.644809 bus /sierra_2: Setting Powered False
Standard Error
Traceback (most recent call last):
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
self.path)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
spec.loader.exec_module( importlib.util.module_from_spec(spec) )
File "<frozen importlib._bootstrap_external>", line 673, in exec_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/sysmocom/suites/handover_2G/handover.py", line 84, in <module>
wait(bsc.bts_is_connected, bts1)
File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/event_loop.py", line 109, in wait
raise log.Error('Wait timeout', condition=condition, timeout=timeout, timestep=timestep)
osmo_gsm_tester.core.log.Error: handover.py: Wait timeout {condition=<bound method OsmoBsc.bts_is_connected of osmo-bsc_10.42.42.7>, timeout=300, timestep=1} [trial-164↪handover_2G:nanobts+bts1-trx-sysmocell5000↪handover.py:84]