Skip to content

Failed

handover_2G:oc2g+bts1-trx-lms.handover.py (from handover_2G_oc2g+bts1-trx-lms)

Failing for the past 24 builds (Since #140 )
Took 2 min 55 sec.

Stacktrace

handover.py:146: not_expected=('0-0-2-0 TCH/F ESTABLISHED',),

Standard Output

----------------------------------------------
trial-163 handover_2G:oc2g+bts1-trx-lms handover.py
----------------------------------------------
12:31:40.735576 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x ip_address (candidates: 6)
12:31:40.749022 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x bts (candidates: 2)
12:31:40.897828 tst                      handover.py: using LAC 16323
12:31:41.039939 tst                      handover.py: using RAC 3
12:31:41.182167 tst                      handover.py: using CellId 16322
12:31:41.323912 tst                      handover.py: using BVCI 3001
12:31:41.352258 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x bts (candidates: 2)
12:31:41.517262 tst                      handover.py: using LAC 16324
12:31:41.658723 tst                      handover.py: using RAC 4
12:31:41.803096 tst                      handover.py: using CellId 16323
12:31:41.947518 tst                      handover.py: using BVCI 3002
12:31:41.975544 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x ip_address (candidates: 6)
12:31:41.989636 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x ip_address (candidates: 6)
12:31:42.004222 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x ip_address (candidates: 6)
12:31:42.018550 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x ip_address (candidates: 6)
12:31:42.033050 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x ip_address (candidates: 6)
12:31:42.047077 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x modem (candidates: 2)
12:31:42.181648 tst    handover_2G:oc2g+bts1-trx-lms: Using 1 x modem (candidates: 2)
12:31:42.753356 run              osmo-hlr_10.42.42.2: Starting osmo-hlr
12:31:43.297167 run          create_hlr_db(pid=7948): Launched
12:31:43.449109 bus                        /sierra_2: Setting Powered False
12:31:44.530269 run          create_hlr_db(pid=7948): Terminated: ok {rc=0}
12:31:44.970846 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7950): Launched
12:31:45.328032 run    osmo-hlr_10.42.42.2(pid=7951): Launched
12:31:45.451070 run              osmo-stp_10.42.42.5: Starting osmo-stp
12:31:46.030724 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7952): Launched
12:31:46.389150 run    osmo-stp_10.42.42.5(pid=7953): Launched
12:31:46.510932 run              osmo-msc_10.42.42.6: Starting osmo-msc
12:31:47.113231 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7954): Launched
12:31:47.464971 run    osmo-msc_10.42.42.6(pid=7955): Launched
12:31:47.605077 run              osmo-mgw_10.42.42.3: Starting osmo-mgw
12:31:48.221630 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7956): Launched
12:31:48.565095 run    osmo-mgw_10.42.42.3(pid=7957): Launched
12:31:48.688093 run              osmo-mgw_10.42.42.4: Starting osmo-mgw
12:31:49.254638 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7958): Launched
12:31:49.612179 run    osmo-mgw_10.42.42.4(pid=7959): Launched
12:31:49.734521 run              osmo-bsc_10.42.42.7: Starting osmo-bsc
12:31:50.504860 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7960): Launched
12:31:50.862761 run    osmo-bsc_10.42.42.7(pid=7961): Launched
12:31:54.570111 run                    osmo-bts-oc2g: Starting OsmoBtsOC2G to connect to osmo-bsc_10.42.42.7
12:31:55.170123 run          rm-remote-dir(pid=7962): Launched
12:31:56.396503 run          rm-remote-dir(pid=7962): Terminated: ok {rc=0}
12:31:56.789078 run          mk-remote-dir(pid=7963): Launched
12:31:58.015023 run          mk-remote-dir(pid=7963): Terminated: ok {rc=0}
12:31:58.344760 run     scp-inst-to-remote(pid=7964): Launched
12:32:04.574827 run     scp-inst-to-remote(pid=7964): Terminated: ok {rc=0}
12:32:04.954230 run          mk-remote-dir(pid=7966): Launched
12:32:06.165434 run          mk-remote-dir(pid=7966): Terminated: ok {rc=0}
12:32:06.488753 run      scp-cfg-to-remote(pid=7967): Launched
12:32:07.711286 run      scp-cfg-to-remote(pid=7967): Terminated: ok {rc=0}
12:32:08.074312 run          osmo-bts-oc2g(pid=7969): Launched
12:32:14.539710 tst                   handover.py:43: ----- Report fragment: handover_2G:oc2g+bts1-trx-lms/handover.py/01_bts0_started: pass (19.9s)
12:32:14.715867 tst                      handover.py: using MSISDN 13178
12:32:14.917349 bus                        /sierra_2: Setting Powered True
12:32:18.445607 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015252', msisdn='13178', subscriber_id=1}
12:32:18.603933 tst                      handover.py: using MSISDN 13179
12:32:20.935684 run              osmo-hlr_10.42.42.2: Add subscriber {algo=1, algo_str='comp128v1', imsi='901700000015253', msisdn='13179', subscriber_id=2}
12:32:21.260301 bus                        /sierra_2: Setting Online False
12:32:22.472458 bus                        /sierra_2: Setting Powered False
12:32:28.879203 bus                        /sierra_2: Setting Powered True
12:32:30.250790 bus                        /sierra_2: Setting Online True
12:32:32.926864 tst                        /sierra_2: Connect to ('901', '70')
12:32:33.330220 bus                        /sierra_3: Setting Online False
12:32:34.582043 bus                        /sierra_3: Setting Powered False
12:32:40.850631 bus                        /sierra_3: Setting Powered True
12:32:45.254223 bus                        /sierra_3: Setting Online True
12:32:47.111740 tst                        /sierra_2: Already registered with network ('901', '70')
12:32:48.028101 tst                        /sierra_3: Connect to ('901', '70')
12:32:48.217918 tst                        /sierra_2: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054228053'
12:32:48.350174 tst                        /sierra_3: Manufacturer: 'Sierra Wireless, Incorporated', Model: 'MC7304', Revision: 'SWI9X15C_06.03.32.04 r28324 CNSHZ-AR-BUILD 2015/05/25 01:09:15', Serial: '356853054229028'
12:32:48.559632 tst                   handover.py:54: waiting for modems to attach...
12:32:51.866992 tst                   handover.py:57: ----- Report fragment: handover_2G:oc2g+bts1-trx-lms/handover.py/02.1_ms0_attach: pass (3.1s)
12:33:03.490624 tst                        /sierra_3: Already registered with network ('901', '70')
12:33:07.863425 tst                   handover.py:60: ----- Report fragment: handover_2G:oc2g+bts1-trx-lms/handover.py/02.2_ms1_attach: pass (15.8s)
12:33:09.407707 tst                   handover.py:63: ----- Report fragment: handover_2G:oc2g+bts1-trx-lms/handover.py/02.3_subscribed_in_msc: pass (1.3s)
12:33:14.017204 tst                   handover.py:68: dial success
12:33:16.723617 tst                   handover.py:75: answer success, call established and ongoing
12:33:17.023884 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
12:33:17.242560 tst                   handover.py:79: ----- Report fragment: handover_2G:oc2g+bts1-trx-lms/handover.py/03_call_established: pass (3.0s)
12:33:17.390023 run                     osmo-bts-trx: Starting to connect to osmo-bsc_10.42.42.7
12:33:17.864908 run                     osmo-trx-lms: OSMOTRX CONF: {'osmo_trx': {'channels': [{'rx_path': 'LNAW'}], 'type': 'lms', 'bts_ip': '10.42.42.53', 'clock_reference': 'external', 'remote_user': 'jenkins', 'dev_args': 'LimeSDR-USB,serial=00090706024F0A22', 'trx_ip': '10.42.42.117', 'launch_trx': 'True', 'egprs': 'disable'}}
12:33:18.494465 run                uname-m(pid=7970): Launched
12:33:19.733387 run                uname-m(pid=7970): Terminated: ok {rc=0}
12:33:20.144029 run          rm-remote-dir(pid=7971): Launched
12:33:21.366991 run          rm-remote-dir(pid=7971): Terminated: ok {rc=0}
12:33:21.775216 run          mk-remote-dir(pid=7972): Launched
12:33:23.016849 run          mk-remote-dir(pid=7972): Terminated: ok {rc=0}
12:33:23.427461 run          mk-remote-dir(pid=7973): Launched
12:33:24.647617 run          mk-remote-dir(pid=7973): Terminated: ok {rc=0}
12:33:24.975141 run     scp-inst-to-remote(pid=7974): Launched
12:33:27.196915 run     scp-inst-to-remote(pid=7974): Terminated: ok {rc=0}
12:33:27.595590 run          mk-remote-dir(pid=7976): Launched
12:33:28.835739 run          mk-remote-dir(pid=7976): Terminated: ok {rc=0}
12:33:29.184767 run      scp-cfg-to-remote(pid=7977): Launched
12:33:30.406389 run      scp-cfg-to-remote(pid=7977): Terminated: ok {rc=0}
12:33:30.812064 run          mk-remote-dir(pid=7979): Launched
12:33:32.052012 run          mk-remote-dir(pid=7979): Terminated: ok {rc=0}
12:33:32.405728 run  scp-wrapper-to-remote(pid=7980): Launched
12:33:33.628826 run  scp-wrapper-to-remote(pid=7980): Terminated: ok {rc=0}
12:33:33.994653 run           osmo-trx-lms(pid=7982): Launched
12:33:34.119023 run                     osmo-bts-trx: Waiting for osmo-trx-lms to start up...
12:33:35.455991 run           osmo-bts-trx(pid=7983): Launched
12:33:38.012284 tst                   handover.py:84: ----- Report fragment: handover_2G:oc2g+bts1-trx-lms/handover.py/04.1_bts1_started: pass (20.5s)
12:33:38.294006 tst                   handover.py:86: wait a bit for modems to see bts1
12:33:48.577305 tst                   handover.py:89: starting the second BTS probably took a while, check on the ongoing call
12:33:48.935103 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
12:33:49.370729 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
12:33:50.825133 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 1 requirements): ['not: 1-0-2-0']
12:33:53.536748 tst                  handover.py:117: trigger handover: bts 0 trx 0 timeslot 2 sub-slot 0 handover 1
12:33:53.916426 tst                  handover.py:120: wait for handover counters to change...
12:33:57.707292 tst                  handover.py:122: 
| rate_ctr.abs.bsc.0.handover:failed = 1
| rate_ctr.abs.bts.0.handover:failed = 1
12:33:57.978993 tst                  handover.py:124: 
�[1;35m20201207123353645 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: Allocated (fsm.c:461)�[0;m
�[0;m�[1;35m20201207123353645 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: is child of SUBSCR_CONN(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000016ea0] (fsm.c:491)�[0;m
�[0;m�[1;35m20201207123353645 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BSC) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123353645 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BTS) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123353645 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: No explicit neighbors, regarding all local cells as neighbors (handover_logic.c:188)�[0;m
�[0;m�[1;35m20201207123353646 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123353646 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123353646 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Selected lchan (bts=1,trx=0,ts=2,ss=0) (handover_fsm.c:395)�[0;m
�[0;m�[1;35m20201207123353646 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{NOT_STARTED}: state_chg to WAIT_LCHAN_ACTIVE (handover_fsm.c:397)�[0;m
�[0;m�[1;35m20201207123353651 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_LCHAN_ACTIVE}: Received Event HO_EV_LCHAN_ACTIVE (lchan_fsm.c:818)�[0;m
�[0;m�[1;35m20201207123353651 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_LCHAN_ACTIVE}: state_chg to WAIT_RR_HO_DETECT (handover_fsm.c:993)�[0;m
�[0;m�[1;35m20201207123354070 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: Received Event HO_EV_RR_HO_FAIL (gsm_04_08_rr.c:946)�[0;m
�[0;m�[1;35m20201207123354070 �[1;31mDRSL�[0;m�[1;35m �[1;31mERROR�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Handover failed in state HANDOVER, Failure (MS sent RR Handover Failure): Received RR Handover Fail message (handover_fsm.c:1155)�[0;m
�[0;m�[1;35m20201207123354070 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:932)�[0;m
�[0;m�[1;35m20201207123354070 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:933)�[0;m
�[0;m�[1;35m20201207123354070 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:934)�[0;m
�[0;m�[1;35m20201207123354070 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:935)�[0;m
�[0;m�[1;35m20201207123354071 �[1;32mDRSL�[0;m�[1;35m �[1;32mINFO�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Result: Failure (MS sent RR Handover Failure) (handover_fsm.c:937)�[0;m
�[1;35m20201207123354072 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: Terminating (cause = OSMO_FSM_TERM_REGULAR) (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123354072 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: Freeing instance (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123354072 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ea0]{WAIT_RR_HO_DETECT}: Deallocated (fsm.c:573)�[0;m
12:33:58.328407 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 3 requirements): ['0-0-3-0 TCH/F ESTABLISHED']
12:33:58.465376 bus Vty(host='10.42.42.7', port=4242): ERR: Found unexpected lchan activity (2 of 3 requirements): ['missing: 1-0-2-0 TCH/F ESTABLISHED', 'unexpected: 0-0-2-0 TCH/F ESTABLISHED']  [trial-163↪handover_2G:oc2g+bts1-trx-lms↪handover.py:126↪handover.py↪osmo-bsc_10.42.42.7↪Vty(host='10.42.42.7', port=4242)]
12:33:58.696870 tst                  handover.py:129: did not work, try again... (attempts left: 4)
12:33:59.026636 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
12:34:00.363802 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 1 requirements): ['not: 1-0-2-0']
12:34:02.806484 tst                  handover.py:117: trigger handover: bts 0 trx 0 timeslot 2 sub-slot 0 handover 1
12:34:03.179678 tst                  handover.py:120: wait for handover counters to change...
12:34:06.894158 tst                  handover.py:122: 
| rate_ctr.abs.bsc.0.handover:failed = 1
| rate_ctr.abs.bts.0.handover:failed = 1
12:34:07.160213 tst                  handover.py:124: 
�[1;35m20201207123402914 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: Allocated (fsm.c:461)�[0;m
�[0;m�[1;35m20201207123402914 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: is child of SUBSCR_CONN(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000016ea0] (fsm.c:491)�[0;m
�[0;m�[1;35m20201207123402914 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BSC) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123402914 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BTS) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123402914 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: No explicit neighbors, regarding all local cells as neighbors (handover_logic.c:188)�[0;m
�[0;m�[1;35m20201207123402915 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123402915 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123402916 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Selected lchan (bts=1,trx=0,ts=2,ss=0) (handover_fsm.c:395)�[0;m
�[0;m�[1;35m20201207123402916 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{NOT_STARTED}: state_chg to WAIT_LCHAN_ACTIVE (handover_fsm.c:397)�[0;m
�[0;m�[1;35m20201207123402922 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_LCHAN_ACTIVE}: Received Event HO_EV_LCHAN_ACTIVE (lchan_fsm.c:818)�[0;m
�[0;m�[1;35m20201207123402922 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_LCHAN_ACTIVE}: state_chg to WAIT_RR_HO_DETECT (handover_fsm.c:993)�[0;m
�[0;m�[1;35m20201207123403329 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: Received Event HO_EV_RR_HO_FAIL (gsm_04_08_rr.c:946)�[0;m
�[0;m�[1;35m20201207123403329 �[1;31mDRSL�[0;m�[1;35m �[1;31mERROR�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Handover failed in state HANDOVER, Failure (MS sent RR Handover Failure): Received RR Handover Fail message (handover_fsm.c:1155)�[0;m
�[0;m�[1;35m20201207123403329 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:932)�[0;m
�[0;m�[1;35m20201207123403329 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:933)�[0;m
�[0;m�[1;35m20201207123403329 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:934)�[0;m
�[0;m�[1;35m20201207123403329 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:935)�[0;m
�[0;m�[1;35m20201207123403329 �[1;32mDRSL�[0;m�[1;35m �[1;32mINFO�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Result: Failure (MS sent RR Handover Failure) (handover_fsm.c:937)�[0;m
�[1;35m20201207123403330 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: Terminating (cause = OSMO_FSM_TERM_REGULAR) (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123403330 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: Freeing instance (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123403330 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000013ba0]{WAIT_RR_HO_DETECT}: Deallocated (fsm.c:573)�[0;m
12:34:07.491052 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 3 requirements): ['0-0-3-0 TCH/F ESTABLISHED']
12:34:07.626546 bus Vty(host='10.42.42.7', port=4242): ERR: Found unexpected lchan activity (2 of 3 requirements): ['missing: 1-0-2-0 TCH/F ESTABLISHED', 'unexpected: 0-0-2-0 TCH/F ESTABLISHED']  [trial-163↪handover_2G:oc2g+bts1-trx-lms↪handover.py:126↪handover.py↪osmo-bsc_10.42.42.7↪Vty(host='10.42.42.7', port=4242)]
12:34:07.858962 tst                  handover.py:129: did not work, try again... (attempts left: 3)
12:34:08.196047 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
12:34:09.531701 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 1 requirements): ['not: 1-0-2-0']
12:34:11.940434 tst                  handover.py:117: trigger handover: bts 0 trx 0 timeslot 2 sub-slot 0 handover 1
12:34:12.316315 tst                  handover.py:120: wait for handover counters to change...
12:34:16.005349 tst                  handover.py:122: 
| rate_ctr.abs.bsc.0.handover:failed = 1
| rate_ctr.abs.bts.0.handover:failed = 1
12:34:16.273356 tst                  handover.py:124: 
�[1;35m20201207123412049 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: Allocated (fsm.c:461)�[0;m
�[0;m�[1;35m20201207123412049 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: is child of SUBSCR_CONN(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000016ea0] (fsm.c:491)�[0;m
�[0;m�[1;35m20201207123412049 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BSC) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123412049 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BTS) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123412050 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: No explicit neighbors, regarding all local cells as neighbors (handover_logic.c:188)�[0;m
�[0;m�[1;35m20201207123412050 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123412050 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123412051 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Selected lchan (bts=1,trx=0,ts=2,ss=0) (handover_fsm.c:395)�[0;m
�[0;m�[1;35m20201207123412051 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{NOT_STARTED}: state_chg to WAIT_LCHAN_ACTIVE (handover_fsm.c:397)�[0;m
�[0;m�[1;35m20201207123412056 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_LCHAN_ACTIVE}: Received Event HO_EV_LCHAN_ACTIVE (lchan_fsm.c:818)�[0;m
�[0;m�[1;35m20201207123412056 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_LCHAN_ACTIVE}: state_chg to WAIT_RR_HO_DETECT (handover_fsm.c:993)�[0;m
�[0;m�[1;35m20201207123412472 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: Received Event HO_EV_RR_HO_FAIL (gsm_04_08_rr.c:946)�[0;m
�[0;m�[1;35m20201207123412472 �[1;31mDRSL�[0;m�[1;35m �[1;31mERROR�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Handover failed in state HANDOVER, Failure (MS sent RR Handover Failure): Received RR Handover Fail message (handover_fsm.c:1155)�[0;m
�[0;m�[1;35m20201207123412472 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:932)�[0;m
�[0;m�[1;35m20201207123412472 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:933)�[0;m
�[0;m�[1;35m20201207123412472 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:934)�[0;m
�[0;m�[1;35m20201207123412472 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:935)�[0;m
�[0;m�[1;35m20201207123412472 �[1;32mDRSL�[0;m�[1;35m �[1;32mINFO�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Result: Failure (MS sent RR Handover Failure) (handover_fsm.c:937)�[0;m
�[1;35m20201207123412473 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: Terminating (cause = OSMO_FSM_TERM_REGULAR) (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123412473 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: Freeing instance (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123412473 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000138a0]{WAIT_RR_HO_DETECT}: Deallocated (fsm.c:573)�[0;m
12:34:16.604112 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 3 requirements): ['0-0-3-0 TCH/F ESTABLISHED']
12:34:16.738853 bus Vty(host='10.42.42.7', port=4242): ERR: Found unexpected lchan activity (2 of 3 requirements): ['missing: 1-0-2-0 TCH/F ESTABLISHED', 'unexpected: 0-0-2-0 TCH/F ESTABLISHED']  [trial-163↪handover_2G:oc2g+bts1-trx-lms↪handover.py:126↪handover.py↪osmo-bsc_10.42.42.7↪Vty(host='10.42.42.7', port=4242)]
12:34:16.972478 tst                  handover.py:129: did not work, try again... (attempts left: 2)
12:34:17.305981 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
12:34:18.640470 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 1 requirements): ['not: 1-0-2-0']
12:34:21.079963 tst                  handover.py:117: trigger handover: bts 0 trx 0 timeslot 2 sub-slot 0 handover 1
12:34:21.453597 tst                  handover.py:120: wait for handover counters to change...
12:34:25.146683 tst                  handover.py:122: 
| rate_ctr.abs.bsc.0.handover:failed = 1
| rate_ctr.abs.bts.0.handover:failed = 1
12:34:25.413659 tst                  handover.py:124: 
�[1;35m20201207123421189 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: Allocated (fsm.c:461)�[0;m
�[0;m�[1;35m20201207123421189 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: is child of SUBSCR_CONN(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000016ea0] (fsm.c:491)�[0;m
�[0;m�[1;35m20201207123421189 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BSC) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123421189 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BTS) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123421189 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: No explicit neighbors, regarding all local cells as neighbors (handover_logic.c:188)�[0;m
�[0;m�[1;35m20201207123421190 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123421190 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123421190 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Selected lchan (bts=1,trx=0,ts=2,ss=0) (handover_fsm.c:395)�[0;m
�[0;m�[1;35m20201207123421191 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{NOT_STARTED}: state_chg to WAIT_LCHAN_ACTIVE (handover_fsm.c:397)�[0;m
�[0;m�[1;35m20201207123421195 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_LCHAN_ACTIVE}: Received Event HO_EV_LCHAN_ACTIVE (lchan_fsm.c:818)�[0;m
�[0;m�[1;35m20201207123421196 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_LCHAN_ACTIVE}: state_chg to WAIT_RR_HO_DETECT (handover_fsm.c:993)�[0;m
�[0;m�[1;35m20201207123421610 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: Received Event HO_EV_RR_HO_FAIL (gsm_04_08_rr.c:946)�[0;m
�[0;m�[1;35m20201207123421610 �[1;31mDRSL�[0;m�[1;35m �[1;31mERROR�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Handover failed in state HANDOVER, Failure (MS sent RR Handover Failure): Received RR Handover Fail message (handover_fsm.c:1155)�[0;m
�[0;m�[1;35m20201207123421610 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:932)�[0;m
�[0;m�[1;35m20201207123421610 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:933)�[0;m
�[0;m�[1;35m20201207123421610 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:934)�[0;m
�[0;m�[1;35m20201207123421611 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:935)�[0;m
�[0;m�[1;35m20201207123421611 �[1;32mDRSL�[0;m�[1;35m �[1;32mINFO�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Result: Failure (MS sent RR Handover Failure) (handover_fsm.c:937)�[0;m
�[1;35m20201207123421612 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: Terminating (cause = OSMO_FSM_TERM_REGULAR) (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123421612 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: Freeing instance (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123421612 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000135a0]{WAIT_RR_HO_DETECT}: Deallocated (fsm.c:573)�[0;m
12:34:25.744626 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 3 requirements): ['0-0-3-0 TCH/F ESTABLISHED']
12:34:25.881278 bus Vty(host='10.42.42.7', port=4242): ERR: Found unexpected lchan activity (2 of 3 requirements): ['missing: 1-0-2-0 TCH/F ESTABLISHED', 'unexpected: 0-0-2-0 TCH/F ESTABLISHED']  [trial-163↪handover_2G:oc2g+bts1-trx-lms↪handover.py:126↪handover.py↪osmo-bsc_10.42.42.7↪Vty(host='10.42.42.7', port=4242)]
12:34:26.113731 tst                  handover.py:129: did not work, try again... (attempts left: 1)
12:34:26.446879 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (2 of 2 requirements): ['0-0-2-0 TCH/F ESTABLISHED', '0-0-3-0 TCH/F ESTABLISHED']
12:34:27.781460 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 1 requirements): ['not: 1-0-2-0']
12:34:30.215411 tst                  handover.py:117: trigger handover: bts 0 trx 0 timeslot 2 sub-slot 0 handover 1
12:34:30.604261 tst                  handover.py:120: wait for handover counters to change...
12:34:34.299974 tst                  handover.py:122: 
| rate_ctr.abs.bsc.0.handover:failed = 1
| rate_ctr.abs.bts.0.handover:failed = 1
12:34:34.566741 tst                  handover.py:124: 
�[1;35m20201207123430324 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: Allocated (fsm.c:461)�[0;m
�[0;m�[1;35m20201207123430324 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: is child of SUBSCR_CONN(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x612000016ea0] (fsm.c:491)�[0;m
�[0;m�[1;35m20201207123430324 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BSC) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123430325 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: (BTS) incrementing rate counter: handover:attempted Intra-BSC handover attempts (handover_fsm.c:333)�[0;m
�[0;m�[1;35m20201207123430325 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-none: No explicit neighbors, regarding all local cells as neighbors (handover_logic.c:188)�[0;m
�[0;m�[1;35m20201207123430325 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123430325 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:attempted Intra-BSC handover attempts (handover_fsm.c:385)�[0;m
�[0;m�[1;35m20201207123430325 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Selected lchan (bts=1,trx=0,ts=2,ss=0) (handover_fsm.c:395)�[0;m
�[0;m�[1;35m20201207123430326 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{NOT_STARTED}: state_chg to WAIT_LCHAN_ACTIVE (handover_fsm.c:397)�[0;m
�[0;m�[1;35m20201207123430331 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_LCHAN_ACTIVE}: Received Event HO_EV_LCHAN_ACTIVE (lchan_fsm.c:818)�[0;m
�[0;m�[1;35m20201207123430331 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_LCHAN_ACTIVE}: state_chg to WAIT_RR_HO_DETECT (handover_fsm.c:993)�[0;m
�[0;m�[1;35m20201207123430749 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: Received Event HO_EV_RR_HO_FAIL (gsm_04_08_rr.c:946)�[0;m
�[0;m�[1;35m20201207123430749 �[1;31mDRSL�[0;m�[1;35m �[1;31mERROR�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Handover failed in state HANDOVER, Failure (MS sent RR Handover Failure): Received RR Handover Fail message (handover_fsm.c:1155)�[0;m
�[0;m�[1;35m20201207123430749 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:932)�[0;m
�[0;m�[1;35m20201207123430749 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BSC) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:933)�[0;m
�[0;m�[1;35m20201207123430749 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: handover:failed Received Handover Fail messages (handover_fsm.c:934)�[0;m
�[0;m�[1;35m20201207123430749 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: (BTS) incrementing rate counter: intra_bsc_ho:failed Received Handover Fail messages (handover_fsm.c:935)�[0;m
�[0;m�[1;35m20201207123430749 �[1;32mDRSL�[0;m�[1;35m �[1;32mINFO�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: (0-0-2-TCH_F-0-SPEECH_V1) --HO-> (1-0-2-TCH/F-0) (subscr subscr-IMSI-901700000015252-TMSI-0xbe65c872) HO-intraBSC: Result: Failure (MS sent RR Handover Failure) (handover_fsm.c:937)�[0;m
�[1;35m20201207123430750 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: Terminating (cause = OSMO_FSM_TERM_REGULAR) (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123430750 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: Freeing instance (handover_fsm.c:970)�[0;m
�[0;m�[1;35m20201207123430750 �[1;34mDRSL�[0;m�[1;35m �[1;34mDEBUG�[0;m�[1;35m handover(intraBSC_msc0-conn5_subscr-IMSI-901700000015252-TMSI-0xbe65c872)[0x6120000132a0]{WAIT_RR_HO_DETECT}: Deallocated (fsm.c:573)�[0;m
12:34:34.897550 bus Vty(host='10.42.42.7', port=4242): Found matching lchan activity (1 of 3 requirements): ['0-0-3-0 TCH/F ESTABLISHED']
12:34:35.032527 bus Vty(host='10.42.42.7', port=4242): ERR: Found unexpected lchan activity (2 of 3 requirements): ['missing: 1-0-2-0 TCH/F ESTABLISHED', 'unexpected: 0-0-2-0 TCH/F ESTABLISHED']  [trial-163↪handover_2G:oc2g+bts1-trx-lms↪handover.py:126↪handover.py↪osmo-bsc_10.42.42.7↪Vty(host='10.42.42.7', port=4242)]
12:34:35.265029 tst                  handover.py:129: did not work, try again... (attempts left: 0)
12:34:35.321829 tst                  handover.py:146: ----- Report fragment: handover_2G:oc2g+bts1-trx-lms/handover.py/05.1_handover_ms0: FAIL (46.3s)
12:34:35.329428 tst                  handover.py:146: ERR: AssertionError: handover.py:146: not_expected=('0-0-2-0 TCH/F ESTABLISHED',),  [trial-163↪handover_2G:oc2g+bts1-trx-lms↪handover.py:146]
12:34:35.335409 tst                  handover.py:146: Test FAILED (174.6 sec)
12:34:35.377329 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7950): Terminating (SIGTERM)
12:34:35.426614 run    osmo-hlr_10.42.42.2(pid=7951): Terminating (SIGTERM)
12:34:35.476736 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7952): Terminating (SIGTERM)
12:34:35.529103 run    osmo-stp_10.42.42.5(pid=7953): Terminating (SIGTERM)
12:34:35.589034 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7954): Terminating (SIGTERM)
12:34:35.661558 run    osmo-msc_10.42.42.6(pid=7955): Terminating (SIGTERM)
12:34:35.711333 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7956): Terminating (SIGTERM)
12:34:35.760494 run    osmo-mgw_10.42.42.3(pid=7957): Terminating (SIGTERM)
12:34:35.810512 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7958): Terminating (SIGTERM)
12:34:35.860089 run    osmo-mgw_10.42.42.4(pid=7959): Terminating (SIGTERM)
12:34:35.910187 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7960): Terminating (SIGTERM)
12:34:35.959673 run    osmo-bsc_10.42.42.7(pid=7961): Terminating (SIGTERM)
12:34:36.008877 run          osmo-bts-oc2g(pid=7969): Terminating (SIGTERM)
12:34:36.058128 run           osmo-trx-lms(pid=7982): Terminating (SIGTERM)
12:34:36.107116 run           osmo-bts-trx(pid=7983): Terminating (SIGTERM)
12:34:36.118299 ---      ParallelTerminationStrategy: PID 7950 died...
12:34:36.184696 run pcap-recorder_any(filters='host 10.42.42.2')(pid=7950): Terminated: ok {rc=0}
12:34:36.196550 ---      ParallelTerminationStrategy: PID 7951 died...
12:34:36.263722 run    osmo-hlr_10.42.42.2(pid=7951): Terminated: ok {rc=0}
12:34:36.275542 ---      ParallelTerminationStrategy: PID 7952 died...
12:34:36.340807 run pcap-recorder_any(filters='host 10.42.42.5 and port not 22')(pid=7952): Terminated: ok {rc=0}
12:34:36.352661 ---      ParallelTerminationStrategy: PID 7953 died...
12:34:36.417823 run    osmo-stp_10.42.42.5(pid=7953): Terminated: ok {rc=0}
12:34:36.429601 ---      ParallelTerminationStrategy: PID 7954 died...
12:34:36.494807 run pcap-recorder_any(filters='host 10.42.42.6 and port not 22')(pid=7954): Terminated: ok {rc=0}
12:34:36.506567 ---      ParallelTerminationStrategy: PID 7956 died...
12:34:36.571710 run pcap-recorder_any(filters='host 10.42.42.3 and port not 22')(pid=7956): Terminated: ok {rc=0}
12:34:36.583507 ---      ParallelTerminationStrategy: PID 7957 died...
12:34:36.648782 run    osmo-mgw_10.42.42.3(pid=7957): Terminated {rc=15}
12:34:36.660683 ---      ParallelTerminationStrategy: PID 7958 died...
12:34:36.725951 run pcap-recorder_any(filters='host 10.42.42.4 and port not 22')(pid=7958): Terminated: ok {rc=0}
12:34:36.737792 ---      ParallelTerminationStrategy: PID 7959 died...
12:34:36.802834 run    osmo-mgw_10.42.42.4(pid=7959): Terminated {rc=15}
12:34:36.814662 ---      ParallelTerminationStrategy: PID 7960 died...
12:34:36.879961 run pcap-recorder_any(filters='host 10.42.42.7 and port not 22')(pid=7960): Terminated: ok {rc=0}
12:34:36.891799 ---      ParallelTerminationStrategy: PID 7969 died...
12:34:36.957227 run          osmo-bts-oc2g(pid=7969): Terminated {rc=36608}
12:34:36.969219 ---      ParallelTerminationStrategy: PID 7982 died...
12:34:37.034974 run           osmo-trx-lms(pid=7982): Terminated {rc=36608}
12:34:39.063209 ---      ParallelTerminationStrategy: PID 7955 died...
12:34:39.141696 run    osmo-msc_10.42.42.6(pid=7955): Terminated: ok {rc=0}
12:34:39.154885 ---      ParallelTerminationStrategy: PID 7961 died...
12:34:39.225714 run    osmo-bsc_10.42.42.7(pid=7961): Terminated: ok {rc=0}
12:34:43.086896 run           osmo-bts-trx(pid=7983): Terminating (SIGINT)
12:34:43.270924 ---      ParallelTerminationStrategy: PID 7983 died...
12:34:43.343611 run           osmo-bts-trx(pid=7983): Terminated: ok {rc=0}
12:34:43.505929 bus                        /sierra_3: Setting Online False
12:34:44.773653 bus                        /sierra_3: Setting Powered False
12:35:00.640770 bus                        /sierra_3: Setting Powered False
12:35:01.766014 bus                        /sierra_3: Setting Powered False
12:35:02.949938 bus                        /sierra_2: Setting Online False
12:35:04.085879 bus                        /sierra_2: Setting Powered False
12:35:15.644986 bus                        /sierra_2: Setting Powered False

Standard Error

Traceback (most recent call last):
  File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/test.py", line 76, in run
    self.path)
  File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/src/osmo_gsm_tester/core/util.py", line 414, in run_python_file
    spec.loader.exec_module( importlib.util.module_from_spec(spec) )
  File "<frozen importlib._bootstrap_external>", line 673, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/home/jenkins/workspace/osmo-gsm-tester_run-prod_handover/osmo-gsm-tester/sysmocom/suites/handover_2G/handover.py", line 146, in <module>
    not_expected=('0-0-2-0 TCH/F ESTABLISHED',),
AssertionError