Changes

hlr: pyhss: run the pyhss_hss service

Database preparations used to be done in PyHSS by all services if they
noticed that this was needed. The time between checking and creating the
tables caused a race condition where two services attempting to create
tables at the same time will result at one of them failing, we have seen
this in our ttcn3-hlr-test-pyhss jobs sometimes:

  [Database] [DEBUG] Table apn already exists
  [Database] [DEBUG] Table auc already exists
  [Database] [DEBUG] Table subscriber already exists
  …
  [testenv][pyhss] pyhss_gsup: setup script failed

I have fixed this upstream by letting only the main service (pyhss_hss)
prepare the database:
https://github.com/nickvsnetworking/pyhss/commit/8b8a2202c345fbb7262c9d07d03003cd2fdb819f

This means that we now need to run the pyhss_hss service in the HLR
tests, so pyhss_gsup doesn't fail with:

  [Database] [INFO] Waiting for the main service to prepare the database
  ERROR: 127.0.0.1:4222 did not become available within 5s!
  [testenv][pyhss] pyhss_gsup: setup script failed

Change-Id: I4fe689c8d8617432175ba403b45021c0f646970b

{hlr,hss}/pyhss/config: remove unused options

PyHSS had several unused config options. They have been removed
upstream, remove them from the osmo-ttcn3-hacks configs as well.

Related: https://github.com/nickvsnetworking/pyhss/pull/284
Change-Id: I87cf01e00fe0a3b32be9eaa4cf4c1ddf02cddc0b

ggsn: TC_pdp(4)6_clients_interact: Expect UEs can interact using global IPv6 address

The link-local IPv6 address is only meant to be used privately between
the UE and the GGSN, but the UEs should be in general able to reach each
other using their global IPv6 address (unless some operator policy
explicitly forbids it).

Since open5gs.git 6cc627c4855703a17c92352660078325c6ce63cc, open5gs-upfd supports
proper routing of IPv6 packets between UEs.

osmo-ggsn apparently doesn't implement that internal routing logic yet,
so these tests will start failing now, but it's actually legit to have
it not passing the test anymore.

Related: OS#6967
Change-Id: I0267a9c3bb85736a2aeb4f7dd91f44dbce626958

Print load before and after each test

Make it easier to figure out why tests are failing sporadically by
printing the load before and after each test. For example:

  ------ MGCP_Test.TC_crcx_sdp ------
  (14:24:42) load average: 0,41, 0,44, 0,54

Change-Id: If7d82c3c8105b3f3ca3470ba8f525ae8468dbeae

5gc: Introduce test TC_handover_inter_ngran_xn

Change-Id: I4de3f0a02ddbccc85988754a4cd83fe67b1453dc

hnbgw: Fix regression in hnbgw-test-latest

Fix unintended if condition removal in recent commit when
in the -latest case. That commit expected to leave the -latest path
untouched but ended up removing the line. Re-add it.

Fixes: 92f92923418c9803a0eb26fae97e56eed47a612d
Change-Id: I0abefedc9f9193b9665c3529a0f0e841db63401d

NGAP_Emulation: Use specific record types for AMF/RAN IDs

Change-Id: Ibd85be865a3a61c97d312df76ab3d1ae7ae32a6a

mgw: Fix sporadic failure in TC_one_crcx_loopback_rtp

Change-Id: I29d8d376e343975e3dfeb018bdfda8350bdfc58a

mgw: Improve failure message in f_TC_one_crcx_loopback_rtp()

Change-Id: I51c656371c330e4689738321d0739e280cab530d

bsc: TC_stat_msc_sctp_disconnected Fix sporadic failure

Since we recently increased a bit the timeout to assume the BSSAP_LE
peer (SMLC) is implicitly up, the function
f_bssap_le_wait_sccp_peer_available() may still be onging when we drop
the sctp connection at the STP in the test, which will generate an
explicit DUNA towards BSSAP_LE and hence fail the test because it never
comes back available before it times out.

Since we don't need nor test SMLC in this test, simply disable starting
the SMLC here to avoid this race condition and simplify the test.

Change-Id: Ib063a481d9039cbcd462609404dbae14922c7510

hnbgw: introduce test TC_pfcp_heartbeat_timeout

Related: SYS#7294
Change-Id: I1e0b50bc19e1fa1240274baea1e7fedd1265ffcb

hnbgw: Introduce test TC_pfcp_heartbeat_recovery_timestamp

Related: SYS#7294
Change-Id: I3f8098d7a34666961437fecc5100422a5e7436a4

hnbgw: Fix race conditions during PFCP Association

Since osmo-hnbgw now sends Heartbeat Requests, we need to take that into
account.
Furthermore, since it also now supports retriggering association when
detecting peer restarted upon rx of Heartbeat Req, use that to trigger
an association to clean up state and have a deterministic way to
synchronize at that point regarding state.

Change-Id: If62bd6c6e4d6cf73791029172d3d8880629f36a8

5gc: Split f_start_handler_with_pars() into multiple helpers

This allows initializing multiple ConnHldr and then connecting them and
starting them later. This is similar to what is is done eg. in
MSC_Tests.

Change-Id: Ic60009c5f7fcdc43f6e13f9ea469bfaddc639c33

BSSAP_LE_Emulation: Increase timeout in f_bssap_le_wait_sccp_peer_available()

It was spotted in SMLC_Tests.ttcn that the current timeout may be too
short, and the test fails because ttcn3 connects to STP and receives a
DUNA, and then the IUT (osmo-smlc) connects to STP around 4 seconds
later, which is too late with existing timeout.

Change-Id: I2d4210bf5785a09bf384a1534c407dcf63b1c38b

5gc: Add missing TC_connection_suspend to control block

Fixes: 1af55d318d43967ee5d056bdf2bc3a50c096389b
Change-Id: I2e9e3647917f22e16d9dd6e7199f62a305cc57b3

5gc: Update details on missing open5gs 5GC features

Change-Id: I7be0bbb70207ac6c0ff77f230da56aace29c1eda

ngap: Update asn files V17.5.0 -> V19.1.0

Copied from wireshark.git be5cfd209f62933bc46744dca3e22e731b24ab6b
epan/dissectors/asn1/ngap/*.asn.

Change-Id: I4838c1031f2e168e6bce59f3d4fc7871fb597466

Fix/change TC_tcap_loadshare_ipa_tcap_udts()

The behaviour changed to not reject tcap messages outside of any valid
route. Change the test expectation that these messages are indeed
forwarded.

Change-Id: Ibffa98702fd6eb276410630ab525d986fa0f36e0

ngap: Add multiple templates related to RRC_INACTIVE

Change-Id: Ie0e6f5054c45e681373c6db59d2b298c71ad4836

5gc: Introduce test TC_connection_inactive

The test is expected to fail because open5gs (as of v2.7.3) doesn't
implement such features; see open5gs ticket referenced below.

Related: SYS#7073
Related: open5gs GH#4319
Change-Id: Icf60dde814b7c8ef4cdb4c886423058c7d741a9b

5gc: Introduce test TC_connection_suspend

The test is expected to fail because open5gs (as of v2.7.3) doesn't
implement such features; see open5gs ticket referenced below.

Related: SYS#7073
Related: open5gs GH#4322

Change-Id: I6da2571518a9b50369fa8e0dd19e447137ea20c6

Makefile: clean-logs: also remove *.merged

Change-Id: I6efe839ae2d527387b533fd110965335c0f850c9

bts: as_rsl_meas_res(): tolerate low RxLev in early reports

The MS is not guaranteed to begin transmitting immediately after
activation of the respective logical channel.  There is typically a
short interval during which the BTS receives no bursts.  Consequently,
the initial RSL measurement reports are expected to indicate poor
RxLev/RxQual values.

We already tolerate bad RxQual values;  however, sporadic failures
still occur due to low RxLev in the very first report.  Extend the
logic in f_build_meas_res_tmpl() to tolerate low RxLev as well.

Change-Id: Ic632917d429ec597dba524bf79749944cf1fc628
Related: 8ce558dd ("bts: as_rsl_meas_res(): tolerate bad RxQual in early reports")
Related: OS#6933

hnbgw: TC_rab_assign_mgcp_to: Allow hnbgw to tear down CN conn at SCCP level

Newer versions of osmo-hnbgw will be tearing down the CN conn at SCCP
level by transmitting an RLSD in this scenario, instead of sending a Iu
RANAP Release Request crafted by the osmo-hnbgw itself.
The HNBGW is expected to forward RANAP messages between UE/HNB and CN,
not to generate them.

Related: SYS#7294
Change-Id: Idb062a072c7403442ce981c6c9caeb7effb86554

msc: f_tc_ho_inter_bsc0(): catch BSSMAP HandoverRequiredReject

It may happen that the target BSC is not yet connected to the MSC.
In this case the MSC rejects handover with HandoverRequiredReject.
Let's catch this PDU to make the verdict cleaner in such case.

Change-Id: I5ba98e323e0c7794554ccb115d7697ad03ccbfa5
Related: OS#6951

msc: TC_ho_inter_bsc: wait for the target BSC to become ready

A race condition may occur between the two testcase components
emulating BSCs: the source BSC initiates handover to the terget
BSC before the later completes connection establishment with the
MSC.  In this case, the MSC fails to find the target peer and
rejects our HandoverRequired with HandoverRequiredReject.

  Target RAN peer from neighbor config is not connected: Cell ID CGI:023-42-5-6 resolves to target address 0.24.2
  Cannot find target peer for cell ID CGI:023-42-5-6
  Attempted Handover to 1 cells without success

Let's avoid this by waiting for the target BSC to become available
before initiating the handover procedure.  Use the COORD port
("intercom") to signal readiness of the target BSC.

Change-Id: I00b8a8b89ec7bb15590d30aadf40055981375a5d
Related: OS#6951

bts: limit stderr logging to ERROR to avoid long write() to ext4 fs

Commit c3ab9ec33f37c6837fb8bcb0fa0bbd2e98781fb7 already decreased
verbosity to NOTICE, but that's not enough since we still get plenty of
logging and hence plenty of long write()s during test execution.

Let's try decreasing to ERROR and see if that's enough for now to
stabilize the tests.

Change-Id: I41982fd83a9f46609464afb6b7242240063d4d91

hnbgw: expected-results.xml: Add missing TC_sccp_cr_limit

Change-Id: I4138cff311f82358f040290f3af483df8facc4cf

bts: as_rsl_meas_res(): tolerate bad RxQual in early reports

Bit errors are expected during early stage of channel establishment.
Ignore bad RxQual in the very first reports (4 * 0.480s = 1.92s).

This reduces probability of sporadic failures.

Change-Id: Ie8acb15b66cd92092ec49bc8caeaf0ad4d471f1d
Related: OS#6933

bts: as_rsl_meas_res(): do not expect dummy UL SACCH

We're now populating the UL SACCH cache before establishing a
decicated channel in f_est_dchan(), so this special case is
no longer needed and can be safely removed.

Change-Id: I6eeee2876a76d41ab453af4c7724173fea773b83
Related: I4781fe7539e64e77d594e84f905646c127f46b64
Related: OS#6933

bts: f_verify_ramp_up(): fix RxLev / dBm mixup

Comparing RxLev and dBm values is comparing apples to oranges.  It's
just a lucky coincidence that fake_trx is using RF path loss value of
110 dB, which is also the offset between RxLev and dBm.

* Add `mp_rf_path_loss` matching the default value (110 dB) in fake_trx.
* Add `f_bts_max_rx_level_dbm()` calculating the expected Rx power level.
* Apply `rxlev2dbm()` on `l1_dl.dl_info.rx_level` whenever needed.
* Clarify the units in verdict / logging messages.

Change-Id: I818d18d6e0711247b73ee1f336133e2ed3f6e2cb
Related: OS#6939

bts: fix TC_rsl_ms_pwr_dyn_ass_updown: simulate a good C/I value

Now that we populate the UL SACCH cache in advance, trxcon is sending
ms-pwr-lvl 7 in the first UL SACCH block (as expected).  This suddenly
makes the testcase fail, because now the MS power loop is perfectly
happy about the received input values and does not order any changes
to the current MS power level anymore, contrary to our expectations.

Why? Short answer: because of C/I (curr 6, avg 6) being outside of
the good range (thresh 13..17).  The MS power loop intentionally
avoids reducing Tx power because that would potentially degrade the
link quality (C/I) even further.  Solution: let's order fake_trx.py
to simulate a C/I value that is within the thresholds.

This alone does not fix the testcase yet, another fix follows.

Change-Id: I8926fc925e930bc2703210931b613988161a72da
Related: c246f207 ("bts: f_est_dchan(): populate UL SACCH cache")
Related: OS#6945

bts: fix TC_rsl_ms_pwr_dyn_ass_updown: adjust the timers

This testcase predates major changes to the MS power control logic
in osmo-bts and was passing thanks to a coincidence (broken UL SACCH
cache in trxcon).  Specifically, the MS power loop is now using
P_Con_INTERVAL=4 by default.  This means that the power control
decision is intentionally delayed and a change may occur only once
in a period of 4 SACCH blocks (N=4 is ~1.92s).

Adjust the timer values to take this into account.  Take a chance
to move comments in-place and expand them with more details.

This patch makes TC_rsl_ms_pwr_dyn_ass_updown pass again.

Change-Id: I36d87c12f49ec13003b708d768285aa6840e81eb
Related: OS#6945

bts: TC_rsl_ms_pwr_dyn_ass_updown: misc improvements

Change-Id: Iae3fa5f09d68bd472f7236bf882f5b168d87384a
Related: OS#6945