Skip to content
Success

Changes

Summary

  1. s1gw: Print idx of component timing out (details)
  2. StatsD_Checker: Allow building without VTY support (details)
  3. StatsD_Checker: Simplify by getting rid of StatsDExpectPriv (details)
  4. StatsD_Checker: Allow running without VTY support (details)
  5. s1gw: Initial StatsD support (details)
  6. s1gw: ConnHdlr: Use stats reported by osmo-s1gw to expect PFCP Assoc (details)
  7. s1gw: TC_setup: Validate some metrics (details)
  8. SIP_Emulation: Allow forwarding ASP_SIP_close (details)
  9. asterisk: Introduce test TC_ims_call_mo_after_tcp_conn_closed (details)
  10. deps/Makefile: Update titan.TestPorts.SIPmsg to current master (details)
  11. testenv: support obtaining talloc reports (details)
  12. */testenv.cfg: add vty_port, vty_host (details)
  13. HTTP_Adapter: Rename s/g_pars/g_http_pars/ (details)
  14. Misc_Helpers: Introduce API helper f_str_startswith (details)
  15. Introduce Prometheus_Checker module (details)
  16. hss: Initial validation of Prometheus metrics (details)
  17. Introduce PCRF_Tests testsuite (details)
  18. pcrf: Validate Gx metrics (details)
  19. cosmetic: Fix same typo in comment in several files (details)
  20. lapdm: remove (details)
  21. upf: remove unused port PFCP from test_CT (details)
  22. hnbgw: move vc_PFCP from ConnHdlr to test_CT (details)
  23. library/PFCP_Emulation: look up by SeqNr and SEID separately (details)
  24. library/PFCP_Emulation: fix routing of incoming PDUs (details)
  25. library/PFCP_Emulation: allow retrieving the Recovery Time Stamp (details)
  26. s1gw: rework f_pfcp_wait_assoc_setup() into f_ConnHdlr_pfcp_expect() (details)
  27. s1gw: cache PFCP Recovery Timestamp in ConnHdlr (details)
  28. s1gw: send the actual Recovery Time Stamp (details)
  29. s1gw: move t_ConnHdlrPars back to S1GW_Tests (details)
  30. s1gw: add TC_pfcp_heartbeat (details)
  31. library/s1ap: add more E-RAB related templates (details)
  32. library/PFCP_Templates: add missing receive templates (details)
  33. testenv: on crash, wait until test stopped (details)
  34. testenv: display current test name on ^C (details)
  35. library/PFCP_Templates: fix missing fields in (details)
  36. library/s1ap: fix formatting in ts_S1AP_RABSetupItemCtxtSURes (details)
  37. sgsn/SGSN_Tests: remove a superfluous/wrong comment (details)
  38. asterisk: TC_ims_reregistration: Expect re-register using existing ipsec (details)
  39. testenv: get coredump + backtrace on crash (details)
Commit 4099510236a40dcabdb3d7a9930e65e26fcdc731 by Pau Espin Pedrol
s1gw: Print idx of component timing out

Change-Id: I354104dd88627a6d15e160d02f5ddf1072745a17
The file was modifieds1gw/S1GW_ConnHdlr.ttcn
Commit c4328a5457001987aef39943fd638cca1079c5e9 by Pau Espin Pedrol
StatsD_Checker: Allow building without VTY support

Some programs may support exporting to statsd, but may not support the
Osmocom VTY set of commands to send reports, or not have a VTY at all.

The current StatsD_Checker implementation as it is now is not totally
useful without VTY code. a Follow-up patch will provide new APIs and
improved implementation to use it without VTY.

Change-Id: I5421c76e4f303fd16d4db945a1c69910e40ac820
The file was modifiedmgw/regen_makefile.sh
The file was modifiedupf/gen_links.sh
The file was modifiedpcu/regen_makefile.sh
The file was modifiedbsc/regen_makefile.sh
The file was modifiedhnbgw/gen_links.sh
The file was removedlibrary/StatsD_Checker.ttcn
The file was modifiedmgw/gen_links.sh
The file was modifiedupf/regen_makefile.sh
The file was modifiedhnbgw/regen_makefile.sh
The file was modifiedpcu/gen_links.sh
The file was modifiedns/regen_makefile.sh
The file was modifiedhnodeb/gen_links.sh
The file was modifiedhnodeb/regen_makefile.sh
The file was modifiedbsc/gen_links.sh
The file was addedlibrary/StatsD_Checker.ttcnpp
The file was modifiedns/gen_links.sh
Commit 1498eeedb4c59c84d0c692c211872f29f606dc87 by Pau Espin Pedrol
StatsD_Checker: Simplify by getting rid of StatsDExpectPriv

Get rid of extra record, we have plenty of them and we'll have more once
we add new APIs to work without VTY.
While doing so, simplify the expect loop by maintaining a counter.

Change-Id: Ib14c838c5f0647d050249853327671542182636f
The file was modifiedlibrary/StatsD_Checker.ttcnpp
Commit a3810cfe130b9c29fc16c86aea6dbaa30fa64515 by Pau Espin Pedrol
StatsD_Checker: Allow running without VTY support

New features are added to the public API of StatsD_Checker which make it
possible to use it...

* without "stats reset":
Feature to take snapshots (f_statsd_snapshot()) which can later be
used to validate expectancies with values relative to the snapshot,
using API f_statsd_expect_from_snapshot().
This way, one can do:
"""
var StatsDExpects statsd_exp := { /* relative expectancies here... */ };
var StatsDMetrics statsd_snapshot := f_statsd_snapshot(f_statsd_keys_from_expect(statsd_exp));
/* do some test stuff here changing the state of the IUT... */
f_statsd_expect_from_snapshot(statsd_exp, snapshot := statsd_snapshot);
"""

* without polling ("stats report"), aka with periodict reporting:
New parameter wait_converge in f_statsd_expect(), which allows
overcoming race conditions with StatsD server processing older incoming
metrics due to periodic reporting.
This feature also allows a test to wait until a state changes in the
IUT.

Change-Id: Ie1180a5b674504864309c3b9b11bfcf5256d9178
The file was modifiedlibrary/StatsD_Types.ttcn
The file was modifiedlibrary/StatsD_Checker.ttcnpp
Commit 956bf05e7ea56db2be0685adc743a92a267b1832 by Pau Espin Pedrol
s1gw: Initial StatsD support

Change-Id: Ib98c2fe68006a8d0f45de6966927b349a3b933ca
The file was modifieds1gw/S1GW_Tests.ttcn
The file was modifieds1gw/regen_makefile.sh
The file was modifieds1gw/S1GW_ConnHdlr.ttcn
The file was modifieds1gw/gen_links.sh
The file was modifieds1gw/S1GW_Tests.cfg
Commit a025436a922c6a6df79bced57522fc77a8899aaa by Pau Espin Pedrol
s1gw: ConnHdlr: Use stats reported by osmo-s1gw to expect PFCP Assoc Setup

Change-Id: I329745125d78786fc9c243f0047cd394de9b16d4
The file was modifieds1gw/S1GW_ConnHdlr.ttcn
The file was modifieds1gw/osmo-s1gw.config
Commit 6a0b01c6159a5d394290ab5e954938e6b606fb4b by Pau Espin Pedrol
s1gw: TC_setup: Validate some metrics

Change-Id: Iae629bc9291ff8b6e482f61d692e4d97ff0136bd
The file was modifieds1gw/S1GW_Tests.ttcn
Commit 1d1b809a956bdb474afba8c0e8e2339d4cba1011 by Pau Espin Pedrol
SIP_Emulation: Allow forwarding ASP_SIP_close

This allows closing SIP connections at will from within the test.

Change-Id: If5c6bb0c0d773e0ba18ec28062663bdec751c8c4
The file was modifiedlibrary/SIP_Emulation.ttcn
Commit 4c4fccdf0dfac1ba763c745a7f109ab23a512906 by Pau Espin Pedrol
asterisk: Introduce test TC_ims_call_mo_after_tcp_conn_closed

Related: SYS#7067
Change-Id: I90df69b2418557d7001dddee4b6930ebbc58066d
The file was modifiedasterisk/expected-results.xml
The file was modifiedasterisk/IMS_ConnectionHandler.ttcn
The file was modifiedasterisk/Asterisk_Tests.ttcn
Commit cbed7a7f4130fe158e184f1ba318e10b0236ecee by Pau Espin Pedrol
deps/Makefile: Update titan.TestPorts.SIPmsg to current master

New HEAD contains 1 more commit which fixes a regression introduced in
previous version, which is needed in order to get
Asterisk_Tests.TC_ims_call_mo_after_tcp_conn_closed working fine.

Change-Id: I9962076666706bdb4f6319ddade53d5622236995
The file was modifieddeps/Makefile
Commit bcee2e6ff23e13699cf01fe0eeb8cae97b09a3ae by Oliver Smith
testenv: support obtaining talloc reports

Allow setting vty_port= and vty_host= (default: 127.0.0.1) for SUT
components in testenv.cfg. Pass the values as OSMO_SUT_PORT and
OSMO_SUT_HOST to the testsuite, so ttcn3-tcpdump-stop.sh generates
talloc reports after running test cases.

The next patch will add vty_port/vty_host to existing testenv.cfg files.

I have considered enabling the talloc report only conditionally (e.g.
with a --talloc-report parameter). But previous behavior in
docker-playground was also that we created them unconditionally where
OSMO_SUT_PORT/HOST was set, and they are generated almost instantly so
it did not seem necessary to make this conditionally.

Related: OS#6552
Change-Id: I50a7035f072668ca2ab65de1bc1f194da8b18610
The file was modified_testenv/testenv/testsuite.py
The file was modified_testenv/README.md
The file was modified_testenv/testenv/testenv_cfg.py
Commit aa4d036af481a5277acd1c298d2750bb01852f41 by Oliver Smith
*/testenv.cfg: add vty_port, vty_host

Add these variables, so a talloc report can be obtained after running
test cases.

Closes: OS#6552
Change-Id: I5c0305c5d2ac6ac79dcf01bed8fb9d5beeeb99a6
The file was modifiedcbc/testenv.cfg
The file was modifiedhlr/testenv.cfg
The file was modifiedsgsn/testenv_sns.cfg
The file was modifiedsgsn/testenv_generic.cfg
The file was modifiedsip/testenv.cfg
The file was modifiedsmlc/testenv.cfg
The file was modifiedhnodeb/testenv.cfg
The file was modifiedupf/testenv.cfg
The file was modifiedmgw/testenv.cfg
Commit 7324d47706d4c564acfd2e5ce2b702608791d81d by Pau Espin Pedrol
HTTP_Adapter: Rename s/g_pars/g_http_pars/

The HTTP_Adapter component is a probable candidate to end up as part of
another component extending it, like the Prometheus_Checker coming up
soon.
Since the fields in HTTP_Adapter are available to subclass components,
that means they cannot use the usual "g_pars".
Rename the self-enclosed "g_pars" in HTTP_Adapter to let subclasses use
it.

Change-Id: Ia07cf8717593073a71632e247d7fe0775e66eb13
The file was modifiedlibrary/HTTP_Adapter.ttcn
Commit c5c4961ea578692d07df43e3e8174e7d369bbe04 by Pau Espin Pedrol
Misc_Helpers: Introduce API helper f_str_startswith

Counterpart of f_str_endswith(), this API is usually available in
several programming languages like python, and it's handy to have since
it's usually used and hence dev doesn't need to think on how to
implement it using regexp, etc.

This API will be used in a follow-up patch implementing Prometheus
metrics checker module.

Change-Id: Iad392ac652714a54a25954ea697f082d55772aa8
The file was modifiedlibrary/Misc_Helpers.ttcn
Commit 6d8b8b6c27a926d466cdcc620366892ca74c4784 by Pau Espin Pedrol
Introduce Prometheus_Checker module

This module serves the same purpose as the existing StatsD_Checker.
It will be used in open5gs, which so far exports its metrics using
Prometheus.

Change-Id: Iec5544ba74978918f1bddba12971f69a1824683e
The file was addedlibrary/Prometheus_Checker.ttcn
Commit 5771875ca76ce65f0074cffdad3b872b6bafee53 by Pau Espin Pedrol
hss: Initial validation of Prometheus metrics

Requires open5gs b2f56b964133bbe2f6ab07ead4cb62524528fcf7 for HSS
diameter metrics.

Change-Id: I507537c66a031b67518294daa8338f3836a90d6b
The file was modifiedhss/HSS_Tests.ttcn
The file was modifiedhss/gen_links.sh
The file was modifiedhss/open5gs-hss.yaml
The file was modifiedhss/HSS_Tests.default
The file was modifiedhss/regen_makefile.sh
Commit e7d3cbac2329fa551daf1315eef0eb0039e63678 by Pau Espin Pedrol
Introduce PCRF_Tests testsuite

Change-Id: I381577111ccadaefe42f6abe3cd9bb30cc8ceb56
The file was addedpcrf/PCRF_Tests.ttcn
The file was modifiedlibrary/DIAMETER_Templates.ttcn
The file was addedpcrf/mongod.conf
The file was modifiedMakefile
The file was addedpcrf/expected-results.xml
The file was addedpcrf/open5gs-pcrf.yaml
The file was addedpcrf/PCRF_Tests.cfg
The file was modifiedlibrary/DIAMETER_ts29_212_Templates.ttcn
The file was addedpcrf/create_test_subscribers.sh
The file was addedpcrf/freediameter.conf
The file was addedpcrf/regen_makefile.sh
The file was addedpcrf/testenv.cfg
The file was addedpcrf/gen_links.sh
The file was addedpcrf/PCRF_Tests.default
Commit 04c681f3916f821cc6e2a9c9d143ca3b50dae545 by Pau Espin Pedrol
pcrf: Validate Gx metrics

Change-Id: I4eab93f14d45f291c391d9519dc1462508f8a1eb
The file was modifiedpcrf/PCRF_Tests.ttcn
Commit 7795249ef78eda7b6d15ce3a19e44c4f8a66bbfa by Pau Espin Pedrol
cosmetic: Fix same typo in comment in several files

Change-Id: Id59e09b0ac89251175ecee23e7f9d1a058644aa8
The file was modifiedggsn_tests/GGSN_Tests.ttcn
The file was modifiedhss/HSS_Tests.ttcn
The file was modifiedpgw/PGW_Tests.ttcn
Commit 48d4a3510b787753dea547ef2a246582e5fa4f3e by laforge
lapdm: remove

In the related patch, lapdm/L1CTL_Test.ttcn has been moved to
bts/BTS_Tests_LAPDm.ttcn. The remaining files in this directory are
unused. Remove them to unclutter the repository a bit.

Related: Ibbc967f0bd9a19ab4bbd17f612654d72cb32d571
Change-Id: I6aae060a5b252d3ab36378da8c721154b3186466
The file was removedlapdm/gen_links.sh
The file was removedlapdm/regen_makefile.sh
Commit 180cfb4f6f9e78d6da534af0df519ab192f69478 by laforge
upf: remove unused port PFCP from test_CT

Change-Id: Iba31a6aaa9a9ec854ba36956d1b853dcf66a804d
The file was modifiedupf/UPF_Tests.ttcn
Commit d13980f024e692222105a173930fb53559182b85 by laforge
hnbgw: move vc_PFCP from ConnHdlr to test_CT

A logical relation between the PFCP emulation component (acting as
the UP function) and its ConnHdlr components is one-to-many.  It's
unusual for the ConnHdlr components to have their own emulation
component, especially when a testcase spawns many of them.  This
renders the dispatch function of the PFCP emulation component useless.

The HNBGW testsuite, which is using the PFCP emulation, does not
spawn more than one ConnHdlr component, so there was no problem
with that so far.  Let's move the PFCP emulation to test_CT, in
order to allow spawning more than one ConnHdlr and to make the
component hierarchy less confusing.

Change-Id: Iaf7a54c674f2dc549572d918beb6f355f48ee700
The file was modifiedlibrary/PFCP_Emulation.ttcn
The file was modifiedhnbgw/HNBGW_Tests.ttcn
Commit 4512e7db1c59a1e605b8de97f17d8b2c2e6e5829 by laforge
library/PFCP_Emulation: look up by SeqNr and SEID separately

This allows routing of PFCP PDUs without SEID by SeqNr.
An example of such a PDU is the Heartbeat Request.

Change-Id: Ic7912d944e94852a587993708d51439ec90f08cd
The file was modifiedlibrary/PFCP_Emulation.ttcn
Commit 2962d17079f26e2d874e82fe019adab34088d270 by laforge
library/PFCP_Emulation: fix routing of incoming PDUs

Before this patch, the PFCP_Emulation component was unable to route
incoming PDUs properly.  The problem is that the SeqNr and SEID were
neither stored nor updated whem sending outgoing PDUs, and thus
remained unbound all the time.

Change-Id: I8e62471915ab9a917ac960d3078e89669b5ac0ea
The file was modifiedlibrary/PFCP_Emulation.ttcn
Commit fbb13fa0db9db46bd15192f83e4268783a04472f by laforge
library/PFCP_Emulation: allow retrieving the Recovery Time Stamp

The Recovery Time Stamp is a mandatory IE for some PDUs, such as:

* Heartbeat Request/Response
* PFCP Association Setup Request/Response

Allow the PFCP_ConnHdlr components to retrieve its value, so that
they can include it in outgoing (and expect in incoming) PDUs.

Change-Id: I23eb2b4e309460f93183df83cdd35e46a93fbb94
The file was modifiedlibrary/PFCP_Emulation.ttcn
Commit 8b6d8aa23db9e623ed36e57b12011a42a022b33e by Vadim Yanitskiy
s1gw: rework f_pfcp_wait_assoc_setup() into f_ConnHdlr_pfcp_expect()

Generalize this function, so that it can be used to expect any
PFCP PDUs, not just the Association Setup.

Change-Id: If691cb9df72672eddfbafdd8e03ae09c81b1ce71
The file was modifieds1gw/S1GW_ConnHdlr.ttcn
Commit 7ad95e1cfb00d269069bd052c44a9cae9027f763 by Vadim Yanitskiy
s1gw: cache PFCP Recovery Timestamp in ConnHdlr

Change-Id: I71f1473b52c893245b70ef67599a2f33569c37dd
The file was modifieds1gw/S1GW_ConnHdlr.ttcn
Commit 9015183e0fc50a28ed719f6ef2530655f042af2f by Vadim Yanitskiy
s1gw: send the actual Recovery Time Stamp

Change-Id: I350d28289526d8264ef9b70e796ef76ee7fcbb4b
The file was modifieds1gw/S1GW_ConnHdlr.ttcn
Commit 67c5196f8b7b550d4c598d44333925313d611fa4 by Vadim Yanitskiy
s1gw: move t_ConnHdlrPars back to S1GW_Tests

This is a partial revert of 956bf05e7, which moved the t_ConnHdlrPars
from module S1GW_Tests to S1GW_ConnHdlr and added f_new_ConnHdlrPars().

The problem is that we want to assign module parameter values to
some fields of the ConnHdlrPars record, but TTCN-3 does not allow
to access module parameters of one module from another.  Having to
add more and more parameters to the proxy function
f_new_ConnHdlrPars() is highly inconvenient.

Change-Id: Ibc34d0219a616a239c0595e61a783f18fbc91b36
Related: 956bf05e7 "s1gw: Initial StatsD support"
The file was modifieds1gw/S1GW_ConnHdlr.ttcn
The file was modifieds1gw/S1GW_Tests.ttcn
Commit b68be3983fe821af306302cb2f8b5a6e5cf31f63 by Vadim Yanitskiy
s1gw: add TC_pfcp_heartbeat

Change-Id: Iadb24c2c346a62aa8b59d24475e71f683fc3dfe6
The file was modifieds1gw/expected-results.xml
The file was modifieds1gw/S1GW_Tests.ttcn
Commit 66c8af3dd4b4d98801537a4ce6e6528b8fc505df by Vadim Yanitskiy
library/s1ap: add more E-RAB related templates

Change-Id: I57df78e72bd2b070a6222c87f6faf4930bf1bdc5
The file was modifiedlibrary/s1ap/S1AP_Templates.ttcn
Commit 59cc891d06625c5a6a5261c4dfeb37694708b82f by Vadim Yanitskiy
library/PFCP_Templates: add missing receive templates

Change-Id: I534d23c4463f9de86bcc0a03b969f22f291ef8c1
The file was modifiedupf/UPF_Tests.ttcn
The file was modifiedlibrary/PFCP_Templates.ttcn
Commit 96e7bf4e978d5a0b0126efc34b8f2abd27ff43ee by laforge
testenv: on crash, wait until test stopped

When the IUT (or other test component) crashes, wait until
ttcn3-tcpdump-stop.sh has closed the pcap, as otherwise the last
buffered data may be lost.

Add a timeout of 2 minutes in case the test doesn't stop on its own.

I considered making this feature optional, but impatient users can still
immediately kill everything with ^C.

Change-Id: If49263869b1d46103813e9d06deff47f8ba72896
The file was modified_testenv/testenv/daemons.py
The file was modified_testenv/testenv/testsuite.py
The file was modifiedttcn3-tcpdump-start.sh
The file was modifiedttcn3-tcpdump-stop.sh
Commit 33dfe43fdc8d2ad320ef7bd742f5611aa5642620 by laforge
testenv: display current test name on ^C

When running a whole testsuite and getting a lot of output, it can be
useful to know which test was currently running when pressing ^C. Now
that we have a function that provides this information, show it to the
user.

Change-Id: I2eb639f47440447ac71027956b9c234323565956
The file was modified_testenv/testenv.py
Commit 74d8e4249de308128041462c59acbcbab5cfe2e4 by Vadim Yanitskiy
library/PFCP_Templates: fix missing fields in {ts,tr}_PFCP_Session_Est_Resp

Change-Id: Ib0887f1150f87746f9048f2944cd18c65224a4c7
The file was modifiedlibrary/PFCP_Templates.ttcn
The file was modifiedhnbgw/HNBGW_Tests.ttcn
Commit dc3ae29fcb09fd5c42c641d154f64fcb23cd7524 by Vadim Yanitskiy
library/s1ap: fix formatting in ts_S1AP_RABSetupItemCtxtSURes

Change-Id: Ie6a367841a927ee0b542ccff2d6eb4ab9e51ca12
The file was modifiedlibrary/s1ap/S1AP_Templates.ttcn
Commit e89e06f003c042557858c8fd3456fac1a098c6e9 by lynxis
sgsn/SGSN_Tests: remove a superfluous/wrong comment

Change-Id: Ib1cdea68e534bcf33aaeb8635d85f1f02cacb148
The file was modifiedsgsn/SGSN_Tests.ttcn
Commit 55bade7a4ef6891410187c90b17e8ee612fcba80 by Pau Espin Pedrol
asterisk: TC_ims_reregistration: Expect re-register using existing ipsec ctx

Change-Id: I8d100dbba6cddfa5f31437c9cdc8c48f1bbd5701
The file was modifiedasterisk/IMS_ConnectionHandler.ttcn
The file was modifiedasterisk/Asterisk_Tests.ttcn
Commit d7531335952fd993f56d7632f8c96bf505c765eb by Oliver Smith
testenv: get coredump + backtrace on crash

If the SUT or another test component crashes, check if a matching
coredump was registered in coredumpctl. If that is the case, then copy
it into the testdir and print + store the backtrace.

This solves the problem that it is especially tricky to get a good
backtrace when a component crashes inside a container. One needs to
grab the coredump from the host (usually handled by systemd-coredump,
we cannot override /proc/sys/kernel/core_pattern for containers so it
can't be handled in the container), then put the coredump into the
container and finally run gdb to get the backtrace inside the container
(where proper libraries and debug symbols are). This patch automates all
of these steps.

Pau requested this feature.

Related: OS#6494
Change-Id: I743c20968bda9b6d6fb9c2d23bef70ee11950761
The file was modified_testenv/testenv/daemons.py
The file was added_testenv/testenv/coredump.py
The file was modified_testenv/testenv/podman.py
The file was modified_testenv/data/podman/Dockerfile