Changes

tcap_as_loadshare: Clarify name of tcap asp selection function

Change-Id: I327ad282d2689be2bcf7d3da23026717923c9325

tcap_as_loadshare: Route msg based on TCAP route if cache lookup fails

Related: SYS#5423
Change-Id: I2fd9068be8a0a512b17c8bd2365aa87c266e5669

stp: Make TCAP tests more realistic

* Use a TCAP id outside of the TCAP routing range for the M3UA TCAP id.
* Test both directions IPA -> M3UA and M3UA -> IPA.
* Test TCAP routing with a different called address answering

Change-Id: Ia807c2b26e5188b700365650311cabc0f9095674
Related: SYS#5423

cosmetic: Rename TCAP Being->Begin

Change-Id: I19de8da6ee72a04365896c7fb7730be4f3ea01d4

smux: allow lchan lookup without remote address

Make rem_addr optional in osmux_lchan_find() by skipping
address comparison when it is NULL

This allows callers without socket-based addressing to
locate lchans by CID only (e.g. callback-based transports)

Change-Id: I7a275dad2353241c7b04cdb5d6cc1436a175c1f3

build: make sched_* usage optional

Add configure checks for sched_* APIs and guard usage of
sched_setscheduler with HAVE_SCHED_SETSCHEDULER.

This avoids build/runtime issues on platforms without
scheduler support.

Change-Id: I5af357c1e2074fa1e05ac6bc20af8535c474d906

trx: handle synchronous TRX control callbacks in provision FSM

Set *_sent flags before issuing TRX control commands to ensure
correct state when responses are delivered synchronously
(e.g. WebUSB callback path)

Also guard FSM state handling in st_open_poweroff() to avoid
duplicate transitions if state was already changed by a
synchronous response

No functional change for asynchronous (socket-based) backends

Change-Id: I50c95e3796839e9fe721bff6f96fed4c490c6018

abis: add helper to update BTS identity

Introduce abis_update_bts_info() to update site_id and bts_id in
bts_dev_info at runtime

This allows adjusting BTS identity after initialization,
e.g. from external control interfaces

Change-Id: Ibafd66a1e1dc42ad0063c412087bee10d212fe33

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

trx: factor out clock indication handling into helper

Extract clock indication handling logic into trx_clk_phy()
and reuse it from trx_clk_read_cb()

This separates core FN processing from socket-specific code
and prepares for alternative clock sources (e.g. non-UDP /
in-memory callbacks used in WebSDR/WASM builds).

No functional changes intended

Change-Id: I0e6b3e25da14ada4d0ef86fd4c4e6bbc12e447d2

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

trx: factor out FN timer processing into common helper

Extract core FN timer handling logic into trx_fn_timer_process()
to decouple it from timerfd-specific code.

This allows reusing the same scheduling logic for alternative
timer backends (e.g. WebSDR) while keeping timerfd integration
as a thin wrapper.

No functional changes intended.

Change-Id: Ia28cce57afb3dc97c8655e855ebd5ace7e096648

trx: factor out burst submission into helper

Extract TRXD burst submission logic from bts_sched_flush_buffers()
into trx_sched_submit_bursts()

No functional changes intended, this prepares for alternative
backends reusing the same scheduling logic

Change-Id: I7ce61bee4ca7ca50ef190982f1a0e4ce57c8c838

trx: split TRXD parsing from socket callback

Extract TRXD message parsing into trx_data_read() and
reuse it from trx_data_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I50ad8b6d2edfd91f2d569cd8badf5c036fbf87ed

trx: factor out TRXD data submission helper

Extract TRXD data sending into trx_data_submit_msg() and
reuse it from trx_if_send_burst()

This separates transport-specific I/O from message construction
and prepares for alternative backends

No functional changes intended

Change-Id: Ie89ac6efe2f5af0b4cbe5a259a18770b8d2de859

build: add WebSDR configure option and Emscripten detection

Detect Emscripten toolchain and introduce --with-websdr
configure option

When enabled, check for libosmo-trx-websdr and required
symbol modulateBits, failing configure if missing

Change-Id: Iff4b76e339e2873686c8c15885dd498458ceb546

trx: split TRX ctrl response parsing from socket callback

Extract TRX control response parsing logic into
trx_ctrl_parse_rsp() and call it from trx_ctrl_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I255c7d554aad3ac6df2c66716fae1c1fb512110e

bts: guard AMR RTP decoder stub with HAVE_LIBOSMOCODEC_STATIC

Disable dummy osmo_amr_rtp_dec() implementation when
linking against static libosmocodec to avoid symbol
conflicts

Apply to omldummy and virtual BTS models

Change-Id: Ie19ae2013893575a2f74a87af2151034abf57870

trx: factor out TRX ctrl message submission

Extract TRX control message sending logic into
trx_ctrl_submit_msg() and reuse it from trx_ctrl_send()

This separates message construction from transport-specific
submission and prepares for alternative backends

No functional changes intended

Change-Id: Ibbd50117db4531efbc374e43e3802f9f995362cd

transceiver: add optional Laurent burst LUT modulation (int16)

Introduce an optional Burst LUT optimization for Laurent-based burst
modulation, enabled via --with-burst-lut.

- Add precomputed Laurent LUT tables (float and int16 I/Q variants)
- Generate LUT at initialization (prepareBurstLUT)
- Use LUT-based modulation for sps=4 when enabled
- Add configure.ac option and USE_BURST_LUT define

This reduces runtime computation by replacing per-burst Laurent
processing with memcpy-based LUT lookups.

Change-Id: I1715c2d33dc55fe1c7be5c6e7259d378ea5e80b2

bts: guard AMR RTP decoder stub with HAVE_LIBOSMOCODEC_STATIC

Disable dummy osmo_amr_rtp_dec() implementation when
linking against static libosmocodec to avoid symbol
conflicts

Apply to omldummy and virtual BTS models

Change-Id: Ie19ae2013893575a2f74a87af2151034abf57870

trx: factor out clock indication handling into helper

Extract clock indication handling logic into trx_clk_phy()
and reuse it from trx_clk_read_cb()

This separates core FN processing from socket-specific code
and prepares for alternative clock sources (e.g. non-UDP /
in-memory callbacks used in WebSDR/WASM builds).

No functional changes intended

Change-Id: I0e6b3e25da14ada4d0ef86fd4c4e6bbc12e447d2

build: make sched_* usage optional

Add configure checks for sched_* APIs and guard usage of
sched_setscheduler with HAVE_SCHED_SETSCHEDULER.

This avoids build/runtime issues on platforms without
scheduler support.

Change-Id: I5af357c1e2074fa1e05ac6bc20af8535c474d906

trx: handle synchronous TRX control callbacks in provision FSM

Set *_sent flags before issuing TRX control commands to ensure
correct state when responses are delivered synchronously
(e.g. WebUSB callback path)

Also guard FSM state handling in st_open_poweroff() to avoid
duplicate transitions if state was already changed by a
synchronous response

No functional change for asynchronous (socket-based) backends

Change-Id: I50c95e3796839e9fe721bff6f96fed4c490c6018

trx: factor out burst submission into helper

Extract TRXD burst submission logic from bts_sched_flush_buffers()
into trx_sched_submit_bursts()

No functional changes intended, this prepares for alternative
backends reusing the same scheduling logic

Change-Id: I7ce61bee4ca7ca50ef190982f1a0e4ce57c8c838

trx: factor out FN timer processing into common helper

Extract core FN timer handling logic into trx_fn_timer_process()
to decouple it from timerfd-specific code.

This allows reusing the same scheduling logic for alternative
timer backends (e.g. WebSDR) while keeping timerfd integration
as a thin wrapper.

No functional changes intended.

Change-Id: Ia28cce57afb3dc97c8655e855ebd5ace7e096648

trx: split TRXD parsing from socket callback

Extract TRXD message parsing into trx_data_read() and
reuse it from trx_data_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I50ad8b6d2edfd91f2d569cd8badf5c036fbf87ed

trx: factor out TRXD data submission helper

Extract TRXD data sending into trx_data_submit_msg() and
reuse it from trx_if_send_burst()

This separates transport-specific I/O from message construction
and prepares for alternative backends

No functional changes intended

Change-Id: Ie89ac6efe2f5af0b4cbe5a259a18770b8d2de859

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

smux: allow lchan lookup without remote address

Make rem_addr optional in osmux_lchan_find() by skipping
address comparison when it is NULL

This allows callers without socket-based addressing to
locate lchans by CID only (e.g. callback-based transports)

Change-Id: I7a275dad2353241c7b04cdb5d6cc1436a175c1f3

trx: split TRX ctrl response parsing from socket callback

Extract TRX control response parsing logic into
trx_ctrl_parse_rsp() and call it from trx_ctrl_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I255c7d554aad3ac6df2c66716fae1c1fb512110e

abis: add helper to update BTS identity

Introduce abis_update_bts_info() to update site_id and bts_id in
bts_dev_info at runtime

This allows adjusting BTS identity after initialization,
e.g. from external control interfaces

Change-Id: Ibafd66a1e1dc42ad0063c412087bee10d212fe33

build: add WebSDR configure option and Emscripten detection

Detect Emscripten toolchain and introduce --with-websdr
configure option

When enabled, check for libosmo-trx-websdr and required
symbol modulateBits, failing configure if missing

Change-Id: Iff4b76e339e2873686c8c15885dd498458ceb546

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

trx: factor out TRX ctrl message submission

Extract TRX control message sending logic into
trx_ctrl_submit_msg() and reuse it from trx_ctrl_send()

This separates message construction from transport-specific
submission and prepares for alternative backends

No functional changes intended

Change-Id: Ibbd50117db4531efbc374e43e3802f9f995362cd

trx: split TRXD parsing from socket callback

Extract TRXD message parsing into trx_data_read() and
reuse it from trx_data_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I50ad8b6d2edfd91f2d569cd8badf5c036fbf87ed

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

bts: guard AMR RTP decoder stub with HAVE_LIBOSMOCODEC_STATIC

Disable dummy osmo_amr_rtp_dec() implementation when
linking against static libosmocodec to avoid symbol
conflicts

Apply to omldummy and virtual BTS models

Change-Id: Ie19ae2013893575a2f74a87af2151034abf57870

trx: factor out burst submission into helper

Extract TRXD burst submission logic from bts_sched_flush_buffers()
into trx_sched_submit_bursts()

No functional changes intended, this prepares for alternative
backends reusing the same scheduling logic

Change-Id: I7ce61bee4ca7ca50ef190982f1a0e4ce57c8c838

trx: factor out FN timer processing into common helper

Extract core FN timer handling logic into trx_fn_timer_process()
to decouple it from timerfd-specific code.

This allows reusing the same scheduling logic for alternative
timer backends (e.g. WebSDR) while keeping timerfd integration
as a thin wrapper.

No functional changes intended.

Change-Id: Ia28cce57afb3dc97c8655e855ebd5ace7e096648

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

trx: split TRX ctrl response parsing from socket callback

Extract TRX control response parsing logic into
trx_ctrl_parse_rsp() and call it from trx_ctrl_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I255c7d554aad3ac6df2c66716fae1c1fb512110e

abis: add helper to update BTS identity

Introduce abis_update_bts_info() to update site_id and bts_id in
bts_dev_info at runtime

This allows adjusting BTS identity after initialization,
e.g. from external control interfaces

Change-Id: Ibafd66a1e1dc42ad0063c412087bee10d212fe33

build: make sched_* usage optional

Add configure checks for sched_* APIs and guard usage of
sched_setscheduler with HAVE_SCHED_SETSCHEDULER.

This avoids build/runtime issues on platforms without
scheduler support.

Change-Id: I5af357c1e2074fa1e05ac6bc20af8535c474d906

build: remove unused sys/timerfd.h include

The header is not used in scheduler_trx.c and breaks builds on
platforms where timerfd is not available (e.g. Web/emscripten).

Remove the unused include.

Change-Id: Id5a700c865f984e8f99199a1e365bb7f1462030a

smux: allow lchan lookup without remote address

Make rem_addr optional in osmux_lchan_find() by skipping
address comparison when it is NULL

This allows callers without socket-based addressing to
locate lchans by CID only (e.g. callback-based transports)

Change-Id: I7a275dad2353241c7b04cdb5d6cc1436a175c1f3

trx: factor out TRXD data submission helper

Extract TRXD data sending into trx_data_submit_msg() and
reuse it from trx_if_send_burst()

This separates transport-specific I/O from message construction
and prepares for alternative backends

No functional changes intended

Change-Id: Ie89ac6efe2f5af0b4cbe5a259a18770b8d2de859

trx: handle synchronous TRX control callbacks in provision FSM

Set *_sent flags before issuing TRX control commands to ensure
correct state when responses are delivered synchronously
(e.g. WebUSB callback path)

Also guard FSM state handling in st_open_poweroff() to avoid
duplicate transitions if state was already changed by a
synchronous response

No functional change for asynchronous (socket-based) backends

Change-Id: I50c95e3796839e9fe721bff6f96fed4c490c6018

trx: factor out clock indication handling into helper

Extract clock indication handling logic into trx_clk_phy()
and reuse it from trx_clk_read_cb()

This separates core FN processing from socket-specific code
and prepares for alternative clock sources (e.g. non-UDP /
in-memory callbacks used in WebSDR/WASM builds).

No functional changes intended

Change-Id: I0e6b3e25da14ada4d0ef86fd4c4e6bbc12e447d2

trx: factor out TRX ctrl message submission

Extract TRX control message sending logic into
trx_ctrl_submit_msg() and reuse it from trx_ctrl_send()

This separates message construction from transport-specific
submission and prepares for alternative backends

No functional changes intended

Change-Id: Ibbd50117db4531efbc374e43e3802f9f995362cd

build: add WebSDR configure option and Emscripten detection

Detect Emscripten toolchain and introduce --with-websdr
configure option

When enabled, check for libosmo-trx-websdr and required
symbol modulateBits, failing configure if missing

Change-Id: Iff4b76e339e2873686c8c15885dd498458ceb546

convolve: avoid function pointer casts by adding wrapper functions

Introduce _base_convolve_*_void() wrappers matching the expected
function pointer signatures and use them instead of casting
_base_convolve_* functions to (void *).

This removes unsafe function pointer casts and improves
type safety, which is required for stricter toolchains and
non-native targets (e.g. WebAssembly)

Change-Id: Idecb118be285eb3e4691d1761d0d8fa24fd80c75

trx: add JSON serialization for 'show trx'

Introduce trx_to_json() helper to serialize the output of
'show trx' into JSON format.

Also initialize talloc/VTY context in websdr entrypoint to
allow accessing TRX configuration programmatically.

Intended for external consumers (e.g. WebSDR frontend).

Change-Id: Icbfb754580daa6db9b5646cc6698a6a6ee1c1610

device: add WebSDR radio backend

Introduce a new optional WebSDR device backend, enabled via
--with-websdr.

Add WebSDRDevice implementation and build integration, providing
a RadioDevice interface backed by callback hooks for control and
sample I/O.

Intended for Web-based deployments where osmo-trx interacts with
SDR hardware via a WebSDR/WebUSB frontend.

Change-Id: Ie459cbd70388dd8ff5b89221d30770bab0bd9014

build: detect pthread_setname_np and sched_* APIs, guard usage

Add configure checks for pthread_setname_np and sched_* functions.
Wrap their usage with HAVE_* guards to avoid build failures on
platforms where they are unavailable.

Return -ENOTSUP for unsupported scheduler operations.

Improves portability across non-Linux and restricted environments
(e.g. WebAssembly).

Change-Id: Ic3f1492544439d88c5e6683330c6696de1c2f85b

transceiver: add optional WebSDR device support

- Add support for WebSDR devices enabled via the --with-websdr
configure flag.
- add libosmo-trx-websdr library and pkg-config file
- update .gitignore for wasm and pkg-config artifacts

Change-Id: Ia0d340c323c2eea28fbe82601ba0af7cfbd68f6d

nokia: respond to BTS_COMMISS_TEST_REQ

In some cases, Flexi Multiradio BTS sends us BTS_COMMISS_TEST_REQ
during its init process, and expects us to respond with
BTS_COMMISS_TEST_COMPL, indicating that we did the requested
commissioning tests.  While we don't do any actual tests currently,
and we have not yet reached an understanding of the exact conditions
as to when the BTS makes this query of us, if it does send us
BTS_COMMISS_TEST_REQ, we need to respond with BTS_COMMISS_TEST_COMPL
to make it happy.

Change-Id: I0f2710c3cdb38a9e5b37e40ca5584237a730a902

nokia vty: change help strings away from "*Site"

Support for Nokia BTS in OsmoBSC is being expanded from just *Site
family to also include Flexi Multiradio - therefore, help strings
and error messages in vty should be changed accordingly.

FIXME: names of Nokia-specific vty settings still begin with
nokia_site, ditto for the name of the BTS model.  TODO: decide
on a proper long-term solution.

Change-Id: I55556b7b5aa5bc6efcbee815e9011497a933697c

nokia: make Rx diversity configurable

The config structure sent to the BTS with BTS_CONF_DATA includes
an IE that enables or disables Rx diversity.  In the original code
this setting was hard-coded to enabled state; make it configurable
via vty.

Change-Id: I73bd91b717a8c8b338bacb6ed9db73bb07245c12

nokia skip-reset: make consistent with other vty settings

All other Nokia-specific vty settings use is_nokia_bts() helper
function, while the code for skip-reset directly compares
bts->type against GSM_BTS_TYPE_NOKIA_SITE.  Change to use the
helper function: it is consistent with other vty settings,
and avoids one more place that erroneously refers to *Site
instead of all Nokia BTS.

Change-Id: Ie8cdac51f82185e2d4f37a8da00acba633670052

nokia: add support for Flexi Multiradio

Previous support existed for Nokia MetroSite, UltraSite and InSite
models, resulting in BTS model name "nokia_site".  However, there
exist newer Nokia BTS models that also support E1 Abis and can be
supported by OsmoBSC with relatively small changes: Flexi EDGE,
Flexi Multiradio and Flexi Multiradio 10.  Out of these newer
models, American 2G Cooperative (the present patch author's current
organization) has only Flexi MR at the present time - hence this
model shall be the first "new" Nokia BTS to be supported.

The first change required in order to support Flexi MR is ability
to receive long OML messages that have been segmented by the BTS:
the new BTS sends a much longer BTS_CONF_REQ message, and we have
to handle it correctly.

Change-Id: Icd429552c3b5e90d6c5d3ae5f52afd6550e3cebc

nokia bts-reset-timer: raise the maximum to 300 s

The previous vty tuning range for this timer (allowing up to 100 s)
is not sufficient for Flexi Multiradio BTS, where we need 120 s
minimum or 130 s for some margin.  Raise the upper limit for vty
config of this parameter to 5 min, which should be long enough
to not require further revisiting.

Change-Id: Ie5b784c9cce56e27800084471a7fb0b867cc0c69

rsl: don't send legacy Chan Ident IE for Nokia BTS

The description of CHAN ACTIV command message in TS 48.058
section 8.4.1 lists Channel Identification as an optional IE,
with a note that reads "Included if compatibility with phase1
is required."  This note may have been misinterpreted as
referring to compatibility with Phase 1 phones - but since
this network-internal Abis detail is invisible to phones,
the real intent of GSM spec authors here must have been
compatibility with Phase 1 BTS.

Previous versions of OsmoBSC would always include this IE,
and it appears that all previously supported BTS models
would simply ignore it.  However, newer Nokia BTS models
(observed on Flexi Multiradio at least) dislike this IE
and return CHAN ACTIV NACK with an optional IE error cause
if it is included.

To avoid changing OsmoBSC behavior for other users, let us
conditionalize removal of this legacy IE on Nokia BTS for now.
This change has been tested on InSite, MetroSite and Flexi
Multiradio - no adverse effects on previously supported
*Site models.

Change-Id: Ibce7a841907ac87ae378ded3ddb174d38b70b7b7

Cosmetic: bts_nokia_site: fix comment formatting

Ensure each line of a multiline comment starts with " * ".

Related: https://gerrit.osmocom.org/c/osmo-bsc/+/42712/comment/6aa80188_f56f2265/
Change-Id: I83ed16ec09fb66076dc2ba6a49ef272657b09407

lint/checkpatch: ignore MULTILINE_DEREFERENCE

This linting rule gives false positives, e.g. for this snippet:

static int abis_nm_rcvmsg_fom(struct e1inp_sign_link *sign_link,
      uint8_t *l3_msg, unsigned l3_msg_len)

It complains with:
src/osmo-bsc/bts_nokia_site.c:1785: WARNING:MULTILINE_DEREFERENCE: Avoid multiple line dereference - prefer 'run.static'

Related: https://gerrit.osmocom.org/c/osmo-bsc/+/42712/comment/fd3a38ef_a59f616f/
Change-Id: If2a1972fe333f51d534a924fd5ac1c1f7eb65b01

nokia: respond to BTS_COMMISS_TEST_REQ

In some cases, Flexi Multiradio BTS sends us BTS_COMMISS_TEST_REQ
during its init process, and expects us to respond with
BTS_COMMISS_TEST_COMPL, indicating that we did the requested
commissioning tests.  While we don't do any actual tests currently,
and we have not yet reached an understanding of the exact conditions
as to when the BTS makes this query of us, if it does send us
BTS_COMMISS_TEST_REQ, we need to respond with BTS_COMMISS_TEST_COMPL
to make it happy.

Change-Id: I0f2710c3cdb38a9e5b37e40ca5584237a730a902

nokia bts-reset-timer: raise the maximum to 300 s

The previous vty tuning range for this timer (allowing up to 100 s)
is not sufficient for Flexi Multiradio BTS, where we need 120 s
minimum or 130 s for some margin.  Raise the upper limit for vty
config of this parameter to 5 min, which should be long enough
to not require further revisiting.

Change-Id: Ie5b784c9cce56e27800084471a7fb0b867cc0c69

nokia: make Rx diversity configurable

The config structure sent to the BTS with BTS_CONF_DATA includes
an IE that enables or disables Rx diversity.  In the original code
this setting was hard-coded to enabled state; make it configurable
via vty.

Change-Id: I73bd91b717a8c8b338bacb6ed9db73bb07245c12

nokia vty: change help strings away from "*Site"

Support for Nokia BTS in OsmoBSC is being expanded from just *Site
family to also include Flexi Multiradio - therefore, help strings
and error messages in vty should be changed accordingly.

FIXME: names of Nokia-specific vty settings still begin with
nokia_site, ditto for the name of the BTS model.  TODO: decide
on a proper long-term solution.

Change-Id: I55556b7b5aa5bc6efcbee815e9011497a933697c

nokia: add support for Flexi Multiradio

Previous support existed for Nokia MetroSite, UltraSite and InSite
models, resulting in BTS model name "nokia_site".  However, there
exist newer Nokia BTS models that also support E1 Abis and can be
supported by OsmoBSC with relatively small changes: Flexi EDGE,
Flexi Multiradio and Flexi Multiradio 10.  Out of these newer
models, American 2G Cooperative (the present patch author's current
organization) has only Flexi MR at the present time - hence this
model shall be the first "new" Nokia BTS to be supported.

The first change required in order to support Flexi MR is ability
to receive long OML messages that have been segmented by the BTS:
the new BTS sends a much longer BTS_CONF_REQ message, and we have
to handle it correctly.

Change-Id: Icd429552c3b5e90d6c5d3ae5f52afd6550e3cebc

nokia skip-reset: make consistent with other vty settings

All other Nokia-specific vty settings use is_nokia_bts() helper
function, while the code for skip-reset directly compares
bts->type against GSM_BTS_TYPE_NOKIA_SITE.  Change to use the
helper function: it is consistent with other vty settings,
and avoids one more place that erroneously refers to *Site
instead of all Nokia BTS.

Change-Id: Ie8cdac51f82185e2d4f37a8da00acba633670052

rsl: don't send legacy Chan Ident IE for Nokia BTS

The description of CHAN ACTIV command message in TS 48.058
section 8.4.1 lists Channel Identification as an optional IE,
with a note that reads "Included if compatibility with phase1
is required."  This note may have been misinterpreted as
referring to compatibility with Phase 1 phones - but since
this network-internal Abis detail is invisible to phones,
the real intent of GSM spec authors here must have been
compatibility with Phase 1 BTS.

Previous versions of OsmoBSC would always include this IE,
and it appears that all previously supported BTS models
would simply ignore it.  However, newer Nokia BTS models
(observed on Flexi Multiradio at least) dislike this IE
and return CHAN ACTIV NACK with an optional IE error cause
if it is included.

To avoid changing OsmoBSC behavior for other users, let us
conditionalize removal of this legacy IE on Nokia BTS for now.
This change has been tested on InSite, MetroSite and Flexi
Multiradio - no adverse effects on previously supported
*Site models.

Change-Id: Ibce7a841907ac87ae378ded3ddb174d38b70b7b7

esim/http_json_api.py: support text/plain response Content-Type

Allow returning text/plain Content-Types as 'data' output argument.

So far, all the esim/http_json_api functions require a JSON response.
However, a specific vendor has a list function where the request is JSON
but the response is text/plain CSV data. Allow and return in a dict.

Change-Id: Iba6e4cef1048b376050a435a900c0f395655a790
Jenkins: skip-card-test

nokia bts-reset-timer: raise the maximum to 300 s

The previous vty tuning range for this timer (allowing up to 100 s)
is not sufficient for Flexi Multiradio BTS, where we need 120 s
minimum or 130 s for some margin.  Raise the upper limit for vty
config of this parameter to 5 min, which should be long enough
to not require further revisiting.

Change-Id: Ie5b784c9cce56e27800084471a7fb0b867cc0c69

rsl: don't send legacy Chan Ident IE for Nokia BTS

The description of CHAN ACTIV command message in TS 48.058
section 8.4.1 lists Channel Identification as an optional IE,
with a note that reads "Included if compatibility with phase1
is required."  This note may have been misinterpreted as
referring to compatibility with Phase 1 phones - but since
this network-internal Abis detail is invisible to phones,
the real intent of GSM spec authors here must have been
compatibility with Phase 1 BTS.

Previous versions of OsmoBSC would always include this IE,
and it appears that all previously supported BTS models
would simply ignore it.  However, newer Nokia BTS models
(observed on Flexi Multiradio at least) dislike this IE
and return CHAN ACTIV NACK with an optional IE error cause
if it is included.

To avoid changing OsmoBSC behavior for other users, let us
conditionalize removal of this legacy IE on Nokia BTS for now.
This change has been tested on InSite, MetroSite and Flexi
Multiradio - no adverse effects on previously supported
*Site models.

Change-Id: Ibce7a841907ac87ae378ded3ddb174d38b70b7b7

nokia vty: change help strings away from "*Site"

Support for Nokia BTS in OsmoBSC is being expanded from just *Site
family to also include Flexi Multiradio - therefore, help strings
and error messages in vty should be changed accordingly.

Change-Id: I55556b7b5aa5bc6efcbee815e9011497a933697c

nokia: add support for Flexi Multiradio

Previous support existed for Nokia MetroSite, UltraSite and InSite
models, resulting in BTS model name "nokia_site".  However, there
exist newer Nokia BTS models that also support E1 Abis and can be
supported by OsmoBSC with relatively small changes: Flexi EDGE,
Flexi Multiradio and Flexi Multiradio 10.  Out of these newer
models, American 2G Cooperative (the present patch author's current
organization) has only Flexi MR at the present time - hence this
model shall be the first "new" Nokia BTS to be supported.

The first change required in order to support Flexi MR is ability
to receive long OML messages that have been segmented by the BTS:
the new BTS sends a much longer BTS_CONF_REQ message, and we have
to handle it correctly.

Change-Id: Icd429552c3b5e90d6c5d3ae5f52afd6550e3cebc

nokia: make Rx diversity configurable

The config structure sent to the BTS with BTS_CONF_DATA includes
an IE that enables or disables Rx diversity.  In the original code
this setting was hard-coded to enabled state; make it configurable
via vty.

Change-Id: I73bd91b717a8c8b338bacb6ed9db73bb07245c12

nokia: respond to BTS_COMMISS_TEST_REQ

In some cases, Flexi Multiradio BTS sends us BTS_COMMISS_TEST_REQ
during its init process, and expects us to respond with
BTS_COMMISS_TEST_COMPL, indicating that we did the requested
commissioning tests.  While we don't do any actual tests currently,
and we have not yet reached an understanding of the exact conditions
as to when the BTS makes this query of us, if it does send us
BTS_COMMISS_TEST_REQ, we need to respond with BTS_COMMISS_TEST_COMPL
to make it happy.

Change-Id: I0f2710c3cdb38a9e5b37e40ca5584237a730a902

fw: add git blame ignore file

Reformatting should not pollute blame, so ignore those revs.

Change-Id: I35b86f52d491c9e28ab0af9e3e3d4fd5f6dbd119

fw source: force reformat once and for all

This commit applies the rules to all files that matter
(obv excluding asf4 code) since it is impossible to modify code
because the code never used the code style enforced by the
linter/clang-format and
1) changing code
2) reformatting code
3) extracting the reformatting changes for the modified lines as a
separate commit
4) commiting standalone reformatting changes
4) commiting the actual code change on top
is a workflow that takes forever and is completely ridiculous because
the whole point of formatter rules is to apply them.

This needs a second commit with .git-blame-ignore-revs to not pollute blame.

Change-Id: Iacc086bb566551225e7a21b639a1ad2ec257484f

formatting: update .clang-format

Updates the formatting rules derived from linux kernel
098b6e44cbaa2d526d06af90c862d13fb414a0ec

At the same time tell clang-format to keep the bespoke formatting
used for the fsm and usb descs.

Change-Id: I1fd4d38897946693b5495fa4d0b61925eb9d07f7

fw source: force reformat once and for all

This commit applies the rules to all files that matter
(obv excluding asf4 code) since it is impossible to modify code
because the code never used the code style enforced by the
linter/clang-format and
1) changing code
2) reformatting code
3) extracting the reformatting changes for the modified lines as a
separate commit
4) commiting standalone reformatting changes
4) commiting the actual code change on top
is a workflow that takes forever and is completely ridiculous because
the whole point of formatter rules is to apply them.

This needs a second commit with .git-blame-ignore-revs to not pollute blame.

Change-Id: Iacc086bb566551225e7a21b639a1ad2ec257484f

fw: add git blame ignore file

Reformatting should not pollute blame, so ignore those revs.

Change-Id: I35b86f52d491c9e28ab0af9e3e3d4fd5f6dbd119

transceiver: add optional Laurent burst LUT modulation (int16)

Introduce an optional Burst LUT optimization for Laurent-based burst
modulation, enabled via --with-burst-lut.

- Add precomputed Laurent LUT tables (float and int16 I/Q variants)
- Generate LUT at initialization (prepareBurstLUT)
- Use LUT-based modulation for sps=4 when enabled
- Add configure.ac option and USE_BURST_LUT define

This reduces runtime computation by replacing per-burst Laurent
processing with memcpy-based LUT lookups.

Change-Id: I1715c2d33dc55fe1c7be5c6e7259d378ea5e80b2

Fix: Only run pcscd with "--diable-polkit" if supported

For example, if bankd tests run on Debian Bookwork, pcscd does not have
the "--disable-polkit" option. This option is not required, because it
does not support Polit anyway.

Only add "--disable-polkit" to the command line if supported by pcscd.

Change-Id: Ib2cdf7fcd698c33810ebfbb1db2fa666b9d7dbb3

jobs/ttcn3-testsuites-testenv: fix rm apt partial

Fix that testenv jobs may fail to remove the cache at the beginning:

  rm: cannot remove '_cache/podman/var-lib-apt-debian-trixie/lists/partial': Permission denied

This happens when the partial directory still exists:

  $ ls -lah ~/jenkins/workspace/ttcn3-bsc-test-sccplite/_cache/podman/var-lib-apt-debian-trixie/lists
  total 12K
  drwxr-xr-x 3 osmocom-build osmocom-build 4.0K Apr 27 08:48 .
  drwxr-xr-x 3 osmocom-build osmocom-build 4.0K Apr 26 08:48 ..
  drwx------ 2        100041 osmocom-build 4.0K Apr 26 08:48 partial

Which in turn is the result of "apt update" failing to download files in
an earlier job:

  E: Failed to fetch https://downloads.osmocom.org/packages/osmocom:/nightly/Debian_13/./Packages.gz  File has unexpected size (210767 != 210491). Mirror sync in progress? [IP: 148.251.236.141 443]
     Hashes of expected file:
      - Filesize:210491 [weak]
      - SHA256:a0cdea838b724880f455d68eb433f6331a7d4113c2eb677ca2f7f6dbdf9a72e1
      - SHA1:71780896257ca550e51b95e77fee4c63fcc9980c [weak]
      - MD5Sum:531a2d49d4e9f3ff121ca6aa5be602fc [weak]
     Release file created at: Sun, 26 Apr 2026 08:48:04 +0000
  E: Some index files failed to download. They have been ignored, or old ones used instead.

Change-Id: I35b9e42b754805b42f0a868a5d0c20c1f1c01731

build: add WebSDR configure option and Emscripten detection

Detect Emscripten toolchain and introduce --with-websdr
configure option

When enabled, check for libosmo-trx-websdr and required
symbol modulateBits, failing configure if missing

Change-Id: Iff4b76e339e2873686c8c15885dd498458ceb546

trx: handle synchronous TRX control callbacks in provision FSM

Set *_sent flags before issuing TRX control commands to ensure
correct state when responses are delivered synchronously
(e.g. WebUSB callback path)

Also guard FSM state handling in st_open_poweroff() to avoid
duplicate transitions if state was already changed by a
synchronous response

No functional change for asynchronous (socket-based) backends

Change-Id: I50c95e3796839e9fe721bff6f96fed4c490c6018

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

trx: factor out TRX ctrl message submission

Extract TRX control message sending logic into
trx_ctrl_submit_msg() and reuse it from trx_ctrl_send()

This separates message construction from transport-specific
submission and prepares for alternative backends

No functional changes intended

Change-Id: Ibbd50117db4531efbc374e43e3802f9f995362cd

smux: allow lchan lookup without remote address

Make rem_addr optional in osmux_lchan_find() by skipping
address comparison when it is NULL

This allows callers without socket-based addressing to
locate lchans by CID only (e.g. callback-based transports)

Change-Id: I7a275dad2353241c7b04cdb5d6cc1436a175c1f3

trx: split TRX ctrl response parsing from socket callback

Extract TRX control response parsing logic into
trx_ctrl_parse_rsp() and call it from trx_ctrl_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I255c7d554aad3ac6df2c66716fae1c1fb512110e

trx: factor out clock indication handling into helper

Extract clock indication handling logic into trx_clk_phy()
and reuse it from trx_clk_read_cb()

This separates core FN processing from socket-specific code
and prepares for alternative clock sources (e.g. non-UDP /
in-memory callbacks used in WebSDR/WASM builds).

No functional changes intended

Change-Id: I0e6b3e25da14ada4d0ef86fd4c4e6bbc12e447d2

trx: factor out FN timer processing into common helper

Extract core FN timer handling logic into trx_fn_timer_process()
to decouple it from timerfd-specific code.

This allows reusing the same scheduling logic for alternative
timer backends (e.g. WebSDR) while keeping timerfd integration
as a thin wrapper.

No functional changes intended.

Change-Id: Ia28cce57afb3dc97c8655e855ebd5ace7e096648

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

bts: guard AMR RTP decoder stub with HAVE_LIBOSMOCODEC_STATIC

Disable dummy osmo_amr_rtp_dec() implementation when
linking against static libosmocodec to avoid symbol
conflicts

Apply to omldummy and virtual BTS models

Change-Id: Ie19ae2013893575a2f74a87af2151034abf57870

abis: add helper to update BTS identity

Introduce abis_update_bts_info() to update site_id and bts_id in
bts_dev_info at runtime

This allows adjusting BTS identity after initialization,
e.g. from external control interfaces

Change-Id: Ibafd66a1e1dc42ad0063c412087bee10d212fe33

trx: split TRXD parsing from socket callback

Extract TRXD message parsing into trx_data_read() and
reuse it from trx_data_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I50ad8b6d2edfd91f2d569cd8badf5c036fbf87ed

build: make timerfd usage optional

Check for sys/timerfd.h in configure and guard its usage
with HAVE_SYS_TIMERFD_H.

This avoids build failures on platforms where timerfd is
not available.

Change-Id: Id5a700c865f984e8f99199a1e365bb7f1462030a

trx: factor out burst submission into helper

Extract TRXD burst submission logic from bts_sched_flush_buffers()
into trx_sched_submit_bursts()

No functional changes intended, this prepares for alternative
backends reusing the same scheduling logic

Change-Id: I7ce61bee4ca7ca50ef190982f1a0e4ce57c8c838

build: make sched_* usage optional

Add configure checks for sched_* APIs and guard usage of
sched_setscheduler with HAVE_SCHED_SETSCHEDULER.

This avoids build/runtime issues on platforms without
scheduler support.

Change-Id: I5af357c1e2074fa1e05ac6bc20af8535c474d906

trx: factor out TRXD data submission helper

Extract TRXD data sending into trx_data_submit_msg() and
reuse it from trx_if_send_burst()

This separates transport-specific I/O from message construction
and prepares for alternative backends

No functional changes intended

Change-Id: Ie89ac6efe2f5af0b4cbe5a259a18770b8d2de859

vty: fix osmo_cpu_sched_vty_init() prototype

Change return type of osmo_cpu_sched_vty_init() from void to int
to match its implementation.

This fixes a prototype mismatch rejected by stricter compilers
(e.g. Emscripten).

Change-Id: Ia86e8063178ad82de098c2ff693cfbcd7e9f2192

vty: make CPU scheduling optional based on platform support

Add configure checks for sched_* APIs and guard their usage
in cpu_sched_vty with HAVE_SCHED_* macros.

If scheduler functions are not available, return -ENOTSUP
instead of failing at build time.

This improves portability on platforms lacking full sched
support (e.g. non-Linux or restricted environments).

Change-Id: Ic5b7e39fac16531d370cb81f769ba87fef18cb81

rspro_util: asn1 decode: fix memleak when decoding fails

Change-Id: I071343d0134dbffb1bb71ea26a5b514be1b5bb27

Fix: Prevent bankd from exiting upon SIGPIPE

If a client disconnects from a worker thread while the worker is
responding to a to a request from that client, the socket is closed and
SIGPIPE is sent to the worker, causing the application to exit.

To prevent this, all workers have a dummy handler to prevent this. Now
the worker can handle the closing of the socket.

Change-Id: I13d6e9da48d12f93c00bd2d021789bde71aca7cf

Remove unconditional assignment of asn_debug = 0

It is now the default and there is no code to set it.
Remove the assignment

Change-Id: I114e9773a1d0aa3c16990a946cf91c69b9c2498a

asn1: move asn_debug into librspro

To allow compiling the asn code with `-DEMIT_ASN_DEBUG=1`.
Otherwise it will fail with an unresolved symbol because
librspro is compiled as shared-object.
Further initialize it as disable (0).

Change-Id: Icb03e8905f3b38ecebfcf294727a01f994511cce

Remove unconditional assignment of asb_debug = 0

It is now the default and there is no code to set it.
Remove the assignment

Change-Id: I114e9773a1d0aa3c16990a946cf91c69b9c2498a

asn1: move asn_debug into librspro

To allow compiling the asn code with `-DEMIT_ASN_DEBUG=1`.
Otherwise it will fail with an unresolved symbol because
librspro is compiled as shared-object.
Further initialize it as disable (0).

Change-Id: Icb03e8905f3b38ecebfcf294727a01f994511cce

rspro_util: asn1 decode: fix memleak when decoding fails

Change-Id: I071343d0134dbffb1bb71ea26a5b514be1b5bb27

asn1: ber_decoder: disable stack size check on ctx = NULL

The stack size check is triggering when using asan or
when compile with -O1.
It is unclear why this is problematic.
This is a not a leak.

Related: OS#7002
Change-Id: Id99531f62ccdf73497bb08483ca6adbabe1a7cd7

Fix: Prevent race conditions when accessing slotmap in bankd

All threads (main and workers) can use slotmap_by_bank() and
slotmap_by_client() to get a mapping entry from the slot map. While
working with this entry (reading values), it could be deleted by a
different thread.

For example, if a client connects to a worker, the worker will check if
there is an existing mapping between a reader and this client. If it
exists, slotmap_by_client() will return a pointer. If the mapping is
deleted at this time by the server, the worker uses a pointer to a
mapping entry that has just been freed.

Change-Id: I3464726f37beb7c47b4e1f00c018ffa4f3948906

Fix: Prevent bankd from exiting upon SIGPIPE

If a client disconnects from a worker thread while the worker is
responding to a to a request from that client, the socket is closed and
SIGPIPE is sent to the worker, causing the application to exit.

To prevent this, all workers have a dummy handler to prevent this. Now
the worker can handle the closing of the socket.

Change-Id: I13d6e9da48d12f93c00bd2d021789bde71aca7cf

Fix: Remove slot mapping at bankd when client disconnects

If a client disconnects before removing the slot mapping, the worker
must undefine its 'bank_id' and 'slot_nr' and must change its state to
'UNMAPPED'.

send_signal_to_worker() searches for a worker that has a given bank and
slot. If a client re-connects to a different worker and if the bank and
slot would be still assigned to the old worker, the old worker could
receive signals when assigning or removing slot mapping. The new worker
would not be mapped.

Change-Id: I2fd03490e2506c55104309a0ef952389119023b8

Fix: Prevent bankd from exiting upon SIGPIPE

If a client disconnects from a worker thread while the worker is
responding to a to a request from that client, the socket is closed and
SIGPIPE is sent to the worker, causing the application to exit.

To prevent this, all workers have a dummy handler to prevent this. Now
the worker can handle the closing of the socket.

Change-Id: I13d6e9da48d12f93c00bd2d021789bde71aca7cf

ts_51_011/EF.SMSP: fix handling of 'alpha_id' field

The field 'alpha_id' is technically not an optional field, even though
the specification describes it as optional. Once the card manufacturer
decides that the field should be present, it must be always present and
vice versa.

(see code comment for a more detailed description)

Related: SYS#7765
Change-Id: I0ec99b2648b22c56f9145345e4cd8776f9217701

pySim-prog/cards: fix programming of EF.SMSP

The legacy code found in legacy/cards.py does not use the modern
construct based encoder (pySim-read uses it). The card classes either
use their own implementation of update_smsp or use the generic method
provided by the SimCard class. The latter one is true for FairwavesSIM
and WavemobileSim.

Unfortunately the implementation found in the SimCard is wrong. It
adds padding at the end of the file instead of the beginning. This
completely messes up the contents of EF.SMSP for the cards using this
method. To fix this, let's use the leftpad feature provided by
the update_record. This will ensure a correct alignment of the file
contents.

Related: SYS#7765
Change-Id: Ie112418f1f1461762d61365d3863181ca6be7245

pySim/transport: fix GET RESPONSE behaviour

The current behavior we implement in the method __send_apdu_T0 is
incomplete. Some details discussed in ETSI TS 102 221,
section 7.3.1.1.4, clause 4 seem to be not fully implemented. We
may also end up sending a GET RESPONSE in other APDU cases than
case 4 (the only case that uses the GET RESPONSE command).

Related: OS#6970
Change-Id: I26f0566af0cdd61dcc97f5f502479dc76adc37cc

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

build: make linux/if_packet.h usage optional

Check for presence of linux/if_packet.h in configure and
guard its usage with HAVE_LINUX_IF_PACKET_H.

This avoids build failures on non-Linux platforms or
environments where the header is unavailable.

Change-Id: I8a02b0676e65fa4b6191073c84e0646a2e67a010

trx: factor out TRXD data submission helper

Extract TRXD data sending into trx_data_submit_msg() and
reuse it from trx_if_send_burst()

This separates transport-specific I/O from message construction
and prepares for alternative backends

No functional changes intended

Change-Id: Ie89ac6efe2f5af0b4cbe5a259a18770b8d2de859

trx: handle synchronous TRX control callbacks in provision FSM

Set *_sent flags before issuing TRX control commands to ensure
correct state when responses are delivered synchronously
(e.g. WebUSB callback path)

Also guard FSM state handling in st_open_poweroff() to avoid
duplicate transitions if state was already changed by a
synchronous response

No functional change for asynchronous (socket-based) backends

Change-Id: I50c95e3796839e9fe721bff6f96fed4c490c6018

trx: split TRXD parsing from socket callback

Extract TRXD message parsing into trx_data_read() and
reuse it from trx_data_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I50ad8b6d2edfd91f2d569cd8badf5c036fbf87ed

bts: guard AMR RTP decoder stub with HAVE_LIBOSMOCODEC_STATIC

Disable dummy osmo_amr_rtp_dec() implementation when
linking against static libosmocodec to avoid symbol
conflicts

Apply to omldummy and virtual BTS models

Change-Id: Ie19ae2013893575a2f74a87af2151034abf57870

smux: allow lchan lookup without remote address

Make rem_addr optional in osmux_lchan_find() by skipping
address comparison when it is NULL

This allows callers without socket-based addressing to
locate lchans by CID only (e.g. callback-based transports)

Change-Id: I7a275dad2353241c7b04cdb5d6cc1436a175c1f3

trx: factor out TRX ctrl message submission

Extract TRX control message sending logic into
trx_ctrl_submit_msg() and reuse it from trx_ctrl_send()

This separates message construction from transport-specific
submission and prepares for alternative backends

No functional changes intended

Change-Id: Ibbd50117db4531efbc374e43e3802f9f995362cd

trx: factor out FN timer processing into common helper

Extract core FN timer handling logic into trx_fn_timer_process()
to decouple it from timerfd-specific code.

This allows reusing the same scheduling logic for alternative
timer backends (e.g. WebSDR) while keeping timerfd integration
as a thin wrapper.

No functional changes intended.

Change-Id: Ia28cce57afb3dc97c8655e855ebd5ace7e096648

build: make timerfd usage optional

Check for sys/timerfd.h in configure and guard its usage
with HAVE_SYS_TIMERFD_H.

This avoids build failures on platforms where timerfd is
not available.

Change-Id: Id5a700c865f984e8f99199a1e365bb7f1462030a

trx: split TRX ctrl response parsing from socket callback

Extract TRX control response parsing logic into
trx_ctrl_parse_rsp() and call it from trx_ctrl_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I255c7d554aad3ac6df2c66716fae1c1fb512110e

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

build: add WebSDR configure option and Emscripten detection

Detect Emscripten toolchain and introduce --with-websdr
configure option

When enabled, check for libosmo-trx-websdr and required
symbol modulateBits, failing configure if missing

Change-Id: Iff4b76e339e2873686c8c15885dd498458ceb546

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

build: make sched_* usage optional

Add configure checks for sched_* APIs and guard usage of
sched_setscheduler with HAVE_SCHED_SETSCHEDULER.

This avoids build/runtime issues on platforms without
scheduler support.

Change-Id: I5af357c1e2074fa1e05ac6bc20af8535c474d906

abis: add helper to update BTS identity

Introduce abis_update() to update site_id and bts_id in
bts_dev_info at runtime

This allows adjusting BTS identity after initialization,
e.g. from external control interfaces

Change-Id: Ibafd66a1e1dc42ad0063c412087bee10d212fe33

trx: factor out clock indication handling into helper

Extract clock indication handling logic into trx_clk_phy()
and reuse it from trx_clk_read_cb()

This separates core FN processing from socket-specific code
and prepares for alternative clock sources

No functional changes intended

Change-Id: I0e6b3e25da14ada4d0ef86fd4c4e6bbc12e447d2

trx: factor out burst submission into helper

Extract TRXD burst submission logic from bts_sched_flush_buffers()
into trx_sched_submit_bursts()

No functional changes intended, this prepares for alternative
backends reusing the same scheduling logic

Change-Id: I7ce61bee4ca7ca50ef190982f1a0e4ce57c8c838

build: make timerfd usage optional

Check for sys/timerfd.h in configure and guard its usage
with HAVE_SYS_TIMERFD_H.

This avoids build failures on platforms where timerfd is
not available.

Change-Id: Id5a700c865f984e8f99199a1e365bb7f1462030a

trx: factor out burst submission into helper

Extract TRXD burst submission logic from bts_sched_flush_buffers()
into trx_sched_submit_bursts()

No functional changes intended, this prepares for alternative
backends reusing the same scheduling logic

Change-Id: I7ce61bee4ca7ca50ef190982f1a0e4ce57c8c838

trx: handle synchronous TRX control callbacks in provision FSM

Set *_sent flags before issuing TRX control commands to ensure
correct state when responses are delivered synchronously
(e.g. WebUSB callback path)

Also guard FSM state handling in st_open_poweroff() to avoid
duplicate transitions if state was already changed by a
synchronous response

No functional change for asynchronous (socket-based) backends

Change-Id: I50c95e3796839e9fe721bff6f96fed4c490c6018

build: make sched_* usage optional

Add configure checks for sched_* APIs and guard usage of
sched_setscheduler with HAVE_SCHED_SETSCHEDULER.

This avoids build/runtime issues on platforms without
scheduler support.

Change-Id: I5af357c1e2074fa1e05ac6bc20af8535c474d906

smux: allow lchan lookup without remote address

Make rem_addr optional in osmux_lchan_find() by skipping
address comparison when it is NULL

This allows callers without socket-based addressing to
locate lchans by CID only (e.g. callback-based transports)

Change-Id: I7a275dad2353241c7b04cdb5d6cc1436a175c1f3

trx: factor out clock indication handling into helper

Extract clock indication handling logic into trx_clk_phy()
and reuse it from trx_clk_read_cb()

This separates core FN processing from socket-specific code
and prepares for alternative clock sources

No functional changes intended

Change-Id: I0e6b3e25da14ada4d0ef86fd4c4e6bbc12e447d2

trx: factor out TRXD data submission helper

Extract TRXD data sending into trx_data_submit_msg() and
reuse it from trx_if_send_burst()

This separates transport-specific I/O from message construction
and prepares for alternative backends

No functional changes intended

Change-Id: Ie89ac6efe2f5af0b4cbe5a259a18770b8d2de859

trx: factor out FN timer processing into common helper

Extract core FN timer handling logic into trx_fn_timer_process()
to decouple it from timerfd-specific code.

This allows reusing the same scheduling logic for alternative
timer backends (e.g. WebSDR) while keeping timerfd integration
as a thin wrapper.

No functional changes intended.

Change-Id: Ia28cce57afb3dc97c8655e855ebd5ace7e096648

trx: split TRXD parsing from socket callback

Extract TRXD message parsing into trx_data_read() and
reuse it from trx_data_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I50ad8b6d2edfd91f2d569cd8badf5c036fbf87ed

trx: split TRX ctrl response parsing from socket callback

Extract TRX control response parsing logic into
trx_ctrl_parse_rsp() and call it from trx_ctrl_read_cb()

This decouples parsing from socket I/O and allows reuse
with alternative transports

No functional changes intended

Change-Id: I255c7d554aad3ac6df2c66716fae1c1fb512110e

bts: guard AMR RTP decoder stub with HAVE_LIBOSMOCODEC_STATIC

Disable dummy osmo_amr_rtp_dec() implementation when
linking against static libosmocodec to avoid symbol
conflicts

Apply to omldummy and virtual BTS models

Change-Id: Ie19ae2013893575a2f74a87af2151034abf57870

build: add WebSDR configure option and Emscripten detection

Detect Emscripten toolchain and introduce --with-websdr
configure option

When enabled, check for libosmo-trx-websdr and required
symbol modulateBits, failing configure if missing

Change-Id: Iff4b76e339e2873686c8c15885dd498458ceb546

abis: add helper to update BTS identity

Introduce abis_update() to update site_id and bts_id in
bts_dev_info at runtime

This allows adjusting BTS identity after initialization,
e.g. from external control interfaces

Change-Id: Ibafd66a1e1dc42ad0063c412087bee10d212fe33

build: make linux/if_packet.h usage optional

Check for presence of linux/if_packet.h in configure and
guard its usage with HAVE_LINUX_IF_PACKET_H.

This avoids build failures on non-Linux platforms or
environments where the header is unavailable.

Change-Id: I8a02b0676e65fa4b6191073c84e0646a2e67a010

trx: add JSON stats export for WebSDR API

Introduce stats_json.c providing JSON serialization for:
- stats and counters
- rate counters
- BTS state
- TRX/transceiver state
- WebSDR runtime metrics

Add osmobts_get_stats() API to expose these via WebSDR
interface and export it in Emscripten build

This enables programmatic access to data previously available
via VTY (e.g. "show stats", "show bts")

Change-Id: Ifb82adfab879a65ecd222c45e06551983aa90a0f

trx, bts: add optional WebSDR backend (callback-based transport)

Introduce WebSDR backend enabled via ENABLE_WEBSDR,
adding an alternative callback-based transport alongside
existing socket-based TRX/OSMUX I/O

Implement:
- TRX control via osmotrxlib_process_command()
- TRXD data via apitrx_tx_call()/trx_data_read()
- clock handling via apibts_clock_ind() and external timers
- Osmux integration via ws_osmux_* helpers

Adapt scheduler to support API-based burst submission and
external timer callbacks in WebSDR mode

Disable signal handling, stats and socket I/O when building
with ENABLE_WEBSDR

Add WebSDR API entrypoint, Emscripten build target and
example configuration

Enables running BTS/TRX stack in WebAssembly environments

Change-Id: Idebd588f33afd85021813ad5821391781db683a9

trx: factor out TRX ctrl message submission

Extract TRX control message sending logic into
trx_ctrl_submit_msg() and reuse it from trx_ctrl_send()

This separates message construction from transport-specific
submission and prepares for alternative backends

No functional changes intended

Change-Id: Ibbd50117db4531efbc374e43e3802f9f995362cd

tcap_as_loadshare: Clarify name of tcap asp selection function

Change-Id: I327ad282d2689be2bcf7d3da23026717923c9325

tcap_as_loadshare: Route msg based on TCAP route if cache lookup fails

Related: SYS#5423
Change-Id: I2fd9068be8a0a512b17c8bd2365aa87c266e5669

tcap_as_loadshare: Don't fill in PC from M3UA

If both calling and called_addr are not point codes then the compare in
tcap_trans_track_entry_find() will ignore them. It still matches on
otid/dtid.

Related: SYS#5423
Change-Id: I4f99fc87cd33ee3f2aad531710cc39a9b9c7e6d9

tcap_as_loadshare: Use tcap_trans_track_begin instead of *_entry_create

*_begin checks for a matching entry and reuses that (with an updated
timestamp). This avoids duplicate entries in the tcap cache.

Related: SYS#5423
Change-Id: Icbaab976ed8207843129291494176679db574304

stp: Test TCAP routing with a different called address answering

Change-Id: I6d586b17b767219d20646f7cd1b59d6719cdbcdb
Related: SYS#5423

stp: Make TCAP tests more realistic

Use a TCAP id outside of the TCAP routing range for the M3UA TCAP id.
Test both directions IPA -> M3UA and M3UA -> IPA.

Change-Id: Ia807c2b26e5188b700365650311cabc0f9095674
Related: SYS#5423

HDLC timeslots: fix Rx of maximum-sized frames

E1D protocol sets the maximum permitted size of an HDLC frame to
264 bytes of useful payload, not counting the 2 bytes of FCS.
This limit is fair enough, as it matches GSM TS 08.56, the primary
intended application.  However, osmo_isdnhdlc engine used internally
by the daemon requires an Rx buffer 2 bytes larger than the maximum
payload length, for temporary storage of FCS bytes: by the nature
of HDLC, a receiver has no way of identifying FCS bytes as such
until after they have been received and stored in the Rx buffer.

This bug was caught when bringing up OsmoBSC with Nokia Flexi
Multiradio BTS.  This BTS emits some long OML messages that get
broken into segments, with each not-last segment traveling over
HDLC as a maximum-sized frame - and these frames then get dropped
by osmo-e1d, causing further breakage downstream.

Change-Id: I479e8fe5118c8eb7d4e6b16b49047a3278bc9808

cosmetic: fix white space in lapd_sap_{start,stop}()

Change-Id: Iadf8629bc3bdf18260b1b83290ce95483d8aac2c

lapd_sap_stop(): fix use of uninitialized stack memory

DL-RELEASE in lapd_core can be done in two ways: by sending DISC
and waiting for UA, or a purely local clearing of state.  The
selection between these two modes is communicated via an extended
field in struct osmo_dlsap_prim that extends beyond the standard
struct osmo_prim_hdr.  lapd_sap_stop() is a wrapper function that
sends DL-RELEASE to lapd_core; its only current users are the
vendor-specific E1 BTS handlers for Ericsson and Nokia in OsmoBSC,
and both of those need local release handling.  This wrapper
function does allocate correctly-sized struct osmo_dlsap_prim
on the stack, but until now it failed to initialize the extended
field - hence the actual behavior was subject to laws of chance,
depending on uninitialized content of stack memory.

Change-Id: Ic1469698ee8edf9b6fb55df8f79cc062be711f61

UppAudit: better indicate exception cause

Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
Jenkins: skip-card-test

test_configurable_parameters.py: add tests for new parameters

For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
Jenkins: skip-card-test

generate sdkey classes from a list

Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
Jenkins: skip-card-test

ConfigurableParameter: safer val length check

validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.

Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.

Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test

trx: add JSON serialization for 'show trx'

Introduce trx_to_json() helper to serialize the output of
'show trx' into JSON format.

Also initialize talloc/VTY context in websdr entrypoint to
allow accessing TRX configuration programmatically.

Intended for external consumers (e.g. WebSDR frontend).

Change-Id: Icbfb754580daa6db9b5646cc6698a6a6ee1c1610

saip/param_source: try to not repeat random values

Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
Jenkins: skip-card-test

personalization.ConfigurableParameter: fix BytesIO() input

Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
Jenkins: skip-card-test

use secrets.SystemRandom as secure random nr source

secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test

personalization: fix EF_SMSP length, alpha_id padding

The efFileSize needs to be updated and the alpha_id needs to be != None.

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test

add test_param_src.py

Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
Jenkins: skip-card-test

saip SmspTpScAddr.get_values_from_pes: allow empty values

Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
Jenkins: skip-card-test

param_source: allow plugging a random implementation (for testing)

Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
Jenkins: skip-card-test

UppAudit: better indicate exception cause

Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
Jenkins: skip-card-test

personalization: add int as input type for BinaryParameter

Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
Jenkins: skip-card-test

add comment about not updating existing key_usage_qualifier

Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
Jenkins: skip-card-test

ConfigurableParameter: safer val length check

validate_val() calls len() to check the value against allow_len,
min_len and max_len. len() requires the object to have a __len__()
method, which integers do not — calling len() on an int raises
TypeError.

Fix this by checking for __len__ first: if present, use len(val) as
usual; otherwise fall back to len(str(val)), which gives the number
of decimal digits for integer values.

Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test

MncLen

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Jenkins: skip-card-test

ConfigurableParameter: do not magically overwrite the 'name' attribute

Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test

generate sdkey classes from a list

Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
Jenkins: skip-card-test

test_configurable_parameters.py: add tests for new parameters

For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
Jenkins: skip-card-test

add test_configurable_parameters.py

Add ConfigurableParameterTest, which applies each parameter to a real
UPP DER template and reads it back, comparing results against a stored
expected-output snapshot (xo/test_configurable_parameters).

Add TestValidateVal covering validate_val() for Iccid, Imsi, Pin1, Puk1
and K, testing both valid inputs and invalid ones expected to raise
ValueError.

Add TestEnumParam covering the EnumParam methods (validate_val,
map_name_to_val, map_val_to_name, name_normalize, clean_name_str) using
AlgorithmID as the concrete subclass, including fuzzy name matching.

Also add get_value_from_pes() to ConfigurableParameter as a convenience
wrapper around get_values_from_pes() that asserts all returned values
are identical and returns the single result.

Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test

use random.SystemRandom as random nr source (/dev/urandom)

/dev/urandom is somewhat better than python's PRNG

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test

saip: add numeric_base indicator to ConfigurableParameter

By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test

personalization audit: optionally audit all (unknown) SD keys

By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test

sdkeys kv40 aes

Change-Id: If5b53c840ebd1f224f9bb4706a602b415194f47b
Jenkins: skip-card-test

personalization: implement UppAudit and BatchAudit

Change-Id: Iaab336ca91b483ecdddd5c6c8e08dc475dc6bd0a
Jenkins: skip-card-test

transitional name mapping

To help existing applications transition to a common naming scheme for
the SdKey classes, offer this intermediate result, where the SdKey
classes' .name are still unchanged as before generating them.

Change-Id: I974cb6c393a2ed2248a6240c2722d157e9235c33
Jenkins: skip-card-test

SdKey KVN4X ID02: set key_usage_qual=0x48

Related: SYS#7865
Change-Id: Idc5d33a4a003801f60c95fff6931706a9aeb6692
Jenkins: skip-card-test

saip SmspTpScAddr.get_values_from_pes: allow empty values

Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
Jenkins: skip-card-test

add comment about not updating existing key_usage_qualifier

Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
Jenkins: skip-card-test

saip SmspTpScAddr: safeguard against decoding error

Reading the TS48 V6.0 eSIM_GTP_SAIP2.1A_NoBERTLV profile results in an
exception [1] in SmspTpScAddr. I have a caller that needs to skip
erratic values instead of raising.

The underlying issue, I presume, is that either the data needs
validation before decode_record_bin(), or decode_record_bin() needs
well-defined error handling.

So far I know only of this IndexError, so, as a workaround, catch that.

[1]
  File "/pysim/pySim/esim/saip/personalization.py", line 617, in get_values_from_pes
    ef_smsp_dec = ef_smsp.decode_record_bin(f_smsp.body, 1)
  File "/pysim/pySim/filesystem.py", line 1047, in decode_record_bin
    return parse_construct(self._construct, raw_bin_data)
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 550, in parse_construct
    parsed = c.parse(raw_bin_data, total_len=length, **context)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 404, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 416, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 820, in _parse
    return self._decode(obj, context, path)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 268, in _decode
    if r[-1] == 'f':
       ~^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/utils.py", line 50, in __getitem__
    return hexstr(super().__getitem__(val))
                  ~~~~~~~~~~~~~~~~~~~^^^^^
IndexError: string index out of range

Change-Id: Ic436e206776b81f24de126e8ee0ae8bf5f3e8d7a
Jenkins: skip-card-test

personalization audit: optionally audit all (unknown) SD keys

By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test

personalization: fix SdKey.apply_val() implementation

'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.

So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.

Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
  an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.

Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.

(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)

Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
Jenkins: skip-card-test

personalization.ConfigurableParameter: fix BytesIO() input

Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
Jenkins: skip-card-test

add test_param_src.py

Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
Jenkins: skip-card-test

generate sdkey classes from a list

Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
Jenkins: skip-card-test

remove transitional name mapping

This reverts commit I974cb6c393a2ed2248a6240c2722d157e9235c33

Now, finally, all SdKey classes have a unified logical naming scheme.

Change-Id: Ic185af4a903c2211a5361d023af9e7c6fc57ae78
Jenkins: skip-card-test

ConfigurableParameter: do not magically overwrite the 'name' attribute

Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test

saip/param_source: try to not repeat random values

Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
Jenkins: skip-card-test

add test_configurable_parameters.py

Add ConfigurableParameterTest, which applies each parameter to a real
UPP DER template and reads it back, comparing results against a stored
expected-output snapshot (xo/test_configurable_parameters).

Add TestValidateVal covering validate_val() for Iccid, Imsi, Pin1, Puk1
and K, testing both valid inputs and invalid ones expected to raise
ValueError.

Add TestEnumParam covering the EnumParam methods (validate_val,
map_name_to_val, map_val_to_name, name_normalize, clean_name_str) using
AlgorithmID as the concrete subclass, including fuzzy name matching.

Also add get_value_from_pes() to ConfigurableParameter as a convenience
wrapper around get_values_from_pes() that asserts all returned values
are identical and returns the single result.

Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Co-authored-by: Vadim Yanitskiy <vyanitskiy@sysmocom.de>
Jenkins: skip-card-test

esim.saip.personalization: fix TLSPSK keys

Add AES variant of TLSPSK DEK (SCP81 KVN40 key_id=0x02).

Change-Id: I713a008fd26bbfcf437e0f29717b753f058ce76a
Jenkins: skip-card-test

use random.SystemRandom as random nr source (/dev/urandom)

/dev/urandom is somewhat better than python's PRNG

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test

personalization: fix EF_SMSP length, alpha_id padding

The efFileSize needs to be updated and the alpha_id needs to be != None.

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test

personalization: add get_typical_input_len() to ConfigurableParameter

The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.

Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
Jenkins: skip-card-test

saip: SdKey.__doc__: update SdKey listing

Change-Id: Ib5011b0c7d76b082231744cf09077628dc4e69b7
Jenkins: skip-card-test

MncLen

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Jenkins: skip-card-test

personalization: add int as input type for BinaryParameter

Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
Jenkins: skip-card-test

comment in uicc.py on Security Domain Keys: add SCP81

Change-Id: Ib0205880f58e78c07688b4637abd5f67ea0570d1
Jenkins: skip-card-test

test_configurable_parameters.py: add tests for new parameters

For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
Jenkins: skip-card-test

use secrets.SystemRandom as secure random nr source

secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test

param_source: allow plugging a random implementation (for testing)

Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
Jenkins: skip-card-test

ConfigurableParameter: safer val length check

Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test

saip: add numeric_base indicator to ConfigurableParameter

By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test

UppAudit: better indicate exception cause

Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
Jenkins: skip-card-test

personalization: make AlgorithmID a new EnumParam

The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.

Implement the enum value part abstractly in new EnumParam.

Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn

Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Jenkins: skip-card-test

personalization: add param_source.py, add batch.py

Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.

Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.

Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
Jenkins: skip-card-test

personalization: implement reading back values from a PES

Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.

Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.

Implementation detail:

Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.

For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.

Rationales for reading back values:

This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).

Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
  I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.

Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Jenkins: skip-card-test

personalization: indicate default ParamSource per ConfigurableParameter

Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.

This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.

For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.

BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:

  my_source = MyParam.default_source.from_str( MyParam.default_value )

Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Jenkins: skip-card-test

abis: align e1 input init prototypes for stricter compilers

Change return types of e1inp_dahdi_init(), e1inp_e1d_init() and
e1inp_rs232_init() from void to int to match their implementations.

Required for stricter compilers (e.g. Emscripten) which reject
mismatched function prototypes.

Change-Id: I93cd079003abb5665f1995647cfa94dc8e901ed9

abis: add optional WebSDR E1 input driver

Add a new optional E1 input backend for WebSDR, enabled via
--with-websdr.

Introduce input/websdr.c and public callback API to replace
socket-based IPA transport with a callback-driven interface
for OML/RSL and OSMUX.

Intended for WebAssembly builds where Osmocom components run
in a browser and communicate with SDR via WebUSB.

Change-Id: Ib804dc8eb67a91678603f54ea0beccd5c61e4a80

remsim: server: add a client timeout test

1. Connect to the server
2. Don't answer to IPA PINGs
3. Server closes the connection
4. Client reconnects
5. Older server versions will crash

Related: OS#6957
Change-Id: Iec52d3ce229c2ab84ab0ff9d8dbbff59b7f3fbd9

remsim: server: add a client timeout test

1. Connect to the server
2. Don't answer to IPA PINGs
3. Server closes the connection
4. Client reconnects
5. Older server versions will crash

Related: OS#6957
Change-Id: Iec52d3ce229c2ab84ab0ff9d8dbbff59b7f3fbd9

jenkins.sh: pin libosmocore version to current version v1.14

Change-Id: I678957099348e067d62a4abd04cbdac822463cf4

personalization: fix SdKey.apply_val() implementation

'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.

So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.

Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
  an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.

Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.

(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)

Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
Jenkins: skip-card-test

saip SmspTpScAddr: safeguard against decoding error

Reading the TS48 V6.0 eSIM_GTP_SAIP2.1A_NoBERTLV profile results in an
exception [1] in SmspTpScAddr. I have a caller that needs to skip
erratic values instead of raising.

The underlying issue, I presume, is that either the data needs
validation before decode_record_bin(), or decode_record_bin() needs
well-defined error handling.

So far I know only of this IndexError, so, as a workaround, catch that.

[1]
  File "/pysim/pySim/esim/saip/personalization.py", line 617, in get_values_from_pes
    ef_smsp_dec = ef_smsp.decode_record_bin(f_smsp.body, 1)
  File "/pysim/pySim/filesystem.py", line 1047, in decode_record_bin
    return parse_construct(self._construct, raw_bin_data)
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 550, in parse_construct
    parsed = c.parse(raw_bin_data, total_len=length, **context)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 404, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 416, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 820, in _parse
    return self._decode(obj, context, path)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 268, in _decode
    if r[-1] == 'f':
       ~^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/utils.py", line 50, in __getitem__
    return hexstr(super().__getitem__(val))
                  ~~~~~~~~~~~~~~~~~~~^^^^^
IndexError: string index out of range

Change-Id: Ic436e206776b81f24de126e8ee0ae8bf5f3e8d7a
Jenkins: skip-card-test

transitional name mapping

To help existing applications transition to a common naming scheme for
the SdKey classes, offer this intermediate result, where the SdKey
classes' .name are still unchanged as before generating them.

Change-Id: I974cb6c393a2ed2248a6240c2722d157e9235c33
Jenkins: skip-card-test

remove transitional name mapping

This reverts commit I974cb6c393a2ed2248a6240c2722d157e9235c33

Now, finally, all SdKey classes have a unified logical naming scheme.

Change-Id: Ic185af4a903c2211a5361d023af9e7c6fc57ae78
Jenkins: skip-card-test

add test_configurable_parameters.py

Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Jenkins: skip-card-test

personalization: implement reading back values from a PES

Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.

Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.

Implementation detail:

Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.

For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.

Rationales for reading back values:

This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).

Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
  I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.

Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Jenkins: skip-card-test

use random.SystemRandom as random nr source (/dev/urandom)

/dev/urandom is somewhat better than python's PRNG

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test

personalization: add get_typical_input_len() to ConfigurableParameter

The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.

Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
Jenkins: skip-card-test

personalization.ConfigurableParameter: fix BytesIO() input

Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
Jenkins: skip-card-test

saip: add numeric_base indicator to ConfigurableParameter

By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test

esim.saip.personalization: fix TLSPSK keys

Add AES variant of TLSPSK DEK (SCP81 KVN40 key_id=0x02).

Change-Id: I713a008fd26bbfcf437e0f29717b753f058ce76a
Jenkins: skip-card-test

param_source: allow plugging a random implementation (for testing)

Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
Jenkins: skip-card-test

ConfigurableParameter: safer val length check

Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test

MncLen

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Jenkins: skip-card-test

personalization: make AlgorithmID a new EnumParam

The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.

Implement the enum value part abstractly in new EnumParam.

Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn

Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Jenkins: skip-card-test

generate sdkey classes from a list

Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
Jenkins: skip-card-test

saip SmspTpScAddr.get_values_from_pes: allow empty values

Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
Jenkins: skip-card-test

saip: SdKey.__doc__: update SdKey listing

Change-Id: Ib5011b0c7d76b082231744cf09077628dc4e69b7
Jenkins: skip-card-test

test_configurable_parameters.py: add tests for new parameters

For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
Jenkins: skip-card-test

personalization: fix EF_SMSP length, alpha_id padding

The efFileSize needs to be updated and the alpha_id needs to be != None.

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test

personalization: add int as input type for BinaryParameter

Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
Jenkins: skip-card-test

personalization: add param_source.py, add batch.py

Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.

Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.

Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
Jenkins: skip-card-test

add comment about not updating existing key_usage_qualifier

Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
Jenkins: skip-card-test

saip/param_source: try to not repeat random values

Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
Jenkins: skip-card-test

sdkeys kv40 aes

Change-Id: If5b53c840ebd1f224f9bb4706a602b415194f47b
Jenkins: skip-card-test

SdKey KVN4X ID02: set key_usage_qual=0x48

Related: SYS#7865
Change-Id: Idc5d33a4a003801f60c95fff6931706a9aeb6692
Jenkins: skip-card-test

use secrets.SystemRandom as secure random nr source

secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test

add test_param_src.py

Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
Jenkins: skip-card-test

personalization audit: optionally audit all (unknown) SD keys

By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test

personalization: indicate default ParamSource per ConfigurableParameter

Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.

This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.

For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.

BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:

  my_source = MyParam.default_source.from_str( MyParam.default_value )

Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Jenkins: skip-card-test

UppAudit: better indicate exception cause

Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
Jenkins: skip-card-test

comment in uicc.py on Security Domain Keys: add SCP81

Change-Id: Ib0205880f58e78c07688b4637abd5f67ea0570d1
Jenkins: skip-card-test

ConfigurableParameter: do not magically overwrite the 'name' attribute

Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test

personalization: implement UppAudit and BatchAudit

Change-Id: Iaab336ca91b483ecdddd5c6c8e08dc475dc6bd0a
Jenkins: skip-card-test

personalization: add param_source.py, add batch.py

Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.

Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.

Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
Jenkins: skip-card-test

jobs/gerrit-verifications.yml: pysim: allow skipping card tests

I've already implemented this logic in 6a3b8a0, however by mistake
I added it to master-builds.yml instead of gerrit-verifications.yml.
This is wrong because we specifically need this feature for Jenkins
verification, not for the master builds.

For daily master builds, on the other hand, we specifically want to
execute all existing job types, so revert my recent changes from
master-builds.yml, but keep the new job type.

Change-Id: I986d88545f64e13cd571ba9ff56bc924822e39a0
Fixes: 6a3b8a0 ("jobs/master-builds.yml: pysim: allow skipping card tests")

debian-trixie-build: add tshark

This is needed for running pysim unit tests on generic nodes.
As of now we're executing them on the 'simtester' node, which cannot
run more than one job at once, making build verification for large
pysim patchsets extremely slow.

Change-Id: Idbe934e5cc29b06fd1b13b85893de5856eca836e
Related: osmo-ci.git I986d88545f64e13cd571ba9ff56bc924822e39a0

saip SmspTpScAddr: safeguard against decoding error

Reading the TS48 V6.0 eSIM_GTP_SAIP2.1A_NoBERTLV profile results in an
exception [1] in SmspTpScAddr. I have a caller that needs to skip
erratic values instead of raising.

The underlying issue, I presume, is that either the data needs
validation before decode_record_bin(), or decode_record_bin() needs
well-defined error handling.

So far I know only of this IndexError, so, as a workaround, catch that.

[1]
  File "/pysim/pySim/esim/saip/personalization.py", line 617, in get_values_from_pes
    ef_smsp_dec = ef_smsp.decode_record_bin(f_smsp.body, 1)
  File "/pysim/pySim/filesystem.py", line 1047, in decode_record_bin
    return parse_construct(self._construct, raw_bin_data)
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 550, in parse_construct
    parsed = c.parse(raw_bin_data, total_len=length, **context)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 404, in parse
    return self.parse_stream(io.BytesIO(data), **contextkw)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 416, in parse_stream
    return self._parsereport(stream, context, "(parsing)")
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2236, in _parse
    subobj = sc._parsereport(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 2770, in _parse
    return self.subcon._parsereport(stream, context, path)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 428, in _parsereport
    obj = self._parse(stream, context, path)
  File "/application/venv/lib/python3.13/site-packages/construct/core.py", line 820, in _parse
    return self._decode(obj, context, path)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/construct.py", line 268, in _decode
    if r[-1] == 'f':
       ~^^^^
  File "/application/venv/lib/python3.13/site-packages/osmocom/utils.py", line 50, in __getitem__
    return hexstr(super().__getitem__(val))
                  ~~~~~~~~~~~~~~~~~~~^^^^^
IndexError: string index out of range

Change-Id: Ic436e206776b81f24de126e8ee0ae8bf5f3e8d7a
Jenkins: skip-card-test

personalization.ConfigurableParameter: fix BytesIO() input

Change-Id: I0ad160eef9015e76eef10baee7c6b606fe249123
Jenkins: skip-card-test

personalization: add param_source.py, add batch.py

Implement pySim.esim.saip.batch.BatchPersonalization,
generating N eSIM profiles from a preset configuration.

Batch parameters can be fed by a constant, incrementing, random or from
CSV rows: add pySim.esim.saip.param_source.* classes to feed such input
to each of the BatchPersonalization's ConfigurableParameter instances.

Related: SYS#6768
Change-Id: I01ae40a06605eb205bfb409189fcd2b3a128855a
Jenkins: skip-card-test

saip: SdKey.__doc__: update SdKey listing

Change-Id: Ib5011b0c7d76b082231744cf09077628dc4e69b7
Jenkins: skip-card-test

esim.saip.personalization: fix TLSPSK keys

Add AES variant of TLSPSK DEK (SCP81 KVN40 key_id=0x02).

Change-Id: I713a008fd26bbfcf437e0f29717b753f058ce76a
Jenkins: skip-card-test

MncLen

Change-Id: I6c600faeab00ffb072acbe94c9a8b2d1397c07d3
Jenkins: skip-card-test

saip/param_source: try to not repeat random values

Change-Id: I4fa743ef5677580f94b9df16a5051d1d178edeb0
Jenkins: skip-card-test

add comment about not updating existing key_usage_qualifier

Change-Id: Ie23ae5fde17be6b37746784bf1601b4d0874397a
Jenkins: skip-card-test

test_configurable_parameters.py: add tests for new parameters

For:
SmspTpScAddr
MilenageRotation
MilenageXoringConstants
TuakNrOfKeccak

Change-Id: Iecbea14fe31a9ee08d871dcde7f295d26d7bd001
Jenkins: skip-card-test

saip SmspTpScAddr.get_values_from_pes: allow empty values

Change-Id: Ibbdd08f96160579238b50699091826883f2e9f5a
Jenkins: skip-card-test

use secrets.SystemRandom as secure random nr source

secrets.SystemRandom is defined as the most secure random source
available on the given operating system.

Change-Id: I8049cd1292674b3ced82b0926569128535af6efe
Jenkins: skip-card-test

UppAudit: better indicate exception cause

Change-Id: I4d986b89a473a5b12ed56b4710263b034876a33e
Jenkins: skip-card-test

personalization: add get_typical_input_len() to ConfigurableParameter

The aim is to tell a user interface how wide an input text field should
be chosen to be convenient -- ideally showing the entire value in all
cases, but not too huge for fields that have no sane size limit.

Change-Id: I2568a032167a10517d4d75d8076a747be6e21890
Jenkins: skip-card-test

use random.SystemRandom as random nr source (/dev/urandom)

/dev/urandom is somewhat better than python's PRNG

Change-Id: I6de38c14ac6dd55bc84d53974192509c18d02bfa
Jenkins: skip-card-test

personalization: fix SdKey.apply_val() implementation

'securityDomain' elements are decoded to ProfileElementSD instances,
which keep higher level representations of the key data apart from the
decoded[] lists.

So far, apply_val() was dropping binary values in decoded[], which does
not work, because ProfileElementSD._pre_encode() overwrites
self.decoded[] from the higher level representation.

Implement using
- ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify
  an exsiting entry, or
- ProfileElementSD.add_key() to create a new entry.

Before this patch, SdKey parameters seemed to patch PES successfully,
but their modifications did not end up in the encoded DER.

(BTW, this does not fix any other errors that may still be present in
the various SdKey subclasses, patches coming up.)

Related: SYS#6768
Change-Id: I07dfc378705eba1318e9e8652796cbde106c6a52
Jenkins: skip-card-test

personalization: indicate default ParamSource per ConfigurableParameter

Add default_source class members pointing to ParamSource classes to all
ConfigurableParameter subclasses.

This is useful to automatically set up a default ParamSource for a given
ConfigurableParameter subclass, during user interaction to produce a
batch personalization.

For example, if the user selects a Pin1 parameter, a calling program can
implicitly set this to a RandomDigitSource, which will magically make it
work the way that most users need.

BTW, default_source and default_value can be combined to configure a
matching ParamSource instance:

  my_source = MyParam.default_source.from_str( MyParam.default_value )

Change-Id: Ie58d13bce3fa1aa2547cf3cee918c2f5b30a8b32
Jenkins: skip-card-test

sdkeys kv40 aes

Change-Id: If5b53c840ebd1f224f9bb4706a602b415194f47b
Jenkins: skip-card-test

SdKey KVN4X ID02: set key_usage_qual=0x48

Related: SYS#7865
Change-Id: Idc5d33a4a003801f60c95fff6931706a9aeb6692
Jenkins: skip-card-test

personalization: implement reading back values from a PES

Implement get_values_from_pes(), the reverse direction of apply_val():
read back and return values from a ProfileElementSequence. Implement for
all ConfigurableParameter subclasses.

Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works
fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this
implementation to use the higher level ProfileElementSD members.

Implementation detail:

Implement get_values_from_pes() as classmethod that returns a generator.
Subclasses should yield all occurences of their parameter in a given
PES.

For example, the ICCID can appear in multiple places.
Iccid.get_values_from_pes() yields all of the individual values. A set()
of the results quickly tells whether the PES is consistent.

Rationales for reading back values:

This allows auditing an eSIM profile, particularly for producing an
output.csv from a batch personalization (that generated lots of random
key material which now needs to be fed to an HLR...).

Reading back from a binary result is more reliable than storing the
values that were fed into a personalization.
By auditing final DER results with this code, I discovered:
- "oh, there already was some key material in my UPP template."
- "all IMSIs ended up the same, forgot to set up the parameter."
- the SdKey.apply() implementations currently don't work, see
  I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.

Change-Id: I234fc4317f0bdc1a486f0cee4fa432c1dce9b463
Jenkins: skip-card-test

personalization: implement UppAudit and BatchAudit

Change-Id: Iaab336ca91b483ecdddd5c6c8e08dc475dc6bd0a
Jenkins: skip-card-test

transitional name mapping

To help existing applications transition to a common naming scheme for
the SdKey classes, offer this intermediate result, where the SdKey
classes' .name are still unchanged as before generating them.

Change-Id: I974cb6c393a2ed2248a6240c2722d157e9235c33
Jenkins: skip-card-test

comment in uicc.py on Security Domain Keys: add SCP81

Change-Id: Ib0205880f58e78c07688b4637abd5f67ea0570d1
Jenkins: skip-card-test

add test_configurable_parameters.py

Change-Id: Ia55f0d11f8197ca15a948a83a34b3488acf1a0b4
Jenkins: skip-card-test

remove transitional name mapping

This reverts commit I974cb6c393a2ed2248a6240c2722d157e9235c33

Now, finally, all SdKey classes have a unified logical naming scheme.

Change-Id: Ic185af4a903c2211a5361d023af9e7c6fc57ae78
Jenkins: skip-card-test

personalization: fix EF_SMSP length, alpha_id padding

The efFileSize needs to be updated and the alpha_id needs to be != None.

Change-Id: Ief6e02517f3e96158a2509d763b88aec4bd5a296
Jenkins: skip-card-test

personalization: add int as input type for BinaryParameter

Change-Id: I31d8142cb0847a8b291f8dc614d57cb4734f0190
Jenkins: skip-card-test

generate sdkey classes from a list

Change-Id: Ic92ddea6e1fad8167ea75baf78ffc3eb419838c4
Jenkins: skip-card-test

saip: SmspTpScAddr: fix get_values_from_pes

Change-Id: I2010305340499c907bb7618c04c61e194db34814
Jenkins: skip-card-test

personalization audit: optionally audit all (unknown) SD keys

By a flag, allow to audit also all Security Domain KVN that we have
*not* created ConfigurableParameter subclasses for.

For example, SCP80 has reserved kvn 0x01..0x0f, but we offer only
Scp80Kvn01, Scp80Kvn02, Scp80Kvn03. So we would not show kvn
0x03..0x0f in an audit.

This patch includes audits of all SD key kvn there may be in the UPP.
This will help to spot SD keys that may already be present in a UPP
template, with unexpected / unusual kvn.

Change-Id: Icaf6f7b589f117868633c0968a99f2f0252cf612
Jenkins: skip-card-test

add test_param_src.py

Change-Id: I03087b84030fddae98b965e0075d44e04ec6ba5c
Jenkins: skip-card-test

personalization: make AlgorithmID a new EnumParam

The AlgorithmID has a few preset values, and hardly anyone knows which
is which. So instead of entering '1', '2' or '3', make it work with
prededined values 'Milenage', 'TUAK' and 'usim-test'.

Implement the enum value part abstractly in new EnumParam.

Make AlgorithmID a subclass of EnumParam and define the values as from
pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn

Related: SYS#6768
Change-Id: I71c2ec1b753c66cb577436944634f32792353240
Jenkins: skip-card-test

personalization: allow reading back multiple values from PES

Change-Id: Iecb68af7c216c6b9dc3add469564416b6f37f7b2
Jenkins: skip-card-test

ConfigurableParameter: do not magically overwrite the 'name' attribute

Change-Id: I6f631444c6addeb7ccc5f6c55b9be3dc83409169
Jenkins: skip-card-test

ConfigurableParameter: safer val length check

Change-Id: Ibe91722ed1477b00d20ef5e4e7abd9068ff2f3e4
Jenkins: skip-card-test

saip: add numeric_base indicator to ConfigurableParameter

By default, numeric_base = None, to indicate that there are no explicit
limitations on the number space.

For parameters that are definitely decimal, set numeric_base = 10.
For definitely hexadecimal, set numeric_base = 16.

Do the same for ConfigurableParameter as well as ParamSource, so callers
can match them up: if a parameter is numeric_base = 10, then omit
sources that are numeric_base = 16, and vice versa.

Change-Id: Ib0977bbdd9a85167be7eb46dd331fedd529dae01
Jenkins: skip-card-test

param_source: allow plugging a random implementation (for testing)

Change-Id: Idce2b18af70c17844d6f09f7704efc869456ac39
Jenkins: skip-card-test

jobs/gerrit-verifications.yml: pysim: allow skipping card tests

I've already implemented this logic in 6a3b8a0, however by mistake
I added it to master-builds.yml instead of gerrit-verifications.yml.
This is wrong because we specifically need this feature for Jenkins
verification, not for the master builds.

For daily master builds, on the other hand, we specifically want to
execute all existing job types, so revert my recent changes from
master-builds.yml, but keep the new job type.

Change-Id: I986d88545f64e13cd571ba9ff56bc924822e39a0
Fixes: 6a3b8a0 ("jobs/master-builds.yml: pysim: allow skipping card tests")

jobs/master-builds.yml: add missing job type 'distcheck'

We do have this job type in gerrit-verifications.yml, so it's executed
for each patch submitted to Gerrit.  However, we are not executing
'distcheck' during the daily master builds - fix this.

Change-Id: I4898ba712a29a6afc34bc22adec6a3131ce8c794
Fixes: 31fd5cf0 ("gerrit-verifications: execute 'distcheck' job for pysim")

jobs/master-builds.yml: fix typo: crad-test -> card-test

Change-Id: I0686af6c026e298c86d3a64cc161e5c84ec7dc57
Fixes: ee1ef65 ("jobs/master-builds.yml: pysim: separate JOB_TYPE for card tests")

contrib/jenkins.sh: separate JOB_TYPE for card tests

A separate job gives us a possibility to skip tests requiring physical
cards for specific commits that do not touch the core logic.  See the
related commits in osmo-ci.git.

Change-Id: If76d812ee43b7eb3b57fdc660c60bf31fbff5b16
Related: osmo-ci.git Ia48d1b468f65d7c2e6b4128eeac36d0f3d03c45e
Related: osmo-ci.git I986d88545f64e13cd571ba9ff56bc924822e39a0

contrib/jenkins.sh: add setup_venv()

Reduece code duplication by factoring out virtualenv setup and
activation into a shell function.

Change-Id: Ibb193d12d5502c78104ef53badc6037f08e92df1

contrib/jenkins.sh: separate JOB_TYPE for card tests

A separate job gives us a possibility to skip tests requiring physical
cards for specific commits that do not touch the core logic.  See the
related commits in osmo-ci.git.

Change-Id: If76d812ee43b7eb3b57fdc660c60bf31fbff5b16
Related: osmo-ci.git Ia48d1b468f65d7c2e6b4128eeac36d0f3d03c45e
Related: osmo-ci.git I986d88545f64e13cd571ba9ff56bc924822e39a0

contrib/jenkins.sh: add setup_venv()

Reduece code duplication by factoring out virtualenv setup and
activation into a shell function.

Change-Id: Ibb193d12d5502c78104ef53badc6037f08e92df1

contrib/jenkins.sh: separate JOB_TYPE for card tests

A separate job gives us a possibility to skip tests requiring physical
cards for specific commits that do not touch the core logic.  See the
related commits in osmo-ci.git.

Change-Id: If76d812ee43b7eb3b57fdc660c60bf31fbff5b16
Related: osmo-ci.git Ia48d1b468f65d7c2e6b4128eeac36d0f3d03c45e
Related: osmo-ci.git I986d88545f64e13cd571ba9ff56bc924822e39a0