Skip to content

Changes

Started by pmaier@sysmocom.de
Started 4 hr 37 min ago
Queued 2 ms
Took 14 min on built-in

Summary

  1. personalization: refactor ConfigurableParameter, Iccid, Imsi (details)
  2. personalization: refactor Puk (details)
  3. personalization: refactor Pin, Adm (details)
  4. personalization: refactor AlgorithmID, K, Opc (details)
  5. personalization: refactor SdKey (details)
  6. personalization: refactor SmspTpScAddr (details)
  7. personalization: set some typical parameter names (details)
  8. personalization: set example input values (details)
  9. pySim-trace: pySim.apdu_source.stdin_hex (details)
  10. Print SMSC in pySim-read.py (details)
  11. pySim/card_key_provider: fix typo in keys (details)
  12. contrib/csv-to-pgsql: add missing copyright header (details)
  13. es9p_client: Fix type conversion in installation result notification (details)
  14. es9p_client: MAke install notification code execute at all (details)
  15. http_json_api: Only require Content-Type if response body is non-empty (details)
  16. esim/http_json_api: add missing apidoc (details)
  17. tests/pySim-smpp2sim_test: add integration test (details)
  18. contrib/smpp-ota-tool/cosmetic: fix sourcecode formatting (details)
  19. contrib/smpp-ota-tool/cosmetic: use lazy formatting for logging (details)
  20. contrib/smpp-ota-tool: use correct kid index (details)
  21. contrib/smpp-ota-tool: warn about mixed up KIC/KIC indexes (details)
  22. contrib/smpp-ota-tool: add missing usage helpstrings (details)
  23. contrib/smpp-ota-tool: fix boolean commandline parameters (details)
  24. tests/pySim-smpp2sim_test: add testcase for aes_cbc/aes_cmac (details)
Commit 742baeab56b6a5f9d6a15daf20ebdd06a9b2c985 by laforge
personalization: refactor ConfigurableParameter, Iccid, Imsi

Main points/rationales of the refactoring, details below:
1) common validation implementation
2) offer classmethods

The new features are optional, and will be heavily used by batch
personalization patches coming soon.

Implement Iccid and Imsi to use the new way, with a common abstract
DecimalParam implementation.

So far leave the other parameter classes working as they always did, to
follow suit in subsequent commits.

Details:

1) common validation implementation:
There are very common validation steps in the various parameter
implementations. It is more convenient and much more readable to
implement those once and set simple validation parameters per subclass.
So there now is a validate_val() classmethod, which subclasses can use
as-is to apply the validation parameters -- or subclasses can override
their cls.validate_val() for specialized validation.
(Those subclasses that this patch doesn't touch still override the
self.validate() instance method. Hence they still work as before this
patch, but don't use the new common features yet.)

2) offer stateless classmethods:
It is useful for...
- batch processing of multiple profiles (in upcoming patches) and
- user input validation
to be able to have classmethods that do what self.validate() and
self.apply() do, but do not modify any self.* members.
So far the paradigm was to create a class instance to keep state about
the value. This remains available, but in addition we make available the
paradigm of a singleton that is stateless (the classmethods).
Using self.validate() and self.apply() still work the same as before
this patch, i.e. via self.input_value and self.value -- but in addition,
there are now classmethods that don't touch self.* members.

Related: SYS#6768
Change-Id: I6522be4c463e34897ca9bff2309b3706a88b3ce8
The file was modifiedpySim/esim/saip/personalization.py
Commit 21641816ea7132c1ad14227f15eac3678d0cec86 by laforge
personalization: refactor Puk

Implement abstract DecimalHexParam, and use it to refactor Puk1 and Puk2
to the new ConfigurableParameter implementation style.

DecimalHexParam will also be used for Pin and Adm soon.

Change-Id: I271e6c030c890778ab7af9ab3bc7997e22018f6a
The file was modifiedpySim/esim/saip/personalization.py
Commit d5b570b01d806571ff705ccca12c35c16071c17a by laforge
personalization: refactor Pin, Adm

Refactor Pin1, Pin2, Adm1 and Adm2 to the new ConfigurableParameter
implementation style.

Change-Id: I54aef10b6d4309398d4b779a3740a7d706d68603
The file was modifiedpySim/esim/saip/personalization.py
Commit ae656c66a3b3a382644ca895d808bca0dad9f2b2 by laforge
personalization: refactor AlgorithmID, K, Opc

Refactor AlgorithmID, K, Opc to the new ConfigurableParameter
implementation style.

K and Opc use a common abstract BinaryParam.

Note from the future: AlgorithmID so far takes "raw" int values, but
will turn to be an "enum" parameter with predefined meaningful strings
in I71c2ec1b753c66cb577436944634f32792353240

Change-Id: I6296fdcfd5d2ed313c4aade57ff43cc362375848
The file was modifiedpySim/esim/saip/personalization.py
Commit 1e9885610585a6b1b22c969b45aaf736fcba4d55 by laforge
personalization: refactor SdKey

Refactor SdKey (and subclasses) to the new ConfigurableParameter
implementation style, keeping the same implementation.

But duly note that this implementation does not work!
It correctly patches pe.decoded[], but that gets overridden by
ProfileElementSD._pre_encode().

For a fix, see I07dfc378705eba1318e9e8652796cbde106c6a52.

Change-Id: I427ea851bfa28b2b045e70a19a9e35d361f0d393
The file was modifiedpySim/esim/saip/personalization.py
Commit 1c082da0ee6cbdcfd08c053f4a2577043f831b9c by laforge
personalization: refactor SmspTpScAddr

Refactor SmspTpScAddr to the new ConfigurableParameter implementation
style.

Change-Id: I2600369e195e9f5aed7f4e6ff99ae273ed3ab3bf
The file was modifiedpySim/esim/saip/personalization.py
Commit db17529136e480563bb4bc86d799511672b60d4a by laforge
personalization: set some typical parameter names

These names better match what humans expect to read, for example "PIN1"
instead of "Pin1".

(We still fall back to the __class__.__name__ if a subclass omits a
specific name, see the ConfigurableParameter init.)

Change-Id: I31f390d634e58c384589c50a33ca45d6f86d4e10
The file was modifiedpySim/esim/saip/personalization.py
Commit ccc1a047abfd367c2a906cee1733b998b115d6a3 by laforge
personalization: set example input values

For all ConfigurableParameter subclasses, provide an example_input.

This may be useful for downstream projects' user interaction, to suggest
a value or prefill an input field, as appropriate.

Related: SYS#6768
Change-Id: I2672fedcbc32cb7a6cb0c233a4a22112bd9aae03
The file was modifiedpySim/esim/saip/personalization.py
Commit 429b12c8b5064f3abc7ca4c1d748cd5544d15b29 by laforge
pySim-trace: pySim.apdu_source.stdin_hex

This introduces an "APDU source" for pySim-trace which enables the
decoding of APDUs that are copy+pasted from elsewhere, for example
APDU logs in text form created by proprietary tools, or to decode
personalization scripts or the like.

Change-Id: I5aacf13b7c27cea9efd42f01dacca61068c3aa33
The file was addedpySim/apdu_source/stdin_hex.py
The file was modifiedpySim-trace.py
Commit ae912455822dce0bea3e0c98abc4c827653c6fe8 by laforge
Print SMSC in pySim-read.py

Change-Id: I17067b68086316d51fd71ba77049874605594e3f
The file was modifiedtests/pySim-prog_test/sysmoISIM-SJA2.ok
The file was modifiedtests/pySim-prog_test/sysmoUSIM-SJS1.ok
The file was modifiedtests/pySim-prog_test/sysmosim-gr1.ok
The file was modifiedpySim-read.py
The file was modifiedtests/pySim-prog_test/fakemagicsim.ok
The file was modifiedtests/pySim-prog_test/sysmoISIM-SJA5.ok
The file was modifiedtests/pySim-prog_test/Fairwaves-SIM.ok
The file was modifiedtests/pySim-prog_test/Wavemobile-SIM.ok
Commit 1f2db11d31d28b796dfe44adad74e36c4cb75b48 by laforge
pySim/card_key_provider: fix typo in keys

Change-Id: Ie76f351ae221da2a0aab65c311fafe8ae6d63663
The file was modifiedpySim/card_key_provider.py
Commit e62f160775f8fa99e83a0b20acaf04054172a912 by laforge
contrib/csv-to-pgsql: add missing copyright header

Change-Id: Iad8b2c1abb6a80764d05c823fbd03a9eae0ec0ab
The file was modifiedcontrib/csv-to-pgsql.py
Commit 4e35e2c357f3a959ba04aff3990816d01f5a3b08 by laforge
es9p_client: Fix type conversion in installation result notification

The asn.1 encoder expects bytes-like objects, we cannot simply pass
hex-strings to it without conversion

Change-Id: I83ad047e043dc6b3462b188ce6dd0b2cc0e52e87
The file was modifiedcontrib/es9p_client.py
Commit 22c3797a89bc3e93c82194dab1af7ffa18f2b977 by laforge
es9p_client: MAke install notification code execute at all

The caller specified 'install' but the do_notification() function
compared with 'download' :(

Change-Id: I2d441cfbc1457688eb163301d3d91a1f1fdc7a8c
The file was modifiedcontrib/es9p_client.py
Commit e0a9e73267f18fed1d81874938f5e2831afd12e6 by laforge
http_json_api: Only require Content-Type if response body is non-empty

If there is an empty body returned, such as in the case of the response
to an es9p notification, then it is of course also legal to not set the
content-type header.

This patch fixes an exception when talking to certain SM-DP+ with
es9p_client.py:

DEBUG:pySim.esim.http_json_api:HTTP RSP-STS: [204] hdr: {'X-Admin-Protocol': 'gsma/rsp/v2.5.0', 'Date': 'Wed, 28 Jan 2026 18:26:39 GMT', 'Server': 'REDACTED'}
DEBUG:pySim.esim.http_json_api:HTTP RSP: b''
{'X-Admin-Protocol': 'gsma/rsp/v2.5.0', 'Date': 'Wed, 28 Jan 2026 18:26:39 GMT', 'Server': 'REDACTED'}
<Response [204]>
Traceback (most recent call last):
  File "gprojects/git/pysim/es9p/../contrib/es9p_client.py", line 315, in <module>
    c.do_notification()
    ~~~~~~~~~~~~~~~~~^^
  File "projects/git/pysim/es9p/../contrib/es9p_client.py", line 159, in do_notification
    res = self.peer.call_handleNotification(data)
  File "projects/git/pysim/contrib/pySim/esim/es9p.py", line 174, in call_handleNotification
    return self.handleNotification.call(data)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "projects/git/pysim/contrib/pySim/esim/http_json_api.py", line 335, in call
    if not response.headers.get('Content-Type').startswith(req_headers['Content-Type']):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'startswith'

Change-Id: I99e8f167b7bb869c5ff6d908ba673dac87fef71a
The file was modifiedpySim/esim/http_json_api.py
Commit 0a1c5a27d744805c65e61f60cd8c0d70f020ec3a by pmaier@sysmocom.de
esim/http_json_api: add missing apidoc

Change-Id: Ibf9cf06197c9e3203c7a3ea5d77004f0ca41cd3f
The file was modifiedpySim/esim/http_json_api.py
Commit 15691233e1b5c1f151b36ca199f2f49581d049cd by laforge
tests/pySim-smpp2sim_test: add integration test

At the moment pySim.ota codebase is not covered by any of the
integration tests (we have only normal unittests so far). To
increase the test coverage, let's add an integration test that
sends exchanges an RFM OTA-SMS with a real-world card.

However, there is no tool avaliable that can be used as an SMPP
client for pySim-smpp2sim yet. Let's use smpp_ota_apdu2.py on
laforge/ota to develop a tool that we can use to exchange SMS-TPDUs
that contain remote APDU scripts (RFM/RAM).

Finally let's use the tool we have created as a basis to create
an integration test that exchanges an SMS-TPDU with the RFM
application of a sysmoISIM-SJA5 card. The testcase shall pass
when we get the expected response from the card.

Related: OS#6868
Change-Id: If25e38be004cc1c7aeeb130431831377e78fe28d
The file was modifiedsetup.py
The file was modifiedpySim/ota.py
The file was modifiedcontrib/jenkins.sh
The file was addedcontrib/smpp-ota-tool.py
The file was addedtests/pySim-smpp2sim_test/pySim-smpp2sim_test.sh
The file was modifiedrequirements.txt
The file was addedtests/pySim-smpp2sim_test/pySim-smpp2sim_test.cfg
Commit 8422bed36d0f6c83d4c9056e7d2a6a711f17f449 by pmaier@sysmocom.de
contrib/smpp-ota-tool/cosmetic: fix sourcecode formatting

Change-Id: Icbce41ffac097d2ef8714bc8963536ba54a77db2
Related: OS#6868
The file was modifiedcontrib/smpp-ota-tool.py
Commit 44fd6dd28aabb034b78fba3109a006edf7f4e019 by pmaier@sysmocom.de
contrib/smpp-ota-tool/cosmetic: use lazy formatting for logging

Change-Id: I2540472a50b7a49b5a67d088cbdd4a2228eef8f4
Related: OS#6868
The file was modifiedcontrib/smpp-ota-tool.py
Commit defa40fb0df348f5fcbbfef70273323d6896ccb9 by pmaier@sysmocom.de
contrib/smpp-ota-tool: use correct kid index

(normally KID index and KIC index should be the same since mixing keys
is a concidered as a security violation. However, in this tool we
want to allow users to specify different indexes for KIC and KIC so that
they can make tests to make sure their cards correctly reject mixed up
key indexes)

Change-Id: I8847ccc39e4779971187e7877b8902fca7f8bfc1
Related: OS#6868
The file was modifiedcontrib/smpp-ota-tool.py
Commit c0ea5107705f5994361dfad2e1d91ddb35adcac2 by pmaier@sysmocom.de
contrib/smpp-ota-tool: warn about mixed up KIC/KIC indexes

Cards usually have multiple sets of KIC, KID (and KIK). The keys
are selected through an index. However, mixing keys from different
sets is concidered as a security violation and cards should reject
such configurations.

Let's print a warning to make users aware that something is off.

Change-Id: Ieb4e14145baba1c2cb4a237b612b04694940f402
Related: OS#6868
The file was modifiedcontrib/smpp-ota-tool.py
Commit 110368b1225a4fa9f197e5d36e30189aeae0db72 by pmaier@sysmocom.de
contrib/smpp-ota-tool: add missing usage helpstrings

Change-Id: Ic1521ba11b405f311a30fdb3585ad518375669ae
Related: OS#6868
The file was modifiedcontrib/smpp-ota-tool.py
Commit 39451dee90db1feaf4c2e4c729ca90bf560b9496 by pmaier@sysmocom.de
contrib/smpp-ota-tool: fix boolean commandline parameters

Boolean parameters should be false by default and use store_true when
set.

Change-Id: I0652b48d2ea5efbaaf5bc147aa8cef7ab8b0861d
Related: OS#6868
The file was modifiedcontrib/smpp-ota-tool.py
Commit cf68b7c79211109392a785beccb08778fa321c2f by pmaier@sysmocom.de
tests/pySim-smpp2sim_test: add testcase for aes_cbc/aes_cmac

Extend the existing test script so that it can handle multiple
testcases, add a new testcase to test OTA-SMS (RFM) with AES
encryption.

Change-Id: I1f10504f3a29a8c74a17991632d932819fecfa5a
Related: OS#6868
The file was addedtests/pySim-smpp2sim_test/testcase_triple_des_cbc2_rfm.cfg
The file was removedtests/pySim-smpp2sim_test/pySim-smpp2sim_test.cfg
The file was addedtests/pySim-smpp2sim_test/testcase_aes_cbc_cmac_rfm.cfg
The file was modifiedtests/pySim-smpp2sim_test/pySim-smpp2sim_test.sh