The module scp.py predates the existence of the pySimLogger and still uses an individually created logger. Let's migrate to pySimLogger to avoid unexpected effects and to be uniform with the other modules.
card_key_provider: move boiler-plate code into helper functions
in pySim-shell.py we add the commandline options for the card key provider and do the setup accordingly. Let's put this boilerplate code into helper functions instead, so that we can re-use it in other pySim programs as well. Let's use pySim.transport as a pattern.
card_key_provider: pass CardKeyFieldCryptor to constructor
We currently create the CardKeyFieldCryptor object inside the constructor of the concrete CardKeyProvider classes. There is currently no problem with that, but when we create the CardKeyFieldCryptor object first and then pass it as parameter to the constructor, we gain more flexibility in case we want to support other CardKeyFieldCryptor variants in the future.
card_key_provider: add a static method to parse --column-keys args
The contents of the --column-keys arguments are currently parsed in init_card_key_provider. Let's add a static method in CardKeyFieldCryptor to simplify re-usage of the CardKeyFieldCryptor
In format string we prepend when we log in verbose mode. We use %(module)s as format string quaifier. This qualifier is replaced with the name of the module from where the logger was called. This is mostly equal to the logger name (__name__) we pass when we create the logger.
However, this is not the behavior we actually want. We want to log the logger name that we passed when the logger was created. For this, we must use %(name)s as qualifier.
The legacy code found in legacy/cards.py does not use the modern construct based encoder (pySim-read uses it). The card classes either use their own implementation of update_smsp or use the generic method provided by the SimCard class. The latter one is true for FairwavesSIM and WavemobileSim.
Unfortunately the implementation found in the SimCard is wrong. It adds padding at the end of the file instead of the beginning. This completely messes up the contents of EF.SMSP for the cards using this method. To fix this, let's use the leftpad feature provided by the update_record. This will ensure a correct alignment of the file contents.
ts_51_011/EF.SMSP: fix handling of 'alpha_id' field
The field 'alpha_id' is technically not an optional field, even though the specification describes it as optional. Once the card manufacturer decides that the field should be present, it must be always present and vice versa.
(see code comment for a more detailed description)
The current behavior we implement in the method __send_apdu_T0 is incomplete. Some details discussed in ETSI TS 102 221, section 7.3.1.1.4, clause 4 seem to be not fully implemented. We may also end up sending a GET RESPONSE in other APDU cases than case 4 (the only case that uses the GET RESPONSE command).
The AlgorithmID has a few preset values, and hardly anyone knows which is which. So instead of entering '1', '2' or '3', make it work with prededined values 'Milenage', 'TUAK' and 'usim-test'.
Implement the enum value part abstractly in new EnumParam.
Make AlgorithmID a subclass of EnumParam and define the values as from pySim/esim/asn1/saip/PE_Definitions-3.3.1.asn
personalization: add get_typical_input_len() to ConfigurableParameter
The aim is to tell a user interface how wide an input text field should be chosen to be convenient -- ideally showing the entire value in all cases, but not too huge for fields that have no sane size limit.
'securityDomain' elements are decoded to ProfileElementSD instances, which keep higher level representations of the key data apart from the decoded[] lists.
So far, apply_val() was dropping binary values in decoded[], which does not work, because ProfileElementSD._pre_encode() overwrites self.decoded[] from the higher level representation.
Implement using - ProfileElementSD.find_key() and SecurityDomainKeyComponent to modify an exsiting entry, or - ProfileElementSD.add_key() to create a new entry.
Before this patch, SdKey parameters seemed to patch PES successfully, but their modifications did not end up in the encoded DER.
(BTW, this does not fix any other errors that may still be present in the various SdKey subclasses, patches coming up.)
Implement pySim.esim.saip.batch.BatchPersonalization, generating N eSIM profiles from a preset configuration.
Batch parameters can be fed by a constant, incrementing, random or from CSV rows: add pySim.esim.saip.param_source.* classes to feed such input to each of the BatchPersonalization's ConfigurableParameter instances.
personalization: implement reading back values from a PES
Implement get_values_from_pes(), the reverse direction of apply_val(): read back and return values from a ProfileElementSequence. Implement for all ConfigurableParameter subclasses.
Future: SdKey.get_values_from_pes() is reading pe.decoded[], which works fine, but I07dfc378705eba1318e9e8652796cbde106c6a52 will change this implementation to use the higher level ProfileElementSD members.
Implementation detail:
Implement get_values_from_pes() as classmethod that returns a generator. Subclasses should yield all occurences of their parameter in a given PES.
For example, the ICCID can appear in multiple places. Iccid.get_values_from_pes() yields all of the individual values. A set() of the results quickly tells whether the PES is consistent.
Rationales for reading back values:
This allows auditing an eSIM profile, particularly for producing an output.csv from a batch personalization (that generated lots of random key material which now needs to be fed to an HLR...).
Reading back from a binary result is more reliable than storing the values that were fed into a personalization. By auditing final DER results with this code, I discovered: - "oh, there already was some key material in my UPP template." - "all IMSIs ended up the same, forgot to set up the parameter." - the SdKey.apply() implementations currently don't work, see I07dfc378705eba1318e9e8652796cbde106c6a52 for a fix.
personalization: indicate default ParamSource per ConfigurableParameter
Add default_source class members pointing to ParamSource classes to all ConfigurableParameter subclasses.
This is useful to automatically set up a default ParamSource for a given ConfigurableParameter subclass, during user interaction to produce a batch personalization.
For example, if the user selects a Pin1 parameter, a calling program can implicitly set this to a RandomDigitSource, which will magically make it work the way that most users need.
BTW, default_source and default_value can be combined to configure a matching ParamSource instance:
contrib/jenkins.sh: separate JOB_TYPE for card tests
A separate job gives us a possibility to skip tests requiring physical cards for specific commits that do not touch the core logic. See the related commits in osmo-ci.git.
