[[ipc_if]] == osmo-trx-ipc IPC Interface This interface is the one used by _osmo_trx_ipc_ backend to communicate to a third party process in charge of driving the lowest layer device-specific bits (from now on the Driver). It consists of a set of Unix Domain (UD) sockets for the control plane, plus a shared memory region for the data plane. Related code can be found in the https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc[Transceiver52M/device/ipc/] directory in _osmo-trx.git_. If you are a potential driver implementator, the various primitives and data structures are publicly available in header file https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]. === Control plane Control plane protocol is transmitted over Unix Domain (UD) sockets using message based primitives. Each primitive has a type identified by an integer, and each type of primitive has a number of extra attributes attached to it. The IPC interface consists of 2 types of UD sockets: * _Master_ UD socket: One per osmo-trx-ipc process. * _Channel_ UD socket: One for each channel managed by osmo-trx-ipc process. The _Driver_ is in all cases expected to take the server role when creating UD sockets, while _osmo-trx-ipc_ takes the client role and connects to sockets provided by the driver. === Master UD socket During startup, _osmo-trx-ipc_ will try connecting to the _Driver_ Master UD socket located in the path provided by its own (VTY) configuration. As a result, it means the _Driver_ process must be running and listening on the Master UD socket before _osmo-trx-ipc_ is started, otherwise _osmo-trx-ipc_ will fail and exit. Once connected, _osmo-trx-ipc_ will submit a `GREETING_REQ` message primitive announcing the maximum supported protocol version (first version ever is `1`, increasing over time). The _Driver_ shall then answer in `GREETING_CNF` message primitive with its own maximum supported version (`<=` version received), providing 0 if none is supported. If _osmo-trx-ipc_ receives back the requested version, then both sides agreed on the protocol version to use. If _osmo-trx-ipc_ receives back a lower version, it shall decide to continue with version negotiation using a lower version, until a supported version or 0 is received. If finally 0 is received, _osmo-trx-ipc_ will disconnect and exit with failure. Once the version is negotiated (`v1` as of current date), _osmo-trx-ipc_ will ask for device information and available characeristics to the _Driver_ using the `INFO_REQ` message primitive. The _Driver_ shall then answer with a `INFO_CNF` message containing information, such as: * String containing device description * Available reference clocks, * {rx,tx} I/Q scaling factors * Maximum number of channels supported * for each channel: ** List of available {rx,tx} paths/antennas. ** {min,max}{rx,tx} gains ** Nominal transmit power All the information received from the _Driver_ during `INFO_CNF` will be used by _osmo-trx-ipc_ to decide whether it can fullfil the requested configuration from the user, and proceed to open the device, or exit with a failure (for instance number of channels, referece clock or tx/rx antenna selected by the user cannot be fullfilled). _osmo-trx-ipc_ will then proceed to open the device and do an initial configuration using an `OPEN_REQ` message, where it will provide the _Driver_ with the desired selected configuration (such as number of channels, rx/tx paths, clock reference, bandwidth filters, etc.). The _Driver_ shall then configure the device and send back a `OPEN_CNF` with: * `return_code` integer attribute set to `0` on success or `!0` on error. * Name of the Posix Shared Memory region where data plane is going to be transmitted. * One path for each channel, containing the just-created UD socket to manage that channel (for instance by taking Master UD socket path and appending `_$chan_idx`). * Path Delay: this is the loopback path delay in samples (= used as a timestamp offset internally by _osmo-trx-ipc_), this value contains the analog delay as well as the delay introduced by the digital filters in the fpga in the sdr devices, and is therefore device type and bandwidth/sample rate dependant. This can not be omitted, wrong values will lead to a _osmo-trx-ipc_ that just doesn't detect any bursts. Finally, _osmo-trx-ipc_ will connect to each channel's UD socket (see next section). Upon _osmo-trx-ipc_ closing the UD master socket connection, the _Driver_ shall go into _closed_ state: stop all processing and instruct the device to power off. TIP: See https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h] for the detailed definition of all the related message primitives and data types for this socket. === Channel UD Socket This socket can be used by _osmo-trx-ipc_ to start/stop data plane processing or change channel's parameters such as Rx/Tx Frequency, Rx/Tx gains, etc. A channel can be either in _started_ or _stopped_ state. When a channel is created (during `OPEN_REQ` in the Master UD Socket), it's by default in _stopped_ state. `START_REQ` and `STOP_REQ` messages control this state, and eventual failures can be reported through `START_CNF` and `STOP_CNF` by the _Driver_. The message `START_REQ` instructs the _Driver_ to start processing data in the data plane. Similary, `STOP_REQ` instructs the _Driver_ to stop processing data in the data plane. Some parameters are usually changed only when the channel is in stopped mode, for instance Rx/Tx Frequency. TIP: See https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h] for the detailed definition of all the related message primitives and data types for this socket. === Data Plane Data plane protocol is implemented by means of a ring buffer structure on top of Posix Shared Memory (see `man 7 shm_overview`) between _osmo-trx-ipc_ process and the _Driver_. The Posix Shared Memory region is created and its memory structure prepared by the _Driver_ and its name shared with _osmo-trx-ipc_ during _OPEN_CNF_ message in the Master UD Socket from the Control Plane. Resource allocation for the shared memory area and cleanup is up to the ipc server, as is mutex initialization for the buffers. ==== Posix Shared Memory structure [[fig-shm-structure]] .General overview of Posix Shared Memory structure [graphviz] ---- digraph hierarchy { node[shape=record,style=filled,fillcolor=gray95] edge[dir=back, arrowtail=empty] SHM[label = "{Posix Shared Memory region|+ num_chans\l+ Channels[]\l}"] CHAN0[label = "{Channel 0|...}"] CHAN1[label = "{Channel 1|...}"] CHANN[label = "{Channel ...|}"] STREAM0_UL[label = "{UL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"] STREAM0_DL[label = "{DL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"] STREAM1_UL[label = "{UL Stream|...}"] STREAM1_DL[label = "{DL Stream|...}"] STREAMN_UL[label = "{UL Stream|...}"] STREAMN_DL[label = "{DL Stream|...}"] BUF_0DL0[label = "{DL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"] BUF_0DLN[label = "{DL Sample Buffer ....|...}"] BUF_0UL0[label = "{UL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"] BUF_0ULN[label = "{UL Sample Buffer ...|...}"] SHM->CHAN0 SHM->CHAN1 SHM->CHANN CHAN0->STREAM0_DL CHAN0->STREAM0_UL STREAM0_DL->BUF_0DL0 STREAM0_DL->BUF_0DLN STREAM0_UL->BUF_0UL0 STREAM0_UL->BUF_0ULN CHAN1->STREAM1_UL CHAN1->STREAM1_DL CHANN->STREAMN_UL CHANN->STREAMN_DL } ---- The Posix Shared Memory region contains an array of _Channels_. Each _Channel_ contains 2 Streams: * Downlink _Stream_ * Uplink _Stream_ Each _Stream_ handles a ring buffer, which is implemented as: * An array of pointers to _Sample Buffer_ structures. * Variables containing the number of buffers in the array, as well as the maximum size in samples for each Sample Buffer. * Variables containing `next_read` and `next_write` _Sample Buffer_ (its index in the array of pointers). * Unnamed Posix semaphores to do the required locking while using the ring buffer. Each _Sample Buffer_ contains: * A `timestamp` variable, containing the position in the stream of the first sample in the buffer * A `data_len` variable, containing the amount of samples available to process in the buffer * An array of samples of size specified by the stream struct it is part of. ==== Posix Shared Memory format The Posix Shared memory region shall be formatted applying the following considerations: * All pointers in the memory region are encoded as offsets from the start address of the region itself, to allow different processes with different address spaces to decode them. * All structs must be force-aligned to 8 bytes * Number of buffers must be power of 2 (2,4,8,16,...) - 4 appears to be plenty * IQ samples format: One (complex) sample consists of 16bit i + 16bit q, so the buffer size is number of IQ pairs. * A reasonable per-buffer size (in samples) is 2500, since this happens to be the ususal TX (downlink) buffer size used by _osmo-trx-ipc_ with the b210 (rx over-the-wire packet size for the b210 is 2040 samples, so the larger value of both is convenient). TIP: See https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h] for the detailed definition of all the objects being part of the Posix Shared memory region structure ==== Posix Shared Memory procedures The queue in the shared memory area is not supposed to be used for actual buffering of data, only for exchange, so the general expectation is that it is mostly empty. The only exception to that might be minor processing delays, and during startup. Care must be taken to ensure that only timed waits for the mutex protecting it and the condition variables are used, in order to ensure that no deadlock occurs should the other side die/quit. Thread cancellation should be disabled during reads/writes from/to the queue. In general a timeout can be considered a non recoverable error during regular processing after startup, at least with the current timeout value of one second. Should over- or underflows occur a corresponding message should be sent towards _osmo-trx-ipc_. Upon **read** of `N` samples, the reader does something like: . Acquire the semaphore in the channel's stream object. . Read `stream->next_read`, if `next_read==next_write`, become blocked in another sempahore (unlocking the previous one) until writer signals us, then `buff = stream->buffers[next_read]` . Read `buff->data_len` samples, reset the buffer data (`data_len=0`), increment `next_read` and if read samples is `buffers[next_write]`. If `data_len!=0`, signal `buffer_overflow` (increase field in stream object) and probably increase next_read`. . Increase `next_write`. . If `next_write` was `== next_read`, signal the reader through the other semaphore that it can continue reading.