RecordStorage

Purpose

The RecordStorage is a small file system for the flash storage. It is used for storing persistent data, for example the enrollment data of a node or the module settings. Internally, it uses the FlashStorage to access the flash and queue its operations. The RecordStorage is fully fail safe against power loss and corruption. It will intelligently write all records to a single flash page until it is full and will swap the valid data to another page (defragmentation) once space is needed thereby minimizing erase operations on the flash.

Functionality

Most important features of RecordStorage are:

  • creating new records in memory

  • updating existing records

  • deleting existing records

  • protection against power loss

  • defragmentation

A record is identified by its recordId, thus every record need to have unique recordId. Records store some metadata like: id, lenght, crc and the record version in addition to the data. The crc of the record is checked every time a record is accessed. Records with invalid crc will be ignored and removed during next defragmentation or swap. The versionCounter is used determine the version of the record. When a record is updated, it is not deleted, but new record with incremented versionCounter is created. When reading, only the record with newest versionCounter is returned, which means that a record becomes inactive as soon as a newer version of this record is stored. Inactive records are removed during defragmentation.

The versionCounter and pageVersion have a maximum of 65535. This is the maximum number of times a record can be updated and the maximum number of times that pages can be swapped.

Record storage needs to be assigned a number of pages in flash memory that are not used by the application. The minimium number of pages is 2 (one data and 1 swap page). The swap page is the page that currently does not contain any data. Whenever all other pages are full, the page which can be defragmented the most is defragmented and copied to the swap page. After validation of the records, the old page is erased and becomes the swap page. During defragmentation, all active records will be moved but inactive records will be omitted.

Usage

Saving or updating records and deleting them are all non-blocking operations which are cached and executed asynchronously. Users can register listener when scheduling operation to get notified once the operation was executed. In the handler, the user will get information about the result of the operation. A userType and user context data can be given to determine the operation.

Reading from recordStorage is done synchronously as a simple access to flash memory.