Base class for serializable objects of dynamic size More...

#include <serializer.h>

Inheritance diagram for olb::BufferSerializable:

Collaboration diagram for olb::BufferSerializable:

Protected Member Functions
template<typename DataType >
void	registerSerializable (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, size_t &sizeBufferIndex, bool *&dataPtr, DataType &data, const bool loadingMode=false)
	Register `Serializable` object of dynamic size.

template<typename DataType >
void	registerStdVectorOfVars (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, size_t &sizeBufferIndex, bool *&dataPtr, std::vector< DataType > &data, const bool loadingMode=false)
	Method for registering a `std::vector<DataType>` of primitive `DataType` (`int`, `double`, ...)

template<typename DataType >
void	registerStdVectorOfSerializablesOfConstSize (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, size_t &sizeBufferIndex, bool *&dataPtr, std::vector< DataType > &data, const bool loadingMode=false)
	Method for registering a `std::vector<DataType>` of constant-sized `Serializable`

template<typename DataType >
void	registerStdVectorOfSerializables (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, size_t &sizeBufferIndex, bool *&dataPtr, std::vector< DataType > &data, const bool loadingMode=false)
	Method for registering a `std::vector<DataType>` of dynamic-sized `DataType`

template<typename DataTypeKey , typename DataTypeValue >
void	registerMap (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, size_t &sizeBufferIndex, bool *&dataPtr, std::map< DataTypeKey, DataTypeValue > &data, const bool loadingMode=false)
	Method for registering a `std::map<DataTypeKey, DataTypeValue>` of fixed-sized types (i.e. `int`, `double`)

size_t	addSizeToBuffer (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, size_t &sizeBufferIndex, bool *&dataPtr, const size_t data) const
	Add a `size_t` to the `sizeBuffer` in the `n-th` util::round and return that `size_t` in all successive rounds.

Protected Member Functions inherited from olb::Serializable
template<typename DataType >
void	registerVar (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, bool *&dataPtr, const DataType &data, const size_t arrayLength=1) const
	Register primitive data types (`int`, `double`, ...) or arrays of those.

template<typename DataType >
void	registerSerializableOfConstSize (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, bool *&dataPtr, DataType &data, const bool loadingMode=false)
	Register `Serializable` object of constant size.

template<typename DataType >
void	registerSerializablesOfConstSize (const std::size_t iBlock, std::size_t &sizeBlock, std::size_t &currentBlock, bool &dataPtr, DataType data, const size_t arrayLength, const bool loadingMode=false)
	Register an array of `Serializable` objects of constant size.

Protected Attributes
std::vector< bool * >	_dataBuffer
	Data buffer for data that has to be buffered between two `getBlock()` iterations.

std::vector< size_t >	_sizeBuffer
	`std::vector` of integer buffers (e.g. for `std::vector` size) to be buffered for the whole iteration process

Additional Inherited Members
Public Member Functions inherited from olb::Serializable
virtual	~Serializable ()=default

virtual bool *	getBlock (const std::size_t iBlock, std::size_t &sizeBlock, const bool loadingMode=false)=0
	Returns the address of the i-th block and its size.

virtual std::size_t	getNblock () const =0
	Returns the number of blocks.

virtual std::size_t	getSerializableSize () const =0
	Returns the binary size of the data to be saved.

template<bool includeLogOutputDir = true>
bool	save (std::string fileName="", const bool enforceUint=false)
	Save `Serializable` into file `fileName`

template<bool includeLogOutputDir = true>
bool	load (std::string fileName="", const bool enforceUint=false)
	Load `Serializable` from file `fileName`

bool	save (std::uint8_t *buffer)
	Save `Serializable` into buffer of length `getSerializableSize`

bool	load (const std::uint8_t *buffer)
	Load `Serializable` from buffer of length `getSerializableSize`

virtual void	postLoad ()

Detailed Description

Base class for serializable objects of dynamic size

All Serializable subclasses with dynamic size (unknown at compile time , e.g. holding std::vector or std::map members) have to inherit from BufferSerializable

For detailed information on the serialization concept, see the Serializable documentation.

BufferSerializable provides register methods for:

Method	Suitable for
`registerSerializable()`	Dynamic-sized `Serializable` object (for constant-sized `Serializable`

| use registerSerializableOfConstSize() registerStdVectorOfVars() | std::vector<DataType> (for primitive DataType, e.g. int, | double, ...) registerStdVectorOfSerializablesOfConstSize() | std::vector<DataType> (for Serializables of constant size) registerStdVectorOfSerializables() | std::vector<DataType> (for Serializables of dynamic size) registerMap() | std::map<DataTypeKey, DataTypeValue> (for primitive types)

Definition at line 299 of file serializer.h.

Member Function Documentation

◆ addSizeToBuffer()

size_t olb::BufferSerializable::addSizeToBuffer	(	const std::size_t	iBlock,
		std::size_t &	sizeBlock,
		std::size_t &	currentBlock,
		size_t &	sizeBufferIndex,
		bool *&	dataPtr,
		const size_t	data ) const

inlineprotected

Add a size_t to the sizeBuffer in the n-th util::round and return that size_t in all successive rounds.

increase currentBlock by one
increase sizeBufferIndex by one.
n-th util::round: push given size_t to sizeBuffer and provide pointer to it.

Definition at line 571 of file serializer.h.

  {
    size_t returnSize = 0;
 
    if (iBlock == currentBlock) {
      // write size into _sizeBuffer vector
      _sizeBuffer.push_back(*new size_t(data));
    }
 
    if (iBlock >= currentBlock) {
      returnSize = _sizeBuffer[sizeBufferIndex];
    }
 
    // register size as var
    registerVar(iBlock, sizeBlock, currentBlock, dataPtr, _sizeBuffer[sizeBufferIndex]);
    sizeBufferIndex++;
 
    return returnSize;
  }

References _sizeBuffer, and olb::Serializable::registerVar().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ registerMap()

template<typename DataTypeKey , typename DataTypeValue >

void olb::BufferSerializable::registerMap	(	const std::size_t	iBlock,
		std::size_t &	sizeBlock,
		std::size_t &	currentBlock,
		size_t &	sizeBufferIndex,
		bool *&	dataPtr,
		std::map< DataTypeKey, DataTypeValue > &	data,
		const bool	loadingMode = false )

inlineprotected

Method for registering a std::map<DataTypeKey, DataTypeValue> of fixed-sized types (i.e. int, double)

This method registers a map of a fixed-sized std::map (consisting of std::pairs<DataTypeKey, DataTypeValue>). The first block holds a size_t sizeOfMap with the size of the map, followed by sizeOfMap blocks of std::pairs.

In case of loading the data, the dataBuffer has to be used in order to provide a valid pointer to a newly created std::pair before inserting that same pair into the map in the next iteration.

The total number of blocks occupied by this method is 1 + sizeOfMap.

Parameters

sizeBufferIndex	see `registerStdVector()`
data	Reference to the data to be registered by this method. Fills `dataPtr` with a `(bool)`-casted pointer (if this is the current block) to: First block* - number of elements in the map second block to last block - pointer to `i-th` map element

Note: In writing mode, dataPtr holds a pointer to the i-th map element (which is a std::pair). In reading mode, dataPtr holds a pointer to the bool* buffer, which holds a newly created std::pair to be filled and that pair is inserted into data in the following util::round.

For information about the other parameters of this method, see registerVar() documentation.

Definition at line 522 of file serializer.h.

  {
    if (iBlock >= currentBlock) {
      // process length of data map
      size_t sizeOfMap = addSizeToBuffer(iBlock, sizeBlock, currentBlock, sizeBufferIndex, dataPtr, data.size());
 
      if (iBlock >= currentBlock && iBlock < currentBlock + sizeOfMap + 1) {
        // determine size of pair
        sizeBlock = sizeof(std::pair<DataTypeKey, DataTypeValue>);
 
        // LOADING MODE
        if (loadingMode) {
          // If pair in dataBuffer => insert into data and delete
          if (iBlock > currentBlock) {
            std::pair<DataTypeKey, DataTypeValue> *pairPtr = (std::pair<DataTypeKey, DataTypeValue> *) _dataBuffer.back();
            data.insert(*pairPtr);  // copy pair into map
            delete pairPtr;         // delete pair object that was created (with new!) in the buffer
            _dataBuffer.pop_back(); // remove pointer to deleted pair from buffer vector
          }
 
          // push new pair into buffer and return pointer
          if (iBlock < currentBlock + sizeOfMap) {
            _dataBuffer.push_back((bool *) new std::pair<DataTypeKey, DataTypeValue>);
            dataPtr = _dataBuffer.back();
          }
        }
 
        // SAVING MODE
        else {
          if (iBlock < currentBlock + sizeOfMap) {
            // advance through iterator to n-th element and return pointer to pair
            auto map_it = data.begin();
            std::advance(map_it, iBlock - currentBlock);
            dataPtr = (bool *) (&(*map_it));
          }
        }
      }
      currentBlock += sizeOfMap;
    }
  }

References _dataBuffer, and addSizeToBuffer().

Here is the call graph for this function:

◆ registerSerializable()

template<typename DataType >

void olb::BufferSerializable::registerSerializable	(	const std::size_t	iBlock,
		std::size_t &	sizeBlock,
		std::size_t &	currentBlock,
		size_t &	sizeBufferIndex,
		bool *&	dataPtr,
		DataType &	data,
		const bool	loadingMode = false )

inlineprotected

Register Serializable object of dynamic size.

This method is suitable for all Serializable objects of _ dynamic size_, e.g. where getNblock() cannot be evaluated correctly on an empty object.

For more information about the parameters of this method, see registerVar().

This method registers a Serializable by buffering the dynamic getNblock() value and afterwards delegating the getBlock() call to the Serializable.

Note: getNblock() only works for writing mode, since it is dynamic.

Definition at line 329 of file serializer.h.

  {
    static_assert(std::is_base_of<Serializable, DataType>::value, "DataType must be a Serializable.");
 
    size_t dataBlockCount = 0;
 
    // hold getNblock() in sizeBuffer
    if (loadingMode) { // loading -> set to 0 and wait for reading next util::round
      dataBlockCount = addSizeToBuffer(iBlock, sizeBlock, currentBlock, sizeBufferIndex, dataPtr, 0);
    }
    else {   // saving -> save getNblock from data object
      dataBlockCount = addSizeToBuffer(iBlock, sizeBlock, currentBlock, sizeBufferIndex, dataPtr, data.getNblock());
    }
 
    if (iBlock >= currentBlock) {
      if (iBlock < currentBlock + dataBlockCount) {
        dataPtr = data.getBlock(iBlock - currentBlock, sizeBlock, loadingMode);
      }
    }
  }

References addSizeToBuffer().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ registerStdVectorOfSerializables()

template<typename DataType >

void olb::BufferSerializable::registerStdVectorOfSerializables	(	const std::size_t	iBlock,
		std::size_t &	sizeBlock,
		std::size_t &	currentBlock,
		size_t &	sizeBufferIndex,
		bool *&	dataPtr,
		std::vector< DataType > &	data,
		const bool	loadingMode = false )

inlineprotected

Method for registering a std::vector<DataType> of dynamic-sized DataType

This method registers a vector of a dynamic-sized Serializable. The first block holds a size_t sizeOfVector with the size of the vector, followed by sizeOfVector many registerSerializable() calls, which occupy data[i].getNblock() blocks each. getNblock() may differ for any data[i].

The sizeBuffer is used to store the length of the registered std::vector as well as the number of blocks occupied by each Serializable in the vector. This method occupies 1 + sizeOfVector many sizeBuffer elements.

The total number of blocks occupied by this method is 1 + sizeOfVector * (1 + data[i].getNblock()).

Parameters

sizeBufferIndex	Index counter for size buffer. Is increased by two by this method, and the size of the registered `std::vector` is stored in the corresponding first `sizeBuffer` element, the fixed number of blocks of `DataType` is stored in the second one.
data	Reference to the data to be registered by this method. Fills `dataPtr` with a `(bool)`-casted pointer (if this is the current block) to: First block* - number of elements in the vector For each `std::vector` element: First Subblock - number of blocks occupied by this `Serializable` object second block to last block - pointer to `i-th` block of `data[i]`

For information about the other parameters of this method, see registerVar() documentation.

Definition at line 471 of file serializer.h.

  {
    static_assert(std::is_base_of<Serializable, DataType>::value, "DataType must be a Serializable.");
 
    if (iBlock >= currentBlock) {
      // process length of data vector
      size_t sizeOfVector = addSizeToBuffer(iBlock, sizeBlock, currentBlock, sizeBufferIndex, dataPtr, data.size());
 
      // resize data vector from buffer (only for loading)
      if (iBlock == currentBlock && loadingMode) {
        data.resize(_sizeBuffer[sizeBufferIndex - 1]);
      }
 
      // process Serializables
      if (iBlock >= currentBlock && sizeOfVector > 0) {
        for ( DataType& dataValue : data ) {
          registerSerializable(iBlock, sizeBlock, currentBlock, sizeBufferIndex, dataPtr, dataValue, loadingMode);
        }
      }
    }
  }

References _sizeBuffer, addSizeToBuffer(), and registerSerializable().

Here is the call graph for this function:

◆ registerStdVectorOfSerializablesOfConstSize()

template<typename DataType >

void olb::BufferSerializable::registerStdVectorOfSerializablesOfConstSize	(	const std::size_t	iBlock,
		std::size_t &	sizeBlock,
		std::size_t &	currentBlock,
		size_t &	sizeBufferIndex,
		bool *&	dataPtr,
		std::vector< DataType > &	data,
		const bool	loadingMode = false )

inlineprotected

Method for registering a std::vector<DataType> of constant-sized Serializable

This method registers a vector of a constant-sized Serializable. The first block holds a size_t sizeOfVector with the size of the vector, the second holding the number of blocks in one DataType, followed by sizeOfVector blocks of DataType, each of which consists of DataType.getNblock() subblocks.

The sizeBuffer is used to store the constant(!) number of subblocks (DataType.getNblock()) of the Serializable.

The total number of blocks occupied by this method is 2 + sizeOfVector*nSubBlock.

Parameters

sizeBufferIndex	Index counter for size buffer. Is increased by two by this method, and the size of the registered `std::vector` is stored in the corresponding first `sizeBuffer` element, the fixed number of blocks of `DataType` is stored in the second one.
data	Reference to the data to be registered by this method. Fills `dataPtr` with a `(bool)`-casted pointer (if this is the current block) to: First block* - number of elements in the vector second block - fixed number of block in `DataType` third block to last block - pointer to `i-th` vector element

For information about the other parameters of this method, see registerVar() documentation.

Definition at line 418 of file serializer.h.

  {
    static_assert(std::is_base_of<Serializable, DataType>::value, "DataType must be a Serializable.");
 
    if (iBlock >= currentBlock) {
      // process length of data vector
      size_t sizeOfVector = addSizeToBuffer(iBlock, sizeBlock, currentBlock, sizeBufferIndex, dataPtr, data.size());
 
      // resize data vector from buffer (only for loading)
      if (iBlock == currentBlock && loadingMode) {
        data.resize(_sizeBuffer[sizeBufferIndex - 1]);
      }
 
 
      // process Serializables
      if (iBlock >= currentBlock && sizeOfVector > 0) {
        for ( DataType& dataValue : data ) {
          registerSerializableOfConstSize(iBlock, sizeBlock, currentBlock, dataPtr, dataValue, loadingMode);
        }
      }
    }
  }

References _sizeBuffer, addSizeToBuffer(), and olb::Serializable::registerSerializableOfConstSize().

Here is the call graph for this function:

◆ registerStdVectorOfVars()

template<typename DataType >

void olb::BufferSerializable::registerStdVectorOfVars	(	const std::size_t	iBlock,
		std::size_t &	sizeBlock,
		std::size_t &	currentBlock,
		size_t &	sizeBufferIndex,
		bool *&	dataPtr,
		std::vector< DataType > &	data,
		const bool	loadingMode = false )

inlineprotected

Method for registering a std::vector<DataType> of primitive DataType (int, double, ...)

This method registers a vector of a primitive DataType. The first block holds a size_t sizeOfVector with the size of the vector, followed by sizeOfVector many blocks of DataType.

The total number of blocks occupied by this method is 1 + sizeOfVector.

Parameters

sizeBufferIndex	Index counter for size buffer. Is increased by one by this method, and the size of the registered `std::vector` is stored in the corresponding `sizeBuffer` element.
data	Reference to the data to be registered by this method. Fills `dataPtr` with a `(bool)`-casted pointer (if this is the current block) to: First block* - number of elements in the vector second block to last block - pointer to `i-th` vector element

For information about the other parameters of this method, see registerVar() documentation.

Definition at line 371 of file serializer.h.

  {
    if (iBlock >= currentBlock) {
      // process length of data vector
      size_t sizeOfVector = addSizeToBuffer(iBlock, sizeBlock, currentBlock, sizeBufferIndex, dataPtr, data.size());
 
      // resize data vector from buffer (only for loading)
      if (iBlock == currentBlock && loadingMode) {
        data.resize(_sizeBuffer[sizeBufferIndex - 1]);
      }
 
      if (iBlock >= currentBlock && iBlock < currentBlock + sizeOfVector) {
        sizeBlock = sizeof(DataType);
        dataPtr = (bool *) (&data[iBlock - currentBlock]);
      }
      currentBlock += sizeOfVector;
    }
  }

References _sizeBuffer, and addSizeToBuffer().

Here is the call graph for this function:

Member Data Documentation

◆ _dataBuffer

std::vector<bool*> olb::BufferSerializable::_dataBuffer

mutableprotected

Data buffer for data that has to be buffered between two getBlock() iterations.

This is currently only used within registerMap().

This is only used for loading.

Definition at line 307 of file serializer.h.

◆ _sizeBuffer

std::vector<size_t> olb::BufferSerializable::_sizeBuffer

mutableprotected

std::vector of integer buffers (e.g. for std::vector size) to be buffered for the whole iteration process

Each register method for dynamic-sized objects (e.g. std::vector or std::map) uses the size buffer to provide to correctly increase the currentBlock variable for all following register methods.

Definition at line 313 of file serializer.h.

The documentation for this class was generated from the following file:

src/core/serializer.h

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ addSizeToBuffer()

◆ registerMap()

◆ registerSerializable()

◆ registerStdVectorOfSerializables()

◆ registerStdVectorOfSerializablesOfConstSize()

◆ registerStdVectorOfVars()

Member Data Documentation

◆ _dataBuffer

◆ _sizeBuffer