#include <Hdf5DataWriter.hpp>

Inheritance diagram for Hdf5DataWriter:

Collaboration diagram for Hdf5DataWriter:

Public Member Functions
	Hdf5DataWriter (DistributedVectorFactory &rVectorFactory, const std::string &rDirectory, const std::string &rBaseName, bool cleanDirectory=true, bool extendData=false, std::string datasetName="Data", bool useCache=false)

virtual	~Hdf5DataWriter ()

void	DefineFixedDimension (long dimensionSize)

void	DefineFixedDimension (const std::vector< unsigned > &rNodesToOuput, long vecSize)

void	DefineUnlimitedDimension (const std::string &rVariableName, const std::string &rVariableUnits, unsigned estimatedLength=1)

void	AdvanceAlongUnlimitedDimension ()

int	DefineVariable (const std::string &rVariableName, const std::string &rVariableUnits)

bool	IsInDefineMode ()

virtual void	EndDefineMode ()

void	PossiblyExtend ()

void	EmptyDataset ()

void	PutVector (int variableID, Vec petscVector)

void	PutStripedVector (std::vector< int > variableIDs, Vec petscVector)

bool	GetUsingCache ()

void	WriteCache ()

void	PutUnlimitedVariable (double value)

void	Close ()

int	GetVariableByName (const std::string &rVariableName)

bool	ApplyPermutation (const std::vector< unsigned > &rPermutation, bool unsafeExtendingMode=false)

void	DefineFixedDimensionUsingMatrix (const std::vector< unsigned > &rNodesToOuput, long vecSize)

void	SetFixedChunkSize (const unsigned &rTimestepsPerChunk, const unsigned &rNodesPerChunk, const unsigned &rVariablesPerChunk)

void	SetTargetChunkSize (hsize_t targetSize)

void	SetAlignment (hsize_t alignment)

Public Member Functions inherited from AbstractHdf5Access
	AbstractHdf5Access (const std::string &rDirectory, const std::string &rBaseName, const std::string &rDatasetName, bool makeAbsolute=true)

	AbstractHdf5Access (const FileFinder &rDirectory, const std::string &rBaseName, const std::string &rDatasetName)

virtual	~AbstractHdf5Access ()

bool	IsDataComplete ()

std::vector< unsigned >	GetIncompleteNodeMap ()

std::string	GetUnlimitedDimensionName ()

std::string	GetUnlimitedDimensionUnit ()

Private Member Functions
void	CheckVariableName (const std::string &rName)

void	CheckUnitsName (const std::string &rName)

void	ComputeIncompleteOffset ()

void	OpenFile ()

hsize_t	CalculateNumberOfChunks ()

void	CalculateChunkDims (unsigned targetSize, unsigned pChunkSizeInBytes, bool pAllOneChunk)

void	SetChunkSize ()

Private Attributes
DistributedVectorFactory &	mrVectorFactory

const bool	mCleanDirectory

const bool	mUseExistingFile

bool	mIsInDefineMode

bool	mIsFixedDimensionSet

unsigned	mEstimatedUnlimitedLength

unsigned	mFileFixedDimensionSize

unsigned	mDataFixedDimensionSize

unsigned	mLo

unsigned	mHi

unsigned	mNumberOwned

unsigned	mOffset

bool	mNeedExtend

bool	mUseMatrixForIncompleteData

std::vector< DataWriterVariable >	mVariables

long unsigned	mCurrentTimeStep

Mat	mSinglePermutation

Mat	mDoublePermutation

Mat	mSingleIncompleteOutputMatrix

Mat	mDoubleIncompleteOutputMatrix

bool	mUseOptimalChunkSizeAlgorithm

hsize_t	mChunkSize [DATASET_DIMS]

hsize_t	mNumberOfChunks

hsize_t	mFixedChunkSize [DATASET_DIMS]

hsize_t	mChunkTargetSize

hsize_t	mAlignment

bool	mUseCache

long unsigned	mCacheFirstTimeStep

std::vector< double >	mDataCache

Friends
class	TestHdf5DataWriter

Additional Inherited Members
Protected Member Functions inherited from AbstractHdf5Access
bool	DoesDatasetExist (const std::string &rDatasetName)

void	SetUnlimitedDatasetId ()

void	SetMainDatasetRawChunkCache ()

Protected Attributes inherited from AbstractHdf5Access
std::string	mBaseName

std::string	mDatasetName

FileFinder	mDirectory

bool	mIsDataComplete

std::string	mUnlimitedDimensionName

std::string	mUnlimitedDimensionUnit

bool	mIsUnlimitedDimensionSet

std::vector< unsigned >	mIncompleteNodeIndices

hid_t	mFileId

hid_t	mUnlimitedDatasetId

hid_t	mVariablesDatasetId

hsize_t	mDatasetDims [DATASET_DIMS]

Static Protected Attributes inherited from AbstractHdf5Access
static const unsigned	DATASET_DIMS =3

Detailed Description

A concrete HDF5 data writer class.

Definition at line 48 of file Hdf5DataWriter.hpp.

Constructor & Destructor Documentation

Hdf5DataWriter::Hdf5DataWriter	(	DistributedVectorFactory &	rVectorFactory,
		const std::string &	rDirectory,
		const std::string &	rBaseName,
		bool	cleanDirectory = `true`,
		bool	extendData = `false`,
		std::string	datasetName = `"Data"`,
		bool	useCache = `false`
	)

Constructor.

Parameters

rVectorFactory	the factory to use in creating PETSc Vec and DistributedVector objects.
rDirectory	the directory in which to write the data to file, relative to chaste test output.
rBaseName	the name of the file in which to write the data
cleanDirectory	whether to clean the directory (defaults to true)
extendData	whether to try opening an existing file and appending to it.
datasetName	The name of the HDF5 dataset to write, defaults to "Data".
useCache	Whether to cache writes so only whole chunks are written to disk.

The extendData parameter allows us to add to an existing dataset. It only really makes sense if the existing file has an unlimited dimension which we can extend. It also only makes sense if cleanDirectory is false, otherwise there won't be a file there to read...

Todo:: 1300 We can't set mDataFixedDimensionSize, because the information isn't in the input file. This means that checking the size of input vectors in PutVector and PutStripedVector is impossible.

Definition at line 52 of file Hdf5DataWriter.cpp.

Hdf5DataWriter::~Hdf5DataWriter ( )

virtual

Destructor.

Definition at line 271 of file Hdf5DataWriter.cpp.

References Close(), PetscTools::Destroy(), mDoubleIncompleteOutputMatrix, mDoublePermutation, mSingleIncompleteOutputMatrix, and mSinglePermutation.

Member Function Documentation

void Hdf5DataWriter::AdvanceAlongUnlimitedDimension ( )

Advance along the unlimited dimension. Normally this will be called when all variables in a row have been input.

Definition at line 1225 of file Hdf5DataWriter.cpp.

References EXCEPTION, mChunkSize, mCurrentTimeStep, AbstractHdf5Access::mDatasetDims, mEstimatedUnlimitedLength, AbstractHdf5Access::mIsUnlimitedDimensionSet, mNeedExtend, mUseCache, and WriteCache().

Referenced by Hdf5DataWriter().

bool Hdf5DataWriter::ApplyPermutation	(	const std::vector< unsigned > &	rPermutation,
		bool	unsafeExtendingMode = `false`
	)

Apply a permutation to all occurences of PutVector Should be called when in define mode

Parameters

rPermutation	a forward/?reverse permutation
unsafeExtendingMode	is true when we are extending a file which requires a permutation to be applied to it. In particular we are extending a cardiac simulation with "original node ordering"

Returns: success value. A value "false" indictates that the permutation was empty or was the identity and was not applied

Definition at line 1295 of file Hdf5DataWriter.cpp.

References EXCEPTION, mDataFixedDimensionSize, mDoublePermutation, mFileFixedDimensionSize, mHi, mIsInDefineMode, mLo, mSinglePermutation, and PetscTools::SetupMat().

void Hdf5DataWriter::CalculateChunkDims	(	unsigned	targetSize,
		unsigned *	pChunkSizeInBytes,
		bool *	pAllOneChunk
	)

private

Calculate (and save in the member variables) chunk dimensions based on a target size. Chunks are kept as "square" as possible while wasting as little space around the edges of the dataset as possible.

Parameters

[in]	targetSize	The target number of entries in each dimension in the chunk.
[out]	pChunkSizeInBytes	The size of the resulting chunk in bytes.
[out]	pAllOneChunk	Whether the dataset is spanned by a single chunk.

Definition at line 1406 of file Hdf5DataWriter.cpp.

References CeilDivide(), AbstractHdf5Access::DATASET_DIMS, mChunkSize, and AbstractHdf5Access::mDatasetDims.

Referenced by SetChunkSize().

hsize_t Hdf5DataWriter::CalculateNumberOfChunks ( )

private

Little method to calculate the number of chunks resulting from given chunk dimensions.

Returns: The number of chunks resulting from chunk dimensions (stored in member variable).

Definition at line 1395 of file Hdf5DataWriter.cpp.

References CeilDivide(), AbstractHdf5Access::DATASET_DIMS, mChunkSize, and AbstractHdf5Access::mDatasetDims.

Referenced by SetChunkSize().

void Hdf5DataWriter::CheckUnitsName ( const std::string & rName )

private

Check name of unit is allowed, i.e. contains only alphanumeric & _, and isn't blank.

Parameters

rName unit name

Definition at line 558 of file Hdf5DataWriter.cpp.

References EXCEPTION.

Referenced by CheckVariableName(), and DefineVariable().

void Hdf5DataWriter::CheckVariableName ( const std::string & rName )

private

Check name of variable is allowed, i.e. contains only alphanumeric & _, and isn't blank.

Parameters

rName variable name

Definition at line 549 of file Hdf5DataWriter.cpp.

References CheckUnitsName(), and EXCEPTION.

Referenced by DefineVariable().

void Hdf5DataWriter::Close ( )

Close any open files.

Definition at line 1182 of file Hdf5DataWriter.cpp.

References AbstractHdf5Access::mFileId, mIsInDefineMode, AbstractHdf5Access::mIsUnlimitedDimensionSet, AbstractHdf5Access::mUnlimitedDatasetId, mUseCache, AbstractHdf5Access::mVariablesDatasetId, and WriteCache().

Referenced by ~Hdf5DataWriter().

void Hdf5DataWriter::ComputeIncompleteOffset ( )

private

Compute mOffset and mNumberOwned from mIncompleteNodeIndices when we have incomplete data.

Definition at line 497 of file Hdf5DataWriter.cpp.

References mHi, AbstractHdf5Access::mIncompleteNodeIndices, mLo, mNumberOwned, and mOffset.

Referenced by DefineFixedDimension(), DefineFixedDimensionUsingMatrix(), and Hdf5DataWriter().

void Hdf5DataWriter::DefineFixedDimension ( long dimensionSize )

Define the fixed dimension, assuming complete data output (all the nodes).

Parameters

dimensionSize The size of the dimension

Definition at line 388 of file Hdf5DataWriter.cpp.

References EXCEPTION, DistributedVectorFactory::GetHigh(), DistributedVectorFactory::GetLocalOwnership(), DistributedVectorFactory::GetLow(), mDataFixedDimensionSize, mFileFixedDimensionSize, mHi, mIsFixedDimensionSet, mIsInDefineMode, mLo, mNumberOwned, mOffset, and mrVectorFactory.

Referenced by DefineFixedDimension(), DefineFixedDimensionUsingMatrix(), ExtendedBidomainProblem< DIM >::save(), and AbstractCardiacProblem< ELEMENT_DIM, SPACE_DIM, 2 >::save().

void Hdf5DataWriter::DefineFixedDimension	(	const std::vector< unsigned > &	rNodesToOuput,
		long	vecSize
	)

Define the fixed dimension, assuming incomplete data output (subset of the nodes).

Parameters

rNodesToOuput	Node indexes to be output (precondition: to be monotonic increasing)
vecSize

Definition at line 413 of file Hdf5DataWriter.cpp.

References ComputeIncompleteOffset(), DefineFixedDimension(), EXCEPTION, mFileFixedDimensionSize, AbstractHdf5Access::mIncompleteNodeIndices, and AbstractHdf5Access::mIsDataComplete.

void Hdf5DataWriter::DefineFixedDimensionUsingMatrix	(	const std::vector< unsigned > &	rNodesToOuput,
		long	vecSize
	)

Define the fixed dimension, assuming incomplete data output (subset of the nodes) and using a matrix to convert from full to incomplete output (rather than picking required data values out one at a time).

Parameters

rNodesToOuput	Node indexes to be output (precondition: to be monotonic increasing)
vecSize

Definition at line 438 of file Hdf5DataWriter.cpp.

References ComputeIncompleteOffset(), DefineFixedDimension(), EXCEPTION, mDataFixedDimensionSize, mDoubleIncompleteOutputMatrix, mFileFixedDimensionSize, mHi, AbstractHdf5Access::mIncompleteNodeIndices, AbstractHdf5Access::mIsDataComplete, mLo, mNumberOwned, mOffset, mSingleIncompleteOutputMatrix, mUseMatrixForIncompleteData, and PetscTools::SetupMat().

void Hdf5DataWriter::DefineUnlimitedDimension	(	const std::string &	rVariableName,
		const std::string &	rVariableUnits,
		unsigned	estimatedLength = `1`
	)

Define a variable.

Parameters

rVariableName	The name of the dimension
rVariableUnits	The physical units of the dimension
estimatedLength	An estimate of the unlimited dimension length for performance reasons

Definition at line 1205 of file Hdf5DataWriter.cpp.

References EXCEPTION, mEstimatedUnlimitedLength, mIsInDefineMode, AbstractHdf5Access::mIsUnlimitedDimensionSet, AbstractHdf5Access::mUnlimitedDimensionName, and AbstractHdf5Access::mUnlimitedDimensionUnit.

Referenced by VoltageInterpolaterOntoMechanicsMesh< DIM >::VoltageInterpolaterOntoMechanicsMesh().

int Hdf5DataWriter::DefineVariable	(	const std::string &	rVariableName,
		const std::string &	rVariableUnits
	)

Define a variable.

Parameters

rVariableName	The name of the dimension
rVariableUnits	The physical units of the dimension

Returns: The identifier of the variable

Definition at line 514 of file Hdf5DataWriter.cpp.

References CheckUnitsName(), CheckVariableName(), EXCEPTION, mIsInDefineMode, DataWriterVariable::mVariableName, mVariables, and DataWriterVariable::mVariableUnits.

Referenced by PostProcessingWriter< ELEMENT_DIM, SPACE_DIM >::WriteOutputDataToHdf5().

void Hdf5DataWriter::EmptyDataset ( )

Reset mCurrentTimeStep to 0 and resize the dataset and unlimited dataset to size 1 in the first dimension. Future writes will therefore overwrite the original contents.

* THIS METHOD (EFFECTIVELY) DELETES THE DATASET! * *

Definition at line 1265 of file Hdf5DataWriter.cpp.

References mCurrentTimeStep, AbstractHdf5Access::mDatasetDims, mNeedExtend, and PossiblyExtend().

void Hdf5DataWriter::EndDefineMode ( )

virtual

bool Hdf5DataWriter::GetUsingCache ( )

Whether we're caching writes

Returns: whether we're caching writes

Definition at line 1090 of file Hdf5DataWriter.cpp.

References mUseCache.

int Hdf5DataWriter::GetVariableByName ( const std::string & rVariableName )

Get the id of the given variable, the variable must already exist or an exception will be thrown.

Parameters

rVariableName variable name to look up

Returns: HDF5 id for the given variable.

Definition at line 1275 of file Hdf5DataWriter.cpp.

References EXCEPTION, and mVariables.

bool Hdf5DataWriter::IsInDefineMode ( )

Check whether writer is in define mode.

When in define mode variables can be defined but no data may be written, and vice versa. When extending, the writer will be in define mode if the dataset does not exist, and vice versa.

Returns: whether in define mode.

Definition at line 570 of file Hdf5DataWriter.cpp.

References mIsInDefineMode.

void Hdf5DataWriter::OpenFile ( )

private

Opens an existing file or creates a new file, depending on mUseExistingFile.

This method sets mFileId.

Definition at line 293 of file Hdf5DataWriter.cpp.

References RelativeTo::Absolute, EXCEPTION, FileFinder::Exists(), FileFinder::GetAbsolutePath(), mAlignment, AbstractHdf5Access::mBaseName, mCleanDirectory, AbstractHdf5Access::mDatasetDims, AbstractHdf5Access::mDirectory, mEstimatedUnlimitedLength, mFileFixedDimensionSize, AbstractHdf5Access::mFileId, mUseExistingFile, mVariables, and AbstractHdf5Access::mVariablesDatasetId.

Referenced by EndDefineMode(), and Hdf5DataWriter().

void Hdf5DataWriter::PossiblyExtend ( )

Extend the dataset to the correct to the correct dimensions if needed.

Definition at line 1255 of file Hdf5DataWriter.cpp.

References AbstractHdf5Access::mDatasetDims, mNeedExtend, AbstractHdf5Access::mUnlimitedDatasetId, and AbstractHdf5Access::mVariablesDatasetId.

Referenced by EmptyDataset(), PutStripedVector(), PutUnlimitedVariable(), and PutVector().

void Hdf5DataWriter::PutStripedVector	(	std::vector< int >	variableIDs,
		Vec	petscVector
	)

Write data for two variables from a PETSc vector to the dataset.

Parameters

variableIDs	vector of IDs of variables of the columns in the HDF5 file
petscVector	the data

Definition at line 917 of file Hdf5DataWriter.cpp.

References PetscTools::CreateVec(), AbstractHdf5Access::DATASET_DIMS, PetscTools::Destroy(), EXCEPTION, mCurrentTimeStep, mDataCache, mDataFixedDimensionSize, AbstractHdf5Access::mDatasetDims, mDoubleIncompleteOutputMatrix, mDoublePermutation, mFileFixedDimensionSize, AbstractHdf5Access::mIncompleteNodeIndices, AbstractHdf5Access::mIsDataComplete, mIsInDefineMode, AbstractHdf5Access::mIsUnlimitedDimensionSet, mLo, mNumberOwned, mOffset, mUseCache, mUseMatrixForIncompleteData, AbstractHdf5Access::mVariablesDatasetId, and PossiblyExtend().

void Hdf5DataWriter::PutUnlimitedVariable ( double value )

Write a single value for the unlimited variable (e.g. time) to the dataset.

Parameters

value the data

Definition at line 1146 of file Hdf5DataWriter.cpp.

References PetscTools::AmMaster(), EXCEPTION, mCurrentTimeStep, mIsInDefineMode, AbstractHdf5Access::mIsUnlimitedDimensionSet, AbstractHdf5Access::mUnlimitedDatasetId, and PossiblyExtend().

void Hdf5DataWriter::PutVector	(	int	variableID,
		Vec	petscVector
	)

Write data for a given variable from a PETSc vector to the dataset.

Parameters

variableID	the variable id of the column in the HDF5 file
petscVector	the data

Definition at line 772 of file Hdf5DataWriter.cpp.

References PetscTools::CreateVec(), AbstractHdf5Access::DATASET_DIMS, PetscTools::Destroy(), EXCEPTION, mCurrentTimeStep, mDataCache, mDataFixedDimensionSize, AbstractHdf5Access::mDatasetDims, mFileFixedDimensionSize, AbstractHdf5Access::mIncompleteNodeIndices, AbstractHdf5Access::mIsDataComplete, mIsInDefineMode, AbstractHdf5Access::mIsUnlimitedDimensionSet, mLo, mNumberOwned, mOffset, mSingleIncompleteOutputMatrix, mSinglePermutation, mUseCache, mUseMatrixForIncompleteData, AbstractHdf5Access::mVariablesDatasetId, and PossiblyExtend().

void Hdf5DataWriter::SetAlignment ( hsize_t alignment )

Set the alignment parameter to pass through to H5Pset_alignment. Every file object will be aligned on the disk to a multiple of this parameter. See the H5P docs for more information.

Especially useful with SetTargetChunkSize to ensure each chunk (each chunk is an 'file object') is aligned to a disk stripe on a striped file system with minimal wastage.

This method only has an effect when creating a NEW HDF5 FILE. Must be called in define mode with extendData set to False.

Parameters

alignment Alignment (bytes)

Definition at line 1488 of file Hdf5DataWriter.cpp.

References EXCEPTION, mAlignment, mIsInDefineMode, and mUseExistingFile.

void Hdf5DataWriter::SetChunkSize ( )

private

This method sets the chunk size by building up in each dimension until a threshold is reached, unless user-specified values have been set using SetFixedChunkSize. By default, chunks of 128 K are used, which seems to be a good compromise. For large problems performance will usually improve by increasing this value (to e.g. 1 M).

Note: The public method for altering the algorithm's target chunk size is SetTargetChunkSize.

Definition at line 1430 of file Hdf5DataWriter.cpp.

References CalculateChunkDims(), CalculateNumberOfChunks(), AbstractHdf5Access::DATASET_DIMS, mChunkSize, mChunkTargetSize, mFixedChunkSize, mNumberOfChunks, and mUseOptimalChunkSizeAlgorithm.

Referenced by EndDefineMode().

void Hdf5DataWriter::SetFixedChunkSize	(	const unsigned &	rTimestepsPerChunk,
		const unsigned &	rNodesPerChunk,
		const unsigned &	rVariablesPerChunk
	)

Use a particular chunk size, ignoring the algorithm that figures out a sensible value.

This method may be useful in very specific circumstances, e.g. if you want chunks to align perfectly with stripes on a Lustre file system, or if you have a parallel problem where every process can be assigned the same size and shape hyperslab of the HDF5 file (thus getting rid of contention between processes when reading/writing).

USE WITH CAUTION - as it can degrade performance.

Parameters

rTimestepsPerChunk	The number of unlimited dimension steps (usually time steps) per chunk.
rNodesPerChunk	The number of objects (usually nodes) per chunk in the second dimension.
rVariablesPerChunk	The number of objects (usually output variables) per chunk in the third dimension.

Definition at line 1383 of file Hdf5DataWriter.cpp.

References mFixedChunkSize, mIsInDefineMode, and mUseOptimalChunkSizeAlgorithm.

void Hdf5DataWriter::SetTargetChunkSize ( hsize_t targetSize )

Adjust the target (max) chunk size in the chunking algorithm. Useful when one knows roughly how big a chunk should be but doesn't care about the exact dimensions or shape. Default is 128 K.

Especially useful with SetAlignment for ensuring each chunk gets its own stripe on striped file systems.

This method only has an effect when creating a NEW DATASET. Must be called in define mode.

Parameters

targetSize Max chunk size (bytes)

Definition at line 1477 of file Hdf5DataWriter.cpp.

References EXCEPTION, mChunkTargetSize, and mIsInDefineMode.

Referenced by AbstractCardiacProblem< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::InitialiseWriter().

void Hdf5DataWriter::WriteCache ( )

Write the cache to disk.

Definition at line 1095 of file Hdf5DataWriter.cpp.

References AbstractHdf5Access::DATASET_DIMS, mCacheFirstTimeStep, mCurrentTimeStep, mDataCache, AbstractHdf5Access::mDatasetDims, mNumberOwned, mOffset, AbstractHdf5Access::mVariablesDatasetId, and PetscTools::ReplicateBool().

Referenced by AdvanceAlongUnlimitedDimension(), and Close().