PetscTools Class Reference

#include <PetscTools.hpp>

Collaboration diagram for PetscTools:

Static Public Member Functions
static void	ResetCache ()
static bool	IsInitialised ()
static bool	IsSequential ()
static bool	IsParallel ()
static bool	IsIsolated ()
static unsigned	GetNumProcs ()
static unsigned	GetMyRank ()
static bool	AmMaster ()
static bool	AmTopMost ()
static void	Barrier (const std::string callerId="")
static void	BeginRoundRobin ()
static void	EndRoundRobin ()
static void	IsolateProcesses (bool isolate=true)
static MPI_Comm	GetWorld ()
static Vec	CreateVec (int size, int localSize=PETSC_DECIDE, bool ignoreOffProcEntries=true)
static Vec	CreateVec (std::vector< double > data)
static Vec	CreateAndSetVec (int size, double value)
static void	SetupMat (Mat &rMat, int numRows, int numColumns, unsigned rowPreallocation, int numLocalRows=PETSC_DECIDE, int numLocalColumns=PETSC_DECIDE, bool ignoreOffProcEntries=true, bool newAllocationError=true)
static bool	ReplicateBool (bool flag)
static void	ReplicateException (bool flag)
static void	DumpPetscObject (const Mat &rMat, const std::string &rOutputFileFullPath)
static void	DumpPetscObject (const Vec &rVec, const std::string &rOutputFileFullPath)
static void	ReadPetscObject (Mat &rMat, const std::string &rOutputFileFullPath, Vec rParallelLayout=NULL)
static void	ReadPetscObject (Vec &rVec, const std::string &rOutputFileFullPath, Vec rParallelLayout=NULL)
static bool	HasParMetis ()
static void	Destroy (Vec &rVec)
static void	Destroy (Mat &rMat)
Static Public Attributes
static const unsigned	MASTER_RANK = 0
Static Private Member Functions
static void	CheckCache ()
Static Private Attributes
static bool	mPetscIsInitialised = false
static unsigned	mNumProcessors = 0
static unsigned	mRank = 0
static bool	mIsolateProcesses = false

Detailed Description

A helper class of static methods.

Any PETSc operation that can be performed using the methods in this class, should be.

This ensures a consistent interface in Chaste even when PETSc arguments change between PETSc versions. For example VecDestroy takes different arguments in 3.2, and using PetscTools::Destroy(vec); takes care of this.

Definition at line 110 of file PetscTools.hpp.

Member Function Documentation

bool PetscTools::AmMaster ( ) [static]

Returns:: whether it is the master process or not.

If not running in parallel, or if IsolateProcesses has been called, always returns true.

Definition at line 120 of file PetscTools.cpp.

References CheckCache(), MASTER_RANK, mIsolateProcesses, and mRank.

bool PetscTools::AmTopMost ( ) [static]

Returns:: whether it is the right-most process or not.

If not running in parallel, or if IsolateProcesses has been called, always returns true.

Definition at line 126 of file PetscTools.cpp.

References CheckCache(), mIsolateProcesses, mNumProcessors, and mRank.

Referenced by NodeBasedCellPopulation< DIM >::AddReceivedCells(), NodeBasedCellPopulation< DIM >::AddReceivedHaloCells(), DistributedTetrahedralMesh< ELEMENT_DIM, SPACE_DIM >::ConstructFromMeshReader(), NodesOnlyMesh< SPACE_DIM >::EnlargeBoxCollection(), NodeBasedCellPopulation< DIM >::GetReceivedCells(), DistributedBoxCollection< DIM >::LoadBalance(), NodeBasedCellPopulation< DIM >::NonBlockingSendCellsToNeighbourProcesses(), NodePartitioner< ELEMENT_DIM, SPACE_DIM >::PetscMatrixPartitioning(), NodeBasedCellPopulation< DIM >::SendCellsToNeighbourProcesses(), DistributedBoxCollection< DIM >::SetupHaloBoxes(), AbstractTetrahedralMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteFilesUsingParallelMesh(), PostProcessingWriter< ELEMENT_DIM, SPACE_DIM >::WriteGenericFileToMeshalyzer(), AbstractTetrahedralMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteNclFile(), and AbstractCellPopulation< ELEMENT_DIM, SPACE_DIM >::WriteResultsToFiles().

void PetscTools::Barrier ( const std::string callerId = "" ) [static]

If MPI is set up, perform a barrier synchronisation. If not, or if IsolateProcesses has been called, it's a noop.

Parameters:

callerId

only used in debug mode; printed before & after the barrier call

Definition at line 134 of file PetscTools.cpp.

References CheckCache(), GetMyRank(), mIsolateProcesses, and mPetscIsInitialised.

void PetscTools::BeginRoundRobin ( ) [static]

Call at the start of a block of code that should be executed by each process in turn.

Note that this is not reliable for printing output to stdout in an ordered fashion, since on some systems each process may have a separate stdout buffer, and there's no way to force a flush to the underlying output stream. See e.g. http://stackoverflow.com/questions/5182045/openmpi-mpi-barrier-problems for more info.

Definition at line 149 of file PetscTools.cpp.

References Barrier(), and GetMyRank().

Referenced by ActivationOutputModifier::FinaliseAtEnd(), GenericEventHandler< 11, MeshEventHandler >::ReportImpl(), ExecutableSupport::ShowParallelLaunching(), AbstractTetrahedralMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteFilesUsingParallelMesh(), PostProcessingWriter< ELEMENT_DIM, SPACE_DIM >::WriteGenericFileToMeshalyzer(), AbstractTetrahedralMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteNclFile(), and AbstractCellPopulation< ELEMENT_DIM, SPACE_DIM >::WriteResultsToFiles().

static void PetscTools::CheckCache ( ) [inline, static, private]

Private method makes sure that (if this is the first use within a test) then PETSc has been probed.

Definition at line 127 of file PetscTools.hpp.

References mNumProcessors, and ResetCache().

Referenced by AmMaster(), AmTopMost(), Barrier(), GetMyRank(), GetNumProcs(), IsInitialised(), IsParallel(), IsSequential(), and ReplicateBool().

Vec PetscTools::CreateAndSetVec	(	int	size,
		double	value
	)			`[static]`

Create a vector of the specified size with all values set to be the given constant. SetFromOptions is called.

Parameters:

	size	the size of the vector
	value	the value to set each entry

Returns:: new PETSc vector

Definition at line 253 of file PetscTools.cpp.

References CreateVec().

Referenced by OdeLinearSystemSolver::OdeLinearSystemSolver(), NodeBasedCellPopulationWithBuskeUpdate< DIM >::UpdateNodeLocations(), and AbstractNonlinearAssemblerSolverHybrid< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::VerifyJacobian().

Vec PetscTools::CreateVec ( std::vector< double > data ) [static]

Create a Vec from the given data.

Parameters:

data

some data

Returns:: new PETSc vector

Definition at line 233 of file PetscTools.cpp.

References CreateVec().

Vec PetscTools::CreateVec	(	int	size,
		int	localSize = `PETSC_DECIDE`,
		bool	ignoreOffProcEntries = `true`
	)			`[static]`

Create a vector of the specified size. SetFromOptions is called.

Parameters:

	size	the size of the vector
	localSize	the local number of items owned by this process
	ignoreOffProcEntries	whether to ignore entries destined to be stored on a separate processor when assembling (eliminates global reductions).

Returns:: new PETSc vector

Definition at line 213 of file PetscTools.cpp.

Referenced by AbstractNonlinearAssemblerSolverHybrid< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::ComputeJacobianNumerically(), PseudoEcgCalculator< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::ComputePseudoEcgAtOneTimeStep(), CreateAndSetVec(), CreateVec(), PetscMatTools::GetMatrixRowDistributed(), LinearSystem::LinearSystem(), PCBlockDiagonal::PCBlockDiagonalCreate(), PCLDUFactorisation::PCLDUFactorisationCreate(), PCTwoLevelsBlockDiagonal::PCTwoLevelsBlockDiagonalCreate(), Hdf5DataWriter::PutStripedVector(), Hdf5DataWriter::PutVector(), PetscVecTools::SetupInterleavedVectorScatterGather(), and VoltageInterpolaterOntoMechanicsMesh< DIM >::VoltageInterpolaterOntoMechanicsMesh().

static void PetscTools::Destroy ( Mat & rMat ) [inline, static]

Destroy method Note that PETSc 3.1 and previous destroy based on a PETSc object but PETSc 3.2 and later destroy based on a pointer to a PETSc object

Parameters:

rMat

a reference to the PETSc object

Definition at line 367 of file PetscTools.hpp.

static void PetscTools::Destroy ( Vec & rVec ) [inline, static]

Destroy method Note that PETSc 3.1 and previous destroy based on a PETSc object but PETSc 3.2 and later destroy based on a pointer to a PETSc object

Parameters:

rVec

a reference to the PETSc object

Definition at line 351 of file PetscTools.hpp.

void PetscTools::DumpPetscObject	(	const Vec &	rVec,
		const std::string &	rOutputFileFullPath
	)			`[static]`

Dumps a given PETSc object to disk.

Parameters:

	rVec	a vector
	rOutputFileFullPath	where to dump the vector to disk

Definition at line 346 of file PetscTools.cpp.

References PETSC_DESTROY_PARAM.

void PetscTools::DumpPetscObject	(	const Mat &	rMat,
		const std::string &	rOutputFileFullPath
	)			`[static]`

Dumps a given PETSc object to disk.

Parameters:

	rMat	a matrix
	rOutputFileFullPath	where to dump the matrix to disk

Definition at line 332 of file PetscTools.cpp.

References PETSC_DESTROY_PARAM.

void PetscTools::EndRoundRobin ( ) [static]

Call at the end of a block of code that should be executed by each process in turn.

Definition at line 159 of file PetscTools.cpp.

References Barrier(), GetMyRank(), and GetNumProcs().

Referenced by ActivationOutputModifier::FinaliseAtEnd(), GenericEventHandler< 11, MeshEventHandler >::ReportImpl(), ExecutableSupport::ShowParallelLaunching(), AbstractTetrahedralMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteFilesUsingParallelMesh(), PostProcessingWriter< ELEMENT_DIM, SPACE_DIM >::WriteGenericFileToMeshalyzer(), AbstractTetrahedralMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteNclFile(), and AbstractCellPopulation< ELEMENT_DIM, SPACE_DIM >::WriteResultsToFiles().

unsigned PetscTools::GetMyRank ( ) [static]

Returns:: our rank.

If PETSc has not been initialized, returns 0.

Definition at line 114 of file PetscTools.cpp.

References CheckCache(), and mRank.

unsigned PetscTools::GetNumProcs ( ) [static]

Returns:: total number of processors.

Definition at line 108 of file PetscTools.cpp.

References CheckCache(), and mNumProcessors.

MPI_Comm PetscTools::GetWorld ( ) [static]

Get the MPI Communicator representing the whole set of running processes. This will normally be PETSC_COMM_WORLD, unless IsolateProcesses has been called, in which case it will be PETSC_COMM_SELF.

Returns:: the MPI Communicator representing the whole set of running processes

Definition at line 173 of file PetscTools.cpp.

References mIsolateProcesses.

Referenced by AbstractCellPopulation< ELEMENT_DIM, SPACE_DIM >::GetCellCyclePhaseCount(), AbstractCellPopulation< ELEMENT_DIM, SPACE_DIM >::GetCellMutationStateCount(), AbstractCellPopulation< ELEMENT_DIM, SPACE_DIM >::GetCellProliferativeTypeCount(), NodeBasedCellPopulation< DIM >::GetSizeOfCellPopulation(), NodesOnlyMesh< SPACE_DIM >::GetWidth(), ObjectCommunicator< CLASS >::IRecvObject(), NodesOnlyMesh< SPACE_DIM >::IsANodeCloseToDomainBoundary(), ObjectCommunicator< CLASS >::ISendObject(), ObjectCommunicator< CLASS >::RecvObject(), GenericEventHandler< 11, MeshEventHandler >::ReportImpl(), ObjectCommunicator< CLASS >::SendObject(), and ObjectCommunicator< CLASS >::SendRecvObject().

bool PetscTools::HasParMetis ( ) [static]

Checks if PETSc has been configured with ParMetis partioning support.

Returns:: true If ParMetis partitioning (via PETSc) is available

Definition at line 444 of file PetscTools.cpp.

References Destroy(), PETSC_DESTROY_PARAM, and SetupMat().

Referenced by NodePartitioner< ELEMENT_DIM, SPACE_DIM >::PetscMatrixPartitioning().

bool PetscTools::IsInitialised ( ) [static]

Returns:: whether PETSc has been initialised yet.

Definition at line 85 of file PetscTools.cpp.

References CheckCache(), and mPetscIsInitialised.

bool PetscTools::IsIsolated ( ) [static]

Returns:: whether this process is isolated, i.e. IsolateProcesses has been called.

Definition at line 103 of file PetscTools.cpp.

References mIsolateProcesses.

Referenced by GenericEventHandler< 11, MeshEventHandler >::ReportImpl().

void PetscTools::IsolateProcesses ( bool isolate = true ) [static]

Where work can be split between isolated processes, it would be nice to be able to do so easily without worrying about collective calls made inside classes such as OutputFileHandler leading to deadlock. This method attempts to enable this behaviour. If the flag is set then AmMaster and AmTopMost always return true, Barrier becomes a no-op, and ReplicateBool doesn't replicate.

Parameters:

isolate

whether to consider processes as isolated

Definition at line 168 of file PetscTools.cpp.

References mIsolateProcesses.

bool PetscTools::IsParallel ( ) [static]

Returns:: whether there is more than one process.

Definition at line 97 of file PetscTools.cpp.

References CheckCache(), and mNumProcessors.

bool PetscTools::IsSequential ( ) [static]

Returns:: whether there is one process or not.

Definition at line 91 of file PetscTools.cpp.

References CheckCache(), mIsolateProcesses, and mNumProcessors.

Referenced by AbstractCardiacTissue< ELEMENT_DIM, SPACE_DIM >::AbstractCardiacTissue(), AbstractContinuumMechanicsSolver< DIM >::AllocateMatrixMemory(), CylindricalHoneycombMeshGenerator::CylindricalHoneycombMeshGenerator(), DistanceMapCalculator< ELEMENT_DIM, SPACE_DIM >::DistanceMapCalculator(), DistributedBoxCollection< DIM >::DistributedBoxCollection(), VertexBasedCellPopulation< DIM >::GetTetrahedralMeshUsingVertexMesh(), HoneycombMeshGenerator::HoneycombMeshGenerator(), AbstractBidomainSolver< ELEMENT_DIM, SPACE_DIM >::InitialiseForSolve(), DistributedBoxCollection< DIM >::IsInteriorBox(), NodeBasedCellPopulationWithParticles< DIM >::NodeBasedCellPopulationWithParticles(), PCTwoLevelsBlockDiagonal::PCTwoLevelsBlockDiagonalCreate(), AbstractNonlinearElasticitySolver< DIM >::SetKspSolverAndPcType(), VtkMeshWriter< ELEMENT_DIM, SPACE_DIM >::SetParallelFiles(), SetupMat(), VtkMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteFilesUsingMesh(), AbstractTetrahedralMeshWriter< ELEMENT_DIM, SPACE_DIM >::WriteNclFile(), NodeBasedCellPopulationWithParticles< DIM >::WriteVtkResultsToFile(), and NodeBasedCellPopulation< DIM >::WriteVtkResultsToFile().

void PetscTools::ReadPetscObject	(	Vec &	rVec,
		const std::string &	rOutputFileFullPath,
		Vec	rParallelLayout = `NULL`
	)			`[static]`

Read a previously dumped PETSc object from disk.

Parameters:

	rVec	a vector
	rOutputFileFullPath	where to read the matrix from
	rParallelLayout	If provided, rMat will have the same parallel layout. Its content is irrelevant.

Definition at line 411 of file PetscTools.cpp.

References PETSC_DESTROY_PARAM.

void PetscTools::ReadPetscObject	(	Mat &	rMat,
		const std::string &	rOutputFileFullPath,
		Vec	rParallelLayout = `NULL`
	)			`[static]`

Read a previously dumped PETSc object from disk.

Parameters:

	rMat	a matrix
	rOutputFileFullPath	where to read the matrix from
	rParallelLayout	If provided, rMat will have the same parallel layout. Its content is irrelevant.

Todo:: : #1082 work out appropriate nz allocation.

Definition at line 360 of file PetscTools.cpp.

References Destroy(), PETSC_DESTROY_PARAM, and SetupMat().

bool PetscTools::ReplicateBool ( bool flag ) [static]

Boolean OR of flags between processes.

Parameters:

flag

is set to true on this process.

Returns:: true if any process' flag is set to true.

Definition at line 185 of file PetscTools.cpp.

References CheckCache(), mIsolateProcesses, and mPetscIsInitialised.

Referenced by AbstractCardiacTissue< ELEMENT_DIM, SPACE_DIM >::AbstractCardiacTissue(), BidomainProblem< DIM >::AnalyseMeshForBath(), DistanceMapCalculator< ELEMENT_DIM, SPACE_DIM >::ComputeDistanceMap(), BoundaryConditionsContainer< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::DefineConstantDirichletOnMeshBoundary(), AbstractBoundaryConditionsContainer< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::HasDirichletBoundaryConditions(), ReplicateException(), CardiacSimulationArchiver< PROBLEM_CLASS >::Save(), and DistanceMapCalculator< ELEMENT_DIM, SPACE_DIM >::UpdateQueueFromRemote().

void PetscTools::ReplicateException ( bool flag ) [static]

Ensure exceptions are handled cleanly in parallel code, by causing all processes to throw if any one does.

Parameters:

flag

is set to true if this process has thrown.

Definition at line 197 of file PetscTools.cpp.

References EXCEPTION, and ReplicateBool().

Referenced by AbstractCardiacTissue< ELEMENT_DIM, SPACE_DIM >::AbstractCardiacTissue(), AbstractFunctionalCalculator< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::Calculate(), DistributedTetrahedralMesh< ELEMENT_DIM, SPACE_DIM >::CalculateBoundingBox(), DistributedTetrahedralMesh< ELEMENT_DIM, SPACE_DIM >::ConstructFromMeshReader(), CellMLToSharedLibraryConverter::ConvertCellmlToSo(), ExtendedBidomainTissue< SPACE_DIM >::CreateExtracellularConductivityTensors(), BidomainTissue< SPACE_DIM >::CreateExtracellularConductivityTensors(), ExtendedBidomainTissue< SPACE_DIM >::CreateGGapConductivities(), AbstractCardiacTissue< ELEMENT_DIM, SPACE_DIM >::CreateIntracellularConductivityTensor(), ExtendedBidomainTissue< SPACE_DIM >::CreateIntracellularConductivityTensorSecondCell(), ExtendedBidomainTissue< SPACE_DIM >::ExtendedBidomainTissue(), ReplicatableVector::Resize(), ElectrodesStimulusFactory< DIM >::SetCompatibleExtracellularStimulus(), AbstractCardiacProblem< ELEMENT_DIM, SPACE_DIM, PROBLEM_DIM >::Solve(), ExtendedBidomainTissue< SPACE_DIM >::SolveCellSystems(), and AbstractCardiacTissue< ELEMENT_DIM, SPACE_DIM >::SolveCellSystems().

void PetscTools::ResetCache ( ) [static]

Reset our cached values: whether PETSc is initialised, how many processors there are, and which one we are.

Definition at line 58 of file PetscTools.cpp.

References mNumProcessors, mPetscIsInitialised, and mRank.

Referenced by CheckCache(), and PetscSetupUtils::ResetStatusCache().

void PetscTools::SetupMat	(	Mat &	rMat,
		int	numRows,
		int	numColumns,
		unsigned	rowPreallocation,
		int	numLocalRows = `PETSC_DECIDE`,
		int	numLocalColumns = `PETSC_DECIDE`,
		bool	ignoreOffProcEntries = `true`,
		bool	newAllocationError = `true`
	)			`[static]`

Set up a matrix - set the size using the given parameters. The number of local rows and columns is by default PETSC_DECIDE. SetFromOptions is called.

Parameters:

	rMat	the matrix
	numRows	the number of rows in the matrix
	numColumns	the number of columns in the matrix
	rowPreallocation	the max number of nonzero entries expected on a row A value of 0 is allowed: no preallocation is then done and the user must preallocate the memory for the matrix themselves.
	numLocalRows	the number of local rows (defaults to PETSC_DECIDE)
	numLocalColumns	the number of local columns (defaults to PETSC_DECIDE)
	ignoreOffProcEntries	tells PETSc to drop off-processor entries
	newAllocationError	tells PETSc whether to set the MAT_NEW_NONZERO_ALLOCATION_ERR. currently only used in PETSc 3.3 and later in PETSc 3.2 and earlier MAT_NEW_NONZERO_ALLOCATION_ERR defaults to false in PETSc 3.3 MAT_NEW_NONZERO_ALLOCATION_ERR defaults to true