Detailed Description

MPI support for nda::basic_array and nda::basic_array_view objects.

nda uses the TRIQS/mpi library to provide functions to broadcast, gather, reduce and scatter arrays and views over MPI processes.

The following example demonstrates some of these features:

#include <mpi/mpi.hpp>
#include <nda/mpi.hpp>
#include <nda/nda.hpp>
 
#include <iostream>
 
int main(int argc, char **argv) {
  // initialize MPI environment
  mpi::environment env(argc, argv);
  mpi::communicator comm;
 
  // create a 2x2 array on each process and fill it with its rank
  nda::array<int, 2> A(2, 2);
  A() = comm.rank();
 
  // reduce the array over all processes
  auto A_sum = mpi::reduce(A);
 
  // print the result
  if (comm.rank() == 0) std::cout << A_sum << std::endl;
}

Running with 4 cores outputs:

[[6,6]

[6,6]]

Classes
struct	mpi::lazy< mpi::tag::gather, A >
	Specialization of the `mpi::lazy` class for nda::Array types and the `mpi::tag::gather` tag. More...

struct	mpi::lazy< mpi::tag::reduce, A >
	Specialization of the `mpi::lazy` class for nda::Array types and the `mpi::tag::reduce` tag. More...

struct	mpi::lazy< mpi::tag::scatter, A >
	Specialization of the `mpi::lazy` class for nda::Array types and the `mpi::tag::scatter` tag. More...

Functions
template<typename A > requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())
auto	nda::lazy_mpi_gather (A &&a, mpi::communicator comm={}, int root=0, bool all=false)
	Implementation of a lazy MPI gather for nda::basic_array or nda::basic_array_view types.

template<typename A > requires (is_regular_or_view_v<A>)
auto	nda::lazy_mpi_reduce (A &&a, mpi::communicator comm={}, int root=0, bool all=false, MPI_Op op=MPI_SUM)
	Implementation of a lazy MPI reduce for nda::basic_array or nda::basic_array_view types.

template<typename A > requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())
auto	nda::lazy_mpi_scatter (A &&a, mpi::communicator comm={}, int root=0)
	Implementation of a lazy MPI scatter for nda::basic_array or nda::basic_array_view types.

template<typename A > requires (is_regular_or_view_v<A>)
void	nda::mpi_broadcast (A &&a, mpi::communicator comm={}, int root=0)
	Implementation of an MPI broadcast for nda::basic_array or nda::basic_array_view types.

template<typename A > requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())
auto	nda::mpi_gather (A const &a, mpi::communicator comm={}, int root=0, bool all=false)
	Implementation of an MPI gather for nda::basic_array or nda::basic_array_view types.

template<typename A1 , typename A2 > requires (is_regular_or_view_v<A1> and std::decay_t<A1>::is_stride_order_C() and is_regular_or_view_v<A2> and std::decay_t<A2>::is_stride_order_C())
void	nda::mpi_gather_capi (A1 const &a_in, A2 &&a_out, mpi::communicator comm={}, int root=0, bool all=false)
	Implementation of an MPI gather for nda::basic_array or nda::basic_array_view types using a C-style API.

template<typename A > requires (is_regular_or_view_v<A>)
auto	nda::mpi_reduce (A const &a, mpi::communicator comm={}, int root=0, bool all=false, MPI_Op op=MPI_SUM)
	Implementation of an MPI reduce for nda::basic_array or nda::basic_array_view types.

template<typename A1 , typename A2 > requires (is_regular_or_view_v<A1> && is_regular_or_view_v<A2>)
void	nda::mpi_reduce_capi (A1 const &a_in, A2 &&a_out, mpi::communicator comm={}, int root=0, bool all=false, MPI_Op op=MPI_SUM)
	Implementation of an MPI reduce for nda::basic_array or nda::basic_array_view types using a C-style API.

template<typename A > requires (is_regular_or_view_v<A>)
void	nda::mpi_reduce_in_place (A &&a, mpi::communicator comm={}, int root=0, bool all=false, MPI_Op op=MPI_SUM)
	Implementation of an in-place MPI reduce for nda::basic_array or nda::basic_array_view types.

template<typename A > requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())
auto	nda::mpi_scatter (A const &a, mpi::communicator comm={}, int root=0)
	Implementation of an MPI scatter for nda::basic_array or nda::basic_array_view types.

template<typename A1 , typename A2 > requires (is_regular_or_view_v<A1> and std::decay_t<A1>::is_stride_order_C() and is_regular_or_view_v<A2> and std::decay_t<A2>::is_stride_order_C())
void	nda::mpi_scatter_capi (A1 const &a_in, A2 &&a_out, mpi::communicator comm={}, int root=0)
	Implementation of an MPI scatter for nda::basic_array or nda::basic_array_view types using a C-style API.

Function Documentation

◆ lazy_mpi_gather()

template<typename A >
requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())

auto nda::lazy_mpi_gather	(	A &&	a,
		mpi::communicator	comm = {},
		int	root = 0,
		bool	all = false )

#include <nda/mpi/gather.hpp>

Implementation of a lazy MPI gather for nda::basic_array or nda::basic_array_view types.

This function is lazy, i.e. it returns an mpi::lazy<mpi::tag::gather, A> object without performing the actual MPI operation. Since the returned object models an nda::ArrayInitializer, it can be used to initialize/assign to nda::basic_array and nda::basic_array_view objects.

The behavior is otherwise similar to nda::mpi_gather.

Warning: MPI calls are done in the invoke and shape methods of the mpi::lazy object. If one rank calls one of these methods, all ranks in the communicator need to call the same method. Otherwise, the program will deadlock.

Template Parameters

A	nda::basic_array or nda::basic_array_view type with C-layout.

Parameters

a	Array/view to be gathered.
comm	`mpi::communicator` object.
root	Rank of the root process.
all	Should all processes receive the result of the gather.

Returns: An mpi::lazy<mpi::tag::gather, A> object modelling an nda::ArrayInitializer.

Definition at line 153 of file gather.hpp.

◆ lazy_mpi_reduce()

template<typename A >
requires (is_regular_or_view_v<A>)

auto nda::lazy_mpi_reduce	(	A &&	a,
		mpi::communicator	comm = {},
		int	root = 0,
		bool	all = false,
		MPI_Op	op = MPI_SUM )

#include <nda/mpi/reduce.hpp>

Implementation of a lazy MPI reduce for nda::basic_array or nda::basic_array_view types.

This function is lazy, i.e. it returns an mpi::lazy<mpi::tag::reduce, A> object without performing the actual MPI operation. Since the returned object models an nda::ArrayInitializer, it can be used to initialize/assign to nda::basic_array and nda::basic_array_view objects:

The behavior is otherwise similar to nda::mpi_reduce and nda::mpi_reduce_in_place.

The reduction is performed in-place if the target and input array/view are the same, e.g.

A = mpi::reduce(A);

Warning: MPI calls are done in the invoke method of the mpi::lazy object. If one rank calls this methods, all ranks in the communicator need to call the same method. Otherwise, the program will deadlock.

Template Parameters

A	nda::basic_array or nda::basic_array_view type.

Parameters

a	Array/view to be reduced.
comm	`mpi::communicator` object.
root	Rank of the root process.
all	Should all processes receive the result of the reduction.
op	MPI reduction operation.

Returns: An mpi::lazy<mpi::tag::reduce, A> object modelling an nda::ArrayInitializer.

Definition at line 173 of file reduce.hpp.

◆ lazy_mpi_scatter()

template<typename A >
requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())

auto nda::lazy_mpi_scatter	(	A &&	a,
		mpi::communicator	comm = {},
		int	root = 0 )

#include <nda/mpi/scatter.hpp>

Implementation of a lazy MPI scatter for nda::basic_array or nda::basic_array_view types.

This function is lazy, i.e. it returns an mpi::lazy<mpi::tag::scatter, A> object without performing the actual MPI operation. Since the returned object models an nda::ArrayInitializer, it can be used to initialize/assign to nda::basic_array and nda::basic_array_view objects.

The behavior is otherwise similar to nda::mpi_scatter.

Warning: MPI calls are done in the invoke and shape methods of the mpi::lazy object. If one rank calls one of these methods, all ranks in the communicator need to call the same method. Otherwise, the program will deadlock.

Template Parameters

A	nda::basic_array or nda::basic_array_view type.

Parameters

a	Array/view to be scattered.
comm	`mpi::communicator` object.
root	Rank of the root process.

Returns: An mpi::lazy<mpi::tag::scatter, A> object modelling an nda::ArrayInitializer.

Definition at line 138 of file scatter.hpp.

◆ mpi_broadcast()

template<typename A >
requires (is_regular_or_view_v<A>)

void nda::mpi_broadcast	(	A &&	a,
		mpi::communicator	comm = {},
		int	root = 0 )

#include <nda/mpi/broadcast.hpp>

Implementation of an MPI broadcast for nda::basic_array or nda::basic_array_view types.

For the root process, the array/view is broadcasted to all other processes. For non-root processes, the array/view is resized/checked to match the broadcasted dimensions and the data is written into the given array/view.

Throws an exception, if

a given view does not have the correct shape,
the array/view is not contiguous with positive strides or
one of the MPI calls fails.

See Broadcasting an array/view for an example.

Template Parameters

A	nda::basic_array or nda::basic_array_view type.

Parameters

a	Array/view to be broadcasted from/into.
comm	`mpi::communicator` object.
root	Rank of the root process.

Definition at line 56 of file broadcast.hpp.

◆ mpi_gather()

template<typename A >
requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())

auto nda::mpi_gather	(	A const &	a,
		mpi::communicator	comm = {},
		int	root = 0,
		bool	all = false )

#include <nda/mpi/gather.hpp>

Implementation of an MPI gather for nda::basic_array or nda::basic_array_view types.

The function gathers C-ordered input arrays/views from all processes in the given communicator and makes the result available on the root process (all == false) or on all processes (all == true). The arrays/views are joined along the first dimension.

It simply constructs an empty array and then calls nda::mpi_gather_capi.

See Gathering an array/view for examples.

Template Parameters

A	nda::basic_array or nda::basic_array_view type with C-layout.

Parameters

a	Array/view to be gathered.
comm	`mpi::communicator` object.
root	Rank of the root process.
all	Should all processes receive the result of the gather.

Returns: An nda::basic_array object with the result of the gathering.

Definition at line 177 of file gather.hpp.

◆ mpi_gather_capi()

template<typename A1 , typename A2 >
requires (is_regular_or_view_v<A1> and std::decay_t<A1>::is_stride_order_C() and is_regular_or_view_v<A2> and std::decay_t<A2>::is_stride_order_C())

void nda::mpi_gather_capi	(	A1 const &	a_in,
		A2 &&	a_out,
		mpi::communicator	comm = {},
		int	root = 0,
		bool	all = false )

#include <nda/mpi/gather.hpp>

Implementation of an MPI gather for nda::basic_array or nda::basic_array_view types using a C-style API.

The function gathers C-ordered input arrays/views from all processes in the given communicator and makes the result available on the root process (all == false) or on all processes (all == true). The arrays/views are joined along the first dimension.

It is expected that all input arrays/views have the same shape on all processes except for the first dimension. The function throws an exception, if

the input array/view is not contiguous with positive strides,
the output array/view is not contiguous with positive strides on receiving ranks,
the output view does not have the correct shape on receiving ranks or
any of the MPI calls fails.

The input arrays/views are simply concatenated along their first dimension. The content of the output array/view depends on the MPI rank and whether it receives the data or not:

On receiving ranks, it contains the gathered data and has a shape that is the same as the shape of the input array/view except for the first dimension, which is the sum of the extents of all input arrays/views along the first dimension.
On non-receiving ranks, the output array/view is ignored and left unchanged.

If mpi::has_env is false or if the communicator size is < 2, it simply copies the input array/view to the output array/view.

Template Parameters

A1	nda::basic_array or nda::basic_array_view type with C-layout.
A2	nda::basic_array or nda::basic_array_view type with C-layout.

Parameters

a_in	Array/view to be gathered.
a_out	Array/view to gather into.
comm	`mpi::communicator` object.
root	Rank of the root process.
all	Should all processes receive the result of the gather.

Definition at line 99 of file gather.hpp.

◆ mpi_reduce()

template<typename A >
requires (is_regular_or_view_v<A>)

auto nda::mpi_reduce	(	A const &	a,
		mpi::communicator	comm = {},
		int	root = 0,
		bool	all = false,
		MPI_Op	op = MPI_SUM )

#include <nda/mpi/reduce.hpp>

Implementation of an MPI reduce for nda::basic_array or nda::basic_array_view types.

The function reduces input arrays/views from all processes in the given communicator and makes the result available on the root process (all == false) or on all processes (all == true).

It simply constructs an empty array and then calls nda::mpi_gather_capi.

See Reducing an array/view for an example.

Template Parameters

A	nda::basic_array or nda::basic_array_view type.

Parameters

a	Array/view to be reduced.
comm	`mpi::communicator` object.
root	Rank of the root process.
all	Should all processes receive the result of the reduction.
op	MPI reduction operation.

Returns: An nda::basic_array object with the result of the reduction.

Definition at line 221 of file reduce.hpp.

◆ mpi_reduce_capi()

template<typename A1 , typename A2 >
requires (is_regular_or_view_v<A1> && is_regular_or_view_v<A2>)

void nda::mpi_reduce_capi	(	A1 const &	a_in,
		A2 &&	a_out,
		mpi::communicator	comm = {},
		int	root = 0,
		bool	all = false,
		MPI_Op	op = MPI_SUM )

#include <nda/mpi/reduce.hpp>

Implementation of an MPI reduce for nda::basic_array or nda::basic_array_view types using a C-style API.

The function reduces input arrays/views from all processes in the given communicator and makes the result available on the root process (all == false) or on all processes (all == true).

It is expected that all input arrays/views have the same shape on all processes. The function throws an exception, if

the input array/view is not contiguous with positive strides (only for MPI compatible types),
the output array/view is not contiguous with positive strides on receiving ranks,
the output view does not have the correct shape on receiving ranks,
the data storage of the output array/view overlaps with the input array/view or
any of the MPI calls fails.

The content of the output array/view depends on the MPI rank and whether it receives the data or not:

On receiving ranks, it contains the reduced data and has a shape that is the same as the shape of the input array/view.
On non-receiving ranks, the output array/view is ignored and left unchanged.

Types which cannot be reduced directly, i.e. which do not have an MPI type, are reduced element-wise.

If mpi::has_env is false or if the communicator size is < 2, it simply copies the input array/view to the output array/view.

Template Parameters

A1	nda::basic_array or nda::basic_array_view type.
A2	nda::basic_array or nda::basic_array_view type.

Parameters

a_in	Array/view to be reduced.
a_out	Array/view to reduce into.
comm	`mpi::communicator` object.
root	Rank of the root process.
all	Should all processes receive the result of the reduction.
op	MPI reduction operation.

Definition at line 114 of file reduce.hpp.

◆ mpi_reduce_in_place()

template<typename A >
requires (is_regular_or_view_v<A>)

void nda::mpi_reduce_in_place	(	A &&	a,
		mpi::communicator	comm = {},
		int	root = 0,
		bool	all = false,
		MPI_Op	op = MPI_SUM )

#include <nda/mpi/reduce.hpp>

Implementation of an in-place MPI reduce for nda::basic_array or nda::basic_array_view types.

The function in-place reduces arrays/views from all processes in the given communicator and makes the result available on the root process (all == false) or on all processes (all == true).

The behavior and requirements are similar to nda::mpi_reduce, except that the function does not return a new array. Instead it writes the result directly into the given array/view on receiving ranks.

See Reducing an array/view for an example.

Template Parameters

A	nda::basic_array or nda::basic_array_view type.

Parameters

a	Array/view to be reduced.
comm	`mpi::communicator` object.
root	Rank of the root process.
all	Should all processes receive the result of the reduction.
op	MPI reduction operation.

Definition at line 197 of file reduce.hpp.

◆ mpi_scatter()

template<typename A >
requires (is_regular_or_view_v<A> and std::decay_t<A>::is_stride_order_C())

auto nda::mpi_scatter	(	A const &	a,
		mpi::communicator	comm = {},
		int	root = 0 )

#include <nda/mpi/scatter.hpp>

Implementation of an MPI scatter for nda::basic_array or nda::basic_array_view types.

The function scatters a C-ordered input array/view from a root process across all processes in the given communicator. The array/view is chunked into equal parts along the first dimension using mpi::chunk_length.

It simply constructs an empty array and then calls nda::mpi_scatter_capi.

See Scattering an array/view for an example.

Template Parameters

A	nda::basic_array or nda::basic_array_view type.

Parameters

a	Array/view to be scattered.
comm	`mpi::communicator` object.
root	Rank of the root process.

Returns: An nda::basic_array object with the result of the scattering.

Definition at line 160 of file scatter.hpp.

◆ mpi_scatter_capi()

template<typename A1 , typename A2 >
requires (is_regular_or_view_v<A1> and std::decay_t<A1>::is_stride_order_C() and is_regular_or_view_v<A2> and std::decay_t<A2>::is_stride_order_C())

void nda::mpi_scatter_capi	(	A1 const &	a_in,
		A2 &&	a_out,
		mpi::communicator	comm = {},
		int	root = 0 )

#include <nda/mpi/scatter.hpp>

Implementation of an MPI scatter for nda::basic_array or nda::basic_array_view types using a C-style API.

The function scatters a C-ordered input array/view from a root process across all processes in the given communicator. The array/view is chunked into equal parts along the first dimension using mpi::chunk_length.

It is expected that all input arrays/views have the same rank on all processes. The function throws an exception, if

the input array/view is not contiguous with positive strides on the root process,
the output array/view is not contiguous with positive strides,
the output view does not have the correct shape or
any of the MPI calls fails.

The input array/view on the root process is chunked along the first dimension into equal (as much as possible) parts using mpi::chunk_length. If the extent of the input array along the first dimension is not divisible by the number of processes, processes with lower ranks will receive more data than processes with higher ranks.

If mpi::has_env is false or if the communicator size is < 2, it simply copies the input array/view to the output array/view.

Template Parameters

A1	nda::basic_array or nda::basic_array_view type with C-layout.
A2	nda::basic_array or nda::basic_array_view type with C-layout.

Parameters

a_in	Array/view to be scattered.
a_out	Array/view to scatter into.
comm	`mpi::communicator` object.
root	Rank of the root process.

Definition at line 94 of file scatter.hpp.

Detailed Description

Classes

Functions

Function Documentation

◆ lazy_mpi_gather()

◆ lazy_mpi_reduce()

◆ lazy_mpi_scatter()

◆ mpi_broadcast()

◆ mpi_gather()

◆ mpi_gather_capi()

◆ mpi_reduce()

◆ mpi_reduce_capi()

◆ mpi_reduce_in_place()

◆ mpi_scatter()

◆ mpi_scatter_capi()