Create Places

Places is an N-dimensional grid representing an environment in the simulation. It is alwasy required in any simulation and all Agents must be associated with a Places.

Define Custom Place Class

Before instantiating an instance of Places, we need to first define the bahavior of each Place instance in the grid by defining a class inheriting from mass::Place.

The methods required to be implemented for a Place class are as follows:

• The constructor. It always takes in an mass::Index representing the flattened local index of this instance in the block, which should be passed to the base class constructor. Usually the body of the constructor should be empty since all the attributes are not created at this time. It should be decorated with __universal__.

• The callMethod. This method is called when the callAll method is invoked to the Places instance. It is reponsible for dispatching method invocation requests to the actual member method based on the given mass::FunctionId and the arguments as void*. Typical implementation of this method is using a big switch statement on the FunctionId. It should be decorated with __device__.

For better identifying the custom methods and dispatch invocation requests in callMethod, it is recomended to use an enum as their identifier (the FunctionId).

#include <mass/Place.h>

class MyPlace final : public mass::Place {
public:
    __universal__ explicit MyPlace(const mass::Index localIndex) : mass::Place(localIndex) {}

    enum Function {
        FUNC1, FUNC2
    };

    __device__ void callMethod(const mass::FunctionId functionId, void*const args) {
        switch (functionId) {
        case FUNC1: func1(); break;
        case FUNC2: func2(*static_cast<const int*>(args)); break;
        default: break;
        }
    }

    __device__ void func1() {}
    __device__ void func2(const int arg) {}
};

Instantiate Places Instance

With the custom Place class defined, we can create an instance of Places using the defined MyPlace class. This is done using the mass::makePlaces function, which takes in the integer handle of the created Places instance and a configuration instance of type mass::PlacesConfig. The integer handle can be any value as long as it is unique among all Places instances in the same simulation. As for the PlacesConfig structure, it contains the following options.

PlacesConfig::size

The n-dimensional size of the Places grid as an mass::NDSize instance. This is currently the only required option that must be provided in the configuration. Typically the value can be provided using a C++ initializer list where each integer in the list corresponds to the size of the grid in the corresponding dimension.

PlacesConfig::ghostSize

The size of the ghost regions of each block on a single GPU. This option is used for multi-GPU support with default value set to 1. The value should be eigher an integer or a mass::GhostSize instance. An integer value means that the size of the ghost region is the same in all directions while a mass::GhostSize instance allows users to specify different sizes on different directions. In most of the simulations, using a single integer for this option should be enough.

A mass::GhostSize can be constructed using an initializer list of std::pair<long, long> where each pair represents the ghost size of a dimension. The first element in the pair is the size of the ghost region in the front along that dimension while the second element is the size of the ghost region in the back. For example, a mass::GhostSize with value { { 2, 1 }, { 1, 3 } }, the resulting ghost region is presented in the figure below, where the area with hatches represents the ghost regions.

PlacesConfig::deviceMap

An N-dimensional array specifying the device assignment of each block. Each cell represents a block and the value on it is the rank of the GPU that is responsible for it. This option is optional. If not provided, MASS CUDA will try to infer a mapping rule, but it is not guaranteed to be optimal in terms of communication efficiency.

To create a deviceMap, we can use the mass::makeDeviceMap function, which takes in the N-dimensional size of the array as template parameters and the actual values as an N-dimensional raw array. For example, if a Places grid is divided by 4 GPUs into 2x2 blocks, where block (0,0) is assigned to GPU 0, block (0,1) is assigned to GPU 1, block (1,0) is assigned to GPU 2 and block (1,1) is assigned to GPU 3, then the device map should be created with the following code:

mass::makeDeviceMap<2, 2>({
    { 0, 1 },
    { 2, 3 }
})

PlacesConfig::memoryOrder

An mass::MemoryOrder enum specifying the underlying memory order for the Places grid, which can be either row major or column major. The default value is ROW_MAJOR.

Places Creation Example

Here is an example of creating a 2D Places split over 2 available GPUs with the following configurations:

• Total grid size is 100x100

• Size of the ghost regions at all directions are 1

• Split the grid into top-half and bottom-half, with the top half on device 0 and the bottom half on device 1

• Memory layout is row major

The corresponding implementation is as follows:

const auto config = mass::PlacesConfig {
    .size = { 100, 100 },
    .ghostSize = 1,
    .deviceMap = mass::makeDeviceMap<2, 1>({ 
        { 0 }, 
        { 1 } 
    }),
    .memoryOrder = mass::MemoryOrder::ROW_MAJOR
};

const auto places = mass::makePlaces<MyPlace>(0, config);