# Create Places

`Places` is an N-dimensional grid representing an environment in the simulation. It is alwasy required in any simulation and all `Agents` must be associated with a `Places`. 

## Define Custom `Place` Class

Before instantiating an instance of `Places`, we need to first define the bahavior of each `Place` instance in the grid by defining a class inheriting from `mass::Place`. 

The methods required to be implemented for a `Place` class are as follows: 

* The constructor. It always takes in an `mass::Index` representing the flattened local index of this instance in the block, which should be passed to the base class constructor. Usually the body of the constructor should be empty since all the attributes are not created at this time. It should be decorated with `__universal__`.

* The `callMethod`. This method is called when the `callAll` method is invoked to the `Places` instance. It is reponsible for dispatching method invocation requests to the actual member method based on the given `mass::FunctionId` and the arguments as `void*`. Typical implementation of this method is using a big `switch` statement on the `FunctionId`. It should be decorated with `__device__`.

For better identifying the custom methods and dispatch invocation requests in `callMethod`, it is recomended to use an enum as their identifier (the `FunctionId`).

```cpp
#include <mass/Place.h>

class MyPlace final : public mass::Place {
public:
    __universal__ explicit MyPlace(const mass::Index localIndex) : mass::Place(localIndex) {}

    enum Function {
        FUNC1, FUNC2
    };

    __device__ void callMethod(const mass::FunctionId functionId, void*const args) {
        switch (functionId) {
        case FUNC1: func1(); break;
        case FUNC2: func2(*static_cast<const int*>(args)); break;
        default: break;
        }
    }

    __device__ void func1() {}
    __device__ void func2(const int arg) {}
};
```


## Instantiate `Places` Instance

With the custom `Place` class defined, we can create an instance of `Places` using the defined `MyPlace` class. This is done using the `mass::makePlaces` function, which takes in the integer handle of the created `Places` instance and a configuration instance of type `mass::PlacesConfig`.  The integer handle can be any value as long as it is unique among all `Places` instances in the same simulation. As for the `PlacesConfig` structure, it contains the following options.

### `PlacesConfig::size`

The n-dimensional size of the `Places` grid as an `mass::NDSize` instance. This is currently the only required option that must be provided in the configuration. Typically the value can be provided using a C++ initializer list where each integer in the list corresponds to the size of the grid in the corresponding dimension.

### `PlacesConfig::ghostSize`

The size of the ghost regions of each block on a single GPU. This option is used for multi-GPU support with default value set to 1. The value should be eigher an integer or a `mass::GhostSize` instance. An integer value means that the size of the ghost region is the same in all directions while a `mass::GhostSize` instance allows users to specify different sizes on different directions. In most of the simulations, using a single integer for this option should be enough.

A `mass::GhostSize` can be constructed using an initializer list of `std::pair<long, long>` where each pair represents the ghost size of a dimension. The first element in the pair is the size of the ghost region in the front along that dimension while the second element is the size of the ghost region in the back. For example, a `mass::GhostSize` with value `{ { 2, 1 }, { 1, 3 } }`, the resulting ghost region is presented in the figure below, where the area with hatches represents the ghost regions.

![GhostSize](../assets/GhostSize.png)

### `PlacesConfig::deviceMap`

An N-dimensional array specifying the device assignment of each block. Each cell represents a block and the value on it is the rank of the GPU that is responsible for it. This option is optional. If not provided, MASS CUDA will try to infer a mapping rule, but it is not guaranteed to be optimal in terms of communication efficiency. 

To create a `deviceMap`, we can use the `mass::makeDeviceMap` function, which takes in the N-dimensional size of the array as template parameters and the actual values as an N-dimensional raw array. For example, if a `Places` grid is divided by 4 GPUs into 2x2 blocks, where block (0,0) is assigned to GPU 0, block (0,1) is assigned to GPU 1, block (1,0) is assigned to GPU 2 and block (1,1) is assigned to GPU 3, then the device map should be created with the following code:

```cpp
mass::makeDeviceMap<2, 2>({
    { 0, 1 },
    { 2, 3 }
})
```

### `PlacesConfig::memoryOrder`

An `mass::MemoryOrder` enum specifying the underlying memory order for the `Places` grid, which can be either row major or column major. The default value is `ROW_MAJOR`.

### Places Creation Example

Here is an example of creating a 2D `Places` split over 2 available GPUs with the following configurations:

* Total grid size is 100x100
* Size of the ghost regions at all directions are 1
* Split the grid into top-half and bottom-half, with the top half on device 0 and the bottom half on device 1
* Memory layout is row major

The corresponding implementation is as follows: 

```cpp
const auto config = mass::PlacesConfig {
    .size = { 100, 100 },
    .ghostSize = 1,
    .deviceMap = mass::makeDeviceMap<2, 1>({ 
        { 0 }, 
        { 1 } 
    }),
    .memoryOrder = mass::MemoryOrder::ROW_MAJOR
};

const auto places = mass::makePlaces<MyPlace>(0, config);
```