This article describes the Java version of the MASS library, a parallel-computing library for Multi-Agent Spatial Simulation. As envisioned from its name, the design is based on multi-agents, each behaving as a simulation entity on a given virtual space. The library is intended to parallelize a program that particularly on multi-entity interaction in physical, biological, social, and strategic domains. The examples include major physics problems (including molecular dynamics, Schrödinger’s wave equation, and Fourier’s heat equation), neural network, artificial society, and battle games.
Places and Agents are keys to the MASS library. Places is a matrix of elements that are dynamically allocated over a cluster of computing nodes. Each element, called a Place, is pointed to by a set of network-independent matrix indices, and is capable of exchanging information with any other Place. On the other hand, Agents is a set of execution instances that can reside on a place, migrate to any other Place with matrix indices, (thus as duplicating themselves), and interact with other Agents as well as multiple Places.
An example of Places and Agents in a battle game could be territories and military units respectively. Some applications may need only either Places or Agents. For instance, Schrödinger’s wave simulation needs only two-dimensional places, each diffusing its wave influence to the neighbors. Molecular dynamics needs only agents, each behaving as a particle since it must collect distance information from all the other particles for computing its next position, velocity, and acceleration.
Parallelization with the MASS library assumes a cluster of multi-core computing nodes as the underlying computing architecture, and thus uses a set of multi-threaded communicating processes that are forked over the cluster and managed under the control of typical message-passing software infrastructure such as sockets and MPI. The library spawns the same number of threads as that of CPU cores per node or per process. Those threads take charge of method call and information exchange among places and agents in parallel.
Places are mapped to threads, whereas Agents are mapped to processes. Unless a programmer indicates his/her places-partitioning algorithm, the MASS library divides Places into smaller stripes in vertical, or in the X-coordinate direction, each of which is then allocated to and executed by a different thread. Contrary to Places, Agents are grouped into bags, each allocated to a different process where multiple threads keep checking in and out one after another Agent from this bag when they are ready to execute a new Agent. If Agents are associated with a particular Place, they are allocated to the same process whose thread takes care of this Place.
The following code shows a Java programming framework that uses the MASS library to simulate a multi-agent spatial simulation:
#!java
import edu.uw.bothell.css.dsl.MASS.Agents;
import edu.uw.bothell.css.dsl.MASS.MASS;
import edu.uw.bothell.css.dsl.MASS.Places;
public class Application {
public void static main( String[] args ) {
// get the max simulation time
int maxTime = Integer.parseInt( args[0] );
// start a process at each computing node
MASS.init();
// distribute places and agents over computing nodes
Places territories = new Places( 1, Territory.class.getName(), null, 100, 100 );
Agents troops = new Agents( 2, Troop.class.getName(), null, territories, 40000 );
// start cyclic simulation in parallel
int[] destination = new int[2]; dest[0] = 0; dest[1] = 1;
for ( int time = 0; time < maxTime - 10; time++ ) {
Object arg = ( Object )( new Integer( time ) );
territories.callAll( Territory.compute_, arg );
territories.exchangeAll( 1, Territory.exchange_, dest );
troops.callAll( Troop.compute_, arg );
troops.manageAll();
}
// terminate the processes
MASS.finish();
}
}
The behavior of the above code is as follows: it synchronizes all processes with MASS.init() and has them spawn multiple threads (line 10). The code thereafter maps a matrix of 100 × 100 “Territory” places as well as 4000 “Troop” agents over these processes (lines 13 – 16). Each process falls into a cyclic simulation (lines 20 – 26) where all its threads repeat calling the following three functions in a parallel fashion:
as well as control the “Troop” agents in manageAll() so as to move, spawn, terminate, suspend, and resume agents. At the end, all the processes get synchronized together for their termination with MASS.finish() (line 29).
In the following sections, we will define the specification of MASS, Place, Place, Agents, and Agent.
All processes involved in the same MASS library computation must call MASS.init() and MASS.finish() at the beginning and end of their code respectively, so as to get started and finished together. Upon a MASS.init() call, each process, running on a different computing node, spawns the same number of threads as that of its local CPU cores, so that all threads can access places and agents. Upon a MASS.finish() call, each process cleans up all its threads as being detached from the Places and Agents objects.
init
public static void init()
Initialize the MASS library (using settings declared in “nodes.xml”). Calling this method effectively begins computation. Use this init when the nodes and other MASS settings are defined in nodes.xml
finish
public static void finish()
Finish computation, terminate remote processes, and perform cleanup and disconnection operations. This method should be called when all computational work has been completed.
setNodeFilePath
public static void setNodeFilePath( String nodeFilePath )
Set the filename for the cluster node definition file. Use this function if the nodes are defined in an xml file not named “nodes.xml”.
Parameters: String nodeFilePath – Directory of the xml
file containing the nodes definition.
debugInit
public void debugInit( int placesHandle, int agentsHandle, int port )
Starts the debugger and waits to connect to the GUI on the provided port. This function will block the application until the GUI has connected, allowing the GUI and the debugger application to be executed in any order. After a connection is made some initial data about the Places and Agents are sent to the GUI.
Parameters: int placesHandle – User defined handle of
the Places object they wish to debug.
int agentsHandle – User defined handle of the Agents
object they wish to debug.
int port – Port with which the user wishes to connect
the debugger to.
debugUpdate
public void debugUpdate()
This method is used to update the information in the debugger. Should be called at the end of the cyclic simulation loop.
setNumThreads
public static void setNumThreads( int numThreads )
Set the number of threads to spawn on each node.
Parameters: int numThreads – The number of threads to
spawn.
Places is a distributed matrix whose elements are allocated to different computing nodes. Each element, (termed a Place) is addressed by a set of network-independent matrix indices. Once the main method has called MASS.init(), it can create as many Place objects as needed, using the Places constructor. Unless a user supplies an explicit mapping method in their Place definition, a Places instance is partitioned into smaller stripes in terms of coordinates[0], and is mapped over a given set of computing nodes, (i.e., processes).
Places manages all Place elements within the simulation space. The class instantiates an array of user-defined Place objects shared among multiple nodes. Array elements are accessed and processed by multi-processes in parallel.
Places
public Places( int handle, String className, Object argument, int... size )
Instantiates a shared array of Place objects with size from the className class as passing an argument to the className constructor. This array is associated with a user-given handle that must be unique over machines.
Parameters: int handle – A unique identifier that
designates a group of places. Must be unique over all machines.
String className – Name of the user-implemented class
Places are constructed from.
Object argument – Arguments for user-defined Place
constructor.
int... size – Size of the matrix the user wants to
create. Given in varargs form, so users can create a matrix of how many
dimensions they want. For example, a 2-dimensional matrix of size 100
would use Places( 1, className, argument, 100, 100 ), while a
3-dimensional matrix of size would use Places( 1, className, argument,
100, 100, 100 ).
** Places**
public Places( int handle, String className, int boundaryWidth, Object argument, int... size )
Instantiates a shared array of Place objects with size from the className class as passing an argument to the className constructor. This array is associated with a user-given handle that must be unique over machines.
Parameters: int handle – A unique identifier that
designates a group of places. Must be unique over all machines. String
className – Name of the user-implemented class Places are constructed
from.
int boundaryWidth – The amount of Place objects shared
between stripes on different nodes. For example, if there are two
computing nodes, sharing 100 Place objects in the x direction and
boundaryWidth is set to 5, node 0 will be responsible for the Place
objects at x-coordinates 0 – 54 and node 1 will be responsible for the
Place objects at 45 – 99. This boundary region can easily be exchanged
using Places.exchangeBoundary(). Default is 0.
Object argument – Arguments for user-defined Place
constructor.
int... size – Size of the matrix the user wants to
create. Given in varargs form, so users can create a matrix of how many
dimensions they want. For example, a 2-dimensional matrix of size 100
would use Places( 1, className, 1, argument, 100, 100 ), while a
3-dimensional matrix of size would use Places( 1, className, 1,
argument, 100, 100, 100 ).
getHandle
public int getHandle()
Returns the handle associated with this array of Place objects.
Returns: The handle associated with this array.
getSize
public int[] getSize()
Returns the size of this multi-dimensional array.
Returns: The size of the array of Place objects.
callAll
public void callAll( int functionId )
Calls the user-defined method specified with functionId of all array elements. Done in parallel among multi-processes/threads. To be used when there are no arguments in the user-defined function to be called.
Parameters: int functionId – The Id of the user-created
function to be called from the overridden callMethod() function that is
derived from Place. The callMethod() function is detailed
in section 4.3.
callAll
public void callAll( int functionId, Object argument )
Calls the user-defined method specified with functionId of all array elements as passing an Object argument to the method. Done in parallel among multi-processes/threads.
Parameters: int functionId – The Id of the user-created
function to be called from the overridden callMethod() function that is
derived from Place. The callMethod() function is detailed
in section 4.3.
Object argument – Argument passed to the user-defined
function.
callAll
public Object[] callAll( int functionId, Object[] arguments )
Calls the user-defined method specified with functionId of all array elements as passing arguments[i] to element[i]’s method, and receives a return value from it into Object[i]. Done in parallel among multi- processes/threads. In case of a multi-dimensional array, i is considered as the index when the array is flattened to a single dimension.
Parameters: int functionId – The Id of the user-created
function to be called from the overridden callMethod() function that is
derived from Place. The callMethod() function is detailed
in section 4.3.
Object[] argument – Arguments passed to the user-defined
function.
Returns: An array of objects from each Place objects callMethod() return value.
exchangeAll
public void exchangeAll( int destinationHandle, int functionId )
Calls the user-defined method specified with functionId of all destination cells, each indexed with a different Vector element. Each vector element, say destination[] is an array of integers where destination[i] includes a relative index (or a distance) on the coordinate i from the current caller to the callee cell. The caller cell’s outMessage is a continuous set of arguments passed to the callee’s method. The caller’s inMessages[] stores values returned from all callees. More specifically, inMessages[i] maintains a set of return values from the ith callee.
Parameters: int destinationHandle – The unique
identifier defined in the Places constructor for the Places object that
is to be interacted with.
int functionId – The id of the user-defined method to
call in callMethod().
exchangeBoundary
public void exchangeBoundary()
Exchanges each node’s boundary region with that node’s neighbors. If there was no boundary defined in the Places constructor, then this will do nothing.
Place is the base class from which a user can derive an application-specific matrix of Place objects. An actual matrix instance is created and maintain within a Places class, so that the user can obtain parallelizing benefits from Places callAll() and exchangeAll() methods that invoke a given method of each matrix element and exchange data between each element and others.
callMethod
public Object callMethod( int functionId, Object arguments )
Is called from Places.callAll() and
Places.exchangeAll(). Invokes the user-created function
pointed to by functionId. An application should override
callMethod() so as to direct Places to invoke an
application-specific method.
Parameters: int functionId – Id of the user-defined
function to call.
Object[] arguments – Arguments passed to the
user-defined function.
Returns: Return value of the user-defined functions.
getAgents
public synchronized Set<Agent> getAgents()
Returns a set containing the Agents that are executing on this Place.
Returns: A set containing the Agents that are executing on this Place.
getNumAgents
public int getNumAgents()
Returns the number of Agents that are executing on this Place.
Returns: The number of Agents that are executing on this Place.
getInMessages
public Object[] getInMessages()
Returns the values from this Places neighbors defined in
setNeighbors() after an exchangeAll()
call.
Returns: The values from this Places neighbors defined in
setNeighbors() after an exchangeAll()
call.
setNeighbors
public void setNeighbors( Vector<int[]> neighbors )
Defined the Places relative to this Place that this Place is to look
to when exchanging information between Places in
exchangeAll().
Parameters: Vector<int[]> neighbors – The
coordinates of Places that are defined to be neighbors if this
Place.
getDebugData
public Number getDebugData()
Debugger method that allows the user return the data for this Place so they can view it in the debugger. To be implemented by the user in their Place class.
Returns: The value the user wants to return for this Place as a Number, a Java class Double, Integer, Byte, etc., are derived from.
setDebugData
public void setDebugData( Number data )
Debugger method that allows the user to inject their own values for this Place in the debugger. To be implemented by the user in their Place class.
Parameters: Number data – The value to be added to this
Place.
Since method names are user-given, it is quite natural to invoke each
array element’s method through Java reflection, which is intolerably
slow for parallel computing. Thus, when deriving from Place, the user
should override the callMethod() function to run user made
functions. The callMethod() function is a user-provided
framework that assists the MASS library in choosing a method to call.
The callMethod() function should include a switch statement
where each function, is identified by a user-defined integer value. This
is shown in the following code sample:
#!java
public class Area extends Place {
// constants
public static final int init_ = 0;
public static final int computeWave_ = 1;
public static final int exchangeWave_ = 2;
public static final int collectWave_ = 3;
// wave height at each cell
double[] wave = new double[3];
int time = 0;
// wave height from four neighbors: north, east, south, and west
private final int north = 0, east = 1, south = 2, west = 3;
double[] neighbors = new double[4];
// simulation constants
private final double c = 1.0; // wave speed
private final double dt = 0.1; // time quantum
private final double dd = 2.0; // change in system
// the array size and my index in (x, y) coordinates
private int sizeX, sizeY;
private int myX, myY;
public Area( Object args ) {
Vector<int[]> placeNeighbors = new Vector<int[]>();
placeNeighbors.add( new int[] { 0, -1 } );
placeNeighbors.add( new int[] { 1, 0 } );
placeNeighbors.add( new int[] { 0, 1 } );
placeNeighbors.add( new int[] { -1, 0 } );
setNeighbors( placeNeighbors );
}
public Object callMethod( int funcId, Object args ) {
switch( funcId ) {
case init_: return init( args );
case computeWave_: return computeWave( args );
case exchangeWave_: return ( Object ) exchangeWave( args );
case collectWave_: return ( Object ) collectWave( args );
}
return null;
}
}Agents are a set of execution instances made up of user-defined Agent objects. Each is capable of residing on a place, migrating to any other Place objects with matrix indices, and interacting with other agents as well as multiple Places.
Once the main method has called MASS.init(), it can
create as many Agent objects as needed, using the
Agents constructor. Unless a user supplies an explicit
mapping method in their Agent definition (see Class Agent),
Agents distribute instances of a given Agent class
uniformly over different computing nodes.
public Agents( int handle, String className, Object argument, Places places, int initPopulation )
Instantiates a set of Agent objects from the className class, passes
the argument object to their constructor, associates them with a given
Place matrix, and distributes them over these places, based the
Agents.map() method that is defined within the Agent class.
If a user does not overload it by themselves, Agents.map()
uniformly distributes an initPopulation number of Agent objects. If a
user-provided Agents.map() method is used, it must return
the number of Agent objects spawned at each Place regardless of the
initPopulation parameter. Each set of Agent objects is associated with a
user-given handle that must be unique over all machines.
Parameters: int handle – A unique identifier that
designates a group of Agent objects. Must be unique over all
machines.
String className – The name of the user-defined class to
instantiate
Object argument – The argument to pass to each Agent as
it is being instantiated
Places places – The Places instance that will contain
the Agents
int initPopulation – The number of Agent Objects to
create.
getHandle
public int getHandle()
Returns the handle associated with this Agent set.
Returns: Unique handle for this Agent set.
nAgents
public int nAgents()
Returns the total number of Agent objects in this set.
Returns: Total number of Agent objects in this set.
callAll
public void callAll( int functionId )
Calls the user-defined method specified with functionId of all Agent objects in this this collection. Done in parallel among multi-processes/threads. To be used when there are no arguments in the user-defined function to be called.
Parameters: int functionId – The Id of the user-created
function to be called from the overridden callMethod()
function that is derived from Agent. This callMethod()
function is similar to the one defined in Places and detailed in section
4.3.
callAll
public void callAll( int functionId, Object argument )
Calls the method specified with functionId of all agents as passing an Object argument to the method. Done in parallel among multi-processes/threads.
Parameters: int functionId – The Id of the user-created
function to be called from the overridden callMethod()
function that is derived from Agent. The callMethod()
function is similar to the one defined in Places and detailed in section
4.3.
Object argument – Argument passed to the user-defined
function.
callAll
public Object callAll(int functionId, Object[] argument)
Calls the method specified with functionId of all agents as passing arguments[i] to agent[i]’s method, and receives a return value from it into an array [i] whose element’s size is return_value. Done in parallel among multi-processes/threads. The order of agents depends on the index of a place where they resides, starts from the place[0][0]…[0], and gets increased with the right-most index first and the left-most index last.
Parameters: int functionId – The Id of the user-created
function to be called from the overridden callMethod()
function that is derived from Agent. The callMethod() function is
similar to the one defined in Places and detailed in section 4.3.
Object argument – Argument passed to the user-defined
function.
manageAll
public void manageAll()
Updates each agent’s status, based on each of its latest migrate(), spawn(), and kill() calls. These methods are defined in the Agent base class and may be invoked from other functions through callAll. Done in parallel among multi-processes/threads.
The following example of code fragment creates a size x size matrix over multiple processes (line 10), distributes a “RandomWalk” agent every four places of this matrix (lines 11 – 12), and simulates random walk of these agents over the matrix using many threads (lines 13 – 17):
#!java
import edu.uw.bothell.css.dsl.MASS.Agents;
import edu.uw.bothell.css.dsl.MASS.MASS;
import edu.uw.bothell.css.dsl.MASS.Places;
public class RandomWalkDriver {
public void static main( String[] args ) {
MASS.init();
int size = Integer.parseInt( args[0] );
int maxTime = Integer.parseInt( args[1] );
Places matrix = new Place( 1, Matrix.class().getName(), null, size, size );
Agents walkers = new Agents( 2, RandomWalk.class().getName(), null, matrix, size * size / 4 );
for ( int time = 0; time < maxTime; time++ ) {
walkers.callAll( RandomWalk.newLocation );
walkers.manageAll( );
}
MASS.finalize();
}
}
Agent is the base class from which a user can derive an application-specific agent that migrates to another Place, forks their copies, and terminate themselves.
getPlace public Place getPlace()
Returns the current place where this agent resides.
Returns: The current user-defined Place where this Agent currently resides.
map
public int map( int initPopulation, int[] size, int[] index )
Returns the number of Agent objects to initially instantiate on a
Place indexed with index. The initPopulation parameter indicates the
number of Agent objects to create over the entire application. The size
argument defines the size of the Place matrix to which a given Agent
class belongs. The system-provided (thus default) map()
method distributes Agent objects over places uniformly as in:
initPopulation / size.length The map() method
may be overloaded by an application-specific method. A user-provided
map() method may ignore initPopulation when creating
agents.
Parameters: int initPopulation – Number of Agent objects
to create over the entire application.
int[] size – Size of the Place matrix to which a given
Agent class belongs.
int[] index – Denotes which Place to initially
instantiate the Agent objects on.
Returns: The number of Agent objects initially instantiated on the Place indexed with index.
migrate
public boolean migrate( int... index )
Initiates an agent migration to a new Place upon a next call to Agents.manageAll(). More specifically, migrate() updates the calling agent’s index[].
Parameters: int... index – Coordinates of the new Place
this Agent will migrate to.
Returns: True if migration is successful, false otherwise.
spawn
public void spawn( int numAgents, Object[] arguments )
Spawns a numAgents amount of new Agent objects, as passing arguments[i] to the i-th new agent upon a next call to Agents.manageAll().
Parameters: int numAgents – Amount of Agent Objects to
create.
Object[] arguments – An array of parameters to give to
the new Agent objects.
kill
public void kill()
Terminates the calling Agent upon a next call to
Agents.manageAll().
callMethod
public Object callMethod( int functionId, Object arguments )
Is called from Agents.callAll(). Invokes the user-created function pointed to by functionId. An application should override callMethod() so as to direct Agents to invoke an application-specific method.
Parameters: int functionId – Id of the user-defined
function to call.
Object[] arguments – Arguments passed to the
user-defined function.
Returns: Return value of the user-defined functions.
getDebugData
public Number getDebugData()
Debugger method that allows the user return the data for this Agent so they can view it in the debugger. To be implemented by the user in their Agent class.
Returns: The value the user wants to return for this Agent as a Number, a Java class Double, Integer, Byte, etc., are derived from.
setDebugData
public void setDebugData( Number data )
Debugger method that allows the user to inject their own values for this Agent in the debugger. To be implemented by the user in their Agent class.
Parameters: Number data – The value to be added to this
Agent.
Parallel I/O in the MASS Library is to increase the large file read performance. In order to implement parallel I/O, each place must be able to open, read, write, and close the same file in parallel. To make these tasks possible, all of the current implementations have been done within the Place class. Currently, parallel I/O is being designed for Netcdf and Text files.
Parallel I/O uses a hash table to store an integer file descriptor as a key and corresponding file attributes as a value. When any file is opened, an unique integer as a file descriptor, and the newly opened file is added to the file Table. When open the file, the first place on each computing node will read the file and save it onto the machine’s memory space. During the reading process, each place will get a specific portion of the file to read from. For example, if there are 5 places in total, place 1 would read the first portion of the file, place 2 would read the second portion of the file, and so on.
Open
protected int open(String filepath, int ioType)
A single Place opens a file specified by the given filePath and ioType. If ioType is 0 then the file is opened for reading, if the ioType is 1 then the file is opened for writing (it is expected that the ioType is either 0 or 1; otherwise, -1 is returned). A file that is opened for reading will be opened in memory and added to the fileTable so that the file can be accessed by all Places. A file that is opened for writing will be opened on the disk and a temporary buffer to write to is added to the fileTable so that the temp buffer can be accessed by all Places. A successfully opened file is given a unique file descriptor (integer) and the file descriptor is returned. An unsuccessful opened file will result in an exception being thrown.
Parameters: String filepath - The filepath of the file
to be opened
int ioType - either 0 for read or 1 for write
Returns: A unique file descriptor for the newly opened file
Read NetCDF
protected Object read(int fileDescriptor, String variableToRead)
Reads a specific portion of the NetCDF file’s variable based on this place’s order and returns the results as a 1-dimensional primitive java array (i.e. a float[]). Each place reads only a portion of the file, but if each place calls this method, then the entire file will be read and parts of the data will be contained on each place involved in the computation.
Parameters: int fileDescriptor - specifies the NetCDF
file to read from (must be in the fileTable)
String variableToRead - specifies the NetCDF variable to
read (must be in the NetCDF file)
Returns: A 1-dimensional primitive java array representing a portion of the NetCDF variable data read by this place - if an error occurs during the reading process, then null is returned
Read TXT
protected byte[] read(int fileDescriptor)
Reads a specific portion of the a TXT file based on this place’s order and returns the results as a byte array. Each place reads only a portion of the TXT file, but if each place calls this method, then the entire file will be read and parts of the data will be contained on each place involved in the computation.
Parameters: int fileDescriptor - unique identifier for
the file to read
Returns: The portion of the file in byte array read by this place - if an error occurs then null is returned
Close
protected synchronized boolean close(int fileDescriptor)
Closes the specified file descriptor and removes it from the file table
Parameters: int fileDescriptor - the file descriptor to
close
Returns: Return true if the file is successfully found in the file table, closed, and removed; otherwise false