Shared Memory

This document describes the shared memory structures, the daemons that access them, and the semaphores that control access.


Shared Memory

Several pieces of information are used by multiple components of the console software and therefore placed in shared memory.

Two pieces of information obtained from the DAP and PP. The SCSI commands which retrieve data and the data they return are:

These two pieces of information are placed into double buffered shared memory segments.

Two other singly-buffered shared memory segments are available: one for various names and values and the other for ownership information. The type definitions in "IPC_routines.h" describe the arrangement of all the shared memory segments:

typedef struct shared_names_tag {
        int             response_number;
        char            pulse_program[256];
        char            name[4][4][256];
        GLOBAL_SYMBOLS  globals;
} SHARED_NAMES;

typedef struct shared_display_tag {
        int                       	buffer_ref;
        GET_NEXT_DISPLAY_DATA_PACKET    display[2];     /* double buffered */
} SHARED_DISPLAY;

typedef struct shared_status_tag {
        int                       	buffer_ref;
        GET_NEXT_STATUS_DATA_PACKET     status[2];     /* double buffered */
} SHARED_STATUS;

typedef struct shared_ownership_tag {
        uid_t   UID;
	char	proc[PROC_TABLE_SIZE][256];
        time_t  owned, connected;
} SHARED_OWNERSHIP;

extern SHARED_NAMES                     *names;
extern SHARED_DISPLAY                   *display;
extern SHARED_STATUS                    *status;
extern SHARED_OWNERSHIP                 *ownership;
extern int                              shmid_names, shmid_display;
extern int				shmid_status, shmid_ownership;
extern int                              semid_names, semid_display;
extern int				semid_status, semid_ownership;


"get_next_~" daemons

Because the information returned by the "get_next_~()" commands needs to be continually updated, it is desirable to have them run in daemon-like processes. Any process desiring FID data or PP status information should start up the appropriate daemons. A daemon begins by checking for the existence of another daemon of the same type. If it finds one, it will exit, leaving the original daemon to handle the SCSI commands. If a daemon survives this check it should enter an infinite loop which issues the appropriate SCSI command. At the beginning of each iteration it should check to see if any other processes still want access to the shared memory segment it controls. If, and only if, all other processes are finished with shared memory, the daemon should clean up the memory and semaphores and exit. "Race conditions" which occur upon daemon startup can be handled by a semaphore which flags the existence of a daemon process.

(Real daemons are started up at boot time and always occupy a slot in the system process table. They are told to awaken or sleep depending on the need for their services. This would be another possible implementation of the "get_next_~" daemons.)


ownership daemon

Like the "get_next_~" daemons, the ownerhip daemon begins by checking for the existence of another ownership daemon and exits if one already exists. If it survives, it will enter an infinite loop. Unlike the "get_next_~" daemons, the owner ship daemon does not issue SCSI commands from inside this loop. Instead, it merely checks the "proc[][]" table against "/proc/#####" to see if processes which previously gained ownership still exist. Note that only one user (UID) may have ownership at a time but they may be running many processes which require ownership.

If the number of owners is zero then the status of the PP is determined. Depending on this check, the daemon either disconnects or disowns the spectrometer. The logic inside the infinite loop is presented schematically below:

while(1) {

	sem_wait(semid_ownership);	/* lock shared memory */

	if ( owner_count > 0 )
		sginap();
	else if ( owner_count == 0 && PP_status == RUNNING )
		disconnect();
		sginap();
	else if ( owner_count == 0 && PP_status != RUNNING )
		disown();
		remove daemon_id;
		exit();

	sem_signal(semid_ownership);	/* unlock */

}


Controlling shared memory access

Shared memory control uses synchronization primitives called semaphores. Semaphores are handled inside the kernel and have the property that multiple semaphore operations can be performed "atomically" with respect to other processes. Thus, agreed upon semaphores can be used to "lock" access to shared resources. semaphores

There are two kinds of shared resources in the console software: daemons and shared memory. The "get_next_display()" and "get_next_status()" commands have daemons which issue the commands and shared memory segments where the information is written.

Each of these four shared resources has a semaphore controlling it. The two semaphores associated with the daemons prevent multiple processes from spawning multiple daemons. A process counter attached to these semaphores flags the existence of other daemon processes.

The two semaphores associated with the shared memory segments each provide two separate functions necessary for memory to be shared properly:

  1. A semaphore lock on the shared memory buffer reference allows processes accessing the shared memory to prevent a daemon from writing to the memory segment currently being accessed.
  2. A process counter in the semaphore allows processes to announce their desire to use the information in shared memory and enables a daemon to cleanup and exit when no more processes need access to shared memory.
The semaphore operations are all defined in "IPC_routines.c" which comes directly from pages 146-151 of "Unix Network Programming" (1st ed.), by W. Richard Stevens.

Each process which intends to use shared memory must begin by:

  1. ensuring the existence of the semaphore controlling a shared memory segment
  2. adding itself to the process count of processes intending to use shared memory
  3. ensuring the existence of the shared memory
  4. starting up the daemons if they don't exist
All processes forked by the main process (ie. the daemons) will inherit the semaphores and shared memory created in the main process. When finished, each process using shared memory must remove itself from the process count so that the daemon can perform cleanup if no more processes want shared memory.

The daemons are responsible for cleaning up shared memory and the controlling semaphores if no other process needs the information in shared memory. When a daemon exits it should:

  1. lock the daemon semaphore
  2. lock the memory semaphore
  3. remove the shared memory
  4. remove the memory semaphore
  5. remove the daemon semaphore
  6. exit
Each "main" process should have the following structure:
manager/FID_display/pulse_program/Felix()
{
	/* other startup */

	create_shared_status();		/* create/get and initialize semaphores
					   and memory for the status */
	create_shared_display();	/* create/get and initialize semaphores
					   and memory for the display */
	
	fork_status_daemon();		/* fork daemon if necessary */
	fork_display_daemon();		/* fork daemon if necessary */

	/* BODY OF PROGRAM */

}

The daemon processes have the following structure:

display/status_daemon()
{
	sem_startup(~_key);		/* exit if another daemon exists,
					   else set daemon semaphore process
					   counter */
		
	Open_DAP/PP();

	while (1) {
		sem_cleanup(id);	/* cleanup if appropriate, else return */
		get_next_~();		/* write to available buffer */
		sem_wait(id);		/* lock buffer reference */
		swap_buffer_reference();
		sem_signal(id);		/* unlock buffer reference */
	}
}


Semaphore operations

All of the semaphore operations found in "IPC_routines.c" are general except for "sem_startup()" and "sem_cleanup()". These functions are specific to the console software and are used to control the creation of daemon processes and the cleanup of the shared memory when all processes are done with it.

The "sem_startup()" function accesses the semaphore flagging the existence of a daemon process. It locks the semaphore and tests the value of its process counter. If another daemon exists it simply exits. Otherwise, the process counter is incremented, the semaphore is unlocked and the function returns. (The code which follows is shematic. See "IPC_routines.h" for the actual function.)

sem_startup(key)
{
	id = sem_get(key);		/* get/create semaphore */
	semop(id, &op_lock[0], 2);	/* lock semaphore */
	if ( process_counter == 1 )
		exit(0);		/* semaphore will be unlocked by
					   "SEM_UNDO" flag used in op_lock[] */
	else {
		process_counter = 1;
		semop(id, &op_unlock[0], 2);
	}
	return
}

The "sem_cleanup()" function is responsible for removing shared memory and removing the memory and daemon semaphores. Access to the semaphore is controlled by maintaining a lock on the samaphore while all this is happening. (The code which follows is shematic. See "IPC_routines.h" for the actual function.)

sem_cleanup(id)
{
	sem_wait(id);			/* lock memory semaphore */
	sem_wait(daemon_id);		/* lock daemon semaphore */
	if ( process_count > 1 ) {	/* more than just the daemon */
		sem_signal(id);		/* unlock memory semaphore */
		sem_signal(daemon_id);	/* unlock memory semaphore */
		return;			/* no cleanup has occurred */
	}
	else {
		detach_shared_memory();
		remove_shared_memory();
		sem_rm(id);		/* remove memory semaphore without
					   ever having unlocked it */
		sem_rm(daemon_id);	/* remove daemon semaphore without
					   ever having unlocked it */
		exit(0);		   
	}
}

Jonathan Callahan