You are here

Collaborative Spatial Decision Making: An Empirical Study of Geographic Information Technology Use in Water Resource Planning

Project summary

The increasing involvement of stakeholder groups in solving spatial decision problems has created a need for information technology capable of supporting collaborative spatial decision making. Such information technology has developed in recent years for the support of groups engaged in solving business problems. Similar information technology is now being developed to support group decision making aimed at solving place-based decision problems, such as site selection, choice of environmental and economic development strategies, and community planning. However, our understanding of the benefits (or costs) of such technology is limited by the shortage of empirical studies of its use. This empirical study provides insights into the dynamics of collaborative work involving true stakeholders in a realistic decision problem of water management planning in the Boise River basin in southwestern Idaho.

Our research team developed a participatory geographic information system called WaterGroup designed to support collaborative decision making among stakeholders tasked with developing recommendations for a water management plan for the Boise River basin. WaterGroup integrates maps, satellite imagery, scenario building tools, dynamic visualizations of scenario impacts, and voting tools. The goal of this study was to evaluate WaterGroup by examining how its use influences the dynamics of group interaction during collaborative decision process. In collaboration with the Idaho Department of Water Resources, we conducted a field experiment. The stakeholders who took part in the experiment included a mix of participants representative of common-pool resource decision problems, included elected officials from local (county) and state government, water district representatives, irrigation district representatives, municipal water provider representatives, and the representatives of private ground water users (irrigators and commercial water users). The experimental design involved two groups of stakeholders – a control group and an experimental group. For the control group, decision agenda activity and WaterGroup software operation were conducted by a facilitator-chauffeur. The WaterGroup display was projected on a large public screen, viewable by all stakeholders. For the experimental group, decision agenda activity was directed by a facilitator, but WaterGroup software operation was up to participants working in pairs, in addition to viewing the facilitator display on a projected screen. Each groups had between 10 and 12 participants representing five stakeholder perspectives. The human-computer-human interaction process during the experiment was captured using professional video cameras and computer activity logging. A tested social-behavioral science technique called “interaction coding” was employed to interpret and summarize the raw data. Exploratory sequential data analysis techniques were used to analyze the coded data. Parametric and non-parametric statistical procedures were used to test research hypotheses. The project results expand our knowledge about collaborative decision processes that engage diverse groups of stakeholders. Moreover, the project helped us to develop and share a better understanding of integration needs for information technology that involves GIS, decision models, and group support systems.

 

Project results

Software usability comparisons between 2001 and 2002 experiments, based on questionnaire responses:

  • The usability of geographic decision support software called GeoChoicePerspectives (GPC), used in 2001 as off-the-shelf software tool not customized for the decision task, was rated an average 13% lower than the usability of the WaterGroup software used in 2002 sessions and built for the decision task. 

  • The standard deviation of the ratings for both experimental groups (using the GCP and WaterGroup software) was on average 14% higher than the average standard deviation of both control groups (using GCP and WaterGroup software). 

  • In both experiments the respective control groups rated the software's support functionality higher than the experimental groups.

Analysis of technology-supported group decision-making process and outcomes (2002 experiment):

  • The control group tended to follow the facilitator's suggested tasks, which appeared on the agenda. The group generated fewer options for addressing the decision problem, and they developed a stronger consensus about the preferred option. However, there was little deliberation about the benefits and drawbacks of each option. 

  • The experimental group, which had access to individual computers, generated more options but did not follow the agenda as closely and was not able to come to a strong consensus in the time frame allotted for the meeting.

  • The experimental group used a much wider array of decision aids than did the control group 1in relation to problem exploration.

  • While there was limited conflict in each decision session, conflict tended to be preceded by criteria identification, valuation, and prioritization for both groups. Additionally, we found that both groups tended to experience conflict during criteria identification, prioritization, and valuation.

  • Survey responses indicate that members of the control group thought their process was more coordinated, understandable, and satisfying, than did the experimental group of their process. However, the experimental group thought their process was fairer than did the control group of their process.

  • Survey results show that the experimental group has a more favorable perception of WaterGroup software than does the control group, for both understanding the data as well as the quality of the outcome. Furthermore, both groups rated the software above 2.5 as the mean mark on the 5 point Likert scale.

  • The control group was more satisfied with their solution quality than was the experimental group with their decision solution. However, both groups indicated that the quality is tending toward very satisfied as both means are above 4.

Analysis of the research design we used to study technology-supported group decision-making process and outcomes:

  • The field experiment research design offered a number of benefits for conducting empirical research into the dynamics of technology-supported collaborative decision making. While laboratory experiments allow researchers to study group process with a great deal of control, the lack of realism results in decision outcomes that are not particularly relevant given that the subjects have no real stake in the decision. In this study we found that a field experiment can be quite useful for studying and comparing both decision process and decision outcomes, due to the increased realism.

  • We also found that field studies are far more difficult to organize than laboratory experiments due to the constraints that other organizations may put on scheduling (e.g., needing to conduct the study before the software is ready).

  • Technology supported collaborative decision making is extraordinarily complex from a social-behavioral point of view. The research strategy employed in this study was useful for systematically gathering, interpreting, and summarizing the many aspects of this process.

  • The process of social-behavioral interaction coding requires a great deal of time and money. Generating enough data for statistical validity is even more resource intensive.

  • The data gathering and summarization techniques we used can be most powerful when integrated with other qualitative research methods in a research design.

 

 

Researchers

Principal researchers

Piotr Jankowski, Ph.D.
Department of Geography
San Diego State University
piotr@geography.sdsu.edu

Timothy Nyerges, Ph.D.
Department of Geography
University of Washington
nyerges@u.washington.edu

Research collaborator

David R. Tuthill, Jr., P.E., Ph.D.
Idaho Dept. of Water Resources
Dave.Tuthill@idwr.idaho.gov

 

Research assistants

Kevin Ramsey, M.A.
Department of Geography
University of Washington
kramsey@u.washington.edu

Steve Robischon, M.A.
Department of Geography
University of Idaho
(Currently at IDWR)
Steve.Robischon@idwr.idaho.gov

David Yarnell
Department of Geography
University of Idaho

Guirong Zhou, M.A.
Department of Geography
University of Washington
grzhou@u.washington.edu

 

Publications
(click to read abstracts)

Design Considerations and Evaluation of a Collaborative, Spatio-Temporal Decision Support System. Jankowski, P., Nyerges, T., Robischon, S., Ramsey, K., and Tuthill, D. Paper submitted to Transactions in GIS.

Participatory GIS Support for Collaborative Water Resource Decision Making: Results of a Field Experiment. Nyerges, T., Jankowski, P., Tuthill, D., and Ramsey, K. Paper submitted to Annals of the Association of American Geographers.

A Data Gathering Strategy for Evaluating Technology-Supported Collaborative Decision Making Processes. Ramsey, K., Nyerges, T., and Jankowski, P. Paper submitted to Behaviour & Information Technology.

Use of PGIS in Water Resources Planning: Evolution from a State of Idaho Experiment to a Multi-State Approach in the Conjunctive Administration of Surface and Ground Water. Tuthill, D., Jankowski, P., Nyerges, T., and Robischon, S. 2003. 2nd Annual Public Participation GIS Conference, July 20-22, 2003, in Portland, OR .

Participatory GIS in Collaborative Water Resource Decision Making: Results of a field experiment. Nyerges, T.L., Jankowski, P., Ramsey, K., and Tuthill, D. 2003. 2nd Annual Public Participation GIS Conference Proceedings, July 20-22, 2003, in Portland, OR.

Design Considerations for Participatory GIS. Robischon, S., Jankowski, P., Nyerges, T., and Tuthill, D. 2003. 2nd Annual Public Participation GIS Conference Proceedings, 20-22 July 2003, in Portland, OR.

Techniques and Measures for the Evaluation of a Participatory GIS: An Interaction Coding System for Collaborative Water Resource Decision Making. Ramsey, K., Nyerges, T.L., Jankowski, P. 2003. 2nd Annual Public Participation GIS Conference, 20-22 July 2003, in Portland, OR .

Utilization of Emerging Geo-Spatial Technologies in the Implementation of Conjunctive Administration of Surface and Ground Water. Tuthill, D. 2002. dissertation, Civil Engineering, University of Idaho, Boise.

 

Acknowledgements

This research was supported in part by National Science Foundation Grant No. SBR-0111481, funded jointly by the Geography and Regional Science Program and the Decision, Risk and Management Science Program.

 

Related research projects

Many of the research team members are currently engaged in this related research project:

Participatory Geographic Information Systems for Transportation (PGIST)
The PGIST project is studying how Geographic Information Systems (GIS) and Internet technologies can improve public participation in transportation decision making. We are working with regional agencies and stakeholders to develop and evaluate on-line tools for expanding public participation in transportation improvement programming for the central Puget Sound region. We aim to improve people's ability to express their views and expand their knowledge, in order to come to a shared understanding about transportation related concerns, and then voice choices about project recommendations.