Application for
The Joanne Hugi Excellence Award in Research Technologies

Name of the practice: Catalyst WebQ 3.0 for Research

Institution name: University of Washington

Department name: Office of Learning Technologies

Principle contact: Oren Sreebny, oren@washington.edu, (206) 543-5415

Names and email addresses of those in the group responsible for the practice:

• encouraging innovation and enhancement in teaching, learning, and research through the use of technology;
• meeting client needs through a user-centered, creative, and adaptable approach;
• using technology as a means – not an end – to help clients of all skill levels achieve their goals, and
• collaborating with clients and partners in a flexible, friendly, and interested manner.

Catalyst Web Tools are a suite of twelve Web-based collaboration and communication tools that we provide free of charge to the University community. Recently, we redesigned WebQ, a survey and quiz tool, to expand its range of application. We originally created this tool for use in a course environment, where instructors could use it to test their students’ knowledge. Now, WebQ also functions as a powerful tool within our research community.

“The primary mission of the University of Washington is the preservation, advancement, and dissemination of knowledge.” People use the Catalyst Tools to advance these goals. WebQ, in particular, allows researchers to obtain knowledge about their field of interest, collecting data from people worldwide. Researchers have published the results of studied conducted using WebQ surveys in academic journals.

WebQ was created with input from people all over campus. As a result, the tool is flexible enough to be used by a wide audience. During the development process, we worked closely with the Human Subjects Division (HSD) staff to create a tool which would fulfill all of their requirements, by providing procedures to protect the privacy and confidentiality of human subjects. The UW HSD is so confident in WebQ that they highly recommend that UW researchers use it to collect data. Because WebQ has been approved by HSD, researchers may simply reference WebQ on their Human Subjects applications, saving them from having to describe and defend their software choice. Using WebQ greatly simplifies the Human Subjects application process for researchers at our University.

    Features

Many new features are particularly useful for researchers, allowing them to customize their surveys. A lot of researchers have found their participation rates are drastically higher when they used WebQ.

We also expanded the security options, providing researchers flexibility in limiting who may access their survey. This allows survey creators to target the group they would like to reach. Researchers may select their security from the following options:

• Researchers may create a confidential or anonymous survey which will be reviewed by HSD, by simply clicking a radio button. This simplified process allows researchers to easily choose the security option which is requested by HSD.
• Survey creators may limit who may take their survey to anyone with UW Identification or to a specific group of people with UW Identification. This option allows researchers to easily notify people about the survey and send reminders to participants who have not completed the survey.
• Researchers may specify a unique identification for each person and then optionally type in their email address. This allows researchers to survey participants outside of the UW and notify and remind them about the survey directly from WebQ. Researchers may create any id for participants; thus, they may use their own, customized system for tracking participants.
• Researchers may limit access to a specified set of email addresses, again allowing people from outside or inside the UW to participate.
• Survey creators may open the survey to anyone who access to the Web. This allows anyone to participate as often as they would like.

In response to requests for more flexibility in creating different types of questions, we have expanded the options available to users:

• For general content and question content, researchers may create and format text with a "what you see is what you get" (WYSIWYG) text editor. This allows researchers to easily customize their questions, adding in graphics and links to Web pages, specifying text color and font, and formatting their text.
• A short text response question allows researchers to limit the response to a certain number of characters and require the response to be a number or integer only. They can also specify text to display initially in the answer box.
• A long text response question allows participants' responses to be several paragraphs or pages long. This allows researchers to collect open-ended text responses. They can customize the size of the answer box, limit the number of characters in the response, and specify text to be displayed initially in the answer box.
• WebQ has four types of multiple choice questions:
• One answer (button) question allows survey participants to select only one of the available options. Researchers can choose to shuffle the answer choices for each participant and allow a write-in response to this question.
• One answer (menu) allows the researcher to provide answer choices and have the participants select the best response. The researcher can choose a default answer to be pre-selected and shuffle answer choices for each participant.
• Multiple answers (check) question allows survey participants to select more than one option in response to this question type. Researchers can specify a minimum and a maximum number of boxes they can select.
• Multiple answers (menubox) question allows participants to select answer choices by highlighting them from a list. Researchers can choose a default answer to be pre-selected, allow a write-in response, shuffle answer choices for each participant, and specify a minimum and/or maximum number of responses.

• Matrix questions allow researchers to create many questions which share a set of answer choices. Multiple questions can be included in the set to save researchers from typing the answers over and over again. There are three types of matrix questions:
• One answer per row (button) questions allow survey participants to select one answer for each question. This allows them to easily create likert scale questions.
• Multiple answers per row (check) questions allow survey participants to select multiple answers for each question. They can specify whether to display the answer choices vertically or horizontally and how often you want to repeat answer choices.
• One answer per row (menu) questions allow survey participants to select one answer for each question. Researchers can use this question as a "ranking" or "ordering" question by specifying that each answer can only be selected once by the participant.

In addition to controlling access to surveys and providing question styles, WebQ has other features that support research:

• Collaborate allows researchers to specify people affiliated with the UW to help create, edit, and review results of a survey. This heavily-requested feature allows researchers to work together more easily.
• WebQ will send automated customizable reminders to those who have not responded, saving the researcher from having to track who has responded.
• Researchers may decide how the participants will take the survey. For example, they allow them to take the survey more than once, save a survey and finish it later, and take a survey within a certain time period.
• Researchers may view and download their results in quick or customized reports. The data is all stored on a secure server, and the amount of file space allocated to researchers is nearly limitless.
• For research projects where the data is collected multiple times over a certain time period, we have added the ability to set time spans when data is collected.
• Splitting surveys into multiple pages allows researchers to further customize the look of their survey. Dividing the survey into multiple pages may improve participation as the participant does not get overwhelmed with a really long screen.

    University Connections

Since we create Catalyst Web Tools on campus, they are well integrated into the campus infrastructure. The large numbers of people using WebQ for research is a testament to the success of the project. Researchers have created over 6,000 surveys which are filed with the Human Subjects Division. Researchers and instructors have created 14,700 general surveys, and participants have taken 557,028 WebQ surveys since the new WebQ was released in late January 2005.

We have received a lot of positive feedback about the new WebQ.

• “WebQ 3.0 is great! I love the new survey features, it's everything I had wanted in the previous version and then some. Great work! You just made my life much easier.” Crystal Eney, UW Staff

• “We used a survey vendor for our work last year, and while there were many good things about it, it cost us a fortune and we still had a ton of technical issues and problems that we had to solve with our survey participants…Thanks so much for our fabulous (and FREE!) tools! That's so incredibly helpful to small grants like ours!!” Angie Windus, UW Staff

In addition to reaching people at the UW, WebQ is used by K-12 instructors in Washington State though the Washington Digital Learning Commons; by doctors, nurses, and staff at Swedish, Harborview, and UW hospitals; and by researchers at other universities such as the University of Montana.

The development process that we use can be replicated at other universities.  We have applied industry practices of user research and iterative user-centered design, creatively adapting them to the university environment and, despite a lack of industry resources, the result is an improved and useable product that meets the needs of faculty, staff, and students.

Over the past year, we transformed our software design and development process by embracing standard project management and user-centered design principles.  WebQ was the first tool to benefit from this new process.  Every project we undertake is in response to the needs of faculty, staff, and students on our campus.  Everyone on our staff is continually interacting with our users --we answer email messages sent to our help account, teach workshops, work with individuals in our drop-in center, answer questions over the phone, and give customized presentations to various groups on campus.  We also are active on numerous committees on campus and nationwide that focus on teaching and technology. These thousands of interactions provide us with immediate feedback about our tools and Web site, and keep us in touch with the needs of our clients.  We track and prioritize feature requests we receive, and develop solutions to implement that will benefit people campus-wide.

Based on feedback gathered over the years, the team meets with users to clarify the general needs of faculty, students, and staff for tools that support communication and collaboration in a multitude of contexts. Their feedback, in turn, informs the development of software requirements, as do informal and formal usability studies of initial designs.  The designers and developers regularly meet with each other, brainstorming about the best way to make the tools user-friendly.  Other helpful feedback comes from our undergraduate and graduate student staff members that participate in the development process.  Once a product is released, the team continues to collect feedback which is used to make improvements to the tool, and members of the research team go out into the campus community to learn how the tools are being used and to offer suggestions for best practices.

Following this community-centered approach, Catalyst Web Tools have become demonstrably more usable. Involving faculty, students, and staff in the development process has the added benefit of increasing awareness of our software on campus.

Although the tools are designed with course instructors in mind, the development process outlined above leads to tools that are flexible enough for other members of the campus community to use.   For example:

• EPost, the online discussion board, was created for UW course discussions, but the most active message board is owned by a staff member who runs a reception desk in one of the dormitories. Campus members use EPost for everything ranging from helping them find nannies to communicating with honors students overseas.
• The online Portfolio tool was developed for courses which already used portfolios, but it’s often used for career and academic planning and highlighting professional skills. Approximately 75% (over 3000) of incoming freshman use it to help them identify and reflect upon their experiences. People even use it for online presentations and quickly posting information on the Web.
• Group Manager was designed to help instructors easily use their class lists, but the student elections had the largest number of people in a group (36,567). The Faculty senate also uses it combined with WebQ to administer elections.
• SimpleSite was originally designed to allow instructors to create course Web sites, however, the students wanted to use SimpleSite so much that they funded the development of student-based templates.

The Catalyst software engineers have re-created the tool framework which will be released to the open source world. This will let people to create their own substantiations of the Catalyst Tools. This will allow, for example, researchers to customize WebQ for their specific research project. It will also give the open source community at other universities the ability to use, improve, and customize the Catalyst Tools.

Catalyst home page
Catalyst Web Tools
Examples of uses of the tools
New features of WebQ
Use WebQ for survey research
Learning Commons