PROJECT OBJECTIVES

Key objectives of the project are:

• to improve the state of Computer Science education research-and thereby ultimately to improve the state of CS education-by developing skills, in the design, conduct and management of research, of Computer Science educators and by exposing them to relevant theory and methods, and
• to facilitate the establishment of research relationships that extend beyond the duration of the workshops, contributing to a research community able to sustain a constructive discourse as well as ongoing collaboration.

PROJECT CHARACTER

The project has three major constituents:

• intensive workshops in two successive years, and
• activities (and associated mechanisms) to sustain participants' work during the intervening period, and
• explicit scaffolding to facilitate the transition from individual researcher to participant member of the wider research community via joint meetings with the members from the Bootstrapping project.

The Scaffolding project aims to foster a community of discourse, in which ideas can be generated, tested, examined and extended. The project has been designed to address directly and effectively the needs and problems faced by the participants.

The biggest cost in any community endeavour is the cost of establishing relationships. Our workshops put participants into a "hothouse"-a place where ideas are nurtured and sustained through intensive interaction. Then, the experiment kit, managed mailing list, and meetings at SIGCSE '04 and '05 will carry this energy and discourse into everyday working practice.

The whole program of activity over time is designed to engender skills and confidence that will allow participants to initiate subsequent research and engage in the wider research community.

PROJECT DESCRIPTION

Each participant will be asked to make a two-year commitment encompassing attendance at two week-long workshops, and conducting a data collection exercise in the interim. In addition, their interim activity will be scaffolded with on-line discussions and additional meetings at SIGCSE Symposia in 2004 and 2005. The first workshop will be held June 13 - 18, 2003 in the Pacific Northwest of the United States. The second workshop will be held June 5 - 10, 2003, also in the Pacific Northwest.

Structure of the project

Initial Workshop (1)

Workshop one is topic-oriented. The intention is to provide an overview of CS education research and to enable participants to think about how to do research in CS education. In each day there will be time for participants to consider their own research questions as a way of reflecting on the material so far. Each day will have a combination of delivered material, guided discussions, and practical exercises. Participants will leave the workshop fully versed in the "experiment kit".

Day 1: Putting research methods in context.

• Indicative Topics
  • Describing the space of methods and building a vocabulary about research methods.
  • What gives research value?
  • Providing a pragmatic perspective (evidence, context, epistemology)
  • Classic concerns in research: validity, bias, statistics, etc.
  • Introduction to theories of learning (building on preliminary reading.)

Day 2: How to ask questions and how to seek answers
• Indicative Topics
  • Introducing a pragmatic approach to study design: the research question demands evidence which constrains techniques
  • Planning research studies
  • Choosing subjects; representativeness
  • Research ethics and informed consent
  • Testing study designs, how and why (pilot studies, bias, populations, etc.)
  • Analysis (what will the data look like and what will you do with it)
  • Bias (a more detailed examination of sources of bias)

Day 3: Working from purpose to technique
• Indicative Topics
  • Review of a variety of techniques and what they’re good for: benefits, limitations, costs
  • Exercises in design reasoning (e.g., If you want to confirm a suspicion, how would you set about it? If you want to elicit and capture a process, what might you do? If you want to see if two factors are related, what might you do? etc)
  • Record-keeping and documentation
  • Reporting

Day 4: Experiment Kit
• Indicative Topic
  • Introduction to the experiment kit

The experiment kit

An "experiment kit" is designed by a principle investigator (within the Scaffolding project, the workshop leaders). It
• defines a research problem,
• details the design of a relevant study from methodology to analysis, and
• situates it in its theoretical context (readings, case studies and further references).

The experiment kit will be introduced in workshop one. Data gathering will be carried out during the interim year in the participants’ classrooms. Participants will be encouraged to maintain contact over the intervening period for problem-solving and support, forming an electronic research network. Preliminary analysis (on the individual data sets) will be undertaken in advance of the second workshop, where the data will be presented, amalgamated, and analysed in aggregate. Participants will contribute to the writing up, and the resultant paper will be submitted to an appropriate journal.

The kit to be introduced in this program will address understanding of design and investigate programming design processes in first- and second-year undergraduate students. Although the topic is heavily researched, results are still equivocal, and so a real contribution can be made.

Components of the experiment kit will include:

• a clear statement of the research question
• a selection of published work on programming understanding that sets the context for the question
• literature concerning the research methodology, its justification and epistemology, and examples of previous use of the techniques
• a description of the techniques for data collection and analysis for this example
• a preliminary framing for a research publication to result from this study

Capstone Workshop (2)

Workshop two is task-oriented. In contrast to Workshop One, most of the input in this workshop will be from the participants, who will present and discuss their experiences using the experiment kit and reflect on research in general.

Each participant will arrive at the workshop having completed their data collection and preliminary analysis, and having read study sketches written by other participants and prepared to discuss a selection that has been assigned to them.

Day 1: Presentation and amalgamation of data

• Indicative Tasks
  • Reflection on the experience of the previous year
  • Review and extraction of lessons learned

Day 2: Analysis of data in aggregate

• Indicative Tasks
  • Aggregation of data
  • Analysis of aggregated data
  • Conclusions drawn and discussion of their implications, assessment of contribution of this work to the field ("pre-writing" exercises)

Day 3: What next?

• Indicative Tasks
  • draft remainder of paper ("draft zero")
  • plenary critique of one or two participants’ new designs for their own research studies (with particular attention to limitations, cost-benefit analysis, and possible bias, design trade-offs.)
  • presentation of all participants’ new designs for their own research studies. This will be conducted in small groups for which participants will have been prepared by their interim critical reading. The basis for these small group discussions will be peer- and self-critique, providing an opportunity to consolidate previous learning, to experience collaboration and discourse, and to see other peoples’ ideas at an early stage.

Day 4: Reflection

• Indicative Tasks
  • draft remainder of paper ("first draft")
  • discuss strategies for sustaining research beyond the workshop, including identifying required resources and support that participants will need from one another and their home institutions.
  • structured reflection on the program