Proposal Number: 0721821 Performing Organization:Univ. of WA, Tacoma NSF Program: CISE Pathways to Integrated Undergraduate Computing Education (CPATH) Principal Investigator: Tenenberg, Josh D Proposal Title: Collaborative research: CPATH CB: Disciplinary Commons in Computing Education Date: June 18, 2007 Panel Summary #1 Proposal Number: 0721821 Panel Summary: Panel Summary Project Description The project provides a mechanism for supporting serious reflection by computing faculty on the manner in which they teach courses. The proposal requires participants to join one of several Disciplinary Commons, each focusing on a specific topic in computing. Participants are expected to reflect on their practice in teaching the topic, and to capture these reflections in course portfolios. At intensive, monthly meetings of each Commons, participants would discuss and critique each others' portfolios, and would exchange ideas on how to teach the topic better. At the end, the resulting portfolios for each commons would be used to reify individual practice and produce peer-reviewed examples of teaching excellence. Intellectual Merit The project addresses an important problem: that of helping teachers improve both by reflection on their own practice and by exchanges with others teaching the same topic. The mechanism proposed, the Disciplinary Commons, is a creative, original, and innovative approach to improving teaching. As presented in the proposal, the project is well-organized, with a detailed plan for community building. The team, involving professionals not only in the United States but also in the United Kingdom, is well qualified to carry out the project, and there is sufficient access to the resources required. The panel believes the project may well contribute to improving the teaching of traditional computing concepts, and will definitely build communities of engaged scholars. The question, however, is whether this will truly transform computing as stated in the CPATH goals (see below). In addition, the project fails to articulate concrete goals, and does not provide specific outcomes. What exactly Commons participants will do is vague. Broader Impact The proposal holds promise for promoting improved teaching, and for advancing our knowledge as to how communities of scholars can collaborate in such improvement. There is potential to enhance the collaborative infrastructure of teaching professionals, especially given the dissemination plan and the mechanisms by which participants are drawn to the Commons. The UK connection is commendable in this regard. There are no specifics about how this project would address the needs of underrepresented groups. Responsiveness to CPATH Solicitation The proposal's primary deficiency is its lack of any plan to truly transform computing education. In the panel's opinion, the resulting changes, while significant, are primarily incremental in nature. This is especially apparent when one considers the titles of the proposed Commons - three of the four clearly focus on traditional computing topics (data structures, software engineering, and introductory courses). It is unclear whether the focus of the fourth Commons on engineering applications is or is not traditional. Rationale for Recommendation (including suggestions for improvement) There was much the panel liked about the proposal: pulling communities together, the framework defined for the Disciplinary Commons, and the outreach to similar efforts in the UK. The focus on traditional courses, however, does not exploit these ideas to their fullest, and fails to meet the goals set for CPATH. We encourage the PIs to consider the application of this approach to truly transformative ideas, including courses, topics, and curricular innovations with high risk and potentially high payoff. Recommendation: After serious discussion, the panel rated this as do not fund. Panel Recommendation: Do Not Fund Review #1 Rating: Good REVIEW: What is the intellectual merit of the proposed activity? This proposal aims to run four disciplinary commons in different regions of U.S. 10-15 educators in each region will be participating in face-face monthly meetings to discuss the teaching and learning practices in their respective classrooms. These educators hope to build a community within each region and connecting with shared educational experiences with educators in the other regions. The expected outcomes include the development of educational leaders to spearhead transformation of computing education in several regions of the U.S., application of scholarly approaches to teaching and learning, and development of teaching portfolios. The topics proposed for the commons are software engineering, data structures, teaching of programming, and computing education in other disciplines. The project draws upon the practices from Scholarship of teaching and learning to change the way computing is taught and provides a forum to exchange best practices What are the broader impacts of the proposed activity? The project can provide methods to influence teaching and learning of computing. It also provides several educators to develop share and refine educational materials. It also has good potential to foster a regional community of computing educators. The project also can benefit a broader segment of students in several regions. The peer-reviewed course materials can benefit other educators also. Summary Statement The proposal addresses the need for building communities.While a set of communities of educators mar result from tis project,broader involvement of community is missing. The project also appears to be more along curricular issues and pedagogy rather than community building. Review #2 Rating: Good REVIEW: What is the intellectual merit of the proposed activity? The proposal focuses on the issue of improved computing education from the bottom up, by improving the teaching skills of professors teaching computing courses. The mechanism is a so-called Disciplinary Commons, combining individual reflection and group discussion of issues across a range of computing areas. 1. Is the proposal systemic in nature? Does it address a broad range of issues? It's systemic at the pedagogical level, but not at the transformation of what computing is all about level. The issues it addresses are broad and important pedagogically. In particular, it focuses on improving the teaching of undergraduate courses via reflection on what works and what doesn't in a community setting. 2. Does the proposal have potential to transform and revitalize undergraduate computing education nationwide? Yes, in the sense of creating more effective and reflective professors. It may indirectly improve the infrastructure through encouraging reflective thought on what we are doing and why. 3. Do the project's vision, goals, objectives and outcomes have the potential to contribute to the CPATH vision? Yes. 4. Is there a Gantt-chart or similar entity that shows the timeline, tasks, milestones and specific responsibilities? Yes, very well done. 5. Is the implementation plan likely to realize the stated vision, goals, objectives and outcomes? The plan is presented well, and shows a lot of thought has been given to how the project should progress. I have high confidence it will meet the vision, goals, objectives, and outcomes. What are the broader impacts of the proposed activity? 1. Will the proposal help create a U.S. workforce with the necessary computing expertise to support the nation's health, security and prosperity? What are the prospects that it will aid U.S. competitiveness? It will probably help, in the sense that better teaching of computing courses will result in a better educated set of graduates. 2. Does the proposal address common challenges and fluctuating enrollments; demographic changes in the student body & workforce; integration of fast-paced innovations; integration of concepts & methods? This is the area where the proposal is weakest. It's impact on any of these is second-order - an indirect result of improved teaching. 3. Is there a really a community that is being supported? How established is it? How broad is it? Is there evidence of ongoing engagement with the community? Yes, there is a community - all those engaged in teaching computing at the undergraduate level. Its size is attested to by the number of attendees at education conferences, most especially SIGCSE. 4. Are the stakeholders identified? How are they being brought together (e.g., forums, workshops, sharing lessons learned)? This is where the proposal shines, with its intensive collaboration between members selected to participate in the commons. The monthly meetings, periodic course portfolio reviews, etc., will serve to improve the teaching of the participants, and should help improve the teaching of their colleagues and others in the community. 5. To what extent are CISE individuals engaged? Are others outside of CISE faculty engaged (e.g., other disciplines, industry, non-profits)? All of the PIs are from computing related disciplines, and all the commons participants will be heavily involved in using computing in their specific disciplines. Summary Statement This is an excellent proposal to update and improve computing pedagogy across a variety of specific subject areas and institutions. The particular way in which the Disciplinary Commons are created is intriguing - some based on specific computing topics, one on the same topic across high school and college, and one on the use of computing across engineering. I have one concern, however. While this project shows great promise in creating communities of reflective practitioners, might it also simply improve the way in which we teach our current materials rather than provide a significant restructuring of the computing disciplines to meet new challenges? The question remains: is improving teaching per se is as important for CPATH goals as is a redefinition of what computing is all about? Review #3 Rating: Fair REVIEW: What is the intellectual merit of the proposed activity? Intellectual merit - How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields? There is not a clear disciplinary motivation for this proposal. - How well qualified is the proposer (individual or team) to conduct the project? PI Tenenberg has extensive experience in producing results from related types of activities (Bootstrapping and Scaffolding) and probably has the skills to manage a community. There is less detailed information on the group organization and facilitation skills of the other investigators. - To what extent does the proposed activity suggest and explore creative and original concepts? It does not propose creative or original concepts - the proposed commons are groups of people. - How well conceived and organized is the proposed activity? The commons, themselves, are very will organized, but the goals and motivation of each are not well conceived. - Is there sufficient access to resources? The primary resources seem to be travel and personnel and these appear adequate. - Project Vision, Goals, Objectives, and Anticipated Outcomes: Comment on the potential of the proposed project and the likelihood that it will contribute in significant ways to realization of the CPATH vision. The proposed effort will create commons for discussion. It's not clear how the outcomes of the commons will revitalize education as it's not clear how the results can be disseminated in any meaningful way. - Implementation Plan. Comment on the likelihood that the proposed implementation plan will result in realization of the proposed vision, goals, objectives and outcomes. If the goal is to simply bring people together to talk and share, the proposal will probably be effective. It's not clear that there can be results from these commons. Describe in detail the activities to be undertaken to realize the project vision, goals ,objectives and anticipated outcomes. The proposal is very clear about its activities and how the communities will be formed. It's not so clear how the outcomes will be met by this process. Identify the types of participants likely to engage in the activity Describe the potential for the project to contribute to the revitalization of undergraduate computing education. Low - the participants may come away with an improved understanding of what others are doing, but it's not clear it will extend beyond this group. Describe the roles that each of the proposing partner organizations will play in the project. Very clearly stated. Describe approaches to engage individuals from CISE disciplines in the activities, both as leaders and as participants. This is not at all specified other than to indicate some possible groups that participants will be selected from. Using a table, a spreadsheet, or one of the many project management tools that are available, summarize the activities that will take place during the lifetime The proposal contains a clearly specified table of activities. Assess and comment upon the degree to which individuals from CISE disciplines will be engaged in the proposed activities. The participants are largely drawn from CISE disciplines. What are the broader impacts of the proposed activity? Broader Impacts - How well does the activity advance discovery and understanding while promoting teaching, training, and learning? It advances understanding of each others' approaches, but is not clear in how it promotes, specifically, teaching, training, and learning, other than through creation of common portfolios. - How well does the proposed activity broaden the participation of underrepresented groups This is vague - the only statement is that participants will be "recruited from a diverse pool of institutions" - To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? It's not clear if the commons will be sustainable beyond the scope of the proposal. - Will the results be disseminated broadly to enhance scientific and technological understanding? Conferences and the internet will be used to disseminate. It's not clear what will be disseminated that will enhance understanding. - What may be the benefits of the proposed activity to society? Not clear. Summary Statement Summary of Project: This project creates a number of disciplinary commons that are both regional and by topic. The idea is to bring groups of educators together to reflect, participate and disseminate (reification). The hope is to improve these processes through a bottom-up approach vs. the top down approach of educational conferences and publications. There are four topic areas proposed, Data Structures, Software Engineering, Introductory Programming, and Computing for Engineers. My general feeling about this project is that communities will certainly bring people together. However, there are not clear motivations nor goals for these communities. Within the topic areas: Data Structures: The goal is to determine a common set of outcomes for data structures. However, hasn't there been a tremendous amount of effort spent by working groups defining the ACM/IEEE curriculum. The proposal does not identify what deficiencies are to be addressed nor value-added aspects of the community. Software Engineering: There is not clear goal of this topic area. It is noted that there are a variety of ways that software engineering may be taught and presumably this will be the topic of discussion in the disciplinary commons. Again, what about using the extensive study within SEEK as a basis for discussion. Introductory Programming: There is no goal nor problem to be addressed in this area. Computing for Engineers: The common thread is that there are many languages used to teach engineers all with a common theme of outcomes. It's difficult to see how or what improvements might result to have positive effects on engineering (what effects, what are the perceived problems) and other technical fields with similar goals (what fields, what are the commonalities?). It's clear that there will be a lot of discussion in each of these commons, but how will there be a productive result? In 7.3 the proposal correctly identifies that such groups are largely concerned with complaint, not plaudit. How will the commons be managed so they are not unproductive "bitch sessions?" How will the results of the commons be encoded for the rest of the community in a cohesive way (reification) that is usable by a larger community? It's not clear that rigorously reviewed portfolios will have a "large effect on the culture of computing education." In summary, there has been a lot of work done in this proposal in creating communities (CB) but the motivating problems and goals of the commons are not well thought out. The proposers should be encouraged to give more thought to specific motivating problems and goals then suggesting some possible outcomes that can be assessed. The evaluation focuses on outcomes of the workshops, but fails to discuss how success of the objectives can be assessed within these outcomes - surveys are nice but how will they be able to influence the methods used? Review #4 Rating: Good REVIEW: What is the intellectual merit of the proposed activity? This proposal would set up four regional Disciplinary Commons in which a group of participants would meet on a monthly basis to share reflections on common courses through development of individual course portfolios. Pilot programs using this model have been conducted in the United States and the UK. Faculty members seldom have a chance to engage in a structured reflection on their teaching objectives, student outcomes, and mechanisms to bring outcomes more in line with objectives. In other words, applying a scientific approach to teaching itself is seldom done. This proposal would buy time, structure, and leadership for such a process. More importantly, it provides "moral support" because such activities would be carried out in a group context. No doubt this process empowers and energizes individual faculty and improves their teaching. But this occurs in the context of "tweaking" individual courses without regard to the larger issues of curriculum reform. The participating project leaders have already engaged in similar projects and are well-qualified to carry out the proposed work. The timeline and milestones are reasonable, and there are built-in opportunities for sharing between the different Commons. What are the broader impacts of the proposed activity? Course portfolios developed and examined through this process will be available for all educators through a public website. The proposers postulate that this will improve the teaching of faculty beyond the individual participants, and that participants will be catalysts for change within their own institutions. Perhaps. They also suggest that the participants, as a result of this experience, will be uniquely qualified to address larger issues in computing education. Summary Statement Improving the teaching of individual courses in such a deliberate manner is laudable, but does not significantly impact the overall approach of computing education. In that sense I do not feel this proposed activity is in line with the CPATH vision.