Course Portfolio: CSCI-161 Fall 2005

Brad Richards
Department of Mathematics and Computer Science
University of Puget Sound

Preface:

This is very much a work in progress. It is not yet intended for the general public, though nothing contained in this document needs to be kept from the public eye. No effort has been made yet to make this document pretty, or easy to navigate.

Course Overview:

This portfolio describes CSCI-161, an introductory programming course taught in Java. It is the entry point to the CS major at the University of Puget Sound, and the starting point for non-majors as well since we don't offer an introductory course specifically for non-majors. CS161 serves approximately 100 students a year. It satisfies a mathematics core requirement for the general population, and is required for all math majors. In the fall of 2005 the course was taught in a procedural-first manner, but switched to an objects-early approach for the spring of 2006.

I've chosen to build a portfolio for CS161 both because it's at the top of the list of courses that I want to teach well, and because efforts directed at 161 will reap the largest rewards given its enrollments and potential for creating majors. Plus, I'm still mystified by the first course after all of these years — how can such a "simple" course be so hard to teach well?

Intended Audience:

I am building this portfolio primarily for my own reference, as I strive to understand the course better and improve my pedagogy. The construction of the portfolio has caused me to think critically about various aspects of the course, and the raw artifacts, motivations, and reflections captured here will be a resource for me as I plan future iterations of 161 and other courses. The portfolio may also be of interest to colleagues at the University of Puget Sound who wish to know more about my teaching philosophy and approach, or as fodder for a conversation about how best to teach object-oriented programming. Finally, others teaching similar courses at other institutions may benefit from some of the discussion and materials presented.

Overview of this Portfolio

This document is broken into four main sections: The next section describes my objectives for the course, along with information about its content and the context in which it's taught. The Teaching and Learning section discusses my teaching philosophy and methods, and analyzes graded student work. Finally, the Conclusions section points out directions for future work and summarizes the experience.

Course Specifics

Course Objectives:

My objectives for the course are shaped in large part by the department's guidelines: I'm expected to cover a variety of specific topics and techniques so that students are prepared for the second course in the introductory sequence. But I also have some high-level objectives of my own that I'd like to make explicit and consider in the portfolio even though there's little evidence of them at work in the fall 2005 sections of 161. The list below moves from specific issues to these higher-level goals:
  1. Students completing the course should have a command of programming fundamentals (e.g. control flow, methods & parameters, etc.), and at least recognize the utility of the object-oriented approach.

  2. Students should know enough about what Computer Science is to be able to make informed decisions about taking more.

  3. Ideally, the structure and flow of the course should be optimized so that the relevance of each new topic is immediately clear. If possible, new topics would even be anticipated by students.

  4. I want to give students better tools and frameworks for the problem-solving aspect of programming.

  5. I see beauty in good code. It would be nice if I could get students to see it too.
The basic purpose of the course is to introduce programming fundamentals, so it should come as no surprise that command of these fundamentals is on the list of objectives. In my experience, few students attain the same command of basic object-oriented programming and design by the end of the first course. My hope is that all students completing the course will have had at least one "ah-hah!" moment where they recognize the value and utility of object-oriented programming. (The best hope for this moment is probably their recognition that extending an existing class is faster than reinventing the wheel.) If they can leave CS161 recognizing the value of OO, they'll be motivated to learn more about it in CS261, the follow-on course.

Any good introductory course should present the discipline in sufficient detail to allow students to determine if they're interested in taking additional courses in the discipline. This overview of the field serves students well even if they choose not to continue — at least they have a basic understanding of what professionals in the field do. Introductory computing courses that focus too tightly on programming perpetuate stereotypes about Computer Science and turn away students who might be attracted by other aspects of CS.

Too often, CS1 feels like an arbitrary exploration of programming constructs with no organizing principles. Contrast this to an intro course in Physics or Biology. You know what needs to be explained, and look forward to the point in the course where it's introduced (e.g. genetics, kinematics, etc.). Is it possible to organize the introductory programming course so that students similarly anticipate upcoming topics and see them as part of a coherent whole? (Perhaps students could work towards the completion of some large and interesting project. They'd know what they needed to learn (e.g. loops, reading from a file, displaying images, etc.), and be motivated to learn each new topic.)

Designing and implementing a program can be bewildering for novices. So many questions! Where does one start? Everybody talks about improving problem solving skills, but I've seen very little that seems useful or appropriate unless you're willing to adopt a whole new religion like "How to Design Programs / Class Hierarchies". Perhaps design patterns can help?

I appreciate programming as a fundamentally creative activity, and take pleasure in reading and writing good, clean, elegant, beautiful code. I would like my students to appreciate good code, and to get the same sense of pleasure from it. (I am convinced that this makes students better programmers, as well as happier human beings.) How best does one train someone in this art? Is repeated exposure enough? Do some never get it?

Context:

As mentioned above, CS161 is an introductory programming course taught in Java. It is the entry point to the CS curriculum for both majors and non-majors, and serves approximately 100 students a year. It satisfies a mathematics core requirement, and is required for all math majors. The University of Puget Sound is a small liberal arts school that operates on the semester system.

Students

UPS has approximately 2600 undergraduate students, virtually all of whom are of traditional age, and live in campus housing. About 25% of the students are from Washington State, with the balance coming from 46 states and 17 countries. Fewer than 10% are transfer students, and about 15% identify as members of ethnic or minority populations. Very few work outside of school, though a large majority have campus jobs as part of their financial aid package. If this semester is any indication, very few have had prior programming experience.

Constraints

Most of my constraints and experiences are colored by the fact that I'm a new arrival at UPS. (I joined the faculty in the fall of 2005, after nine years at Vassar College in Poughkeepsie, NY.) I'm still figuring out how things work at UPS, including what I have control over, how things came to be the way they are, and which topics may be contentious or controversial. Some specifics:

Significance

My course has been shaped to the greatest extent by the choice of textbook. The author and I disagree on too many philosophical points (everything from the importance of recursion to the proper way to compare floating-point values) for me to be completely comfortable with the book. I'm not satisfied with the topics it chooses to emphasize, or the order in which it presents them. (To be fair, I was offered the opportunity to select a different text, and chose not to for the reasons outlined above.) The student demographics have a substantial impact as well — I'm able to assume that my students are on campus or have easy access throughout the day, and therefore that they have access to specific computers and software, and can be expected to make it to lectures and office hours, etc. Also, the homogeneity of the student body in terms of age and experience makes it easier to deliver the course material than if I was dealing with a more diverse population.

Course Content:

The course web site for the Fall 2005 semester is online here. The index page contains links to the labs and assignments given throughout the semester, as well as to a directory containing sample code written in class. Grading details can be found at the bottom of the index page. The skeleton of the course — the language, book, and choice and ordering of topics — was carried over from previous semesters, and I had little input into its structure. However, the syllabus does show evidence of my influence with regard to the following issues:

Why labs?

The CS1 course has typically been taught without closed labs, but I chose to dedicate one of my lecture days each week to a lab session. I think it's essential that students get a chance to practice the material for themselves on a small scale — especially in a setting where I can walk them through the process without fear of "giving away" the solution. It also allows me an opportunity to work with students one-on-one, and provides me with valuable feedback on the course pacing and delivery. Unfortunately, my labs are limited to 50 minutes, which significantly limits the scope of the activities.

Why DrJava?

The official IDE of the intro course is Eclipse, but I elected to use DrJava instead. I like the fact that students can evaluate simple expressions, experiment with statements, and interact with objects in real-time, all without having to write complete programs. These facilities make it an excellent tool for working with code during lecture as well, and I like to use the same IDE that the students do. I also think DrJava's interface and feature set is more appropriate for CS1 students (e.g. no need to build a project first, then define classes). Finally, I'm an Eclipse newbie, and I was more comfortable using a tool that I knew well.

Why these assignments?

I'm responsible for the programming assignments for my two sections of CS1, for better or worse. I do my best to create assignments that make use of the topics of the week, are of appropriate scale, and will hold students' interests. I try to suggest ways in which students could extend the basic assignment, so stronger students remain challenged.

Why a reading and essay assignment?

The course is advertised as an introduction to Computer Science, but there's little evidence of that in the syllabus. I added the reading assignment in an effort to introduce additional breadth, and convince students that Computer Science is more than just syntax errors and objects. Students read two of three chapters selected from Henry Walker's "The Limits of Computing". The chapters deal with computability, feasibility, and security.

Teaching and Learning

Teaching Philosophy:

My beliefs about student learning have shaped the course in a variety of ways. My primary belief is that students learn CS1 material best by doing, and this is reflected in the number of closed labs and assignments given during the semster. (There were 12 labs and 9 programming assignments in the span of the 14-week semester.) But I also think that the timing and delivery of new material is key: Students are most receptive once they've reached a point where they recognize the value, if not the need, for a new tool or technique, and the programming activities were structured each week to emphasize a few carefully-chosen concpets and constructs. I try to provide appropriate scaffolding for the students — especially during the lab exercises. For example, I often provide students with working code as a starting point, and have them extend it during lab. This reduces initial frustration, gets students to the "interesting part" more quickly, and gives me a chance to put good code in front of them with some frequency. The lab topics and activities are designed to prepare students for the week's assignment, so the lab itself serves as scaffolding for the assignment. Finally, I believe that self confidence is essential for novices, and I work hard to foster it. I try to be encouraging in lab, where I get to spend time one-on-one with students, and I emphasize the positive in the written comments on assignments — especially on those from weaker students.

Teaching Methods:

Most of the teaching that I do in the classroom uses Direct Teaching to illustrate particular techniques. [Need a link or reference here, as well as a better explanation of the term.] I prepare carefully beforehand and come to lecture with a set of examples that illustrate the application of a concept. These examples typically build gradually upon one another until we've explored the issue to the appropriate depth. As a concrete example, I used this approach when introducing actions and listeners: I started with a one-button example, and a simple listener that printed action events. I extended the code so that the button itself was modified when pressed, then moved to a two-button example in which a common listener distinguished between buttons. This led to an example with a full calculator keypad where button presses updated a display field.

With Direct Teaching, I can plan ahead and devise a sequence of well-chosen examples that lead the students to the desired outcome. Often, the process of thinking through a sequence of examples helps refine my own ideas about how topics are related, and ensures that I come to class well prepared. Another advantage of Direct Teaching is that I can leave a record of our work for the students. The sample code that I develop with them in class can be posted online so that they can work through the sequence again on their own, or refer back to the examples when questions arise.

When appropriate, I use Lecture to introduce new concepts — topics that require an extended introduction before students would benefit from the Direct Teaching approach described above. Topics like classes and instances, object references, or inheritance, for example, require appropriate introduction before demonstrating the concept in action. This introduction typically involves well-chosen references to previous material, use of analogy, diagrams on the board, wild gesticulation, etc.

When transitioning from concepts to application, I use Lecture with Discussion. I ask questions that are designed to promote discussion, that require student participation, or that probe the depths of their knowledge. This helps me gauge when it's time to start using Direct Teaching, and gives students a chance to verify their understanding of the topic or ask about additional or related topics. If all goes well, the examples used in the Direct Teaching phase will tie in to issues that arise during the discussion.

Student Learning

The philosophy and teaching methods outlined above are well and good, but the real issue is whether the students are learning the material as a result. This section presents one specific lab and assignment in depth, as a representative sample, instead of looking at all of the student work. The assignment and lab are described, graded student submissions are shown, and an effort is made to summarize the outcomes and link them back to the course objectives.

A Representative Assignment

The selected assignment asks students to approximate the value of π pi via a Monte Carlo method, and was chosen because the problem is complex enough that the solutions exhibit interesting variations, yet the solutions are short enough to be able to inspect fairly quickly. It was the first assignment to require the use of a loop, and also involves a helper method for simulating the throw of a single dart, and user input for determining how many darts to throw. (Our textbook used JOptionPane dialog boxes and Integer.parseInt() to get numeric inputs.)

The week of the assignment, students worked through a lab that introduced loops, and reinforced the notion of conditionals and user input. In keeping with my "scaffolding" philosophy, students were given a working program as the starting point for the lab. This straight-line program lacked many of the desired attributes, but ensured that they had working code from the start. During the course of the lab, students made a series of carefully-planned modifications that produced a complete "guessing game" program. The lab also contained a series of proposed extensions to keep the stronger students involved. Parts of this lab were used verbatim in the assignment (e.g. getInteger()), while the experience with loops prepared them more generally.

Grading the Assignment

Programming is a skill that must be practiced, and the goal of my assignments is to give students an opportunity to practice specific skills in a controlled setting. Grading is both a means by which to coerce students into completing the assignment, and a mechanism through which students obtain feedback on their performance.

As I graded the assignments shown in the previous section, I recorded the number of points lost for various offenses across the set of assignments. The largest penalties were associated with defects that affected correctness — only performing the main task once instead of repeatedly, flawed arithmetic, etc. The next largest penalties were associated with failure to follow the specifics of the assignment. Programs that lost points for these issues typically ran and produced correct results, but didn't structure their solutions as required by the assignment. The guidelines set out in the assignment were not arbitrary — they were intended to make the students practice particular skills — and circumventing these guidelines allows students to "get by" without honing the desired skills. Points were lost for style issues, but these penalties were smaller in general than those above. (The defects and associated penalties are listed here.)

What does this say to students? I hope that it tells them that it's important to get the programs working, but that there's more to a good solution than that — failure to follow the assignment or to use good style will cost points. I care about those other issues.

One could argue that the modest penalties for style issues tells students that I care less about those defects, but that's not my intention. Students are understandably concerned when they're told that points will be taken off for style issues — new programmers often have no idea what good style looks like — and I've always thought it was unfair to penalize heavily until they've had a chance to learn good style. Thus, the penalties are enough to get their attention, but not enough to substantially lower their grades. I'll often write comments about a particular style issue without any penalties on a particular assignment, and start taking off points for the issue on the next assignment, after I've had a chance to discuss the results with the class.

After inspecting my rubric, I went through copies of the graded assignments, and compiled a list of the comments that appeared. The lists for correctness issues, style issues, and following assignment guidelines were of roughly equal length. What had escaped my notice until this review was the number of encouraging comments I'd made — both in recognition of a job well done for students who had mastered the assignment ("nice code"), and general encouragement for students who were struggling ("not a bad start").

Graded Student Work

All but three out of 34 students granted permission for their anonymized work to be included in this portfolio. Their graded submissions are all online, and illustrate both my grading style and their mastery (or lack thereof) of the material. For this portfolio, I went back through the submissions and recorded how well they met the requirements of the assignment.

Of the 31 submissions, 21 arrived at the correct structure of the nested loops required by the assignment. The assignment specified that the programs should simulate the throw of 1000 darts, report the approximation of π, then give the user the option of throwing additional batches of 1000. Six of the submissions asked the user before the first batch of 1000, and again after each subsequent batch, and are not included in the 21 completely-correct submissions. Still, those six managed to construct working nested loops that, while not in strict compliance with the requirements, computed the correct value. Thus, 27 out of the 31 submissions wrote functioning nested loops, and several others had at least one syntactically-correct loop in their submission, indicating that the majority of the students had successfully learned to write loops.

The assignment required that students write a Boolean-valued helper method to simulate the throw of a single dart. All 31 submissions contained an attempt at such a helper method, though only 8 of these were completely correct. Here as well, there were many "almost correct" submissions — methods that were int-valued instead of Boolean, that generated random numbers improperly, or that built in values instead of using the parameter. Others worked correctly but did not follow the requirements of the assignment (e.g. throwing a batch of darts within the method instead of just one). Still, it came as somewhat of a surprise that students were less successful in writing the helper than they were with nested loops.

Almost all (27 of 31) of the submissions had perfect I/O. This result is perhaps to be expected, since students had had more practice with I/O than with loops and methods by the time of this assignment. Finally, 28 of the 31 submissions were properly commented — they had a main comment at the top of the program, and comments before each of the methods, if not more. These weren't all good comments, but I was pleased to see so many of the students were making an effort, and giving me something upon which to give feedback.

To Do: Lots more analysis here...

Conclusions

Revisiting the Objectives

The real goal of this portfolio was to make my assumptions, philosophies, and approaches explicit — to open them to analysis and discussion, and use the process to improve my pedagogy. From this perspective, the project has been a resounding success, though the proof will only appear in the years to come. Thus, the project is less about my specific objectives than about the process, but it's still worth trying to relate my objectives to specific course experiences and outcomes. The objectives stated at the beginning of the portfolio were:
  1. Students completing the course should have a command of programming fundamentals (e.g. control flow, methods & parameters, etc.), and at least recognize the utility of the object-oriented approach.

  2. Students should know enough about what Computer Science is to be able to make informed decisions about taking more.

  3. Ideally, the structure and flow of the course should be optimized so that the relevance of each new topic is immediately clear. If possible, new topics would even be anticipated by students.

  4. I want to give students better tools and frameworks for the problem-solving aspect of programming.

  5. I see beauty in good code. It would be nice if I could get students to see it too.
I've made some progress on establishing the first of the objectives: The analysis of Assignment #4 shows that the vast majority of the students had a command of programming fundamentals, though more analysis will be required to demonstrate that they had a grasp of object-oriented programming. I believe that the course does an adequate job of fulfilling the second objective — students have a reasonable grasp of what Computer Science entails, largely thanks to the writing assignment, though nothing I captured or measured can support that assertion. There is some hope that the portfolio can say something meaningful about the third objective with some additional work. I tried to stage the labs and assignments with this sort of flow in mind, though that hasn't been made explicit yet in the section on student learning.

The last two objectives have no obvious connection to the portfolio. This is both because they're difficult to assess, and because I wasn't explicitly aware of either of them until the portfolio project began (and therefore after the course had been planned). I haven't given up on these objectives — they're just going to have to wait for a more thorough exploration in future courses and portfolios.

Future Work

The future work breaks down into two categories: Work to be done on the current portfolio, and items that need to be handled in future courses and portfolios. As mentioned above, there's much still to be done with the artifacts from the current (fall 2005) course. I need to analyze additional assignments — in particular, assignments that bridge the transition to object-oriented programming. I also need to make explicit the structure of the labs and assignments, and the flow of topics as the semester progressed. I put a lot of thought into that structure, but the structure has yet to be exposed in the portfolio.

The higher-level objectives are topics for another portfolio. I need to spend time reflecting on how I can actualize these objectives (the kinds of assignments or labs that might be introduced, or other sorts of activities), and think about the evidence that would convince me that I'm achieving my goals. In short, I haven't designed in these objectives yet, and I certainly haven't captured any data or artifacts to measure success. The first order of business, therefore, is deciding how the introductory course might be redesigned to incoroporate the objectives, and what sorts of evidence I can and should collect.

Brad Richards, 6/3/2006