Highlights of the First Annual CHI Play Conference

Posted by Daniel Perry on November 11, 2014
ParticipatoryDesignWorkshop

The Participatory Design Workshop at CHI Play

Posted by PhD student Daniel Perry

In late October I attended the first annual conference on Computer-Human Interaction in Play (CHI Play 2014) in Toronto. The conference brought together some 150 researchers, academics, and game designers across numerous areas of serious game research and HCI. Sessions at the conference covered a variety of topic areas, including collaboration and communication in serious games, games for health, gamification and education, and game analytics. Research I conducted on co-design with high school youth for the bioinformatics educational game MAX5 was one of several accepted abstracts in a workshop titled Participatory Design for Serious Game Design.

The Participatory Design for Serious Game Design workshop marked one of the highlights of the conference for me, as it brought together participants from all over the world (Malta, Belgium, Germany, Taiwan, Canada, and the U.S to name just a few) to explore the philosophical and methodological challenges of integrating participatory design and serious games. Participatory design (PD) has its roots in Scandinavia some forty years ago as way to empower workers and involve them more directly in the software design process. PD techniques have been adapted and used in a variety of fields and contexts with communities ranging from toddlers to the elderly. While PD often works hand-in-hand with user-centered design approaches, PD’s focus on directly integrating participant design concepts takes a more democratic stance on the design process. In the workshop, we worked in small groups mapping the field of participatory design as we saw it in our own research on games. Topics that emerged included: the use of PD to design games, as well as the use of games as a methodological way to design other types of software systems; the challenge of conveying domain knowledge to participants for learning games (in my own research, this involved getting high school youth up-to-speed with biology and computer science topics in our game); and deciding what design outcomes should be integrated into the final game. While there were no easy answers that came out of the workshop, the importance of transparency in design, as well as providing a tangible sense of participant contribution came up as important issues to address. It was exciting to feel that in many ways we were putting forth a new global agenda for the future of PD in serious games research.

Another highlight of the conference included a keynote by Mike Ambinder, an experimental psychologist at Valve Software (the Seattle-area company behind a host of game favorites including Portal and Left for Dead). In his talk, Mike discussed the current state of the art in gathering game data, and the frequent biases and challenges inherent in the process. He encouraged the audience to imagine the tools and methods that would fill in the data gaps in an ideal research world. I was left with the impression that if a game powerhouse like Valve was facing a daunting data landscape, there is much to gain from further discussions between industry veterans and the academic researcher community. I’m looking forward to attending CHI Play next year.

 

 

Hacked Ethnographic Fieldnotes from Astro Hack Week

Posted by fioreb on October 29, 2014

First posted at the Astrohackweek blog

What is data science ethnography anyway?

As an ethnographer of data science, I immerse myself in particular communities to understand how they make sense of the world, how they communicate, what motivates them, and how they work together. I spent a week at astro data hack week, which might as well have been a foreign culture to me. I participated as an active listener, trying to sensitize myself to the culture and discern patterns that may not be self-evident to people within the community. Ethnography can have the effect of making the ordinary strange, such that the norms, objects, and practices that the community takes for granted become fascinating, informative sites for learning and discovery. Many of the astro hackers were probably thinking, “Why is this woman hanging around watching me code on my laptop? There is nothing interesting here.” But I assured them it was interesting to me because I was seeing their everyday practice in the context of a complex social and technical world that is in flux.

Ethnography can be thought of as a form of big data. Typically hundreds of pages of fieldnotes, interview transcripts, and artifacts from the field would be recorded over a long period of time until the ethnographer determines they have reached a point of saturation. The analysis process co-occurs with the data collection, iteratively shaping the focus of the research and observation strategy. Across this massive dataset with an abundance of unwieldy dimensions, the ethnographer has to make sense. The ethnographer works with members of the community to help them interpret what they are observing. Ethnographic insights, what many may term “findings”, emerge as patterns and themes are detected. Theory and new questions are generated, rather than tested. In this process I also acknowledge my own biases and prior assumptions and use them as ways to probe deeper and understand through them rather than ignore them. For instance, I came to astro data hack week not understanding much of anything people were talking about. It made me prone to feeling intimidated and I recognized with this intimidation my own reticence to ask questions. My own experience with this feeling helped me identify in others that were also feeling variations of this and also be able to identify what helped transform that feeling throughout the week into a more comfortable and curious state.

I only spent 5 days among the community of astro hackers, but in the spirit of hacking, I have a few “hacked” fieldnotes to share. Sharing is a key component of the hack week and as a participant I feel it is important to follow suit. But bear in mind these thoughts are preliminary. So, what have I been working on this week?

Initial descriptive observations from an outsider (a little tongue-in-cheek, forgive me):

  • Astro hackers live in a very dusty, dirty, and noisy environment! Very hard to keep clean and elaborate measures are taken to obtain a signal. But when the signal is too strong or the data too clean, there is a feeling of mistrust.
  • The common language is Python, although there are many other dialects, some entirely made of acronyms, others sound like common names, such as George and Julia.
  • When talking there is always some form of data, documentation or model that mediates the conversation, whether it is on the white board, on the screen, or through representational gestures.
  • Although most people are studying something that has to do with astronomy, they can literally be operating on “different wavelengths”!
  • Astro hackers play with “toys” and “fake data” as much as “real world data”!
  • Coffee and beer fuel interactivity!

Themes


Josh Bloom teaches Machine Learning

Data science at the community level: From T to Pi to Gamma-shaped (Josh Bloom’s term) scientists: Across the group I heard over and over again in various ways reference and deference to those who are more expert, those who are smarter or those who know more than I do. Granted, this is a somewhat common occurrence in the culture of academia as we are continuously humbled by the expertise around us. However, I found this particularly acute and concentrated within this community. What I heard across students, postdocs, and research scientists was more than the typical imposter syndrome. It was the feeling that they are expected to be experts or at the very least fluent in a range of computing and statistical areas in addition to their own domain. While this motivates people to be at a hack week such as this, it can also have the unintended effect of making people intimidated and overwhelmed with having to know everything themselves. This can have a chilling effect across the community. This means the feeling that other people know more than they do is pervasive and this often leads to thinking their questions aren’t valuable for the rest of the group, and therefore, not worth sharing. This is a negative thing and we want to ensure this effect is minimized. Not only is it bad for morale; it is bad for science. We should consider who feels comfortable taking a risk in these settings? A risk might be asking a question that they fear isn’t scientifically interesting for others. Or sharing something that isn’t complete or isn’t perfect. If we take what Josh Bloom says, that we might be better off thinking about data science on the community level, happening in a more distributed way, rather than data science on the individual level, we can begin to paint a different picture and change some of the expectations that may trigger this negative effect.

Josh Bloom’s lecture on machine learning explained the popular idea of “Pi-shaped” individuals (a buzz word for the academic data science community) and his preference, for talking about “Gamma-shaped” individuals. Rather than promote the idea that there is an expectation of individuals having expert-level depth in two domains, which is unrealistic for the majority of people, what if we thought of people as Gamma-shaped? These people would have expert-level depth in one domain and also be versed and proficient in other domains. Someone with their PhD in biology may be conversant in the language and culture of computer science enough to have conversations and collaborate, but they don’t necessarily need to be an expert in computer science to the extent that they are able to advance the discipline. These Gamma-shaped individuals can work with each other to bridge multiple domains of expertise. This Gamma symbol better reflects the makeup of individuals in this astro hack week community and this view of data science allows for the expectations to shift to the community and to the collaborative interactions between people. This shift is important and has implications for thinking about how to better structure hack week. For instance, with these tweaked expectations a learning goal of the hack week might be working together across Gamma-shaped individuals.

Categorizing hacking interactions I categorized the different kinds of hacking interactions I observed over the course of the week. This list is not meant to be exhaustive, but it might be helpful in understanding the diversity of interactions and how to facilitate the types of hacking interactions desired.

  • Resource Seeking: An individual works on their hack idea and uses other people as sources of expertise when they need help
  • Asymmetical Synergy: A pair or small group joins together to work on a hack idea in which one person is learning something, such as an algorithm, and the other has more advanced knowledge and is exploring what that algorithm can do. They are generating something together but getting different things out of the activity.
  • Symmetrical Synergy: A pair or small group joins together to work on a hack idea and iteratively discovers how their expertise informs the other, or how interests synergize. Then, they generate something new together.
  • Comparing Notes: An individual works on their hack idea and shares it with others based on their common interest. A form of comparing notes in which they are talking about the work more broadly and loosely.
  • Learning Collective: A semi-structured activity that draws multiple people in to learn something collectively, thus creating a learning collective.

The Importance of “Connective Tissue”

Across this community there is great diversity across institution, dataset, data source, methodology, computing tools and packages, statistical approach, status within academia, and level of knowledge in different arenas. This creates many opportunities for discovering connections, for sharing, and working together. Yet this also presents challenges for forging these connections especially within the broader academic environment which in many ways doesn’t incentivize collaboration and “failing fast”. Failing fast refers to the capacity to be highly experimental, to take risks, and invest a little bit often, such that when things don’t work, it is framed much more as part of the iterative process rather than as a significant loss. In a culture where people are failing fast, people are more likely to take risks and learning can happen more rapidly.

A key and essential role that emerged this week was the set of capacities for facilitating connection across people and ideas, what Fernando Perez has called the “connective tissue”. There is a need both the people and the organizational structure that supports social and technical resonances across a wide range of people and can facilitate connections among them. These people can play a role of translation across ideas that might appear otherwise unrelated. They also provide coaching (as opposed to teaching) to help both identify and achieve their goals. We should all be learning from these people so that we can all contribute to the connective tissue. This connective tissue developed further throughout the week. Specifically, the more semi-structured collective learning activities and the emphasis on working in pairs greatly increased the productivity across the group (there was more to show at the end of the day) and the interaction (fewer people with earphones in and more talking). I also observed many more small and big shared victories. I hadn’t yet seen a high five and I saw two instances on Thursday, which reflected the overall sense that the victory was about more than an individual completing the hack, rather it was shared and celebrated together.

This hack week performs as a kind of lab space where people can take risks and work together in new ways that they might not be incentivized to do otherwise. It is an opportunity to change the incentives for a short period of time. In fact, the frictions that we see emerge in this hack week (i.e. people needing to work towards publications) reflect some of the default incentives clashing with hack week incentives. For future hack weeks it might be important to advocate failing fast through normalizing it and facilitating a supportive environment for risk taking. In addition, part of the goal of a future hack week might be more explicitly to learn about how to work together and what it takes to develop connective tissue through incentivizing a range of different hacking interactions.

Work Life Balance

Posted by Katie Kuksenok on October 13, 2014

During the first meeting of the new quarter, our lab meeting consisted of each member talking about what they did this summer: be it professional achievement or a personal one. We laughed together. We ate pizza and a root vegetable medley made by one of the students, as per last year’s tradition to share food during meetings which had to be during mealtimes due to our excessively overwhelming schedules. We applauded for especially noteworthy remarks, such as: making a plan to graduate soon (2x), submitting a paper to the most recent Big Deal Conference deadline (4x), getting married (1x),and managing to have an actual honest-to-goodness vacation (3x). In our meetings for the last few years, we have allowed the unrelated to seep in, and I think it has improved both the variety and the caliber of our work. Instead of seeing these asides as distractions, we engaged with each other about a huge variety of research topics, as well as human topics.

In my own multi-year struggle with work-life balance (aka, “four years of grad school”), I have found it useful to have one core assumption. Even though I work on a million of seemingly-unrelated projects, they are necessarily and fundamentally related: because they are mine, and are built on the same body of knowledge. In this sense, every intellectually-stimulating conversation that grabs my attention is, by definition, relevant. It is relevant to my perception of the world, and I take note of it. Incidentally, when I began to pursue this sense of “wholeness,” it helped to ease the dreaded (and all-too-common) “impostor syndrome,” the haunting sense of being found out as far less competent than I appear. On the one hand, yes, with anything that I do, there are many people in the world who are much better at that thing than I am. But they are all not me, they do not have the combined idiosyncratic background I bring to the table: the whole has more creative variety to draw from than the sum of its parts. So I can feel both more secure in myself, and relieved that there is always someone to save you from excruciating (and boring) intellectual solitude with advice, feedback, and debate.

“How did you get over anxiety during giving talks?” one of the students asked Cecilia in an aside in a meeting a few years ago. “Well, when you’ve flown a plane straight at the ground at 250 mph at an airshow with hundreds of thousands of people watching, it’s difficult to be too stressed out about other things.” Professor Aragon leads our lab, teaches classes, and occasionally shares what she learned from the time she was an aerobatic champion. Instead of viewing “work life balance” as something of a separation between our “work” selves and our “life” selves, we’re building empathy within the group, as well as sharing with one another our wonderful variety of experiences and lessons.

Oberlin Winter Term 2013

Posted by Katie Kuksenok on April 02, 2013

For the month of January, three Oberlin College undergraduates – Dan Barella, Sayer Rippey, and Eli Rose – joined SCCL to work on extending our command-line tool for affect detection using machine learning, ALOE. The Winter Term internship was initially conceived by Katie Kuksenok, one of the two Oberlin alumni in SCCL; the other, Michael Brooks, also helped in mentoring the students while they were on campus.

obies2013

Each of the visiting Obies contributed a new functionality and compared its performance to that reported in our CSCW report; Dan implemented a novel segmentation algorithm, Sayer extended feature extraction to process French chat messages rather than only English, and Eli worked on HMM classification. Having returned to Oberlin, Sayer continues to work on analyzing the French portions of the dataset as an independent research project, collaborating over distance.

It has been an incredible month. Besides being blown away by the Seattle public transit system, I got to learn so much about machine learning, language, and grad school, and I got to meet a lot of smart, passionate, inspiring people.
The work I did applying the ALOE pipeline to French was completely fascinating. It was great because I got to be doing something very practical, trying to get the labeler working for the rest of the pipeline, but it also brought up some really interesting differences between French and English.

- Sayer Rippey

So, here I am at the end of Winter Term. I’m already nostalgic! This project was really enrapturing, and the whole experience thoroughly enjoyable. … I will say that, I’m proud of the work I’ve done. There are some places where I know there’s room for improvement, but to believe otherwise would perhaps be worse. I can’t claim that it’s all perfect, but I can claim that I did nearly all that I set out to do, and then some that I hadn’t expected to do. I didn’t expect I’d have to put together a profiling script to test my project, and yet this turned out to be one of the most invaluable tools I’ve had for code analysis (hopefully for others as well). I didn’t expect to find such a subtle tradeoff between a small tweaking of time variables, and yet this became a central issue of the last two weeks of my project. I didn’t think comparing pipeline statistics would be so nuanced, but now I’m beginning to see all the ways that a visualization can change the way people perceive information. I could go on, but what I’m really trying to say is: I learned so many new things!

But the most exciting parts of this Winter Term were not the components of my project. They were the incredible people at the SCCL, who brought me to lectures and talks on the nature of artificial intelligence and information visualization, who always provided novel viewpoints and provoking discussions, who were dedicated to sharing their unbelievable experience in so many topics. I was honored to work with Eli, Sayer, Katie, Michael, Megan, Cecilia, and the rest of this great team. They’ve humbled and challenged me, and for that I thank all of them; as this term comes to a close, I hope only that I should be so lucky in pursuit of future endeavors as I was in finding this one. So to everyone at the SCCL, so long, and thanks for all the fish!

- Dan Barella