Crowdlicit

Abdullah X. Ali, University of Washington
Meredith Ringel Morris, Google Research
Jacob O. Wobbrock, University of Washington

Work done while at Microsoft Research

About

Crowdlicit is a web-based tool that allows system creators to conduct distributed end-user elicitation and identification studies. Crowdlicit generates custom web interfaces that present end-users with prompts in a variety of formats (e.g., text, still images, audio clips, video clips), and facilitates the collection of user proposals such as voice commands, button labels, icons, interface sketches, or stroke or hand gestures.


Crowdlicit has been folded into the Crowd Design Engine.
 

Citation

Ali, A.X., Morris, M.R. and Wobbrock, J.O. (2019). Crowdlicit: A system for conducting distributed end-user elicitation and identification studies. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '19). Glasgow, Scotland (May 4-9, 2019). New York: ACM Press. Paper No. 255.

Related Project

Crowdsensus is a crowd-powered tool for clustering symbols and calculating agreement in elicitation studies.

Acknowledgements

This work was supported in part by funding from Microsoft Research, the Mani Charitable Foundation, and the National Science Foundation under grant IIS-1702751. Any opinions, findings, conclusions or recommendations expressed in our work are those of the authors and do not necessarily reflect those of any supporter.

Our End-User Elicitation Publications

  1. Vatavu, R.-D. and Wobbrock, J.O. (2022). Clarifying agreement calculations and analysis for end-user elicitation studies. ACM Transactions on Computer-Human Interaction 29 (1). Article No. 5.
  2. Ali, A.X., Morris, M.R. and Wobbrock, J.O. (2019). Crowdlicit: A system for conducting distributed end-user elicitation and identification studies. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '19). Glasgow, Scotland (May 4-9, 2019). New York: ACM Press. Paper No. 255.
  3. Ali, A.X., Morris, M.R. and Wobbrock, J.O. (2018). Crowdsourcing similarity judgments for agreement analysis in end-user elicitation studies. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '18). Berlin, Germany (October 14-17, 2018). New York: ACM Press, pp. 177-188.
  4. Vatavu, R.-D. and Wobbrock, J.O. (2016). Between-subjects elicitation studies: Formalization and tool support. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '16). San Jose, California (May 7-12, 2016). New York: ACM Press, pp. 3390-3402.
  5. Vatavu, R.-D. and Wobbrock, J.O. (2015). Formalizing agreement analysis for elicitation studies: New measures, significance test, and toolkit. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '15). Seoul, Korea (April 18-23, 2015). New York: ACM Press, pp. 1325-1334. Honorable Mention Paper.
  6. Morris, M.R., Danielescu, A., Drucker, S., Fisher, D., Lee, B., schraefel, m.c. and Wobbrock, J.O. (2014). Reducing legacy bias in gesture elicitation studies. ACM Interactions 21 (3), pp. 40-45.
  7. Morris, M.R. (2012). Web on the wall: insights from a multimodal interaction elicitation study. Proceedings of the ACM Conference on Interactive Tabletops and Surfaces (ITS '12). Cambridge, Massachusetts (November 11-14, 2012). New York: ACM Press, pp. 95-104.
  8. Morris, M.R., Wobbrock, J.O. and Wilson, A.D. (2010). Understanding users' preferences for surface gestures. Proceedings of Graphics Interface (GI '10). Ottawa, Ontario, Canada (May 31-June 2, 2010). Toronto, Ontario: Canadian Information Processing Society, pp. 261-268.
  9. Wobbrock, J.O., Morris, M.R. and Wilson, A.D. (2009). User-defined gestures for surface computing. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '09). Boston, Massachusetts (April 4-9, 2009). New York: ACM Press, pp. 1083-1092. Best Paper Nominee.
  10. Wobbrock, J.O., Aung, H.H., Rothrock, B. and Myers, B.A. (2005). Maximizing the guessability of symbolic input. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI '05). Portland, Oregon (April 2-7, 2005). New York: ACM Press, pp. 1869-1872.

Copyright © 2019 Jacob O. Wobbrock. All rights reserved.
Last updated January 8, 2022.