$N Multistroke Recognizer

Lisa Anthony, University of Maryland—Baltimore County
Jacob O. Wobbrock, University of Washington [contact]

Currently at the University of Florida

Download

$N source code: JavaScript, C#
Pseudocode: $N, $N-Protractor
Multistroke gesture logs: XML
Papers: $N, $N-Protractor

This software is distributed under the New BSD License agreement.

About

The $N Multistroke Recognizer is a 2-D stroke recognizer designed for rapid prototyping of gesture-based user interfaces. In machine learning terms, $N is an instance-based nearest-neighbor classifier with a Euclidean scoring function, i.e., a geometric template matcher. $N is built upon the $1 Unistroke Recognizer. $N automatically generalizes examples of multistrokes to encompass all possible stroke orders and directions. This means you can make and define multistrokes using any stroke order and direction you wish, provided you begin at either endpoint of each component stroke. A version of $N utilizing Protractor, optional here, improves $N's speed. The $P Point-Cloud Recognizer is the latest in the dollar family, performing unistroke and multistroke recognition without the combinatoric overhead of $N. Read about the limitations of $N, which $P largely avoids.

Demo

In the demo below, only one multistroke template is loaded for each of the 16 gesture types. You can add additional multistrokes as you wish, and even define your own custom multistrokes.


Use Golden Section Search (original)
Use Protractor (faster)
Use bounded rotation invariance (±45°)
Require same no. of component strokes

Make strokes on this canvas. Right-click the canvas to recognize. If a misrecognition occurs, add the mis- recognized gesture as an example of the intended gesture.

The <canvas> element is not supported by this browser.
Add as example of existing type:
Add as example of custom type:
Delete all user-defined gestures:  

$N By Others

Our Related Software Projects

Our Related Publications

Vatavu, R.-D., Anthony, L. and Wobbrock, J.O. (2014). Gesture heatmaps: Understanding gesture performance with colorful visualizations. Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI '14). Istanbul, Turkey (November 12-16, 2014). New York: ACM Press, pp. 172-179.

Vatavu, R.-D., Anthony, L. and Wobbrock, J.O. (2013). Relative accuracy measures for stroke gestures. Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI '13). Sydney, Australia (December 9-13, 2013). New York: ACM Press, pp. 279-286.

Anthony, L., Vatavu, R.-D. and Wobbrock, J.O. (2013). Understanding the consistency of users' pen and finger stroke gesture articulation. Proceedings of Graphics Interface (GI '13). Regina, Saskatchewan (May 29-31, 2013). Toronto, Ontario: Canadian Information Processing Society, pp. 87-94.

Vatavu, R.-D., Anthony, L. and Wobbrock, J.O. (2012). Gestures as point clouds: A $P recognizer for user interface prototypes. Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI '12). Santa Monica, California (October 22-26, 2012). New York: ACM Press, pp. 273-280.

Anthony, L. and Wobbrock, J.O. (2012). $N-Protractor: A fast and accurate multistroke recognizer. Proceedings of Graphics Interface (GI '12). Toronto, Ontario (May 28-30, 2012). Toronto, Ontario: Canadian Information Processing Society, pp. 117-120.

Anthony, L. and Wobbrock, J.O. (2010). A lightweight multistroke recognizer for user interface prototypes. Proceedings of Graphics Interface (GI '10). Ottawa, Ontario (May 31-June 2, 2010). Toronto, Ontario: Canadian Information Processing Society, pp. 245-252.

Wobbrock, J.O., Wilson, A.D. and Li, Y. (2007). Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '07). Newport, Rhode Island (October 7-10, 2007). New York: ACM Press, pp. 159-168.


Copyright © 2010-2014 Jacob O. Wobbrock