UW Aquatic & Fishery Sciences Quantitative Seminar

Neeraj Kumar

University of Washington
Computer Science and Engineering

Leafsnap - A Computer Vision System for Automatic Plant Species Identification


Botanists in the field are racing to capture the complexity of Earth's flora before climate change and development erase their living record. To greatly speed up the process of plant species identification, collection, and monitoring, we have built and publicly released the first hand-held botanical identification system. The system -- called Leafsnap -- identifies tree species from photographs of their leaves. With nearly a million downloads, we believe it is the most widely deployed non-commercial computer vision system in the world. In this talk, I will describe the development of this system from its early motivations to its current success. In particular, I will cover our efforts to digitize the Smithsonian's collection of type specimens of 100,000 plant species, and the subsequent creation of an app for public use.

The key technical pieces of the app are segmenting the leaf from the background, computing curvature features robustly along the boundary of the segmented leaf, and performing identification using these extracted features. I'll also describe the insights learned from creating and deploying such a large-scale system to a non-technical set of users. Finally, I'll try to place this work in the larger context of the vision community, in particular within the burgeoning "fine-grained visual recognition" area.

Quantitative Seminar Home