AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections

People

• Jun-Yan Zhu
• Yong Jae Lee
• Alexei Efros

Video

(Watch in HD!)

Popular Press

Abstract

This paper proposes an interactive framework that allows a user to rapidly explore and visualize a large image collection using the medium of average images. Average images have been gaining popularity as means of artistic expression and data visualization, but the creation of compelling examples is a surprisingly laborious and manual process. Our interactive, real-time system provides a way to summarize large amounts of visual data by weighted average(s) of an image collection, with the weights reflecting user-indicated importance. The aim is to capture not just the mean of the distribution, but a set of modes discovered via interactive exploration. We pose this exploration in terms of a user interactively “editing” the average image using various types of strokes, brushes and warps, similar to a normal image editor, with each user interaction providing a new constraint to update the average. New weighted averages can be spawned and edited either individually or jointly. Together, these tools allow the user to simultaneously perform two fundamental operations on visual data: user-guided clustering and user-guided alignment, within the same framework. We show that our system is useful for various computer vision and graphics applications

Paper

SIGGRAPH paper. (pdf, 5MB)

Presentation

(pptx + videos), 363MB

Citation

Jun-Yan Zhu, Yong Jae Lee, Alexei A. Efros. AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections. ACM Transactions on Graphics (SIGGRAPH 2014). August 2014, vol. 33, No. 4.
Bibtex

Additional Results

Online Shopping Our system could be useful to an online shopper when browsing products.(e.g. on AMAZON)

 

Average Cat Face for Different Cat Breeds Trying moving your mouse over the image. Click the image to see the keypoints/parts annotation.

 

Image Average "Refocusing" We can focus on different parts of the "Bridge of Sighs". We can even generate the object-centric average in real-time.

Additional Materials

• Video (Download), 163MB

• Supplemental Material, 3.6MB
  Include more comparison results and the details of our user studies.

Related Work

• Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman and Alexei A. Efros. "Generative Visual Manipulation on the Natural Image Manifold", in ECCV 2016.

Other work in Image Averaging

• Sir Francis Galton, "Composite Portraits" (1880s). See his Nature paper in 1878
• Krzysztof Pruszkowski, e.g. "60 passagers de 2e classe du metro. Ligne: Clignancourt-Orleans. Paris" (1985)
• Jason Salavon, e.g. "The Class of 1988 & The Class of 1967" (1998) and "100 Special Moments" (2004)
• Antonio Torralba, "Object-Centric Averages" (2001)
• James Campbell, e.g. "Illuminated Average Series" (2000-2001)
• Idris Khan, e.g. "Every...Bernd And Hilla Becher Gable Sided Houses" (2004)

Acknowledgement

We thank Tinghui Zhou, Abhinav Shrivastava, Carl Doersch, and Shiry Ginosar for helpful insights and discussions, and the anonymous reviewers for valuable comments.

Funding

This research is supported in part by:

• Google Research Grant
• Adobe Research Grant
• ONR MURI N000141010934