Using Large-Scale Web Data to Facilitate Textual Query Based Retrieval of Consumer Photos.
ACM Multimedia 2009 (Full Paper).
Its extended version is to appear in IEEE Transactions on Pattern Analysis and Machine Intelligence.
The rapid popularization of digital cameras and mobile phone cameras has lead to an explosive growth of consumer photo collections. In this paper, we present a real-time textual query based personal photo retrieval system by leveraging millions of web images and their associated rich textual descriptions (captions, categories, etc.). After a user provides a textual query, our system exploits the inverted file method to automatically find the positive web images that are related to the textual query as well as the negative web images which are irrelevant to the textual query. Based on these automatically retrieved relevant and irrelevant web images, we employ two simple but effective classification methods, k Nearest Neighbor (kNN) and decision stumps, to rank personal consumer photos. To further improve the photo retrieval performance, we propose three new relevance feedback methods via cross-domain learning. These methods effectively utilize both the web images and the consumer images. In particular, our proposed cross-domain learning methods can learn robust classifiers with only a very limited amount of labeled consumer photos from the user by leveraging the pre-learned decision stumps at interactive response time. Extensive experiments on both consumer and professional stock photo datasets demonstrated the effectiveness and efficiency of our system, which is also inherently not limited by any predefined lexicon.
[ abstract ] [ paper ] [ demo paper ] [ talk ]
Texture Splicing.
Pacific Graphics 2009.
We propose a new texture editing operation called texture splicing. For this operation, we regard a texture as having repetitive elements (textons) seamlessly distributed in a particular pattern. Taking two textures as input, texture splicing generates a new texture by selecting the texton appearance from one texture and distribution from the other. Texture splicing involves self-similarity search to extract the distribution, distribution warping, context-dependent warping, and finally, texture refinement to preserve overall appearance. We show a variety of results to illustrate this operation.
[ abstract ] [ paper ] [ talk ] [ video ] [ supplementary material ]
The Dual-Microfacet Model for Capturing Thin Transparent Slabs.
Pacific Graphics 2009. Distinguished Paper Award.
We present a new model, called the dual-microfacet, for those materials such as paper and plastic formed by a thin, transparent slab lying between two surfaces of spatially varying roughness. Light transmission through the slab is represented by a microfacet-based BTDF which tabulates the microfacet’s normal distribution (NDF) as a function of surface location. Though the material is bounded by two surfaces of different roughness, we approximate light transmission through it by a virtual slab determined by a single spatially-varying NDF. This enables efficient capturing of spatially variant transparent slices. We describe a device for measuring this model over a flat sample by shining light from a CRT behind it and capturing a sequence of images from a single view. Our method captures both angular and spatial variation in the BTDF and provides a good match to measured materials.