Projects > Media Annotation > Advancing Digital Imagery Technologies

Advancing Digital Imagery Technologies
for Art and Cultural Heritages

Featured on PBS NOVA, The Associated Press and The Philadelphia Inquirer
Our art authentication project, jointly with Van Gogh Museum, Kröller-Müller Museum, Cornell, Princeton, and Maastrichit Univ., is featured on PBS NOVA ScienceNow TV Program, July 2, 2008. (watch online) It is also reported by The Associated Press, The Philadelphia Inquirer (pdf) and The Economist, UK. Read the technical publication.

This project focuses on the preservation, retrieval, and dissemination of digital imagery for art and cultural heritages. For more information on image retrieval, please refer to the main Media Annotation site. This specific project focuses on using that technology on art and cultural images. Image retrieval allows a user to search for images similar to the one they are looking at. This technology therefor does not have any language barriers since words are not used. It is also very useful when the user does not know what keywords to use. For example, I may want to do a report comparing modern day artists to van Gogh. However, I do not know any artists that base their work off of van Gogh. By searching for artwork that is similar to a digital image of his work, I am able to find information for my report. We also develop image analysis tools for art historians. We collaborate with various institutions including The Van Gogh Museum in Amsterdam. More info on a workshop series related to this project can be found at digitalpaintinganalysis.org .

Project Directors

This research is being continued at The Pennsylvania State University by:

Project Background

This collaborative research project, started in August 2002, aims at advancing information technologies related to the preservation, retrieval, and dissemination of digital imagery for Asian art and cultural heritages. This project will contribute to the fundamental knowledge and technologies required to create and maintain information systems that can operate in multiple languages, formats, media, and social and organizational contexts. The US research team will focus on (1) multimodal media management, (2) distributed data mining, and (3) image storage and display. The team will develop theoretical foundations to associate linguistic terms with image features. The research work will show that (1) modern machine learning and statistical data mining tools are capable of learning from non-structured or semi-structured input data such as human annotations, (2) statistical image modeling techniques can be used in automatic linguistic indexing and concept dictionary building.

The content-based image retrieval and automatic learning-based linguistic indexing part of the project was started in 1995 when James Z. Wang developed an art image retrieval system for the Stanford University Libraries. He has later worked for the IBM QBIC project, the NEC AMORA project, and the AI Center of SRI International. From 1999 to 2000, the project was funded by the National Science Foundation under the Digital Library Initiative II (DLI-II) program. The technology research has been continued at Penn State University since 2001, with funding from the National Science Foundation, PNC Foundation, and SUN Microsystems. If you have any questions or comments about the image retrieval project, please send a message to James Z. Wang (jwang @ ist.psu. edu). For more technical information, please refer to our publications listed below.

Demonstrations: Searching the Art Databases

Photographic Art Demo

This Photographic Art Demo has 13,000 images. They are copyright of images by QT Luong.

Art Demo

This Art Demo has 132,000 thumbnail images. They are copyright by various owners.

Global Memory Net - memorynet.org

Enter Global Memory Net.

Paintings Demo

This Paintings Demo has 1,200 images. They are copyright of various museums.

Copyright Notice:

Images we used are for our scientific research and for viewing ONLY. Please do NOT copy or download any images. This project is on developing digital imagery technologies.

The copyrights of the images of ancient works belong to the museums or the owners of the images. Some images are used in the scholarly papers for the purposes of teaching, scholarship or research, which are considered as "Fair Use" because (1) the amount of copyrighted work used in the scholarly papers is reasonable, (2) the importance of that part of the work is not substantial to the whole work, and (3) the effect of the use upon the value or potential value of the copyrighted work is not significant. (See 17 U.S.C.A. 107).

Related Publications

  1. C. Richard Johnson, Jr., Ella Hendriks, Igor Berezhnoy, Eugene Brevdo, Shannon Hughes, Ingrid Daubechies, Jia Li, Eric Postma and James Z. Wang, ``Image Processing for Artist Identification - Computerized Analysis of Vincent van Gogh's Painting Brushstrokes,'' IEEE Signal Processing Magazine, Special Issue on Visual Cultural Heritage, vol. 25, no. 4, pp. 37-48, 2008. (download) (g-scholar)

  2. Dean R. Snow, Mark Gahegan, C. Lee. Giles, Kenneth G. Hirth, George R. Milner, Prasenjit Mitra and James Z. Wang, ``Cybertools and Archaeology,'' Science, vol. 311, issue. 5763, pp. 958-959, February 17, 2006. (download) (g-scholar)

  3. James Z. Wang, Kurt Grieb, Ya Zhang, Ching-chih Chen, Yixin Chen and Jia Li, ``Machine Annotation and Retrieval for Digital Imagery of Historical Materials,'' International Journal on Digital Libraries, Special Issue on Multimedia Contents and Management in Digital Libraries, vol. 6, no. 1, pp 18-29, Springer-Verlag, 2006. (download) (g-scholar)

  4. Ching-chih Chen, Howard Wactlar, James Z. Wang and Kevin Kiernan, ``Digital Imagery for Significant Cultural and Historical Materials,'' International Journal on Digital Libraries, Special Issue: Towards the New Generation Digital Libraries: Recommendations of the US-NSF/EU-DELOS Working Groups, vol. 5, no. 4, pp. 275-286, 2005. (download) (g-scholar)

  5. Jia Li and James Z. Wang, ``Studying Digital Imagery of Ancient Paintings by Mixtures of Stochastic Models,'' IEEE Transactions on Image Processing, vol. 13, no. 3, pp. 340-353, 2004. (download) (g-scholar)

  6. James Z. Wang, Jia Li and Ching-chih Chen, ``Machine Annotation for Digital Imagery of Historical Materials Using the ALIP System,'' Proceedings of the DELOS-NSF Workshop on Multimedia in Digital Libraries, 5 pages, Crete, Greece, 2003. In e-proceedings. [invited] (download) (g-scholar)

  7. Ching-chih Chen and James Z. Wang, ``Large-Scale Emperor Digital Library and Semantics-Sensitive Region-Based Retrieval,'' Proceedings of the International Conference on Digital Library -- IT Opportunities and Challenges in the New Millennium, pp. 454-462, Beijing, China, July 9-11, 2002. (download) (g-scholar)

  8. James Z. Wang, Jia Li and Ching-chih Chen, ``Interdisciplinary Research to Advance Digital Imagery Indexing and Retrieval Technologies for Asian Art and Cultural Heritages,'' Proceedings of the 4th ACM International Workshop on Multimedia Information Retrieval, in conjunction with ACM Multimedia, 6 pages, Juan Les Pins, France, ACM, December 2002. [invited] (download) (g-scholar)

More Information

  1. Research Group
  2. Related projects done by this group