Digital Forensics. Think of that phrase. It could be the title of a new TV crime show. However, recently developed software by Dr. Hany Farid of Dartmouth College is far more serious than an hour of frivolous, sensationalist TV.

In an e-mail interview I recently conducted with Dr. Farid he discusses the ramifications of how photos, and, to a degree, video, can undergo manipulation in this era of 2.0, and as we slide faster than we know into 3.0.

As the interview makes clear, manipulation and doctoring of photos has a long history. Yet digital, with new tools that seem to increase daily, allows those without journalistic ethics, and those who have something to sell, whether politics, sex, celebrity, stocks or what have you, to alter individual images and sets of images for his or her gain. Some photographers, editors, plain folk, politicians and propagandists feel they have the right to alter any image or set of images for personal, political and commercial gain, often depending on his or her point of view.

For these people, the original or "real" photo may not be good enough. The image they see needs work, they think, and they know exactly how to reorder reality. It is a growing problem and one that any of us who has an interest in photography and truth should be alert to and understand.

After each answer, there are links supplied by Dr. Farid for those interested in more details about his software, his research and conclusions.

Q. Photo manipulation is nothing new. Explain. A brief history, if you can. Add links that are pertinent, especially the one you call, "Photo Tampering Throughout History."

A. We may have the sense that photo manipulation is a byproduct of digital photography. The reality is, however, that photographs have been manipulated for nearly as long as photography has been around. The nearly iconic portrait of Abraham Lincoln (circa 1860), for example, is a composite of Lincoln's head and Senator John Calhoun's body. Famed Civil War photographer Mathew Brady routinely doctored photographs to create more compelling images. And, Stalin, Mao, Hitler, Mussolini, Castro, and many more each had photographs manipulated in an attempt to alter history. Although photo manipulation is not new, digital photography has certainly made it easier to manipulate photographs in ways that can be difficult to detect.

Photo Tampering Throughout History: http://www.cs.dartmouth.edu/farid/research/digitaltampering/

Q. Why did you develop Digital Forensics? In other words, why now? Is it because you can? Because it is possible to create software to protect the professional and the casual viewer from fraud?

A. Ten years ago I stumbled upon a clause in the Federal Rules of Evidence that stated that a 35mm negative and a digital image are equivalent when photographs are introduced as evidence into a court of law. That is, film or digital photos can be introduced as evidence, and they hold equal weight. At the time, I thought that this was a problem, and that it was going to become a bigger problem as digital photography became more ubiquitous. So I started to think about how we could authenticate digital images. After three years of thinking and many dead ends, my graduate students and I started to make progress. And over the past seven years, we have developed a suite of tools for detecting digital tampering. While I was primarily motivated by the issue of photographic evidence in the courts, my hope is that our work will also help the media, among others, contend with the issue of digital manipulation.

Federal Rules of Evidence (Article X, Rule 1001 (3)): http://www.law.cornell.edu/rules/fre

Digital Tampering and Forensics: http://www.cs.dartmouth.edu/farid/research/tampering.html

Q. You say, "A digitally altered photograph, often leaving no visual clues of having been tampered with, can be indistinguishable from an authentic photograph. As a result, photographs no longer hold the unique stature as a definitive record of events." Is this a warning? Are you saying, consumer beware? Does your comment mean that we never trust what we see, even if it comes from a usual reliable source?

A. The fact is that many of the images we consume daily have been manipulated and altered. And it isn't just the tabloid magazines and fashion magazines that are doctoring photographs. The New York Times, the Los Angeles Times, Reuters, Newsweek, to name a few, have each recently published manipulated photographs, for which they have been widely criticized. I don't advocate never trusting anything, but I do advocate being aware that images are regularly being manipulated and that we should be careful to consider this when consuming mass media.

Q. Explain some of the more commonly used methods for tampering with images.

A. While the digital technology to manipulate photographs is relatively new, the types of photographic fakes being created have not changed much throughout history. As with the Lincoln/Calhoun fake, attaching a person's head to another person's body remains popular: Oprah Winfrey's head on Ann-Margret's body on the cover of TV Guide, Texas Governor Ann Richards' head on a model's body on the cover of Texas Monthly, and Martha Stewart's head on a model's body on the cover of Newsweek. Compositing techniques have also remained popular: in 1950 a photo of Senator Millard Tydings conversing with a leader of the American Communist Party is thought to have contributed to Tydings' electoral defeat. In 2004, a doctored photo was widely circulated showing then-presidential hopeful Senator John Kerry sharing a stage with anti-war activist Jane Fonda, and, of course, tabloids routinely create composites of various Hollywood stars with rumored sweethearts. It is also popular and easy to airbrush out of digital images everything from small details to people and buildings.

Photo Tampering Throughout History: http://www.cs.dartmouth.edu/farid/research/digitaltampering/

Q. You have thoughts on what you call digital doctoring – how to tell the real from the fake. Explain.

A. My students and I have developed mathematical and computational programs to detect tampering in digital images (and audio and video). Our general philosophy is that there are many ways to doctor an image and that we need at least as many different tools to detect tampering. Our philosophy is also that there is never going to be a magic button that you can push to determine authenticity. A suite of tools will always be needed, where each tool detects a specific form of tampering. It is also important to remember that, like spam/anti-spam and virus/anti-virus, we are involved in a form of an arms race. Techniques to doctor images will continue to improve, techniques in digital forensics will continue to improve, and on and on we will go. In the end, however, it will always be easier to create a fake image than it will be to detect it, and forensics will never stop photographic fakery – it will only make it increasingly harder and more time-consuming to create a compelling fake.

Q. Is there a way a casual viewer can tell what is real from what may have been fixed?

A. While a good forgery can be very difficult to detect visually, there are certain clues that a viewer can look for. Lighting, in particular, is a powerful cue. When creating a forgery of two people, for example, it is often very difficult to perfectly match the lighting. Differences in shadow direction and softness, and lighting gradients can be a good cue. Similarly, a person's eye often contains a small white specularity which is a reflection of the light in their surroundings. Differences in the shape and color of this specularity can reveal tampering. A popular, but in my opinion unreliable, visual analysis is to magnify an image by several hundred percent and then posit about the source of small pixel artifacts. Many of these artifacts are due to compression or noise, so care must be taken when undergoing any type of visual inspection.

Exposing Digital Forgeries Through Specular Highlights on the Eye: http://www.cs.dartmouth.edu/farid/publications/ih07.html

Exposing Digital Forgeries by Detecting Inconsistencies in Lighting: http://www.cs.dartmouth.edu/farid/publications/acm05.html

Q. To protect against tampering a digital image, briefly tell me how, in your words, your system can "make passive detection of digital tampering possible." And as part two of this question, should editors use this system automatically for all images or only those they suspect of tampering?

A. We have developed a suite of tools to detect tampering in images (and video). Some of these tools can run in batch mode and automatically process many images very efficiently. Other tools require several manual steps and are best suited for processing a relatively small number of images. As such, it is possible that editors could use some of our tools to automatically process all images, and then other tools to analyze those that are flagged as possibly suspicious. One of the biggest challenges here is that virtually all images are cropped and color corrected prior to processing, so differentiating between acceptable and unacceptable alterations is difficult.

Link to all technical papers describing these tools: http://www.cs.dartmouth.edu/farid/research/tampering.html

Q. This is not an afterthought, but with video in the ascendancy, especially on news Web sites and broadband in general, what must we look at to make sure what we are seeing in any video is honest?

A. We have recently developed tools to detect tampering in video. Although it is decidedly harder to tamper with video, it is also harder to analyze because of the sheer volume of data from even a short video. In addition, because of the relatively fast frame rate at which video is displayed, it can be hard to visually detect any static inconsistencies. In addition to continually developing new image forensic tools, I see video (and audio and scanned document) forensics as an important frontier in digital forensics.

Exposing Digital Forgeries in Interlaced and De-Interlaced Video: http://www.cs.dartmouth.edu/farid/publications/tifs07.html

Exposing Digital Forgeries in Video by Detecting Double MPEG Compression: http://www.cs.dartmouth.edu/farid/publications/acm06b.html

Q. As a final word, do you have any advice for photojournalists and editors?

A. Digital technology has made it possible for photojournalists to easily manipulate images. And while I believe that the technology of digital forensics can partially combat this problem, technology alone will not solve this problem. Media outlets have to develop clear and practical guidelines for photojournalists, they have to teach journalistic ethics, and they have to have zero-tolerance for the types of photographic fakes that are currently being published in many newspapers and news magazines.

Hany Farid received his undergraduate degree in Computer Science and Applied Mathematics from the University of Rochester in 1989. He received his Ph.D. in Computer Science from the University of Pennsylvania in 1997. Following a two-year post-doctoral position in Brain and Cognitive Sciences at MIT, he joined the Dartmouth faculty in 1999. Hany is the David T. McLaughlin Distinguished Professor of Computer Science and Associate Chair of Computer Science. He is also affiliated with the Institute for Security Technology Studies at Dartmouth. Hany is the recipient of an NSF CAREER award, a Sloan Fellowship and a Guggenheim Fellowship.

From working with federal law enforcement agencies on digital forensics, to the digital reconstruction of Ancient Egyptian tombs, Hany works and plays with digital media at the crossroads of computer science, engineering, mathematics, optics, and psychology.