Paint By Numbers

Ingrid Daubechies’ eyes dart down at her plate of mixed salad greens. She stabs a hefty chunk of endive hiding beneath an arugula leaf and chews it quickly. The words are coming fast now.

“We don’t get a three-dimensional map,” she says. “We have a much higherdimensional map. More like eighty. But I can only explain it in three dimensions.”

Backlit by a wall of windows in the Refectory, a bustling cafeteria in the divinity school made bright by a gush of autumn sun, Daubechies is describing how she turns a painting into a massive stream of data. It’s esoteric stuff, even when she describes it in three dimensions. The process revolves around a mathematical tool known as a wavelet transform, essentially a formula for identifying patterns in large sets of data. Daubechies (pronounced doh-bee-SHEE), who arrived at Duke in 2011 as a James B. Duke Professor of mathematics, has pioneered the use of wavelet transforms with complex data sets such as digital images—whenever you upload a cell-phone picture to your Facebook page, her formulas are at work compressing the image.

Daubechies’ mastery of wavelets—not to mention her ability to think in eighty dimensions— has made her one of the most prominent mathematicians in the world. Born in Houthalen, Belgium, she studied physics at the Vrije Universiteit in Brussels, completing a doctorate in theoretical physics in 1980. She met American mathematician Robert Calderbank (now dean of natural sciences in Trinity College of Arts & Sciences) five years later and moved to the U.S. when they married in 1987.

After several years of research work at AT&T Bell Laboratories in New Jersey, she joined Princeton’s faculty in 1993, garnering recognition for her work in the interdisciplinary Program in Applied and Computational Mathematics. In 2000, she became the first woman to receive the National Academy of Sciences Award in Mathematics. Ten years later, she was the first woman elected president of the International Mathematical Union.

But for some years now, she’s been as interested in art as math. While picking through her salad, she spells out an idea she’s been working on for several years—that mathematical analysis can illuminate art, that it can help us see things that are hidden to even the most well-trained eye. Like style. Like spontaneity. And like forgery.

Since arriving at Duke, Daubechies has been honing a tool that uses nothing but computational analysis to detect an artist’s original work from a copy. And while reducing art to numbers may unsettle some purists, it’s a high-stakes prospect for collectors and museum directors who may fork over tens of millions of dollars for a masterpiece.

Daubechies grew up appreciating art from family visits to European museums. “Often it was old art—medieval, Renaissance mostly—and then maybe more recent, seventeenth and eighteenth century,”she recalls. “I like the Flemish Primitives very much because people in Belgium are proud of them. I also like Impressionism, and I liked Fauvism very much at some point.” She has even tried her hand at the arts, dabbling in sculpture and ceramics. To her, multidimensionality means direct involvement in an art context. She prefers the handling of clay to the daubing of a paintbrush.

But her involvement with art became a professional matter in 2007, when she was at Princeton. The television series NOVA staged a friendly competition, challenging mathematicians at three universities to come up with a way to ferret out a commissioned forgery from a stack of original Van Gogh masterpieces. Under the supervision of the Van Gogh Museum in Amsterdam, teams from Princeton, Penn State, and Maastricht universities all took similar approaches to analyzing the paintings, using mathematical processes to convert high-resolution scans into detailed maps of the painter’s brushstrokes.

Why brushstrokes? “If you try to make a copy, you pay so much attention to what you’re doing that you probably paint it more slowly and with a more restrained hand than Van Gogh himself would have painted it,” Daubechies explains. A forger will be less spontaneous, making shorter, more precise brushstrokes, she says. The difference may not be easily detectable to the casual eye—the pictures may appear identical. But it’s quantifiable.

Using this analysis, all three teams picked the same painting—a version of Van Gogh’s elegant 1899 work Reaper With Sickle After Millet—as the fake. In the show’s big “reveal” scene, the teams are assembled in the Van Gogh Museum when Charlotte Caspers, a young Dutch artist and conservator, enters with her copy of the painting. The mathematicians all cheer when they find out they got it right. Daubechies even high-fives one of her colleagues.

But for Daubechies, the celebration marked a beginning, rather than an end. After the show, she contacted Caspers and asked her to make more copies of paintings. The painter and the mathematician formed a partnership to expand on the time-constrained NOVA experiment to see what else wavelet transforms might reveal.

Last year, Daubechies helped Caspers secure a joint grant from the North Carolina Museum of Art and the Duke Council for the Arts’ Visiting Artist program to spend two months at Duke. Working out of a makeshift studio in Duke’s Smith Warehouse, Caspers could be found most days in deep concentration, brush or pencil in hand, her nose a few inches from her work surface. For much of September, she painted small images of songbirds that would make Audubon jealous. During October, she copied each of her own songbird paintings with the same diligence with which she’d copied Van Gogh’s Reaper five years before.

The plan is for Daubechies to analyze the pairs of paintings without knowing which is original. In the NOVA test, she reasons, there were two variables— Caspers’ copy of the Reaperwas the only painting not created spontaneously, but it was also the only one not painted by Van Gogh. These experiments will test whether Daubechies’ algorithms can distinguish between an original and a copy painted by the same artist.

“We will have a richer data set than what we had for the Van Goghs,” Daubechies explains. “It’s not that there’s all of a sudden a quantum leap to a different kind of data. It’s just that we will have many more things in our new data set. We have many more paintings. And we will have copies that are painted immediately after the originals were painted.”

“I think Ingrid is just interested to see what happens,” Caspers says while working on a redheaded woodpecker’s wing. “So I paint some originals and copies with different materials, and I change technique sometimes. They have to find out what things the mathematics can do.” As she paints, she makes meticulous notes about the brushes and materials she uses. Every nuance is documented, from differences in underdrawings to variations in stylistic bases.

Caspers’ training in art history and conservation has taught her that copying a painting means more than just duplicating its brushstrokes. When painting the woodpecker’s tail feather, for instance, she’s not overly concerned about exactly matching the number of stripes. It’s about capturing the spirit of the tail feather, she says.

“If you’re copying you could be counting strokes, but a good copy’s also about doing it in the same way and trying to get the same atmosphere. And of course every painting is quite personal. The copy as well.”

Therein lies much of what makes Daubechies’ project so counterintuitive. It seems impossible to capture something as intangible and personal as the spirit of a tail feather in the absolute realities of data.

It’s certainly possible, Daubechies argues. Just incredibly complicated. See: eighty dimensions.

The first step in crunching a painting into data is to make a high-resolution scan of the work. A scanner breaks the painting down into tiny squares—pixels—and assigns each a color value. Daubechies uses an eight-bit color scan, which is capable of expressing 256 shades of red, green, and blue, creating more than 16 million distinct hues. Each one has a specific numeric code. A nice olive green would show up as R:128 G:128 B:0. Plug in R:128 G:0 B:128 and you get magenta.

“All we have are pixels and their neighbors,” Daubechies points out. “Colors and their contrasts and similarities are all we have to work with. However, we have it in such a fine scale that indirectly you have brushstroke information there.”

She slices the paint-by-numbers scan into half-inch-square “patches” before applying formulas to transform the data in various ways. These transformations are strictly mathematical—they don’t relate to any specific aspect of the painting or how it was made—but they sometimes reveal patterns of similar numbers, which Daubechies calls “clusters.” Those clusters don’t map to any line or shape that the human eye could detect on the canvas, but she thinks they may represent unique characteristics of an artist’s style. A deep, granular look into color patterns is like a fingerprint in which swoops and swirls can reveal a person’s identity while telling you nothing about personality or physical appearance.

Daubechies offers a sonic metaphor. “In sound, in voice, we’re very good at recognizing the speaker—not just in understanding what is said, but also in recognizing the speaker. We’re good at that even if we’re hearing an utterance that we’ve never heard that speaker say,” she says. “You can tape sound with a microphone and sample it. You analyze it piece by piece, and you can find ways of visualizing it as spots in a many-dimensional space, particularly using two dimensions. Where those cluster islands lie is not dependent, if you think for a minute, on what words you’ve said. But the whole archipelago is really characteristic of the speaker.”

And that’s where the multiple dimensions come into play. Sorting through the patches of a painting, Daubechies says, is like matching pieces of a jigsaw puzzle. “Even if you don’t know what the pieces stand for, you can already start sorting them. You say, ‘This looks very similar to that. And this one looks lighter in color.’And so you sort them and this helps you solve the puzzle.”

But, she adds, there are multiple ways to sort the pieces. “I’ve had times that I was making a jigsaw puzzle with my daughter, for instance, and there were pieces that I thought were in similar families and that she thought were completely different. So we were not looking at the same characteristics.”

Daubechies’ formulas essentially sort the pieces again and again, each time using a different set of characteristics, each time looking at the unsorted patches in a new dimension. “What we are trying to find is the right characteristics so that when we sort them according to those characteristics, they end up in a pile that is mostly correctly segregated between original and copy pieces.”

Her programs are trained to search out clusters that occur again and again in a particular artist’s work—and not in works by others—suggesting that they are inherent to that artist’s particular style. Many dimensions will yield no insight, but Daubechies is banking on the chance that some will.

“A data set doesn’t know anything, but it is a collection of things that have a lot in common,” she says. “What we’re trying to do is to learn how to see it.”

The wooden panel shines with the flat gold of gaudy tableware. William Brown, chief conservator at the North Carolina Museum of Art, plunks it onto a table and holds up a finger to indicate he’ll be right back. On the panel, St. John the Evangelist peers from the gold field in half-profile, depicted in fresh, bright tempera colors. His arm is raised in a benevolent gesture, hand extending from his sleeve.

In a moment Brown returns with a second panel, which he sets alongside the first. This one has a similar scene but is dull and clouded, its beige and brown tones oxidized by time. There’s St. John again, but it’s harder to tell what he’s up to.

We’re in the basement of the museum, in a spacious laboratory that houses the museum’s conservation department. Science is no stranger here—Brown and his staff are used to crossing art history with scientific techniques in their efforts to preserve and restore ancient works of art. Brown is intrigued by the idea that statistical analysis could be another tool at his disposal, even if it is something of a black box.

Brown glances back and forth between the two panels on the table. One is ancient, part of an altarpiece created by fourteenthcentury Italian painter Francescuccio di Cecco Ghissi. At some point, the altarpiece, depicting the life of St. John the Evangelist, was cut into eight pieces. The NCMA owns three, and it has made plans to borrow four others from museums in New York, Chicago, and Portland, Oregon, for a 2013 exhibition. The eighth panel is likely lost to history.

Enter Caspers, with her knowledge of medieval and early Renaissance materials and techniques. Based on contemporaneous works on the same subject, art historians know what scene the missing panel likely portrayed. As part of her grant with NCMA, Caspers recreated the scene in historically accurate fashion, using eggs to make her tempera paints and applying gold with the technique of Ghissi’s day.

Daubechies, meanwhile, is unleashing her wavelets on the original Ghissi panels in the NCMA collection. Using Caspers’ new painting as a baseline, she’s building a mathematical model to quantify the effects of more than six centuries of aging. Her tool will analyze patterns of color degradation and cracks that have formed in the surface of the paint and then “undo” them, offering a digital picture of what Ghissi’s panels might have looked like on the day the altarpiece was installed. She also is developing a tool to work in the opposite direction, aging Caspers’ new panel to show how it might look if it were 650 years old.

“It helps us visualize what a painting would look like without the crack pattern,” Brown says, “which would help us understand what the painting would have looked like closer to when it was an original. But sometimes it helps with restoration issues, too. So it’s an interesting practical application that has connoisseurship applications and also puts it in more of a historical perspective.”

“That’s the benefit of these interdisciplinary interactions, and I know Duke is all about that,” he says.

It’s not the first time Brown has seen the power of math to enlighten others about art. Earlier he worked with one of Daubechies’ graduate students in studying panels of an altarpiece by the fourteenth- century painter Giotto. Brown and his team had been trying to discern the master’s handiwork from that of his studio’s hired hands. With no background in art history and perhaps never having heard of Giotto, the student independently reached the same conclusions that professional art historians had.

“A connoisseur breaks it down visually based on iconography and other tools that the art historian has,” Brown says, caressing the airspace above St. John’s purposeful gesture. “And the conservator understands the materials and how they were applied. Art historians don’t always know that. So you apply that to the Giotto, and you can see differences in how the brushstrokes were applied and so on. But that’s not what Ingrid’s doing. She’s breaking it down into a total abstraction.”

Looking up from the panels, Brown’s tone of wonderment downshifts to matter- of-fact. “But then where she got with that was very consistent with where the connoisseur got through breaking it down subjectively. She applies her totally objective mathematical analysis and gets to the same point.”

A curator might shrug, but every museum has a file on every work in its collection. Those files are filled with opinions about each work, gathered over generations of scholarship. But opinions can change.

There’s a moment in the NOVA episode, for example, when a museum curator marvels at the uncanny similarity of Caspers’ Reaper to its original. At a casual glance, any museum visitor would mistake it for a Van Gogh. But Daubechies believes that won’t always be the case.

She points to the case of Han van Meegeren, a Dutch artist who in 1937 claimed to have found a new Vermeer painting, a discovery from which he profited handsomely. By 1942, six more Vermeers had surfaced, stunning experts who unanimously and vehemently declared their authenticity. Only in 1945, facing trial as a Nazi collaborator for having sold one of his supposed Vermeers to Nazi officer Hermann Goering, did van Meegeren admit to forging the paintings. His technique, using handmade pigments that mimicked Vermeer’s finish, wouldn’t fool anyone today. “You can’t understand how anybody could have thought these were Vermeers,” Daubechies says. “I mean, there’s no way.

“I don’t believe that we’ve become that much smarter than the experts were then. I think we just have added a dimension to our looking. We have learned to see things that they weren’t paying attention to. Seeing is very different. It’s not just that there’s an objective reality and we sense it. We do a lot with it in our brains, and some people see things that others don’t see.”

What Daubechies understands is that it’s not about whether her wavelet functions can beat an art connoisseur in identifying a masterpiece. Connoisseurship and science are both accretive processes, where new information is quickly absorbed across a wider community of practitioners. She’s envisioning a day when statistical analysis will teach us how to look at art differently, to appreciate a dimension we haven’t been able to grasp before. And when it comes to grasping new dimensions, Daubechies is already about seventy-seven steps ahead of the rest of us.

Vitiello is a writer and art critic based in Durham. His most recent book of poetry, Obedience, was published in March 2012.