"Video is much better for showing when a piece of fruit reaches the consumer quality threshold."
Tim Cronshaw, Apeel Software Engineer
It's impressive to read about Apeel's experiments that double or even quintuple the shelf life of fruits and vegetables using the natural, plant-based Apeel product. It's jaw-dropping to watch the time-lapse videos.
Typical footage features untreated fruit on one side of the video frame and the fruit treated with Apeel on the other, with the number of days since the start of the test displayed in the center. As the untreated sample shrivels and becomes covered in gray mold, the Apeel-protected side remains visibly fresher longer.
In the string beans video, the untreated produce shrinks and twirls in a dance of decay as the day count ticks upward. The string beans treated with Apeel, meanwhile, appear relatively unchanged and far more appetizing. A common viewer reaction to such dramatic proof of shelf-life extension: "Now that's game-changing."
Apeel's in-house developed time-lapse video system, known internally as the Time Machine, highlights versatility in delaying the onset of spoilage in a range of produce including avocados, bananas, berries, cassava, citrus, mangoes and tomatoes.
But the Time Machine does more than capture raw video. The automated, teachable system also records and analyzes vital data about produce color, texture, size and volume.
"None of the technology is never-before-seen," said Apeel software engineer Tim Cronshaw, "but the result seems to be. The Time Machine is a creative blend of different disciplines that combine to effectively demonstrate the purpose and power of Apeel products."
It took about six months to design and build Apeel's Time Machine, and another six months to fine-tune its operation. Cronshaw noted that a faster approach would have been simply to use still photos to compare fruit samples at the beginning and end of a study, but key information about trends in color loss and volume loss would have been missed.
"Video is much better for showing when a piece of fruit reaches the consumer quality threshold," he said, referring to the point at which shoppers consider produce to be no longer fresh and worth buying. "A video might show untreated fruit crossing the consumer quality threshold on Day 10, for example, but treated fruit not crossing that threshold until Day 20."
Time Machine hardware consists of a camera mounted on a track above a rack that holds produce samples, a stepper motor that slides the camera between samples and several Raspberry Pi mini-computers that control the camera and store images. Running on a server computer, a "computational pipeline" then pulls images from the Raspberry Pis in order to produce analytical reports and post-process the images into finalized videos.
TensorFlow, Google's open-source software for machine learning, empowers an especially promising type of image analysis. As Apeel technicians continue to mark thousands of individual images as either acceptable or unacceptable examples of color and other produce traits, the Time Machine is teaching itself to recognize new images as being inside or outside of quality parameters.
The system also uses a volume analysis formula to estimate the decrease in a sample as it loses moisture content. "We're able to translate video pixels to real-life scale," said Cronshaw, adding that Time Machine volume estimates have been validated using the physical method of water displacement. The Time Machine might be adapted in the future to quantify plant growth, he added, or even deployed in the field to analyze fruits and vegetables prior to harvest.
The Time Machine showcases the innovative technical talent pool at Apeel. "None of us were experts in photography, image processing or machine learning in the beginning," Cronshaw recalled. "But we are now. It speaks to the incredible potential of everyone here, and to one of our core values: no job is too big or too small. People from all over the company would lean over and help out with a piece of the project."
One step in the time-lapse video system still needs to be performed manually. After placing a fruit or vegetable sample on the rack, the operator must initialize the experiment by manually selecting the specific Apeel formula being tested. The machine cannot detect Apeel because it's invisible as well as tasteless and edible.
Defining terms like fresh, ripe, peak color, shelf life and consumer quality threshold also remains an imprecise and subjective exercise. But anyone who has watched an Apeel time-lapse video would much rather dine on the Apeel side of the screen.
Tim Cronshaw marks precise locations for the camera to shoot.
A partially transparent hood optimizes the light for each kind of fruit.
The most similar fruits are isolated from a group, then randomly selected for the Apeel side or the untreated side.