Smartphones have changed the way we interact with photos, and soon there will be cameras everywhere.
A decade ago, I had one of the most formative experience of my life, spending 10 weeks in Europe, traveling by van to 13 different countries. Of course I wanted to document my trip, and this being 2007, it was all done on digital camera. That plants the experience at a very specific moment in time: just a few years earlier, I would have had a regular camera with film in it. Just a few years later, it would have been all on my smartphone.
That just shows how rapidly the technology and our experiences with digital images have evolved over the last 10 to 15 years. Those advancements are not slowing down, and continue to change quickly, thanks to mobile devices as well as social media, and advancements in artificial intelligence and machine learning. I recently wrote a piece about some of the startups taking advantage of these new technologies. I also reached out to the investors in these companies to ask them how the space has evolved, what's happening now and what the future of digital imaging will look like.
The evolution of digital imaging has had three revolutions so far, according to James Joaquin, co-founder and Managing Director at Obvious Ventures, who also co-founded Ofoto, an online photography service that was acquired by Kodak in 2001, becoming Kodak Gallery. The first one he identified goes back to the early days of Kodak. It's one that will be familiar to anyone who remembers the pre-digital camera days. This is when cameras still took film, and you had to take those rolls to someone to get them developed. Those physical photos would be put into albums, or be mailed out to relatives. The whole process was offline.
The second revolution came about with the release of the digital camera, and which precipitated the launch of companies like Ofoto, as well as Shutterfly and Snapfish, where users could suddenly share their photos and albums online. Suddenly taking photos, and sharing them, became a much, much easier experience with far less friction. That second revolution was relatively short-lived, however, especially compared to the first one, which lasted for around 100 years. Only a few years after changing photography forever, the digital camera was then supplanted by the emergence of an even newer technology just a few years later: the camera in your phone. "We are now in the third revolution, which is being driven by the smartphone. We're taking more pictures than ever before because we have a camera in our pocket at all times. That has given rise to great new services like Facebook, which was built on photo sharing, along with Snap and others," said Joaquin.
The smartphone has created a world where photos can be taken and shared with just a click of a button, allowing for more photos to be taken now than ever before. Deloitte Global predicted that 2.5 trillion photos would be shared or stored online in 2016, an increase of 15 percent from 2015. "The move to having a high-quality camera on your phone has upped the number of pictures that are accessible by 1000 percent. Before, a lot of pictures were trapped on a camera; that's where they’d stay, because there certainly wasn't a lot of sharing being done from the camera itself," said Matt Murphy, Managing Director at Menlo Ventures. There are so many photos now being taken, he told me, that the picture has even begun to replace words as the primary way that people communicate with each other. "With social media, and the ability to post and share wherever you want, the picture is now what you’re sharing rather than words. We socialize more with photos and selfies."
The proliferation of smartphones and all of the photos that are now being taken have, of course, created its own issues and problems. "The number of photos being taken has created a new problem, and it's one I see every day, which is photo fragmentation. With the explosion of capture devices, we use different ones for different occasions. You might use a digital SLR for your vacation, while kids have their smartphones, and you have an iPad. There are all these different devices, and that ends up creating photos that are fragmented into lots of different places," said Joaquin.
That is where the next revolution comes in, with new technologies that allow for easier searching and sharing thanks to advance in artificial intelligence and machine learning, which allow people to do more detailed, binary searches for their photos. That includes object recognition, as well as facial recognition and even the ability to detect emotions. So, say you're looking for a specific photo, of your sibling wearing a specific piece of clothing, and standing in a specific setting. Thanks to AI, all of those things will be keyworded and tagged automatically, rather than having to be done manually. This was not possible even a few years ago.
One of the companies that Murphy invested in is Clarifai, an artificial intelligence company that does visual recognition in photos. "Clarifai is good at tagging all the photos and all the videos. Rather than having humans needing to go through and write tags, now a machine built by Clarifai can recognize what’s in the photo, provide more tags, at a higher veracity, making all photos and video indexable. So, on a website like Hotels.com, if someone wants to find a hotel with white sand beaches, the site can search through all the hotels and tell you which ones have a white beach, or which have a mountain near a lake. All of these things in the images are tagged, and that’s why it’s so powerful to add that to search. A picture is worth 1000 words, and now we can unlock that and make it part of the search paradigm," said Murphy.
Joaquin, however, does not believe that being able to do that kind of search will resonate with the average consumer. "I'm a contrarian, in that I don't think that's meaningful to average consumer. Simplicity is more powerful," he said, pointing to a company like Apple, which has not had the same success with photos that it has had with music, simply because its photo solution tries to lock the consumer into only using Apple products. Meanwhile, most people have a slew of different devices that they want to use to access those same photos.
"A solution needs to work in a heterogeneous environment. It needs to work so my in-laws or my grandparents can figure it out. I don't think that grandma wants AI tagging. She wants something where she can easily access and share." Instead he pointed out two important innovations in how we are interacting with photos. One company that Obvious invested in is Joy, which offers consumers a digital photo album that they can display in their home, giving the average user a simple solution. While Joy is a startup, and has raised around $14 million in venture funding, some of the major corporations are doing something similar, including Samsung and Sony, which are also incorporating cameras into their consumer electronics, such as televisions.
"You'll be able to sit on the couch and relive your photos on your TV. There have been many failed attempts at making that a great experience. You should be able to bring your photo album up in your living room or your kitchen, but the challenge is how to make it simple and effortless. Until now it's been complicated, needing a TV remote and then having to navigate to the folder, so consumers said, 'I don't want this.' The simple way is to do it through voice control, so you can literally say 'vacation,' and the album will come up. That could be transforming," he said. "It's a fundamental human behavior, to want to share and connect with friends and family, so this would be a powerful tool."
One suprising entrant into the world of digital imaging is Amazon, which has begun to make photos a larger part of its business. The company introduced two photo products recently: the Echo Show, which is an Alexa speaker that contains a screen to view photos and videos, and the Echo Look, which uses its camera to take photos of user's clothing, and then recommend other things they might want to buy. Evan Nisselson, General Partner at LDV Capital, sees that as the beginning of a new trend in which all household objects will work on digital imaging.
"The Internet of Eyes will be exponentially larger than the Internet of Things. All inanimate objects will have at least one or more cameras on them. I wrote that a year and half ago, and then about four months ago that thesis was validated by the Amazon Echo Look coming out and having a camera on it. That is now starting this trend, which I'm calling the "war of the camera." When I say "camera," I'm not talking about a DSLR. Those are dead, except for the one percent of professional photographers in the world. It's mostly the cameras on phones, or on an inanimate object," he said.
"There will be cameras in your refrigerator, tracking what you're eating. There will be cameras in our cars, since autonomous cars cannot operate without cameras to see. There was a story recently about a photojournalist who made a photo essay with the cameras from her Volvo. I think that's exciting and it's not a surprise to me, but it's a surprise to a lot of people. So the value of a camera is exponentially expanding, either for creative content, or for e-commerce or for purely searching via Google Lens, or the Amazon Echo. Images, computer vision and artificial intelligence will augment search everywhere and this will decrease the need to search with keywords, and that's the direction where it's going. There's more and more places where cameras will be."
With artificial intelligence and machine learning already taking photos to the next level, where do they go from here? What will digital imaging look like 10 years from now. Sean Flynn, Managing Director and Partner at Shasta Ventures, believes that they will be used in almost every vertical, from e-commerce to healthcare. "The advancements in computer vision in the last five years have been astounding, as a result of what's happening in deep learning and neural networks. Computers, in many ways, are smarter and better at identifying objects than humans are, and that's really the first time in history that's been the case. What are the implications of that? I think we're going to see implications across healthcare, across consumer social networking and communications. You're going to see it across lots of different verticals and use cases. Once these devices that we have with us 24/7 in the palm of our hand move beyond just being able to capture, to having a sense of understanding, you're going to see lots of really innovative use cases come from that," he said.
In healthcare, for example, processing medical imaging can either replace radiologists altogether, or at least make them more effective. In dermatology, people are able to take photos of their skin problems over time to see if they are growing, and then send the picture to their doctor for an immediate response, rather than waiting for an appointment. The same thing is being done with wound care, where doctors and nurses are taking photos and uploading them directly to the patient's EMR. "There's interesting example of companies that will be able to diagnose potential medical issues. If you use baby photos, it can actually look at the orientation of a baby's eyes and face, and then determine, with the same accuracy or greater as a DNA test, whether or not that child is predisposed to having different types of diseases in the future. When you're no longer relying on humans, but you can rely on computers to make those types of judgments, you're going to see lots of really interesting stuff," Flynn said.
"There's a very, very long list of examples of how computer vision is going to dramatically change the world going forward." Nisselson sees many innovative ways that we will be taking photos on the consumer end, and a future where we will be able to take photos without even needing to carry around a camera.
"I can't wait for my satellite selfie. We're going to be able to grab a friend and say, 'Hey, join me for a selfie and look up,' click a button on your phone and a satellite is going to take your photo. That will definitely happen, it's just a matter of time and who will be the provider of that. I like minimalism. I don't like carrying around lot of things and I would be happy to avoid carrying a camera, and not use a horrible selfie stick, that's going to be great. People laugh but that's the reality and it's something that I've been talking about for years. Beyond that, years later, I think there’s no doubt we'll have a contact lens camera, a retina cam. There's already companies like Samsung and Google that have published patents that relate to having electronics and a sensor on a contact lens that we'll put on our eye. Obviously that brings up science fiction movies and books, but I don't think it’s too far off in the future," he told me.
"There's very interesting stuff that can come out of visual data. 90 percent of the data that our brains analyze is visual data. So, in order to succeed in delivering true AI, 90 percent of the data artificial intelligence will analyze will have to be visual. In order to do that, it will have to understand computer vision and machine learning to find the signal in that visual data, whether it's photos from our camera phone, or if it's photos from a car, or satellites. The ability to see these trends, and hopefully improve our lives, is our goal."