How I use Siri mode in the iOS 27 Camera app to ask questions about anything I see

Follow ZDNET: Add us as a preferred source on Google.

When using my iPhone's camera, I'll sometimes see an animal, plant, landmark, painting, or other object that strikes my curiosity. In those moments, I want a quick way to get information on the item. Well, that's something I can now do easily thanks to the new Siri mode in the iOS 27 Camera app.

Also: I put Siri AI through the same tests I use for ChatGPT and Gemini on MacOS 27 - here's how it did

Currently available in the iOS 27 developer beta, Siri mode is another addition to the camera modes, alongside Photo, Video, Portrait, and Pano. When you see something through the camera, just tap the Siri mode icon. From there, you can ask Siri about it by voice, run a Google Search on it, or send an image of the object directly to Siri for analysis.

What's the big Siri improvement in iOS 27?

If this sounds like the Visual Intelligence feature introduced in iOS 18.2, that's because it is. But here's the difference. With the current version of iOS on a supported phone, you have to trigger Visual Intelligence separately. That means you must exit the Camera app and then launch the feature, a clumsy process. With iOS 27, you can stay in the Camera app and kick off Siri without losing sight of the object. That's a big improvement, and one I appreciate.

As with any photo, you can zoom in or out in Siri mode to capture as much or as little of the object as you want. Depending on the object, Siri may display a name identifying the item. Tapping the shutter button then captures the image for Siri to further analyze. Tapping the picture icon on the right runs a Google search on the photo to show you similar images. Tapping the Ask Siri icon on the left opens a prompt with a thumbnail of the image, so you can type or dictate a question about it.

Also: Will your iPhone support Siri AI? The answer is complicated

Other types of information in the photo will display additional options. Aim your camera at a date, time, and place, and Siri may offer to add it to your calendar. Aim it at a menu, sign, or other text in a different language, and Siri could offer to translate it.

What do you need for this to work?

First, you need an iPhone that supports Apple Intelligence and the new Siri AI. That includes the iPhone 15 Pro, the iPhone 15 Pro Max, or any model in the iPhone 16 or 17 lineup.

Second, you need to download and install the iOS 27 developer beta. For that, I strongly urge you to use a spare or test iPhone and not your main or go-to device. Developer betas can be unstable, and you don't want to risk any adverse effects on your primary iPhone.

Third, you need to join the Siri AI waitlist. And waitlist is the right term, as you may need to wait quite a while before Apple grants you access. After I joined the list, I had to wait almost a week before Siri AI popped up on my phone.

With iOS 27 and Siri AI on my test iPhone 15 Pro, I've been using the new Siri AI mode with my camera. Here's how.

Here, I aimed my camera at a pigeon I saw on the street. Tapping Siri mode in the Camera app identified it as a rock dove. I then tapped the shutter button, and Siri expanded on the description. I asked Siri if these birds are common in New York City. In response, Siri told me that they are common in New York and that tall buildings provide an ideal alternative to their natural cliffside habitats.

Here, I captured the Statue of Liberty through the camera. After I pressed the shutter button, Siri quickly identified it. I followed up by asking Siri how it was brought to the US, and it explained how the statue was disassembled, shipped across the Atlantic, and then reassembled on what was then called Bedloe's Island, now known as Liberty Island.

Next, I asked Siri to identify some artwork at the Metropolitan Museum of Art. Here, the initial results were disappointing. Siri misidentified two paintings -- one by Mary Cassatt and another by Pierre-Auguste Renoir. In each case, I told Siri that it was wrong. It was then able to correctly identify the Cassatt painting but told me it couldn't confidently identify the Renoir painting.

The third time, I showed it a painting by Vincent Van Gogh, and this time it gave me the correct name of the artist and the painting right off the bat. I then tapped the button to run a Google search, and the resulting page showed me links to websites where I could buy a print of the painting.

Next, I aimed the phone at a page in a booklet from the Metropolitan Opera announcing upcoming operas for the next season. Specifically, I pointed at a page about Mozart's opera The Magic Flute. Here, I tapped the Ask Siri button and asked Siri for a synopsis of the opera. In response, Siri described the plot, the main characters, and the highlights.

Next, I aimed the camera at a menu written in Chinese. Siri briefly described Chinese restaurant menus in general. But then I tapped the Translate button at which point the menu appeared in English. The items on the menu appeared too small to read, and I couldn't zoom in on the image. But by taking a screenshot, I was able to open the shot in the Photos app and zoom in to better read the text.

Next, I captured a shot of the outside of the Museum of the City of New York. After Siri gave me a general description of the museum, I asked for its location and hours. Siri then gave me the address and the days and hours when it's open.

Finally, I pointed the camera at a writing event in Brooklyn that included its dates. The Siri screen displayed a button to add it to my calendar. After doing that, the event appeared in my calendar with the appropriate date, time, and other details.

Since Siri AI is still in the early beta stages, I did bump into a few problems. In some cases, Siri couldn't identify or incorrectly identified the object. In other cases, the AI took a long time to respond or even gave up and told me to try again later. But hopefully these bugs will be fixed by the time the new Siri makes its official debut with iOS 27 in September.

View original source — ZDNet ↗

ShareShare on X Share on Facebook