Posted on Leave a comment

Apple’s camera-toting Apple Watch band employs facial recognition tech for clear FaceTime calls

Apple is investigating methods of incorporating camera hardware into its Apple Watch product lineup, focusing specifically on systems that avoid the inherent pitfalls of embedding image capturing equipment in a device not well-suited to the task.

Apple Watch Camera

Source: USPTO

According to a patent granted to Apple by the U.S. Patent and Trademark Office on Tuesday, the tech giant is looking to broaden the feature set of its wearable by incorporating a novel camera system capable of automatically cropping in on subject matter, tracking objects like a user’s face and generating angle-adjusted avatars for FaceTime calls.

Apple’s U.S. Patent No. 10,129,503 for an “Image-capturing watch” details a hardware and software solution that makes a camera-toting Apple Watch not only feasible, but useful.

As noted in the property’s language, such designs would allow users to be less reliant on their iPhones, perhaps ditching the handset altogether in certain situations. With a camera-enabled Watch, users would be able to leave bulky handheld equipment behind when playing sports, working out or performing other strenuous activities.

A workable smartwatch camera solution is difficult to execute. Simply graphing a traditional handheld image capture system, like an iPhone camera, onto Apple Watch would be a user experience nightmare. Owners would be hard pressed to aim the camera at a subject without moving the entire platform — their wrist — to check framing, while at the same time dealing with a host of other concerns, from video stability to ergonomics.

To address the litany of anticipated issues, Apple proposes a system in which a wide-angle lens is fitted to a sensor built into a watch band. Alternatively, multiple cameras might simultaneously capture image data that is seamlessly stitched together to create one continuous image with a field of view that is much wider than any single-lens camera can provide.

The resulting wide-angle image reduces the need for precise aiming. Instead of framing subject matter beforehand, the final photo or video can be cropped after the fact to home in on a target or point of interest. This process can be conducted algorithmically or manually by the user.

Apple notes the technique not only works well for initial image acquisition — users need only point the lens in a target’s general direction — but also adds a new level of convenience not usually available to portable camera users. Crop-after-capture can assist in recording spur of the moment events, for example.

Apple Watch Camera

Illustration of teleconference call technique.

Beyond basic photography, the patent delves into teleconferencing. Here, too, Apple provides thoughtful context to the proposed technology.

In some embodiments, the watch processes a captured image and applies facial recognition software to recognize a user’s face. The device dynamically outputs a cropped image centered around the identified visage, making continuous on-the-fly adjustments to keep the face in frame even if the watch is in motion.

Perhaps most interesting is a contingency for rectifying unflattering “up nose” angles. If a user were to employ Watch in a FaceTime call, and do so without holding the device directly in front of their face as promised by Apple’s auto-cropping tech, the resulting image would at times be captured from a low vantage point.

To negate “up nose” shots, Apple proposes outputting a representation of a user’s face that appears to be taken from a straight-on perspective.

As noted in the patent, Watch’s onboard processor might generate an angle-corrected image or representation of a user’s face by compiling stored facial data, perhaps information gathered during biometric registration. Motion data captured by the camera and processed by the watch is then mapped onto the computer generated representation, which mimics a user’s facial expressions and movements in real time.

Alternatively, source motion data can be used to inform the movements of a non-human avatar like Apple’s Animoji and Memoji.

Apple Watch Camera

Illustration of automated cropping (top) and “up nose” angle correction.

Whether Apple intends to bring its Apple Watch camera band technology remains unknown. The company was in 2015 rumored to integrate a FaceTime camera into a future Apple Watch model, though the rumblings failed to bear fruit.

Apple’s camera watch band patent was first filed for in September 2016 and credits Seung Wook Kim and Megan A. McClain as its inventors.

Posted on Leave a comment

Apple Heart Study not used to gain FDA clearance for Apple Watch Series 4 ECG

 

The Apple Heart Study, conducted in partnership with Stanford Medicine, collected heart rate data from more than 400,000 Apple Watch users in its attempt to determine whether wearable devices can effectively detect irregular heart rhythms. Contrary to previous reports, however, the results were not used to gain clearance from the U.S. Food and Drug Administration for Apple Watch Series 4’s ECG feature.

Apple Watch ECG

The ECG app on Apple Watch Series 4.

Stanford revealed the scope of the Apple Heart Study in an announcement Thursday, saying the clinical trial was the largest screening study on atrial fibrillation ever conducted. A paper detailing the study and its design was published in the American Heart Journal today.

Apple closed enrollment in August, some eight months after it launched the program in 2017. Participants began receiving word that the study was complete in September, though Stanford says data collection will be completed early next year, in line with previous statements from Apple.

Following the closure to enrollment, Apple quietly submitted a de novo request for FDA clearance to two Apple Watch apps that would feature prominently in Apple Watch Series 4. The first app handles interpretation of and display of electrocardiogram readings from the wearable’s new ECG system, while a second uses optical sensors to identify irregular heart rhythms.

The FDA issued regulatory Class II clearances — over-the-counter access — for both.

Previous reports claimed Apple leveraged the Heart Study in both de novo requests, but Stanford says the trial was used only in respect to atrial fibrillation notifications.

The clarification makes sense, as the Heart Study related to atrial fibrillation, not ECG systems or data collection. During the months-long trial, a specialized app collected pulse rate data from Apple Watch Series 1, 2 and 3 hardware. In some cases, the app was able to detect and notify users of irregular pulse rate episodes.

The study, according to Stanford, sought to determine how many patients who received irregular pulse notifications were found to have atrial fibrillation, and how many went on to get medical attention. Calculating the accuracy of the system against simultaneous ECG recordings was a tertiary goal of the trial.

Though it did not factor into regulatory approval of Apple’s ECG solution as previously thought, the Heart Study can be considered an important first step toward providing consumers with easily accessible medical hardware.

“We now have access to high-quality sensors that can measure and detect changes in our bodies in entirely new and insightful ways without even needing to go to the doctor, but we need to rigorously evaluate them,” said Mintu Turakhia, MD, associate professor of cardiovascular medicine at Stanford. “There’s never really been a study like this done before.”

Apple is expected to activate the ECG feature in Apple Watch Series 4 later this year.

Posted on Leave a comment

Apple Heart Study data reportedly used to win FDA approval for Apple Watch ECG

 

The two FDA clearances that Apple announced Wednesday for the Apple Watch Series 4’s ECG capabilities came from data collected via the Apple Heart Study, according to a report Thursday.

Apple Watch warning

Apple announced Wednesday at its “Gather Round” event that the new Apple Watch Series 4 comes with the ability to take an electrocardiogram — the first Apple product to receive clearance from the FDA. Apple COO Jeff Williams said on stage that the company received the clearances for its ECG and atrial fibrillation testing on the Apple Watch via a “de novo” pathway, which means it supplied data to the agency to prove the product both worked and is safe.

Quartz reported Thursday, citing FDA documents, that the FDA used data from the Apple Heart Study in order to grant Apple those clearances. That study, conducted by Apple along with Stanford Health, launched last November and began winding down earlier this month.

The Watch’s abilities don’t actually mean much, according to one doctor.

Andrew Moore, an emergency department physician at the Oregon Health and Science University, told Quartz Thursday that the Series 4 doesn’t rise to the level of a medical device.

“The tech that Apple is working with is very rudimentary compared to what we’d do for someone in a hospital or health care setting,” Moore told the site. “The ECG thing is a little bit overhyped in terms of what it will really provide.”

Apple has never quite claimed that this Apple Watch, or any other product it makes, is meant to serve as a substitute for full-fledged medical devices or professional medical attention.

It says right there on the Apple Watch, that if the Watch detects atrial fibrillation (AFib), “you should talk to your doctor.” At the same time, Apple’s Williams admitted that the Series 4 won’t always catch AFib every time.

Posted on Leave a comment

What it was like to give up my Apple Watch after three years of constant use

After three years of daily use, I’ve decided to give up my Apple Watch for a week to prove to myself how useful and important it really is —or maybe, it isn’t.

[embedded content]

I’ve been using Apple’s wearable since it debuted and for the most part, I’ve been a vocal fan.

Soon, Apple is expected to debut their latest ‘Series 4’ Apple Watch. Before I jump in head-first and upgrade yet again, I wanted to test myself and see if I truly did need my Apple Watch. Is this something I can’t live without, or have I just fallen for my own hype?

Before I ditched the watch for a week, there were a few things I wanted to sort out. What did I use my watch for most, and what — if anything — would be the replacement.

Considering my replacement

After closely studying my habits, there are a few things I identified as relying on most frequently.

I have a bad history of not paying attention to my phone, and my Apple Watch notifications help make sure I don’t miss anything. So, that’s high on the list.

The Apple Watch has truly helped me stay active. Whether it is making sure I’m not lounging too long on the couch or making sure I head to the gym on a regular interval. Each day I make an honest effort to fill my rings.

Possibly most importantly, I’m a Type 1 diabetic and use my Dexcom CGM to view my glucose levels on my Apple Watch. It is useful to see on my watch, but luckily I can still view it directly on my phone.

There are plenty of other ancillary tasks I rely on my Apple Watch for such as Siri, hands-free phone calls, heart rate monitoring, HomeKit control, Wallet/Apple Pay, quick access to weather, and more. Though these are all to a lesser extent than the three above.

I considered a lot of options for my possible Apple Watch replacement, and I came to the conclusion that I couldn’t ditch a wearable completely. I still needed something — however minimal — to help with at least a couple of my frequent tasks.

Mi Band 2

That’s why when I shed my Apple Watch, I went with a super cheap fitness tracker that I had reviewed in the past — the Mi Band 2. It has a surprisingly large number of features and I was able to pick it up for less than $30 bucks.

Starting the week

Heading into the week, I was getting along just fine with the Mi Band 2. I got notifications for social media, texts, and phone calls. Idle alerts made sure I never sat around too long. It was also easily capable of tracking steps, calories, distance, and workouts throughout the day. It also didn’t need charged the entire week, though I plugged it in once just to top it off.

My iPhone and HomePod served as suitable replacements for many frequent tasks. Granted, it was ever so slightly less convenient, but I was starting to gain confidence in my ability to forgo Apple’s wearable.

Comparing the heart rate tracking to my Polar heart monitor also was confidence boosting, with the numbers falling very close to one another and appearing very accurate for such a small and cheap band.

Eventually, though, things started to turn.

Missing my old friend

Around day three I started to really yearn for my Apple Watch.

Without my Apple Watch I found myself using my iPhone more than in the past. The new Screen Time digital health features in iOS 12 even confirmed that. Since I had to pull my iPhone out for so many tasks, I ended up diving into other apps and wasting more time there than I normally would have.

I also missed my Siri Watch face. Sports are just kicking into gear and I missed being able to glance down at the watchOS 5 Siri face and see the score of the Buckeyes, Browns, or Indians game without having to actually launch any apps. I miss having my frequent HomeKit scenes presented before I need to think about them.

Leaving to take the dogs for a run/walk was also a bummer. I love heading out, locking the door with my Apple Watch, turning on my Apple Music playlist with my AirPods, and starting a workout. Cellular connectivity means I also am able to get security alerts from HomeKit or my security cameras while I’m away. Mi Band wasn’t up for any of this.

I went without music once and another time I tried to bring my iPhone but it was super uncomfortable to keep in my workout shorts while running. When I was without my phone, I had a bit of a sense of worry knowing I couldn’t be reached and that I couldn’t watch after my home.

Paying for things was more difficult having to always pull my phone out. When going through the drive-through at Starbucks, I had to unplug my iPhone from CarPlay to scan my Starbucks card rather than being able to have it scanned off my Apple Watch. Minor annoyances, but they add up and ultimately what gives Apple Watch the better user experience.

Workouts also provided much less information and Mi Band lacked any of the social aspects found in the Apple Watch. I’ve several friends using Apple Watch where we compete and challenge one another in our workouts and the Mi Band made me feel a bit left out, and less motivated.

Apple Watch unlock Mac

Another thing I noticed right away was that the Apple Watch is a great way to unlock your Mac. It is almost instant whereas even using Touch ID feels like it lags. I forgot about this feature until I no longer had it to rely on.

Choosing one or the other

Apple Watch is a personal device. Everyone uses it differently. On the surface, it seemed I could easily replace the Apple Watch with a cheaper, more minimalistic device and get the same outcome. But Apple’s blend of features and excellent design make that a lot harder to pull off than I imagined.

The Mi Band 2 was a great device. It was a fantastic fitness tracker that is significantly more cost effective than Apple Watch or even a Fitbit. I actually preferred the Mi Band 2 over any of the Fitbit models I’ve used in the past.

Sleep tracking was also something that I’ve been very interested in and have yet to find an excellent option with the Apple Watch. I’ve used Pillow, Auto Track, SleepScore (and SleepScore Max) and others but it was nice that the Mi Band 2 had this automatically built in.

Apple’s Health app syncs with Mi Band so I was able to somewhat keep track of my activity versus others, but it wasn’t quite the same.

There is one area that I really loved when not having my Apple Watch. I had the opportunity once more to wear other watches. I’ve quite a few watches I’ve gotten over the years myself or as gifts and it was a struggle for me to keep my Apple Watch activity streak and to get the chance to sport one of my other timepieces.

Looking to the future

Apple Watch vs Mi Band 2

I always knew I was going back to my Apple Watch. I haven’t worn it nearly every day for the past three years for fun.

I’m still glad I tried this experiment, though. It reinforced my love of my Apple Watch and helped me put everything into perspective. What I love is that I don’t have to force the Apple Watch into my life. It fits and offers me many things from peace of mind, to more workout notification, and lots of convenience.

Apple Watch isn’t just a simple fitness tracker, but it unlocks your computer, controls your home, finds your phone, acts as a flashlight, and so much more.

Right now, the Series 4 Apple Watch looks great. I’m very excited to try out the new complication-dense watch face that got leaked ahead of time because glanceable information is some of the most useful.

I can’t say with certainty I will be upgrading to the Series 4, but I can say without a doubt I love my Apple Watch.