Apple's Smart Sensor Technologies -- market research report (sample)

www.memsjournal.com | info@memsjournal.com | Copyright 2015 MEMS Journal, Inc. | All rights reserved. | Proprietary and confidential.
TECHNOLOGIES
APPLE’S
SMART SENSOR
Market Research Report
August 2015
Sample Version

2
Apple’s Smart Sensor Technologies | Market Research Report | mr@memsjournal.com
Copyright 2015 MEMS Journal, Inc. | All rights reserved. | Proprietary and confidential. | www.memsjournal.com
Confidentiality and Disclaimer Notice
This document and its associated attachments contain information that is confidential and
proprietary to MEMS Journal, Inc. If you are not the intended recipient of this document,
please notify MEMS Journal immediately by email or telephone, delete the original
document, emails and attachment files from your computer system and destroy any hard
copies of this document and its supplements. Be aware that any disclosure, copying,
distribution, or use of this file or its accompanying attachments (if any) with any parties is
prohibited. The content of this document is for information purposes only and is not
intended to provide tax, legal or investment advice. You should not fully rely on any material
contained in this document and should seek independent advice wherever necessary. Any
decisions you make based upon any information contained in this document are your sole
responsibility. The information does not constitute a solicitation of any order to buy or sell
any securities.
For further questions, please contact us at info@memsjournal.com.

3
Table of Contents
I. Executive Summary ………………………………………………….……….…
II. Consumer Electronics Smart Sensor Landscape …………..………
1. Smart Sensor Applications and Evolution
2. Market Size and Growth
3. Top Buyers and Suppliers
4. Market Trends and Forces
i. Consumer Demand
ii. Technology Advances and Challenges
iii. Shift to Software
iv. Data Analytics and Storage
v. Privacy and Security
III.Apple Smart Sensor Use ……………………………………..……………….
1. Product and Sensor Roadmap
2. iPhone and iPad Smart Sensors (Historic and Current)
3. Expected Apple Watch Smart Sensors
4. Sensor Suppliers
5. Sensor Hardware and Software Deep Dive
i. Accelerometer
ii. Gyroscope
iii. 6-axis Combo Motion (IMU)
iv. Magnetometer
v. Pressure Sensor
vi. Microphones
vii. Fingerprint Sensor
viii.Ambient Light and Proximity Sensors
ix. Hall Effect (Smart Cover) Sensor
x. Heart Rate Sensor
xi. Force Sensor
6. Sensor Hub and Sensor Fusion Deep Dive
i. Motion Co-processor Hub
ii. Core Motion and Core Location Software
iii. Watch Software
IV.Apple Emerging Smart Sensor Technology ………..…………………
1. Areas of Sensor Development
2. Rumored Next Generation iPhone and Watch Sensors
3. Emerging Sensor Technology Deep Dive
i. Health/Fitness Sensors and Wearables
ii. Enhanced Security Systems
iii. Sound and Image Capturing
iv. User Interface and Accessories
v. Networking
vi. Positioning and Navigation
vii. Device Management and Protection
viii.Smart Home
5
14
36
78
V. Apple Smart Sensor Ecosystem ………..…………..…………………..
1. Internet of Things Platform Overview
2. Apple Internet of Things Reputation
3. HealthKit Deep Dive
4. HomeKit Deep Dive
5. Software-Focused Approach
6. Privacy and Security
7. Regulations
8. Data Analytics and Storage
VI.Third-Party Accessories …..…………..………..…………..…………….
1. Smartphone/Tablet Accessory Landscape
2. Types of Sensors in Accessories
3. Health/Medical Accessories
4. Fitness/Sports Accessories
5. Smart Home Accessories
6. Lifestyle Accessories
7. Environmental Monitoring Accessories
8. Gaming/User Interface Accessories
VII.Competitor Smart Sensor Use …..………..………..………..……….
1. Smartphone and Tablet Competitors and Market Share
2. Overview of Sensor Use by Competitor
3. Wearables Competitors and Market Share
4. Wearables Sensors and Use by Competitor
5. Health Platforms by Competitor
6. Smart Home/IoT Sensors and Use by Competitor
7. Smart Home Platforms by Competitor
8. Competitor Sensor and Platform Deep Dive
i. Samsung
ii. Google
iii. Microsoft
iv. Blackberry
v. Xiaomi
vi. Other Chinese Manufacturers
Appendix: Apple Sensor Project Details …..………..………..……….
About Us: MEMS Journal Market Intelligence Services ..……….
118
134
147
182
199

4
List of Figures
1. Consumer Electronics Smart Sensor Applications
2. Smart Sensor Development in Consumer Electronics
3. Global MEMS Unit Shipments (Consumer Electronics)
4. Sensor Market Size and Forecast (Smartphones & Tablets)
5. Sensor Market Size and Forecast (Wearables)
6. Top Buyers of Consumer Electronics MEMS
7. Top Suppliers of Phone & Tablet MEMS
8. Global Smartphone and Tablet Market Size
9. Global Wearables Market Size by Device Category
10. Global Internet Device Installed Base Forecast
11. MEMS Accelerometers Cost and Size Reductions
12. Sensor Hub Market Size and Forecast
13. Examples of Sensor Hub Architectures
14. Market Value Repartition in IoT Structure
15. Big Data Size Reference
16. Timeline of Apple’s Product and Sensor Milestones
17. iPhone Sensors by Model
18. Snapshot of iPhone6 Sensors and Suppliers
19. iPad Sensors by Model
20. Snapshot of iPad Air 2 Sensors and Suppliers
21. Snapshot of Expected Apple Watch Sensors
22. Snapshot of iPhone Sensor Suppliers Over Time
23. Apple MEMS Microphone Primary Suppliers by Product
24. Knowles MEMS Microphone Market Share
25. Apple Motion Co-processor Sensor Hub Architecture
26. Apple Motion Co-Processor Evolution
27. Examples of Potential Future Apple Sensors
28. Hypothesized Future Sensor Technology by Product Line
29. Timeline of Sensor-Related Emerging Technology
30. Rumored iPhone 7 Sensor Additions
17
18
19
20
21
22
23
26
27
28
29
30
31
33
34
41
42
43
44
45
46
47
49
50
72
74
82
83
84
85
31. Apple Internet of Things Platforms and Product Synergy
32. Timeline of Apple’s Platform Development
33. Most Influential Internet of Things Companies
34. Smartphone/Tablet Accessory Landscape
35. Types of Sensors in Third-Party Accessories
36. Handset Market Share
37. Handset Profit Share
38. Tablet Market Share
39. Overview of Smart Sensors by Competitor
40. Smart Sensor Hardware by Competitor
41. Wearable Device Release Timeline
42. Global Smartwatch Market Share
43. Global Wearable Bands Market Share
44. Examples of Wearables Sensors
45. Leading Smartwatch Sensor Comparison
46. Wearables (Excluding Smartwatch) Sensor Comparison
47. Health Platforms by Competitor
48. Examples of Smart Home/Internet of Things Sensors
49. Competitor Smart Home Sensor Comparison
50. Smart Home Platforms by Competitor
51. Samsung Galaxy S5 Sensors (Compared to iPhone)
52. Samsung Simband
53. Samsung Smart Home
54. Google Smartphone Sensors (Compare to Apple)
55. Google Moto 360 Sensors
56. Microsoft Smartphone & Tablet Sensors (Compared Apple)
57. Microsoft Band Sensors
58. Blackberry Smartphone Sensors (Compared to Apple)
59. Xiaomi Smartphone & Tablet Sensors (Compared Apple)
60. Consumption of Motion Sensors and Microphones by OEM
61. China Semiconductor Industry Revenue
122
123
124
136
137
152
152
153
154
155
156
157
157
158
159
160
161
162
163
164
165
168
169
170
172
174
176
177
178
180
181

5
Get ahead of the competition -- order the full version!
Please click here to order
the enterprise license for the
full version of this report.
For further questions,
please contact us at
info@memsjournal.com.

66
VII Competitor Smart Sensor Use
VI Third-Party Accessories
I Executive Summary
II Consumer Electronics Smart Sensor Landscape
III Apple Smart Sensor Use
IV Apple Emerging Smart Sensor Technology
V Apple Smart Sensor Ecosystem

14
Apple’s competitors in wearables and smart home use a wide
variety of smart sensors
• In wearables, Apple has introduced its Watch and now faces competition from a wider range of
companies including Samsung, Google, Microsoft, LG, Sony, Fitbit, and Jawbone; Samsung, Google, and
Microsoft have also launched health/wearables software platforms that compete with Apple’s
HealthKit
–Common biometric sensors in wearables include heart rate, galvanic skin response, bio-impedance,
and skin temperature sensors; most wearables also have basic motion sensors, and smartwatches
have microphones and other user interface sensors
–The Microsoft Band has the most sensors of any major manufacturer’s wearable and also has a
GPS, making it more functional as a stand-alone tracker without a phone
–Samsung has a line of wearables including a smartwatch, headphones, and virtual reality glasses and
Google offers a smartwatch, all with basic biometric and motion sensors
–Xiaomi offers a band with only basic functionality but very low price and long battery life that make
it distinct
• In the smart home space, Samsung and Google have platforms that compete with Apple’s HomeKit
–Common smart home sensors in platform accessories include occupancy, ultrasonic, open/close,
temperature/humidity, and ambient light and sound sensors; communication systems like WiFi
also play an important role in connecting devices and enabling presence sensing
–Google acquired leading smart home company Nest as its platform foundation; Nest devices use
occupancy, ultrasonic, temperature/humidity, smoke and carbon dioxide sensors among others
–Samsung acquired software-focused smart home company SmartThings as its platform foundation;
SmartThings makes a hub and a range of compatible devices that use vibration, open/close, and
temperature sensors, among others, and use communications systems for presence sensing
–Xiaomi is also entering the smart home space with a platform and range of smart home hardware

20
The variety of smart sensors available for consumer
electronics has exploded in recent years
Sources: This little motion sensor went to the market..., Sonja Thompson, IT News Digest, March 22, 2007; Willie D. Jones, IEEE Spectrum, A Compass in Every Smartphone, January 29,
2010; Consumers boost MEMS combo sensors, Electronic Product Design and Test, March 19, 2014; Samsung Turns up the Pressure on Competition with Pressure Sensor in Galaxy S4,
IHS, March 20, 2013; Behind the sixth sense of smartphones: the Snapdragon processor sensor engine, Qualcomm, April 24, 2014; MEMS for Cell Phones & Tablets, Yole
Developpement, May 2012; Fairchild, Emergence of a $Trillion MEMS Sensor Market, SensorCon, 2012; MEMS Microphone Market Tops 2 Billion Units, Mobile Dev Design, March 4, 2013
2005 2015+
Accelerometer
(Samsung
SCH-S310
possibly 1st
application)
Gyroscope
(iPhone4)
2007 2010 2014
Accel/
Magneto combo
Accel/
Gyro combo
2011
Pressure/Barometer
(Samsung Galaxy)
2012
Temperature
Humidity
(Samsung
Galaxy)
2013
Hall Effect
(Samsung Galaxy)
2006 2008 2009
Magnetometer
(iPhone3)
Heart rate
(Samsung
Galaxy)
Fingerprint
(iPhone 5s 1st major use)
Emerging tech:
•Force
•Pulse
•Gas/chemical
•UV
•Thermal imaging
•Etc.
RGB
(iPhone4)
RGB, proximity,
gesture
(Samsung
Galaxy)
Smart Sensor Development in Consumer Electronics
9-axis motion
combo
Apple iPhone introduced
with 3 sensors
Samsung Galaxy has more than 10
sensors, most of any phone on market
High performance
MEMS microphones
(iPhone4)
Proximity
Ambient Light
(Original
iPhone)

24
Apple and Samsung dominate the market for smart sensors in
consumer electronics
Note: Data from 2011 but trend of Apple and Samsung dominating the market continues today
Sources: Apple and Samsung Are Top Consumer & Mobile MEMS Buyers, IHS, August 13, 2012; Apple and Samsung Drive Adoption of Next-Generation Sensors, IHS, October 23, 2014
0
100
200
300
400
500
600
Expenditures[US$Billions]
Top Buyers of Consumer Electronics MEMS (2011)Apple
Samsung
LG
Electronics
Song
Nintendo
Nokia
HTC
Motorola
Canon
Sony
Ericsson
• Apple and Samsung have led adoption of
sensors in consumer electronics; recent
developments include:
–Apple initiated the mainstream market
for finger print sensors in the iPhone5s
–Samsung pioneered health (pulse) and
environmental (humidity, UV) sensors
in recent models
• As of 2011, Apple and Samsung
accounted for 60% of value in the
consumer electronics MEMS market; their
share was projected to reach 67% by 2016
• Market dominance gives Apple and
Samsung purchasing power, and in the
past they have aggressively negotiated
lower prices on components like motion
sensors than other handset makers

26
A variety of trends and forcing are impacting the smart sensor
industry and its players
Consumer
Demand
Technology
Advances
Technology
Challenges
Shift to
Software
Data Analytics
& Storage
Privacy &
Security
Consumer demand
for mobile device
functionality &
interconnectedness
is driving the smart
sensor market. Key
trends include
demand for more
smartphones and
more smartphone
features, growth in
wearables, and
emergence of the
Internet of Things.
Improvements in
sensor technology
are enabling use of
more sensors in
consumer
electronics. Key
improvements
include reduced
sensor size and
cost, development
of sensor hubs and
fusion, and reduced
data processing
costs.
There are still
technological
challenges to
overcome to meet
market demand for
smart sensors. Size,
cost, and power
consumption need
to decrease further,
intelligence needs
to improve, and
R&D needs to
become more
streamlined.
The proliferation of
sensors and
emergence of the
Internet of Things
will shift the focus
and value in the
industry from
hardware to
software and value-
added services. This
will impact value
chain dynamics and
may require
existing players to
adjust their
business models.
The large volumes
of data being
produced by
sensors poses
unique challenges
for analytics and
storage. Big data
analytics
approaches will
become important,
and there may be a
shift towards more
processing on
sensors to reduce
the data that has to
be transmitted and
stored.
Privacy, security,
and regulatory
issues also pose a
challenge to smart
sensor and related
Internet of Things
markets.
Approaches and
standards for data
ownership and use
will have to be
created and more
oversight from
regulatory agencies
may become the
norm.
Sources: See following pages

27
Consumer demand for device functionality is driving the smart
sensor market
Sources: Emergence of Trillion Sensors Movement, Fairchild, IEEE MEMS, January 22, 2014; Sensors for cellphones and tablets, Yole Developpement, 2014; Wearable Sensor Market to
Expand Sevenfold in Five Years, October 16, 2014; The Internet of Things: Sizing up the opportunity, McKinsey, December 2014; MEMS: How smart devices and wearables will keep
getting smarter, Francine Hardaway, Phoenix Business Journal, November 12, 2014
Smartphones and tablets
• Demand for smartphones and tablets continues to increase
• Additionally, consumers are using these devices to do more
(communication, entertainment, business, fitness, etc. ) and
have higher expectations in terms of functionality
• There will be demand for more sensors
–One estimate suggests the IoT will
require 1 trillion sensors
–MEMS are projected to grow 10x
further because of the IoT
• There will also be demand for new types
of sensors as well as combo sensors and
other integrated solutions
• There will be need for more advanced
sensor algorithms and need for closer
integration between hardware and
software
• There will be a need for solutions
(analytics, storage, etc.) to address the
data generated by the rapidly expanding
installed base of sensors
Trends Impact
Wearables
• The wearables market is gaining traction as consumers look
for more advanced fitness, wellness, and health monitoring
• Device manufacturers are responding with new products,
including sophisticated multipurpose devices
Internet of Things (IoT)
• The emerging Internet of Things category is connecting
sensor-laden devices through network architecture
• Wearables and home monitoring (often with a smartphone
or tablet as a hub) are key verticals in consumer electronics
• It is expected that 25-35 million objects will be connected to
the Internet of Things by 2020

29
The wearables category is growing rapidly as new types of
products are introduced and adopted
Global Wearables Market Size by Device Category
Note: Data is for developed markets
Sources: The smart wearables market will be worth USD22.9 billion worldwide by 2020, Analysis Mason, September 16, 2014; Wearable sensor market to expand sevenfold in 5 years,
MEMS Journal, October 16, 2014
0
20
40
60
80
100
120
140
160
180
200
2013 2014 2015 2016 2017 2018 2019 2020
Shipments[Millions]
Smart wristbands
Smart watches
Smart glasses
Smart clothing/accessories
Other
Wearables will incorporate sensors that are already utilized by smartphones today,
including motion and user interface sensors, and expand into new and emerging sensor
types such as bio and chemical sensors
Forecast

33
There are several sensor hub architectures in use, and they
continue to evolve
External hub, typically a
dedicated microcontroller
(MCU); this is the dominant
architecture in consumer
electronics applications
today and will likely remain
so in the near term due to
it’s superior low power
performance
Low-power sensor hub
incorporated in an
application processor; m ay
overtake the dedicated
MCU architecture by 2016.
Combo of one or more sensors
(typically accelerometer and
gyroscope) and low-power
processor
Images from EE Catelog, Qualcomm, InvenSense
Sources: Sensor Hubs Make Big Gains While Enabling the Future of Sensing, IHS, April 16, 2014; Atmel, Qualcomm, NXP Lead Booming Sensor Hub Market, IHS, April 21, 2014
Qualcomm and InvenSense websites
• Manufacturers include:
–InvenSense
–STMicroelectronics
–Bosch
• Device users include:
–Apple (6-axis IMU - InvenSense)
–Qualcomm (29% share
sensor hubs
–Intel
–Nvidia
–Atmel (32% share sensor hubs)
–NXP Semiconductor (24% share
sensor hubs)
–STMicroelectronics
• Device users include:
–Apple (M8 co-processor - NXP)
–Samsung
–Motorola/Google
1
2
3
Examples of Sensor Hub Architectures

38

45
New iPhone generations have always come with new sensors
Sources: Driven by Apple and Samsung, Light Sensors Achieve Double-Digit Revenue Growth, IHS, June 30, 2013; MEMS: Looking back at 2014 and 5 years outlook, IHS, November
2014; Light and Proximity Sensors - A Market Ready for Explosive Growth, Tony Rizzo, Mobility TechZone, July 30, 2013; iPhone 6 Teardown, iFixit, 2014; Apple 3G iPhone Teardown
Report, Portelligent, 2008; MEMS Microphone Market Tops 2 Billion Units, Mobile Dev Design, March 4, 2013
2007 2008/9
iPhone 1st
Generation
iPhone 3
2010/11
iPhone 4
2012/13
Magnetometer
(3GS)
RGB/proximity
combo (4)
Magnetometer
High performance
Microphone
iPhone 5
Fingerprint (5s)
Sensor Hub (5s)
6-axis accel + gyro IMU
Accelerometer (discrete)
Proximity
Ambient Light
Magnetometer
Microphone
iPhone 6/
6Plus
2014
Fingerprint
Pressure
Magnetometer
Microphone
GyroscopeGyroscope
Accelerometer
Proximity
Ambient Light
Accelerometer
Proximity
Ambient Light
Accelerometer
Proximity
Ambient Light
Sensor Hub
Accelerometer
Proximity (4s)
Ambient Light (4s)
iPhone Sensors by Model

52
Apple’s microphone supplier lineup has seen many changes
Sources: Teardown of the Apple iPhone 4 Smart Phone, Chipworks, June 23, 2014; iPhone 5 MEMS Microphones, PR Newswire, February 5, 2014; MEMS Microphone 2014, Yole
Developpement, 2014 ; In Complex Supply Chain, Knowles and Infineon are Top Suppliers of Packaged MEMS Microphones and Bare Microphone Dies, IHS, June 18, 2014; Top MEMS
Microphone Suppliers All Can Count on Apple for Clear and Resounding Success, IHS, April 29, 2013; MEMS: Looking back at 2014 and 5 years outlook, IHS, November 2014
iPhone 4 iPhone 5 iPhone 6/6PlusiPhone 5s
iPad 2 iPad Mini iPad Air & Mini 2/3
It has been suggested Apple (and other device makers), use multiple microphone suppliers
to ensure they have a reliable supply; it likely also serves to keep costs lower
Apple Microphone Primary Suppliers by Product
Knowles expected
starting Q2 2015 –
see following slides

66
A fingerprint sensor is behind the Touch ID feature in the
iPhone and iPad
Images from Apple and iFixit
Sources: iPhone 5s teardown, iFixit, 2013; iPhone 5S Fingerprint Scanner Details Surface; Sweaty Fingers Not Allowed, Daily Tech, September 16, 2014; How the iPhone 5S Fingerprint
Scanner Works—And What It Means For You, Andrew Tarantola, Gizmodo, September 10, 2013; Apple preparing software update to enhance functionality of iPhone 5s Touch ID, Daniel
Eran Dilger, Apple Insider, February 28, 2014
• What it is:
–CMOS chip using sensitive capacitance method
–Chip has an array of minuscule capacitive cells, each less
than a finger ridge wide, that consist of two conductor
plates separated by an insulating layer
• How it works:
–Senses fingerprint ridges and valleys and combines the
data to create an image of the print
–Apple says the sensor searches for vitality signs to ensure
that the finger is attached to the hand of a living person
–The technology is generally recognized as being leading
edge, but with some potential issues:
The capacitive technology offers advantages in image quality and in the coatings that the
sensor can image through as compared to optical, thermal and other solutions
There are questions about the durability; because the sensor insulation layer has to
endure sweat, sebum, water, and dirt, there is expected to be decay over time
There are some inherent issues with fingerprint recognition like tissue damage or wet
conditions that could make a fingerprint difficult to recognize or completely unreadable

72
Optical sensors for heart rate monitoring are expected in the
Apple Watch
• What it is:
–According to Apple, the heart rate monitor will include optical
sensors that use infrared and visible-light LEDs and photodiodes
• How it works:
–The system is rumored to include a photoplethysmogram (PPG)
sensor that tracks changes in blood-flow by shining a light on the
skin and measuring how it scatters off blood vessels
–The system may leverage other sensors in the Watch and iPhone
It will likely work with the accelerometer in the Watch and GPS
and WiFi on the iPhone to track activity and calorie burn
Image from Apple
Sources: Apple Watch's Wristful of Sensors and MEMS, Tekla Perry, IEEE Spectrum, September 9, 2014; The Apple Watch May Solve the Usual Smart-Watch Annoyances, Rachel Metz,
MIT Technology Review, September 9, 2014; Next-generation Apple Watch rumored to boast more sensors, fitness capabilities, Mikey Campbell, Apple Insider, September 11, 2014;
Apple Watch coming this April: What you need to know about Apple's first smartwatch, CNET, February 6, 2015; Apple website
There are apps that use the iPhone camera to calculate heart rate through changes in
facial skin color, and it has been suggested that this technology might be incorporated
–It is expected that the heart rate monitoring in the Watch will activate only during workouts,
as opposed to the 24-hour tracking of other devices (e.g. Microsoft Band)
–There is some question as to how accurate the heart rate sensing will be
The top of the wrist is not as good a location as the bottom for measuring heart rate
The wrist in general is hard place to accurately gauge signals because there is noise from
motions unrelated to overall activity (other sensors in the Watch and iPhone may help,
and it is theorized that Apple has a way of using visible light to improve reliability)

75
The latest iPhones and iPads use a sensor hub for always-on
control and monitoring of motion sensors
• What it is: The M8 motion co-processor is a dedicated chip for collecting and processing the
outputs from the accelerometer, 6-axis IMU, magnetometer, and pressure sensors
• How it works: The M8 provides always-on functionality for the motion sensors
–In the past, motion sensors were kept off when the device was asleep to avoid tying up
the main processor and draining the battery
–The M8 operates separately from the main processor and at much lower power levels,
allowing the sensors to stay active and be analyzed all the time
–When the device is brought out of sleep, the M8 can feed the gathered information to the
processor and, in turn, to compatible apps
Images from Apple and sources noted on individual sensor pages
Sources: What Apple’s M7 Motion-Sensing Chip Could Do, David Talbot, MIT Technology Review, September 25, 2013; Apple's new M7 motion coprocessor to empower new breed of
fitness apps, Kevin Bostic, Apple Insider, September 11, 2013; Apple website
Motion co-processor
Application processor
Accelerometer
6-axis IMU
Magnetometer
:
:
Apps
Apple Motion Co-processor Sensor Hub Architecture
Pressure

81

88
Apple’s sensor-related development has been very active in
recent years
Development Timeline of Representative Sensor Technology
201520142012 2013
Acquired AuthenTec;
Fingerprint recognition
sensors and other security-
related technologies
Acquired WiFiSLAM;
Technology enables indoor
navigation and mapping
using WiFi and inertial
sensor data
Acquired PrimeSense;
Technology uses motion
and depth-sensing optical
sensors for 3D sensing
Hiring experts in biometrics/wearables and fashion
Patents filed
for iPhone as
wearables
hub
Multiple patents for wearables and related technology
Patent filed
for enhanced
fingerprint
security
Patent for
heart rate
sensors for
security
Patents for
3D gesture
control UI
Patent for
“intelligent
projector”
with sensors
Latest of
several patents
for “smart
stylus”
Patent for
virtual
keyboard with
3D sensors
Patent
application
for home
button joystick
Patent for
iPad “smart
bezel” (one of
several over
the years)
Several Smart
Cover patents,
including
sensor fusion
Patent for
drop
protection
Patent filed for
microphone
noise
cancellation
Patent
extension for
watch gesture
control

98
In addition to adding health sensors to headphones, Apple
may be exploring other sensor-enhanced designs
• In 2012, Apple filed patents for sensors in headphones that are used to
detect a user's presence and turn certain functions on or off.
–The system would use capacitive touch sensors, accelerometers, and
force sensors
–The sensors would detect if the earbuds are in the user’s ear so that
the noise-cancellation feature could be turned on or off in order to
save power or the music could be played or paused
• In 2013, Apple filed a patent for headphones that recognize voice
activity to activate and tune beamforming for advanced noise
cancellation
–The system would use accelerometers, two microphones in the
earpod, and an array of microphones on the wire
–Voice would be detected by the accelerometer via vibrations
propagating through tissue and bone and the array of microphones
would either be targeted at the face or at ambient noise that could
then be removed from the signal
• Apple’s acquisition of the Beats headphones brand in 2014 has been
identified as a possible step towards Apple making “smart headphones”
(with new health monitoring and/or audio technology), because it
comes with the infrastructure for significant headphone production
Image from Apple patent and Apple
Sources: With Beats, Apple Buys A Quick Start On Smart Headphones, Sam Sanders, NPR, May 31, 2014; Apple's latest patent application: Smart headphones that recognize your
voice, Barry Levine, Venture Beat, April 3, 2014; Future EarPods may use sensors to detect users' ears, control noise cancellation and music playback, Mikey Campbell, Apple Insider, May
29, 2014; Apple working on voice-recognizing headphones with built-in accelerometer, beamforming mics, Mikey Campbell, Apple Insider, April 3, 2014

104
• Apple has patented a design for 3D gesture control
software that works with PrimeSense’s motion and depth-
sensing optical sensors and is capable of recognizing and
translating user gestures (for example, a command could
be executed with a hand rising gesture)
• Apple has also patented a motion sensing system for a 3D
GUI concept for the iPhone that would use sensors to
determine and display a perspective projection of the 3D
display environment and detect gestures made above the
display with proximity sensors
The PrimeSense acquisition and other developments suggest
Apple is broadly exploring 3D technology
Image from Waak (not the Apple system)
Sources; Apple's secret plans for PrimeSense 3D tech hinted at by new itSeez3D iPad app, Daniel Eran Dilger, Apple Insider, July 11, 2014; New Apple Patent From PrimeSense Covers
Kinect-Style 3D Space Mapping, Darrell Etherington , Tech Crunch, December 9, 2014; Apple patent points to in-air 3D gesture UI for Apple TV, Mac and beyond, Mikey Campbell, Apple
Insider, January 13, 2015; Apple wins patent for 3D iPhone UI w/ motion gestures, reassigns PrimeSense 3D mapping patent, Jordan Kahn, 9 to 5 Mac, December 9, 2014; Apple patents
and other sources
–Rumors have emerged recently that Apple is working on glasses-free 3D iPhone display technology,
and, at the end of 2014, Apple had listed a number of job openings for developers with experience
developing on virtual reality and motion sensing technology
• Other Apple projects also include enhanced 3D experiences:
–3D user interface effects: GPS location, position (using motion sensors), and lighting conditions are
combined to create a better 3D experience in a mobile device (e.g. for more sophisticated gaming)
–3D virtual navigation: includes creating virtual 3D scenes from real-world scenes, and the ability to
navigate the 3D virtual environment; motion sensors track movement of the device to navigate or
create 3D representations of objects
–3D remote controller wands: Use motion sensors to recognize user gestures for zooming and
scrolling functions for media systems

119
An Apple patent describes the use of sensors to protect a
dropped device
• In 2014, Apple was granted a patent for a system that
uses sensors to protect a dropped iPhone
–The iPhone would be equipped with a “drop
sensor” (similar to the ones already used in Apple
laptops) based on motion sensors
–If the sensor detects it is falling, the drop processor
would communicate with a small motor to shift the
phone’s internal weight and control which side it
lands on (specifically, a side made more rugged for
this purpose)
• The iPhone already has many of the building blocks for
such a system with the motion processor and a small
motor used for vibration, so experts have suggested
the system could be relatively easily implemented
Sources: US Trademark Office Approves Apple Patent That Would Give iPhones Cat-Like Reflexes, Drew Prindle, Digital Trends, December 4, 2014; Apple patents mid-air flip
technology for protecting phones, Nic Healey, CNET, October 10, 2013

129
Apple’s early moves have built its reputation as a key player in
the Internet of Things landscape
Note: Chart from Appinions; based on identifying relevant opinions from within over 200 million documents from news, blogs, forums, and social media over the preceding 90
days; data captured July 2014
Sources: Internet Of Things Influence Study, Appinions, July 2014
Most Influential Internet of Things Companies
RANK
NET
INFLUENCE
SCORES
GOOGLE ACQUIRED NEST

134
Although not yet launched, HomeKit has already attracted
device partners
• When Apple announced HomeKit at WWDC in June 2014, it announced partnerships with
many leading manufacturers, including iHome, Haier, Withings, Philips, iDevices, Belkin,
Honeywell, and Kwikset
Image from Insteon
Sources: Apple's HomeKit rollout to require Apple TV for remote Siri control, Rich Brown, CNET, January 8, 2015; Apple HomeKit Automation Gear Sees Slow Ramp, Dawn Chmielewski
and Ina Fried, Recode, January 23, 2015; Apple details HomeKit compatibility with competing home automation platforms, rules out rival Wi-Fi gear, Jordan Kahn, 9to5 Mac, January 22,
2015
• At the 2015 Consumer Electronics Show, the first
round of HomeKit-compatible products were
unveiled; they include a wide variety of products
and sensors, including:
–Insteon Hub that bridges HomeKit-compatible
devices with Insteon’s vast catalog of otherwise
incompatible smart home accessories
–Smart sensors by Elgato for measurements like
air quality, temperature, humidity, air pressure,
energy and water consumption
–ConnectSense Smart Outlet, an Internet-
connected electrical socket that turns ordinary
household appliances, like lamps, into devices
that can be controlled via HomeKit
–Smart versions of basic products like lightbulbs
and garage door openers

142
A variety of sensor-based third-party accessories are on the
market or in development
Home Environment
These encompass and go beyond HomeKit and HealthKit platform accessories
Quantified Self
Health/Medical
Detect a variety of body conditions to
provide on-going monitoring or evaluation
of a specific condition
Fitness
Detect motion and body conditions such as
heart rate or perspiration
Sports
Track movement of sports equipment to
provide feedback on performance
Detect conditions or problems
in the home and/or enable
smart control of home devices
Detect environmental conditions
such as weather, pollution
levels, or water quality
Lifestyle Gaming/UI
Use sensors to capture
information about mobile
device surroundings or control
the device
Smartphone/Tablet Accessory Landscape
Wide variety of devices from pet
monitors to devices that provide
smart support for hobbies.

158
Four things are crucial in IoT: sensing, connectivity, data
analyses and services. We have our technology in sensing
and have [memos] in place with companies that are great in
sensors… [IoT] has the potential to transform our economy,
our society and how we live our lives.
“
“
BK Yoon, CEO Samsung, January 2015
What competitors say about sensors and the IoT
There will be So many…devices, sensors, things that you are
wearing, things that you are interacting with that you won't
even sense it. the Internet will be part of your presence all
the time.
“ “
Eric Schmidt, Executive Chairman, Google,
January 2015

166
The leading smartwatches all have accelerometers, but use of
other sensors varies
Sources: What Exactly Is an Apple Watch For?, Daisuke Wakabayashi, The Wall Street Journal, February 16, 2015; Best Smartwatches 2015, Mike Prospero, Tom’s Guide, February 26,
2015; Sony website, Samsung website; Google website, Pebble website; LG website; Vandrico Wearables Database
Apple Watch Samsung
Gear 2
Google
Moto 360
Sony
SmartWatch 3
Pebble LG G Watch R
Starting Price $349 $300 $250 $249 $99 $250
Sensors:
Accelerometer      
Gyroscope     
Compass   
Microphone     
Force 
Ambient light   
Heart rate    
Pressure/barometer 
Other features (used with sensors to enhance functionality):
GPS 
WiFi 
Bluetooth      
Leading Smartwatch Sensor Comparison

187
Beyond Xiaomi, other Chinese manufacturers are also
increasingly using sensors
• Among the top Chinese manufacturers (which include Xiaomi), most
devices have the common smart sensors and sensor hubs and are
adding advanced features like fingerprint sensors in their latest models
–Huawai was the first Android smartphone with a touch fingerprint
sensor (like Apple’s Touch ID)
• As a result, Chinese manufacturers are increasingly dominating the
global market for sensors
Sources: MEMS: Looking back at 2014 and 5 years outlook, IHS, November 2014; Huawei sets a world first with Precise BioMatch™ Mobile, Percise Biometrics; China Smartphones: Hot
for Sensor Hubs, Junko Yoshida, EE Times, June 23, 2014; The rise of China's smartphone makers, Matt Hamblen, Computer World, December 30, 2014
Chinese Mfr
0%
20%
40%
60%
80%
100%
2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Revenue[$100M]
Other OEMs Chinese OEMs Apple Samsung
Consumption of Motion Sensors and MEMS Microphones by OEM

190190
Appendix
Apple Sensor Project Details

191
Apple Sensor Project Summary
Motion Sensors 25
Multiple Sensors 14
Accelerometer 7
Magnetometer 4
Temperature Sensor 4
MEMS Actuator 2
Gyroscope 2
Microphone 2
Light Sensor 1
Ultrasonic Transmitter 1
MEMS Mirror 1
Proximity Sensor 1
Pressure Sensor 1
Switch 1
Other 2
User Interface 21
Positioning & Navigation 9
Imaging & Video 8
Camera 7
Health & Fitness 5
Power Management 5
Sound 5
Networking 4
Motors & Actuators 2
Display 1
Security 1
Mobile 63
Computer Mouse 3
Personal Computer 1
Remote Control 1
Application Sub-System End Application

192
Apple Sensor Project Details (1/15)
Project Title Summary Sensor
Technology
Sub-System End
Application
Key person(s)
IR rejection filter for
ambient light sensor
The ability to accurately measure ambient light in handheld devices is
important for a adjusting the display brightness and color scheme. Ambient
light sensors have a tendency to report that the ambient light is brighter than
it is. This occurs in environments lit by incandescent lamps compared to
fluorescent lamps. In this project, a filter has been developed to accurately
measure and calculate the amount of visible light (i.e. subtracting the amount
of IR light).
Sensor application:
- Use a combination of clear and color sensors to cancel out the IR in the
ambient light source.
Light Sensor Camera Mobile Ching Yu Tam (Cupertino,
CA)
Camera actuator Digital cameras are adding more functionality by introducing additional optics
for new applications, such as autofocus. The optical assembly and associated
driving circuit involved must be small (to fit in flat smartphone), low cost and
consume little power. This project is using a low-profile MEMS actuator for
adjusting a camera lens to change the focal point.
Sensor application:
- Piezoelectric MEMS actuator for adjusting a camera lens.
MEMS
actuator
Camera Mobile Richard Topliss (Campbell,
CA)
Camera autofocus Autofocus is one of the "holy grails" for digital cameras in handheld devices.
This requires additional optics and driving circuitry. This project uses software
to control a MEMS actuator that drives an optical assembly. The camera can
then automatically change focal points when the user aims the camera at
different objects.
Sensor application:
- Piezoelectric MEMS actuator for autofocus.
MEMS
actuator
Camera Mobile Richard Topliss (Campbell,
CA)
Tripod detection for
imaging
When a handheld device is mounted on a tripod or otherwise held still, image
stabilization is not needed. This project is using motion sensor data to
determine if stabilization is needed, before the actual image is captured.
Sensor application:
- Use of motion sensor data to detect if a camera device is stationary.
Motion
Sensors
Camera Mobile Jianping Zhou (Fremont, CA)
Sebastien Beysserie
(Milpitas, CA)
Tripod detection for
video
When a handheld device is mounted on a tripod or otherwise held still, video
stabilization is not needed. This project is using motion sensor data to
determine if stabilization is needed, before the actual video is captured.
Sensor application:
- Use of motion sensor data to detect if a camera device is stationary.
Motion
Sensors
Camera Mobile Jianping Zhou (Fremont, CA)
Sebastien Beysserie
(Milpitas, CA)
George Williams
(Pleasanton, CA)
Rolf Toft (Palo Alto, CA)

202
Apple Sensor Project Details (11/15)
Project Title Summary Sensor
Technology
Sub-System End
Application
Key person(s)
Audio alert silencing Mobile devices provide audible alerts to the user for email, phone, timer, etc.
Silencing these alarms can be inconvenient for the user, who typically needs to
bring out the device and press a button or touchscreen. In this project, an
accelerometer is used to silence alarms. The user simply shakes the device,
without having to touch it.
Sensor application:
- Accelerometer detecting when the device is being shaken to silence alarms.
Accelerometer User Interface Mobile Scott Herz (Santa Clara, CA)
Dan Keen (Cupertino, CA)
Motion sensor to
navigate a cursor on a
display
As mobile device displays are becoming smaller (especially on, for example, smart
watches), it becomes increasingly more challenging for users to interact with
them. New, innovative user interfaces are being developed. In this project, the
user can move the device to navigate a cursor on a display.
Sensor application:
- Accelerometer for user motion tracking.
Accelerometer User Interface Mobile Farshid Moussavi (Oakland,
CA)
3D remote controller
wand for media
systems
The classic remote control for media entertainment systems is slowly being
replaced by "smarter" remote controls. This project is using motion sensors to
recognize user gestures for zooming and scrolling functions.
Sensor application:
- Sensors for motion tracking in a remote control.
Motion
Sensors
User Interface Remote
Control
Duncan Kerr (San Francisco,
CA)
Nicholas King (San Jose, CA)
3D virtual navigation 3D virtual environments is an extension of the existing set of features on handheld
devices. This particular project has developed a system for creating virtual 3D
scenes from a real-world scenes, and the ability to navigate the 3D virtual
environment by motion tracking.
Sensor application:
- Motion sensors to track movement of a device to navigate or create 3D
representations of objects.
Motion
Sensors
User Interface Mobile Richard Tsai (Cupertino, CA)
Andrew Just (Cupertino, CA)
Brandon Harris (Palo Alto, CA)
Computer mouse with
motion sensors
Computer mice equipped with a surface tracking sensor (i.e. optical) can only
measure user movements in two directions. This project has added motion sensors
to detect rotational movement, which enables more functionality and a better
user experience.
Sensor application:
- Motion sensors for tracking rotational movements of a computer mouse.
Motion
Sensors
User Interface Computer
Mouse
Michael Cretella, JR. (Santa
Clara, CA)
David Amm (Sunnyvale, CA)
Motion and touch
hybrid user input
Touch screens are inadequate in tracking both large and fine motions. For
example, moving a cursor from one side of a touch display to another might take
several swiping motions. This project is combining motion sensors for large
movement and touch screen for fine movements in a hybrid approach.
Sensor application:
- Motion sensors for tracking user movement
Motion
Sensors
User Interface Computer
Mouse
Farshid Moussavi (Oakland,
CA)

207

208208
About Us
For more information, please email us at info@memsjournal.com

209
About MEMS Journal, Inc.
• MEMS Journal was founded in 2003
• 35 full-time and part-time staff and are headquartered in Southfield, Michigan
• Largest MEMS publication worldwide, with 34,800+ subscribers, growing quickly
• Published over 9,800 stories, interviews, and articles and have reviewed more than 26,000
patents and patent applications
• Services include:
–Strategic planning (focus and positioning, marketplace perception, competitive analysis,
roadmap development)
–Market research and intelligence (technology scouting and assessment, system and
component teardowns, patent/IP analysis)
–Event organization and management (organize 4-5 events and sponsor 20-25 events per
year)
–Marketing and advertising (brand building, lead generation)
–Content development (whitepapers, presentations, reports)

210
Our Upcoming Conferences

211
Market Intelligence Services
For questions or more information, please email us at info@memsjournal.com.

212
Market Intelligence Services
For questions or more information, please email us at info@memsjournal.com.

213
Contact us
MEMS Journal, Inc.
2000 Town Center, Suite 1900
Southfield, Michigan 48075
info@memsjournal.com
248.792.9618 (office phone)
http://www.memsjournal.com

214

Apple's Smart Sensor Technologies -- market research report (sample)

More Related Content

Viewers also liked (20)

Similar to Apple's Smart Sensor Technologies -- market research report (sample) (20)

Recently uploaded (20)

Apple's Smart Sensor Technologies -- market research report (sample)