SlideShare a Scribd company logo
134 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
4 Processing IoT Data
Lesson Time: 3 hours, 50 minutes
Lesson Introduction
An important design consideration for Internet of Things (IoT) is how to distribute the
processing tasks involved in acquiring, aggregating, filtering, and analyzing data. Some tasks
should be performed directly on the IoT devices that acquire data from sensors. Other tasks
may be delegated to a local IoT gateway or cloud services.
Lesson Objectives
In this lesson, you will:
• Use microcontroller I/O to interact with actuators and digital and analog sensors
attached to an IoT device.
• Distribute data processing tasks to the most appropriate location within IoT
infrastructure.
• Provide M2M communication between systems used in IoT.
134 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
TOPIC A
Process IoT Device Input and Output
IoT devices acquire data from sensors, and perform initial processing on that data. A staggering
variety of sensors are available for use in IoT devices. When designing an IoT device, you'll need to
know what types of sensors are available, the criteria for selecting the right sensors for your
application, how sensors connect to a microcontroller, and how to read data from them. IoT devices
may also include actuators, LEDs, and other displays that enable them to interact with users and the
environment around them.
DAQ
Data acquisition (DAQ) is the process of measuring real world conditions and converting the
measurements into a form that can be processed electronically—by a digital computer, for example.
Readings may be taken (or sampled) at a fixed time interval, or on demand.
The process of data acquisition includes some form of signal conditioning, which entails
converting raw data into a suitable form for processing and analysis by a computer.
Sensors
Sensors are electronic components designed to perform the task of data acquisition. They detect
information about the world around them, such as air temperature and humidity, and convert it into
a signal that can be processed electronically.
Some sensors function like a switch, making or breaking a connection depending on whether a
measurement has reached a particular value, such as a temperature, water level, or air pressure.
Other sensors can provide a continuous range of data values.
IoT devices typically include sensors, and can be installed virtually anywhere—in a forest,
refrigerator, machine, body, or garbage can, at the top of a tower, underneath a road surface, or in
orbit around the planet. IoT devices can be located in places you might not be able to go, but
sensors mounted on the device can apprise you of the situation there, enabling you to answer a wide
range of questions about a remote entity, such as:
• Where is it, and where is it headed? (location, bearing, and speed sensors)
• Is it positioned where it needs to be? (proximity sensors)
• What is the weather there? (environment and weather sensors)
• What does it look like there? (camera and imaging sensors)
• What types of forces are at play there? (energy and force sensors)
• What are things there made of? (chemical sensors)
• How long does it take to do something there, and when was the last time it happened? (clock)
Sensors can answer questions like these—and many more.
Multiplexing
Developer boards provide various pins you can use to attach sensors, and you need to match the
type of I/O pin on the board (analog or digital) to the type of data being provided from a sensor. In
some cases, the number of I/O pins may be quite limited, so you need to ensure you have enough
connections for your sensors.
It's often possible to read more sensors than you have microcontroller I/O pins to accommodate. It
may be possible to multiplex the sensors—essentially taking turns reading the different sensors
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 135
through a shared I/O pin. It requires additional code and electronic components to accomplish this,
so it is simpler when you have enough pins to match the number of sensors you require.
Location, Bearing, and Speed Sensors
Four common sensing devices often work together to track location, bearing, and speed.
• Accelerometers measure changes in momentum and inertia, including the pull of gravity.
• Gyro sensors sense changes in twist and angle, using a mechanism that resists such changes
(such as a vibrating fork or spinning wheel) as a reference.
• Magnetometers sense a magnetic field, and can use the Earth's magnetic field as a reference to
get a compass bearing.
• Global Positioning System (GPS) sensors use man-made satellites as a reference to obtain the
sensor's position on the planet.
Each of these has benefits and drawbacks. For example, GPS is very precise and has very little drift,
but is subject to the sensor's ability to communicate with satellites. In a tunnel, you might
temporarily lose the connection. GPS also uses more power than the other sensor types.
Accelerometers and gyro sensors are not dependent on a satellite connection, but they are subject to
drift. Magnetometers are prone to interference from magnetic sources other than Earth's magnetic
field. However, used together, these sensors can provide a resilient and accurate data stream.
For specialized position sensing, other types of sensors may be used.
• Wi-Fi, cellular, and other network connections can be analyzed to determine where the
device attaches to the network, which can often be used to provide a coarse measurement (for
example, within a few city blocks) of the user's location.
• Variable resistors (rheostats or potentiometers) are implemented as sliding or rotating
controls, like the volume control on a radio. Used as sensors, these components can be attached
to various types of levers to measure position. For example, a float sensor is typically a variable
resistor attached to a lever with a float on the end of it that follows the surface of a liquid,
enabling the level of liquid to be measured.
Proximity Sensors
Proximity sensors can detect the presence of a nearby object without having to actually come in
contact with the object. Different types include:
• Echo-based proximity sensors include sonar, radar, and lidar and go by various other names,
such as laser or optical rangefinders, ultrasonic sensors, and so forth. This type of sensor emits a
field, wave, or beam of some sort, and measures the time it takes for it to bounce back. If you
know how fast the field or beam travels, you can determine the distance to the device it bounced
off of.
• Sonar (sound navigation and ranging) use sound.
• Radar (radio detection and ranging) sensors use radio waves.
• Lidar (light detection and ranging) sensors use light waves.
Different frequencies result in different abilities, such as the distance they can cover and the
types of materials they can penetrate. So for example, different forms of light (e.g., UV or
infrared), sound frequencies, or radio waves may be used by different sensors.
• Electromagnetic sensors detect proximity to an electromagnetic field. Some, like Hall effect
sensors can measure flow and variations in the magnetic field, whereas others, like reed switches,
make or break a connection (on or off) based on the proximity and strength of a magnet. Reed
switches can be mounted on moving devices like a door or window to determine when they are
open or closed, without requiring any mechanical levers or contact between the moving entities.
• Capacitive touch sensors sense when a finger (or some other conductive object) is touching or
in close proximity to the sensor. Many smartphone display screens use capacitive touch sensing.
Fingerprint sensors are another application of this sensor type.
Lesson 4: Processing IoT Data | Topic A
136 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
If contact with an object is acceptable, a limit switch may be used to detect when the object is in
the desired position. Limit switches typically have a spring-loaded lever, often with a wheel or
bumper attached to it, which switches when an object bumps into it (indicating that a limit has been
reached).
Environment and Weather Sensors
There are many types of sensors that can measure the environment around an IoT device.
• Temperature
• Humidity
• Wind speed and direction
• Rainfall
Cameras and Imaging Sensors
Cameras and imaging sensors detect various forms of light and record data representing what they
"see," which may include spectrums of light the human eye can't see, such as ultraviolet or infrared.
• Cameras can provide an image or video of the world around the device. Sensing cameras can
provide early detection of cracks in infrastructure, hot spots, flames, edges (of a road or a
package, for example), and other visual events.
• Motion detection algorithms can be applied to camera data, so a device can be programmed to
send you data only when something new appears on the scene.
Energy and Force Sensors
Force and pressure sensors measure the amount of force exerted over a particular area. The force
may be exerted through solids (mechanical pressure), liquids (such as water pressure), or gases (such
as air pressure).
Pressure sensors can be used to indirectly estimate variables that are a function of pressure, such as
altitude (in the atmosphere), depth within a body of water, or weather (barometric pressure).
Pressure may be measured in relative amounts. For example, vacuum is measured as a difference
from normal atmospheric pressure.
• Force sensors can measure mechanical force exerted upon the sensor through a wide range of
sources such as human touch (touch sensors).
• Load cells create an electrical signal that is proportional to the amount of force being applied to
the sensor.
• Load bolts are specialized load cells shaped like a bolt and used to measure the amount of
mechanical stress upon a machine at that location.
• Impact sensors are another type of load cell. They are optimized to measure mechanical force
experienced when the sensor is struck.
• Weight sensors measure gravity's pull on the object being weighed.
• Air pressure sensors may monitor barometric pressure, vacuum, or line pressure.
• Sound sensors detect periodic changes in local air pressure. Humans can hear frequencies
between 20 and 20,000 vibrations per second (Hertz), but various types of electronic sensors can
detect sounds below or above this range. A common form of sound sensor is a microphone, which
converts sound waves into electronic signals that can be monitored for changes in volume,
frequency, and other qualities.
• Flow sensors measure the volume and rate of movement of a liquid or gas through a conduit
such as ductwork or pipes.
• Resistive touch sensors are true force sensors. They function through physical force, unlike
capacitive touch sensors (described previously), which sense a finger's proximity. In comparison
to capacitive touch sensors, resistive touch sensors tend to:
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 137
• Function better in harsh environments than capacitive touch sensors.
• Function better when the user is wearing gloves or using a non-conductive stylus.
• Be less responsive than capacitive touch sensors (which is why most tablets and smartphones
use capacitive touch sensors).
Electrical energy can be measured by its various attributes, such as voltage, current, and resistance.
• Voltage sensors measure the voltage difference across two lines.
• Electrical current sensors measure the amperage of current flowing through an electrical circuit.
Split core current transformers can be mounted around an electrical wire without actually cutting
into the wire, although they must surround only one of the two wires in the power cable.
• Resistance is measured using an ohmmeter. Many types of sensors change their resistance to
correspond with an attribute they're measuring, such as light (photoresistor), electromagnetic
force, and so forth.
Electromagnetic radiation (EMR) is energy emitted from a source. It takes on many forms that
people interact with every day, such as light, radio, microwave, X-rays, and gamma rays. All of these
forms of radiation can be detected by sensors:
• Radio
• Infrared light
• Visible light
• Ultraviolet light
• X-rays
• Electromagnetic fields
Chemical Sensors
Chemical sensor options are improving as the technology advances from multiple specialized
devices to a single generalized chip or module that can detect a wider array of chemical measures.
Relatively inexpensive chemical sensors can now provide molecular analysis of foods, medicines,
soil, indoor and outdoor air, rainwater, biological samples, and countless other substances.
• Gas sensors can detect a variety of volatile organic compounds (VOCs), breath alcohol levels,
toxic gases like carbon monoxide, and pollutants.
• PH sensors can measure acidity or alkalinity.
• Organic substances such as alcohol can be detected and measured.
Timer
When multiple readings are taken over a specific time, an additional dimension of measurement can
be provided. For example, using a proximity sensor, the difference between two distance
measurements divided by the time between those measurements produces a measurement of velocity
(e.g., meters per second). With another reading of time and distance, you can determine acceleration—
how much faster (or slower) the object goes over time.
There are various ways to measure time. You can measure relative time, such as how long a device
has been running, how long since the battery was charged, and so forth. You can also measure local
time, which is indicated relative to coordinated universal time (UTC) or Greenwich Mean Time
(GMT).
Processors and microcontrollers typically measure relative time using their internal clocks, which are
based on the consistent vibration of crystals. The device's internal clock can be kept in sync by
periodically using an Internet connection to compare the processor's local time with that of a time
server, using a protocol such as Simple Network Time Protocol (SNTP). A microprocessor or
microcontroller may be connected to an external real-time clock (RTC) chip. RTCs run continuously
using their own power supply (such as a battery), and enable a device to stay consistent with world
time, even if the device is powered down.
Lesson 4: Processing IoT Data | Topic A
138 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Note: All of the Guidelines for this lesson are available as checklists from the Checklist tile on
the CHOICE Course screen.
Guidelines for Selecting Sensors
Follow these guidelines for selecting sensors.
Select Sensor Types for IoT Projects
When considering what types of sensors to use for your IoT projects, consider such things as:
• Cost: While the cost of one sensor might not make much difference in a single device, if you
plan to scale to hundreds of devices or if you are developing a commercial product, it is
important to monitor costs. The same component can vary widely in price from one vendor to
another. Furthermore, there are often many different components to choose from. To manage
cost, it is important to keep requirements in mind. You might over-engineer the project, using
costly components when much less expensive components would meet requirements. In most
cases, components cost much less when you purchase them in volume.
• Power: What power source (e.g., voltage and current) does the sensor require? Will it operate
with the power you plan to provide to the device? Does the sensor support your IoT device's
power management requirements?
• Signaling: What form of signal does the sensor provide (analog or digital, voltage levels, etc.)?
• Resolution and range: How sensitive, and with how much detail, can the sensor take
measurements? What are the smallest and largest values the sensor can read? How frequently can
measurements be sampled?
• Physical interface: How does the sensor interact with its environment? For example, does it
need to directly attach, or can it measure simply by being located near the thing it is measuring?
• Operating environment: Are there factors that affect the sensor's reliability and durability, such
as operating temperature, exposure to liquids, dust, chemicals, ambient electrical noise, shock,
and vibration?
• Development support: What software libraries and APIs are provided to support software
development using the development tools you plan to use?
• Supplier: If you need to scale, producing many devices over time, it becomes especially
important to carefully select your supplier. If time is a consideration, make sure the supplier
carries an adequate supply of inventory and can get parts to you in an appropriate timeframe.
• Form factor: How large is the sensor? How will it mount or attach to your IoT device? Sensors
targeted toward prototyping and hobbyists are often pre-packaged on a shield or small circuit
board that includes supporting components and is easy to use on a prototyping breadboard. For
production, it is often more appropriate to use sensors provided in a small package (a chip with
dual inline pins, for example) that is not already mounted on a board, and is ready to be mounted
on and soldered to a custom circuit board.
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 139
Item Description
Scenario
Once you've determined what kind of data you need to obtain, your next task will be to identify the
type of sensor or sensors that will provide that data. Many different types of sensors are available for
use in IoT devices. In many cases, numerous options will seem to meet your requirements, and
you'll need to narrow them down to identify your best option. In this activity, you'll consider project
requirements and identify various sensor options that might support those requirements.
1. You need to detect the number of cars passing over a particular section of
road. What types of sensors might you use for this?
2. You are designing a home automation device to monitor a home's sump
pump. You'd like to be able to send alerts to the home owner when the pump
is cycling excessively or the water level gets too high.
Location Awareness
Location services use a series of satellites in Earth orbit to enable a receiver to determine where it is
located. The satellites broadcast information about their orbit and timing. The receiver combines
this information from multiple satellites to triangulate its location. Commercial systems are accurate
to within a couple meters, but high-end systems can pinpoint location to within a few centimeters.
GPS Global Positioning System (GPS) is a satellite-based radio navigation
system, also known as NAVSTAR, operated by the United States Air
Force. GPS employs 31 satellites, and was first launched in 1978. Users
do not have to transmit any data to use GPS for positioning. The system
is freely accessible to anyone with a GPS receiver.
Lesson 4: Processing IoT Data | Topic A
ACTIVITY 4-1
Selecting Sensors for IoT Applications
140 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
GLONASS GLONASS, which stands for Globalnaya Navigazionnaya Sputnikovaya
Sistema (Global Navigation Satellite System), is Russia's version of
GPS. GLONASS employs 24 satellites, and was first launched in 1982.
BeiDou The BeiDou Navigation Satellite System (BDS) is made up of two satellite
constellations. BeiDou-1 is composed of three satellites, offering limited
coverage and applications primarily targeted to customers in China and
neighboring regions. BeiDou-2 (also known as Compass) is a
comprehensive global navigation system currently under development
and will include 35 satellites when completed in 2020.
Galileo Galileo is a global navigation system currently under development by the
European Union and the European Space Agency and due to be
completed by 2019. It employs 30 satellites, to provide a precise
positioning system for Europe that is independent from GPS,
GLONASS, and other systems.
Actuators and Other Output Devices
Actuators provide physical output, typically in the form of motion produced through electrical
motors and servos, pneumatic or hydraulic cylinders, and so forth. The term may also refer to small
electronic components that produce actions or signaling on a device, such as lights, vibrators, force
feedback, and buzzers. Essentially any sort of energy that can be controlled electronically can be
used for output. Some examples are listed here.
Lesson 4: Processing IoT Data | Topic A
Item Description
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 141
Motion A wide range of actuator devices and technologies can be used to control
motion. Typically, actuators provide a certain type of motion (circular or
linear, for example) and range of motion (distance the actuator travels)
and include a means for other levers or devices to be attached to them.
Levers, gears, and other mechanisms can be used to adjust the type of
motion produced by the actuator.
Electromechanical actuators typically operate by applying electrical
current to an electromagnet or motor, whereas hydraulic and pneumatic
actuators are controlled by opening and closing valves that provide
pressurized liquid or air to push or pull a piston within an enclosed
cylindrical space. The control valves themselves may be controlled
through another type of actuator, such as a stepper motor.
Various types of sensors (such as limit switches, proximity sensors, or
quadrature encoders) may be used in combination with an actuator to
determine when it has arrived at the desired position.
• Solenoids: When electrical current is provided to an electric coil, an
electromagnetic force is created, sufficient to pull a lever. Typically a
spring or some other form of resistance pulls the solenoid back to its
original position when current to the solenoid is removed.
• Linear actuators: Use an electrical, hydraulic, pneumatic, or some
other force to provide movement in a straight line.
• Motors: Motors provide circular motion, which may be reversible
(clockwise or counter-clockwise).
• Stepper motors: Stepper motors are optimized for moving an object
to a precise position within a circular motion, rotating in defined steps.
• Servos: Similar in purpose to a stepper motor, servos rotate a shaft to
a specific position.
• Vibrators: Vibrating mechanisms are typically used to provide
feedback, as in the silent ringing mode on a smartphone.
• Muscle wire: Electricity flowing through a muscle wire causes it to
flex or relax, similar to muscles in a human or animal.
Lesson 4: Processing IoT Data | Topic A
Type of Energy Description
142 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Electrical power Various types of electronic devices can be used as power switches,
enabling circuits operating on one power level to control circuits with a
different (typically higher) power level.
Which type of switching device you use depends to some extent on the
amount of power you need to control.
• Relays: Relays are traditionally electromechanical devices, although
solid-state (non-mechanical) relays do exist. A typical relay has a coil
of wire that creates a magnetic field when electricity is applied. The
pull of magnetism closes a mechanical switch, which is opened again
when electricity is no longer applied to the coil. In typical applications,
a small amount of electrical current controls the switch, while the
switch closes a circuit that conducts higher current. Relays provide
good electrical isolation between the two circuits.
• Transistors: Transistors can provide switching capabilities when
power levels between the microcontroller and the device being
switched aren't extreme. For example, you might use a transistor to
enable a 3 volt microcontroller to control a small 9 volt motor. While
transistors tend to be limited to lower power levels than relays, they
generally switch much more quickly since they are not mechanical.
Special types of transistors, such as metal-oxide-semiconductor field-
effect transistors (MOSFETs) can operate with higher power levels
than normal transistors. For example, a MOSFET might enable a
microcontroller to control automobile headlights.
Sound There are various types of sound-generating devices.
• Transducers: Transducers are commonly electromechanical or
piezoelectric devices. They may essentially be a mechanical relay that is
switched on and off frequently, or a piezoelectric device that uses a
vibrating crystal or metallic fork to produce tones, clicks, or buzzing.
• Speakers: Speakers are fundamentally similar to some transducer
designs (using piezoelectric or coil-based vibration to generate sound
waves) but the frequency of the sound waves is modulated to produce
a wider range of tones, even reproducing human voice or music.
• Sirens, horns, and whistles: When loud, penetrating sounds are
required (for alarms in an industrial environment, for example), a
compressed air source (such as an air pump) can be used to generate
loud wailing or whistling noises.
Light • Annunciator (status indication) lights may be implemented through
various lighting technologies, such as incandescent, fluorescent, or
light emitting diodes (LEDs).
• LEDs can be used as annunciators or for illumination. A single LED
device may contain just one color LED or several colors (typically red,
green, and blue), which can be separately controlled to provide color-
coded indicators. Multiple LEDs arrayed in a matrix can be used for
segmented displays.
• Special light sources, such as laser and infrared, can provide
signaling for specialized situations. For example, laser can be used for
long-distance line-of-sight signaling or remote control, and infrared is
frequently used for short distance wireless control (as used in TV
remote controls).
Lesson 4: Processing IoT Data | Topic A
Type of Energy Description
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 143
Case Study: Analog Light Sensor
The GL5516 photoresistor is a variable resistor. The amount of resistance it provides in a circuit
varies based on the amount of light it is exposed to. However, the ESP8266 microcontroller's analog
input pin measures voltage, not resistance. But there is a relationship between voltage, current, and
resistance. By changing resistance, you can change voltage, which you can measure on the
microcontroller's analog pin.
For example, you could connect 3.3 volt power to the microcontroller's analog pin through the
photoresistor, as shown in the following figure.
Figure 4-1: Photoresistor adjusting voltage to the analog pin.
As the resistance provided by the photoresistor changes with the amount of light, it allows more or
less voltage to be provided to the analog pin.
This circuit is almost right, but unfortunately, the analog pin on the ESP8266 takes its maximum
reading at only 1 volt, so the 3.3 volt power provided by the development board will be too much.
And the development board provides only two power options, 3.3 volts and 5 volts. There are no 1
volt power pins. However, by adding a second resistor to the circuit, you can divert some of the
electrical energy to ground so the full voltage won't reach the analog pin, as shown in the following
figure.
Lesson 4: Processing IoT Data | Topic A
144 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Figure 4-2: Voltage to the analog pin divided using a second resistor.
This type of circuit is called a voltage divider. Voltage dividers and other techniques are commonly
used to match required signal levels between a sensor and an I/O pin on a microcontroller.
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 145
Before You Begin
The course data files are installed on your computer in C:095024Data. These files include a
customized version of the Arduino IDE (located in C:095024DataToolsarduino), which
contains example programs that you will use in this course. The example programs are contained in
the File > Sketchbook sub-menus inside the Arduino IDE.
The ESP8266 microcontroller is connected to a USB port on the development computer. You have
successfully connected the device to the Wi-Fi network, and the SSID and password are now cached
on the device. The Arduino IDE is not running.
Scenario
You will use a photoresistor to add light-sensing capability to your IoT device. You will use a second
(fixed value) resistor to reduce the voltage reaching the analog pin on the microcontroller, so you
can take a full range of readings.
The analog-to-digital converter on the ESP8266 returns values in the range 0 to 1023, so the reading
will return a relative light value in that range.
Jumper wires on the prototyping board provide power to the left power rails on the development
board, so you will connect your resistors to the power and ground provided in the power rails,
rather than connecting them directly to the development board.
For now, you will simply output the sensor readings to the serial monitor.
1. Remove power from the IoT device.
a) Disconnect the USB power cable from the IoT device.
When installing components, remove power from the board so you don't damage them.
2. Observe how power will be provided to the power rails.
Lesson 4: Processing IoT Data | Topic A
ACTIVITY 4-2
Reading Analog Sensor Data
146 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
a) Examine the jumper wires on the left side of the breadboard.
• Between socket A26 and the positive (+) power rail.
• Between socket A29 and the negative (−) power rail.
• These wires tap electrical power from pins on the microcontroller.
• The 3V3 pin connects to socket A26, and provides 3.3 volts from the board.
• The GND pin connects to socket A29, and provides ground from the board.
• Because of these connections, components you add to the breadboard can tap power from the
rails in the left two columns of the breadboard.
3. Locate the light sensor and 10K ohm resistor in the development kit.
a) In your development kit, locate the light sensor (photoresistor) and 10K ohm resistor.
You will add these components to the prototyping breadboard. You will team the light sensor with
the 10K ohm resistor to enable it to function at the logic levels used by the microcontroller.
4. Set up the components on the breadboard.
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 147
Note: Position the leads in the positive and negative sockets away from each
other so they don't short out.
a) Insert the 10K ohm resistor wires into the breadboard sockets as shown.
• Resistors have no polarity, so it doesn't matter which end goes into each socket.
• One end goes in the negative (−) power rail on the left side of the prototyping board.
• The other end goes in socket C9.
• The jumper wire already located in socket D9 connects to the analog I/O pin (A0) on the
microcontroller, so the resistor is now effectively connected between ground and the analog I/O
pin.
b) Insert the light sensor into the breadboard sockets as shown.
Connect the light sensor leads with:
• Polarity doesn't matter for this component either.
• One end in socket E9.
• The other end in the nearest socket in the positive (+) power rail on the left side of the
prototyping board.
c) Observe the current configuration.
Lesson 4: Processing IoT Data | Topic A
148 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
• One leg of the light sensor is powered with 3.3 volts.
• One leg of the 10K ohm resistor is grounded.
• The other legs of the light sensor, 10K ohm resistor, and analog I/O are connected to each other
through row 9 of the breadboard.
• The analog pin A0 on the microcontroller can now read light values from the light sensor.
5. Load a sketch that reads values from the A0 analog pin.
a) Display the Windows Start menu, and select the Arduino tile to launch the Arduino IDE.
b) Select File→Sketchbook→Processing IoT Data→Analog_Sensor.
c) Close the other sketch window so Analog_Sensor is the only sketch showing.
6. Start the process to compile and upload the sketch.
a) Plug the USB cable back into the microcontroller board to reconnect the device.
b) Select Tools→Serial Monitor to display the serial monitor window, and ensure the baud rate is set to
9600 baud.
c) Select Sketch→Upload.
7. While the sketch is being compiled, examine the source code.
a) Examine lines 1 through 18 in the Analog_Sensor sketch.
• Lines 3 through 6 initialize various variables used to track the sensor readings. For example,
updateInterval, defined in line 3, determines how much time will elapse between each
sensor reading. Right now, that value is set to 1000 milliseconds, or one second.
• The setup() function in lines 8 through 18 initialize the connection to the serial monitor, where
you'll view the sensor readings.
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 149
b) Examine lines 20 through 39 in the Analog_Sensor sketch.
• The loop() function checks how long it has been since the last sensor reading.
• Line 22 determines if the time between the last update and the current time (in milliseconds) has
exceeded the value specified in updateInterval. If it has, line 23 resets the time of the last
update (to the current time), and then calls other functions to update the sensor values and to log
them to the serial monitor.
• In line 31, the updateSensorValues() function reads the current data value from pin A0 into
the light variable.
• In line 37, the logSensorValues() function prints the value of the light variable.
When the program runs, the current value read by the light sensor will be displayed in the serial
monitor.
8. Examine the sensor readings.
a) When the program runs, examine the output in the serial monitor. Position your hand over the
sensor and observe how the values change.
• Every second, a new value appears.
• The analog pin reads values between 0 and 1023.
• There are 1024 possible values because the ESP8266 microcontroller's analog-to-digital
converter (on pin A0) provides output in 10 bits. There are 1024 different combinations of 1 and
0 in 10 bits (0000000000, 0000000001, 0000000010, 0000000011, and so forth).
• In the current configuration, the brightest light readings would result in the largest numbers, and
the dimmest light would result in the smallest numbers.
Lesson 4: Processing IoT Data | Topic A
150 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Case Study: Digital Humidity and Temperature Sensor
The DHT11 and DHT22 digital humidity and temperature sensors are popular for prototyping and
hobbyist use because they are inexpensive while providing good performance. The two devices are
similar, operating on 3 to 5 volts, and drawing very low current (no more than 2.5 milliamperes) to
operate. Within their plastic shell, the devices enclose a humidity-sensing component, a thermistor
(temperature-sensing transistor), and an integrated circuit that converts their signals to digital form
for communication with a digital I/O pin on the microcontroller. The device requires a positive and
negative connection to provide power, and uses a single data pin for serial communication. The
DHTesp software library provides the logic to enable the ESP8266 microcontroller to communicate
with the sensor.
The DHT22 is typically more expensive and reads a wider range of temperatures. The DHT22
measures temperatures from -40 to +125 degrees Centigrade (-40 to 257 Fahrenheit), accurate to
half a degree (.9 degrees Fahrenheit). The DHT11 measures temperatures from 0 to 50 degrees
Centigrade (32 to 122 Fahrenheit) with +-2 degrees (3.6 degrees Fahrenheit) accuracy. The DHT22
sensor has a better humidity measuring range, from 0 to 100% with 2-5% accuracy, while the
DHT11 measures humidity from 20 to 80% with 5% accuracy.
There is a tradeoff with the DHT22's greater range and precision; it takes longer to sample data. The
DHT22 can take a reading once every two seconds, whereas the DHT11 can take a reading once per
second.
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 151
Before You Begin
The course data files are installed on your computer in C:095024Data. These files include a
customized version of the Arduino IDE (located in C:095024DataToolsarduino), which
contains example programs that you will use in this course. The example programs are contained in
the File > Sketchbook sub-menus inside the Arduino IDE. The ESP8266 microcontroller is
connected to a USB port on the development computer. You have successfully connected the
device to the Wi-Fi network, and the SSID and password are now cached on the device. You have
installed the light sensor and 10K resistor on the breadboard. The Arduino IDE is running and the
serial monitor is showing.
Scenario
Now you will attach a DHT11 digital sensor to take temperature and humidity readings. This sensor
has three connections: power, ground, and a single data line.
1. Locate the DHT11 sensor in the development kit.
a) In your development kit, locate the DHT11 temperature and humidity sensor.
2. Install components on the breadboard.
a) Unplug the USB cable from the device to remove power while you add the sensor.
Lesson 4: Processing IoT Data | Topic A
ACTIVITY 4-3
Reading Digital Sensor Data
152 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
b) Attach the labeled wires from the DHT11 sensor to the breadboard as shown.
• + wire to the positive (+) power rail
• G wire to the negative (−) power rail
• D wire to the F1 socket
c) Observe how the DHT11 sensor is now connected.
• The signal from jumper wire D (the data line) is conducted by a jumper wire to digital I/O pin D1
on the microcontroller.
• The DHT11 sensor data can now be read using pin D1 on the microntroller.
• The two other wires provide power to the sensor.
3. Load a sketch that reads values from the D1 digital pin.
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 153
Note: Output from the program currently installed on the device may appear in
the console before the new program is uploaded to the device.
a) Select File→Sketchbook→Processing IoT Data→Digital_Sensor.
b) Close the other sketch window so Digital_Sensor is the only sketch showing.
4. Start the process to compile and upload the sketch.
a) Plug the USB cable back in to the development computer to reconnect the device.
b) Make sure the serial monitor window is still showing.
c) Select Sketch→Upload.
5. While the sketch is being compiled, examine the source code.
a) Examine lines 9 through 12.
Initial values for the humidity and temperature (in Centigrade and Fahrenheit) are set.
b) Examine lines 25 and 27.
• The DHT sensor will be read from pin D1.
• Line 27 will output the headings above the data table that will be output to the serial port.
c) Examine lines 45 through 50.
• The humidity and temperature values are read each time the program loops.
• Since the sensor might not be ready, its status is checked before obtaining the sensor readings
to ensure no false values will be obtained.
Lesson 4: Processing IoT Data | Topic A
154 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Note: To influence the sensor readings, you can cover and uncover the light
sensor and breathe on the temperature and humidity sensor.
d) Examine lines 54 through 69.
• Lines 54 through 56 contain the function that prints the heading.
• Lines 59 through 69 contain the function that outputs a row of current data values.
6. a) When the program runs, examine the output in the serial monitor.
Every second, new values are printed.
7. Clean up the workspace.
a) In the Arduino IDE, select File→Quit.
Lesson 4: Processing IoT Data | Topic A
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 155
Item Description
TOPIC B
Process Data in the Cloud
The processing tasks performed by IoT devices revolve around data acquisition. Other, more
powerful devices are typically used to perform the heavy processing tasks. Now that you've enabled
your IoT device to acquire data, you'll set up a sensor network to enable the devices to offload their
data for aggregation by an IoT gateway as it prepares the data for processing by cloud services.
Deployment Models
As more organizations have become comfortable using cloud services, they have migrated systems
from on premises (using the company's own computers and data centers) to the cloud. The cloud,
with its ability to provide virtually unlimited access to resources for computing, storage, and
networking, provides a viable platform for IoT.
Public cloud • Most common way to deploy cloud computing.
• Hardware, software, and supporting infrastructure are owned and
operated by a cloud services provider.
• Tenants (customers) share hardware, storage, and network devices
with other tenants.
• Services are typically accessed and managed through a web browser.
• Deployment costs are contained, as you don't have to purchase
hardware or software, and you pay only for resources you use.
• Decreased maintenance over on-premises deployment since the
service provider maintains the hardware and some of the software.
• Resources are virtually unlimited, as they can be scaled up to meet
demand.
• Reliability tends to be quite high, as service providers build in
redundancy and backups.
Private cloud • Often used by government agencies, financial institutions, other
organizations that need to maintain control over business-critical
operations.
• Cloud computing deployment dedicated solely to use by one
organization.
• Can be physically located on premises, or hosted by a third-party
service provider.
• More flexible than public cloud, making it easier to customize
resources for very specific IT or business requirements.
• Potentially improves security since resources are not shared with
others, facilitating higher levels of control and security.
• Still provides the scalability and efficiency of a public cloud.
Lesson 4: Processing IoT Data | Topic B
156 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Hybrid • Combined use of public and private clouds, to take advantage of both.
• May move data and applications between private and public clouds as
needed to meet demand, provide more flexibility, or increase
deployment options.
• Enables the organization to maintain a private infrastructure for
sensitive assets.
• Provides the flexibility to use resources in the public cloud as needed.
• Enables scaling to the public cloud only as needed, helping to contain
costs.
• Facilitates a gradual move to the public cloud over time.
Cloud Computing
Cloud computing is a model for sharing data processing, storage, and software applications among
multiple customers or organizations across a network, such that individual customers can provision
computing resources instantly as needed and release them when they are no longer needed.
Companies such as Amazon, Google, and Microsoft have created large data centers with computing
resources they rent to customers, who then do not have to buy and maintain their own dedicated
servers. These cloud data centers provide a pool of computing resources, and expert staff to run
them, that are available to customers whenever they need them.
The National Institute of Standards and Technology (NIST) has identified five principles that define
cloud services and describe how they are typically designed. These are also core benefits to cloud
computing customers. They are:
• Resource pooling: Anyone who wants access to computing resources can get access to the
resources they need, as long as they pay for them.
• On-demand self-service: Customers can add resources they need, when they need them.
• Rapid elasticity: Customers can rapidly scale up or scale down their resources, if need be.
• Measured services: The services used are metered. Customers pay only for what they use, for
the duration that they use it.
• Broad network access: Allowing a range of devices, including both desktop and mobile, access
to applications and data stored in the cloud in private cloud implementations.
There are several benefits to cloud computing models. One of the biggest is that paying for access
to cloud computing resources enables businesses to bypass the up-front costs associated with buying
space for a computer room, buying computer and network hardware, and paying IT support staff to
set up and maintain that computer hardware.
Lesson 4: Processing IoT Data | Topic B
Item Description
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 157
Cloud Service Models
Figure 4-3: Cloud service models.
Because there are so many different cloud service offerings, it can be helpful to consider them in
terms of the general type of service they provide.
Software as a Service
(SaaS)
Platform as a Service
(PaaS)
Infrastructure as a
Service (IaaS)
Cloud Platforms
Applications such as customer relationship management software running
on the service provider's environment. The services are released in line
with the changing demands.
Platforms such as web applications, database servers, enterprise service
buses, and other middleware, with associated security mechanisms that let
you develop, test, and deploy applications.
Computer and storage resources maintained on a service provider's
environment.
Numerous cloud platforms have arisen to support IoT. The features offered by these platforms vary
widely. Examples include:
• Microsoft Azure
• Amazon Web Services
• IBM Cloud
• Oracle Cloud
• SAP Cloud Platform
• Google Cloud Platform
• Huawei FusionSphere
• Bosch IoT Cloud
• General Electric Predix
• Salesforce
Lesson 4: Processing IoT Data | Topic B
Cloud Service Models Description
158 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Concept Description
Note: It is not necessary to use a single cloud vendor. Model your solution on paper to
determine whether the benefits of using multiple cloud service providers outweigh the
disadvantages. A hybrid solution may involve services from multiple cloud platforms,
including private cloud.
Virtualization Technologies
Cloud services depend on virtualization technologies. Virtualization simulates one or more
computing devices within a physical computer. The following are various concepts related to
virtualization.
Virtual machine Virtualization technology makes it possible to run multiple instances of an
operating system configuration on the same hardware. For example, a
physical server with eight processor cores, 64 gigabytes (GB) of Random
Access Memory (RAM), and 4 terabytes (TB) of storage can be
configured with four virtual machines, which are each assigned two
processor cores, 16 GB of RAM, and 1 TB of storage.
The physical computer is the host, and the simulated devices are typically
called virtual machines (VMs).
Virtual networking Just as computing, memory, and storage resources can be subdivided into
virtual machines, network resources can be virtualized as well. For
example, if the host machine previously described included four network
interface cards (NICs), they could be virtualized so that each VM has a
single virtual NIC. Likewise, network switches can be virtualized to
enable virtual networks to be defined.
Software-defined networking is accomplished through virtualization.
Hypervisor The software or firmware that manages virtualization of resources on the
host hardware is called a hypervisor. The hypervisor provides the guest
operating systems with a virtual operating platform and manages the
execution of the guest operating systems. Different operating systems
may share the virtualized hardware resources.
Container Requirements for a particular task may not call for using an entire virtual
machine. You may only need to access an application instance.
Application containers include just enough virtual machine environment
(such as files, environment variables, and libraries) to contain an
application, hence the name container. Containerization enables you to
optimize your use of virtual machine resources when all you need to run
is an application. Docker is a common technology used for
containerization.
Guidelines for Evaluating Cloud Platforms
Follow these guidelines when evaluating cloud platforms for use in IoT.
Evaluate Cloud Platforms
When you evaluate cloud platforms for your projects, consider points such as the following.
• Services offered. Different vendors tend to have better capabilities in some areas rather than
others, so you should align the vendor's capabilities with your project's requirements. For
example, one vendor may offer superior analytics tools, but may not offer other services you
require.
Lesson 4: Processing IoT Data | Topic B
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 159
Note: When evaluating services for pricing and performance, it is critical to model how your
system will work. For example, one vendor may charge less to store data from their cloud
storage service, but may charge more to retrieve data. Whether it will be cost-effective to use
that vendor's solution depends on how frequently your solution will need to retrieve data, which
you can only determine by planning its design.
• Compatibility with other systems you're using. If your existing infrastructure is already based
on products or services from a particular vendor, it might simplify some of your integration tasks
to stick with that vendor for your IoT needs.
• The location and geographic distribution of data centers. Having data centers located near
your on-premises systems and customers may help to improve overall performance and the user
experience. Having data centers that are distributed throughout different regions will make the
system more resilient and less prone to problems like local outages.
• Pricing. Vendors price their services differently, which makes it difficult to compare costs. In
some cases, vendors offer a pricing calculator, which can help you estimate your costs by
modeling the services and usage you think you'll require.
• Reliability and service level agreements. Evaluate the level of service the vendor will
guarantee through a service level agreement (SLA), a contract that ensures a certain level of
service will be guaranteed, and which may specify consequences if that level of service is not met.
The service level might include measures such as guaranteed uptime (e.g., 99.9 percent), data
rates, and throughput. The SLA identifies repercussions or responsibilities if the service level is
not maintained.
• Security and compliance. Since you'll ultimately be responsible to ensure the security and
compliance of solutions you put into place, work with a cloud service provider that provides
effective tools to facilitate cybersecurity and compliance.
• Programming platforms and interfaces. Open programming platforms and interfaces will
help you avoid vendor lock-in. Also, using prevalent tools will make it easier to find technical
talent to implement solutions.
Cloud vendors may offer a free tier, which you can use to evaluate their service offerings.
IoT Services Performed in the Cloud
Various IoT services are typically performed in the cloud, as shown here.
Lesson 4: Processing IoT Data | Topic B
160 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Figure 4-4: IoT services typically performed in the cloud.
Using cloud services for these tasks provide the ability to scale up and down as needed to handle
varying demands and changing requirements. Trying various iterations of cloud service
configurations enables you to evolve and optimize the system quickly, without extensive investment
in hardware and infrastructure.
Device Registry and Management
Managing IoT devices registered with the system is another service that may be performed by cloud
services. IoT deployments may involve large numbers of devices the IoT system may communicate
with, control, provision, or manage, making it essential to maintain a scalable database, or registry,
of devices. Various applications throughout the IoT system may need to refer to this information,
making the cloud a natural place to store it.
To manage the registry, some sort of device management system should be implemented through
which administrators can securely and reliably provision devices, manage and control them remotely,
update their software, and monitor them through logging and other features. The device
management function may communicate with devices through a management agent running on each
device.
Data Management
IoT devices may generate large quantities of real-time data. While initially this data may be stored
and processed by an IoT gateway, it is typically moved to cloud storage where it can be integrated
with processes and applications that are part of the IoT system. Since data can become voluminous,
the data store must be elastic and scalable, making cloud storage a good choice for data
management. Different cloud storage services provide different costs, capabilities, performance, and
limitations, so it is common to use multiple storage services to meet project requirements. Big data
tools provided by cloud services are appropriate for the analysis of the vast quantities of data
produced by IoT.
Lesson 4: Processing IoT Data | Topic B
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 161
IoT Data Analytics Tools
Data analytics tools can analyze historical data to identify patterns that can predict actions that will
produce good results in the future. Analyzing complex data such as video and audio requires
extensive processing capabilities. Using cloud services for this task enables large amounts of
computing capability to be provisioned on demand.
An IoT data analytics capability includes:
• Streaming analytics: Data analysis may be time-sensitive, so data needs to be received and
analyzed from a variety of inputs in real time. As data arrives, it may be processed immediately
through streaming analytics, and it may also be forwarded to a data warehouse for deeper
background analysis.
• Predictive Analytics: To perform predictive analytics, the system extracts information from
existing datasets to determine the current state, identify patterns, and predict future trends. It
provides a predictive score for the concept being evaluated (such as whether a particular
component is likely to fail before the next scheduled maintenance, how many vehicles are likely
to cross a particular bridge in the next three minutes, and so forth). Decisions can be informed
based on the score.
• Content analytics: Although structured data comes from sources like IoT sensors, databases,
online transactions, and so forth, companies may also produce a large amount of unstructured
data, such as audio, video, office documents, text messages, email, chat messages with customers,
and so forth. Natural language analysis of such unstructured content may help to uncover
patterns and trends that can also serve as a source for data analytics.
• Visualization and reporting: Tools such as statistical graphics, charts, dashboards, and
summaries enable users to view summarized data and predictions in a form that helps to reveal
insights resulting from data.
• Analytics data repository: Data may come from a variety of new and legacy sources, such as
IoT devices, streaming analytics, enterprise applications, enterprise data, and reference data. Data
may be structured or unstructured. An analytics data repository may provide a variety of tools,
such as:
• Exploration and archiving: Tools should be provided to manage the massive data sets in
the repository, enabling it to be explored, augmented, and archived.
• Interactive analysis and reporting: The system should provide tools to view and report on
data in large data sets to answer immediate business and operations questions.
• Data catalog: To facilitate effective data analytics, the system should provide a consolidated
view of all information being used for the purpose of data analytics, called a data catalog.
The catalog is produced through data curation by data professionals, and may be enhanced
through automated cataloging tasks performed by the system, making it easier for analysts to
find the data they need, ensure data isn't duplicated or incorrect, and adhere to privacy
statutes and other compliance requirements.
Following are examples of predominant data analytics tools.
• SAP Analytics Cloud
• Microsoft Azure Stream Analytics
• AWS IoT Analytics
• IBM Watson IoT Platform
• Cisco Data Analytics
• Oracle Stream Analytics and Edge Analytics
Components of a Real-Time Analytics System
A number of capabilities are required to implement real-time analytics.
Lesson 4: Processing IoT Data | Topic B
162 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Messaging A messaging system should be able to provide a low latency, high
throughput, fault tolerant publish and subscribe pipeline between content
sources and analytics tools.
Streaming It should be possible to process incoming data as it arrives from a wide
variety of data streams.
Scalable database Due to the large quantities of data likely to be processed, the database
should be highly scalable, geared toward big data applications.
Commercial systems geared toward real-time analytics are available. Alternatively, a system could
also be constructed through free open source tools. For example, a real-time data pipeline could be
implemented using Apache Kafka for messaging, Apache Spark Streaming for real-time streaming
analytics, and Apache Hadoop or Apache Cassandra as the database.
While the solution may involve different software components, due to the demanding performance
requirements of any real-time system, you carefully plan where you deploy and how you integrate
the various capabilities. For example, eliminate network communication by locating the streaming
and database capabilities on the same host.
AI
Artificial intelligence (AI) is a technology that enables computer systems to mimic cognitive
functions normally associated with human intelligence, enabling computers to perform tasks that
would normally be done by a human, such as making decisions, recognizing a visual stimulus
(computer vision), responding to the spoken or written word (natural language processing), and so
forth. In IoT systems, AI capabilities enable computers on the edge (such as IoT gateways) and in
cloud services to automate such tasks with little or no intervention by people, or to perform them
on a scale that would not be practical to perform if human personnel were required.
For example, AI can be used to monitor the huge amounts of data being produced by IoT sensors,
and to notify human operators when a problem pattern is identified. Or AI functions might be
programmed to directly trigger an immediate, appropriate response when there is insufficient time to
involve human intervention.
One application of AI is machine learning (ML), through which a computer system can adapt its
behavior or outputs based on interactions with data, human data analysts, and other entities. With
Internet-connected systems, the information sources that inform machine learning do not
necessarily need to be local, so edge devices such as sensors can provide data that informs machine
learning in cloud services.
Traditionally, computers have been programmed directly to perform certain types of data analysis
tasks. However some types of tasks are not practical to implement solely through code, but can be
learned effectively through AI. As with human learners, artificial intelligence improves with
experience. Using a process called deep learning, AI can be trained through exposure to large
historical datasets to identify patterns, and improve its predictive analysis capabilities. Through this
process, predictive analysis capabilities of AI have been trained to meet or exceed the capabilities of
human analysts in tasks such as transcribing audio, predicting crop yields, and diagnosing illness.
AI Tools
Artificial intelligence is a rapidly developing field, and deep learning products have improved
dramatically in just a few years, making them more accessible to organizations that don't already
have AI experts on staff, although there is still a significant learning curve to using these
technologies. The following are some of the artificial intelligence tools now being used for IoT data
analytics.
Lesson 4: Processing IoT Data | Topic B
Capabilities Description
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 163
Google TensorFlow Open source framework (with some closed-source components) that uses
Python to perform numerical computations using data flow graphs.
TensorFlow can function on a wide range of systems that can run
Python, and is compatible with the TensorBoard visualization framework.
Theano Another Python-based tool that supports numerical operations
supporting efficient high performance processing on a GPU.
Caffe A C++ deep learning framework that supports the training of models
without having to write code.
Torch A library based on the Lua programming language that includes many
pre-trained models.
IBM Watson A cloud-based service that interfaces with IoT systems through a variety
of programming interfaces.
Sagemaker SageMaker, provided by Amazon Web Services, is based on Jupyter
Notebook, a Python-based machine learning service. SageMaker
streamlines much of the complexity involved in constructing, training,
and deploying machine learning models, and is optimized for use with
huge, distributed data sets.
Drawbacks of Centralized Cloud Computing in IoT
With IoT, much of the action takes place out on the edges of the network. IoT devices and sensors
acquire data. After data is analyzed, it may be determined that action is needed. Actuators on the
devices make adjustments as needed. When only the central nodes of the network (cloud services)
have the capability to store, process, and analyze data, you've committed to a design that requires
extensive, long-distance communication.
Problems with this design include:
• The time delay in moving data from where it is acquired to where it is processed.
• The time delay in communicating commands from where data is analyzed to where actions are
effected to make appropriate adjustments.
• High volume and frequency of data communications between edge devices and the cloud
computing "core."
Lesson 4: Processing IoT Data | Topic B
Item Description
164 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Note: Unfortunately, these terms mean different things to different people, and are not always
used precisely the same way. Always consider the context when interpreting these terms to
ensure you understand the underlying points being made about them.
Where to Perform Processing Tasks
Figure 4-5: Different processing tasks are performed throughout the continuum, spanning cloud
services, the edge, and devices.
While recent IT trends have shifted data processing and analytics tasks to the cloud, with IoT, many
organizations are opting to perform many of these tasks out at the edges of the network. This is part
of a trend called fog computing or distributed cloud computing. The objective of fog
computing is to provide better quality of service, make more efficient use of bandwidth by
processing data where it's needed, improve user experience, and minimize exposure of sensitive data
to security and privacy risks.
Fog computing builds upon the edge computing concept, distributing cloud computing, storage,
control, and networking functions closer to where the effort is needed. It distributes cloud services
throughout the continuum between cloud, edge, and devices. The term "fog" is used because data
processing clouds may be formed close to "ground level"—near sensor networks, IoT devices, and
end users. Private clouds and IoT gateways may be used for this purpose.
Complementing the fog computing effort, some processing may be performed directly on IoT
devices and client applications. Processing that occurs here is sometimes called mist computing.
When you determine where processing should occur in an IoT system, there are tradeoffs to
consider. Local processing is immediate but constrained by local limitations, such as the processing
power of IoT devices. Extensive resources and scalability are provided by cloud services, but it takes
longer to send data there and aggregate it for processing.
Guidelines for IoT Processing and Analytics
Follow these guidelines when planning how to distribute IoT processing tasks.
Lesson 4: Processing IoT Data | Topic B
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 165
Determine Where Processing Should Be Performed
The following examples describe various types of processing tasks and identify where such tasks
might typically be performed within an IoT architecture.
Task Edge Devices Fog Cloud
Read data from sensors x
Control actuators x
Validate raw sensor data x x
M2M communication x x
Initial data validation, filtering, and
normalization
x x
Aggregate data from individual sensor devices
within a single sensor network
x
Process real time data (shallow data analysis and
quick decision-making and response)
x x
Aggregate batches of data forwarded from
various sensor networks
x
Process big data (deep data analysis, in-depth
decision-making and analysis)
x
Data warehousing x
Business logic x
Reporting and visualization x
Lesson 4: Processing IoT Data | Topic B
166 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Scenario
As you continue to prototype your IoT devices, you're also planning the infrastructure that will
support them—networks, gateways, servers, and so forth. Here's a summary of the system as you
currently envision it:
• You will start with approximately 300 rechargeable battery-powered sensor devices, most of
which will be mobile, located in growing trays and on carts that may be moved throughout the
greenhouse.
• The sensor devices will attach to the network through Wi-Fi.
• Each greenhouse has its own Wi-Fi router, which is attached to the farm's network through
cabled Ethernet.
• A program running on a computer located on premises will obtain data from the sensors, analyze
that data, and send commands to separate actuator devices throughout the farm to adjust
lighting, air flow, heating and cooling, irrigation, and other factors as needed.
• You also intend to forward the sensor data, along with other data you manually update in a
database application, to a cloud-based application that will perform analytics to help Greene
Organix determine the most profitable time to grow specific crops, improve its agricultural
practices, and so forth.
1. How would you transport sensor data to the cloud application?
2. How would you decide which processing tasks should be performed locally,
and which tasks should be performed in cloud services?
Lesson 4: Processing IoT Data | Topic B
ACTIVITY 4-4
Planning IoT Processing
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 167
TOPIC C
Provide M2M Communication
Processing IoT data involves moving data from one machine to another, and machines do not have
the same communication requirements as people.
Industrial M2M Communication
Industrial machine-to-machine (M2M) communication typically involves the Supervisory Control
and Data Acquisition (SCADA) control system architecture, which is typically used in industrial,
energy, and utility companies. While originally a monolithic system, SCADA has evolved to become
a standard for data acquisition and control so process automation and control products from
different manufacturers can interoperate effectively. SCADA systems now support multiple plant
locations, with remote access over a wide area. Systems using SCADA acquire data from sensors,
and provide a graphical user interface (also called a human-machine interface (HMI) (in the
context of industrial control system). The user interface presents data graphically, provides
notifications, alarms, and other functions. On-screen controls enable users to control valves, pumps,
motors, and other devices as well.
IoT M2M Communication
Familiar web technologies such as Transmission Control Protocol (TCP) and Hypertext Transfer
Protocol/Hypertext Transfer Protocol Secure (HTTP/HTTPS) have been leveraged for M2M
communication in business and consumer IoT. Some new technologies have also been developed,
and some technologies have been borrowed from other domains. While it is possible to implement
M2M communication using only traditional web technologies, they are not always a good fit. The
web was originally designed for machine-to-person communication, not machine-to-machine
communication.
For example, TCP supports acknowledgment of transmitted packets, retransmission of lost packets,
and flow control. These are appropriate when a live customer is at the other end of the
transmission, waiting to consume the information being sent. Likewise, HTTP/HTTPS is based on
a two-way request-response model that makes sense for human users interacting live with a website.
But a more streamlined one-way messaging approach may be more appropriate for machine-to-
machine communication, in which communication must occur quickly and with little overhead.
These features come with overhead in terms of time, processing cycles, and network bandwidth,
which may not be necessary in machine-to-machine communications.
The much more efficient User Datagram Protocol (UDP) may be more appropriate for many M2M
communication tasks. If data does not arrive at its destination in time, it may no longer be relevant,
so retransmission would be pointless anyhow. And a one-way messaging protocol might be more
appropriate than the two-way HTTP/HTTPS request-response model.
Communication Patterns
Identifying the prevalent communication pattern can help you evaluate what communication
protocols are most appropriate for the situation. A request-response protocol like HTTP/HTTPS
might be appropriate for two-way communications, while a publish-subscribe protocol might be
more appropriate for ongoing, one-way status updates, particularly when timing is critical and when
constrained devices are involved.
Lesson 4: Processing IoT Data | Topic C
168 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Telemetry
Notification
Inquiry
• Message Originator: IoT device
• Data Flow: One direction, from the IoT device to the external entity.
• Data sent by the IoT device: The IoT device sends data such as
sensor readings to another entity, such as an IoT gateway or cloud
services.
• Message Originator: External entity
• Data Flow: One direction, from the external entity to the IoT device.
• Data sent by the external entity: The external entity, such as a web
services provider, provides the IoT device with data it would not
otherwise have, such as weather data for local presentation to the user,
news, pollution alerts, and so forth.
• Message Originator: IoT device
• Data Flow: Data flows in two directions, with the request sent away
from the device, and a response returned to the device.
• Request sent by the IoT device: The IoT device requests that an
external entity provide data or perform an action.
• Response returned by the external entity: May result in either:
• Return of data (as in a database lookup)
• Activity being performed by the external entity, such as:
• Change the state of something in the cloud or in the physical
world, such as a request by a smartwatch to change the heat
setting on a thermostat at home, submitting an order for a
product when a dash button is pressed, and so forth.
• If the inquiry was a subscribe request, starting to send
notifications to the IoT device.
Lesson 4: Processing IoT Data | Topic C
Pattern Description (from the perspective of the IoT device)
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 169
Command
• Message Originator: External entity
• Data Flow: Data flows in two directions, with the request initiated by
the external entity, and a response returned by the IoT device.
• Request sent by the external entity: The external entity requests
that the IoT device provide data or perform an action.
• Response returned by the IoT device: May result in:
• Return of data, such as requesting that the IoT device send the last
sensor reading.
• Activity being performed by the IoT device, such as:
• Change the state of the device, perhaps with effects in the
physical world such as changing the heat setting on a
thermostat, turning on a light, and so forth.
• Starting to send telemetry data as an ongoing stream.
Messaging Protocols
While a variety of communication and messaging protocols can be used for IoT M2M
communication, several protocols have emerged to serve this role, including:
• AMQP
• CoAP
• MQTT
AMQP
Advanced Message Queuing Protocol (AMQP) is an open standard publish-and-subscribe
protocol. AMQP was designed for JPMorgan Chase in 2004, for use in the financial industry.
AMQP provides a rich set of features related to messaging, including reliable queuing, topic-based
publish-and-subscribe messaging, flexible routing, transactions, and security. AMQP runs over TCP,
providing reliable (acknowledged) delivery. Messages are encrypted through transport layer security.
A message broker controls communication services that are used by publishers and subscribers. The
message broker has two main components: the exchange and queues. The exchange receives
messages from publishers and distributes them to various queues based on pre-defined roles and
conditions. A queue holds communication on a particular topic. Subscribers subscribe to a particular
queue in order to be notified when publishers send any message to that queue.
MQTT
The Message Queuing Telemetry Transport (MQTT), also known as the ISO/IEC 20922
standard, is another publish-and-subscribe messaging protocol. It is much simpler and more focused
than AMQP, and is often used for IoT data communication because it is well suited to operate with
devices that have limited memory, processing power, and network bandwidth, which describes many
IoT devices.
Lesson 4: Processing IoT Data | Topic C
Pattern Description (from the perspective of the IoT device)
170 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
MQTT:
• Operates on top of UDP, so the network doesn't provide reliability (acknowledged delivery) or
encryption through Secure Sockets Layer and Transport Layer Security (SSL/TLS).
• Provides its own reliability (quality of service) capabilities.
• Does not provide encryption. (The communicating applications must provide their own
encryption.)
• Provides a very simple programming interface that is well suited for communication with
lightweight remote devices, requiring little processing power and network bandwidth to operate
as a client.
• Enables communicating devices (publishers and subscribers) to be separated by distance and
time.
• Does not require communicating devices to communicate directly with each other or even know
each other's identity to communicate.
These objectives are accomplished through a scheme that involves a third party, an MQTT broker,
which facilitates communication. The MQTT broker processes all incoming messages and
distributes them to the appropriate subscribers based on various filters, including message subject,
content, and type or class of message.
Premises Underlying MQTT
While MQTT is well suited for IoT, it is important that developers be aware of premises underlying
the protocol. For example:
• The publisher can't assume that any subscriber is actually listening to messages that are sent.
• Both publisher and subscriber must know how data is structured beforehand.
• With subject-based filtering, both parties must know which topics will be used.
• Publishers and subscribers must know the hostname/IP address and port of the broker through
which they communicate.
• While communications can be described as "near real time," they don't need to be particularly
fast, and may actually take seconds to complete.
In spite of the word "queuing" in its name, MQTT (unlike AMQP) is not actually a messaging
queue. It is more like a broadcast. There may be multiple subscribers (possibly thousands)
consuming the published data, and MQTT does not "queue" or hold on to messages until they are
received by a client. MQTT offers three qualities of service:
• QoS 0: At most once delivery: This default QoS doesn't guarantee message delivery. This level
minimizes communication and is used in situations where the network is reliable and it is
acceptable to occasionally lose messages, such as a sensor feed that publishes readings frequently,
where missing some readings will not have an adverse impact on the system.
• QoS 1: At least once delivery: Ensures that each receiver gets message at least one time, with
the inconvenience of perhaps receiving the same message multiple times. The subscriber sends a
message back to the broker to acknowledge receipt. The subscriber must provide its own logic to
deal with the possibility of receiving duplicate messages.
• QoS 2: Exactly once delivery: Delivers the highest quality of service, where neither the loss nor
duplication of messages is considered acceptable. This comes at the cost of increased overhead,
make it the slowest of the three levels.
MQTT Communication Process
MQTT communication between clients (publishers and subscribers) is accomplished through a third
party, the broker. MQTT clients vary widely in capability and purpose. They might be a tiny IoT
device consisting of not much more than a Wi-Fi connected microcontroller and a sensor or two up
to a high-powered server used to process big data. In any case, the client is running an application
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 171
written in a language like C, C#, C++, Java, JavaScript, or Go that includes a software library
providing the support needed to process MQTT communication.
The broker provides the hub through which communication is accomplished. Publishers send all
messages to the broker. Subscribers receive all messages from the broker. No two clients are ever in
direct communication. The broker filters all incoming messages, determines which subscribers want
to receive them, and forwards the message on to those subscribers. If authentication and
authorization are used, the broker manages these processes for all clients.
Message Persistence
Depending on your needs, you might set up an IoT messaging system to support only direct
messaging in which the receiver must be online in order to receive messages. This approach is
commonly used for telemetry data.
Alternatively, you might set up a messaging system to hold messages for a time in a cache or queue.
If a device is offline, it will still receive the messages when it comes back online. The device could
then service the messages in the order in which they were sent.
MQTT provides an option to enable or disable message queuing, depending on the requirements for
your application. Of course, this adds some complexity to the system. Often you can configure
messages so they remain viable for a limited time. A time to live (TTL) parameter can be placed on
the message to prevent old messages from being acted on when a device comes back online long
after the message was originally sent.
Web-Based Communication
While lightweight messaging protocols such as MQTT are often recommended for communication
involving constrained devices, traditional communications based on HTTP/HTTPS may
nonetheless be appropriate for other parts of the IoT system. HTTP/HTTPS is well known, and is
well supported for software development through existing protocol stacks and development tools.
In situations where latency and bandwidth are not an issue, and when the devices involved have the
processing power to implement them, web protocols are a good choice for IoT.
Representational State Transfer Application Programming Interfaces (REST APIs), Constrained
Application Protocol (CoAP), and WebSocket are web-based communication methods currently
useful for IoT, and new standards are under development that will improve web protocols for use in
IoT. For example, HTTP/2 will provide more efficient data formats and communication methods.
REST APIs
Over the years, a variety of approaches have been developed to communicate with web servers as an
alternative to the original conventions of HTTP. Representational State Transfer (REST)
Application Programming Interfaces (APIs) prescribes a pattern of communication between
web clients and servers that proponents of REST say uses HTTP as it was originally envisioned.
For example:
• GET requests are issued to look up information.
• PUT requests are issued to change information.
• POST requests are issued to create information.
• DELETE requests are issued to delete information.
REST is popular because it is simple to program, and builds upon existing systems and features of
HTTP. Features such as encryption and reliability are easily implemented using well-known
protocols such as SSL/TLS and TCP, with which most developers are already familiar, and HTTP
can be programmed through almost any modern application programming language. For example,
requesting a sensor reading might be as simple as issuing a GET request on a URL such as
https://192.168.1.67/sensors/temperature.
Lesson 4: Processing IoT Data | Topic C
172 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Note: Note that WebSocket does not define a messaging format. If you use WebSocket, you
need to provide your own protocol for the transmission of data. Some IoT developers use
MQTT, running over WebSocket, for messaging. However, this diminishes some of the benefits
of using MQTT directly over UDP, as WebSocket requires that a web client protocol stack be
implemented on the device.
A REST request can return various types of documents, such as simple text or structured formats
like XML, JSON, and CBOR.
WebSocket
Web technologies were designed based on the assumption of a client-server architecture. Users issue
requests through a web browser, and a server provides a response. Clever techniques such as AJAX
(asynchronous JavaScript and XML) were developed to enable two-way communication without
having to load an entire web page, but this approach still assumed that clients initiated
communication in the request-response model.
The WebSocket protocol, which operates over TCP, has been introduced to address these
shortcomings. WebSocket is part of the HTML5 specification. It was standardized by the Internet
Engineering Task Force (IETF) in 2011, and is currently supported in all of the major web
browsers, including Google Chrome, Microsoft Edge, Internet Explorer, Firefox, Safari, and Opera.
Various programming libraries have been developed, enabling programmers to use WebSocket
directly in their own applications. To use WebSocket, the server must also support the protocol.
Server support can be added through a web application platform such as Node.js.
With WebSocket, once both parties (client and server) agree to establish a persistent two-way
connection, either the client or server can send messages at any time. The overhead of a full HTTP
request is not incurred, so the communication is much faster than conventional HTTP
communication. Communication is done over TCP port number 80 (or 443 in the case of TLS-
encrypted connections), which is useful in environments that block non-web Internet connections
using a firewall.
Like other protocols (such as MQTT and AMQP), this efficient approach is well suited for IoT
communication, particularly when they just involve the periodic sending of sensor readings.
CoAP
Constrained Application Protocol(CoAP) provides a lightweight REST API designed for low-
power web clients, such as IoT devices. CoAP runs over UDP instead of TCP, which is normally
used for REST APIs. CoAP provides a lightweight mechanism for acknowledged delivery. CoAP
has two main components: messaging and request-response. The messaging component is
responsible for reliability and duplication of messages while the request-response component is
responsible for communication. CoAP can run over Datagram Transport Layer Security
(DTLS) to encrypt communications.
CoAP shares many similarities with HTTP/HTTPS, for easier integration with web applications,
and making it easier for developers familiar with HTTP/HTTPS to work with CoAP. Like HTTP/
HTTPS, CoAP uses GET, PUT, POST, and DELETE requests to retrieve, create, update, and
delete, respectively, and CoAP can carry different types of payloads, such as XML, JSON, CBOR,
and other data formats.
Guidelines for M2M Communication
Follow these guidelines when planning and implementing M2M communication for IoT.
Construct an MQTT CONNECT Message
A minimal MQTT CONNECT message includes the following fields of data:
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 173
• ClientId: Provides each MQTT client connecting to a MQTT broker with a unique name.
• Clean Session: Set to false, and the broker will store all subscriptions for the client and all
missed messages, when the subscriber specifies a QoS of 1 or 2. Set to true, and the broker won't
store anything for the client and will also purge all information from a previous persistent
session.
• Username/Password: MQTT can be configured to authenticate the client to authorize access
to various topics. However, the password is sent in plaintext unless it is encrypted or hashed by
the application, or run over a secure transport layer. Some brokers support using an SSL
certificate for authentication, in which case no username and password is needed.
• Will Message: MQTT can be configured to automatically notify other clients when a client
disconnects ungracefully (due to power loss, for example). The message and topic that will be
sent for this notification are specified in this field.
• KeepAlive: This field specifies the time interval after which the client will send a ping message
to the broker to keep the connection alive. This mechanism enables the client and broker to
determine if the other machine is still alive and reachable.
Evaluate an MQTT CONNACK Message
An MQTT CONNACK message includes two fields of data, which the client can evaluate to
determine if connection to the broker was successful:
• sessionPresent: Indicates whether the broker already has a persistent session with the client
from a previous interaction. If a client connects and has set CleanSession to true, this flag will
always be false, because no session is available. If CleanSession is set to false, the flag value
depends on whether session information is available for the ClientId. If stored session
information exists, then the flag is true. Otherwise, it is false.
• returnCode: The value returned here indicates either a successful attempt or the reason why it
was unsuccessful:
• 0 - Connection Accepted
• 1 - Connection Refused, unacceptable protocol version
• 2 - Connection Refused, identifier rejected
• 3 - Connection Refused, Server unavailable
• 4 - Connection Refused, bad user name or password
• 5 - Connection Refused, not authorized
Select an M2M Communication Protocol
The HTTP/HTTPS web protocols and REST are well known and widely implemented technologies
for web and cloud services. Although they have many applications in IoT, there are some situations
for which they are not well suited, and other protocols may be more appropriate. For example:
• Information publishing: HTTP was designed to support a request-and-response model, where
the data exchange is initiated by the requester. However, many IoT scenarios work best with a
publish-and-subscribe model, where a publisher (such as an IoT sensor device) initiates the
exchange of data by publishing it to a particular topic. Any clients that have subscribed to that
topic are then notified that new data has been sent.
• Distributing very small packets of data in huge quantities efficiently: Some IoT
applications require large numbers of sensing devices, which typically deliver very small chunks
of data. The containing headers for HTTP may easily outsize the tiny portions of data an IoT
device may send, making it an inefficient format for sending small packages. This inefficiency is
magnified by the number of devices sending data and the frequency with which they send data.
• Real-time delivery of data: When small portions of data must be delivered in real time, it is
essential to keep communication tasks streamlined and fast. It requires a lot of processing to
compose an HTTP request or response, particularly when encryption is involved. Simpler
protocols may be more appropriate in some circumstances.
Lesson 4: Processing IoT Data | Topic C
174 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
• Minimal consumption of battery power: Simpler protocols help to conserve battery power on
IoT devices by reducing the processing tasks they must perform.
Case Study: MQTT
Figure 4-6: MQTT publishers and subscribers.
In the Greene Organix greenhouses, separate IoT devices provide sensors and actuators.
Sensor devices use MQTT to publish their sensor data, which other clients on the local network
may subscribe to.
The local IoT gateway performs local data collection and local command and control tasks.
• Local data collection: The IoT gateway subscribes to sensor topics to obtain the sensor data. It
runs a background process to filter and aggregate the data, which it periodically forwards to
cloud services.
• Local command and control: The IoT gateway also runs a process to perform command and
control operations on devices within the greenhouses, controlling lighting, air flow, heating and
cooling, and irrigation. Based on the data it collects and analyzes, it publishes messages to
identify actions that need to be taken to adjust these factors.
• Actuator devices subscribe to topics pertaining to actuator commands. They respond to
commands that pertain to them, making adjustments to the various factors under their control.
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 175
Note: Your instructor may direct you to enter a different path, depending on
where your course data files were set up.
Before You Begin
The course data files are installed on your computer in C:095024Data. These files include a
customized version of the Arduino IDE (located in C:095024DataToolsarduino), which
contains example programs that you will use in this course. The example programs are contained in
the File > Sketchbook sub-menus inside the Arduino IDE.
The ESP8266 microcontroller is connected to a USB port on the development computer. You have
successfully connected the device to the Wi-Fi network, and the SSID and password are now cached
on the device. You have installed the light sensor and 10K resistor on the breadboard. The Arduino
IDE is not running.
Scenario
You are going to program the IoT device to communicate using MQTT messaging. But before you
modify the device, you will experiment with the MQTT broker and command-line clients to see
how it operates and demonstrate it is working effectively.
1. View the files in the Mosquitto application directory.
a) Select the Windows Start button to show the Start menu.
(Or from the keyboard, you can press Ctrl+Esc.)
b) Type cmd and press Enter to open a Command Prompt console.
c) In the command console, enter the cd c:095024DataToolsmqtt_broker command to
switch to the directory where the Mosquitto MQTT broker is stored.
Lesson 4: Processing IoT Data | Topic C
ACTIVITY 4-5
Launching and Testing an MQTT Broker
176 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
d) Enter the dir /w command to list files in the directory.
This directory contains various Windows executable files. In this activity, you will use:
• mosquitto.exe: An MQTT broker application
• mosquitto_sub.exe: An MQTT publisher client
• mosquitto_pub.exe: An MQTT subscriber client
e) Enter the title Broker command to give the Command Prompt window a title.
In this activity, you'll open three more Command Prompt windows. Titling your windows will help you
to keep them straight. You'll open one window to use as a message publisher, and you'll open two
other windows to use as message subscribers. Normally, the broker and clients would be running on
different computers or devices, but this will enable you to test how MQTT works.
2. Open and arrange three more Command Prompt windows.
a) In the Broker window, enter the start "Publisher" cmd command to start another Command
Prompt console.
• When you use the start command to open another command console this way, it opens at the
directory you were in (c:095024DataInstallmqtt_broker), saving you the extra step of having to
enter the cd command again.
• You also titled the window as you opened it.
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 177
b) Arrange the two windows on the left half of your desktop, with Broker on top and Publisher on
bottom, as shown.
c) In the Publisher window, enter the start "Subscriber One" cmd command to start another
Command Prompt console.
d) In the Subscriber One window, enter the start "Subscriber Two" cmd command to start
another Command Prompt console.
e) Arrange the two subscriber windows on the right half of your desktop, with Subscriber One on top
and Subscriber Two on bottom, as shown.
3. Show help for the mosquitto command.
Lesson 4: Processing IoT Data | Topic C
178 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Note: This is a network application. If you are prompted with a Windows
security alert, check the type of network you are using, and select Allow
access.
a) In the Broker window, enter the mosquitto --help command to view help on using the mosquitto
command.
• The command accepts several optional parameters.
• The -v parameter displays all logging information in the console window, which is useful for
testing MQTT messaging.
4. Run the MQTT broker application.
a) In the Broker window, enter the mosquitto -v command to launch the Mosquitto broker in
verbose mode.
• Mosquitto launches using the default configuration.
• In the default configuration, it listens for messages from clients on port 1883.
• You are not returned to a command prompt because the Mosquitto broker is still running. It is
now waiting for messages from publishers and subscribers.
5. Subscribe to two MQTT topics.
a) In the Subscriber One window, enter the mosquitto_sub -t building/light -t
building/temperature -v command.
• For this test, you will use the default IP address (localhost) and port (1883), so you don't need to
specify them here.
• You will subscribe to two topics: building/light and building/temperature.
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 179
b) Observe the Subscriber One console (where you just entered the command to subscribe).
The mosquitto_sub command seems to have hung, since no response is shown. However, it hasn't
hung. It is just waiting to receive the messages it has subscribed to.
c) Observe the Broker window.
The broker updated its display with several log entries as soon as you subscribed.
• The broker responded to the connection request by sending a CONNACK message back to the
subscriber.
• The broker acknowledged the two SUBSCRIBE requests by returning a SUBACK message for
each one.
• The subscriber sends periodic PINGREQ ping request messages to keep the connection alive,
and the broker responds to each one with a PINGRESP message.
6. Publish to an MQTT topic.
a) In the Publisher window, enter the mosquitto_pub -t building/temperature -m "68.0"
command.
• You are publishing to the MQTT broker using the default address of 127.0.0.1 (on the local
computer) on port 1883.
• The topic is "building/temperature".
• The message is "68.0".
• In a real-world implementation, the publisher might be an IoT device posting the latest
temperature reading from its sensors.
Lesson 4: Processing IoT Data | Topic C
180 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
b) Observe the Broker console.
• The broker responded to the connection request by sending a CONNACK message back to the
publisher.
• The publisher sent a PUBLISH request to send the message using the topic "building/
temperature".
• The broker received the PUBLISH request, and then sent the message to the subscriber. (In
actual use, the message might have multiple subscribers.)
• The publisher then disconnected.
c) Observe the Subscriber One console.
The subscriber has received the message, and shows the value that was sent.
In a real-world implementation, the subscriber might be a storage system that is collecting the data
for later analysis.
7. Launch a new subscriber client.
a) In the Subscriber Two window, enter the mosquitto_sub -t building/temperature -v
command.
b) Observe the Broker window.
The broker acknowledged the new connection request and subscription to the building/temperature
topic.
c) Observe the Subscriber Two window.
The client is waiting for messages. A building/temperature message had been sent earlier, but by
default old messages aren't retained and sent to new subscribers, so the message you sent to
building/temperature is not received by the new client.
8. Publish messages on different topics.
a) In the Publisher window, enter the mosquitto_pub -t building/temperature -m "72.0"
command.
b) Observe the Subscriber One and Subscriber Two windows.
Both subscribers receive the message, and show the new value (72.0).
c) In the Publisher window, enter the mosquitto_pub -t building/light -m "100" command.
d) Observe the Subscriber One and Subscriber Two windows.
Only Subscriber One received the message. Subscriber Two did not subscribe to the building/light
topic.
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 181
9. Which communication pattern is used by the Publisher (sensor device) in this
scenario—telemetry, notification, inquiry, or command?
10.Why is a publish-subscribe model appropriate for this scenario?
Lesson 4: Processing IoT Data | Topic C
182 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Note: You will need to refer to this address several times in the activities that
follow.
Before You Begin
The course data files are installed on your computer in C:095024Data. These files include a
customized version of the Arduino IDE (located in C:095024DataToolsarduino), which
contains example programs that you will use in this course. The example programs are contained in
the File > Sketchbook sub-menus inside the Arduino IDE.
The ESP8266 microcontroller is connected to a USB port on the development computer. You have
successfully connected the device to the Wi-Fi network, and the SSID and password are now cached
on the device. You have installed the light sensor, a 10K resistor, and a DHT11 sensor on the
breadboard. The Arduino IDE is not running.
In separate command-line consoles, you have set up an MQTT broker, publisher, and two
subscriber clients.
Scenario
Now that you've set up and tested the MQTT broker, you'll enable the IoT device to use it to
publish sensor data.
1. Load a sketch that publishes data values using MQTT.
a) From the Windows Start menu, select Arduino to start the Arduino IDE.
b) Select File→Sketchbook→Processing IoT Data→MQTT_Publish.
c) Close the sketch that is still open in the background so MQTT_Publish is the only sketch that is
open.
2. Set the MQTT server address.
a) In the Publisher command prompt window, enter the ipconfig command and note the IPv4
address your MQTT broker is running on. Write your computer's IP address in the space provided.
My IP Address: .
b) Revise line 9 to replace YourName with your name.
c) In line 10 of MQTT_Publish, change the IP address to match yours.
3. Start the process to compile and upload the sketch.
Lesson 4: Processing IoT Data | Topic C
ACTIVITY 4-6
Using MQTT to Send Data from an IoT Device
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 183
a) Select Tools→Serial Monitor to display the serial monitor window, and ensure the baud rate is set to
9600 baud.
The previous program is still running on the IoT device. For now, you can ignore the output shown in
the serial monitor.
b) Select Sketch→Upload.
4. While the sketch is being compiled, examine the source code.
a) Examine code that has been added to support MQTT.
• The IP address and port of the MQTT server are identified in lines 10 and 11.
• An object that will be used to program the MQTT communication is created in line 17. This object
uses the software library identified in line 1.
b) Examine the code that publishes the sensor data to various topics.
Lesson 4: Processing IoT Data | Topic C
184 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
Note: Many ESP8266 boards have an LED light that flashes when the device
is communicating over Wi-Fi.
5. Examine how the MQTT broker and subscribing clients process data
published by the remote sensor device.
a) When the program is uploaded and running on the device, examine the Subscriber One and
Subscriber Two command consoles.
Subscriber One is receiving temperature and light values from the remote sensor device, and
Subscriber Two is receiving temperature values.
b) Examine the Broker command console.
The MQTT broker is processing messages received from the sensor device, and forwarding them to
subscribers.
6. Clean up the desktop.
a) Close the Subscriber One and Subscriber Two command console windows, but leave the other two
console windows open.
7. What are the benefits of minimizing the processing done by sensor devices
and having them use protocols like MQTT to forward data to subscribers?
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 185
Before You Begin
The course data files are installed on your computer in C:095024Data. These files include a
customized version of the Arduino IDE (located in C:095024DataToolsarduino), which
contains example programs that you will use in this course. The example programs are contained in
the File > Sketchbook sub-menus inside the Arduino IDE.
You have successfully connected the device to the Wi-Fi network, and the SSID and password are
now cached on the device. You have installed the light sensor, a 10K resistor, and a DHT11 sensor
on the breadboard. The Arduino IDE is running.
In separate command-line consoles, the Broker and Publisher windows are open.
Scenario
Your ESP8266 may light an on-board LED to indicate when the Wi-Fi connection is in use. You
can program this LED for other purposes. There is another LED on the board that you may also
use. These LEDs can be useful to display diagnostic information during testing and setup. When
they are not needed, it would be good to disable them. You could add a switch for this purpose, but
since a messaging interface is already available through MQTT, you'll use it to enable the LED state
to be controlled through messaging. A client app on a mobile device can be used to turn them on or
off as needed.
1. Load a sketch that publishes data values using MQTT.
a) In the Arduino IDE, select File→Sketchbook→Processing IoT Data→Control_Instructions.
b) Close the MQTT_Publish window that is still open in the background so Control_Instructions is the
only sketch that is open.
2. Set the MQTT server address.
a) Revise line 9 to replace YourName with your name.
b) In line 10, change the IP address to match the IP address of your computer, which you wrote down
earlier.
3. Start the process to compile and upload the sketch.
a) Make sure the serial monitor window is shown, and ensure the baud rate is set to 9600 baud.
b) Select Sketch→Upload.
4. While the sketch is being compiled, examine the source code.
Lesson 4: Processing IoT Data | Topic C
ACTIVITY 4-7
Using MQTT to Receive Control Instructions
186 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)
a) Examine code that has been added to process incoming MQTT messages in lines 29 through 38.
LED visibility is set based on the incoming value from the building/set_leds_visible topic.
5. After the program is compiled and uploaded, test your ability to set the LED
state through MQTT.
a) On the IoT device, examine the onboard LEDs.
They are turned off by default.
b) In the Publisher command console, enter the mosquitto_pub -t building/
set_leds_visible -m "on" command.
A message in the console indicates that the LED is now on.
c) On the IoT device, examine the LEDs.
• After the message is received and processed, the LEDs update the device status each time
sensor readings are taken.
• The LED that normally shows the Wi-Fi state (typically blue on the NodeMCU ESP8266)
illuminates when the light reading is lower than the previous reading.
• The other LED (typically red) illuminates when the light reading increases.
• When the reading is the same as before, neither LED illuminates.
6. Turn the LEDs off again.
a) In the Publisher command console, press F3 to recall the previous command.
b) Press Backspace three times and type off" to revise the command so it will turn the LEDs off.
The completed command is mosquitto_pub -t building/set_leds_visible -m "off"
c) Press Enter.
A message in the console indicates that the LEDs are now off.
d) On the IoT device, examine the LEDs.
After the message is received and processed, the LEDs stop flashing.
7. Clean up the desktop.
a) Exit the Arduino IDE.
b) Close all command consoles.
Lesson 4: Processing IoT Data | Topic C
Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 187
Note: Check your CHOICE Course screen for opportunities to interact with your classmates,
peers, and the larger CHOICE online community about the topics covered in this course or
other topics you are interested in. From the Course screen you can also access available
resources for a more continuous learning experience.
Summary
In this lesson, you used microcontroller I/O to interact with components attached to an IoT device,
and you implemented M2M communication to enable data processing tasks to be delegated to the
most appropriate location within IoT infrastructure.
What communication protocols will you use for M2M communication in your own
projects, and why?
Why are edge computing approaches so important for IoT data processing?
Lesson 4: Processing IoT Data |
190 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)

More Related Content

Similar to Input Output devices for Processing IoT Data (20)

PDF
How physical sensors in IoT work, what they can do and what they cannot.
Simple Hardware
 
PDF
IoT elements of Iot
SumitMali22
 
PPT
iotsensorpov-sensortypes fujbgtyuujjo.ppt
flowermv2020
 
PPT
chaptertaaaaaaaaaaaaaaadddddddd2222 4.ppt
tadudemise
 
PPTX
MEMS Sensors Overview
Jennifer Chin
 
PPTX
Introduction to IoT and Sensors_new.pptx
tanishqgupta1102
 
PPTX
Report on internship evaluation on topic iot
priyanjalipandey67
 
PDF
iot_module2.pdf
ranjanraj56
 
PDF
SENSORS, ACTUATORS, AND SMART OBJECTS IOT
ssuserfa1ab9
 
PPTX
Introduction to IoT. What is IoT. Notes for mba
srishtisingh1813
 
PDF
Best HW for IoT project-1.pdf
Kunal Pandhram
 
PPTX
IOT sensor actuators for different devices
ChandraPrakash715640
 
PPTX
Mobile Sensors
RAHUL KANEKAR
 
PPTX
IOT FOR DATA SCIENCE AND ANALYTICSMODULE 2
MadhurimaDas52
 
PPTX
Emerging chapter 4.pptx
AderawAlemie
 
PPTX
Introduction to Internet of Things
Sayed Chhattan Shah
 
PDF
IoT PPT.pdf
aman400979
 
PDF
IRJET- IoT based Industrial Level Sensor Data Acquisition & Monitoring
IRJET Journal
 
PPT
Emergig-Technology-ch-4.ppt
LammiiGurmeessaa
 
PDF
Lecture No. One -chapter 1 Introduction.pdf
HashSoft1
 
How physical sensors in IoT work, what they can do and what they cannot.
Simple Hardware
 
IoT elements of Iot
SumitMali22
 
iotsensorpov-sensortypes fujbgtyuujjo.ppt
flowermv2020
 
chaptertaaaaaaaaaaaaaaadddddddd2222 4.ppt
tadudemise
 
MEMS Sensors Overview
Jennifer Chin
 
Introduction to IoT and Sensors_new.pptx
tanishqgupta1102
 
Report on internship evaluation on topic iot
priyanjalipandey67
 
iot_module2.pdf
ranjanraj56
 
SENSORS, ACTUATORS, AND SMART OBJECTS IOT
ssuserfa1ab9
 
Introduction to IoT. What is IoT. Notes for mba
srishtisingh1813
 
Best HW for IoT project-1.pdf
Kunal Pandhram
 
IOT sensor actuators for different devices
ChandraPrakash715640
 
Mobile Sensors
RAHUL KANEKAR
 
IOT FOR DATA SCIENCE AND ANALYTICSMODULE 2
MadhurimaDas52
 
Emerging chapter 4.pptx
AderawAlemie
 
Introduction to Internet of Things
Sayed Chhattan Shah
 
IoT PPT.pdf
aman400979
 
IRJET- IoT based Industrial Level Sensor Data Acquisition & Monitoring
IRJET Journal
 
Emergig-Technology-ch-4.ppt
LammiiGurmeessaa
 
Lecture No. One -chapter 1 Introduction.pdf
HashSoft1
 

Recently uploaded (20)

PDF
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
PDF
Blockchain Transactions Explained For Everyone
CIFDAQ
 
PDF
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
PPTX
Top iOS App Development Company in the USA for Innovative Apps
SynapseIndia
 
PDF
July Patch Tuesday
Ivanti
 
PDF
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
PPTX
"Autonomy of LLM Agents: Current State and Future Prospects", Oles` Petriv
Fwdays
 
PDF
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
PPTX
MSP360 Backup Scheduling and Retention Best Practices.pptx
MSP360
 
PDF
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
PDF
SFWelly Summer 25 Release Highlights July 2025
Anna Loughnan Colquhoun
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PDF
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
PDF
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
PPTX
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
PDF
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
PDF
Windsurf Meetup Ottawa 2025-07-12 - Planning Mode at Reliza.pdf
Pavel Shukhman
 
PPTX
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
PDF
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
PDF
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 
Presentation - Vibe Coding The Future of Tech
yanuarsinggih1
 
Blockchain Transactions Explained For Everyone
CIFDAQ
 
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
Top iOS App Development Company in the USA for Innovative Apps
SynapseIndia
 
July Patch Tuesday
Ivanti
 
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
 
"Autonomy of LLM Agents: Current State and Future Prospects", Oles` Petriv
Fwdays
 
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
 
MSP360 Backup Scheduling and Retention Best Practices.pptx
MSP360
 
Using FME to Develop Self-Service CAD Applications for a Major UK Police Force
Safe Software
 
SFWelly Summer 25 Release Highlights July 2025
Anna Loughnan Colquhoun
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
 
Windsurf Meetup Ottawa 2025-07-12 - Planning Mode at Reliza.pdf
Pavel Shukhman
 
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
How Startups Are Growing Faster with App Developers in Australia.pdf
India App Developer
 
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 
Ad

Input Output devices for Processing IoT Data

  • 1. 134 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) 4 Processing IoT Data Lesson Time: 3 hours, 50 minutes Lesson Introduction An important design consideration for Internet of Things (IoT) is how to distribute the processing tasks involved in acquiring, aggregating, filtering, and analyzing data. Some tasks should be performed directly on the IoT devices that acquire data from sensors. Other tasks may be delegated to a local IoT gateway or cloud services. Lesson Objectives In this lesson, you will: • Use microcontroller I/O to interact with actuators and digital and analog sensors attached to an IoT device. • Distribute data processing tasks to the most appropriate location within IoT infrastructure. • Provide M2M communication between systems used in IoT.
  • 2. 134 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) TOPIC A Process IoT Device Input and Output IoT devices acquire data from sensors, and perform initial processing on that data. A staggering variety of sensors are available for use in IoT devices. When designing an IoT device, you'll need to know what types of sensors are available, the criteria for selecting the right sensors for your application, how sensors connect to a microcontroller, and how to read data from them. IoT devices may also include actuators, LEDs, and other displays that enable them to interact with users and the environment around them. DAQ Data acquisition (DAQ) is the process of measuring real world conditions and converting the measurements into a form that can be processed electronically—by a digital computer, for example. Readings may be taken (or sampled) at a fixed time interval, or on demand. The process of data acquisition includes some form of signal conditioning, which entails converting raw data into a suitable form for processing and analysis by a computer. Sensors Sensors are electronic components designed to perform the task of data acquisition. They detect information about the world around them, such as air temperature and humidity, and convert it into a signal that can be processed electronically. Some sensors function like a switch, making or breaking a connection depending on whether a measurement has reached a particular value, such as a temperature, water level, or air pressure. Other sensors can provide a continuous range of data values. IoT devices typically include sensors, and can be installed virtually anywhere—in a forest, refrigerator, machine, body, or garbage can, at the top of a tower, underneath a road surface, or in orbit around the planet. IoT devices can be located in places you might not be able to go, but sensors mounted on the device can apprise you of the situation there, enabling you to answer a wide range of questions about a remote entity, such as: • Where is it, and where is it headed? (location, bearing, and speed sensors) • Is it positioned where it needs to be? (proximity sensors) • What is the weather there? (environment and weather sensors) • What does it look like there? (camera and imaging sensors) • What types of forces are at play there? (energy and force sensors) • What are things there made of? (chemical sensors) • How long does it take to do something there, and when was the last time it happened? (clock) Sensors can answer questions like these—and many more. Multiplexing Developer boards provide various pins you can use to attach sensors, and you need to match the type of I/O pin on the board (analog or digital) to the type of data being provided from a sensor. In some cases, the number of I/O pins may be quite limited, so you need to ensure you have enough connections for your sensors. It's often possible to read more sensors than you have microcontroller I/O pins to accommodate. It may be possible to multiplex the sensors—essentially taking turns reading the different sensors Lesson 4: Processing IoT Data | Topic A
  • 3. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 135 through a shared I/O pin. It requires additional code and electronic components to accomplish this, so it is simpler when you have enough pins to match the number of sensors you require. Location, Bearing, and Speed Sensors Four common sensing devices often work together to track location, bearing, and speed. • Accelerometers measure changes in momentum and inertia, including the pull of gravity. • Gyro sensors sense changes in twist and angle, using a mechanism that resists such changes (such as a vibrating fork or spinning wheel) as a reference. • Magnetometers sense a magnetic field, and can use the Earth's magnetic field as a reference to get a compass bearing. • Global Positioning System (GPS) sensors use man-made satellites as a reference to obtain the sensor's position on the planet. Each of these has benefits and drawbacks. For example, GPS is very precise and has very little drift, but is subject to the sensor's ability to communicate with satellites. In a tunnel, you might temporarily lose the connection. GPS also uses more power than the other sensor types. Accelerometers and gyro sensors are not dependent on a satellite connection, but they are subject to drift. Magnetometers are prone to interference from magnetic sources other than Earth's magnetic field. However, used together, these sensors can provide a resilient and accurate data stream. For specialized position sensing, other types of sensors may be used. • Wi-Fi, cellular, and other network connections can be analyzed to determine where the device attaches to the network, which can often be used to provide a coarse measurement (for example, within a few city blocks) of the user's location. • Variable resistors (rheostats or potentiometers) are implemented as sliding or rotating controls, like the volume control on a radio. Used as sensors, these components can be attached to various types of levers to measure position. For example, a float sensor is typically a variable resistor attached to a lever with a float on the end of it that follows the surface of a liquid, enabling the level of liquid to be measured. Proximity Sensors Proximity sensors can detect the presence of a nearby object without having to actually come in contact with the object. Different types include: • Echo-based proximity sensors include sonar, radar, and lidar and go by various other names, such as laser or optical rangefinders, ultrasonic sensors, and so forth. This type of sensor emits a field, wave, or beam of some sort, and measures the time it takes for it to bounce back. If you know how fast the field or beam travels, you can determine the distance to the device it bounced off of. • Sonar (sound navigation and ranging) use sound. • Radar (radio detection and ranging) sensors use radio waves. • Lidar (light detection and ranging) sensors use light waves. Different frequencies result in different abilities, such as the distance they can cover and the types of materials they can penetrate. So for example, different forms of light (e.g., UV or infrared), sound frequencies, or radio waves may be used by different sensors. • Electromagnetic sensors detect proximity to an electromagnetic field. Some, like Hall effect sensors can measure flow and variations in the magnetic field, whereas others, like reed switches, make or break a connection (on or off) based on the proximity and strength of a magnet. Reed switches can be mounted on moving devices like a door or window to determine when they are open or closed, without requiring any mechanical levers or contact between the moving entities. • Capacitive touch sensors sense when a finger (or some other conductive object) is touching or in close proximity to the sensor. Many smartphone display screens use capacitive touch sensing. Fingerprint sensors are another application of this sensor type. Lesson 4: Processing IoT Data | Topic A
  • 4. 136 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) If contact with an object is acceptable, a limit switch may be used to detect when the object is in the desired position. Limit switches typically have a spring-loaded lever, often with a wheel or bumper attached to it, which switches when an object bumps into it (indicating that a limit has been reached). Environment and Weather Sensors There are many types of sensors that can measure the environment around an IoT device. • Temperature • Humidity • Wind speed and direction • Rainfall Cameras and Imaging Sensors Cameras and imaging sensors detect various forms of light and record data representing what they "see," which may include spectrums of light the human eye can't see, such as ultraviolet or infrared. • Cameras can provide an image or video of the world around the device. Sensing cameras can provide early detection of cracks in infrastructure, hot spots, flames, edges (of a road or a package, for example), and other visual events. • Motion detection algorithms can be applied to camera data, so a device can be programmed to send you data only when something new appears on the scene. Energy and Force Sensors Force and pressure sensors measure the amount of force exerted over a particular area. The force may be exerted through solids (mechanical pressure), liquids (such as water pressure), or gases (such as air pressure). Pressure sensors can be used to indirectly estimate variables that are a function of pressure, such as altitude (in the atmosphere), depth within a body of water, or weather (barometric pressure). Pressure may be measured in relative amounts. For example, vacuum is measured as a difference from normal atmospheric pressure. • Force sensors can measure mechanical force exerted upon the sensor through a wide range of sources such as human touch (touch sensors). • Load cells create an electrical signal that is proportional to the amount of force being applied to the sensor. • Load bolts are specialized load cells shaped like a bolt and used to measure the amount of mechanical stress upon a machine at that location. • Impact sensors are another type of load cell. They are optimized to measure mechanical force experienced when the sensor is struck. • Weight sensors measure gravity's pull on the object being weighed. • Air pressure sensors may monitor barometric pressure, vacuum, or line pressure. • Sound sensors detect periodic changes in local air pressure. Humans can hear frequencies between 20 and 20,000 vibrations per second (Hertz), but various types of electronic sensors can detect sounds below or above this range. A common form of sound sensor is a microphone, which converts sound waves into electronic signals that can be monitored for changes in volume, frequency, and other qualities. • Flow sensors measure the volume and rate of movement of a liquid or gas through a conduit such as ductwork or pipes. • Resistive touch sensors are true force sensors. They function through physical force, unlike capacitive touch sensors (described previously), which sense a finger's proximity. In comparison to capacitive touch sensors, resistive touch sensors tend to: Lesson 4: Processing IoT Data | Topic A
  • 5. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 137 • Function better in harsh environments than capacitive touch sensors. • Function better when the user is wearing gloves or using a non-conductive stylus. • Be less responsive than capacitive touch sensors (which is why most tablets and smartphones use capacitive touch sensors). Electrical energy can be measured by its various attributes, such as voltage, current, and resistance. • Voltage sensors measure the voltage difference across two lines. • Electrical current sensors measure the amperage of current flowing through an electrical circuit. Split core current transformers can be mounted around an electrical wire without actually cutting into the wire, although they must surround only one of the two wires in the power cable. • Resistance is measured using an ohmmeter. Many types of sensors change their resistance to correspond with an attribute they're measuring, such as light (photoresistor), electromagnetic force, and so forth. Electromagnetic radiation (EMR) is energy emitted from a source. It takes on many forms that people interact with every day, such as light, radio, microwave, X-rays, and gamma rays. All of these forms of radiation can be detected by sensors: • Radio • Infrared light • Visible light • Ultraviolet light • X-rays • Electromagnetic fields Chemical Sensors Chemical sensor options are improving as the technology advances from multiple specialized devices to a single generalized chip or module that can detect a wider array of chemical measures. Relatively inexpensive chemical sensors can now provide molecular analysis of foods, medicines, soil, indoor and outdoor air, rainwater, biological samples, and countless other substances. • Gas sensors can detect a variety of volatile organic compounds (VOCs), breath alcohol levels, toxic gases like carbon monoxide, and pollutants. • PH sensors can measure acidity or alkalinity. • Organic substances such as alcohol can be detected and measured. Timer When multiple readings are taken over a specific time, an additional dimension of measurement can be provided. For example, using a proximity sensor, the difference between two distance measurements divided by the time between those measurements produces a measurement of velocity (e.g., meters per second). With another reading of time and distance, you can determine acceleration— how much faster (or slower) the object goes over time. There are various ways to measure time. You can measure relative time, such as how long a device has been running, how long since the battery was charged, and so forth. You can also measure local time, which is indicated relative to coordinated universal time (UTC) or Greenwich Mean Time (GMT). Processors and microcontrollers typically measure relative time using their internal clocks, which are based on the consistent vibration of crystals. The device's internal clock can be kept in sync by periodically using an Internet connection to compare the processor's local time with that of a time server, using a protocol such as Simple Network Time Protocol (SNTP). A microprocessor or microcontroller may be connected to an external real-time clock (RTC) chip. RTCs run continuously using their own power supply (such as a battery), and enable a device to stay consistent with world time, even if the device is powered down. Lesson 4: Processing IoT Data | Topic A
  • 6. 138 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Note: All of the Guidelines for this lesson are available as checklists from the Checklist tile on the CHOICE Course screen. Guidelines for Selecting Sensors Follow these guidelines for selecting sensors. Select Sensor Types for IoT Projects When considering what types of sensors to use for your IoT projects, consider such things as: • Cost: While the cost of one sensor might not make much difference in a single device, if you plan to scale to hundreds of devices or if you are developing a commercial product, it is important to monitor costs. The same component can vary widely in price from one vendor to another. Furthermore, there are often many different components to choose from. To manage cost, it is important to keep requirements in mind. You might over-engineer the project, using costly components when much less expensive components would meet requirements. In most cases, components cost much less when you purchase them in volume. • Power: What power source (e.g., voltage and current) does the sensor require? Will it operate with the power you plan to provide to the device? Does the sensor support your IoT device's power management requirements? • Signaling: What form of signal does the sensor provide (analog or digital, voltage levels, etc.)? • Resolution and range: How sensitive, and with how much detail, can the sensor take measurements? What are the smallest and largest values the sensor can read? How frequently can measurements be sampled? • Physical interface: How does the sensor interact with its environment? For example, does it need to directly attach, or can it measure simply by being located near the thing it is measuring? • Operating environment: Are there factors that affect the sensor's reliability and durability, such as operating temperature, exposure to liquids, dust, chemicals, ambient electrical noise, shock, and vibration? • Development support: What software libraries and APIs are provided to support software development using the development tools you plan to use? • Supplier: If you need to scale, producing many devices over time, it becomes especially important to carefully select your supplier. If time is a consideration, make sure the supplier carries an adequate supply of inventory and can get parts to you in an appropriate timeframe. • Form factor: How large is the sensor? How will it mount or attach to your IoT device? Sensors targeted toward prototyping and hobbyists are often pre-packaged on a shield or small circuit board that includes supporting components and is easy to use on a prototyping breadboard. For production, it is often more appropriate to use sensors provided in a small package (a chip with dual inline pins, for example) that is not already mounted on a board, and is ready to be mounted on and soldered to a custom circuit board. Lesson 4: Processing IoT Data | Topic A
  • 7. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 139 Item Description Scenario Once you've determined what kind of data you need to obtain, your next task will be to identify the type of sensor or sensors that will provide that data. Many different types of sensors are available for use in IoT devices. In many cases, numerous options will seem to meet your requirements, and you'll need to narrow them down to identify your best option. In this activity, you'll consider project requirements and identify various sensor options that might support those requirements. 1. You need to detect the number of cars passing over a particular section of road. What types of sensors might you use for this? 2. You are designing a home automation device to monitor a home's sump pump. You'd like to be able to send alerts to the home owner when the pump is cycling excessively or the water level gets too high. Location Awareness Location services use a series of satellites in Earth orbit to enable a receiver to determine where it is located. The satellites broadcast information about their orbit and timing. The receiver combines this information from multiple satellites to triangulate its location. Commercial systems are accurate to within a couple meters, but high-end systems can pinpoint location to within a few centimeters. GPS Global Positioning System (GPS) is a satellite-based radio navigation system, also known as NAVSTAR, operated by the United States Air Force. GPS employs 31 satellites, and was first launched in 1978. Users do not have to transmit any data to use GPS for positioning. The system is freely accessible to anyone with a GPS receiver. Lesson 4: Processing IoT Data | Topic A ACTIVITY 4-1 Selecting Sensors for IoT Applications
  • 8. 140 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) GLONASS GLONASS, which stands for Globalnaya Navigazionnaya Sputnikovaya Sistema (Global Navigation Satellite System), is Russia's version of GPS. GLONASS employs 24 satellites, and was first launched in 1982. BeiDou The BeiDou Navigation Satellite System (BDS) is made up of two satellite constellations. BeiDou-1 is composed of three satellites, offering limited coverage and applications primarily targeted to customers in China and neighboring regions. BeiDou-2 (also known as Compass) is a comprehensive global navigation system currently under development and will include 35 satellites when completed in 2020. Galileo Galileo is a global navigation system currently under development by the European Union and the European Space Agency and due to be completed by 2019. It employs 30 satellites, to provide a precise positioning system for Europe that is independent from GPS, GLONASS, and other systems. Actuators and Other Output Devices Actuators provide physical output, typically in the form of motion produced through electrical motors and servos, pneumatic or hydraulic cylinders, and so forth. The term may also refer to small electronic components that produce actions or signaling on a device, such as lights, vibrators, force feedback, and buzzers. Essentially any sort of energy that can be controlled electronically can be used for output. Some examples are listed here. Lesson 4: Processing IoT Data | Topic A Item Description
  • 9. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 141 Motion A wide range of actuator devices and technologies can be used to control motion. Typically, actuators provide a certain type of motion (circular or linear, for example) and range of motion (distance the actuator travels) and include a means for other levers or devices to be attached to them. Levers, gears, and other mechanisms can be used to adjust the type of motion produced by the actuator. Electromechanical actuators typically operate by applying electrical current to an electromagnet or motor, whereas hydraulic and pneumatic actuators are controlled by opening and closing valves that provide pressurized liquid or air to push or pull a piston within an enclosed cylindrical space. The control valves themselves may be controlled through another type of actuator, such as a stepper motor. Various types of sensors (such as limit switches, proximity sensors, or quadrature encoders) may be used in combination with an actuator to determine when it has arrived at the desired position. • Solenoids: When electrical current is provided to an electric coil, an electromagnetic force is created, sufficient to pull a lever. Typically a spring or some other form of resistance pulls the solenoid back to its original position when current to the solenoid is removed. • Linear actuators: Use an electrical, hydraulic, pneumatic, or some other force to provide movement in a straight line. • Motors: Motors provide circular motion, which may be reversible (clockwise or counter-clockwise). • Stepper motors: Stepper motors are optimized for moving an object to a precise position within a circular motion, rotating in defined steps. • Servos: Similar in purpose to a stepper motor, servos rotate a shaft to a specific position. • Vibrators: Vibrating mechanisms are typically used to provide feedback, as in the silent ringing mode on a smartphone. • Muscle wire: Electricity flowing through a muscle wire causes it to flex or relax, similar to muscles in a human or animal. Lesson 4: Processing IoT Data | Topic A Type of Energy Description
  • 10. 142 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Electrical power Various types of electronic devices can be used as power switches, enabling circuits operating on one power level to control circuits with a different (typically higher) power level. Which type of switching device you use depends to some extent on the amount of power you need to control. • Relays: Relays are traditionally electromechanical devices, although solid-state (non-mechanical) relays do exist. A typical relay has a coil of wire that creates a magnetic field when electricity is applied. The pull of magnetism closes a mechanical switch, which is opened again when electricity is no longer applied to the coil. In typical applications, a small amount of electrical current controls the switch, while the switch closes a circuit that conducts higher current. Relays provide good electrical isolation between the two circuits. • Transistors: Transistors can provide switching capabilities when power levels between the microcontroller and the device being switched aren't extreme. For example, you might use a transistor to enable a 3 volt microcontroller to control a small 9 volt motor. While transistors tend to be limited to lower power levels than relays, they generally switch much more quickly since they are not mechanical. Special types of transistors, such as metal-oxide-semiconductor field- effect transistors (MOSFETs) can operate with higher power levels than normal transistors. For example, a MOSFET might enable a microcontroller to control automobile headlights. Sound There are various types of sound-generating devices. • Transducers: Transducers are commonly electromechanical or piezoelectric devices. They may essentially be a mechanical relay that is switched on and off frequently, or a piezoelectric device that uses a vibrating crystal or metallic fork to produce tones, clicks, or buzzing. • Speakers: Speakers are fundamentally similar to some transducer designs (using piezoelectric or coil-based vibration to generate sound waves) but the frequency of the sound waves is modulated to produce a wider range of tones, even reproducing human voice or music. • Sirens, horns, and whistles: When loud, penetrating sounds are required (for alarms in an industrial environment, for example), a compressed air source (such as an air pump) can be used to generate loud wailing or whistling noises. Light • Annunciator (status indication) lights may be implemented through various lighting technologies, such as incandescent, fluorescent, or light emitting diodes (LEDs). • LEDs can be used as annunciators or for illumination. A single LED device may contain just one color LED or several colors (typically red, green, and blue), which can be separately controlled to provide color- coded indicators. Multiple LEDs arrayed in a matrix can be used for segmented displays. • Special light sources, such as laser and infrared, can provide signaling for specialized situations. For example, laser can be used for long-distance line-of-sight signaling or remote control, and infrared is frequently used for short distance wireless control (as used in TV remote controls). Lesson 4: Processing IoT Data | Topic A Type of Energy Description
  • 11. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 143 Case Study: Analog Light Sensor The GL5516 photoresistor is a variable resistor. The amount of resistance it provides in a circuit varies based on the amount of light it is exposed to. However, the ESP8266 microcontroller's analog input pin measures voltage, not resistance. But there is a relationship between voltage, current, and resistance. By changing resistance, you can change voltage, which you can measure on the microcontroller's analog pin. For example, you could connect 3.3 volt power to the microcontroller's analog pin through the photoresistor, as shown in the following figure. Figure 4-1: Photoresistor adjusting voltage to the analog pin. As the resistance provided by the photoresistor changes with the amount of light, it allows more or less voltage to be provided to the analog pin. This circuit is almost right, but unfortunately, the analog pin on the ESP8266 takes its maximum reading at only 1 volt, so the 3.3 volt power provided by the development board will be too much. And the development board provides only two power options, 3.3 volts and 5 volts. There are no 1 volt power pins. However, by adding a second resistor to the circuit, you can divert some of the electrical energy to ground so the full voltage won't reach the analog pin, as shown in the following figure. Lesson 4: Processing IoT Data | Topic A
  • 12. 144 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Figure 4-2: Voltage to the analog pin divided using a second resistor. This type of circuit is called a voltage divider. Voltage dividers and other techniques are commonly used to match required signal levels between a sensor and an I/O pin on a microcontroller. Lesson 4: Processing IoT Data | Topic A
  • 13. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 145 Before You Begin The course data files are installed on your computer in C:095024Data. These files include a customized version of the Arduino IDE (located in C:095024DataToolsarduino), which contains example programs that you will use in this course. The example programs are contained in the File > Sketchbook sub-menus inside the Arduino IDE. The ESP8266 microcontroller is connected to a USB port on the development computer. You have successfully connected the device to the Wi-Fi network, and the SSID and password are now cached on the device. The Arduino IDE is not running. Scenario You will use a photoresistor to add light-sensing capability to your IoT device. You will use a second (fixed value) resistor to reduce the voltage reaching the analog pin on the microcontroller, so you can take a full range of readings. The analog-to-digital converter on the ESP8266 returns values in the range 0 to 1023, so the reading will return a relative light value in that range. Jumper wires on the prototyping board provide power to the left power rails on the development board, so you will connect your resistors to the power and ground provided in the power rails, rather than connecting them directly to the development board. For now, you will simply output the sensor readings to the serial monitor. 1. Remove power from the IoT device. a) Disconnect the USB power cable from the IoT device. When installing components, remove power from the board so you don't damage them. 2. Observe how power will be provided to the power rails. Lesson 4: Processing IoT Data | Topic A ACTIVITY 4-2 Reading Analog Sensor Data
  • 14. 146 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) a) Examine the jumper wires on the left side of the breadboard. • Between socket A26 and the positive (+) power rail. • Between socket A29 and the negative (−) power rail. • These wires tap electrical power from pins on the microcontroller. • The 3V3 pin connects to socket A26, and provides 3.3 volts from the board. • The GND pin connects to socket A29, and provides ground from the board. • Because of these connections, components you add to the breadboard can tap power from the rails in the left two columns of the breadboard. 3. Locate the light sensor and 10K ohm resistor in the development kit. a) In your development kit, locate the light sensor (photoresistor) and 10K ohm resistor. You will add these components to the prototyping breadboard. You will team the light sensor with the 10K ohm resistor to enable it to function at the logic levels used by the microcontroller. 4. Set up the components on the breadboard. Lesson 4: Processing IoT Data | Topic A
  • 15. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 147 Note: Position the leads in the positive and negative sockets away from each other so they don't short out. a) Insert the 10K ohm resistor wires into the breadboard sockets as shown. • Resistors have no polarity, so it doesn't matter which end goes into each socket. • One end goes in the negative (−) power rail on the left side of the prototyping board. • The other end goes in socket C9. • The jumper wire already located in socket D9 connects to the analog I/O pin (A0) on the microcontroller, so the resistor is now effectively connected between ground and the analog I/O pin. b) Insert the light sensor into the breadboard sockets as shown. Connect the light sensor leads with: • Polarity doesn't matter for this component either. • One end in socket E9. • The other end in the nearest socket in the positive (+) power rail on the left side of the prototyping board. c) Observe the current configuration. Lesson 4: Processing IoT Data | Topic A
  • 16. 148 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) • One leg of the light sensor is powered with 3.3 volts. • One leg of the 10K ohm resistor is grounded. • The other legs of the light sensor, 10K ohm resistor, and analog I/O are connected to each other through row 9 of the breadboard. • The analog pin A0 on the microcontroller can now read light values from the light sensor. 5. Load a sketch that reads values from the A0 analog pin. a) Display the Windows Start menu, and select the Arduino tile to launch the Arduino IDE. b) Select File→Sketchbook→Processing IoT Data→Analog_Sensor. c) Close the other sketch window so Analog_Sensor is the only sketch showing. 6. Start the process to compile and upload the sketch. a) Plug the USB cable back into the microcontroller board to reconnect the device. b) Select Tools→Serial Monitor to display the serial monitor window, and ensure the baud rate is set to 9600 baud. c) Select Sketch→Upload. 7. While the sketch is being compiled, examine the source code. a) Examine lines 1 through 18 in the Analog_Sensor sketch. • Lines 3 through 6 initialize various variables used to track the sensor readings. For example, updateInterval, defined in line 3, determines how much time will elapse between each sensor reading. Right now, that value is set to 1000 milliseconds, or one second. • The setup() function in lines 8 through 18 initialize the connection to the serial monitor, where you'll view the sensor readings. Lesson 4: Processing IoT Data | Topic A
  • 17. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 149 b) Examine lines 20 through 39 in the Analog_Sensor sketch. • The loop() function checks how long it has been since the last sensor reading. • Line 22 determines if the time between the last update and the current time (in milliseconds) has exceeded the value specified in updateInterval. If it has, line 23 resets the time of the last update (to the current time), and then calls other functions to update the sensor values and to log them to the serial monitor. • In line 31, the updateSensorValues() function reads the current data value from pin A0 into the light variable. • In line 37, the logSensorValues() function prints the value of the light variable. When the program runs, the current value read by the light sensor will be displayed in the serial monitor. 8. Examine the sensor readings. a) When the program runs, examine the output in the serial monitor. Position your hand over the sensor and observe how the values change. • Every second, a new value appears. • The analog pin reads values between 0 and 1023. • There are 1024 possible values because the ESP8266 microcontroller's analog-to-digital converter (on pin A0) provides output in 10 bits. There are 1024 different combinations of 1 and 0 in 10 bits (0000000000, 0000000001, 0000000010, 0000000011, and so forth). • In the current configuration, the brightest light readings would result in the largest numbers, and the dimmest light would result in the smallest numbers. Lesson 4: Processing IoT Data | Topic A
  • 18. 150 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Case Study: Digital Humidity and Temperature Sensor The DHT11 and DHT22 digital humidity and temperature sensors are popular for prototyping and hobbyist use because they are inexpensive while providing good performance. The two devices are similar, operating on 3 to 5 volts, and drawing very low current (no more than 2.5 milliamperes) to operate. Within their plastic shell, the devices enclose a humidity-sensing component, a thermistor (temperature-sensing transistor), and an integrated circuit that converts their signals to digital form for communication with a digital I/O pin on the microcontroller. The device requires a positive and negative connection to provide power, and uses a single data pin for serial communication. The DHTesp software library provides the logic to enable the ESP8266 microcontroller to communicate with the sensor. The DHT22 is typically more expensive and reads a wider range of temperatures. The DHT22 measures temperatures from -40 to +125 degrees Centigrade (-40 to 257 Fahrenheit), accurate to half a degree (.9 degrees Fahrenheit). The DHT11 measures temperatures from 0 to 50 degrees Centigrade (32 to 122 Fahrenheit) with +-2 degrees (3.6 degrees Fahrenheit) accuracy. The DHT22 sensor has a better humidity measuring range, from 0 to 100% with 2-5% accuracy, while the DHT11 measures humidity from 20 to 80% with 5% accuracy. There is a tradeoff with the DHT22's greater range and precision; it takes longer to sample data. The DHT22 can take a reading once every two seconds, whereas the DHT11 can take a reading once per second. Lesson 4: Processing IoT Data | Topic A
  • 19. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 151 Before You Begin The course data files are installed on your computer in C:095024Data. These files include a customized version of the Arduino IDE (located in C:095024DataToolsarduino), which contains example programs that you will use in this course. The example programs are contained in the File > Sketchbook sub-menus inside the Arduino IDE. The ESP8266 microcontroller is connected to a USB port on the development computer. You have successfully connected the device to the Wi-Fi network, and the SSID and password are now cached on the device. You have installed the light sensor and 10K resistor on the breadboard. The Arduino IDE is running and the serial monitor is showing. Scenario Now you will attach a DHT11 digital sensor to take temperature and humidity readings. This sensor has three connections: power, ground, and a single data line. 1. Locate the DHT11 sensor in the development kit. a) In your development kit, locate the DHT11 temperature and humidity sensor. 2. Install components on the breadboard. a) Unplug the USB cable from the device to remove power while you add the sensor. Lesson 4: Processing IoT Data | Topic A ACTIVITY 4-3 Reading Digital Sensor Data
  • 20. 152 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) b) Attach the labeled wires from the DHT11 sensor to the breadboard as shown. • + wire to the positive (+) power rail • G wire to the negative (−) power rail • D wire to the F1 socket c) Observe how the DHT11 sensor is now connected. • The signal from jumper wire D (the data line) is conducted by a jumper wire to digital I/O pin D1 on the microcontroller. • The DHT11 sensor data can now be read using pin D1 on the microntroller. • The two other wires provide power to the sensor. 3. Load a sketch that reads values from the D1 digital pin. Lesson 4: Processing IoT Data | Topic A
  • 21. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 153 Note: Output from the program currently installed on the device may appear in the console before the new program is uploaded to the device. a) Select File→Sketchbook→Processing IoT Data→Digital_Sensor. b) Close the other sketch window so Digital_Sensor is the only sketch showing. 4. Start the process to compile and upload the sketch. a) Plug the USB cable back in to the development computer to reconnect the device. b) Make sure the serial monitor window is still showing. c) Select Sketch→Upload. 5. While the sketch is being compiled, examine the source code. a) Examine lines 9 through 12. Initial values for the humidity and temperature (in Centigrade and Fahrenheit) are set. b) Examine lines 25 and 27. • The DHT sensor will be read from pin D1. • Line 27 will output the headings above the data table that will be output to the serial port. c) Examine lines 45 through 50. • The humidity and temperature values are read each time the program loops. • Since the sensor might not be ready, its status is checked before obtaining the sensor readings to ensure no false values will be obtained. Lesson 4: Processing IoT Data | Topic A
  • 22. 154 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Note: To influence the sensor readings, you can cover and uncover the light sensor and breathe on the temperature and humidity sensor. d) Examine lines 54 through 69. • Lines 54 through 56 contain the function that prints the heading. • Lines 59 through 69 contain the function that outputs a row of current data values. 6. a) When the program runs, examine the output in the serial monitor. Every second, new values are printed. 7. Clean up the workspace. a) In the Arduino IDE, select File→Quit. Lesson 4: Processing IoT Data | Topic A
  • 23. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 155 Item Description TOPIC B Process Data in the Cloud The processing tasks performed by IoT devices revolve around data acquisition. Other, more powerful devices are typically used to perform the heavy processing tasks. Now that you've enabled your IoT device to acquire data, you'll set up a sensor network to enable the devices to offload their data for aggregation by an IoT gateway as it prepares the data for processing by cloud services. Deployment Models As more organizations have become comfortable using cloud services, they have migrated systems from on premises (using the company's own computers and data centers) to the cloud. The cloud, with its ability to provide virtually unlimited access to resources for computing, storage, and networking, provides a viable platform for IoT. Public cloud • Most common way to deploy cloud computing. • Hardware, software, and supporting infrastructure are owned and operated by a cloud services provider. • Tenants (customers) share hardware, storage, and network devices with other tenants. • Services are typically accessed and managed through a web browser. • Deployment costs are contained, as you don't have to purchase hardware or software, and you pay only for resources you use. • Decreased maintenance over on-premises deployment since the service provider maintains the hardware and some of the software. • Resources are virtually unlimited, as they can be scaled up to meet demand. • Reliability tends to be quite high, as service providers build in redundancy and backups. Private cloud • Often used by government agencies, financial institutions, other organizations that need to maintain control over business-critical operations. • Cloud computing deployment dedicated solely to use by one organization. • Can be physically located on premises, or hosted by a third-party service provider. • More flexible than public cloud, making it easier to customize resources for very specific IT or business requirements. • Potentially improves security since resources are not shared with others, facilitating higher levels of control and security. • Still provides the scalability and efficiency of a public cloud. Lesson 4: Processing IoT Data | Topic B
  • 24. 156 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Hybrid • Combined use of public and private clouds, to take advantage of both. • May move data and applications between private and public clouds as needed to meet demand, provide more flexibility, or increase deployment options. • Enables the organization to maintain a private infrastructure for sensitive assets. • Provides the flexibility to use resources in the public cloud as needed. • Enables scaling to the public cloud only as needed, helping to contain costs. • Facilitates a gradual move to the public cloud over time. Cloud Computing Cloud computing is a model for sharing data processing, storage, and software applications among multiple customers or organizations across a network, such that individual customers can provision computing resources instantly as needed and release them when they are no longer needed. Companies such as Amazon, Google, and Microsoft have created large data centers with computing resources they rent to customers, who then do not have to buy and maintain their own dedicated servers. These cloud data centers provide a pool of computing resources, and expert staff to run them, that are available to customers whenever they need them. The National Institute of Standards and Technology (NIST) has identified five principles that define cloud services and describe how they are typically designed. These are also core benefits to cloud computing customers. They are: • Resource pooling: Anyone who wants access to computing resources can get access to the resources they need, as long as they pay for them. • On-demand self-service: Customers can add resources they need, when they need them. • Rapid elasticity: Customers can rapidly scale up or scale down their resources, if need be. • Measured services: The services used are metered. Customers pay only for what they use, for the duration that they use it. • Broad network access: Allowing a range of devices, including both desktop and mobile, access to applications and data stored in the cloud in private cloud implementations. There are several benefits to cloud computing models. One of the biggest is that paying for access to cloud computing resources enables businesses to bypass the up-front costs associated with buying space for a computer room, buying computer and network hardware, and paying IT support staff to set up and maintain that computer hardware. Lesson 4: Processing IoT Data | Topic B Item Description
  • 25. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 157 Cloud Service Models Figure 4-3: Cloud service models. Because there are so many different cloud service offerings, it can be helpful to consider them in terms of the general type of service they provide. Software as a Service (SaaS) Platform as a Service (PaaS) Infrastructure as a Service (IaaS) Cloud Platforms Applications such as customer relationship management software running on the service provider's environment. The services are released in line with the changing demands. Platforms such as web applications, database servers, enterprise service buses, and other middleware, with associated security mechanisms that let you develop, test, and deploy applications. Computer and storage resources maintained on a service provider's environment. Numerous cloud platforms have arisen to support IoT. The features offered by these platforms vary widely. Examples include: • Microsoft Azure • Amazon Web Services • IBM Cloud • Oracle Cloud • SAP Cloud Platform • Google Cloud Platform • Huawei FusionSphere • Bosch IoT Cloud • General Electric Predix • Salesforce Lesson 4: Processing IoT Data | Topic B Cloud Service Models Description
  • 26. 158 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Concept Description Note: It is not necessary to use a single cloud vendor. Model your solution on paper to determine whether the benefits of using multiple cloud service providers outweigh the disadvantages. A hybrid solution may involve services from multiple cloud platforms, including private cloud. Virtualization Technologies Cloud services depend on virtualization technologies. Virtualization simulates one or more computing devices within a physical computer. The following are various concepts related to virtualization. Virtual machine Virtualization technology makes it possible to run multiple instances of an operating system configuration on the same hardware. For example, a physical server with eight processor cores, 64 gigabytes (GB) of Random Access Memory (RAM), and 4 terabytes (TB) of storage can be configured with four virtual machines, which are each assigned two processor cores, 16 GB of RAM, and 1 TB of storage. The physical computer is the host, and the simulated devices are typically called virtual machines (VMs). Virtual networking Just as computing, memory, and storage resources can be subdivided into virtual machines, network resources can be virtualized as well. For example, if the host machine previously described included four network interface cards (NICs), they could be virtualized so that each VM has a single virtual NIC. Likewise, network switches can be virtualized to enable virtual networks to be defined. Software-defined networking is accomplished through virtualization. Hypervisor The software or firmware that manages virtualization of resources on the host hardware is called a hypervisor. The hypervisor provides the guest operating systems with a virtual operating platform and manages the execution of the guest operating systems. Different operating systems may share the virtualized hardware resources. Container Requirements for a particular task may not call for using an entire virtual machine. You may only need to access an application instance. Application containers include just enough virtual machine environment (such as files, environment variables, and libraries) to contain an application, hence the name container. Containerization enables you to optimize your use of virtual machine resources when all you need to run is an application. Docker is a common technology used for containerization. Guidelines for Evaluating Cloud Platforms Follow these guidelines when evaluating cloud platforms for use in IoT. Evaluate Cloud Platforms When you evaluate cloud platforms for your projects, consider points such as the following. • Services offered. Different vendors tend to have better capabilities in some areas rather than others, so you should align the vendor's capabilities with your project's requirements. For example, one vendor may offer superior analytics tools, but may not offer other services you require. Lesson 4: Processing IoT Data | Topic B
  • 27. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 159 Note: When evaluating services for pricing and performance, it is critical to model how your system will work. For example, one vendor may charge less to store data from their cloud storage service, but may charge more to retrieve data. Whether it will be cost-effective to use that vendor's solution depends on how frequently your solution will need to retrieve data, which you can only determine by planning its design. • Compatibility with other systems you're using. If your existing infrastructure is already based on products or services from a particular vendor, it might simplify some of your integration tasks to stick with that vendor for your IoT needs. • The location and geographic distribution of data centers. Having data centers located near your on-premises systems and customers may help to improve overall performance and the user experience. Having data centers that are distributed throughout different regions will make the system more resilient and less prone to problems like local outages. • Pricing. Vendors price their services differently, which makes it difficult to compare costs. In some cases, vendors offer a pricing calculator, which can help you estimate your costs by modeling the services and usage you think you'll require. • Reliability and service level agreements. Evaluate the level of service the vendor will guarantee through a service level agreement (SLA), a contract that ensures a certain level of service will be guaranteed, and which may specify consequences if that level of service is not met. The service level might include measures such as guaranteed uptime (e.g., 99.9 percent), data rates, and throughput. The SLA identifies repercussions or responsibilities if the service level is not maintained. • Security and compliance. Since you'll ultimately be responsible to ensure the security and compliance of solutions you put into place, work with a cloud service provider that provides effective tools to facilitate cybersecurity and compliance. • Programming platforms and interfaces. Open programming platforms and interfaces will help you avoid vendor lock-in. Also, using prevalent tools will make it easier to find technical talent to implement solutions. Cloud vendors may offer a free tier, which you can use to evaluate their service offerings. IoT Services Performed in the Cloud Various IoT services are typically performed in the cloud, as shown here. Lesson 4: Processing IoT Data | Topic B
  • 28. 160 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Figure 4-4: IoT services typically performed in the cloud. Using cloud services for these tasks provide the ability to scale up and down as needed to handle varying demands and changing requirements. Trying various iterations of cloud service configurations enables you to evolve and optimize the system quickly, without extensive investment in hardware and infrastructure. Device Registry and Management Managing IoT devices registered with the system is another service that may be performed by cloud services. IoT deployments may involve large numbers of devices the IoT system may communicate with, control, provision, or manage, making it essential to maintain a scalable database, or registry, of devices. Various applications throughout the IoT system may need to refer to this information, making the cloud a natural place to store it. To manage the registry, some sort of device management system should be implemented through which administrators can securely and reliably provision devices, manage and control them remotely, update their software, and monitor them through logging and other features. The device management function may communicate with devices through a management agent running on each device. Data Management IoT devices may generate large quantities of real-time data. While initially this data may be stored and processed by an IoT gateway, it is typically moved to cloud storage where it can be integrated with processes and applications that are part of the IoT system. Since data can become voluminous, the data store must be elastic and scalable, making cloud storage a good choice for data management. Different cloud storage services provide different costs, capabilities, performance, and limitations, so it is common to use multiple storage services to meet project requirements. Big data tools provided by cloud services are appropriate for the analysis of the vast quantities of data produced by IoT. Lesson 4: Processing IoT Data | Topic B
  • 29. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 161 IoT Data Analytics Tools Data analytics tools can analyze historical data to identify patterns that can predict actions that will produce good results in the future. Analyzing complex data such as video and audio requires extensive processing capabilities. Using cloud services for this task enables large amounts of computing capability to be provisioned on demand. An IoT data analytics capability includes: • Streaming analytics: Data analysis may be time-sensitive, so data needs to be received and analyzed from a variety of inputs in real time. As data arrives, it may be processed immediately through streaming analytics, and it may also be forwarded to a data warehouse for deeper background analysis. • Predictive Analytics: To perform predictive analytics, the system extracts information from existing datasets to determine the current state, identify patterns, and predict future trends. It provides a predictive score for the concept being evaluated (such as whether a particular component is likely to fail before the next scheduled maintenance, how many vehicles are likely to cross a particular bridge in the next three minutes, and so forth). Decisions can be informed based on the score. • Content analytics: Although structured data comes from sources like IoT sensors, databases, online transactions, and so forth, companies may also produce a large amount of unstructured data, such as audio, video, office documents, text messages, email, chat messages with customers, and so forth. Natural language analysis of such unstructured content may help to uncover patterns and trends that can also serve as a source for data analytics. • Visualization and reporting: Tools such as statistical graphics, charts, dashboards, and summaries enable users to view summarized data and predictions in a form that helps to reveal insights resulting from data. • Analytics data repository: Data may come from a variety of new and legacy sources, such as IoT devices, streaming analytics, enterprise applications, enterprise data, and reference data. Data may be structured or unstructured. An analytics data repository may provide a variety of tools, such as: • Exploration and archiving: Tools should be provided to manage the massive data sets in the repository, enabling it to be explored, augmented, and archived. • Interactive analysis and reporting: The system should provide tools to view and report on data in large data sets to answer immediate business and operations questions. • Data catalog: To facilitate effective data analytics, the system should provide a consolidated view of all information being used for the purpose of data analytics, called a data catalog. The catalog is produced through data curation by data professionals, and may be enhanced through automated cataloging tasks performed by the system, making it easier for analysts to find the data they need, ensure data isn't duplicated or incorrect, and adhere to privacy statutes and other compliance requirements. Following are examples of predominant data analytics tools. • SAP Analytics Cloud • Microsoft Azure Stream Analytics • AWS IoT Analytics • IBM Watson IoT Platform • Cisco Data Analytics • Oracle Stream Analytics and Edge Analytics Components of a Real-Time Analytics System A number of capabilities are required to implement real-time analytics. Lesson 4: Processing IoT Data | Topic B
  • 30. 162 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Messaging A messaging system should be able to provide a low latency, high throughput, fault tolerant publish and subscribe pipeline between content sources and analytics tools. Streaming It should be possible to process incoming data as it arrives from a wide variety of data streams. Scalable database Due to the large quantities of data likely to be processed, the database should be highly scalable, geared toward big data applications. Commercial systems geared toward real-time analytics are available. Alternatively, a system could also be constructed through free open source tools. For example, a real-time data pipeline could be implemented using Apache Kafka for messaging, Apache Spark Streaming for real-time streaming analytics, and Apache Hadoop or Apache Cassandra as the database. While the solution may involve different software components, due to the demanding performance requirements of any real-time system, you carefully plan where you deploy and how you integrate the various capabilities. For example, eliminate network communication by locating the streaming and database capabilities on the same host. AI Artificial intelligence (AI) is a technology that enables computer systems to mimic cognitive functions normally associated with human intelligence, enabling computers to perform tasks that would normally be done by a human, such as making decisions, recognizing a visual stimulus (computer vision), responding to the spoken or written word (natural language processing), and so forth. In IoT systems, AI capabilities enable computers on the edge (such as IoT gateways) and in cloud services to automate such tasks with little or no intervention by people, or to perform them on a scale that would not be practical to perform if human personnel were required. For example, AI can be used to monitor the huge amounts of data being produced by IoT sensors, and to notify human operators when a problem pattern is identified. Or AI functions might be programmed to directly trigger an immediate, appropriate response when there is insufficient time to involve human intervention. One application of AI is machine learning (ML), through which a computer system can adapt its behavior or outputs based on interactions with data, human data analysts, and other entities. With Internet-connected systems, the information sources that inform machine learning do not necessarily need to be local, so edge devices such as sensors can provide data that informs machine learning in cloud services. Traditionally, computers have been programmed directly to perform certain types of data analysis tasks. However some types of tasks are not practical to implement solely through code, but can be learned effectively through AI. As with human learners, artificial intelligence improves with experience. Using a process called deep learning, AI can be trained through exposure to large historical datasets to identify patterns, and improve its predictive analysis capabilities. Through this process, predictive analysis capabilities of AI have been trained to meet or exceed the capabilities of human analysts in tasks such as transcribing audio, predicting crop yields, and diagnosing illness. AI Tools Artificial intelligence is a rapidly developing field, and deep learning products have improved dramatically in just a few years, making them more accessible to organizations that don't already have AI experts on staff, although there is still a significant learning curve to using these technologies. The following are some of the artificial intelligence tools now being used for IoT data analytics. Lesson 4: Processing IoT Data | Topic B Capabilities Description
  • 31. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 163 Google TensorFlow Open source framework (with some closed-source components) that uses Python to perform numerical computations using data flow graphs. TensorFlow can function on a wide range of systems that can run Python, and is compatible with the TensorBoard visualization framework. Theano Another Python-based tool that supports numerical operations supporting efficient high performance processing on a GPU. Caffe A C++ deep learning framework that supports the training of models without having to write code. Torch A library based on the Lua programming language that includes many pre-trained models. IBM Watson A cloud-based service that interfaces with IoT systems through a variety of programming interfaces. Sagemaker SageMaker, provided by Amazon Web Services, is based on Jupyter Notebook, a Python-based machine learning service. SageMaker streamlines much of the complexity involved in constructing, training, and deploying machine learning models, and is optimized for use with huge, distributed data sets. Drawbacks of Centralized Cloud Computing in IoT With IoT, much of the action takes place out on the edges of the network. IoT devices and sensors acquire data. After data is analyzed, it may be determined that action is needed. Actuators on the devices make adjustments as needed. When only the central nodes of the network (cloud services) have the capability to store, process, and analyze data, you've committed to a design that requires extensive, long-distance communication. Problems with this design include: • The time delay in moving data from where it is acquired to where it is processed. • The time delay in communicating commands from where data is analyzed to where actions are effected to make appropriate adjustments. • High volume and frequency of data communications between edge devices and the cloud computing "core." Lesson 4: Processing IoT Data | Topic B Item Description
  • 32. 164 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Note: Unfortunately, these terms mean different things to different people, and are not always used precisely the same way. Always consider the context when interpreting these terms to ensure you understand the underlying points being made about them. Where to Perform Processing Tasks Figure 4-5: Different processing tasks are performed throughout the continuum, spanning cloud services, the edge, and devices. While recent IT trends have shifted data processing and analytics tasks to the cloud, with IoT, many organizations are opting to perform many of these tasks out at the edges of the network. This is part of a trend called fog computing or distributed cloud computing. The objective of fog computing is to provide better quality of service, make more efficient use of bandwidth by processing data where it's needed, improve user experience, and minimize exposure of sensitive data to security and privacy risks. Fog computing builds upon the edge computing concept, distributing cloud computing, storage, control, and networking functions closer to where the effort is needed. It distributes cloud services throughout the continuum between cloud, edge, and devices. The term "fog" is used because data processing clouds may be formed close to "ground level"—near sensor networks, IoT devices, and end users. Private clouds and IoT gateways may be used for this purpose. Complementing the fog computing effort, some processing may be performed directly on IoT devices and client applications. Processing that occurs here is sometimes called mist computing. When you determine where processing should occur in an IoT system, there are tradeoffs to consider. Local processing is immediate but constrained by local limitations, such as the processing power of IoT devices. Extensive resources and scalability are provided by cloud services, but it takes longer to send data there and aggregate it for processing. Guidelines for IoT Processing and Analytics Follow these guidelines when planning how to distribute IoT processing tasks. Lesson 4: Processing IoT Data | Topic B
  • 33. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 165 Determine Where Processing Should Be Performed The following examples describe various types of processing tasks and identify where such tasks might typically be performed within an IoT architecture. Task Edge Devices Fog Cloud Read data from sensors x Control actuators x Validate raw sensor data x x M2M communication x x Initial data validation, filtering, and normalization x x Aggregate data from individual sensor devices within a single sensor network x Process real time data (shallow data analysis and quick decision-making and response) x x Aggregate batches of data forwarded from various sensor networks x Process big data (deep data analysis, in-depth decision-making and analysis) x Data warehousing x Business logic x Reporting and visualization x Lesson 4: Processing IoT Data | Topic B
  • 34. 166 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Scenario As you continue to prototype your IoT devices, you're also planning the infrastructure that will support them—networks, gateways, servers, and so forth. Here's a summary of the system as you currently envision it: • You will start with approximately 300 rechargeable battery-powered sensor devices, most of which will be mobile, located in growing trays and on carts that may be moved throughout the greenhouse. • The sensor devices will attach to the network through Wi-Fi. • Each greenhouse has its own Wi-Fi router, which is attached to the farm's network through cabled Ethernet. • A program running on a computer located on premises will obtain data from the sensors, analyze that data, and send commands to separate actuator devices throughout the farm to adjust lighting, air flow, heating and cooling, irrigation, and other factors as needed. • You also intend to forward the sensor data, along with other data you manually update in a database application, to a cloud-based application that will perform analytics to help Greene Organix determine the most profitable time to grow specific crops, improve its agricultural practices, and so forth. 1. How would you transport sensor data to the cloud application? 2. How would you decide which processing tasks should be performed locally, and which tasks should be performed in cloud services? Lesson 4: Processing IoT Data | Topic B ACTIVITY 4-4 Planning IoT Processing
  • 35. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 167 TOPIC C Provide M2M Communication Processing IoT data involves moving data from one machine to another, and machines do not have the same communication requirements as people. Industrial M2M Communication Industrial machine-to-machine (M2M) communication typically involves the Supervisory Control and Data Acquisition (SCADA) control system architecture, which is typically used in industrial, energy, and utility companies. While originally a monolithic system, SCADA has evolved to become a standard for data acquisition and control so process automation and control products from different manufacturers can interoperate effectively. SCADA systems now support multiple plant locations, with remote access over a wide area. Systems using SCADA acquire data from sensors, and provide a graphical user interface (also called a human-machine interface (HMI) (in the context of industrial control system). The user interface presents data graphically, provides notifications, alarms, and other functions. On-screen controls enable users to control valves, pumps, motors, and other devices as well. IoT M2M Communication Familiar web technologies such as Transmission Control Protocol (TCP) and Hypertext Transfer Protocol/Hypertext Transfer Protocol Secure (HTTP/HTTPS) have been leveraged for M2M communication in business and consumer IoT. Some new technologies have also been developed, and some technologies have been borrowed from other domains. While it is possible to implement M2M communication using only traditional web technologies, they are not always a good fit. The web was originally designed for machine-to-person communication, not machine-to-machine communication. For example, TCP supports acknowledgment of transmitted packets, retransmission of lost packets, and flow control. These are appropriate when a live customer is at the other end of the transmission, waiting to consume the information being sent. Likewise, HTTP/HTTPS is based on a two-way request-response model that makes sense for human users interacting live with a website. But a more streamlined one-way messaging approach may be more appropriate for machine-to- machine communication, in which communication must occur quickly and with little overhead. These features come with overhead in terms of time, processing cycles, and network bandwidth, which may not be necessary in machine-to-machine communications. The much more efficient User Datagram Protocol (UDP) may be more appropriate for many M2M communication tasks. If data does not arrive at its destination in time, it may no longer be relevant, so retransmission would be pointless anyhow. And a one-way messaging protocol might be more appropriate than the two-way HTTP/HTTPS request-response model. Communication Patterns Identifying the prevalent communication pattern can help you evaluate what communication protocols are most appropriate for the situation. A request-response protocol like HTTP/HTTPS might be appropriate for two-way communications, while a publish-subscribe protocol might be more appropriate for ongoing, one-way status updates, particularly when timing is critical and when constrained devices are involved. Lesson 4: Processing IoT Data | Topic C
  • 36. 168 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Telemetry Notification Inquiry • Message Originator: IoT device • Data Flow: One direction, from the IoT device to the external entity. • Data sent by the IoT device: The IoT device sends data such as sensor readings to another entity, such as an IoT gateway or cloud services. • Message Originator: External entity • Data Flow: One direction, from the external entity to the IoT device. • Data sent by the external entity: The external entity, such as a web services provider, provides the IoT device with data it would not otherwise have, such as weather data for local presentation to the user, news, pollution alerts, and so forth. • Message Originator: IoT device • Data Flow: Data flows in two directions, with the request sent away from the device, and a response returned to the device. • Request sent by the IoT device: The IoT device requests that an external entity provide data or perform an action. • Response returned by the external entity: May result in either: • Return of data (as in a database lookup) • Activity being performed by the external entity, such as: • Change the state of something in the cloud or in the physical world, such as a request by a smartwatch to change the heat setting on a thermostat at home, submitting an order for a product when a dash button is pressed, and so forth. • If the inquiry was a subscribe request, starting to send notifications to the IoT device. Lesson 4: Processing IoT Data | Topic C Pattern Description (from the perspective of the IoT device)
  • 37. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 169 Command • Message Originator: External entity • Data Flow: Data flows in two directions, with the request initiated by the external entity, and a response returned by the IoT device. • Request sent by the external entity: The external entity requests that the IoT device provide data or perform an action. • Response returned by the IoT device: May result in: • Return of data, such as requesting that the IoT device send the last sensor reading. • Activity being performed by the IoT device, such as: • Change the state of the device, perhaps with effects in the physical world such as changing the heat setting on a thermostat, turning on a light, and so forth. • Starting to send telemetry data as an ongoing stream. Messaging Protocols While a variety of communication and messaging protocols can be used for IoT M2M communication, several protocols have emerged to serve this role, including: • AMQP • CoAP • MQTT AMQP Advanced Message Queuing Protocol (AMQP) is an open standard publish-and-subscribe protocol. AMQP was designed for JPMorgan Chase in 2004, for use in the financial industry. AMQP provides a rich set of features related to messaging, including reliable queuing, topic-based publish-and-subscribe messaging, flexible routing, transactions, and security. AMQP runs over TCP, providing reliable (acknowledged) delivery. Messages are encrypted through transport layer security. A message broker controls communication services that are used by publishers and subscribers. The message broker has two main components: the exchange and queues. The exchange receives messages from publishers and distributes them to various queues based on pre-defined roles and conditions. A queue holds communication on a particular topic. Subscribers subscribe to a particular queue in order to be notified when publishers send any message to that queue. MQTT The Message Queuing Telemetry Transport (MQTT), also known as the ISO/IEC 20922 standard, is another publish-and-subscribe messaging protocol. It is much simpler and more focused than AMQP, and is often used for IoT data communication because it is well suited to operate with devices that have limited memory, processing power, and network bandwidth, which describes many IoT devices. Lesson 4: Processing IoT Data | Topic C Pattern Description (from the perspective of the IoT device)
  • 38. 170 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) MQTT: • Operates on top of UDP, so the network doesn't provide reliability (acknowledged delivery) or encryption through Secure Sockets Layer and Transport Layer Security (SSL/TLS). • Provides its own reliability (quality of service) capabilities. • Does not provide encryption. (The communicating applications must provide their own encryption.) • Provides a very simple programming interface that is well suited for communication with lightweight remote devices, requiring little processing power and network bandwidth to operate as a client. • Enables communicating devices (publishers and subscribers) to be separated by distance and time. • Does not require communicating devices to communicate directly with each other or even know each other's identity to communicate. These objectives are accomplished through a scheme that involves a third party, an MQTT broker, which facilitates communication. The MQTT broker processes all incoming messages and distributes them to the appropriate subscribers based on various filters, including message subject, content, and type or class of message. Premises Underlying MQTT While MQTT is well suited for IoT, it is important that developers be aware of premises underlying the protocol. For example: • The publisher can't assume that any subscriber is actually listening to messages that are sent. • Both publisher and subscriber must know how data is structured beforehand. • With subject-based filtering, both parties must know which topics will be used. • Publishers and subscribers must know the hostname/IP address and port of the broker through which they communicate. • While communications can be described as "near real time," they don't need to be particularly fast, and may actually take seconds to complete. In spite of the word "queuing" in its name, MQTT (unlike AMQP) is not actually a messaging queue. It is more like a broadcast. There may be multiple subscribers (possibly thousands) consuming the published data, and MQTT does not "queue" or hold on to messages until they are received by a client. MQTT offers three qualities of service: • QoS 0: At most once delivery: This default QoS doesn't guarantee message delivery. This level minimizes communication and is used in situations where the network is reliable and it is acceptable to occasionally lose messages, such as a sensor feed that publishes readings frequently, where missing some readings will not have an adverse impact on the system. • QoS 1: At least once delivery: Ensures that each receiver gets message at least one time, with the inconvenience of perhaps receiving the same message multiple times. The subscriber sends a message back to the broker to acknowledge receipt. The subscriber must provide its own logic to deal with the possibility of receiving duplicate messages. • QoS 2: Exactly once delivery: Delivers the highest quality of service, where neither the loss nor duplication of messages is considered acceptable. This comes at the cost of increased overhead, make it the slowest of the three levels. MQTT Communication Process MQTT communication between clients (publishers and subscribers) is accomplished through a third party, the broker. MQTT clients vary widely in capability and purpose. They might be a tiny IoT device consisting of not much more than a Wi-Fi connected microcontroller and a sensor or two up to a high-powered server used to process big data. In any case, the client is running an application Lesson 4: Processing IoT Data | Topic C
  • 39. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 171 written in a language like C, C#, C++, Java, JavaScript, or Go that includes a software library providing the support needed to process MQTT communication. The broker provides the hub through which communication is accomplished. Publishers send all messages to the broker. Subscribers receive all messages from the broker. No two clients are ever in direct communication. The broker filters all incoming messages, determines which subscribers want to receive them, and forwards the message on to those subscribers. If authentication and authorization are used, the broker manages these processes for all clients. Message Persistence Depending on your needs, you might set up an IoT messaging system to support only direct messaging in which the receiver must be online in order to receive messages. This approach is commonly used for telemetry data. Alternatively, you might set up a messaging system to hold messages for a time in a cache or queue. If a device is offline, it will still receive the messages when it comes back online. The device could then service the messages in the order in which they were sent. MQTT provides an option to enable or disable message queuing, depending on the requirements for your application. Of course, this adds some complexity to the system. Often you can configure messages so they remain viable for a limited time. A time to live (TTL) parameter can be placed on the message to prevent old messages from being acted on when a device comes back online long after the message was originally sent. Web-Based Communication While lightweight messaging protocols such as MQTT are often recommended for communication involving constrained devices, traditional communications based on HTTP/HTTPS may nonetheless be appropriate for other parts of the IoT system. HTTP/HTTPS is well known, and is well supported for software development through existing protocol stacks and development tools. In situations where latency and bandwidth are not an issue, and when the devices involved have the processing power to implement them, web protocols are a good choice for IoT. Representational State Transfer Application Programming Interfaces (REST APIs), Constrained Application Protocol (CoAP), and WebSocket are web-based communication methods currently useful for IoT, and new standards are under development that will improve web protocols for use in IoT. For example, HTTP/2 will provide more efficient data formats and communication methods. REST APIs Over the years, a variety of approaches have been developed to communicate with web servers as an alternative to the original conventions of HTTP. Representational State Transfer (REST) Application Programming Interfaces (APIs) prescribes a pattern of communication between web clients and servers that proponents of REST say uses HTTP as it was originally envisioned. For example: • GET requests are issued to look up information. • PUT requests are issued to change information. • POST requests are issued to create information. • DELETE requests are issued to delete information. REST is popular because it is simple to program, and builds upon existing systems and features of HTTP. Features such as encryption and reliability are easily implemented using well-known protocols such as SSL/TLS and TCP, with which most developers are already familiar, and HTTP can be programmed through almost any modern application programming language. For example, requesting a sensor reading might be as simple as issuing a GET request on a URL such as https://192.168.1.67/sensors/temperature. Lesson 4: Processing IoT Data | Topic C
  • 40. 172 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Note: Note that WebSocket does not define a messaging format. If you use WebSocket, you need to provide your own protocol for the transmission of data. Some IoT developers use MQTT, running over WebSocket, for messaging. However, this diminishes some of the benefits of using MQTT directly over UDP, as WebSocket requires that a web client protocol stack be implemented on the device. A REST request can return various types of documents, such as simple text or structured formats like XML, JSON, and CBOR. WebSocket Web technologies were designed based on the assumption of a client-server architecture. Users issue requests through a web browser, and a server provides a response. Clever techniques such as AJAX (asynchronous JavaScript and XML) were developed to enable two-way communication without having to load an entire web page, but this approach still assumed that clients initiated communication in the request-response model. The WebSocket protocol, which operates over TCP, has been introduced to address these shortcomings. WebSocket is part of the HTML5 specification. It was standardized by the Internet Engineering Task Force (IETF) in 2011, and is currently supported in all of the major web browsers, including Google Chrome, Microsoft Edge, Internet Explorer, Firefox, Safari, and Opera. Various programming libraries have been developed, enabling programmers to use WebSocket directly in their own applications. To use WebSocket, the server must also support the protocol. Server support can be added through a web application platform such as Node.js. With WebSocket, once both parties (client and server) agree to establish a persistent two-way connection, either the client or server can send messages at any time. The overhead of a full HTTP request is not incurred, so the communication is much faster than conventional HTTP communication. Communication is done over TCP port number 80 (or 443 in the case of TLS- encrypted connections), which is useful in environments that block non-web Internet connections using a firewall. Like other protocols (such as MQTT and AMQP), this efficient approach is well suited for IoT communication, particularly when they just involve the periodic sending of sensor readings. CoAP Constrained Application Protocol(CoAP) provides a lightweight REST API designed for low- power web clients, such as IoT devices. CoAP runs over UDP instead of TCP, which is normally used for REST APIs. CoAP provides a lightweight mechanism for acknowledged delivery. CoAP has two main components: messaging and request-response. The messaging component is responsible for reliability and duplication of messages while the request-response component is responsible for communication. CoAP can run over Datagram Transport Layer Security (DTLS) to encrypt communications. CoAP shares many similarities with HTTP/HTTPS, for easier integration with web applications, and making it easier for developers familiar with HTTP/HTTPS to work with CoAP. Like HTTP/ HTTPS, CoAP uses GET, PUT, POST, and DELETE requests to retrieve, create, update, and delete, respectively, and CoAP can carry different types of payloads, such as XML, JSON, CBOR, and other data formats. Guidelines for M2M Communication Follow these guidelines when planning and implementing M2M communication for IoT. Construct an MQTT CONNECT Message A minimal MQTT CONNECT message includes the following fields of data: Lesson 4: Processing IoT Data | Topic C
  • 41. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 173 • ClientId: Provides each MQTT client connecting to a MQTT broker with a unique name. • Clean Session: Set to false, and the broker will store all subscriptions for the client and all missed messages, when the subscriber specifies a QoS of 1 or 2. Set to true, and the broker won't store anything for the client and will also purge all information from a previous persistent session. • Username/Password: MQTT can be configured to authenticate the client to authorize access to various topics. However, the password is sent in plaintext unless it is encrypted or hashed by the application, or run over a secure transport layer. Some brokers support using an SSL certificate for authentication, in which case no username and password is needed. • Will Message: MQTT can be configured to automatically notify other clients when a client disconnects ungracefully (due to power loss, for example). The message and topic that will be sent for this notification are specified in this field. • KeepAlive: This field specifies the time interval after which the client will send a ping message to the broker to keep the connection alive. This mechanism enables the client and broker to determine if the other machine is still alive and reachable. Evaluate an MQTT CONNACK Message An MQTT CONNACK message includes two fields of data, which the client can evaluate to determine if connection to the broker was successful: • sessionPresent: Indicates whether the broker already has a persistent session with the client from a previous interaction. If a client connects and has set CleanSession to true, this flag will always be false, because no session is available. If CleanSession is set to false, the flag value depends on whether session information is available for the ClientId. If stored session information exists, then the flag is true. Otherwise, it is false. • returnCode: The value returned here indicates either a successful attempt or the reason why it was unsuccessful: • 0 - Connection Accepted • 1 - Connection Refused, unacceptable protocol version • 2 - Connection Refused, identifier rejected • 3 - Connection Refused, Server unavailable • 4 - Connection Refused, bad user name or password • 5 - Connection Refused, not authorized Select an M2M Communication Protocol The HTTP/HTTPS web protocols and REST are well known and widely implemented technologies for web and cloud services. Although they have many applications in IoT, there are some situations for which they are not well suited, and other protocols may be more appropriate. For example: • Information publishing: HTTP was designed to support a request-and-response model, where the data exchange is initiated by the requester. However, many IoT scenarios work best with a publish-and-subscribe model, where a publisher (such as an IoT sensor device) initiates the exchange of data by publishing it to a particular topic. Any clients that have subscribed to that topic are then notified that new data has been sent. • Distributing very small packets of data in huge quantities efficiently: Some IoT applications require large numbers of sensing devices, which typically deliver very small chunks of data. The containing headers for HTTP may easily outsize the tiny portions of data an IoT device may send, making it an inefficient format for sending small packages. This inefficiency is magnified by the number of devices sending data and the frequency with which they send data. • Real-time delivery of data: When small portions of data must be delivered in real time, it is essential to keep communication tasks streamlined and fast. It requires a lot of processing to compose an HTTP request or response, particularly when encryption is involved. Simpler protocols may be more appropriate in some circumstances. Lesson 4: Processing IoT Data | Topic C
  • 42. 174 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) • Minimal consumption of battery power: Simpler protocols help to conserve battery power on IoT devices by reducing the processing tasks they must perform. Case Study: MQTT Figure 4-6: MQTT publishers and subscribers. In the Greene Organix greenhouses, separate IoT devices provide sensors and actuators. Sensor devices use MQTT to publish their sensor data, which other clients on the local network may subscribe to. The local IoT gateway performs local data collection and local command and control tasks. • Local data collection: The IoT gateway subscribes to sensor topics to obtain the sensor data. It runs a background process to filter and aggregate the data, which it periodically forwards to cloud services. • Local command and control: The IoT gateway also runs a process to perform command and control operations on devices within the greenhouses, controlling lighting, air flow, heating and cooling, and irrigation. Based on the data it collects and analyzes, it publishes messages to identify actions that need to be taken to adjust these factors. • Actuator devices subscribe to topics pertaining to actuator commands. They respond to commands that pertain to them, making adjustments to the various factors under their control. Lesson 4: Processing IoT Data | Topic C
  • 43. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 175 Note: Your instructor may direct you to enter a different path, depending on where your course data files were set up. Before You Begin The course data files are installed on your computer in C:095024Data. These files include a customized version of the Arduino IDE (located in C:095024DataToolsarduino), which contains example programs that you will use in this course. The example programs are contained in the File > Sketchbook sub-menus inside the Arduino IDE. The ESP8266 microcontroller is connected to a USB port on the development computer. You have successfully connected the device to the Wi-Fi network, and the SSID and password are now cached on the device. You have installed the light sensor and 10K resistor on the breadboard. The Arduino IDE is not running. Scenario You are going to program the IoT device to communicate using MQTT messaging. But before you modify the device, you will experiment with the MQTT broker and command-line clients to see how it operates and demonstrate it is working effectively. 1. View the files in the Mosquitto application directory. a) Select the Windows Start button to show the Start menu. (Or from the keyboard, you can press Ctrl+Esc.) b) Type cmd and press Enter to open a Command Prompt console. c) In the command console, enter the cd c:095024DataToolsmqtt_broker command to switch to the directory where the Mosquitto MQTT broker is stored. Lesson 4: Processing IoT Data | Topic C ACTIVITY 4-5 Launching and Testing an MQTT Broker
  • 44. 176 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) d) Enter the dir /w command to list files in the directory. This directory contains various Windows executable files. In this activity, you will use: • mosquitto.exe: An MQTT broker application • mosquitto_sub.exe: An MQTT publisher client • mosquitto_pub.exe: An MQTT subscriber client e) Enter the title Broker command to give the Command Prompt window a title. In this activity, you'll open three more Command Prompt windows. Titling your windows will help you to keep them straight. You'll open one window to use as a message publisher, and you'll open two other windows to use as message subscribers. Normally, the broker and clients would be running on different computers or devices, but this will enable you to test how MQTT works. 2. Open and arrange three more Command Prompt windows. a) In the Broker window, enter the start "Publisher" cmd command to start another Command Prompt console. • When you use the start command to open another command console this way, it opens at the directory you were in (c:095024DataInstallmqtt_broker), saving you the extra step of having to enter the cd command again. • You also titled the window as you opened it. Lesson 4: Processing IoT Data | Topic C
  • 45. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 177 b) Arrange the two windows on the left half of your desktop, with Broker on top and Publisher on bottom, as shown. c) In the Publisher window, enter the start "Subscriber One" cmd command to start another Command Prompt console. d) In the Subscriber One window, enter the start "Subscriber Two" cmd command to start another Command Prompt console. e) Arrange the two subscriber windows on the right half of your desktop, with Subscriber One on top and Subscriber Two on bottom, as shown. 3. Show help for the mosquitto command. Lesson 4: Processing IoT Data | Topic C
  • 46. 178 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Note: This is a network application. If you are prompted with a Windows security alert, check the type of network you are using, and select Allow access. a) In the Broker window, enter the mosquitto --help command to view help on using the mosquitto command. • The command accepts several optional parameters. • The -v parameter displays all logging information in the console window, which is useful for testing MQTT messaging. 4. Run the MQTT broker application. a) In the Broker window, enter the mosquitto -v command to launch the Mosquitto broker in verbose mode. • Mosquitto launches using the default configuration. • In the default configuration, it listens for messages from clients on port 1883. • You are not returned to a command prompt because the Mosquitto broker is still running. It is now waiting for messages from publishers and subscribers. 5. Subscribe to two MQTT topics. a) In the Subscriber One window, enter the mosquitto_sub -t building/light -t building/temperature -v command. • For this test, you will use the default IP address (localhost) and port (1883), so you don't need to specify them here. • You will subscribe to two topics: building/light and building/temperature. Lesson 4: Processing IoT Data | Topic C
  • 47. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 179 b) Observe the Subscriber One console (where you just entered the command to subscribe). The mosquitto_sub command seems to have hung, since no response is shown. However, it hasn't hung. It is just waiting to receive the messages it has subscribed to. c) Observe the Broker window. The broker updated its display with several log entries as soon as you subscribed. • The broker responded to the connection request by sending a CONNACK message back to the subscriber. • The broker acknowledged the two SUBSCRIBE requests by returning a SUBACK message for each one. • The subscriber sends periodic PINGREQ ping request messages to keep the connection alive, and the broker responds to each one with a PINGRESP message. 6. Publish to an MQTT topic. a) In the Publisher window, enter the mosquitto_pub -t building/temperature -m "68.0" command. • You are publishing to the MQTT broker using the default address of 127.0.0.1 (on the local computer) on port 1883. • The topic is "building/temperature". • The message is "68.0". • In a real-world implementation, the publisher might be an IoT device posting the latest temperature reading from its sensors. Lesson 4: Processing IoT Data | Topic C
  • 48. 180 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) b) Observe the Broker console. • The broker responded to the connection request by sending a CONNACK message back to the publisher. • The publisher sent a PUBLISH request to send the message using the topic "building/ temperature". • The broker received the PUBLISH request, and then sent the message to the subscriber. (In actual use, the message might have multiple subscribers.) • The publisher then disconnected. c) Observe the Subscriber One console. The subscriber has received the message, and shows the value that was sent. In a real-world implementation, the subscriber might be a storage system that is collecting the data for later analysis. 7. Launch a new subscriber client. a) In the Subscriber Two window, enter the mosquitto_sub -t building/temperature -v command. b) Observe the Broker window. The broker acknowledged the new connection request and subscription to the building/temperature topic. c) Observe the Subscriber Two window. The client is waiting for messages. A building/temperature message had been sent earlier, but by default old messages aren't retained and sent to new subscribers, so the message you sent to building/temperature is not received by the new client. 8. Publish messages on different topics. a) In the Publisher window, enter the mosquitto_pub -t building/temperature -m "72.0" command. b) Observe the Subscriber One and Subscriber Two windows. Both subscribers receive the message, and show the new value (72.0). c) In the Publisher window, enter the mosquitto_pub -t building/light -m "100" command. d) Observe the Subscriber One and Subscriber Two windows. Only Subscriber One received the message. Subscriber Two did not subscribe to the building/light topic. Lesson 4: Processing IoT Data | Topic C
  • 49. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 181 9. Which communication pattern is used by the Publisher (sensor device) in this scenario—telemetry, notification, inquiry, or command? 10.Why is a publish-subscribe model appropriate for this scenario? Lesson 4: Processing IoT Data | Topic C
  • 50. 182 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Note: You will need to refer to this address several times in the activities that follow. Before You Begin The course data files are installed on your computer in C:095024Data. These files include a customized version of the Arduino IDE (located in C:095024DataToolsarduino), which contains example programs that you will use in this course. The example programs are contained in the File > Sketchbook sub-menus inside the Arduino IDE. The ESP8266 microcontroller is connected to a USB port on the development computer. You have successfully connected the device to the Wi-Fi network, and the SSID and password are now cached on the device. You have installed the light sensor, a 10K resistor, and a DHT11 sensor on the breadboard. The Arduino IDE is not running. In separate command-line consoles, you have set up an MQTT broker, publisher, and two subscriber clients. Scenario Now that you've set up and tested the MQTT broker, you'll enable the IoT device to use it to publish sensor data. 1. Load a sketch that publishes data values using MQTT. a) From the Windows Start menu, select Arduino to start the Arduino IDE. b) Select File→Sketchbook→Processing IoT Data→MQTT_Publish. c) Close the sketch that is still open in the background so MQTT_Publish is the only sketch that is open. 2. Set the MQTT server address. a) In the Publisher command prompt window, enter the ipconfig command and note the IPv4 address your MQTT broker is running on. Write your computer's IP address in the space provided. My IP Address: . b) Revise line 9 to replace YourName with your name. c) In line 10 of MQTT_Publish, change the IP address to match yours. 3. Start the process to compile and upload the sketch. Lesson 4: Processing IoT Data | Topic C ACTIVITY 4-6 Using MQTT to Send Data from an IoT Device
  • 51. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 183 a) Select Tools→Serial Monitor to display the serial monitor window, and ensure the baud rate is set to 9600 baud. The previous program is still running on the IoT device. For now, you can ignore the output shown in the serial monitor. b) Select Sketch→Upload. 4. While the sketch is being compiled, examine the source code. a) Examine code that has been added to support MQTT. • The IP address and port of the MQTT server are identified in lines 10 and 11. • An object that will be used to program the MQTT communication is created in line 17. This object uses the software library identified in line 1. b) Examine the code that publishes the sensor data to various topics. Lesson 4: Processing IoT Data | Topic C
  • 52. 184 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) Note: Many ESP8266 boards have an LED light that flashes when the device is communicating over Wi-Fi. 5. Examine how the MQTT broker and subscribing clients process data published by the remote sensor device. a) When the program is uploaded and running on the device, examine the Subscriber One and Subscriber Two command consoles. Subscriber One is receiving temperature and light values from the remote sensor device, and Subscriber Two is receiving temperature values. b) Examine the Broker command console. The MQTT broker is processing messages received from the sensor device, and forwarding them to subscribers. 6. Clean up the desktop. a) Close the Subscriber One and Subscriber Two command console windows, but leave the other two console windows open. 7. What are the benefits of minimizing the processing done by sensor devices and having them use protocols like MQTT to forward data to subscribers? Lesson 4: Processing IoT Data | Topic C
  • 53. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 185 Before You Begin The course data files are installed on your computer in C:095024Data. These files include a customized version of the Arduino IDE (located in C:095024DataToolsarduino), which contains example programs that you will use in this course. The example programs are contained in the File > Sketchbook sub-menus inside the Arduino IDE. You have successfully connected the device to the Wi-Fi network, and the SSID and password are now cached on the device. You have installed the light sensor, a 10K resistor, and a DHT11 sensor on the breadboard. The Arduino IDE is running. In separate command-line consoles, the Broker and Publisher windows are open. Scenario Your ESP8266 may light an on-board LED to indicate when the Wi-Fi connection is in use. You can program this LED for other purposes. There is another LED on the board that you may also use. These LEDs can be useful to display diagnostic information during testing and setup. When they are not needed, it would be good to disable them. You could add a switch for this purpose, but since a messaging interface is already available through MQTT, you'll use it to enable the LED state to be controlled through messaging. A client app on a mobile device can be used to turn them on or off as needed. 1. Load a sketch that publishes data values using MQTT. a) In the Arduino IDE, select File→Sketchbook→Processing IoT Data→Control_Instructions. b) Close the MQTT_Publish window that is still open in the background so Control_Instructions is the only sketch that is open. 2. Set the MQTT server address. a) Revise line 9 to replace YourName with your name. b) In line 10, change the IP address to match the IP address of your computer, which you wrote down earlier. 3. Start the process to compile and upload the sketch. a) Make sure the serial monitor window is shown, and ensure the baud rate is set to 9600 baud. b) Select Sketch→Upload. 4. While the sketch is being compiled, examine the source code. Lesson 4: Processing IoT Data | Topic C ACTIVITY 4-7 Using MQTT to Receive Control Instructions
  • 54. 186 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110) a) Examine code that has been added to process incoming MQTT messages in lines 29 through 38. LED visibility is set based on the incoming value from the building/set_leds_visible topic. 5. After the program is compiled and uploaded, test your ability to set the LED state through MQTT. a) On the IoT device, examine the onboard LEDs. They are turned off by default. b) In the Publisher command console, enter the mosquitto_pub -t building/ set_leds_visible -m "on" command. A message in the console indicates that the LED is now on. c) On the IoT device, examine the LEDs. • After the message is received and processed, the LEDs update the device status each time sensor readings are taken. • The LED that normally shows the Wi-Fi state (typically blue on the NodeMCU ESP8266) illuminates when the light reading is lower than the previous reading. • The other LED (typically red) illuminates when the light reading increases. • When the reading is the same as before, neither LED illuminates. 6. Turn the LEDs off again. a) In the Publisher command console, press F3 to recall the previous command. b) Press Backspace three times and type off" to revise the command so it will turn the LEDs off. The completed command is mosquitto_pub -t building/set_leds_visible -m "off" c) Press Enter. A message in the console indicates that the LEDs are now off. d) On the IoT device, examine the LEDs. After the message is received and processed, the LEDs stop flashing. 7. Clean up the desktop. a) Exit the Arduino IDE. b) Close all command consoles. Lesson 4: Processing IoT Data | Topic C
  • 55. Certified Internet of Things (IoT) Practitioner (Exam ITP-110) | 187 Note: Check your CHOICE Course screen for opportunities to interact with your classmates, peers, and the larger CHOICE online community about the topics covered in this course or other topics you are interested in. From the Course screen you can also access available resources for a more continuous learning experience. Summary In this lesson, you used microcontroller I/O to interact with components attached to an IoT device, and you implemented M2M communication to enable data processing tasks to be delegated to the most appropriate location within IoT infrastructure. What communication protocols will you use for M2M communication in your own projects, and why? Why are edge computing approaches so important for IoT data processing? Lesson 4: Processing IoT Data |
  • 56. 190 | Certified Internet of Things (IoT) Practitioner (Exam ITP-110)