Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014, Barcelona
2 likes•1,598 views
The document discusses the extensive tracking capabilities of smartphones, which utilize numerous sensors to monitor human behavior and environmental data. It highlights the creation of the 'explore' app that collects and analyzes sensor data, aimed at understanding user behavior while ensuring privacy and data protection. The document also raises philosophical questions about the implications of such tracking technologies on privacy, identity, and societal norms.
Behavioral Analytics with Smartphone Data. Talk at Strata + Hadoop World 2014, Barcelona
- 1. Telling the Story of People's Behavior with Smartphone Data J oerg Blumtritt @jbenno 1
- 2. Mobile 2
- 3. Mobile 3 • Two billion people use smartphones (three times more than users of PCs) • Smartphones are far more than just „mobile computers“: they carry more than twenty sensors or probes, continuously monitoring our behavior and our environment. • Mobile is becoming the most important source of human generated data and surpasses social networks. • Apart from people using their phones, there are more than five billion mobile devices, connected to objects, like e.g. cars. These build the Internet of Things.
- 4. 4
- 5. 5 • Smartphones carry a phalanx of sensors and track all kind of environmental data. • Our movements and immediate surroundings are monitored by gyroscope, accelerometer, luminosity sensor in the camera, microphone etc. • The location is captured by satellite connection and mobile network. • Proximity can be trackt via bluetooth or Wifi signal (which just becomes systematically useable with the iBeacon)
- 6. 6 Satellites MEMS Radio GPS NFC Wifi, Bluetooth 4G Gyroscope, Accelerometer Microphone, Camera Temperature, Air Pressure, Compass, ... Supplies Battery
- 7. 7 Rich Context Simple Context Events Raw Sensor Data After Pei et.al.: "Human Behavior Cognition Using Smartphone Sensors", Sensors 2013, 13, 1402-1424; doi:10.3390/s130201402
- 8. Our App: explore • We started our own app 'explore': • explore tracks all kinds of sensor data on the smartphone. The data can be collected for analysis, and it can trigger interactions (like asking questions or offering suggestions). • The open beta is available on Google Play Store; the iOS version should be ready by J uly 2014. 8 Datarella - Joerg Blumtritt
- 9. 9 Rich Context Simple Context Events Raw Sensor Data
- 10. Data • Sensor data is generated mostly in forms of tables, locally stored as SQL databases for each app. We transfer the data to analyze it, e.g. visualize geo-location on a map. 10 Datarella - Joerg Blumtritt
- 11. • Not all data is telling streightforeward like geolocation. Gyroscope data e.g. is measured in three dimensions. • This plot shows typical artefacts: the spikes shooting out of the clutter in regular intervals. These are caused by hardware inaccuracies, or also by aliasing effects. • The artefacts are unique to each device, like a fingerprint, and can identify the source of the data. 11
- 12. 12 Rich Context Simple Context Events Raw Sensor Data
- 13. Events • To see what happens, we have to process the data. How people move arround is visible through the gyroscope - you see the turns, changes in directions ect. • With gyroscopic data in combination with acceleration and speed, also the means of transportation can be revealed: walking has a distinct signature, driving by car shows more changes in directions then sitting on a train, etc. • However: the data is noisy; artefacts emerge from different brands of the sensors, of glitches in the operating systems, and also can be caused by environmental influences. • Take e.g. the rhytmik spikes in the picture below: nobody would turn rhythmically and so fast. • So we have to preprocess the data in the app, to really see, what happens. 13 Datarella - Joerg Blumtritt
- 14. What is behavior? • The normalized gyroscopic data on the right shows the movements of a person going from her desk into the kitchen, fixing a pot of tea, leaving the kitchen and returning to her desk. • Sampling rate was 10s, timeframe is 15min. • We notice episodes of different behavior: • turning sharply • walking • turning smoothly • walking again • entering the kitchen, preparing the pot • waiting for the water to boil • standing up, leaving the kitchen • sitting down again 14 0,0 1,0 2,0 3,0 4,0 5,0 6,0 7,0 8,0 9,0 1 4 7 1013161922252831343740434649525558616467707376798285 Datarella - Joerg Blumtritt
- 15. Complex Event Processing • Simple events, like changing direction, entering an area of specific geo- coordinates, or having moved for a specific time span can be combined to complex events. • EPL (event processing language) offeres a way to listen to the data stream and detect the occurance of events. • EPL looks like SQL, but instead of tables, the search goes into the data stream. • For our app, we define events, boolean- combine these events in a GUI and parse the definition in the app via J SON doc. • The event processing itself takes place in the app - no network connection is needed. 15 SELECT ID AS sensorId FROM ExampleStream RETAIN 60 SECONDS WHERE Observation= '' Outlet" Datarella - Joerg Blumtritt
- 16. 16 Rich Context Simple Context Events Raw Sensor Data
- 17. Travel • Studying the means of transportation, they paths, people choose for their communte or travel, is a streightforward application of our data. • We work e.g. for airports to optimize the shops they would offer to passengers. Since many passengers come from other cultures, it is not an easy task for an airport (or in general for a shopping mall) to learn the preferences of potential clients - not consitent shopping data or market research is available. • So, e.g. we incentivize passengers from China to let us accompany their stay in Euorpe with our app 'explore'. So we can understand, what they wanted to buy, if they succeded and if they would have missed anything, that an airport could have offered to them. 17 Datarella - Joerg Blumtritt
- 18. 18 Rich Context Simple Context Events Raw Sensor Data
- 19. Usability • To avoid the shortcommings of the other social research apps, we focused on the user interface, to make it as "mobile" as possible. 19 Datarella - Joerg Blumtritt
- 20. Interactions • explore can ask questions or offer suggestions, triggered by data. 20 Datarella - Joerg Blumtritt
- 21. Quantified Self • explore offers clear and simple analytics of the data collected. People can also get their raw-data for their own purposes. • We want people to be aware what we (and other apps) do on the phone. So we do not only tell in advance, we also show what sensors are activated and give the opportunity to opt-out per sensor. • Since we reflect the results of our tracking as well as questionnairs and interactions in form of diagrams and sumaries, we hope, people will realize what we are doing and can act self-determined. • Of couse we respect take-down notices: if people ask for their data to be deleted, we follow their request (which btw is also required by German data protection laws); this is also a reason for us not to use common cloud storage and cloud computing platforms, since we would not have control over the back-up. Datarella - Joerg Blumtritt21
- 22. 22 • Use case: Tracking Chinese Passengers. • An international airport wants to learn, what Chinese passengers buy, what they consider a pleasent shopping experience, and what they expect. • We recruited a panel of Chinese passengers before they left from China to Europe, and accompanied them with our app. • We learned where they went and could ask them about their experiences.
- 23. 23 • Use case: Driver Timeline • We built an app that tracks the driving and computes more abstract events from its data, like „stuck in a traffic jam“. • The events are displayed in form of a timeline, and can be shared to others.
- 24. Battery • Battery data is both interesting in itself, and also important to maintain the app usable. • Battery consumptions is telling a lot about the environment of the phone: temperature, moisture, even air pressure can be derived using the change in charge. 24 Datarella - Joerg Blumtritt
- 25. Identify whereabouts by location-specific magnetic field • Every place has a distinct signature of the magnetic field (in strenght like shown on my own tracking data on the right as well as in bearing). • So even if someone decided to not-track geolocation, we might still get sufficient information on their whereabouts via other measurements. • That this is not hypothetical can be seen on the diagram: the field's signature of my home is different from other places, I stayed during that week. 25 Datarella - Joerg Blumtritt
- 26. 26 • We try to help people understand what their phone does: • First, in our apps, we provide info on which probes are active and give the choice to switch a single probe off to the user. • Second, we also feedback visualizations on the data readings to the users in our apps (next slide). • Third, we give users access to their data via web api.
- 27. 27
- 28. 28
- 29. Spooky Wifi Self-Tracking vs. Others-Tracking • You can't avoid tracking others involunarily, too. This is a problem. People might be aware, what they themselves are doing. But others might be tracked along without giving their consent. • Wifi is a good example of "others- tracking": all wifi signals within reach are tracked by the phone. It tells a lot about other people; not only about the devices they use. 29 Datarella - Joerg Blumtritt
- 30. Postprivacy, and communalization of private life • In Neal Stephenson's "Snow Crash", we read about the 'Central Intelligence Corporation' - a commercialized version of today's NSA. Mobile health, computational social science, and mass measurement of environmental influences are obvious and benign applications of QS for the public good. With quantifying and making public, what European data protection law defines as "the most intimate personal data", however do we transform the current "knowledge-database" character of the Net along with its communication-networks to something new, something that might become similar to Stephenson's vision? • Could this even lead to Teilhard's (resp. McLuhan's) angelization of humans, not only connected via social media but bodily knit into the data? Would we rather end up in a rally bucolic global village with moral control by the panoptic community (and an inherent abelism that comes with a village life)? In both aspects, representative aggregates like society as well as the concept of the individual might be rendered obsolete. 30 Datarella - Joerg Blumtritt
- 31. "Privacy" 31 Datarella - Joerg Blumtritt
- 32. "Data Protection" 32 Datarella - Joerg Blumtritt
- 33. Becoming cyborgs? • The bodily extension into the data squere - this is what cyborgism is really about. • People like Neal Harbison or Enno Park are pushing the discussion in that direction: How do we maintain posession of our bodies? What ethic framework has there to be set-up? How do we avoid technological extensions becoming "black boxes" that control us, rather than we do them? • So it is worthwhile to follow the proceedings of the Cyborg e.V that Enno founded. 33 Datarella - Joerg Blumtritt
- 34. Some links • http://datarella.com/blog • http://beautifuldata.com my blogs. • http://twitter.com/jbenno/bigdata a data science related twitter list. 34 Datarella - Joerg Blumtritt
- 35. J örg Blumtritt @jbenno Datarella GmbH Oskar-von-Miller-Ring 36 80333 München 089/44 23 69 99 [email protected] Datarella - Joerg Blumtritt35
- 36. 36 @jbenno