Delivering near real time mobility insights at swisscom
0 likes•220 views
The document discusses Swisscom's approach to delivering near real-time mobility insights using smart data and big data architecture, emphasizing privacy preservation while providing aggregated analytics for urban planning. Key applications include optimizing transit routes, parking expansions, and electric vehicle charging station deployment. Swisscom enforces data protection laws and ensures that individual user identification is neutralized in its analytics processes.
![Scala (1/2)
typeChronoHistory = List[UEupdate] @@ Chronological
typeAnteChronoHistory = List[UEupdate] @@ AnteChronological
implicit classChrono(l: List[UEupdate]){
def asChrono: ChronoHistory = {
chronoCheck(l)
l.asInstanceOf[ChronoHistory]
}
def asAnteChrono: AnteChronoHistory = {
anteChronoCheck(l)
l.asInstanceOf[AnteChronoHistory]
}
}](https://image.slidesharecdn.com/deliveringnearreal-timemobilityinsightsatswisscom-161003110840/85/Delivering-near-real-time-mobility-insights-at-swisscom-23-320.jpg)
![Locality-sensitive hashing short histories
A family H of hashing functions is -sensitive if:(r, cr, , )p1 p2
if then
if then
p–q ≤ r P [h(q) = h(p)] ≥rH p1
p–q ≥ cr P [h(q) = h(p)] ≤rH p2
More :
Locality Sensitive Hashing By Spark, Uber, Spark Summit
A Gentle Introduction to Locality-Sensitive Hashing with Apache Spark,
Scala by The Bay](https://image.slidesharecdn.com/deliveringnearreal-timemobilityinsightsatswisscom-161003110840/85/Delivering-near-real-time-mobility-insights-at-swisscom-28-320.jpg)
Delivering near real time mobility insights at swisscom
- 1. Delivering near real-time mobility insights at Swisscom François Garillot [email protected] @huitseeker
- 4. Positioning users in a modern network no radio-goniometer at scale cell of attachment has position, beam characteristics over history, best position ~200m
- 5. Positioning at specific locations handovers at specific cell-to-cell location phone needs to be active
- 6. Positioning with more precision better positioning with excellent data sources: 3G : GPEH 4G: LTE-CTR
- 7. Trajectory data mining time series reconstruction trajectory segmentation map matching, clustering mode of transport detection ...
- 8. How to create value with positioning at Swisscom ? with competitive analytics & data sources, and by making sure it embodies the right values.
- 9. Smart Data
- 10. On (not) tracking (any users) "Swisscom strictly complies with all applicable legislations, in particular with the telecommunications law and the data protection initiative." Jürg Studerus, Swisscom Senior Manager, Corporate Responsibility
- 11. Smart Data : Big Data without Big Brother Privacy preservation is an asset It makes sense to care as much about your customer as they do about you. We technically enforce this answering only synoptic questions, no individual ones, with data flow control : we neutralize quasi-identifiers at every stage
- 12. Swisscom mobile subscribers source: xavierstuder.com, MD&A reports
- 13. Our choices public good applications: making Switzerland run better, understanding places, not individuals, all results presented aggregated, anonymized.
- 14. Markets
- 15. A first product : City "It's a dream for civil engineers" -- Alexandre Machu, Urban systems engineer, Pully
- 16. Demo time
- 17. Usages New roads to divert transit traffic out of downtown (informs a 50M$ project) Parking lot expansion and transformation (informs a 10M$ project) Electric car charging station deployment
- 19. In the backend
- 20. Spark configuration essentials for enterprise jobs spark.executor.memory="not the default 1g" spark.kryo.registrator="something custom"// and companions spark.shuffle.service.enabled="true" spark.dynamicAllocation.enabled="true" spark.deploy.recoveryMode="ZOOKEEPER" spark.deploy.recoveryDirectory="/path/to/state" spark.deploy.zookeeper.url="quorumMachine1:2181, ..." NOT the only valuable settings, see https://techsuppdiva.github.io for more
- 21. See Also
- 22. In the front-end
- 23. Scala (1/2) typeChronoHistory = List[UEupdate] @@ Chronological typeAnteChronoHistory = List[UEupdate] @@ AnteChronological implicit classChrono(l: List[UEupdate]){ def asChrono: ChronoHistory = { chronoCheck(l) l.asInstanceOf[ChronoHistory] } def asAnteChrono: AnteChronoHistory = { anteChronoCheck(l) l.asInstanceOf[AnteChronoHistory] } }
- 24. Scala (2/2) implicit def reverseChrono(l: ChronoHistory): AnteChronoHistory = l.reve implicit def reverseAnteChrono(l: AnteChronoHistory): ChronoHistory = l.
- 26. Selecting users on a path of Interest Massive discrepancy between # of users (2-3E6) and # of interesting users (1.5E3 on test segments) Filtering interesting time series.
- 27. Graph matching
- 28. Locality-sensitive hashing short histories A family H of hashing functions is -sensitive if:(r, cr, , )p1 p2 if then if then p–q ≤ r P [h(q) = h(p)] ≥rH p1 p–q ≥ cr P [h(q) = h(p)] ≤rH p2 More : Locality Sensitive Hashing By Spark, Uber, Spark Summit A Gentle Introduction to Locality-Sensitive Hashing with Apache Spark, Scala by The Bay
- 29. Computing speeds: Solving graph constraints a speed comes from a user well-positioned, twice plus route knowledge given a history of cells, where was the user, exactly ?
- 30. Solving graph constraints just a few users left in computation at this stage so a lot invested in > linear complexity algorithms
- 31. Data Challenges
- 32. Crucial elements Quality, reliability of data sources Automated ground truth checking sensors TEMS fleet What's the ground truth for mode of transport, domicile, etc ? Colleagues and friends volunteers
- 33. In the works Accuracy improvements More features (see you Spark Summit EU!) Streaming for city Thank you