Build Your Own Custom Mobile Analytics with Node and MongoDB

Editor's Notes

• #2: Hello My Name is MATT and I am a DEVELOPER ADVOCATE for STRONG LOOPBefore we get into the code I would like to take 2 Slides & 30 seconds and tell you a little about STRONGLOOP and NODEJS , and then Im going to jump into the demo and the technology, we have a lot of ground to cover and only about 20 minutes to do it in, and I want to make sure and save some time for any Q&A you might have. SO LETS GET STARTED.
• #3: STRONGLOOP IS ALL ABOUT NODE StrongLoop founders Bert Belder and Ben Noordhuis are the core Node contributors We offer PRODUCT --> Solutions and Support around NodeJS WITH - OUR STRONGLOOP DISTRO THE StrongLoop NodeJS DISTRO makes it easy for you to get up and going fast with NODE We have created a few Node Modules as well -Config Loader and Module Loader to help you recursively load configure node modules FOR YOUR NODE STACK MQ: application message queues for node clusters AND Task Emitter to help you run task operations recursively and in parallel.
• #4: STRONGLOOP IS ALL ABOUT NODE StrongLoop founders Bert Belder and Ben Noordhuis are the core Node contributors We offer PRODUCT --> Solutions and Support around NodeJS WITH - OUR STRONGLOOP DISTRO THE StrongLoop NodeJS DISTRO makes it easy for you to get up and going fast with NODE We have created a few Node Modules as well -Config Loader and Module Loader to help you recursively load configure node modules FOR YOUR NODE STACK MQ: application message queues for node clusters AND Task Emitter to help you run task operations recursively and in parallel.
• #11: NodeFly:Operational console(available on-prem)Performance and Memory MonitoringCluster management & scalingError reportingProfiling
• #12: Welcome BACK !!! … I NEVER LEFT.ApplicationTabGroup.jsvar NEWWIN = require('ui/handheld/WindowEngagement');var APPLISTING = require('ui/handheld/SLWindowAppListingNodeApps');Ui/SLNetworkAccessLayer.jsui/handheld/SLWindowAppDetail.js//Ti.App.fireEvent('WindowAppListingRefreshData',{data:['1','2','3']});
• #13: BLE- Xamarin - Realtime/localhost/Users/mattschmulen/trees/strongloop/demoApp-ble-xamarin/presentation-app/public/slides/deck-strongloop/index.html#calltoaction⌘-Shift-F Chrome Full Screen. Hello My Name is MATT and I am a DEVELOPER ADVOCATE for STRONG LOOPBefore we get into the code I would like to take 2 Slides & 30 seconds and tell you a little about STRONGLOOP and NODEJS , and then Im going to jump into the demo and the technology, we have a lot of ground to cover and only about 20 minutes to do it in, and I want to make sure and save some time for any Q&A you might have. SO LETS GET STARTED.------STRONGLOOP IS ALL ABOUT NODE StrongLoop founders Bert Belder and Ben Noordhuis are the core Node contributors We offer PRODUCT --> Solutions and Support around NodeJS WITH - OUR STRONGLOOP DISTRO THE StrongLoop NodeJS DISTRO makes it easy for you to get up and going fast with NODE We have created a few Node Modules as well -Config Loader and Module Loader to help you recursively load configure node modules FOR YOUR NODE STACK MQ: application message queues for node clusters AND Task Emitter to help you run task operations recursively and in parallel. ------Why is Node So popular for mobile API servers ? First, NODE releases JAVA-SCRIPT from the browser Allowing you to run javascript code on your SERVER. Unlike legacy server technology NODE IS ASYNCHRONOUS by DE-SIGN this means LESS BLOCKING AND faster RESPONSE RATE to your mobile APP it's LIGHT WEIGHT implementation MEANS THAT YOU CAN SUPPORT MORE END USERS WITH LESS HARDWARE. THIS MEANS YOUR Server API-TIER Responds faster to the Client App , reducing latency improving user experience Additionally ...Javascript's popularity as a client language means that it is likely you can use the same LANGUAGE you use for your client .. TO BUILD your SERVER API Giving you the added benefit of leveraging the same developer, and the same skill set, using javascript front to back, ALSO means reusable code and less bugs THIS helps you DELIVER your app FASTER The large ecosystem of tools and components that are available for you to use currently NPM ( Node Package Manager ) has over 32 THOUSAND PackagesJavascript is now a language for your full Application lifecycle: Create your client, your Server, and use it for tooling in your Delivery and Continue. Integration the power of javascript and Node JS is #REMARKABLE---This makes node great for mobile API Middle Tiers, connecting to embedded devices AND OF COURSE THE WEB--- Todays demo is about BLE mobile devices and storing "LOTS & LOTS" of analytics information in MongoDB through a NodeJS API middle Tier. I just finished up the Demo on the flight so if you want the source code and the Guide make sure and checkout our StrongLoop Blog, Im not going to drop the Blog until mid July, so if you want to the code and step-by-step GUIDE before then… Just drop me an email. --- A Quick overview of BlueTooth and how this works I was first turned on To BLE by a little company in Austin Tx called Emmoco, I would really recommend checking them out. They released a nice BLE board that helps you integrate mobile control with everything from Industrial Robotics to your coffee pot. One of their better know integrations is the Auto system found in Silver Car. - great group with some cool tech BLE or BlueTooth Low Energy or BT 4.0 is significantly different From the BT 3.2 or ( "Classic" ) Blue Tooth that you use to talk to your wireless headset or mouse. What you need to know, is: - Bluetooth low energy is really good for applications requiring episodic or periodic transfer of small amounts of data. - if your Streaming ( like your headset, use classic ) - Super low Power Consumption - its in sleep most of the time the peak ( max ) power consumption is 15 mA and the average power is ~ 1 micro Amps ( 1 uA ) - Low power consumption means that embedded devices like this can run for about a year with a small button *Watch* Battery. It is actually low-power (as the name suggests). How much? Bluetooth 4.0 consumes half as much energy when active and transmitting, and 1/100 the energy when sleeping. - Is super fast at handshaking - actual connection times are of only a few mS and thereby the connection is quickly initiated as the device wakes up.. - it still has some of the same Robust Characteristics as Classic BT - SUCH as : - Adaptive Frequency Hopping (AFH) - parts of the Logical Link Control and Adaptation Protocol (L2CAP) interface - NO SUPPORT for Serial Port Profile (SPP) in the standard v4.0 Spec, also no support for other profiles that are support in 'Classic BT' such as - headset (HSP), object exchange (OBEX), audio distribution (A2DP), video distribution (VDP) and file transfer (FTP). - Bluetooth low energy technology is based on a master ( CENTRAL ) connected to a number of slaves (PERIPHERALS) . However, in Bluetooth low energy technology the number of slaves can be very large; how large depends on the implementation and available memory. - Advertising: The new "advertising" functionality of Bluetooth low energy technology makes it possible for a slave to announce that it has something to transmit to other devices that are "scanning." "Advertising" messages can also include an event or a measurement value. such as RSSI - Parameters have a state that is accessed using the Attribute Protocol. Attributes are represented as characteristics that describe signal value, presentation format, client configuration, etc. - Shorter Range, by about a 1/4 ( ~ 200 m ) - Supported on nearly all ( forthcoming mobile devices ) and a firm footprint in current devics - with iPhone 4S ( yes thats right, the 4S is a 3year old device now ) - so chances are you have a device in your pocket that supports the "new" BLE protocol. --- APPLE APPLE --- http://appleinsider.com/articles/13/06/19/inside-ios-7-ibeacons-enhance-apps-location-awareness-via-bluetooth-le - APPLE SUPPORT JUST GOT BETTER WITH iOS7 - Apple's new iBeacons feature for iOS 7 implements a Bluetooth Low Energy (BLE) profile for micro-location, enabling a series of new geofencing capabilities for app developers. <http://appleinsider.com/articles/13/06/19/inside-ios-7-ibeacons-enhance-apps-location-awareness-via-bluetooth-le > <http://photos.appleinsider.com/wwdc2013.ibeacons.png> Micro-location door activationEssentially, rather than using satellite signals to locate a device anywhere on Earth as GPS does, BLE can enable a mobile user to navigate and interact with specific regions geofenced by low cost signal emitters that can be placed anywhere, including indoors, and even on moving targets. Additionally, it appears iOS devices can also act as an iBeacon:Acting as iBeacon, a user with an iOS 7 device in hand could trigger events around them, allowing them to, for example, turn on lights and unlock and open doors simply by signaling the user's proximity to devices listening for it via BLE. specific feature set for iBeacons still under NDA - so I can't talk about it :( --- APPLE APPLE --- - Use your imagination - the BLE specification also supports the concept of device leashing, which could, for example, enable a peripheral like a watchband to communicate its location to a configured smartphone. The specification also supports peripheral push notifications, which Apple supports in new APIs in both OS X Mavericks and iOS 7. -- BLE BASICS BLE devices run in 2 modes ( this includes your mobile phone ) Central Mode & Peripheral Mode - Central Mode, means the device ( phone in this case ) … - These are hubs' who listen for Peripherals and engage with them. - so in this case thats our iPhone or iPad - Peripheral Mode, usually means the embedded device ( often in this is an embedded system or beacon : it can be a smoke detector, SMART watches, heart rate monitors, or just a simple Beacon - like what we are going to be DEMO-ING today. - These guys are .. and in our case we are just using it as a simple BLE - Beacon. - In this case the TI CC2540 ( or Emmaco Board ) acts as a peripheral beacon. Peripherals - Advertise themselves to CENTRALS -> with Advertise data, UUIDs' and RSSI Signals - and make themselves available to be connected to ( maybe as in the case of our Dev kit to tell the peripheral about something, like sensor data - in this chip we have ( Temp, MAGNA -amometera, ACCELERA- meter , humidity, ) - these devices come in a wide range of real world touch points. YOU can see the world turning into, a BLE connected world where your mobile device talk's to and is informed by these little BLE SENSORS and BEACONS in the stores you walk into, the homes you live in, and the offices you WORK at. So this means more data. LOTS of It ! Were just going to touch on the analytic data. that is tracking engagement with these devices. AND for that we need a powerful FAST data Store to persist this Exponential GRowth of information that is going to be happening between our Mobile Devices and these BLUE tooth Embedded Peripherals. AND FOR THAT… OF COURSE WE ARE GOING TO USE MONGO - DB & NODE JS.-- - So its also pretty cool that we can have our mobile device ( iPhone ) also run in peripheral mode. and advertise to other mobile devises that are running in Central Mode. - for this we put custom data into advertisement packet specifically the ID on "Local name" field - advertisement data.--- Our iPhone App example runs in both modes Central & Peripheral so lets take a quick look at the mobile app code First off, for this demo I used a mobile X-Platform Technology called Xamarin.Xamarin is the commercial offering and evolution of the Mono Project. - it allows you to write cross platform Mobile Apps in C#, its pretty cool, you should check it out if your looking for "REACH" in your mobile or desktop initiatives.Lets take A quick Walkthrough of our Xamarin App. Our Xamarin App is really about 3 things, - listening for BLE Device. - Broadcasting as a BLE Device. - Report back to the Analytics Collector engagements and interactions between the two Lets take a quick look at some of the code that is doing this for us in the demo app! First a quick look at the BLE Managers. /Users/mattschmulen/trees/strongloop/demoApp-ble-xamarin/BLEFlyTrap/BLECore//Central Mode _Centralmanager = new CBCentralManager(this, DispatchQueue.MainQueue);Terminal.WriteLine( "_Centralmanager Constructed"); //Peripheral Mode _PeripheralManagerDelegate = new BTPeripheralManagerDelegate(); _PeripheralManager = new CBPeripheralManager( _PeripheralManagerDelegate, DispatchQueue.MainQueue );Terminal.WriteLine( "_PeripheralManager Constructed"); _BLEPeripheralList = new List<BLEPeripheral>(); //Just a little code overview 2 main classes:BTManager : CBCentralManagerDelegateBTPeripheralManagerDelegate : CBPeripheralManagerDelegateBLEPeripheral-- Node JS and Middle Tier Technologies. Node Stack: - Square Analytics Node JS Package // take a look at it //-- Data Store-- Deploy --- Mongo$Mongod$Mongodb >show dbs -- show the datasets >use euro2012> db.teams.save({country:"France",GroupName:"D"})> db.teams.save({country:"Sweden",GroupName:"D"})> db.teams.save({country:"Ukraine",GroupName:"D"})> db.teams.find() . . .Here’s another difference of MongoDB from traditional relational databases: there are no tables, rows or columns. In their place, MongoDB uses collections and objects. Think of collections as tables, and objects as table rows. The main difference is that we’re not stuck into a rigid column structure.A database holds a set of collectionsA collection holds a set of documentsA document is a set of fieldsA field is a key-value pairA key is a name (string)A value is abasic type like string, integer, float, timestamp, binary, etc.,a document, oran array of values--- NODE SERVERNow that we have our mobile app lets create our StrongLoop Node.js Mobile APIslnode create web restapp -m -r slnode run app.js + routes/index.jsvardataStoreNodeApps = require('./nodeapps'); //module.exports = function(app, options) {app.get('/json/nodeapps/:id', dataStoreNodeApps.findById);app.get('/json/nodeapps', dataStoreNodeApps.findAll); + routes/nodeapps.js--- Thanks And make sure and checkout the Strong Loop Node Distro at StrongLoop.com And Appcelerators Titanium at Appcelerator.com---------------------------------------------------------------------------http://square.github.io/https://github.com/square/cubehttps://github.com/square/cube/wiki/Scalinghttps://github.com/partyio/heroku-cube//CHECK IT OUT AThttp://matt-evaluator-app.herokuapp.com/?step=1e4&//Time Series Visualizationhttp://square.github.io/cubism///http://square.github.io/cube///An event in Cube is simply a JSON object with a type, time, and arbitrary data. For example, to record an HTTP request sent to a web server, you might emit:Events can also include an id attribute, allowing you to update the data associated with an earlier event.{ "type": "request", "time": "2013-06-20T00:54:30.660Z", "data": { "path": "/cube/", "duration": 383, "status": 200, "browser": { "os": "Mac", "name": "Chrome", "version": 27 } }}//https://github.com/square/cube/wiki/Events//https://github.com/square/cube/wiki/Evaluator#wiki-event_get//For example, to get the time and IP address of the most recent "cube_request" event, say:http://matt-evaluator-app.herokuapp.com/1.0/event?expression=cube_request(ip)&limit=1//For example, to get the number of "cube_request" events in the last minute, say:http://matt-evaluator-app.herokuapp.com/1.0/metric?expression=sum(cube_request)&step=6e4&limit=1//To see all the requests to this endpoint on April 16, 2012, say:http://localhost:1081/1.0/event?expression=cube_request.eq(path,'/1.0/event')&start=2012-04-16&stop=2012-04-17//To see all the requests on April 16, 2012 between 4:00 PM and 5:00 PM UTC, say:http://matt-evaluator-app.herokuapp.com/1.0/metric?expression=sum(cube_request)&start=2012-04-16T16:00Z&stop=2012-04-16T17:00Z&step=6e4//Cube supports five metric resolutions:1e4 - 10-second6e4 - 1-minute3e5 - 5-minute36e5 - 1-hour864e5 - 1-day/Users/mattschmulen/scratch/curlcurl -X POST -d @minevent.json http://matt-collector-app.herokuapp.com:1080/1.0/event/put---------------------------------------------------------------------------git clone git://github.com/mongolab/hello-mongoose.git && cd hello-mongooseheroku createherokuaddons:addmongolabgit push heroku masterheroku open---------------------------------------------------------------------------++ configure the database - Already have mongo DB installed $ mongo > show dbs$mongostatnetstat -na | grep 27017---------------------------------------------------------------------------https://dashboard.heroku.com/apps++ build slnode create web node-app -m -r // references http://theholmesoffice.com/mongoose-and-node-js-tutorial/ http://theholmesoffice.com/how-to-create-a-mongodb-database/https://github.com/mongolab/hello-mongoose/blob/master/README.md#deploymenthttp://backbonetutorials.com/nodejs-restify-mongodb-mongoose/http://addyosmani.github.io/backbone-fundamentals/---http://prinzhorn.github.io/skrollr/https://github.com/nodejitsu/require-analyzerhttp://apiary.io/https://github.com/strongloop/sl-remoting/tree/docs/example/documentationhttps://github.com/strongloop/sl-remoting/tree/docs/example/documentation