From the course: Advanced Data Processing: Batch, Real-Time, and Cloud Architectures for AI
Join today to access over 24,500 courses taught by industry experts.
Batch AI architecture example: Solution
From the course: Advanced Data Processing: Batch, Real-Time, and Cloud Architectures for AI
Batch AI architecture example: Solution
- [Instructor] Let's now implement the customer segmentation architecture with Batch AI. We begin with feature engineering. We have three sources of data, namely the Salesforce Marketing Cloud, the enterprise sales database, and the contact center history files. We add an Apache Spark job to fetch marketing responses data periodically using REST APIs from the Salesforce Marketing Cloud. We add another Apache Spark job to read the sales data from the enterprise sales database using SQL. We also have an Apache Spark job that reads contact center history files using FTP. All the three jobs will write the acquired data into a data lake. We could build a data lake using MongoDB as it provides a flexible document format to dump any kind of data. Then we have an Apache Spark job to merge the three data sets. It will first summarize the data by each customer and then merge them across the data sets using the customer ID as the key. The merge data is then stored into a temporary database…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.