Awsome Strategic Intelligence - BigData

It is astonishing how much data is available now from both inside and outside an enterprise. The data is of no use unless it can be accessed, analyzed and predictive analytics are applied. The amount of data that is accessed and analyzed on a daily basis within our enterprise is over 6 pedabytes. The amount of date is projected to significantly increase indefinitely. Since this data is very beneficial in strategic and tactical planning, decision, operational intelligence and security, the need to analyze larger amounts of data is essential. But there are obstacles that need to be considered. It is unlikely that no enterprise has the resources to keep up with this data trend. This is why a cloud model makes sense. For example, the AWS cloud can be used to store and compute on a massive scale that most enterprises can not afford. The cloud also is the best place to put data so that it can be analyzed from multiple locations.

Global data from the Internet of Things (IoT) will become so massive that it will not be analyzed for business insight unless the resources for big data can be massively scaled. Also there are multiple data sources from governments, businesses and associations that could be used to gain more insight to finance, marketing, macro economics, security and more.

The use of machine learning (Artifical Intelligence) and multivariate analysis for predictive purposes are essential for gaining intelligence from this data. But so are business tools that can easily connect to these various data sources and answer questions that business people need. For example, before introducting a new product, it would be very useful to understand the market demand and the customers most likely to purchase it. This can be achieved rapidly with the use of some business intelligence tools. The best business tool that I have become aware is Qlik. It is both easy to use by anyone and extremely powerful. There is no need to build expensive data wareshouses anymore, since a tool like Qlik enables you to connect easily to both structured and big data to answer business questions that will enable the enterprise to increase sales and profitability.

I would expect that business management will understand that all this structured and big data is essential for increasing sales revenue and profitability. Much of this data is available and the tools needed to analyze it are also available. To make use of it will require massive scaling for storage and compute which only a cloud service like AWS can provide in a cost effective way. The major limitation that most businesses will encounter is finding data scientists that are essential for analyzing this data. I have over 30 years experience in data science and now function as the Chief Data Scientist for the enterprise. The lack of qualified data scientists is the greatest limitation that our enterprise has encountered.