Artificial intelligence in industry

Editor's Notes

• #6: Increase the enrolment ratios in higher education: Part of the skills gap problem is that only a very small percentage of India’s youth go on to higher education. No more than 7% of Indian youth between the age group of 18 and 25 years go to college. Also, 40% of people over the age of 15 are illiterate. While the government and academia work towards increasing the enrolment ratios by providing various schemes, it is also your responsibility, as job aspirants to complete education effectively. If we increase the ratio of enrolments to universities, a part of the problem could be resolved. Alignment of curriculum with industry requirements: Considering the evolving nature of industry and changes in the labor market, it has become imperative to ascertain how best the academic curriculum can address various requirements and demands of the dynamic industry. Despite exponential changes in almost every field, college curriculum is often rigid and hard to revise. It is advisable that curriculum is revised regularly and is developed in accordance with what industry needs. Academia can also look at existing pedagogies and see if the same can be improved and made more practical in approach. So far there has been immense reliance on the classroom methodology and a theory-heavy approach, however, the need is to make the entire approach more practical case studies; regularly calling guest speakers from different industries and letting students interact with them can prove to be extremely beneficial for everyone. Furthermore, there are a number of fantastic, industry relevant online courses available which can be merged with the syllabus to give students more insights about the industry. Emphasis on skill-based education: A recent NASSCOM report suggests that by 2020, 2 out of 3 millennials would like to take up a managerial job within 3 years of their joining and the core skill set required by the industry will change fast and newer skills like creative thinking, higher level problem solving, interpersonal skills, innovation, decision making will be in great demand. However, “Skill-based education is somewhat lacking in all the higher education fields in India and management is no different. The focus of management institutes need to shift from theoretical knowledge to skill based education with a more practical and dynamic approach,” says Roshni Chakrabarty noting recent trends in MBA education. Therefore, besides imparting the core or technical knowledge, academia should also try to focus on the softer and behavioural aspects such as interpersonal skills, leadership capability, attitude, communication skills (with increased emphasis on learning the English language) team spirit and alike which will play critical role in determining the success of individuals once they join the industry.  Internships that give workplace exposure: As students, you would benefit greatly if higher education faculty either had prior work experience or would be required to spend some time on short-term assignments with employers. But you would also hit the ground running with employers if part of their programme had internships with employers. Even if these internships are not in the industry or function where you find permanent jobs, these stints will give you an appreciation for the realities of the workplace. The lack of organised apprenticeships in India (we have only three lakh trainees compared to Germany’s six million and Japan’s 10 million) sabotages employability by undermining an effective vehicle of learning-by-doing and learning-while-earning. Internships combine education with training and enable the industry to take the candidates on a test drive. Up-skilling the faculty: Apart from only focusing on the curriculum structure, it is also imperative to provide the right exposure and training to the faculty. Most of the faculty do not possess requisite industry experience which comes in the way of imparting practical knowledge about industries. It will be great if the faculty can regularly undertake short industrial projects in collaboration with industry experts. This will help ensure that the faculty is in line with the current industrial trends.
• #8: Acting humanly: The Turing Test approach The Turing Test, proposed by Alan Turing(1950), was designed to provide a satisfactory operational definition of intelligence. Turing defined intelligent behavior as the ability to achieve human-level performance in all cognitive tasks, sufficient to fool an interrogator. Capabilities include – Natural language processing to enable it to communicate successfully in English (or some other human language) Knowledge representation to store information provided before or during the interrogation Automated reasoning to use the stored information to answer questions and to draw new conclusions; Machine learning to adapt to new circumstances and to detect and extrapolate patterns. Computer vision Robotics Think humanly (The cognitive modelling approach) Capabilities include – Cognitive Science - the study of thought, learning, and mental organization, which draws on aspects of psychology, linguistics, philosophy, and computer modelling Think rationally – Thinking with logic. Act rationally – Always doing the right thing.
• #9: Computer vision - is an interdisciplinary field that deals with how computers can be made to gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to automate tasks that the human visual system can do. Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions. Understanding in this context means the transformation of visual images (the input of the retina) into descriptions of the world that can interface with other thought processes and elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory. As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner. As a technological discipline, computer vision seeks to apply its theories and models for the construction of computer vision systems. Sub-domains of computer vision include scene reconstruction, event detection, video tracking, object recognition, 3D pose estimation, learning, indexing, motion estimation, and image restoration. Deep learning (also known as deep structured learning or hierarchical learning) is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms. Learning can be supervised, semi-supervised or unsupervised. Deep learning architectures such as deep neural networks, deep belief networks and recurrent neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design and board game programs, where they have produced results comparable to and in some cases superior to human experts. Deep learning models are vaguely inspired by information processing and communication patterns in biological nervous systems yet have various differences from the structural and functional properties of biological brains (especially human brain), which make them incompatible with neuroscience evidences. Robotic process automation (RPA) is the application of technology that allows employees in a company to configure computer software or a “robot” to capture and interpret existing applications for processing a transaction, manipulating data, triggering responses and communicating with other digital systems. Big data is a term used to refer to the study and applications of data sets that are too complex for traditional data-processing application software to adequately deal with. Big data challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating, information privacy and data source. Cloud computing is shared pools of configurable computer system resources and higher-level services that can be rapidly provisioned with minimal management effort, often over the Internet. Cloud computing relies on sharing of resources to achieve coherence and economies of scale, similar to a public utility. Third-party clouds enable organizations to focus on their core businesses instead of expending resources on computer infrastructure and maintenance.[1] Advocates note that cloud computing allows companies to avoid or minimize up-front IT infrastructure costs. Proponents also claim that cloud computing allows enterprises to get their applications up and running faster, with improved manageability and less maintenance, and that it enables IT teams to more rapidly adjust resources to meet fluctuating and unpredictable demand.[1][2][3] Cloud providers typically use a "pay-as-you-go" model, which can lead to unexpected operating expenses if administrators are not familiarized with cloud-pricing models.
• #13: Demand Insight – SocialIdentifies important data across forces News News helps identify the intensity of local incidents, events and weather Events Important local, national and promotional events that influence customer behavior Weather Weather events – actual and forecasted - that influence behavior Competitors Competitor’s actions and their effects on consumer’s change in behavior Customer Factor behaviors, shopping habits, price sensitivities, lifestyles Supplier Supplier’s actions (new product, late and incomplete shipments) can cause disruptions Economy Economic conditions at national, state and local levels Trends New and upcoming trends, regulations, law changes impact demand Machine Learning for Retail Stocking and Inventory Optimizing inventory is by no means a problem exclusive to retail – but it is an inescapable consideration for businesses predicated on holding physical items to be sold in-person to customers (not that eCommerce doesn’t have it’s hassles, but optimizing per location for a retail chain is often a logistics concern that eCommerce players don’t have to deal with). What may be possible in the near-term: Retailers will be able to use purchase data (real time and historical, online and offline) to predict inventory needs in real time (based on the season, the day of the week, activity at other area stores, eCommerce activity from customers in that geo area, etc…) This information may prompt a daily dashboard of suggested orders to a purchasing manager, or some AI systems may be approved to make small orders automatically without requiring human approval Using machine vision for cameras (either installed throughout the building, or carried by employees), inventory systems of the future will be able to generate accurate, real-time estimates of all products in a given store. Such a system could notify store managers of unusual patterns of inventory data, such as suspected theft, or an unusual uptick in sales of a particular product that may warrant a faster re-order than might have been previously planned
• #14: Personalized Product Recommendations (with help of chat service) - Through information provided by consumers about a gift recipient, the software tailors gift recommendations by comparing specifics provided to gifts purchased for similar recipients. The GWYN experience attempts to replicate the role of a concierge at a store through a personal and detailed conversation with users. Published 2015 pilot results, based on data collected from 55,000 users, resulted in a 60% click-through rate and 75% total sales conversions. It’s important to note that we weren’t able to tell if these results represented more or less than North Face’s normal results, and whether these results are sustainable or are merely driven by initial novelty in the user interface. The advantage of using machine learning in such a recommendation Q-and-A interface, is that North Face can potentially run tens of thousands of consumers through this conversational engine. At a certain volume of customer interactions, the system might be expected to glean important insights and patterns on suggestions that “work” (high take-rate to purchase, or high cart value), and those that don’t – allowing the company potentially gain higher and higher conversions over time.
• #28: The role of: data scientist One critical job on an analytics team is to figure out how to analyze the data. This is the person who designs and tests new models, algorithms, or presentations of the data. Needs to understand what’s being studied (i.e. the business process) and be able to work in an iterative, creative manner. Some may be experts in statistics, some in visualization, some in machine learning or other tools. The role of: data wrangler Data preparation (also called: data hygiene, data munging, data wrangling, or ETL) turns out to be a major part of the work in analytics (more than 50%, cf. Lohr, 2014). Many companies exist just to prepare and sell structured datasets to others. Skills: scripting languages (Perl, Python, R), regular expressions, Hadoop/MapReduce, SQL, traditional ETL tools, API hacking, “problem solving with programming” The role of: data communicator Several interviewees discussed the need to have people on their teams who specialize in data visualization or in communicating analytical findings to business users. These professionals could come from the business side or from the analytics team; what is needed is to bridge the gap between analysis and the decision it informs. Needs to understand: how analysis translates into cause-and-effect relationships that managers can use (and which analytics do not); visual and written communication The role of: data engineer Some of the biggest potential gains are in turning analytical models into data products, that is, automating data analysis and making it self-service. These are specialists who understand data streams, databases, cloud platforms, and all the “infrastructure” that makes data apps possible. They build analytical systems (like traditional DW/BI) but also help embed analytical functionality within operational systems.
• #43: Scatterplot - A scatterplot is a useful summary of a set of bivariate data (two variables), usually drawn before working out a linear correlation coefficient or fitting a regression line. It gives a good visual picture of the relationship between the two variables, and aids the interpretation of the correlation coefficient or regression model.   A scatterplot is often employed to identify potential associations between two variables, where one may be considered to be an explanatory variable (such as years of education) and another may be considered a response variable (such as annual income). A positive association between education and income would be indicated on a scatterplot by a upward trend (positive slope), where higher incomes correspond to higher education levels and lower incomes correspond to fewer years of education. A negative association would be indicated by the opposite effect (negative slope), where the most highly educated individuals would have lower incomes than the least educated individuals. Or, there might not be any notable association, in which case a scatterplot would not indicate any trends whatsoever. The following plots demonstrate the appearance of positively associated, negatively associated, and non-associated variables: Correlation Coefficient – The strength of the linear association between two variables is quantified by the correlation coefficient. Given a set of observations (x1, y1), (x2,y2),...(xn,yn), the formula for computing the correlation coefficient is given by  The correlation coefficient always takes a value between -1 and 1, with 1 or -1 indicating perfect correlation (all points would lie along a straight line in this case). A positive correlation indicates a positive association between the variables (increasing values in one variable correspond to increasing values in the other variable), while a negative correlation indicates a negative association between the variables (increasing values is one variable correspond to decreasing values in the other variable). A correlation value close to 0 indicates no association between the variables. Since the formula for calculating the correlation coefficient standardizes the variables, changes in scale or units of measurement will not affect its value. For this reason, the correlation coefficient is often more useful than a graphical depiction in determining the strength of the association between two variables.
• #45: Let's try out the notation on our example with the trend summarized by the line w = -266.53 + 6.1376 h. (Note that this line is just a more precise version of the above solid line, w = -266.5 + 6.1 h.) The first data point in the list indicates that student 1 is 63 inches tall and weighs 127 pounds. That is, x1 = 63 and y1 = 127 . Do you see this point on the plot? If we know this student's height but not his or her weight, we could use the equation of the line to predict his or her weight. We'd predict the student's weight to be -266.53 + 6.1376(63) or 120.1 pounds. That is,  ^y1 = 120.1. Clearly, our prediction wouldn't be perfectly correct — it has some "prediction error" (or "residual error"). In fact, the size of its prediction error is 127-120.1 or 6.9 pounds. As you can see, the size of the prediction error depends on the data point. If we didn't know the weight of student 5, the equation of the line would predict his or her weight to be -266.53 + 6.1376(69) or 157 pounds. The size of the prediction error here is 162-157, or 5 pounds. In general, when we use ^yi=b0+b1xi to predict the actual response yi, we make a prediction error (or residual error) of size: ei=yi−^yi
• #46: Based on the least squares criterion, which equation best summarizes the data? The sum of the squared prediction errors is 766.5 for the dashed line, while it is only 597.4 for the solid line. Therefore, of the two lines, the solid line, w = -266.53 + 6.1376h, best summarizes the data. But, is this equation guaranteed to be the best fitting line of all of the possible lines we didn't even consider? Of course not! If we used the above approach for finding the equation of the line that minimizes the sum of the squared prediction errors, we'd have our work cut out for us. We'd have to implement the above procedure for an infinite number of possible lines — clearly, an impossible task! Fortunately, somebody has done some dirty work for us by figuring out formulas for the intercept b0 and the slope b1 for the equation of the line that minimizes the sum of the squared prediction errors.
• #50: Let's start our investigation of the coefficient of determination, r2, by looking at two different examples — one example in which the relationship between the response y and the predictor x is very weak and a second example in which the relationship between the response y and the predictor x is fairly strong. If our measure is going to work well, it should be able to distinguish between these two very different situations. Here's a plot illustrating a very weak relationship between y and x. There are two lines on the plot, a horizontal line placed at the average response, ¯y , and a shallow-sloped estimated regression line, ^y. Note that the slope of the estimated regression line is not very steep, suggesting that as the predictor x increases, there is not much of a change in the average response y. Also, note that the data points do not "hug" the estimated regression line: Top left figure in the slide The calculations on the right of the plot show contrasting "sums of squares" values: SSR is the "regression sum of squares" and quantifies how far the estimated sloped regression line, ^yi , is from the horizontal "no relationship line," the sample mean or ¯y. SSE is the "error sum of squares" and quantifies how much the data points, yi, vary around the estimated regression line, ^yi. SSTO is the "total sum of squares" and quantifies how much the data points, yi, vary around their mean, ¯y. Note that SSTO = SSR + SSE. The sums of squares appear to tell the story pretty well. They tell us that most of the variation in the response y (SSTO = 1827.6) is just due to random variation (SSE = 1708.5), not due to the regression of y on x (SSR = 119.1). You might notice that SSR divided by SSTO is 119.1/1827.6 or 0.065. Do you see where this quantity appears on Minitab's fitted line plot? Contrast the above example with the following one in which the plot illustrates a fairly convincing relationship between y and x. The slope of the estimated regression line is much steeper, suggesting that as the predictor x increases, there is a fairly substantial change (decrease) in the response y. And, here, the data points do "hug" the estimated regression line: Top right figure in the slide The sums of squares for this data set tell a very different story, namely that most of the variation in the response y (SSTO = 8487.8) is due to the regression of y on x (SSR = 6679.3) not just due to random error (SSE = 1708.5). And, SSR divided by SSTO is 6679.3/8487.8 or 0.799, which again appears on Minitab's fitted line plot. The previous two examples have suggested how we should define the measure formally. In short, the "coefficient of determination" or "r-squared value," denoted r2, is the regression sum of squares divided by the total sum of squares. Alternatively, as demonstrated in this, since SSTO = SSR + SSE, the quantity r2 also equals one minus the ratio of the error sum of squares to the total sum of squares:  r2=SSR/SSTO=1−SSE/SSTO Here are some basic characteristics of the measure: Since r2 is a proportion, it is always a number between 0 and 1. If r2 = 1, all of the data points fall perfectly on the regression line. The predictor x accounts for all of the variation in y! If r2 = 0, the estimated regression line is perfectly horizontal. The predictor x accounts for none of the variation in y! We've learned the interpretation for the two easy cases — when r2 = 0 or r2 = 1 — but, how do we interpret r2 when it is some number between 0 and 1, like 0.23 or 0.57, say? Here are two similar, yet slightly different, ways in which the coefficient of determination r2 can be interpreted. We say either: "r2 ×100 percent of the variation in y is reduced by taking into account predictor x" or: "r2 ×100 percent of the variation in y is 'explained by' the variation in predictor x.“ Students often ask: "what's considered a large r-squared value?" It depends on the research area. Social scientists who are often trying to learn something about the huge variation in human behavior will tend to find it very hard to get r-squared values much above, say 25% or 30%. Engineers, on the other hand, who tend to study more exact systems would likely find an r-squared value of just 30% merely unacceptable. The moral of the story is to read the literature to learn what typical r-squared values are for your research area!
• #51: So, what does the correlation of -0.825 between skin cancer mortality and latitude tell us? It tells us: The relationship is negative. As the latitude increases, the skin cancer mortality rate decreases (linearly). The relationship is quite strong (since the value is pretty close to -1)