SlideShare a Scribd company logo

A.Levenchuk -- Machine learning engineering

Download as PPTX, PDF
Anatoly Levenchuk
Anatoly Levenchuk

Machine learning engineering involves applying scientific and engineering principles to design, build, and improve machine learning systems. It is a competency that draws from fields like software engineering, control systems engineering, and data science. Effective machine learning engineering requires understanding complexity and how to manage technical debt in machine learning systems.

Technology
1 of 44
Downloaded 217 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
Machine Learning Engineering Anatoly Levenchuk Copyright Β© 2016 by Anatoly Levenchuk. Permission granted to DeepHack and INCOSE to publish and use.
What is machine learning as a human activity? β€’ Ontological question (Aristotle definition: via class-subclass specialization) β€’ Why it is important? – How to pay? [grants, investmetns, charity] – How to teach? [science, engineering, arts/crafts] – How to name and distinguish in communication (hiring – participation in division of labor)? 2 How you name yourself to colleagues, when hacking machine learning system?
Machine learning is a… β€’ Science! MSc. in Machine Learning [BigData] β€’ Research? β€’ Engineering? β€’ Art? Programming is a… β€’ Science? Computer science, MSc. β€’ Research? Computer science, MSc. β€’ Engineering? Software engr., MSc. and MSE! β€’ Art? Master of Art in Mathematics! 3 http://www.computer.org/web/education/professional-competency-certifications https://www.kaggle.com/competitions
Test: Why is my program not working? Why is my program not working? You need to know why? To repair compiler? Software engineer (systems) To advance theory? Computer Scientist You need to program working properly? Software engineer (application) 4
Science Resulting in models, descriptions (theories), ontologies: β€’ M0 – manufacturing (not science!). Programmers are engineers: software is physical system! β€’ M1 – design/applied research (Edison) β€’ M2 – basic research (Einstein) β€’ M3 – philosophical logic/mathematics β€’ There are multiple meta-levels. β€’ Scientists produce these meta-descriptions 5
Engineering β€’ Engineering – discipline, art, skill and profession of acquiring and applying scientific, mathematical, economic, social, and practical knowledge, in order to design and build structures, machines, devices, systems, materials and processes that safely realize improvements to the lives of people. β€’ Engineering is the application of mathematics, empirical evidence and scientific, economic, social, and practical knowledge in order to invent, innovate, design, build, maintain, research, and improve structures, machines, tools, systems, components, materials, and processes. 6 https://en.wikipedia.org/wiki/Engineering https://en.wikipedia.org/wiki/Outline_of_engineering
Data scientists – ML Engineers 7 Model/Theory [metamodel] Engineering/Applied Research Reality/Data/Model Science/Basic Research If it is not about budgeting and social status, it need not to distinguish science and engineering! Practice both of them!
Engineering for science 8 http://blogs.nvidia.com/blog/2016/01/12/accelerating-ai-artificial-intelligence-gpus/ Scientists are mere owner-operators of instruments. Who built the Big Hadron Collider? Experiments order by scientists, builds and carried by engineers, interprets by scientists.
The sunset of the professions, not jobs! 9 β€’ Life-long β€’ Special education β€’ No other professions in a mix β€’ Several years long β€’ Additional training β€’ One competence in the mix Machine learning engineering is not a profession. It is a competency!
Machine learning (systems) engineering β€’ Control (systems) engineering β€’ Machine Learning (systems) engineering 10 ? http://www.payscale.com/research/US/Job=Controls_Engineer/Salary β€’ Systems Engineer (IT) β€’ Cognitive/Machine Intelligence Systems Engineer ?
What about jobs? 11 Algorithms + Data Structures = Programs (Niklaus Wirth) Scientist is not an engineer, data is not a system
Kind of Engineerings β€’ Mechanical engineering β€’ Agriculture engineering β€’ Aerospace engineering – aircraft architecture β€’ Systems engineering β€’ System of systems engineering β€’ … β€’ Software engineering β€’ Control [systems] engineering – control [system] architecture β€’ Knowledge engineering -- architecture β€’ Machine learning [system] engineering β€’ … β€’ Neural engineering β€’ neural network engineering -- neural [network] architecture β€’ Feature engineering -- ??? 12
Systems, Software, Machine Learning Engineerings β€’ Systems engineering [Bell Labs in 1940s, boosted as a profession by NCOSE 1990] β€’ Software engineering [term appeared in 1965, boosted by NATO as a profession in 1968] β€’ Machine learning engineering [term appeared in 2011] 13https://www.google.com/trends/explore#q=machine%20learning%20engineering&cmpt=q&tz=Etc%2FGMT-3
Conversion of engineerings and Disruption of engineerings 14 Software Engineering Machine Learning Engineering Janosh Szepanovits. Convergence: Model- Based Software, Systems And Control Engineering + http://www.infoq.com/presentations/Model-Based-Design-Janos-Sztipanovits Le Bottou – Β«Machine Learning disrupts software engineeringΒ» http://leon.bottou.org/slides/2challenges/2challenges.pdf We can add: β€’ Machine learning disrupts systems engineering β€’ Machine learning disrupts control engineering β€’ … β€’ Machine learning disrupts contemporary engineering
Can we use systems and software engineering wisdom in MLE? Le Bottou http://leon.bottou.org/slides/2challenges/2challenges.pdf β€’ Models as modules: problematic due to weak contracts (models behave differently on different input data) β€’ Learning algorithms as modules: problematic due to output depends on the training data which itself depends on every other module Engineering is not only about modularity and modular synthesis! What about other aspects?! β€’ More attention to left part of V-diagram β€’ Optimizations later β€’ … β€’ What else? 15
Technical Debt 16 Machine Learning: The High-Interest Credit Card of Technical Debt http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43146.pdf Hidden Technical Debt in Machine Learning Systems http://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf β€’ Hack now, pay later (with interest, of course!). β€’ Based on heuristics from software engineering (same approach as our: usage of software and systems engineering wisdom in machine learning engineering). β€’ Set of domain-specific heuristics for machine learning
Bionics and machine learning systems engineering β€’ In short: brain is only an inspiration, not a model for reproducing! β€’ There are other β€œlearning systems engineerings”: e.g. neural engineering (https://en.wikipedia.org/wiki/Neural_engineering). β€’ AGI (artificial general intelligence) is a far goal, but magnet for freaks of all sorts. Better not mention it. β€’ Biologically plausible machine learning is about science, not engineering. 17
Knowledge engineering β€’ Ontology engineering (manually) β€’ Solutions are (manually) programmed. β€’ Example: robot-Β«butterflyΒ», https://youtu.be/kyvW5sOcZHU, https://youtu.be/V30e77x8BQA – Every type of movement should be programmed anew – Non-adaptable to changes of environment and device – The best science available up today! – Perfect, if CPS perform only one or two movements. Not for robots, definitely! β€’ No learning! 18
Tribes Shallow Learning Big Data Deep Learning Neuro evolution Bayes Army Symbolic 19
Our definition of complexity Complex system – the one that does not fit in the sole engineer’s head, thus collaboration of a team and automation of a knowledge work are mandatory. E.g.: β€’ Aircraft β€’ programming-in-the small vs.programming in the large β€’ VLSI – very large scale integration, more than 1000 transistors on a single chip (now transistor count is more than 20bln. – FPGA Virtex-Ultrascale XCVU440) β€’ Artificial neural network – 16bln. parameters. 20
Comlexity β€’ Systems Engineering β€’ Machine learning 21 Complex system: not fit into one (hundred) heads for its development. Stellarators, Tokamaks, BHC, aerospace and VLSI engineering. IBM Watson (up to 2011): team of 40. Still not very complex from engineering point of view. http://josephpcohen.com/w/visualizing-cnn-architectures-side-by-side-with-mxnet/http://787updates.newairplane.com/787-Suppliers/World-Class-Supplier-Quality
CNN Architecture/complexity Growth 22 1998 2012 9/2014 2/2015 12/2015 9/2014 http://josephpcohen.com/w/visualizing-cnn-architectures-side-by-side-with-mxnet/ LeNet 28*28 LeNet 28*28 VGG 224x224 GoogLeNet 224x224 Inception V3 299x299 Inception BN 224x224
AutoML β€’ Generative design/architecturing of networks β€’ Bayesian convergence β€’ Neuroevolution β€’ Dynamic neural description languages (e.g. Chainer) 23 Automatization of machine learning, CAMLE (computer-aided machine learning engineering) is the main trend of today and tomorrow!
Master Algorithm Pedro Domingos [module/construction] β€’ Symbolic β€’ Evolution β€’ Connectivist β€’ Bayesian β€’ Analogy [No free lunch!] Sarath Chandar [component/function] β€’ multi-task learning β€’ transfer learning β€’ zero-shot/one-shot learning β€’ multi-modal learning β€’ reinforcement learning 24 http://apsarath.github.io/2016/01/19/agi/ http://www.amazon.com/dp/0465065708/
Intellect-stack is only about one aspect of a whole intellect system. Intellect-stack is about Platforms (modules) = Β«how to make itΒ» Based on Fig.3 ISO 81346-1 -Modules =Components +Allocations 25 Modules and interfaces: platforms/layers Stack
Platform β€’ This is module viewpoint (Β«how to makeΒ») β€’ Platform is a technology stack layer β€’ Cohesive set of modules with published API β€’ Can be based on top of other platform 26
Intelligence Platform Stack and machine learning engineering in it 27 Application (domain) Platform Cognitive Architecture Platform Learning Algorithm Platform Computational library General Computer Language CPU GPU/FPGA/Physical computation Drivers GPU/FPGU/Physical computation Accelerator Neurocompiler Neuromorphic driver Neuromorphic chip Disruptionenablers Disruptiondemand Thanks for computer gamers for their disruption demand to give us disruption enabler such as GPU!
Alternative deep learning stack (as viewed by GPU hardware people) 28http://www.nextplatform.com/2015/12/07/gpu-platforms-emerge-for-longer-deep-learning-reach/ β€’ No cognitive and application levels β€’ Languages unimportant β€’ Chassis, backplane, blades importans (separate layer) β€’ No neuromorphic processing
Hardware Acceleration (except GPU) Is this machine learning engineering? No! But… β€’ Algorithm-dependent β€’ Need compilation (drivers) β€’ Speed rules β€’ Power rules β€’ Scale rules β€’ GPU β€’ FPGA β€’ ASIC β€’ Neuromorphic chips β€’ Physical computing 29 http://lighton.io/ β€’ Approximating kernels at the speed of light http://arxiv.org/abs/1510.06664 Analog, optical device, that performs the random projections literally at the speed of light without having to store any matrix in memory. This is achieved using the physical properties of multiple coherent scattering of coherent light in random media. β€’ Towards Trainable Media: Using Waves for Neural Network-style Learning β€’ Bitwise Neural Networks http://arxiv.org/abs/1601.06071 β€’ Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware http://arxiv.org/abs/1601.04187 http://arxiv.org/abs/1510.03776
General Computer Language Computer science + Software engineering β€’ Important! Separate layer in intellect-stack! β€’ 2 language problem β€’ experiment and production, like deep learning frameworks (speed) β€’ Β«WrappersΒ» in libraries (thresholds in understanding of a full stack up to hardware bottom) β€’ My preference: Julia (http://julialang.org/) β€’ Scientific computing is design goal of Julia, MATLAB-similar syntax β€’ 2 language problem solved (speed of computation as in C, speed of writing code as in Python) β€’ Extensive mathematical function library, Base library and external packages in native Julia β€’ Parallel computing supported (GPU supported too) β€’ Not object-oriented, using multiple dispatch as expression problem solution (good modularity) β€’ Version 0.4.3 now (1.0 expecting in one year) β€’ Caution: slightly more complex than Python, should not be your first computer language… β€’ MXNet deep learning framework have Julia wrapper β€’ DSL for deep learning is not General Computer Language β€’ Probabilistic programming languages -- http://probabilistic-programming.org/wiki/Home β€’ DNN description languges, like in CNTK -- https://github.com/Microsoft/CNTK 30
Computation libraries/frameworks/platforms Not a machine learning engineering! β€’ Computation libraries οƒŸοƒ  Drivers+Hardware (GPU, clusters) β€’ Linear algebra, optimization, autodiff, symbolic computations, etc. β€’ Can be standalone platform, thus differ from machine learning libraries (general algorithms for multiple purposes: bioinformatics, physics, astronomy, engineering, machine learning etc.) β€’ Deep learning frameworks often includes such a library (Torch, Theano, …). β€’ Scikit (NumPy, SciPy, and matplotlib) β€’ Nd4j (n-dimentional arrays for Java) β€’ Julia packages β€’ … β€’ Non-opensource: Mathematica, Maple… 31 Machine learning is β€œyet another domain modules and DSL” for them!
Learning algorithm frameworks (not systems)! Machine learning engineering! β€’ Gentleman algorithm set (CNN, RNN,…) β€’ Updating with an arxiv.org papers rhythm! β€’ Network description language – DSL for machine learning engineering β€’ Experiments and production (scalable!) β€’ Extensibility (on base of general computing language and scientific computing library: on base of another layer platform in intellect-stack) β€’ Presented as The Machine Learning Platform (including all lower levels assembled and tuned) β€’ There are hundreds of its: no less then Β«web frameworksΒ» in early web 32 http://www.slideshare.net/yutakashino/ss-56291783 β€’ Google β€’ Facebook β€’ Microsoft β€’ Baidu β€’ IBM β€’ Samsung β€’ … + standard datasets for comparisons and benchmarking + other tribes platforms
Construction (type of modules) in machine learning β€’ Deep learning classics (DSL in deep learning frameworks) β€’ Probabilistic languages http://probabilistic-programming.org/, https://probmods.org/ β€’ Deep learning and Bayesian conversion -- )http://www.nextplatform.com/2015/09/24/deep-learning-and-a-new-bayesian-golden-age/, http://blog.shakirm.com/2015/10/bayesian-reasoning-and-deep-learning/, http://arxiv.org/abs/1512.05287 β€’ Differentiable languages and datatypes http://colah.github.io/posts/2015-09-NN- Types-FP/, http://www.blackboxworkshop.org/pdf/nips2015blackbox_zenna.pdf, http://arxiv.org/abs/1506.02516 β€’ … β€’ Blends and hybrids of many other learning architectures 33 Varieties in representations: in deep learning abstraction is architecturally layered, in other approaches it different!
Algorithm platform + Hardware platform = Algorithm platform (hardware is not visible for a platform user, but still matter!) 34 http://blogs.microsoft.com/next/2016/01/25/microsoft-releases-cntk-its-open-source-deep-learning-toolkit-on-github/
Cognitive systems/architectures Learning, communications, reasoning, planning β€’ Cognitive = knowledge processing. Knowledge is information that is useful in variety of situations. β€’ Cognitive architecture/system is a platform for multiple application systems. β€’ Ensembles of learning algorithms: it is close to cognitive systems engineering β€’ Cognitive systems engineering is a machine learning systems engineering plus something else β€’ Something else: e.g. knowledge engineering: manual coding (formalization) of knowledge. β€’ Machine learning systems engineering is not cognitive systems engineering, it is smaller! 35
Machine Learning and Cognitive Level β€’ Β«deep learning research is likely to continue its expansion from traditional pattern recognition jobs to full-scale AI tasks involving symbolic manipulation, memory, planning and reasoning. This will be important for reaching to full understanding of natural language and dialogue with humans (i.e., pass the Turing test). Similarly, we are seeing deep learning expanding into the territories of reinforcement learning, control and robotics and that is just the beginningΒ» -- Joshua Bengio https://www.quora.com/Where-is-deep-learning-research-headed 36 If we can learn to reason, plan, model, act – then machine learning engineering will be cognitive systems engineering! Machine intelligence vs. artificial intelligence
Example: MANIC A Minimal Architecture for General Cognition (http://arxiv.org/abs/1508.00019) β€’ Keywords: action, planning, observation, decisions, knowledge, … β€’ Is it keywords for learning systems engineering? 37
Application level of intellect-stack β€’ Killer application for learning systems is here! β€’ Domain specificity and data is here! β€’ End users and money are here! β€’ Systems engineering is here! 38 This chart is only about enterprise AI systems market. https://www.tractica.com/newsroom/press-releases/artificial-intelligence-for-enterprise-applications-to-reach-11-1-billion-in-market-value-by-2024/ If you have no application of interest, there will be no data, no money, no developments, no engineering. Most machine learning engineering is applied. Only small part is machine learning platform development.
Application level: systems engineering β€’ Strategizing and Conceptual design β€’ Requirements engineering β€’ System Architecture β€’ V&V β€’ Configuration management β€’ Machine learning engineers is one of multiple engineers that participate in a cyber- physical system project team. 39 Sensors Consoles http://www.nist.gov/el/nist-releases-draft-framework-cyber-physical-systems-developers.cfm Actuators Monitors
Life cycle stages dictionary 40 Machine learning Systems engineering Conception and requirements Conception and requirements Architecture and Design Architecture and Design Training Manufacturing Transfer learning, ensembling Integration Validation and verification Validation and verification Inference Operation
Stakeholders concerns Domain-specific concern: β€’ Expressivity β€’ Computational efficiency β€’ Trainability β€’ Good generalization (not overfitting) Traditional concerns β€’ Composability – layering, ensembling β€’ Compositiality – transfer learning β€’ Resilience 41
Intellect-stack and machine learning (systems) engineering β€’ Machine learning (systems) engineering cover now only small part of the whole intellect-stack but interact with all levels. β€’ No one Googbookdu can develop all levels in intellect- stack platforms (from hardware accelerators in the bottom up to application on the top) by itself. Maybe except IBM that can span from TrueNorth to IBM Watson applications ;-) β€’ Interfaces from supporting platforms will be stabilizing and… in constant update (like in software engineering APIs: change of everything once in 5 years). β€’ Technology disruption starts with low (enabling) levels of a stack, demand provides from upper level, thus nobody in the middle can ignore developments in other layer platforms. 42
Intellect-Stack 43 Application (domain) Platform Cognitive Architecture Platform Learning Algorithm Platform Computational library General Computer Language CPU GPU/FPGU/Physical computation Drivers GPU/FPGA/Physical computation Accelerator Neurocompiler Neuromorphic driver Neuromorphic chip Disruptionenablers Disruptiondemand Where are you now? Where are you tomorrow?
Thank you! Anatoly Levenchuk, TechInvestLab, president INCOSE Russian chapter, research director https://ru.linkedin.com/in/ailev ailev@asmp.msk.su Blog in Russian: http://ailev.ru 44

More Related Content

Viewers also liked (20)

PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π‘ΡƒΠ΄ΡƒΡ‰Π΅Π΅ проСктирования
Anatoly Levenchuk
Β 
PDF
М.Гайворонский -- ΠΎΠΏΡ‹Ρ‚ Ρ€Π°Π·Ρ€Π°Π±ΠΎΡ‚ΠΊΠΈ БАУ двигатСля
Anatoly Levenchuk
Β 
PDF
Ali Mousavi -- Event modeling
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- плохая ΠΌΠΎΠ΄ΡƒΠ»ΡŒΠ½ΠΎΡΡ‚ΡŒ
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Ρ‚Ρ€Π΅Π½Π΄Ρ‹ Π² ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ Ρ‚Ρ€Π΅Π±ΠΎΠ²Π°Π½ΠΈΠΉ
Anatoly Levenchuk
Β 
PPTX
А.Π•Ρ„Ρ€Π΅ΠΌΠΎΠ² -- встрСчи Русского отдСлСния INCOSE
Anatoly Levenchuk
Β 
PPTX
О.Π‘Π°Π²ΠΈΠ½ -- Modelica Π² Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π½ΠΎΠΌ ΠΌΠΎΠ΄Π΅Π»ΠΈΡ€ΠΎΠ²Π°Π½ΠΈΠΈ
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠžΠ±Ρ€Π°Π·ΠΎΠ²Π°Π½ΠΈΠ΅ ΠΏΠΎ ΠΌΠ°ΡˆΠΈΠ½Π½ΠΎΠΌΡƒ ΠΎΠ±ΡƒΡ‡Π΅Π½ΠΈΡŽ
Anatoly Levenchuk
Β 
PPTX
М.Π‘ΡƒΡ…Π°Ρ€ΠΈΠ½ -- DSM Π² Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π½ΠΎΠΌ ΠΏΡ€ΠΎΠ΅ΠΊΡ‚ΠΈΡ€ΠΎΠ²Π°Π½ΠΈΠΈ
Anatoly Levenchuk
Β 
PPTX
ΠœΠΎΠ΄Π΅Π»Π΅ΠΎΡ€ΠΈΠ΅Π½Ρ‚ΠΈΡ€ΠΎΠ²Π°Π½Π½ΠΎΡΡ‚ΡŒ Π² ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€Π½ΠΎΠ΅ ΠΎΠ±Ρ€Π°Π·ΠΎΠ²Π°Π½ΠΈΠ΅: ΠΈΡ‚ΠΎΠ³ΠΈ 2014 ΠΈ ΠΏΠ»Π°Π½Ρ‹.
Anatoly Levenchuk
Β 
PPTX
Π‘.Позин, Π•.Π“ΠΎΡ€Π±ΡƒΠ½ΠΎΠ²Π° -- Ρ€Π°Π·Π²ΠΈΡ‚ΠΈΠ΅ ядра Essence для стадии сопровоТдСния
Anatoly Levenchuk
Β 
PDF
Π‘.ΠšΠΎΠ²Π°Π»Ρ‘Π² -- тСория ΠΊΠ°Ρ‚Π΅Π³ΠΎΡ€ΠΈΠΉ ΠΊΠ°ΠΊ матСматичСскоС основаниС MBSE
Anatoly Levenchuk
Β 
PDF
Tim Weilkiens - Systems engineering: consulting services, masters curriculum ...
Alexander Shamanin
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Ρ‚Ρ€Π΅Π½Π°ΠΆΡ‘Ρ€ ΠΊΠ»ΡƒΠ±Π° ΠΎΠ΄ΠΈΠ½ΠΎΠΊΠΈΡ… ΠΌΠΎΠ·Π³ΠΎΠ²
Anatoly Levenchuk
Β 
PPTX
А.Иванов -- БистСмная инТСнСрия SmartGrid
Anatoly Levenchuk
Β 
PPTX
К.Π“Π°ΠΉΠ΄Π°ΠΌΠ°ΠΊΠ°, А.Π•Ρ„Ρ€Π΅ΠΌΠΎΠ² -- ArchiMate + Essence = ArchiEssence
Anatoly Levenchuk
Β 
PPTX
ВячСслав ΠœΠΈΠ·Π³ΡƒΠ»ΠΈΠ½ - Π Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ Ρ€Π°Π±ΠΎΡ‚Ρ‹ Π½Π° INCOSE WS 2017
Alexander Shamanin
Β 
PDF
М.АкоСв -- систСмная Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΠ° ΠΈ ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅
Anatoly Levenchuk
Β 
PDF
Π’.Π‘Π°Ρ‚ΠΎΠ²Ρ€ΠΈΠ½ -- Основания систСмной ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π‘ΡƒΠ΄ΡƒΡ‰Π΅Π΅ проСктирования
Anatoly Levenchuk
Β 
М.Гайворонский -- ΠΎΠΏΡ‹Ρ‚ Ρ€Π°Π·Ρ€Π°Π±ΠΎΡ‚ΠΊΠΈ БАУ двигатСля
Anatoly Levenchuk
Β 
Ali Mousavi -- Event modeling
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- плохая ΠΌΠΎΠ΄ΡƒΠ»ΡŒΠ½ΠΎΡΡ‚ΡŒ
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Ρ‚Ρ€Π΅Π½Π΄Ρ‹ Π² ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ Ρ‚Ρ€Π΅Π±ΠΎΠ²Π°Π½ΠΈΠΉ
Anatoly Levenchuk
Β 
А.Π•Ρ„Ρ€Π΅ΠΌΠΎΠ² -- встрСчи Русского отдСлСния INCOSE
Anatoly Levenchuk
Β 
О.Π‘Π°Π²ΠΈΠ½ -- Modelica Π² Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π½ΠΎΠΌ ΠΌΠΎΠ΄Π΅Π»ΠΈΡ€ΠΎΠ²Π°Π½ΠΈΠΈ
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠžΠ±Ρ€Π°Π·ΠΎΠ²Π°Π½ΠΈΠ΅ ΠΏΠΎ ΠΌΠ°ΡˆΠΈΠ½Π½ΠΎΠΌΡƒ ΠΎΠ±ΡƒΡ‡Π΅Π½ΠΈΡŽ
Anatoly Levenchuk
Β 
М.Π‘ΡƒΡ…Π°Ρ€ΠΈΠ½ -- DSM Π² Π°Ρ€Ρ…ΠΈΡ‚Π΅ΠΊΡ‚ΡƒΡ€Π½ΠΎΠΌ ΠΏΡ€ΠΎΠ΅ΠΊΡ‚ΠΈΡ€ΠΎΠ²Π°Π½ΠΈΠΈ
Anatoly Levenchuk
Β 
ΠœΠΎΠ΄Π΅Π»Π΅ΠΎΡ€ΠΈΠ΅Π½Ρ‚ΠΈΡ€ΠΎΠ²Π°Π½Π½ΠΎΡΡ‚ΡŒ Π² ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€Π½ΠΎΠ΅ ΠΎΠ±Ρ€Π°Π·ΠΎΠ²Π°Π½ΠΈΠ΅: ΠΈΡ‚ΠΎΠ³ΠΈ 2014 ΠΈ ΠΏΠ»Π°Π½Ρ‹.
Anatoly Levenchuk
Β 
Π‘.Позин, Π•.Π“ΠΎΡ€Π±ΡƒΠ½ΠΎΠ²Π° -- Ρ€Π°Π·Π²ΠΈΡ‚ΠΈΠ΅ ядра Essence для стадии сопровоТдСния
Anatoly Levenchuk
Β 
Π‘.ΠšΠΎΠ²Π°Π»Ρ‘Π² -- тСория ΠΊΠ°Ρ‚Π΅Π³ΠΎΡ€ΠΈΠΉ ΠΊΠ°ΠΊ матСматичСскоС основаниС MBSE
Anatoly Levenchuk
Β 
Tim Weilkiens - Systems engineering: consulting services, masters curriculum ...
Alexander Shamanin
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Ρ‚Ρ€Π΅Π½Π°ΠΆΡ‘Ρ€ ΠΊΠ»ΡƒΠ±Π° ΠΎΠ΄ΠΈΠ½ΠΎΠΊΠΈΡ… ΠΌΠΎΠ·Π³ΠΎΠ²
Anatoly Levenchuk
Β 
А.Иванов -- БистСмная инТСнСрия SmartGrid
Anatoly Levenchuk
Β 
К.Π“Π°ΠΉΠ΄Π°ΠΌΠ°ΠΊΠ°, А.Π•Ρ„Ρ€Π΅ΠΌΠΎΠ² -- ArchiMate + Essence = ArchiEssence
Anatoly Levenchuk
Β 
ВячСслав ΠœΠΈΠ·Π³ΡƒΠ»ΠΈΠ½ - Π Π΅Π·ΡƒΠ»ΡŒΡ‚Π°Ρ‚Ρ‹ Ρ€Π°Π±ΠΎΡ‚Ρ‹ Π½Π° INCOSE WS 2017
Alexander Shamanin
Β 
М.АкоСв -- систСмная Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΠ° ΠΈ ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅
Anatoly Levenchuk
Β 
Π’.Π‘Π°Ρ‚ΠΎΠ²Ρ€ΠΈΠ½ -- Основания систСмной ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ
Anatoly Levenchuk
Β 

Similar to A.Levenchuk -- Machine learning engineering (20)

PPTX
Tutorial helsinki 20180313 v1
home
Β 
PPTX
Future of Engineering
Anatoly Levenchuk
Β 
PPTX
Fundamntl of computer programing in python.pptx
RahulSingh190790
Β 
PPTX
01 computing
Sulman Ahmed
Β 
PDF
Implementation of Machine Learning Algorithms Using Control Flow and Dataflow...
hittemozald8
Β 
PPTX
Artificial Intelligence in Laymen Terms
Amity University | FMS - DU | IMT | Stratford University | KKMI International Institute | AIMA | DTU
Β 
PDF
Pydata Chicago - work hard once
Ji Dong
Β 
PDF
Implementation of Machine Learning Algorithms Using Control Flow and Dataflow...
dwzlauffsw3467
Β 
PPTX
TRIK robotics
Iakov Kirilenko
Β 
PPTX
MODEL-DRIVEN ENGINEERING (MDE) in Practice
Hussein Alshkhir
Β 
PPTX
Recent Advances in Machine Learning: Bringing a New Level of Intelligence to ...
Brocade
Β 
PPTX
AI-Driven Science and Engineering with the Global AI and Modeling Supercomput...
Geoffrey Fox
Β 
PPTX
Addressing learning gaps and career oppurtunities after B.Sc computer science
sandhya12bansal
Β 
PDF
Foundation of High Performance Computing HPC
nadiabha
Β 
PPTX
Artificial Intelligence in Industry 5.pptx
Rukminisrikant Revuru
Β 
PPTX
Teaching Machine Learning with Physical Computing - July 2023
Hal Speed
Β 
PPTX
hardware security security-210714143458.pptx
BEVARAVASUDEVAAP1813
Β 
PDF
Data-X-v3.1
Ikhlaq Sidhu
Β 
PDF
Data-X-Sparse-v2
Ikhlaq Sidhu
Β 
PPTX
Computer science & IT Engineering.
Samson2323
Β 
Tutorial helsinki 20180313 v1
home
Β 
Future of Engineering
Anatoly Levenchuk
Β 
Fundamntl of computer programing in python.pptx
RahulSingh190790
Β 
01 computing
Sulman Ahmed
Β 
Implementation of Machine Learning Algorithms Using Control Flow and Dataflow...
hittemozald8
Β 
Artificial Intelligence in Laymen Terms
Amity University | FMS - DU | IMT | Stratford University | KKMI International Institute | AIMA | DTU
Β 
Pydata Chicago - work hard once
Ji Dong
Β 
Implementation of Machine Learning Algorithms Using Control Flow and Dataflow...
dwzlauffsw3467
Β 
TRIK robotics
Iakov Kirilenko
Β 
MODEL-DRIVEN ENGINEERING (MDE) in Practice
Hussein Alshkhir
Β 
Recent Advances in Machine Learning: Bringing a New Level of Intelligence to ...
Brocade
Β 
AI-Driven Science and Engineering with the Global AI and Modeling Supercomput...
Geoffrey Fox
Β 
Addressing learning gaps and career oppurtunities after B.Sc computer science
sandhya12bansal
Β 
Foundation of High Performance Computing HPC
nadiabha
Β 
Artificial Intelligence in Industry 5.pptx
Rukminisrikant Revuru
Β 
Teaching Machine Learning with Physical Computing - July 2023
Hal Speed
Β 
hardware security security-210714143458.pptx
BEVARAVASUDEVAAP1813
Β 
Data-X-v3.1
Ikhlaq Sidhu
Β 
Data-X-Sparse-v2
Ikhlaq Sidhu
Β 
Computer science & IT Engineering.
Samson2323
Β 
Ad

More from Anatoly Levenchuk (20)

PPTX
Contemporary Systems Engineering (oct 2022)
Anatoly Levenchuk
Β 
PPTX
Open-endedness curriculum at EEM Institute
Anatoly Levenchuk
Β 
PPTX
ΠŸΡ€Π°ΠΊΡΠΈΠΎΠ»ΠΎΠ³ΠΈΡ ΠΈ систСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Ρ€Π°Π·Π²ΠΈΡ‚ΠΈΠ΅ личности
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- стСйкхолдСрскоС мастСрство
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- SysArchi
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠΊΠ°ΠΊ Π²Ρ‹ΠΆΠΈΡ‚ΡŒ Π² эпоху ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½ ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠŸΡ€Π°ΠΊΡ‚ΠΈΠΊΠΈ систСмной ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π²ΠΈΠ·ΡƒΠ°Π»ΡŒΠ½ΠΎΠ΅ ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- систСмноС Ρ€Π°Π·Π²ΠΈΡ‚ΠΈΠ΅ личности
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π‘ΡƒΠ΄ΡƒΡ‰Π΅Π΅ Π΄Π΅Π²Π΅Π»ΠΎΠΏΠΌΠ΅Π½Ρ‚Π°
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- БистСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅ Π² ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ прСдприятий
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- БистСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅ ΠΈ ΡƒΠΏΡ€Π°Π²Π»Π΅Π½ΠΈΠ΅ ΠΊΠΎΠ½Ρ„ΠΈΠ³ΡƒΡ€Π°Ρ†ΠΈΠ΅ΠΉ
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π°ΠΏΠΏΠ°Ρ€Π°Ρ‚Π½ΠΎΠ΅ ускорСниС Π°Π½Π°Π»ΠΈΡ‚ΠΈΠΊΠΈ Π² BigData
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π‘ΡƒΠ΄ΡƒΡ‰Π΅Π΅ проСктирования
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π±Π΅Π·Π»ΡŽΠ΄Π½Ρ‹Π΅ (Π΄Π΅Π·)ΠΎΡ€Π³Π°Π½ΠΈΠ·Π°Ρ†ΠΈΠΈ
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠΏΡ€Π΅Π΄ΠΏΡ€ΠΈΠ½ΠΈΠΌΠ°Ρ‚Π΅Π»ΡŒΡΡ‚Π²ΠΎ: кСйс NVIDIA
Anatoly Levenchuk
Β 
PPTX
БистСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅ -- нСпопсовый ΠΎΠ±Π·ΠΎΡ€ курса
Anatoly Levenchuk
Β 
PPTX
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- систСмный фитнСс
Anatoly Levenchuk
Β 
PPTX
Π‘Π΅Π·Π»ΡŽΠ΄Π½Ρ‹Π΅ ΠΎΡ€Π³Π°Π½ΠΈΠ·Π°Ρ†ΠΈΠΈ ΠΈ ΠΈΡ… ΠΏΡ€ΠΎΠ±Π»Π΅ΠΌΡ‹
Anatoly Levenchuk
Β 
Contemporary Systems Engineering (oct 2022)
Anatoly Levenchuk
Β 
Open-endedness curriculum at EEM Institute
Anatoly Levenchuk
Β 
ΠŸΡ€Π°ΠΊΡΠΈΠΎΠ»ΠΎΠ³ΠΈΡ ΠΈ систСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Ρ€Π°Π·Π²ΠΈΡ‚ΠΈΠ΅ личности
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- стСйкхолдСрскоС мастСрство
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- SysArchi
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠΊΠ°ΠΊ Π²Ρ‹ΠΆΠΈΡ‚ΡŒ Π² эпоху ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½ ΠΏΠ΅Ρ€Π΅ΠΌΠ΅Π½
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠŸΡ€Π°ΠΊΡ‚ΠΈΠΊΠΈ систСмной ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π²ΠΈΠ·ΡƒΠ°Π»ΡŒΠ½ΠΎΠ΅ ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- систСмноС Ρ€Π°Π·Π²ΠΈΡ‚ΠΈΠ΅ личности
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π‘ΡƒΠ΄ΡƒΡ‰Π΅Π΅ Π΄Π΅Π²Π΅Π»ΠΎΠΏΠΌΠ΅Π½Ρ‚Π°
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- БистСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅ Π² ΠΈΠ½ΠΆΠ΅Π½Π΅Ρ€ΠΈΠΈ прСдприятий
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- БистСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅ ΠΈ ΡƒΠΏΡ€Π°Π²Π»Π΅Π½ΠΈΠ΅ ΠΊΠΎΠ½Ρ„ΠΈΠ³ΡƒΡ€Π°Ρ†ΠΈΠ΅ΠΉ
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π°ΠΏΠΏΠ°Ρ€Π°Ρ‚Π½ΠΎΠ΅ ускорСниС Π°Π½Π°Π»ΠΈΡ‚ΠΈΠΊΠΈ Π² BigData
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π‘ΡƒΠ΄ΡƒΡ‰Π΅Π΅ проСктирования
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- Π±Π΅Π·Π»ΡŽΠ΄Π½Ρ‹Π΅ (Π΄Π΅Π·)ΠΎΡ€Π³Π°Π½ΠΈΠ·Π°Ρ†ΠΈΠΈ
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- ΠΏΡ€Π΅Π΄ΠΏΡ€ΠΈΠ½ΠΈΠΌΠ°Ρ‚Π΅Π»ΡŒΡΡ‚Π²ΠΎ: кСйс NVIDIA
Anatoly Levenchuk
Β 
БистСмноС ΠΌΡ‹ΡˆΠ»Π΅Π½ΠΈΠ΅ -- нСпопсовый ΠΎΠ±Π·ΠΎΡ€ курса
Anatoly Levenchuk
Β 
А.Π›Π΅Π²Π΅Π½Ρ‡ΡƒΠΊ -- систСмный фитнСс
Anatoly Levenchuk
Β 
Π‘Π΅Π·Π»ΡŽΠ΄Π½Ρ‹Π΅ ΠΎΡ€Π³Π°Π½ΠΈΠ·Π°Ρ†ΠΈΠΈ ΠΈ ΠΈΡ… ΠΏΡ€ΠΎΠ±Π»Π΅ΠΌΡ‹
Anatoly Levenchuk
Β 
Ad

Recently uploaded (20)

PPTX
MSP360 Backup Scheduling and Retention Best Practices.pptx
MSP360
Β 
PPTX
✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨
SanjeetMishra29
Β 
PDF
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
Β 
PDF
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
Β 
PDF
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
Β 
PDF
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
Β 
PDF
Python basic programing language for automation
DanialHabibi2
Β 
PDF
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
Β 
PDF
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
Β 
PDF
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
Β 
PDF
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
Β 
PDF
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
Β 
PPTX
Building Search Using OpenSearch: Limitations and Workarounds
Sease
Β 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
Β 
PDF
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
Β 
PDF
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
Β 
PDF
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
Β 
PPTX
UiPath Academic Alliance Educator Panels: Session 2 - Business Analyst Content
DianaGray10
Β 
PDF
Jak MŚP w Europie Środkowo-Wschodniej odnajdujΔ… siΔ™ w Ε›wiecie AI
dominikamizerska1
Β 
PDF
SFWelly Summer 25 Release Highlights July 2025
Anna Loughnan Colquhoun
Β 
MSP360 Backup Scheduling and Retention Best Practices.pptx
MSP360
Β 
✨Unleashing Collaboration: Salesforce Channels & Community Power in Patna!✨
SanjeetMishra29
Β 
DevBcn - Building 10x Organizations Using Modern Productivity Metrics
Justin Reock
Β 
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
Β 
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
Β 
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
Β 
Python basic programing language for automation
DanialHabibi2
Β 
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
Β 
Chris Elwell Woburn, MA - Passionate About IT Innovation
Chris Elwell Woburn, MA
Β 
LLMs.txt: Easily Control How AI Crawls Your Site
Keploy
Β 
Bitcoin for Millennials podcast with Bram, Power Laws of Bitcoin
Stephen Perrenod
Β 
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
Β 
Building Search Using OpenSearch: Limitations and Workarounds
Sease
Β 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
Β 
The Builder’s Playbook - 2025 State of AI Report.pdf
jeroen339954
Β 
CIFDAQ Market Insights for July 7th 2025
CIFDAQ
Β 
CIFDAQ Weekly Market Wrap for 11th July 2025
CIFDAQ
Β 
UiPath Academic Alliance Educator Panels: Session 2 - Business Analyst Content
DianaGray10
Β 
Jak MŚP w Europie Środkowo-Wschodniej odnajdujΔ… siΔ™ w Ε›wiecie AI
dominikamizerska1
Β 
SFWelly Summer 25 Release Highlights July 2025
Anna Loughnan Colquhoun
Β 

A.Levenchuk -- Machine learning engineering

  • 1. Machine Learning Engineering Anatoly Levenchuk Copyright Β© 2016 by Anatoly Levenchuk. Permission granted to DeepHack and INCOSE to publish and use.
  • 2. What is machine learning as a human activity? β€’ Ontological question (Aristotle definition: via class-subclass specialization) β€’ Why it is important? – How to pay? [grants, investmetns, charity] – How to teach? [science, engineering, arts/crafts] – How to name and distinguish in communication (hiring – participation in division of labor)? 2 How you name yourself to colleagues, when hacking machine learning system?
  • 3. Machine learning is a… β€’ Science! MSc. in Machine Learning [BigData] β€’ Research? β€’ Engineering? β€’ Art? Programming is a… β€’ Science? Computer science, MSc. β€’ Research? Computer science, MSc. β€’ Engineering? Software engr., MSc. and MSE! β€’ Art? Master of Art in Mathematics! 3 http://www.computer.org/web/education/professional-competency-certifications https://www.kaggle.com/competitions
  • 4. Test: Why is my program not working? Why is my program not working? You need to know why? To repair compiler? Software engineer (systems) To advance theory? Computer Scientist You need to program working properly? Software engineer (application) 4
  • 5. Science Resulting in models, descriptions (theories), ontologies: β€’ M0 – manufacturing (not science!). Programmers are engineers: software is physical system! β€’ M1 – design/applied research (Edison) β€’ M2 – basic research (Einstein) β€’ M3 – philosophical logic/mathematics β€’ There are multiple meta-levels. β€’ Scientists produce these meta-descriptions 5
  • 6. Engineering β€’ Engineering – discipline, art, skill and profession of acquiring and applying scientific, mathematical, economic, social, and practical knowledge, in order to design and build structures, machines, devices, systems, materials and processes that safely realize improvements to the lives of people. β€’ Engineering is the application of mathematics, empirical evidence and scientific, economic, social, and practical knowledge in order to invent, innovate, design, build, maintain, research, and improve structures, machines, tools, systems, components, materials, and processes. 6 https://en.wikipedia.org/wiki/Engineering https://en.wikipedia.org/wiki/Outline_of_engineering
  • 7. Data scientists – ML Engineers 7 Model/Theory [metamodel] Engineering/Applied Research Reality/Data/Model Science/Basic Research If it is not about budgeting and social status, it need not to distinguish science and engineering! Practice both of them!
  • 8. Engineering for science 8 http://blogs.nvidia.com/blog/2016/01/12/accelerating-ai-artificial-intelligence-gpus/ Scientists are mere owner-operators of instruments. Who built the Big Hadron Collider? Experiments order by scientists, builds and carried by engineers, interprets by scientists.
  • 9. The sunset of the professions, not jobs! 9 β€’ Life-long β€’ Special education β€’ No other professions in a mix β€’ Several years long β€’ Additional training β€’ One competence in the mix Machine learning engineering is not a profession. It is a competency!
  • 10. Machine learning (systems) engineering β€’ Control (systems) engineering β€’ Machine Learning (systems) engineering 10 ? http://www.payscale.com/research/US/Job=Controls_Engineer/Salary β€’ Systems Engineer (IT) β€’ Cognitive/Machine Intelligence Systems Engineer ?
  • 11. What about jobs? 11 Algorithms + Data Structures = Programs (Niklaus Wirth) Scientist is not an engineer, data is not a system
  • 12. Kind of Engineerings β€’ Mechanical engineering β€’ Agriculture engineering β€’ Aerospace engineering – aircraft architecture β€’ Systems engineering β€’ System of systems engineering β€’ … β€’ Software engineering β€’ Control [systems] engineering – control [system] architecture β€’ Knowledge engineering -- architecture β€’ Machine learning [system] engineering β€’ … β€’ Neural engineering β€’ neural network engineering -- neural [network] architecture β€’ Feature engineering -- ??? 12
  • 13. Systems, Software, Machine Learning Engineerings β€’ Systems engineering [Bell Labs in 1940s, boosted as a profession by NCOSE 1990] β€’ Software engineering [term appeared in 1965, boosted by NATO as a profession in 1968] β€’ Machine learning engineering [term appeared in 2011] 13https://www.google.com/trends/explore#q=machine%20learning%20engineering&cmpt=q&tz=Etc%2FGMT-3
  • 14. Conversion of engineerings and Disruption of engineerings 14 Software Engineering Machine Learning Engineering Janosh Szepanovits. Convergence: Model- Based Software, Systems And Control Engineering + http://www.infoq.com/presentations/Model-Based-Design-Janos-Sztipanovits Le Bottou – Β«Machine Learning disrupts software engineeringΒ» http://leon.bottou.org/slides/2challenges/2challenges.pdf We can add: β€’ Machine learning disrupts systems engineering β€’ Machine learning disrupts control engineering β€’ … β€’ Machine learning disrupts contemporary engineering
  • 15. Can we use systems and software engineering wisdom in MLE? Le Bottou http://leon.bottou.org/slides/2challenges/2challenges.pdf β€’ Models as modules: problematic due to weak contracts (models behave differently on different input data) β€’ Learning algorithms as modules: problematic due to output depends on the training data which itself depends on every other module Engineering is not only about modularity and modular synthesis! What about other aspects?! β€’ More attention to left part of V-diagram β€’ Optimizations later β€’ … β€’ What else? 15
  • 16. Technical Debt 16 Machine Learning: The High-Interest Credit Card of Technical Debt http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43146.pdf Hidden Technical Debt in Machine Learning Systems http://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf β€’ Hack now, pay later (with interest, of course!). β€’ Based on heuristics from software engineering (same approach as our: usage of software and systems engineering wisdom in machine learning engineering). β€’ Set of domain-specific heuristics for machine learning
  • 17. Bionics and machine learning systems engineering β€’ In short: brain is only an inspiration, not a model for reproducing! β€’ There are other β€œlearning systems engineerings”: e.g. neural engineering (https://en.wikipedia.org/wiki/Neural_engineering). β€’ AGI (artificial general intelligence) is a far goal, but magnet for freaks of all sorts. Better not mention it. β€’ Biologically plausible machine learning is about science, not engineering. 17
  • 18. Knowledge engineering β€’ Ontology engineering (manually) β€’ Solutions are (manually) programmed. β€’ Example: robot-Β«butterflyΒ», https://youtu.be/kyvW5sOcZHU, https://youtu.be/V30e77x8BQA – Every type of movement should be programmed anew – Non-adaptable to changes of environment and device – The best science available up today! – Perfect, if CPS perform only one or two movements. Not for robots, definitely! β€’ No learning! 18
  • 20. Our definition of complexity Complex system – the one that does not fit in the sole engineer’s head, thus collaboration of a team and automation of a knowledge work are mandatory. E.g.: β€’ Aircraft β€’ programming-in-the small vs.programming in the large β€’ VLSI – very large scale integration, more than 1000 transistors on a single chip (now transistor count is more than 20bln. – FPGA Virtex-Ultrascale XCVU440) β€’ Artificial neural network – 16bln. parameters. 20
  • 21. Comlexity β€’ Systems Engineering β€’ Machine learning 21 Complex system: not fit into one (hundred) heads for its development. Stellarators, Tokamaks, BHC, aerospace and VLSI engineering. IBM Watson (up to 2011): team of 40. Still not very complex from engineering point of view. http://josephpcohen.com/w/visualizing-cnn-architectures-side-by-side-with-mxnet/http://787updates.newairplane.com/787-Suppliers/World-Class-Supplier-Quality
  • 23. AutoML β€’ Generative design/architecturing of networks β€’ Bayesian convergence β€’ Neuroevolution β€’ Dynamic neural description languages (e.g. Chainer) 23 Automatization of machine learning, CAMLE (computer-aided machine learning engineering) is the main trend of today and tomorrow!
  • 24. Master Algorithm Pedro Domingos [module/construction] β€’ Symbolic β€’ Evolution β€’ Connectivist β€’ Bayesian β€’ Analogy [No free lunch!] Sarath Chandar [component/function] β€’ multi-task learning β€’ transfer learning β€’ zero-shot/one-shot learning β€’ multi-modal learning β€’ reinforcement learning 24 http://apsarath.github.io/2016/01/19/agi/ http://www.amazon.com/dp/0465065708/
  • 25. Intellect-stack is only about one aspect of a whole intellect system. Intellect-stack is about Platforms (modules) = Β«how to make itΒ» Based on Fig.3 ISO 81346-1 -Modules =Components +Allocations 25 Modules and interfaces: platforms/layers Stack
  • 26. Platform β€’ This is module viewpoint (Β«how to makeΒ») β€’ Platform is a technology stack layer β€’ Cohesive set of modules with published API β€’ Can be based on top of other platform 26
  • 27. Intelligence Platform Stack and machine learning engineering in it 27 Application (domain) Platform Cognitive Architecture Platform Learning Algorithm Platform Computational library General Computer Language CPU GPU/FPGA/Physical computation Drivers GPU/FPGU/Physical computation Accelerator Neurocompiler Neuromorphic driver Neuromorphic chip Disruptionenablers Disruptiondemand Thanks for computer gamers for their disruption demand to give us disruption enabler such as GPU!
  • 28. Alternative deep learning stack (as viewed by GPU hardware people) 28http://www.nextplatform.com/2015/12/07/gpu-platforms-emerge-for-longer-deep-learning-reach/ β€’ No cognitive and application levels β€’ Languages unimportant β€’ Chassis, backplane, blades importans (separate layer) β€’ No neuromorphic processing
  • 29. Hardware Acceleration (except GPU) Is this machine learning engineering? No! But… β€’ Algorithm-dependent β€’ Need compilation (drivers) β€’ Speed rules β€’ Power rules β€’ Scale rules β€’ GPU β€’ FPGA β€’ ASIC β€’ Neuromorphic chips β€’ Physical computing 29 http://lighton.io/ β€’ Approximating kernels at the speed of light http://arxiv.org/abs/1510.06664 Analog, optical device, that performs the random projections literally at the speed of light without having to store any matrix in memory. This is achieved using the physical properties of multiple coherent scattering of coherent light in random media. β€’ Towards Trainable Media: Using Waves for Neural Network-style Learning β€’ Bitwise Neural Networks http://arxiv.org/abs/1601.06071 β€’ Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware http://arxiv.org/abs/1601.04187 http://arxiv.org/abs/1510.03776
  • 30. General Computer Language Computer science + Software engineering β€’ Important! Separate layer in intellect-stack! β€’ 2 language problem β€’ experiment and production, like deep learning frameworks (speed) β€’ Β«WrappersΒ» in libraries (thresholds in understanding of a full stack up to hardware bottom) β€’ My preference: Julia (http://julialang.org/) β€’ Scientific computing is design goal of Julia, MATLAB-similar syntax β€’ 2 language problem solved (speed of computation as in C, speed of writing code as in Python) β€’ Extensive mathematical function library, Base library and external packages in native Julia β€’ Parallel computing supported (GPU supported too) β€’ Not object-oriented, using multiple dispatch as expression problem solution (good modularity) β€’ Version 0.4.3 now (1.0 expecting in one year) β€’ Caution: slightly more complex than Python, should not be your first computer language… β€’ MXNet deep learning framework have Julia wrapper β€’ DSL for deep learning is not General Computer Language β€’ Probabilistic programming languages -- http://probabilistic-programming.org/wiki/Home β€’ DNN description languges, like in CNTK -- https://github.com/Microsoft/CNTK 30
  • 31. Computation libraries/frameworks/platforms Not a machine learning engineering! β€’ Computation libraries οƒŸοƒ  Drivers+Hardware (GPU, clusters) β€’ Linear algebra, optimization, autodiff, symbolic computations, etc. β€’ Can be standalone platform, thus differ from machine learning libraries (general algorithms for multiple purposes: bioinformatics, physics, astronomy, engineering, machine learning etc.) β€’ Deep learning frameworks often includes such a library (Torch, Theano, …). β€’ Scikit (NumPy, SciPy, and matplotlib) β€’ Nd4j (n-dimentional arrays for Java) β€’ Julia packages β€’ … β€’ Non-opensource: Mathematica, Maple… 31 Machine learning is β€œyet another domain modules and DSL” for them!
  • 32. Learning algorithm frameworks (not systems)! Machine learning engineering! β€’ Gentleman algorithm set (CNN, RNN,…) β€’ Updating with an arxiv.org papers rhythm! β€’ Network description language – DSL for machine learning engineering β€’ Experiments and production (scalable!) β€’ Extensibility (on base of general computing language and scientific computing library: on base of another layer platform in intellect-stack) β€’ Presented as The Machine Learning Platform (including all lower levels assembled and tuned) β€’ There are hundreds of its: no less then Β«web frameworksΒ» in early web 32 http://www.slideshare.net/yutakashino/ss-56291783 β€’ Google β€’ Facebook β€’ Microsoft β€’ Baidu β€’ IBM β€’ Samsung β€’ … + standard datasets for comparisons and benchmarking + other tribes platforms
  • 33. Construction (type of modules) in machine learning β€’ Deep learning classics (DSL in deep learning frameworks) β€’ Probabilistic languages http://probabilistic-programming.org/, https://probmods.org/ β€’ Deep learning and Bayesian conversion -- )http://www.nextplatform.com/2015/09/24/deep-learning-and-a-new-bayesian-golden-age/, http://blog.shakirm.com/2015/10/bayesian-reasoning-and-deep-learning/, http://arxiv.org/abs/1512.05287 β€’ Differentiable languages and datatypes http://colah.github.io/posts/2015-09-NN- Types-FP/, http://www.blackboxworkshop.org/pdf/nips2015blackbox_zenna.pdf, http://arxiv.org/abs/1506.02516 β€’ … β€’ Blends and hybrids of many other learning architectures 33 Varieties in representations: in deep learning abstraction is architecturally layered, in other approaches it different!
  • 34. Algorithm platform + Hardware platform = Algorithm platform (hardware is not visible for a platform user, but still matter!) 34 http://blogs.microsoft.com/next/2016/01/25/microsoft-releases-cntk-its-open-source-deep-learning-toolkit-on-github/
  • 35. Cognitive systems/architectures Learning, communications, reasoning, planning β€’ Cognitive = knowledge processing. Knowledge is information that is useful in variety of situations. β€’ Cognitive architecture/system is a platform for multiple application systems. β€’ Ensembles of learning algorithms: it is close to cognitive systems engineering β€’ Cognitive systems engineering is a machine learning systems engineering plus something else β€’ Something else: e.g. knowledge engineering: manual coding (formalization) of knowledge. β€’ Machine learning systems engineering is not cognitive systems engineering, it is smaller! 35
  • 36. Machine Learning and Cognitive Level β€’ Β«deep learning research is likely to continue its expansion from traditional pattern recognition jobs to full-scale AI tasks involving symbolic manipulation, memory, planning and reasoning. This will be important for reaching to full understanding of natural language and dialogue with humans (i.e., pass the Turing test). Similarly, we are seeing deep learning expanding into the territories of reinforcement learning, control and robotics and that is just the beginningΒ» -- Joshua Bengio https://www.quora.com/Where-is-deep-learning-research-headed 36 If we can learn to reason, plan, model, act – then machine learning engineering will be cognitive systems engineering! Machine intelligence vs. artificial intelligence
  • 37. Example: MANIC A Minimal Architecture for General Cognition (http://arxiv.org/abs/1508.00019) β€’ Keywords: action, planning, observation, decisions, knowledge, … β€’ Is it keywords for learning systems engineering? 37
  • 38. Application level of intellect-stack β€’ Killer application for learning systems is here! β€’ Domain specificity and data is here! β€’ End users and money are here! β€’ Systems engineering is here! 38 This chart is only about enterprise AI systems market. https://www.tractica.com/newsroom/press-releases/artificial-intelligence-for-enterprise-applications-to-reach-11-1-billion-in-market-value-by-2024/ If you have no application of interest, there will be no data, no money, no developments, no engineering. Most machine learning engineering is applied. Only small part is machine learning platform development.
  • 39. Application level: systems engineering β€’ Strategizing and Conceptual design β€’ Requirements engineering β€’ System Architecture β€’ V&V β€’ Configuration management β€’ Machine learning engineers is one of multiple engineers that participate in a cyber- physical system project team. 39 Sensors Consoles http://www.nist.gov/el/nist-releases-draft-framework-cyber-physical-systems-developers.cfm Actuators Monitors
  • 40. Life cycle stages dictionary 40 Machine learning Systems engineering Conception and requirements Conception and requirements Architecture and Design Architecture and Design Training Manufacturing Transfer learning, ensembling Integration Validation and verification Validation and verification Inference Operation
  • 41. Stakeholders concerns Domain-specific concern: β€’ Expressivity β€’ Computational efficiency β€’ Trainability β€’ Good generalization (not overfitting) Traditional concerns β€’ Composability – layering, ensembling β€’ Compositiality – transfer learning β€’ Resilience 41
  • 42. Intellect-stack and machine learning (systems) engineering β€’ Machine learning (systems) engineering cover now only small part of the whole intellect-stack but interact with all levels. β€’ No one Googbookdu can develop all levels in intellect- stack platforms (from hardware accelerators in the bottom up to application on the top) by itself. Maybe except IBM that can span from TrueNorth to IBM Watson applications ;-) β€’ Interfaces from supporting platforms will be stabilizing and… in constant update (like in software engineering APIs: change of everything once in 5 years). β€’ Technology disruption starts with low (enabling) levels of a stack, demand provides from upper level, thus nobody in the middle can ignore developments in other layer platforms. 42
  • 43. Intellect-Stack 43 Application (domain) Platform Cognitive Architecture Platform Learning Algorithm Platform Computational library General Computer Language CPU GPU/FPGU/Physical computation Drivers GPU/FPGA/Physical computation Accelerator Neurocompiler Neuromorphic driver Neuromorphic chip Disruptionenablers Disruptiondemand Where are you now? Where are you tomorrow?
  • 44. Thank you! Anatoly Levenchuk, TechInvestLab, president INCOSE Russian chapter, research director https://ru.linkedin.com/in/ailev [email protected] Blog in Russian: http://ailev.ru 44