TensorFlow 2.0 Autographs - For TFUG - Vik Pant
0 likes139 views
The document discusses TensorFlow 2.0's use of dataflow computation graphs for parallel computing, highlighting their advantages such as simple deployment and graph-based optimizations, alongside disadvantages like unnatural code structure and limitations in debugging. It emphasizes the importance of eager execution for interactive experimentation and debugging, while recommending code that supports both eager and graph execution for production. Additionally, it introduces Autograph, which allows for writing Python code while converting it to TensorFlow graph code, though with certain limitations in converting arbitrary Python code.
TensorFlow 2.0 Autographs - For TFUG - Vik Pant
- 2. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation Dataflow Computation Graphs • TensorFlow implements dataflow, which is a common programming model for parallel computing. • In a dataflow graph, the nodes represent units of computation, and the edges represent the data consumed or produced by a computation. • TensorFlow uses a dataflow graph to represent your computation in terms of the dependencies between individual operations. • This leads to a low-level programming model in which you first define the dataflow graph, then create a TensorFlow session to run parts of the graph across a set of local and remote devices.
- 3. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation Advantages of using TensorFlow graphs • Simple deployment to a platform independent server. • Automatic distribution and replication (placing nodes on the distributed system). • Graph-based optimizations • common subexpression elimination • constant-folding • … • Compilation and kernel fusion.
- 4. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation Disadvantages of Graph Specification • Structure of code is unnatural and does not use Python data structures. • Graph control flow must be used rather than Python control flow and this complicates the specification of dynamic models. • Standard Python debugging tools cannot be used for immediate error reporting. • Not possible to quickly iterate on small code changes and minor reconfigurations.
- 5. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation Eager Execution • TensorFlow's eager execution is an imperative programming environment that evaluates operations immediately. • It does not build graphs so operations return concrete values instead of constructing a computational graph to run later. • tf.Tensor objects reference concrete values instead of symbolic handles to nodes in a computational graph. • Since there isn't a computational graph to build and run later in a session, it's easy to inspect results using print() or a debugger. • This makes it easy to get started with TensorFlow and debug models, and it reduces boilerplate as well.
- 6. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation Best Practice • It's best to write code for both eager execution and graph execution. • This gives you • eager's interactive experimentation and debuggability • distributed performance benefits of graph execution • You should • write, debug, and iterate in eager execution • then import the model graph for production deployment
- 7. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation AutoGraph: Easy control flow for graphs • AutoGraph helps you write complicated graph code using normal Python. • Behind the scenes, AutoGraph automatically transforms your code into the equivalent TensorFlow graph code. • Equivalent graph code means code that generates a TensorFlow graph when run. • The generated graph has the same effects as the original code when executed (for example with tf.function or tf.compat.v1.Session.run). • In other words, using AutoGraph can be thought of as running Python in TensorFlow.
- 8. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation TensorFlow AutoGraph Capabilities and Limitations • TF AutoGraph converts Eager Python code into TensorFlow graph- mode code. • For example, users write code with if and while and AutoGraph automatically converts it into the equivalent tf.cond, and tf.while_loop. • Python is a large language, so hoping to convert arbitrary Python code directly to TF graphs is overly ambitious. • However, the Python code written to metaprogram TF graphs is in practice a restricted subset. • TensorFlow aims to support as much of this subset as possible.
- 9. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation Already Supported Now
- 10. This presentation may contain copyrighted materials that have been used under the fair use provisions of copyright laws. Further use, duplication, or distribution is prohibited. Source: TensorFlow Documentation Not or Partially Supported Now