Gen AI: tech du-jour or the next big thing?

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Gen AI: tech du-jour or the next big thing?
Massimo Re Ferrè
Senior Principal Product Manager
AWS

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2
Disclaimer: Gen AI is much bigger than this content
• This talk covers Gen AI from the (philosophical) angle of my day-by-day (NGDE)
• I can’t speak to why Gen AI is making Hollywood mad

Mainframes
Technology wave #1 Technology wave #2
Personal Computers
Functions
Technology wave #3
T
e
c
h
n
o
l
o
g
y
d
e
l
i
v
e
r
y
m
o
d
e
l
Data center
Cloud
The IT world (through my lenses, over 30 years)
Servers
VMs
Containers

My aha moment for Gen AI

submit
How does Gen AI work (in simple terms)?

How can I use this knowledge and reason about it to create a new asset?
An asset being a piece of code, a whole program, a blog, an architecture, a troubleshooting
workflow, a db query and more outside of the IT realm (a poem, a picture, a receipt …)
Source of unstructured knowledge
Why is Gen AI a big deal (in my opinion)?

7
You
Read and
memorize it all
(LOL – yeah sure)
LLM
Train on it
(doable)
(1)
(2)
(3a)
Natural language
conversation
(3b)
Source of unstructured knowledge
asset
asset
Search engines (possibly
not relevant and still hard
- you are the integrator
and generator of a new
asset – text or code)
The big deal
Why is Gen AI a big deal (in my opinion)?

Generative AI applicability in your stack
Builder
User
The builder writes code
The user interacts with the
application using a query language
The code is a bunch of functions

User
https://stackoverflow.com/questions/16864564/complex-sql-select-statement
application in English
Builder
The builder writes code
Gen AI @ Consumption Time

“Create a function that does string comparisons”
Builder
The builder writes in English a
description of the code they need
Gen AI @ Development Time
User

https://www.haihai.ai/programming-with-llm/
Gen AI @ Run Time
“Create a function that does string comparisons”
Builder
The builder writes in English a
description of the code they need
Gen AI @ Development Time
User
The code is a set of English statements
Let’s talk about
“Gen AI
assistants”

Where are these assistants going?
How to measure productivity?
Can you trust them?
Are they going to replace us?
Gen AI assistant(s) FAQs
12

Can you trust them?
13

The progression of assistants - inline coding

The progression of assistants – chat

The progression of assistants – agents
(1) Prompt
(2) Plan
(3) Code

From “english-to-code” to “code-to-english”
Rewriting a README file
Documenting functions in the code
https://it20.info/2024/3/using-the-amazon-q-feature-development-capability-to-produce-documentation/

Can you trust them?
18

Acceptance rate? This is useful because it doesn’t require explicit feedback (but is it a good proxy?)

Thumbs up / thumbs down? This is useful because it is a good proxy (but requires explicit feedback)

Surveys This is useful because it is a good proxy (but requires time and it’s more work for eveyone)

Tracking “time to job done” This is one of the best dimension to track (in my opinion), but…

From an AWS colleague:
“… they [ customer ] see developers wanting to use this tool [ Q ] to complete their existing tasks faster and get off work early”
23
All we know for sure is…
https://twitter.com/astasiamyers/status/1771344440617246774?s=43
Use a Gen AI assitant if you want to go home early!

Can you trust them?
24

2024 (the emergence of Generative AI)
25
“How can you trust a Gen AI assistant? How can you make sure it’s not hallucinating?
You’d need to verify every answer and every artifact!”

Throw back to 2001 (the emergence of VMs)
26
“How can you trust a Gen AI assistant? How can you make sure it’s not hallucinating?
You’d need to verify every answer and every artifact!”
VM the bug is not in the hypervisor
reproduce your bug on a physical server

• Can you trust…
- Consultants?
- Solutions Architects?
- TAMs?
- Me? (hint: you shouldn’t)
• Can you trust a pipeline? A test?
• At the end of the day, it is about controls and risk mitigation
• Can you absorb a certain amount of risk for the value that is being returned?
• What are the controls you have in place to mitigate the risk?
Do you trust everything and everyone?

“Everything fails all the time”
(why Gen AI shouldn’t?)

• Something I hear regularly: “this is not (entirely) correct”
• A better mental model (IMO): ”did it help me saving time?”
A problem of mindset?

Can you trust them?
30

The role of Gen AI assistants
31
https://x.com/AuthorJMac/status/1773679197631701238
“I want AI to do my laundry and
dishes so that I can do art and
writing, not for AI to do my art
and writing so that I can do my
laundry and dishes.”

32
IaC
Progress
over
time
Cloudformation template
What if this is just yet another abstraction?
CDK program
Natural language interface
Aspirationally

Developer
Gen AI Assistant
Boost zone Learning zone Danger zone
The longer this line the riskier it becomes
The longer this line the less useful it becomes
You are out of control
You are in control
Limited value zone
High return
Low return
A tentative framework for Gen AI assistant adoption (today)
Low task
complexity
High task
complexity
Low level of
knowledge
High level of
knowledge
You are here

• Keep an open mind
• Proving these things wrong is extremely easy
• But extracting value from them can be very useful
• You can rent a car and drive it into a wall or go to the beach with it
• Where do you want to go?
Conclusions

Thank you!
Massimo Re Ferrè
Mail: mreferre@amazon.com
Twitter: @mreferre
35

Gen AI: tech du-jour or the next big thing?

More Related Content

Similar to Gen AI: tech du-jour or the next big thing? (20)

Recently uploaded (20)

Gen AI: tech du-jour or the next big thing?