Understanding Gemini 1.5 Pro's 1M Token Context Window

The release of Google's Gemini Pro 1.5 is, IMO, the biggest piece of A.I. news yet this year. The LLM has a gigantic million-token context window, multimodal inputs (text, code, image, audio, video) and GPT-4-like capabilities despite being much smaller and faster. Key Features 1. Despite being a mid-size model (so much faster and cheaper), its capabilities rival the full-size models Gemini Ultra 1.0 and GPT-4, which are the two most capable LLMs available today. 2. At a million tokens, its context window demolishes Claude 2, the foundation LLM with the next longest context window (Claude 2's is only a fifth of the size at 200k). A million tokens corresponds to 700,000 words (seven lengthy novels) and Gemini Pro 1.5 accurately retrieves needles from this vast haystack 99% of the time! 3. Accepts text, code, images, audio (a million tokens corresponds to 11 hours of audio), and video (1MM tokens = an hour of video). Today's episode contains an example of Gemini Pro 1.5 answering my questions about a 54-minute-long video with astounding accuracy and grace. How did Google pull this off? • Gemini Pro 1.5 is a Mixture-of-Experts (MoE) architecture, routing your input to specialized submodels (e.g., one for math, one for code, etc.), depending on the broad topic of your input. This allows for focused processing and explains both the speed gains and high capability level despite being a mid-size model. • While OpenAI also uses the MoE approach in GPT-4, Google seems to have achieved greater efficiency with the approach. This edge may stem from Google's pioneering work on MoE (Google were the first to publish on MoE, way back in 2017) and their resultant deep in-house expertise on the topic. • Training-data quality is also a likely factor in Google's success. What's next? • Google has 10-million-token context-windows in testing. That order-of-magnitude jump would correspond to future Gemini releases being able to handle ~70 novels, 100 hours of audio or 10 hours of video. • If Gemini Pro 1.5 can achieve GPT-4-like capabilities, the Gemini Ultra 1.5 release I imagine is in the works may allow Google to leapfrog OpenAI and reclaim their crown as the world's undisputed A.I. champions (unless OpenAI gets GPT-5 out first)! Want access? • Gemini Pro 1.5 is available with a 128k context window through Google AI Studio and (for enterprise customers) through Google Cloud's Vertex AI. • There's a waitlist for access to the million-token version (I had access through the early-tester program). Check out today's episode (#762) for more detail on all of the above (including Gemini 1.5 Pro access/waitlist links). The Super Data Science Podcast is available on all major podcasting platforms and a video version is on YouTube. #superdatascience #machinelearning #ai #llms #geminipro #geminiultra

Really excited about Gemini 1.5 Pro for... you guessed it! #accessibility! Here's me checking if a code snippet meets a specific success criteria, and the response nails it! It's helpful, accurate, and to the point. It's also easy to read (succinct with a clear structure). Overall great, and a big improvement over the other models I've used so far. Bonus: with a 1 million token context window, I'm able to add the entire set of Web Content Accessibility Guidelines with failures, techniques, and understanding documents, and getting improved responses as a result. This will likely help with more open ended prompts, like "Which success criteria does this fail for?" Lot's more to discover there. Super excited for this to roll out to the community. In the meantime, let me know if you have ideas you want me to try! (HT Leonie Watson's post on TetraLogical blog for prompt inspiration) --- Prompt (including all of W3C WCAG): Does this meet WCAG 2.2 Level AA SC 2.5.3, which states: For user interface components with labels that include text or images of text, the name contains the text that is presented visually. <p>It was reported today that a dog ate my homework. <a href="news.html" aria-label="Dog ate my homework">Read more...</a> </p> --- Output: No, the provided code snippet does not meet WCAG 2.2 Level AA SC 2.5.3. Here's why: 1. The link contains text ("Read more...") that is presented visually.  2. However, the `aria-label` attribute overrides the visible text and sets the accessible name to "News". This mismatch between the visual label and the accessible name violates SC 2.5.3. To fix this, you have two options: Option 1: Remove the `aria-label` attribute. This will allow the accessible name to default to the visible text "Read more...". Option 2: Modify the `aria-label` attribute to include the visible text. For example, you could use `aria-label="Read more about news"`. This ensures that the accessible name contains the text presented visually. Remember, the goal is to ensure consistency between what sighted users see and what users of assistive technologies like screen readers hear or read. This consistency is crucial for efficient navigation and understanding of the content. --- #WCAG #Gemini #A11y #AI Image shows a screenshot of Google AI Studio showing all of WCAG loaded into the prompt!

Re: Gemini 1.5’s 1M token window, I saw Mike Kaput test it on the Marketing AI Podcast with Paul Roetzer using a 500-page (possibly boring) government document. It worked so well I needed to try it. This week Gemini expanded its capabilities to have it ingest video, so I started thinking of ways to experiment. My test: summarizing a movie from the Public Domain - the Night of the Living Dead.   First, I could only get about 40 minutes of the movie in – a little over 715,000 tokens. Gemini 1.5 Pro seemed to balk at higher amounts even though the token window should have been able to handle it.  Still, it did an excellent job of summarizing the video. Really excellent.  The use cases:  1. If you have a large repository of videos from webinars, you could summarize them for better engagement and more engagement.  2. If you are producing video content today, this gives you another and better path for doing summaries. Sure, recorders can get a transcript and build a summary from the transcript, but so far, I’ve found these summaries to be just okay.  The irony is that Gemini will let you do this with a video you upload, but not by pointing it at a YouTube video. Once this happens, think of the applications.    Also, I think Gemini 1.5 Pro seems to handle transcript summaries better than the other LLMs even though the token count is low.