![E-Commerce Scraping: Prompt and Output
Prompt:
Extract data in JSON format: sku, name, price, description, sizes, colors
CONTENT: {main_markdown}
Output (product.json):
{
"sku": "ATJ-001",
"name": "Adrienne Trek Jacket",
"price": "$89.99",
"description": "Lightweight, water-resistant jacket...",
"sizes": ["S", "M", "L", "XL"],
"colors": ["Black", "Blue", "Green"]
}
NextGen_Outlier 8](https://image.slidesharecdn.com/webscrapinggemini-250711102523-39afc979/85/Web-Scraping-with-Google-Gemini-2-0-pdf-8-320.jpg)
![Example: Airbnb Reviews
Open Airbnb listing, enable screen sharing in Google AI Studio
Voice command: "Extract all reviews visible on the screen in JSON"
Scroll to load more reviews
Sample Output:
{
"reviews": [
{"name": "Maria", "date": "March 2023", "rating": "5 stars", ...},
{"name": "John", "date": "April 2023", "rating": "4 stars", ...}
]
}
NextGen_Outlier 9](https://image.slidesharecdn.com/webscrapinggemini-250711102523-39afc979/85/Web-Scraping-with-Google-Gemini-2-0-pdf-9-320.jpg)
Web Scraping with Google Gemini 2.0 .pdf
- 1. Web Scraping with Google Gemini 2.0 By - Tamanna NextGen_Outlier 1
- 2. Overview What is Web Scraping? Why Google Gemini 2.0? Setup Workflow Example: E-Commerce Scraping Example: Airbnb Reviews Advanced Features Use Cases Limitations Conclusion NextGen_Outlier 2
- 3. What is Web Scraping? Automatically extracting data from websites Converts unstructured web content into structured formats (e.g., JSON, CSV) Examples: Product prices from e-commerce sites Customer reviews from Airbnb Traditionally requires coding (e.g., BeautifulSoup, Scrapy) NextGen_Outlier 3
- 4. Why Use Google Gemini 2.0? No Coding Required: Use natural language or voice commands Dynamic Websites: Handles JavaScript-heavy pages Structured Output: Returns data in JSON or CSV Cost-Effective: ~$0.075 per million tokens (Gemini 1.5 Flash) Beginner-Friendly: Ideal for marketers, researchers, non-coders NextGen_Outlier 4
- 5. Setting Up Gemini 2.0 1. Access Google AI Studio: https://aistudio.google.com/apikey 2. Generate and secure your API key 3. Enable screen sharing for voice-based scraping (optional) 4. Install Python libraries: import os os.system("pip install requests beautifulsoup4 markdownify google-generativeai") NextGen_Outlier 5
- 6. Web Scraping Workflow Step Tools Used Fetch Webpage requests Parse HTML BeautifulSoup Clean Content markdownify AI Extraction google-generativeai Save Output json , pandas NextGen_Outlier 6
- 7. Example: E-Commerce Scraping Goal: Extract product details (name, price, etc.) from a webpage import requests, google.generativeai as genai from bs4 import BeautifulSoup from markdownify import markdownify genai.configure(api_key="YOUR_GEMINI_API_KEY") url = "https://www.scrapingcourse.com/ecommerce/..." response = requests.get(url) soup = BeautifulSoup(response.content, "html.parser") main_html = str(soup.select_one("#main")) main_markdown = markdownify(main_html) NextGen_Outlier 7
- 8. E-Commerce Scraping: Prompt and Output Prompt: Extract data in JSON format: sku, name, price, description, sizes, colors CONTENT: {main_markdown} Output (product.json): { "sku": "ATJ-001", "name": "Adrienne Trek Jacket", "price": "$89.99", "description": "Lightweight, water-resistant jacket...", "sizes": ["S", "M", "L", "XL"], "colors": ["Black", "Blue", "Green"] } NextGen_Outlier 8
- 9. Example: Airbnb Reviews Open Airbnb listing, enable screen sharing in Google AI Studio Voice command: "Extract all reviews visible on the screen in JSON" Scroll to load more reviews Sample Output: { "reviews": [ {"name": "Maria", "date": "March 2023", "rating": "5 stars", ...}, {"name": "John", "date": "April 2023", "rating": "4 stars", ...} ] } NextGen_Outlier 9
- 10. Advanced Tips CAPTCHAs: Use Crawlbase Smart Proxy Rate Limits: Add delays (e.g., time.sleep(1) ) Dynamic Content: Use Puppeteer or Playwright Cost Optimization: Convert HTML to Markdown NextGen_Outlier 10
- 11. Common Use Cases Use Case Example E-Commerce Scrape Amazon prices Market Research Gather Airbnb reviews Real Estate Extract Zillow listings News Aggregation Scrape news headlines NextGen_Outlier 11
- 12. Limitations to Consider Speed: Slower than traditional parsers Cost: Token usage adds up for large HTML Accuracy: May misinterpret complex layouts Legal: Check robots.txt and terms of service NextGen_Outlier 12
- 13. Conclusion Google Gemini 2.0 simplifies web scraping for all users No coding needed with natural language or voice commands Ideal for e-commerce, research, and more Start today at https://aistudio.google.com NextGen_Outlier 13