Puppeteer - Headless Chrome Node API
5 likes2,160 views
The document provides an overview of headless browsers, particularly focusing on Puppeteer, a Node.js API for controlling headless Chrome. It details various functionalities such as taking screenshots, generating PDFs, scraping content, and automating interactions, along with example code snippets. Additionally, it highlights potential use cases and mentions other related web automation tools.
![Scraping Content From Websites
1. const puppeteer = require('puppeteer');
2. (async () => {
3. const browser = await puppeteer.launch();
4. const page = await browser.newPage();
5. await page.goto('https://www.npmjs.com/search?q=react');
6. const packages = await page.evaluate(() => {
7. var nodes = document.querySelectorAll('.packageName');
8. return [...nodes].map(el => el.textContent);
9. });
10. console.log(packages);
11. browser.close();
12. })();](https://image.slidesharecdn.com/sharing-puppeteer-headlesschromenodeapi-170918082240/85/Puppeteer-Headless-Chrome-Node-API-10-320.jpg)
![Emulating Device Metrics and User Agent
1. const puppeteer = require('puppeteer');
2. const devices = require('puppeteer/DeviceDescriptors');
3. const iphone6 = devices['iPhone 6'];
4. (async () => {
5. const browser = await puppeteer.launch();
6. const page = await browser.newPage();
7. await page.emulate(iphone6);
8. await page.goto('https://www.facebook.com/');
9. await page.screenshot({ path: 'facebook.png' });
10. const host = await page.evaluate(() => location.host);
11. console.log(host); // 'm.facebook.com'
12. browser.close();
13. })();](https://image.slidesharecdn.com/sharing-puppeteer-headlesschromenodeapi-170918082240/85/Puppeteer-Headless-Chrome-Node-API-11-320.jpg)
![Handling Events
1. const puppeteer = require('puppeteer');
2. (async () => {
3. const browser = await puppeteer.launch();
4. const page = await browser.newPage();
5. page.on('console', (...args) => {
6. console.log('[Browser]', ...args);
7. });
8. await page.goto('http://jsbin.com');
9. browser.close();
10. })();](https://image.slidesharecdn.com/sharing-puppeteer-headlesschromenodeapi-170918082240/85/Puppeteer-Headless-Chrome-Node-API-12-320.jpg)
Puppeteer - Headless Chrome Node API
- 1. Puppeteer Headless Chrome Node API Wilson Su
- 2. What Is a Headless Browser? A headless browser is a web browser without a graphical user interface. Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but are executed via a command-line interface or using network communication. – Wiki
- 3. Headless Browser Command Line Features 1. // Taking screenshots 2. $ google-chrome --headless --disable-gpu --screenshot https://example.com/ 3. 4. // Creating a PDF 5. $ google-chrome --headless --disable-gpu --print-to-pdf https://example.com/
- 4. Puppeteer https://github.com/GoogleChrome/puppeteer
- 5. Use Cases ✓ Take screenshots ✓ Generate PDFs ✓ Scrape content ✓ Automate UI testing ✓ Capture a timeline trace ✗ DDoS attack ✗ Fake page loads ✗ Credential stuffing
- 6. Examples 1. Take screenshosts 2. Generate PDFs 3. Automate interactions 4. Scrape content from websites 5. Emulate device metrics and user agent 6. Handle events 7. Create trace files
- 7. Taking Screenshots 1. const puppeteer = require('puppeteer'); 2. (async () => { 3. const browser = await puppeteer.launch(); 4. const page = await browser.newPage(); 5. await page.setViewport({ width: 767, height: 1024 }); 6. await page.goto('https://getbootstrap.com/'); 7. await page.screenshot({ 8. path: 'bootstrap.png', 9. fullPage: true 10. }); 11. browser.close(); 12. })();
- 8. Generating PDFs 1. const puppeteer = require('puppeteer'); 2. (async () => { 3. const browser = await puppeteer.launch(); 4. const page = await browser.newPage(); 5. await page.goto('https://github.com/'); 6. await page.pdf({ path: 'github.pdf', format: 'A4' }); 7. browser.close(); 8. })();
- 9. Automating Interactions 1. const puppeteer = require('puppeteer'); 2. (async () => { 3. const browser = await puppeteer.launch(); 4. const page = await browser.newPage(); 5. await page.goto('https://www.npmjs.com'); 6. await page.focus('#site-search'); 7. await page.type('react'); 8. await page.click('#npm-search > button'); 9. await page.waitForNavigation(); 10. await page.screenshot({ path: 'npmjs.png' }); 11. browser.close(); 12. })();
- 10. Scraping Content From Websites 1. const puppeteer = require('puppeteer'); 2. (async () => { 3. const browser = await puppeteer.launch(); 4. const page = await browser.newPage(); 5. await page.goto('https://www.npmjs.com/search?q=react'); 6. const packages = await page.evaluate(() => { 7. var nodes = document.querySelectorAll('.packageName'); 8. return [...nodes].map(el => el.textContent); 9. }); 10. console.log(packages); 11. browser.close(); 12. })();
- 11. Emulating Device Metrics and User Agent 1. const puppeteer = require('puppeteer'); 2. const devices = require('puppeteer/DeviceDescriptors'); 3. const iphone6 = devices['iPhone 6']; 4. (async () => { 5. const browser = await puppeteer.launch(); 6. const page = await browser.newPage(); 7. await page.emulate(iphone6); 8. await page.goto('https://www.facebook.com/'); 9. await page.screenshot({ path: 'facebook.png' }); 10. const host = await page.evaluate(() => location.host); 11. console.log(host); // 'm.facebook.com' 12. browser.close(); 13. })();
- 12. Handling Events 1. const puppeteer = require('puppeteer'); 2. (async () => { 3. const browser = await puppeteer.launch(); 4. const page = await browser.newPage(); 5. page.on('console', (...args) => { 6. console.log('[Browser]', ...args); 7. }); 8. await page.goto('http://jsbin.com'); 9. browser.close(); 10. })();
- 13. Creating Trace Files 1. const puppeteer = require('puppeteer'); 2. (async () => { 3. const browser = await puppeteer.launch(); 4. const page = await browser.newPage(); 5. await page.tracing.start({ path: 'trace.json' }); 6. await page.goto('https://webpack.js.org/', { 7. waitUntil: 'networkidle' 8. }); 9. await page.tracing.stop(); 10. browser.close(); 11. })();
- 15. Further Information ● Selenium - Web Browser Automation ● WebdriverIO - WebDriver bindings for Node.js ● PhantomJS ● SlimerJS ● Chromeless
- 16. Reference ● Headless browser - Wikipedia ● Getting Started with Headless Chrome | Web | Google Developers ● What Is a Headless Browser, and What's It Good For? ● Jack Histon - Making a Master Puppeteer