puppeteer-walker
puppeteer crawler
Last updated 2 years ago by lrlna .
Apache-2.0 · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install puppeteer-walker 
SYNC missed versions from official npm registry.

puppeteer-walker

npm version build status downloads js-standard-style

A crawler to go through your given site in a headless chrome using puppeteer. Returns an object containing host, current path, and current DOM object

Usage

var Walker = require('puppeteer-walker')

var walker = Walker()

walker.on('end', () => console.log('finished walking'))
walker.on('error', (err) => console.log('error', err))
walker.on('page', async (page) => {
  var title = await page.title()
  console.log(`title: ${title}`)
})

walker.walk('https://avocado.choo.io')

API

walker = PuppeteerWalker()

Create a new walker instance.

walker.on('page', async cb(Page, push))

Listen to a page event. Returns an instance of the puppeteer Page Class. The callback has to be an Async Function.

Use the push(url) method to add more pages into the internal walker queue. This is useful for busting past login forms, and the like.

walker.on('error', cb(err))

Listen to error events.

walker.on('end', cb)

Listen to an end event.

walker.walk(url)

Start walking the URL.

See Also

License

Apache-2.0

Current Tags

  • 1.2.0                                ...           latest (2 years ago)

8 Versions

  • 1.2.0                                ...           2 years ago
  • 1.1.2                                ...           3 years ago
  • 1.1.1                                ...           3 years ago
  • 1.1.0                                ...           3 years ago
  • 1.0.4                                ...           3 years ago
  • 1.0.3                                ...           3 years ago
  • 1.0.2                                ...           3 years ago
  • 1.0.0                                ...           3 years ago
Maintainers (1)
Downloads
Today 0
This Week 0
This Month 1
Last Day 0
Last Week 1
Last Month 7
Dependencies (3)
Dev Dependencies (1)
Dependents (0)
None

Copyright 2014 - 2016 © taobao.org |