@coreycollins/nightcrawler
scraping streams for puppeteer
Last updated 8 months ago by coreycollins .
ISC · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install @coreycollins/nightcrawler 
SYNC missed versions from official npm registry.

Nightcrawler

CircleCI

Nightcrawler is a node library that makes it easy to stream scraping requests to Puppeteer using native NodeJS streams.

Getting Started

Install

  npm i @coreycollins/nightcrawler

Important: Nightcrawler has a dependency to puppeteer-core. This allows for a custom executable to be used if desired. If you would like to use the chrome executable bundled with puppeteer, please install it like so:

npm i puppeteer

Usage

const Nightcrawler = require('@coreycollins/nightcrawler')
const JSONStream = require('JSONStream')

let nc = new Nightcrawler()
let qStream = nc.createStream()

// Pipe to standard out
qStream.pipe(JSONStream.stringify(false)).pipe(process.stdout)

let q = nc
  .get('http://example.com')
  .waitFor('body')
  .groupBy('body > div')
  .select({ title: nc.$('p') })

qStream.write(q)

qStream.end()

Current Tags

  • 0.0.15                                ...           latest (8 months ago)

15 Versions

  • 0.0.15                                ...           8 months ago
  • 0.0.14                                ...           8 months ago
  • 0.0.13                                ...           8 months ago
  • 0.0.12                                ...           8 months ago
  • 0.0.11                                ...           8 months ago
  • 0.0.10                                ...           8 months ago
  • 0.0.9                                ...           8 months ago
  • 0.0.8                                ...           8 months ago
  • 0.0.7                                ...           a year ago
  • 0.0.6                                ...           a year ago
  • 0.0.5                                ...           a year ago
  • 0.0.4                                ...           a year ago
  • 0.0.3                                ...           a year ago
  • 0.0.2                                ...           a year ago
  • 0.0.1                                ...           a year ago
Maintainers (1)
Downloads
Today 15
This Week 15
This Month 15
Last Day 0
Last Week 0
Last Month 15
Dependencies (4)
Dev Dependencies (5)
Dependents (0)
None

Copyright 2014 - 2016 © taobao.org |