@compass-marketing/puppet-stream
scraping streams for puppeteer
Last updated 5 months ago by compass-marketing .
ISC · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install @compass-marketing/puppet-stream 
SYNC missed versions from official npm registry.

Nightcrawler

CircleCI

Nightcrawler is a node library that makes it easy to stream scraping requests to Puppeteer using native NodeJS streams.

Getting Started

Install

  npm i @coreycollins/nightcrawler

Important: Nightcrawler has a dependency to puppeteer-core. This allows for a custom executable to be used if desired. If you would like to use the chrome executable bundled with puppeteer, please install it like so:

npm i puppeteer

Usage

const Nightcrawler = require('@coreycollins/nightcrawler')
const JSONStream = require('JSONStream')

let nc = new Nightcrawler()
let qStream = nc.createStream()

// Pipe to standard out
qStream.pipe(JSONStream.stringify(false)).pipe(process.stdout)

let q = nc
  .get('http://example.com')
  .waitFor('body')
  .groupBy('body > div')
  .select({ title: nc.$('p') })

qStream.write(q)

qStream.end()

Current Tags

  • 0.0.15                                ...           latest (5 months ago)

1 Versions

  • 0.0.15                                ...           5 months ago
Maintainers (1)
Downloads
Today 1
This Week 1
This Month 1
Last Day 0
Last Week 0
Last Month 1
Dependencies (4)
Dev Dependencies (5)
Dependents (0)
None

Copyright 2014 - 2017 © taobao.org |