9gag
Download 9GAG posts and comments
Last updated 2 months ago by ruial .
MIT · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install 9gag 
SYNC missed versions from official npm registry.

9gag

A Node.js package to download 9GAG posts and comments.

It can be used as a module to scrap memes or as a command line application to save data to a local folder and generate an HTML file for offline use.

Node.js 8 or above required due to Async/Await usage.

Install

npm install 9gag

Since Puppeteer is needed to handle Cloudflare, check this troubleshooting page to setup required dependencies.

Examples

CLI

9gag <post_count> [<folder> default:output] [<section> default:hot] [<comment_count> default:0]
9gag 10
9gag 20 output trending 3

Testing

# unit
npm run test:unit
# unit + integration
npm test

Scrap posts and comments with images

const NineGag = require('9gag');
const HttpClient = NineGag.HttpClient;
const Scraper = NineGag.Scraper;
const Downloader = NineGag.Downloader;

async function memes() {
    try {
        const httpClient = new HttpClient();
        await httpClient.init();
        const scraper = new Scraper(httpClient, 10, 'hot', 3);
        const posts = await scraper.scrap();
        await httpClient.close();
        posts.forEach(p => console.log(`${p.title} -> ${p.content} -> ${p.comments.map(c => c.content)[0]}`));
        await new Downloader('output').downloadPosts(posts);
    }
    catch (err) {
        console.error(err);
    }
}

memes();

HTML output

demo

Current Tags

  • 3.2.1                                ...           latest (2 months ago)

9 Versions

  • 3.2.1                                ...           2 months ago
  • 3.2.0                                ...           2 months ago
  • 3.1.0                                ...           2 months ago
  • 3.0.0                                ...           2 months ago
  • 2.0.1                                ...           2 years ago
  • 2.0.0                                ...           3 years ago
  • 1.0.2                                ...           3 years ago
  • 1.0.1                                ...           3 years ago
  • 1.0.0                                ...           3 years ago
Maintainers (1)
Downloads
Today 0
This Week 0
This Month 0
Last Day 0
Last Week 10
Last Month 32
Dependencies (4)
Dev Dependencies (3)
Dependents (1)

Copyright 2014 - 2017 © taobao.org |