@moonset/executor
The Moonset Executor
Last updated 12 days ago by down-up-down .
MIT · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install @moonset/executor 
SYNC missed versions from official npm registry.

Moonset EMR Executor

EMR Executor can run hive & spark tasks in a brand new account without any setting up, either in schedule mode or adhoc mode. It uses AWS CDK to setup the infrastructure. Pick a brand new aws account, and try the following command.

# config the credentials
npx moonset config

# run a job
npx moonset deploy --job '{
    "input": [
      {"glue": { "db": "foo", "table": "apple"}}
    ],
    "task": [
      {"hive": {"sqlFile": "s3://foo/hive.sql"}}
    ],
    "output": [
      {"glue": { "db": "foo", "table": "orange"}}
    ]
}' 

Compose time: 0.03 seconds
Synthesis time: 0.113 seconds
Deploy time: 309.98 seconds
Invoke time: 1.308 seconds
Total time: 311.431 seconds

You will see a step function is setup and an EMR is invoked as following.

You can set environment LOG_LEVEL=debug to see more verbose error.

Current Tags

  • 0.0.20                                ...           latest (12 days ago)

21 Versions

  • 0.0.20                                ...           12 days ago
  • 0.0.19                                ...           13 days ago
  • 0.0.18                                ...           13 days ago
  • 0.0.17                                ...           13 days ago
  • 0.0.16                                ...           a month ago
  • 0.0.15                                ...           a month ago
  • 0.0.14                                ...           a month ago
  • 0.0.13                                ...           a month ago
  • 0.0.12                                ...           2 months ago
  • 0.0.11                                ...           3 months ago
  • 0.0.10                                ...           3 months ago
  • 0.0.9                                ...           3 months ago
  • 0.0.8                                ...           3 months ago
  • 0.0.7                                ...           3 months ago
  • 0.0.6                                ...           3 months ago
  • 0.0.5                                ...           3 months ago
  • 0.0.4                                ...           3 months ago
  • 0.0.3                                ...           3 months ago
  • 0.0.2                                ...           3 months ago
  • 0.0.1                                ...           3 months ago
  • 0.0.0                                ...           3 months ago
Maintainers (1)
Downloads
Today 0
This Week 0
This Month 0
Last Day 0
Last Week 1
Last Month 61
Dependencies (16)
Dev Dependencies (7)

Copyright 2014 - 2016 © taobao.org |