@builtioflow/enterprise-gateway
Enterprise gateway connects enterprise applications (on-premise) to Built.io flow.
Last updated 2 years ago by pradeepmishra .
Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install @builtioflow/enterprise-gateway 
SYNC missed versions from official npm registry.

Enterprise Gateway

Enterprise Gateway allows you to access on-premise databases and applications that are normally inaccessible due to security restrictions and firewalls. This gateway establishes a secure connection (or a safe tunnel) between Built.io Flow and your server behind the firewall. As a result, you can easily consume data from your on-premise databases, such as DB2, MySQL, MongoDB or any other on-premise applications, and execute workflows using Built.io Flow.

enterprise-gateway

While using Enterprise Gateway, you do not have to provide any information about your server, such as hostname, port, or any other connection details. Enterprise Gateway uses a unique gateway key to register itself and create a secure connection to Built.io Flow.

Using Enterprise Gateway

prequisites

  • Gateway key (Enterprise Gateway should be enabled for your account. If this is not enabled, contact support.
  • Enterprise Gateway installed on your machine ( npm i @builtioflow/enterprise-gateway )
  • Node.js (4 or above)
  • Redis

Overview of steps

To use Enterprise Gateway to interact with your enterprise services, you need to perform the following steps:

Step 1. Create Enterprise Gateway agent key
Step 2. Install Enterprise Gateway
Step 3. Update Enterprise Gateway config file
Step 4. Start Enterprise Gateway
Step 5. Add new plugins

Detailed steps

Step 1. Create Enterprise Gateway key

To connect Built.io Flow with the gateway on your server, you will need to first create an agent key in the Built.io Flow UI. Here’s how to get the key.

  1. Log in to your Built.io Flow account

  2. Click your profile name located on the top right-hand side corner. From the drop down options, select ‘Globals’.

  3. If Enterprise Gateway is enabled for your account, you will see an option (‘Enterprise Gateway’) on the left-side menu of the Settings panel. Click this option.

  4. Click the ‘+’ icon to create a new enterprise connection agent. Provide a suitable name for this agent and ‘Save’.

  5. Once created, hover on the agent name, and click the ellipses or more options icon (three vertical dots), and select ‘View.

  6. This will display the gateway key for the enterprise agent that you just created. Note down this key as it will be used in the config.json file (in step 2).

Enterprise gateway

Step 2. Install Enterprise Gateway

Enterprise gateway requires Redis as its dependency. Read more about Redis here. Download and install Redis.

Install Enterprise Gateway using ( npm i @builtioflow/enterprise-gateway )

After installing the Enterprise Gateway package, install the required dependencies of specific plugin using the following command.

npm run install <inbuilt-plugin> // example npm run install mysql

List of inbuilt plugins.

  • file
  • http
  • mongodb
  • msaccess
  • mssql
  • mysql
  • oracle
  • spreadsheet
  • unix
  • saphanadb

Step 3. Update the Enterprise Gateway config file (./config.json)

Open the ‘config’ file that is located under at the root level of your Enterprise Gateway.

First, specify the Enterprise Gateway key that you created in step 1.


"gateway_key": "PASTE YOUR EG KEY HERE"

Under ‘application’, adding ‘workers’ would add Enterprise Gateway instances by the corresponding number. The more the number of workers, the better is the performance of your Enterprise Gateway.


"application": {
      "workers": 4
}

Specify the Enterprise Gateway key that you created in step 1.

Then, under ‘Redis’, specify the port information of the locally installed Redis.


"redis": {
        "sockets": [
            {
                 "host": "127.0.0.1",
                 "port": 26379
             }
        ]

Specify the triggers that should be active. ‘Timeout’ should be in milliseconds, and the ‘interval’ is the time interval (in milliseconds) at which the triggers would run.


 "triggers": {
    "active": false,
    "timeout": 120000,
    "interval": 30000,
    "events": [
        {
           "plugin": "mongodb",
           "event": "new_row",
           "connection": "mongodb1",
           "webhook_url": "PASTE FLOW WEBHOOK URL HERE",
           "input": {
               "collection": "test"
            }
        }
   ]
}

Under events, specify the details of the trigger event. Connection should be the ID of the connection that you have created (see below). Webhook URL is the URl of the workflow that you want to trigger when the trigger event happens. Lastly, you can provide any input data for the workflow.

You can create multiple connections for multiple systems. You need to provide an ID for each connection (which will be used in triggers as given in the above step), and other connection details, such as username, password, port, hostname, and so on. These details vary depending on the system that you are connecting.

Step 4. Start Enterprise Gateway

To start Enterprise Gateway, first run the Redis server:

redis-server

Then, use the following command to start Enterprise Gateway:

npm run start

This will start Enterprise Gateway on your machine.

Here are some other helpful commands for your Enterprise Gateway application:


npm run restart  // To restart Enterprise Gateway
npm run stop  // To stop Enterprise Gateway 
npm run delete  // To delete the stopped instances of older running Enterprise Gateway
npm run list  // To see the status of the instances of Enterprise Gateway
npm run actionlogs  // To start monitoring action logs
npm run applogs  // To start monitoring master app logs
npm run triggerlogs  // To start monitoring trigger logs

Step 5: Add New Plugins

Plugins are ways to connect to your on-prem systems and communicate with them by using custom actions. The Enterprise Connector action houses all the plugins that you add in Enterprise Gateway. Each plugin represents and connects to an on-prem system (such as MySQL). This plugin further has actions, which are tasks (such as create new row) that you wish to perform in the defined on-prem system.

So, once Enterprise Gateway is installed and enabled for your account, you need to create new plugins, and actions within each plugin. For example, to connect to Mongodb, you need to first create a new ‘Mongodb’ plugin in Enterprise Gateway (steps given below). Then, you need to add a ‘new row’ action inside this plugin (steps given below).

Here, we will learn how to create actions and triggers for your plugins.

Actions

An action, within a plugin, is a task that you wish to perform in your on-premise app using Enterprise Gateway. For example, create object in Mongodb.

Let’s quickly look at the steps for adding a new plugin and a new action within it. (Scroll down for triggers)

  • Create a new folder inside the ‘/plugins’ directory. The folder name should be the name of the service you are adding the plugin for. For e.g., /plugins/mongodb

  • Inside this new folder, add another folder called ‘actions’.

  • Add a new index.js file, and copy the code given below. This file contains all the actual working of the action. You can, therefore, change the code as per your requirement.

Example code for /plugins/mongodb/actions/index.js file:


module.exports = {
    find : find
}
function find () {
  //specify Json input schema to draw input form in Enterprise connector action to get input from user
  this.input = {
    'type': 'object',
    'properties': {
    }
  }
  // specify Json schema to define the structure of output this specific action will return to flow
  this.output = {
    'type': 'object',
    'properties': {
    }
  }
  this.execute = function (input, output) {
    // input contains all information filled by user
    //var connection = this.getConnectionById('mongodb1')
    // you can get connection data from config.json using getConnectionById helper function.
    // your action's code goes here
    // output is callback function which accepts arguments like this
    // output(error, response)
    output(null, { success : true})
  }
}

Once you add an action, restart your Enterprise Gateway by using the ‘npm run restart’ command. Then, login to your Built.io Enterprise account and configure a new workflow. Drag and drop the Enterprise Connector action (read more about it here) onto the canvas. Double click on the Enterprise Connector action and select the plugin you just configured, listed under the 'Plugins' field. You will now see the actions associated with that plugin under the 'Action' drop down field. Select the required action and use it like any other action of your workflow to perform the task within your on-prem system.

EG-example.png

Triggers

A trigger is some event that fires off a workflow. Within Enterprise Gateway, you can create custom triggers that fires off a workflow when something happens in your local server or on-premise app, such as new row added in your MongoDB database. Within the trigger, you can specify which workflow to be executed in case the trigger event happens.

This helps you listen to the events of your on-prem system, and accordingly perform tasks automatically in any web application or other on-prem system.

To use a trigger, you need to first create a trigger using the steps given below. Then, you need to define the list of active triggers in the ./config.json file (as explained in step 3 above).

Here are the steps for adding a new plugin and a new trigger within it. (Scroll down for triggers)

  • Create a new folder inside the ‘/plugins’ directory. The folder name should be the name of the service you are adding the plugin for. For e.g., /plugins/mongodb. Ignore this steps if you have already created a folder in the above step.

  • Inside this new folder, add another folder called ‘triggers’.

  • Add a new index.js file, and copy the code given below. This file contains all the actual working of the trigger. You can, therefore, change the code as per your requirement.

Example code for /plugins/mongodb/triggers/index.js file:


module.exports = {
    new_row : newRow
}
function newRow () {
  this.input = {
    'type': 'object',
    'properties': {
    }
  }
  this.output = {
    'type': 'object',
    'properties': {
    }
  }
this.activate = function(input, output){
  var that = this
  var mongo = require('mongoose');
  var connection = this.getConnectionById('mongodb1')
  // you can get connection data from Enterprise Gateway config using getConnectionById helper function.
  app.logger.info('Trigger activate function')
  mongo.connect(connection, function(err, db){
    if(err){
      return output(err)
    }
    var collection = db.collection(data.collection);
    collection.find({}, {'created_at': 1}).toArray(function(err, docs) {
      if (docs && docs.length) {
        that.setMeta({'last_sync_token': {'created_at': _.last(docs).created_at}})
      }else{
        that.setMeta({'last_sync_token': null }) 
      }
      db.close()
      output()     
    });
  })
}
  this.execute = function (input, output) {
    var that = this;
    //app.logger.info('execute function called')
    var connection = this.getConnectionById('mongodb1')
    // you can get connection data from Enterprise Gateway config using getConnectionById helper function.
    mongo.connect(connection, function(err, db){
      //app.logger.info('error came',err)
      if(err){
        return output(err)
      }
      var query = null
      if(that.meta && that.meta.last_sync_token && that.meta.last_sync_token.created_at){
        var lastSyncToken = that.meta.last_sync_token
        query = {'created_at': {'$gt': new Date(lastSyncToken.created_at)}} 
      }else{
        query = { }
      }
      var collection = db.collection(data.collection);
      //app.logger.info('query is', query)
      collection.find(query, {'created_at': 1}).toArray(function(err, docs) {
        //app.logger.info('pk lets c', err,docs)
        if (docs && docs.length) {
          that.setMeta({'last_sync_token': {'created_at': _.last(docs).created_at}})
        }     
        output(err, docs) 
        db.close()    
      });
    })
  }
}

Once you add a trigger, you need to specify the list of active triggers in the ./config.json file. To do this, edit the ./config.json file, and copy the snippet given below. Make changes to the code as per your preference.

Set ‘active’ to ‘true’ to activate the trigger. Specify trigger ‘timeout’ in milliseconds. ‘Interval’ is the time interval (in milliseconds) at which the event keeps triggering. Under events, specify the ‘plugin’ and ‘event’ name, and the ‘connection’ it should use to run. Lastly, enter the webhook URL of the workflow that should be executed when the event is triggered.


"triggers": {
  "active": false,
  "timeout": 120000,
  "interval": 30000,
  "events": [
    {
      "plugin": "mongodb",
      "event": "New Row",
      "connection": "mongodb1",
      "webhook_url": "Paste flow webhook url here run it",
      "input": {
        "collection": "test"
      }
    }
  ]
}

After making changes to the ./config.json file, you need to restart the Enterprise Gateway. Now, whenever the defined event happens in the defined on-prem system, the trigger will fire off the specified workflow through its webhook URL.

VM Sizing Requirements

As Enterprise Gateway is backed by Job Queue to serve the user requests, it requires minimal resources (for Redis and Node) to get it running. Apart from this, it depends on how many workers you configure (through config) to run Enterprise Gateway as a cluster to fully utilize your CPU cores, so that parallel actions/tasks can be run by the workers simultaneously. Typically Enterprise Gateway can run smoothly on 1GB VM with a single worker. So, if you want to run four workers of Enterprise Gateway on your machine, you would require minimum 4 GB VM for it.

Current Tags

  • 1.1.10                                ...           latest (2 years ago)

11 Versions

  • 1.1.10                                ...           2 years ago
  • 1.1.9                                ...           2 years ago
  • 1.1.8                                ...           2 years ago
  • 1.1.7                                ...           2 years ago
  • 1.1.6                                ...           2 years ago
  • 1.1.5                                ...           2 years ago
  • 1.1.4                                ...           2 years ago
  • 1.1.3                                ...           2 years ago
  • 1.1.2                                ...           2 years ago
  • 1.1.1                                ...           2 years ago
  • 1.1.0                                ...           2 years ago
Downloads
Today 0
This Week 11
This Month 11
Last Day 0
Last Week 1
Last Month 12
Dependencies (11)
Dev Dependencies (13)
Dependents (0)
None

Copyright 2014 - 2017 © taobao.org |