@adobe/asset-compute-commons
Common utilities needed by all Adobe Asset Compute serverless actions
Last updated 3 months ago by jdelbick .
Apache-2.0 · Repository · Bugs · Original npm · Tarball · package.json
$ cnpm install @adobe/asset-compute-commons 
SYNC missed versions from official npm registry.

Version

asset-compute-commons

Common utilities needed by all Asset Compute serverless actions

Installation

npm install @adobe/asset-compute-commons

API Details

Asset Compute commons contains many different libraries that take care of common functionalities for workers like Adobe IO Events, New Relic metrics, and custom errors.

Asset Compute Events

Asset Compute event handler for sending Adobe IO Events

Constructor parameters

AssetComputeEvents constructor supports the following mandatory parameters:

  • params: object must contain authorization parameters:
    • auth.accessToken
    • orgId or auth.orgId
    • clientId or auth.clientId

AssetComputeEvents constructor supports the following optional parameters:

  • retry: retry options from node-fetch-retry for sending IO events. (defaults to node-fetch-retry default options)

Sending Events

The sendEvent method sends an Adobe IO Event.

AssetComputeEvents.sendEvent method takes in two required parameters: type and payload.

  • type: String containing the Adobe IO Event type (either rendition_created or rendition_failed)
  • payload: Object containing the event payload

see Asset Compute API Document: Asynchronous Events for more on this topic

Examples

Example with custom retry options:

const { AssetComputeEvents } = require('@adobe/asset-compute-commons');

const params = {
	auth: {
		accessToken: '12345',
		orgId: 'orgId',
		clientId: 'clientId`
	}
}
const retry = {
	retryMaxDuration: 1000 // in ms
}
const eventsHandler  = new AssetComputeEvents(params, retry);

await eventsHandler.sendEvent('rendition_created', {
            rendition: {
				name: 'rendition.jpg',
				fmt: 'jpg'
			}
      });

Example with a rendition_failed event type:

const { AssetComputeEvents } = require('@adobe/asset-compute-commons');

const params = {
	auth: {
		accessToken: '12345',
		orgId: 'orgId',
		clientId: 'clientId`
	}
}
const eventsHandler  = new AssetComputeEvents(params);

await eventsHandler.sendEvent('rendition_failed', {
            rendition: {
				name: 'rendition.21',
				fmt: '21'
			},
			errorReason: 'RenditionFormatUnsupported',
			errorMessage: 'Rendition format `21` is not supported'
     });

Asset Compute Metrics

Asset Compute metrics handler for sending New Relic metrics. It uses a node js agent node-openwhisk-newrelic to send metrics to New Relic.

Constructor parameters

AssetComputeMetrics constructor supports the following mandatory parameters:

  • params: Object must contain New Relic metrics parameters:
    • newRelicEventsURL: New Relic Insights Events url (should be of the form: https://insights-collector.newrelic.com/v1/accounts/<YOUR_ACOUNT_ID>/events)
    • newRelicApiKey: New Relic Insights API key (see the "Register an Insert API key" section here)

AssetComputeMetrics constructor supports the following optional parameters:

Simple example

Initiates metrics handler, sends metrics and stops metrics agent:

const { AssetComputeMetrics } = require('@adobe/asset-compute-commons');

const params = {
	newRelicEventsURL: 'https://insights-collector.newrelic.com/v1/accounts/<YOUR_ACOUNT_ID>/events',
	newRelicApiKey: 'YOUR_API_KEY',
	// ... rest of the Asset Compute parameters

}
const metricsHandler = new AssetComputeMetrics(params);
const metrics = {
	downloadDuration: 200,
	size: 3000
}
await metricsHandler.sendMetrics('rendition', metrics);
await metricsHandler.activationFinished(); // see https://github.com/adobe/node-openwhisk-newrelic#usage for information about `activationFinished()`

Other additional functions

Adding custom metrics:

metricsHandler.add({
	uuid: '12345',
	count: 2
});

Get current state of metrics:

console.log(metricsHandler.get()); // should print out metrics added in `metricsHandler.add()`

Sending error metrics (sends metrics with event type error):

const location = 'upload_worker_flite';
const message = 'Invalid file format';
const metrics = {
	downloadDuration: 200,
	size: 3000
}
await metricsHandler.sendErrorMetrics(location, message, metrics);

Sending client error metrics (sends metrics with event type client_error):

const location = 'upload_worker_flite';
const message = 'Invalid file format';
const metrics = {
	downloadDuration: 200,
	size: 3000
}
await metricsHandler.sendClientErrorMetrics(location, message, metrics);

Sending error metrics by exceptions thrown (sends metrics with event type client_error or error depending on the error):

foo() {
	try {
		console.log('hello!');
	} catch (error) {
		metricsHandler.handleError(error, {
			location: "mylocation",
			message: "something failed",
			metrics: {}
		});
	}
}

Asset Compute Errors

There are several custom errors used in Asset Compute workers:

Error types

Type Description Properties
error unexpected errors and exceptions message, date, location
client_error errors caused by client misconfiguration message, date, reason

Error Properties

  • message: error message
  • date: current time in UTC of when the error was thrown
  • location: location where the error took place (only in type error)
  • reason: the reason for the error (only in client errors)
    • must be one of list: "SourceFormatUnsupported", "RenditionFormatUnsupported", "SourceUnsupported", "SourceCorrupt", "RenditionTooLarge" }

Custom Errors

Name Description Type
SourceFormatUnsupportedError The source is of an unsupported type. client error
RenditionFormatUnsupportedError The requested format is unsupported. client error
SourceUnsupportedError The specific source is unsupported even though the type is supported. client error
SourceCorruptError The source data is corrupt. Includes empty files. client error
RenditionTooLargeError The rendition could not be uploaded using the pre-signed URL(s) provided in target. The actual rendition size is available as metadata in repo:size and can be used by the client to re-process this rendition with the right number of pre-signed URLs. client error
ArgumentError Wrong arguments (type, structure, etc.) error
GenericError Any other error. error

Examples

Generic error downloading the source file in action worker-pie:

const { GenericError } = require('@adobe/asset-compute-commons');

const message = 'Error while downloading source file!'
const location = 'download_worker_pie'
throw new GenericError(message, location);

Rendition format is unsupported:

const { RenditionFormatUnsupportedError } = require('@adobe/asset-compute-commons');

const message = 'Rendition format `sdfg` is not supported'
throw new RenditionFormatUnsupportedError(message);

Asset Compute Log Utils

Utilities for removing sensitive information from Asset Compute worker logs

Examples

Redacting access token from logs:

const { AssetComputeLogUtils } = require('@adobe/asset-compute-commons');

params = {
	accessToken: '123453467',
	fmt: 200
}
console.log("Asset Compute parameters:", AssetComputeLogUtils.redactUrl(params)); // should replace access token with "[...REDACTED...]"

Prints out exact same logs using AssetComputeLogUtils.log method:

const { AssetComputeLogUtils } = require('@adobe/asset-compute-commons');

params = {
	accessToken: '123453467',
	fmt: 200
}
AssetComputeLogUtils.log(params, "Asset Compute parameters"); // should replace access token with "[...REDACTED...]"

Apache OpenWhisk Action Name

A simple way to get information about the Apache OpenWhisk action.

Properties (all default to empty strings)

  • name: base Apache OpenWhisk action name
  • package: Apache OpenWhisk package name
  • namespace: Apache OpenWhisk namespace
  • fullname: full Apache OpenWhisk action name, including namespace, package and action name from environment variable __OW_ACTION_NAME

Examples:

const actionInfo = new OpenwhiskActionName();
console.log(actionInfo.name) // prints out something like `worker-pie`
console.log(actionInfo.package) // prints package name, ex: `experimental`
console.log(actionInfo.namespace) // prints namespace, ex: `stage`
console.log(actionInfo.fullname) // prints full name, ex: /stage/experimental/worker-pie

Asynchronous Events

When processing is finished, or if errors occurred, events are sent through Adobe I/O Events. Events are JSON objects in the event field of the objects in the events array of the jorunal response.

The I/O event type for all events of the Asset Compute service is asset_compute. The journal will be automatically subscribed to this event type only, and consumers are not expected to filter based on the i/o event type.

The service specific event types are available in the type property of the event.

Event Types

Event Description
rendition_created Sent for each successfully processed and uploaded rendition.
rendition_failed Sent for each rendition that failed to process or upload.

Event Attributes

Attribute Type Event Description
date string * Timestamp when event was sent in simplified extended ISO-8601 format (as defined by Javascript Date.toISOString()).
requestId string * The request id of the original request to /process, same as X-Request-Id header.
source object * The source of the /process request.
userData object * The userData of the /process request if set.
rendition object rendition_* The corresponding rendition object passed in /process.
metadata object rendition_created The metadata properties of the rendition.
errorReason string rendition_* Rendition failure reason if any.
errorMessage string rendition_* Text giving more detail about the rendition failure if any.

Metadata

Property Description
tiff:ImageWidth The width of the rendition in pixels. Will not be present if the rendition is not an image.
tiff:ImageLength The length of the rendition in pixels. Will not be present if the rendition is not an image.
repo:size The size of the rendition in bytes.
repo:sha1 The sha1 digest of the rendition.

Error Reasons

Reason Description
SourceFormatUnsupported The source is of an unsupported type.
RenditionFormatUnsupported The requested format is unsupported.
SourceUnsupported The specific source is unsupported even though the type is supported.
SourceCorrupt The source data is corrupt. Includes empty files.
RenditionTooLarge The rendition could not be uploaded using the pre-signed URL(s) provided in target. The actual rendition size is available as metadata in repo:size and can be used by the client to re-process this rendition with the right number of pre-signed URLs.
GenericError Any other error.

Contributing

Contributions are welcomed! Read the Contributing Guide for more information.

Licensing

This project is licensed under the Apache V2 License. See LICENSE for more information.

Current Tags

  • 1.1.2                                ...           latest (3 months ago)

5 Versions

  • 1.1.2                                ...           3 months ago
  • 1.1.1                                ...           4 months ago
  • 1.1.0                                ...           5 months ago
  • 1.0.1                                ...           7 months ago
  • 1.0.0                                ...           7 months ago

Copyright 2014 - 2016 © taobao.org |