一.Description

CDF(Cloudera DataFlow)源自HDF（Hortonworks DataFlow），HDF是一个可缩放的实时流分析平台，它可以提取、组织和分析数据，以获得关键洞察和即时的可操作情报。
CDF是由CFM（Cloudera Flow Manager）、CEM（Cloudera Edge Manager）和CSP（Cloudera Stream Processing）3部分组成，其中，CEM包括EFM（Edge Flow Manager）和2类MiNiFi(Java Agent、C++ Agent)。

二.Install CFM

1.install nifi-registry

a).download nifi-registry

b).configration

vi nifi-registry-0.3.0.1.0.0.0-90/conf/nifi-registry.properties

# web properties #
nifi.registry.web.war.directory=./lib
nifi.registry.web.http.host=hostname
nifi.registry.web.http.port=18080
nifi.registry.web.https.host=
nifi.registry.web.https.port=
nifi.registry.web.jetty.working.directory=./work/jetty
nifi.registry.web.jetty.threads=200

c).start service

nifi-registry-0.3.0.1.0.0.0-90/bin/nifi-registry.sh start

d).view log

tail -400f logs/nifi-registry-app.log

e).access page

http://hostname:18080/nifi-registry

2.install EFM

a).download EFM

b).configration

efm-1.0.0.1.0.0.0-54/conf/efm.properties

# Web Server Properties
#  address: the hostname or ip address of the interface to bind to; to bind to all, use 0.0.0.0
efm.server.address=hostname
efm.server.port=10080
efm.server.servlet.contextPath=/efm

# NiFi Registry Properties
#  url: the base URL of a NiFi Registry instance
#  bucket: Only set one of bucketId OR bucketName
#  flowRefreshInterval: specify value and units (d=days, h=hours, m=minutes, s=seconds, ms=milliseconds)
efm.nifi.registry.enabled=true
efm.nifi.registry.url=http://hostname:18080
efm.nifi.registry.bucketId=
efm.nifi.registry.bucketName=minifi
efm.nifi.registry.flowRefreshInterval=60s

c).start service

efm-1.0.0.1.0.0.0-54/bin/efm.sh start

d).view log

tail -400f logs/efm-app.log

e).access page

http://localhost:10080/efm/ui

3.install java agent

a).download EFM

b).configration

vi minifi-0.6.0.1.0.0.0-54/conf/bootstrap.conf

# MiNiFi Command & Control Configuration
# C2 Properties
# Enabling C2 Uncomment each of the following options
# define those with missing options
nifi.c2.enable=true
## define protocol parameters
nifi.c2.rest.url=http://hostname:10080/efm/api/c2-protocol/heartbeat
nifi.c2.rest.url.ack=http://hostname:10080/efm/api/c2-protocol/acknowledge
## heartbeat in milliseconds.  defaults to once a second
nifi.c2.agent.heartbeat.period=10000
## define parameters about your agent
nifi.c2.agent.class=java-agent-100
# Optional.  Defaults to a hardware based unique identifier
nifi.c2.agent.identifier=100

c).start service

minifi-0.6.0.1.0.0.0-54/bin/minifi.sh start

d).view log

tail -400f logs/minifi-app.log

4.install C++ agent

a).download EFM

b).configration

nifi-minifi-cpp-0.6.0/conf/minifi.properties

## Enabling C2 Uncomment each of the following options
## define those with missing options
nifi.c2.enable=true
## define protocol parameters
## The default is CoAP, if that extension is built.
## Alternatively, you may use RESTSender if http-curl is built
#nifi.c2.agent.protocol.class=CoapProtocol
#nifi.c2.agent.coap.host=
#nifi.c2.agent.coap.port=
nifi.c2.rest.url=http://hostname:10080/efm/api/c2-protocol/heartbeat
nifi.c2.rest.url.ack=http://hostname:10080/efm/api/c2-protocol/acknowledge
#nifi.c2.root.classes=DeviceInfoNode,AgentInformation,FlowInformation
## heartbeat 4 times a second
nifi.c2.agent.heartbeat.period=5000
## define parameters about your agent
nifi.c2.agent.class=c_agent
nifi.c2.agent.identifier=6

c).start service

nifi-minifi-cpp-0.6.0/bin/minifi.sh start

d).view log

tail -400f logs/minifi-app.log

三.Mysql Data

When MySQL is used as a metadata base

1.create database and user

CREATE DATABASE EFM DEFAULT CHARACTER SET utf8;

CREATE USER 'efm'@'%' IDENTIFIED BY '2wsx@WSX';

GRANT ALL PRIVILEGES ON *.* TO 'efm'@'%' WITH GRANT OPTION;

commit;

2.change EFM config

vi efm-1.0.0.1.0.0.0-54/conf/efm.properties

# Database Properties
efm.db.url=jdbc:mysql://hostname:3306/EFM
efm.db.driverClass=com.mysql.jdbc.Driver
efm.db.username=efm
efm.db.password=
efm.db.maxConnections=5
efm.db.sqlDebug=false

3.restart EFM

efm-1.0.0.1.0.0.0-54/bin/efm.sh restart

四.Build C++ Agent

1.check environment

[root@localhost ~]# cmake --version
cmake version 3.12.0

[root@localhost ~]# gcc --version
gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)

[root@localhost ~]# bison --version
bison (GNU Bison) 3.0.4

[root@localhost ~]# flex --version
flex 2.5.37

[root@localhost ~]# patch --version
GNU patch 2.7.1

[root@localhost ~]# autoconf --version
autoconf (GNU Autoconf) 2.69

[root@localhost ~]# automake --version
automake (GNU automake) 1.13.4

[root@localhost ~]# libtool --version
libtool (GNU libtool) 2.4.2