E-MapReduce在emr-2.0.0版本开始,在集群中会集成进phoenix-4.7.0,用户在创建集群的软件列表中需要勾选HBase,集群创建后即可使用phoenix。
创建集群
Phoenix使用HBase作为底层存储,所以创建集群时需要勾选HBase软件,如下:
集群空闲后,可以从集群的详情页面查看Zookeeper节点,如下:
phoenix使用上面Zookeeper节点进行相关操作。
Phoenix使用示例
1.示例一
执行:
sudo su hadoop
performance.py localhost 1000
输出(截取部分):
CSV Upsert complete. 1000 rows upserted
Time: 1.298 sec(s)
COUNT(1)
----------------------------------------
2000
Time: 0.437 sec(s)
HO
--
CS
EU
NA
Time: 0.2 sec(s)
DOMAIN
----------------------------------------
Apple.com
Google.com
Salesforce.com
Time: 0.079 sec(s)
DAY
-----------------------
2016-06-13 00:00:00.000
Time: 0.076 sec(s)
COUNT(1)
----------------------------------------
45
Time: 0.068 sec(s)
2.示例二
执行:
sudo su hadoop
cd ~
echo '100,Jack,Doe' >>example.csv
echo '200,Tony,Poppins' >>example.csv
sqlline.py localhost
0: jdbc:phoenix:localhost> CREATE TABLE example (
my_pk bigint not null,
m.first_name varchar(50),
m.last_name varchar(50)
CONSTRAINT pk PRIMARY KEY (my_pk));
psql.py -t EXAMPLE localhost example.csv
验证:
sqlline.py localhost
0: jdbc:phoenix:localhost> select * from example;
+--------+-------------+------------+
| MY_PK | FIRST_NAME | LAST_NAME |
+--------+-------------+------------+
| 100 | Jack | Doe |
| 200 | Tony | Poppins |
+--------+-------------+------------+
3.示例三
执行:
sudo su hadoop
cd ~
echo '100,Jack,Doe' >>example.csv
echo '200,Tony,Poppins' >>example.csv
hadoop dfs -put example.csv /
sqlline.py localhost
0: jdbc:phoenix:localhost> CREATE TABLE example (
my_pk bigint not null,
m.first_name varchar(50),
m.last_name varchar(50)
CONSTRAINT pk PRIMARY KEY (my_pk));
hadoop jar /opt/apps/phoenix-4.7.0-HBase-1.1-bin/phoenix-4.7.0-HBase-1.1-client.jar org.apache.phoenix.mapreduce.CsvBulkLoadTool -Dyarn.application.classpath='/opt/apps/phoenix-4.7.0-HBase-1.1-bin/*,$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/*,$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,$HADOOP_YARN_HOME/share/hadoop/yarn/*,$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*,/opt/apps/extra-jars/*' --input /example.csv -z localhost -t example
验证:
sqlline.py localhost
0: jdbc:phoenix:localhost> select * from example;
+--------+-------------+------------+
| MY_PK | FIRST_NAME | LAST_NAME |
+--------+-------------+------------+
| 100 | Jack | Doe |
| 200 | Tony | Poppins |
+--------+-------------+------------+