小功告成,在WIN7上配置上了HADOOP,终于可以一窥真容啦

简介:

主要参考文档:

http://blog.csdn.net/ruby97/article/details/7423088

以后,慢慢熟悉具体操作吧。

sky@koala

/home/hadoop-0.20.2

$ cd conf

sky@koala

/home/hadoop-0.20.2/conf

$ dos2unix hadoop-env.sh

dos2unix: converting file hadoop-env.sh to Unix format ...

sky@koala

/home/hadoop-0.20.2/conf

$ cd ..

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop namenode -format

13/03/19 10:19:46 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = koala/192.168.7.9

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 0.20.2

STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010

************************************************************/

13/03/19 10:19:47 INFO namenode.FSNamesystem: fsOwner=sky,Domain,Users,Administrators,Remote,Desktop,Users,Users,Remote,Desktop,Users,Administrators

13/03/19 10:19:47 INFO namenode.FSNamesystem: supergroup=supergroup

13/03/19 10:19:47 INFO namenode.FSNamesystem: isPermissionEnabled=true

13/03/19 10:19:47 INFO common.Storage: Image file of size 101 saved in 0 seconds.

13/03/19 10:19:47 INFO common.Storage: Storage directory \tmp\hadoop-Sky\dfs\name has been successfully formatted.

13/03/19 10:19:47 INFO namenode.NameNode: SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at koala/192.168.X.X

************************************************************/

sky@koala

/home/hadoop-0.20.2

$ bin/start-all.sh

starting namenode, logging to /home/hadoop-0.20.2/bin/../logs/hadoop-sky-namenode-koala.out

localhost: starting datanode, logging to /home/hadoop-0.20.2/bin/../logs/hadoop-sky-datanode-koala.out

localhost: cygwin warning:

localhost:   MS-DOS style path detected: C:\cygwin\home\hadoop-0.20.2\/build/native

localhost:   Preferred POSIX equivalent is: /home/hadoop-0.20.2/build/native

localhost:   CYGWIN environment variable option "nodosfilewarning" turns off this warning.

localhost:   Consult the user's guide for more details about POSIX paths:

localhost:     http://cygwin.com/cygwin-ug-net/using.html#using-pathnames

localhost: starting secondarynamenode, logging to /home/hadoop-0.20.2/bin/../logs/hadoop-sky-secondarynamenode-koala.out

localhost: log4j:ERROR setFile(null,true) call failed.

localhost: java.io.FileNotFoundException: C:\cygwin\home\hadoop-0.20.2\logs\hadoop-sky-secondarynamenode-koala.log (Access is denied)

localhost:      at java.io.FileOutputStream.openAppend(Native Method)

localhost:      at java.io.FileOutputStream.<init>(FileOutputStream.java:192)

localhost:      at java.io.FileOutputStream.<init>(FileOutputStream.java:116)

localhost:      at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)

localhost:      at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)

localhost:      at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)

localhost:      at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)

localhost:      at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)

starting jobtracker, logging to /home/hadoop-0.20.2/bin/../logs/hadoop-sky-jobtracker-koala.out

localhost: starting tasktracker, logging to /home/hadoop-0.20.2/bin/../logs/hadoop-sky-tasktracker-koala.out

localhost: log4j:ERROR setFile(null,true) call failed.

localhost: java.io.FileNotFoundException: C:\cygwin\home\hadoop-0.20.2\logs\hadoop-sky-tasktracker-koala.log (Access is denied)

localhost:      at java.io.FileOutputStream.openAppend(Native Method)

localhost:      at java.io.FileOutputStream.<init>(FileOutputStream.java:192)

localhost:      at java.io.FileOutputStream.<init>(FileOutputStream.java:116)

localhost:      at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)

localhost:      at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)

localhost:      at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)

localhost:      at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)

localhost:      at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)

sky@koala

/home/hadoop-0.20.2

$ jps

956 JobTracker

7344 NameNode

6912 Jps

sky@koala

/home/hadoop-0.20.2

$ ps

      PID    PPID    PGID     WINPID   TTY     UID    STIME COMMAND

    10252       1   12080       9788  ?       1001 10:22:03 /cygdrive/c/Java/jdk1.6.0_43/bin/java

    10812       1   11428        956  pty0    1001 10:22:04 /cygdrive/c/Java/jdk1.6.0_43/bin/java

     5344   10852    5344      10076  pty0    1001 10:22:28 /usr/bin/ps

    10852    1824   10852      12052  pty0    1001 22:50:39 /usr/bin/bash

     2032       1    9660      11484  ?       1001 10:22:01 /cygdrive/c/Java/jdk1.6.0_43/bin/java

     1824       1    1824       1824  ?       1001 22:50:39 /usr/bin/mintty

     7692       1   11428       7344  pty0    1001 10:21:59 /cygdrive/c/Java/jdk1.6.0_43/bin/java

     5228       1   10496       8688  ?       1001 10:22:06 /cygdrive/c/Java/jdk1.6.0_43/bin/java

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop fs -mkdir In

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop fs -put *.txt In

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop fs -ls

Found 1 items

drwxr-xr-x   - supergroup          0 2013-03-19 10:23 /user/In

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop fs -put *.pdf In

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop fs -ls /user/

Found 1 items

drwxr-xr-x   - supergroup          0 2013-03-19 10:28 /user/In

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop fs -ls /user/In

Found 5 items

-rw-r--r--   1 supergroup     348624 2013-03-19 10:23 /user/In/CHANGES.txt

-rw-r--r--   1 supergroup      13366 2013-03-19 10:23 /user/In/LICENSE.txt

-rw-r--r--   1 supergroup        101 2013-03-19 10:23 /user/In/NOTICE.txt

-rw-r--r--   1 supergroup   38321433 2013-03-19 10:28 /user/In/ObjectLearn.pdf

-rw-r--r--   1 supergroup       1366 2013-03-19 10:23 /user/In/README.txt

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop dfsadmin -report

Configured Capacity: 104752738304 (97.56 GB)

Present Capacity: 61319459840 (57.11 GB)

DFS Remaining: 61280444416 (57.07 GB)

DFS Used: 39015424 (37.21 MB)

DFS Used%: 0.06%

Under replicated blocks: 0

Blocks with corrupt replicas: 0

Missing blocks: 0

-------------------------------------------------

Datanodes available: 1 (1 total, 0 dead)

Name: 127.0.0.1:50010

Decommission Status : Normal

Configured Capacity: 104752738304 (97.56 GB)

DFS Used: 39015424 (37.21 MB)

Non DFS Used: 43433278464 (40.45 GB)

DFS Remaining: 61280444416(57.07 GB)

DFS Used%: 0.04%

DFS Remaining%: 58.5%

Last contact: Tue Mar 19 10:40:52 CST 2013

 

 

sky@koala

/home/hadoop-0.20.2

$ bin/hadoop dfs -lsr /

sky@koala

/home/hadoop-0.20.2

$

目录
相关文章
|
11月前
|
XML 分布式计算 资源调度
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece(一)
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece(一)
417 5
|
分布式计算 Hadoop Devops
Hadoop集群配置https实战案例
本文提供了一个实战案例,详细介绍了如何在Hadoop集群中配置HTTPS,包括生成私钥和证书文件、配置keystore和truststore、修改hdfs-site.xml和ssl-client.xml文件,以及重启Hadoop集群的步骤,并提供了一些常见问题的故障排除方法。
314 3
Hadoop集群配置https实战案例
|
11月前
|
分布式计算 Hadoop Shell
Hadoop-35 HBase 集群配置和启动 3节点云服务器 集群效果测试 Shell测试
Hadoop-35 HBase 集群配置和启动 3节点云服务器 集群效果测试 Shell测试
221 4
|
11月前
|
XML 资源调度 网络协议
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece(二)
大数据-02-Hadoop集群 XML配置 超详细 core-site.xml hdfs-site.xml 3节点云服务器 2C4G HDFS Yarn MapRedece(二)
459 4
|
11月前
|
分布式计算 资源调度 Hadoop
大数据-01-基础环境搭建 超详细 Hadoop Java 环境变量 3节点云服务器 2C4G XML 集群配置 HDFS Yarn MapRedece
大数据-01-基础环境搭建 超详细 Hadoop Java 环境变量 3节点云服务器 2C4G XML 集群配置 HDFS Yarn MapRedece
277 4
|
11月前
|
SQL 存储 分布式计算
Hadoop-16-Hive HiveServer2 HS2 允许客户端远程执行HiveHQL HCatalog 集群规划 实机配置运行
Hadoop-16-Hive HiveServer2 HS2 允许客户端远程执行HiveHQL HCatalog 集群规划 实机配置运行
185 3
|
11月前
|
SQL 分布式计算 Hadoop
Hadoop-34 HBase 安装部署 单节点配置 hbase-env hbase-site 超详细图文 附带配置文件
Hadoop-34 HBase 安装部署 单节点配置 hbase-env hbase-site 超详细图文 附带配置文件
374 2
|
11月前
|
SQL 存储 数据管理
Hadoop-15-Hive 元数据管理与存储 Metadata 内嵌模式 本地模式 远程模式 集群规划配置 启动服务 3节点云服务器实测
Hadoop-15-Hive 元数据管理与存储 Metadata 内嵌模式 本地模式 远程模式 集群规划配置 启动服务 3节点云服务器实测
157 2
|
11月前
|
分布式计算 Hadoop
Hadoop-27 ZooKeeper集群 集群配置启动 3台云服务器 myid集群 zoo.cfg多节点配置 分布式协调框架 Leader Follower Observer
Hadoop-27 ZooKeeper集群 集群配置启动 3台云服务器 myid集群 zoo.cfg多节点配置 分布式协调框架 Leader Follower Observer
200 1
|
11月前
|
存储 SQL 消息中间件
Hadoop-26 ZooKeeper集群 3台云服务器 基础概念简介与环境的配置使用 架构组成 分布式协调框架 Leader Follower Observer
Hadoop-26 ZooKeeper集群 3台云服务器 基础概念简介与环境的配置使用 架构组成 分布式协调框架 Leader Follower Observer
149 0

相关实验场景

更多