hadoop-env.sh
是 Hadoop 配置中的一个关键脚本,它定义了 Hadoop 运行环境的变量。这个脚本通常位于 Hadoop 安装目录下的 etc/hadoop/
目录中。在该脚本中,你可以设置各种环境变量来影响 Hadoop 的行为,例如 Java 的路径、Hadoop 的临时目录、以及 JVM 的内存参数等。
下面是一个典型的 hadoop-env.sh
文件的示例内容:
#!/bin/bash
# The java implementation to use.
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
# Extra Java CLASSPATH elements. This is for Hadoop internal use and
# shouldn't normally be modified.
export HADOOP_CLASSPATH=
# Extra Java runtime options. Empty by default.
export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true"
# Where log files are stored. $HADOOP_HOME/logs by default.
export HADOOP_LOG_DIR=${HADOOP_HOME}/logs
# A string representing this instance of hadoop. $USER by default.
export HADOOP_IDENT_STRING=$USER
# The scheduling priority for daemon processes. See 'man nice'.
export HADOOP_NICENESS=10
# The directory where pid files are stored. /tmp by default.
export HADOOP_PID_DIR=/var/run/hadoop
# The directory where temporary files are stored. /tmp by default.
export HADOOP_TMP_DIR=/var/tmp/hadoop-${HADOOP_IDENT_STRING}
# Extra ssh options. Empty by default.
export HADOOP_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HADOOP_CONF_DIR"
# The amount of heap to reserve before the GC kicks in (in MB).
# By default, we set it to 85% of physical memory.
# export HADOOP_HEAPSIZE=1024
# The maximum virtual memory a Hadoop child process can consume before
# it is killed. Specify -1 for unlimited.
# export HADOOP_CHILD_VMEM_LIMIT=-1
# The maximum paged memory a Hadoop child process can consume before
# it is killed. Specify -1 for unlimited.
# export HADOOP_CHILD_PMEM_LIMIT=-1
你可能需要根据你的系统环境和需求修改这些变量。例如,如果你使用的是不同的 Java 版本,你需要更新 JAVA_HOME
变量以指向正确的 Java 安装路径。同样,HADOOP_OPTS
可以用于调整 JVM 参数,比如增加或减少分配给 Hadoop 进程的堆内存。
请确保在修改后保存文件并重新启动 Hadoop 服务以便让新的设置生效。