Hadoop--1.0伪分布式安装

2016-04-09

Hadoop

配置环境
修改配置文件
注意事项

　　本篇博客主要讲解Hadoop1.0伪分布式的安装配置(以hadoop-1.1.2.tar.gz为例)，虽然现在都在使用2.x的版本但是对于1.0的我们还是应该知道怎么去配置，当然他们只有细微的差别。如果对于环境还没准备好的，可以阅读Hadoop-安装前环境准备。如需安装2.0请查看博客：http://blog.xiaoxiaomo.com/2016/05/08/Hadoop-2-0伪分布式安装/

配置环境

首先我们需要去下载hadoop-1.1.2.tar.gz，然后上传到Linux上，解压后配置环境变量。

1	命令：tar -zxvf hadoop-1.1.2.tar.gz #解压tar.gz文件

解压hadoop文件

vi /etc/profile修改环境变量并重启配置：

1
2
3

export HADOOP_HOME=/usr/local/hadoop-1.1.2 #指定目录
export PATH=.:$JAVA_HOME/bin:$HADOOP_HOME/bin:$PATH 
#注意PATH只在原基础上添加了:$HADOOP_HOME/bin

环境变量配置

记得重启配置：source /etc/profile
重启配置并检测是否已配置好

上面Warning: $HADOOP_HOME is deprecated.在配置文件中添加export HADOOP_HOME_WARN_SUPPRESS=0 就不会再有该提示。

修改配置文件

配置文件

修改Hadoop安装目录/conf目录/下的配置文件

hadoop-env.sh
core-site.xml
hdfs-site.xml
mapred-site.xml

A、hadoop-env.sh

1	export JAVA_HOME=/usr/local/jdk1.6 #这里修改为jdk的安装目录

B、core-site.xml

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://xiaoxiaomo01:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/usr/local/hadoop-1.1.2/tmp</value>
    </property>  
</configuration>

C、hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>
</configuration>

D、mapred-site.xml

<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>xiaoxiaomo01:9001</value>
    </property>
</configuration>
`

格式化磁盘

hadoop namenode -format #格式化磁盘

[root@xiaoxiaomo01 bin]# hadoop namenode -format
16/04/10 00:15:00 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = xiaoxiaomo01/192.168.3.221
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 1.1.2
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1 -r 1440782; compiled by 'hortonfo' on Thu Jan 31 02:03:24 UTC 2013
************************************************************/
16/04/10 00:15:00 INFO util.GSet: VM type       = 64-bit
16/04/10 00:15:00 INFO util.GSet: 2% max memory = 19.33375 MB
16/04/10 00:15:00 INFO util.GSet: capacity      = 2^21 = 2097152 entries
16/04/10 00:15:00 INFO util.GSet: recommended=2097152, actual=2097152
16/04/10 00:15:00 INFO namenode.FSNamesystem: fsOwner=root
16/04/10 00:15:00 INFO namenode.FSNamesystem: supergroup=supergroup
16/04/10 00:15:00 INFO namenode.FSNamesystem: isPermissionEnabled=false
16/04/10 00:15:01 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
16/04/10 00:15:01 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
16/04/10 00:15:01 INFO namenode.NameNode: Caching file names occuring more than 10 times 
16/04/10 00:15:01 INFO common.Storage: Image file of size 110 saved in 0 seconds.
16/04/10 00:15:01 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/usr/local/hadoop-1.1.2/tmp/dfs/name/current/edits
16/04/10 00:15:01 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/usr/local/hadoop-1.1.2/tmp/dfs/name/current/edits
16/04/10 00:15:01 INFO common.Storage: Storage directory /usr/local/hadoop-1.1.2/tmp/dfs/name has been successfully formatted.
16/04/10 00:15:01 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at xiaoxiaomo01/192.168.3.221
************************************************************/

启动服务

start-all.sh #启动服务

starting namenode, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-root-namenode-xiaoxiaomo01.out
localhost: starting datanode, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-root-datanode-xiaoxiaomo01.out
localhost: starting secondarynamenode, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-root-secondarynamenode-xiaoxiaomo01.out
starting jobtracker, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-root-jobtracker-xiaoxiaomo01.out
localhost: starting tasktracker, logging to /usr/local/hadoop-1.1.2/libexec/../logs/hadoop-root-tasktracker-xiaoxiaomo01.out

注意，如果没有配置ssh就会提示输入多次密码

验证

（1）、执行命令jps，查看进程，分别是：
NameNode、SecondaryNameNode、DataNode、JobTracker、TaskTracker

（2）、在浏览器查看，http://xiaoxiaomo01:50070、http://xiaoxiaomo01:50030
（如果访问的电脑没有配置hosts，192.168.3.221 xiaoxiaomo01就不能通过hostsname访问，只能通过ip，eg：http://192.168.3.221:50030/）

停止服务

命令：stop-all.sh

[root@xiaoxiaomo01 bin]# ./stop-all.sh 
stopping jobtracker
localhost: stopping tasktracker
stopping namenode
localhost: stopping datanode
localhost: stopping secondarynamenode

注意事项

启动时没有NameNode的可能原因:

(1)没有格式化
(2)环境变量设置错误
(3)ip与hostname绑定失败
(4)配置文件写错了

这时候我们可以到Hadoop安装目录/logs/下面查看日志文件，进行排错。

上一篇：Hadoop-HDFS架构和Shell