Hadoop--2.0伪分布式安装

  本篇博客主要讲解Hadoop2.0伪分布式的安装配置(以hadoop-2.6.0.tar.gz为例),使用centsos6.5,jdk使用1.7。安装环境比较简单我就直接贴步骤了。如想安装1.0可查看博客http://blog.xiaoxiaomo.com/2016/04/09/Hadoop-1-0伪分布式安装/

准备

安装/配置

  • 三、上传hadoop后解压文件到/use/local/目录下

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    [root@xxo03 up]# tar -zxvf hadoop-2.6.0.tar.gz -C /usr/local/
    [root@xxo03 up]# cd /usr/local/hadoop-2.6.0/
    [root@xxo03 hadoop-2.6.0]# ll
    total 52
    drwxr-xr-x. 2 20000 20000 4096 Nov 14 2014 bin
    drwxr-xr-x. 3 20000 20000 4096 Nov 14 2014 etc
    drwxr-xr-x. 2 20000 20000 4096 Nov 14 2014 include
    drwxr-xr-x. 3 20000 20000 4096 Nov 14 2014 lib
    drwxr-xr-x. 2 20000 20000 4096 Nov 14 2014 libexec
    -rw-r--r--. 1 20000 20000 15429 Nov 14 2014 LICENSE.txt
    -rw-r--r--. 1 20000 20000 101 Nov 14 2014 NOTICE.txt
    -rw-r--r--. 1 20000 20000 1366 Nov 14 2014 README.txt
    drwxr-xr-x. 2 20000 20000 4096 Nov 14 2014 sbin
    drwxr-xr-x. 4 20000 20000 4096 Nov 14 2014 share
  • 四、修改配置文件
    修改/usr/local/hadoop-2.6.0/etc/hadoop/下的配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    [root@xxo03 hadoop-2.6.0]# cd /usr/local/hadoop-2.6.0/etc/hadoop/
    ##############修改如下8个配置文件##############################
    ############### 一. 2个sh文件、1个日志文件、1个主从文件##########
    hadoop-env.sh
    yarn-env.sh
    log4j.properties
    slaves
    ############## 二. 4个xml文件#################################
    core-site.xml
    hdfs-site.xml
    mapred-site.xml
    yarn-site.xml

修改后配置文件,点击可查看,这里就不一一贴出了(:我主机为xxo03,请把所有xxo03修改为你的主机名)

  • 五、创建目录,为了存放数据文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    ################### 创建目录 ##########################
    [root@xxo03 hadoop-2.6.0]# cd /usr/local/
    [root@xxo03 local]# mkdir -p /usr/local/hadoop_repo/name
    [root@xxo03 local]# cd hadoop_repo/
    [root@xxo03 hadoop_repo]# mkdir data
    [root@xxo03 hadoop_repo]# mkdir namesecondary
    [root@xxo03 hadoop_repo]# mkdir logs
    [root@xxo03 hadoop_repo]# mkdir tmp

    ################### 查看创建结果 ######################
    [root@xxo03 hadoop_repo]# ll
    total 20
    drwxr-xr-x. 2 root root 4096 May 9 05:21 data
    drwxr-xr-x. 2 root root 4096 May 9 05:21 logs
    drwxr-xr-x. 2 root root 4096 May 9 05:20 name
    drwxr-xr-x. 2 root root 4096 May 9 05:21 namesecondary
    drwxr-xr-x. 2 root root 4096 May 9 05:21 tmp
  • 六、配置环境变量,把bin和sbin目录配置到PATH路径下

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    ################### 修改文件 ##########################
    [root@xxo03 hadoop_repo]# vim /etc/profile
    JAVA_HOME=/usr/local/jdk1.7.0_79
    export JRE_HOME=${JAVA_HOME}/jre
    export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
    export HADOOP_HOME=/usr/local/hadoop-2.6.0
    PATH=.:${JAVA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH

    ################### 加载配置 ##########################
    [root@xxo03 hadoop_repo]# source /etc/profile
  • 七.格式化文件系统

    1
    2
    [root@xxo03 hadoop_repo]# hdfs namenode -format
    Formatting using clusterid: CID-c80f2abf-fc78-48fc-b1b3-84476587cce4
  • 八、启动HDFS

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    ################### 启动HDFS ##########################
    [root@xxo03 hadoop_repo]# start-dfs.sh
    Starting namenodes on [xxo03]
    The authenticity of host 'xxo03 (192.168.33.68)' can't be established.
    RSA key fingerprint is 3c:55:d3:bb:e9:15:5a:48:07:c4:22:6a:01:a5:45:cc.
    Are you sure you want to continue connecting (yes/no)? yes
    xxo03: Warning: Permanently added 'xxo03,192.168.33.68' (RSA) to the list of known hosts.
    xxo03: starting namenode, logging to /usr/local/hadoop_repo/logs/hadoop-root-namenode-xxo03.out
    xxo03: starting datanode, logging to /usr/local/hadoop_repo/logs/hadoop-root-datanode-xxo03.out
    Starting secondary namenodes [xxo03]
    xxo03: starting secondarynamenode, logging to /usr/local/hadoop_repo/logs/hadoop-root-secondarynamenode-xxo03.out

    ################### JPS测试 ##########################
    [root@xxo03 hadoop_repo]# jps
    1551 SecondaryNameNode
    1391 DataNode
    1663 Jps
    1312 NameNode

  • 浏览器测试:http://xxo03:50070/
    测试hadoop HDFS

  • 九、启动YARN

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    ################### 启动YARN ##########################
    [root@xxo03 hadoop_repo]# start-yarn.sh
    starting yarn daemons
    starting resourcemanager, logging to /usr/local/hadoop_repo/logs/yarn-root-resourcemanager-xxo03.out
    xxo03: starting nodemanager, logging to /usr/local/hadoop_repo/logs/yarn-root-nodemanager-xxo03.out

    ################### JPS测试 ##########################
    [root@xxo03 hadoop_repo]# jps
    2132 Jps
    1551 SecondaryNameNode ##start-yarn
    1391 DataNode
    1747 ResourceManager ##start-yarn
    1312 NameNode
    1830 NodeManager
  • 浏览器测试:http://xxo03:8088/
    测试hadoop HDFS

当前网速较慢或者你使用的浏览器不支持博客特定功能,请尝试刷新或换用Chrome、Firefox等现代浏览器