HBase--通过idea搭建源码阅读环境

2019-04-07

HBase

下载HBase源码
编译
启动服务
查看

　　该篇文章主要是，搭建一个HBase的源码阅读环境。主要是通过github下载源码，编译导入到开发环境idea中，然后启动相关服务，做DeBug调试来阅读相关源码。

下载HBase源码

通过github下载相关源码(我下载到目录：/opt/dev/idea/hbase )

注意：默认我们克隆时，会把所有的历史commit信息也会克隆下载。
对于一个非常活跃的开源项目来说这些历史信息非常占用空间，下载时就会很慢。
我们可以在git clone 后面加一个--depth 1这样只克隆最新的一次提交（深度为1），就会快很多，如下：

git clone https://github.com/apache/hbase.git –depth 1

编译

命令：mvn clean install -DskipTests

启动服务

这里启动服务主要是启动HBase单节点，HBase单节点只有一个HMaster线程，zookeeper以及HRegionServer都是以进程的形式存在，在后面的源码阅读中可以很好的发现。

配置HBase服务

在idea中添加一个配置（Configuration）- Application。
主要配置（注意自己的的项目路径）：

Main Class：org.apache.hadoop.hbase.master.HMaster
VM Options：-Dlog4j.configuration=file:/opt/dev/idea/hbase/conf/log4j.properties
参数：start
工作目录：/opt/dev/idea/hbase

如图所示：

配置HBase客户端

在idea中添加一个配置（Configuration）- Application。
主要配置（注意自己的的项目路径）：

Main Class：org.jruby.Main
VM Options：-Dhbase.ruby.sources=/opt/dev/idea/hbase/hbase-shell/src/main/ruby -Dlog4j.configuration=file:/opt/dev/idea/hbase/conf/log4j.properties
参数：/opt/dev/idea/hbase/bin/hirb.rb
工作目录：/opt/dev/idea/hbase

如图所示：

修改配置

hbase-common 下面的resource，修改下面几个配置，如下：

<property>
        <name>hbase.tmp.dir</name>
        <value>/opt/tmp/hbase-${user.name}</value>
        <description>Temporary directory on the local filesystem.
            Change this setting to point to a location more permanent
            than '/tmp', the usual resolve for java.io.tmpdir, as the
            '/tmp' directory is cleared on machine restart.
        </description>
    </property>

<property>
        <name>hbase.zookeeper.quorum</name>
        <value>txd</value>
        <description>Comma separated list of servers in the ZooKeeper ensemble
            (This config. should have been named hbase.zookeeper.ensemble).
            For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
            By default this is set to localhost for local and pseudo-distributed modes
            of operation. For a fully-distributed setup, this should be set to a full
            list of ZooKeeper ensemble servers. If HBASE_MANAGES_ZK is set in hbase-env.sh
            this is the list of servers which hbase will start/stop ZooKeeper on as
            part of cluster start/stop. Client-side, we will take this list of
            ensemble members and put it together with the hbase.zookeeper.property.clientPort
            config. and pass it into zookeeper constructor as the connectString
            parameter.
        </description>
    </property>

<property>
    <name>hbase.defaults.for.version.skip</name>
    <value>true</value>
    <description>Set to true to skip the 'hbase.defaults.for.version' check.
    Setting this to true can be useful in contexts other than
    the other side of a maven generation; i.e. running in an
    IDE.  You'll want to set this boolean to true to avoid
    seeing the RuntimeException complaint: "hbase-default.xml file
    seems to be for and old version of HBase (\${hbase.version}), this
    version is X.X.X-SNAPSHOT"</description>
  </property>

启动服务

服务启动，debug等调试操作

查看

通过zk查看

下载一个zookeeper，在/etc/profile或用户目录下配置好环境变量，如：export ZK_HOME=/opt/local/zookeeper-3.4.12

就可以运行zk的客户端了，查看hbase信息如下：

txd:Downloads tangxuandong$ cd /opt/local/zookeeper-3.4.12/
txd:zookeeper-3.4.12 tangxuandong$ cd bin/
txd:bin tangxuandong$ zkCli.sh 
Connecting to localhost:2181
2019-04-09 19:00:16,993 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.12-e5259e437540f349646870ea94dc2658c4e44b3b, built on 03/27/2018 03:55 GMT
2019-04-09 19:00:16,997 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=10.106.195.85
2019-04-09 19:00:16,997 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_161
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/opt/local/zookeeper-3.4.12/bin/../build/classes:/opt/local/zookeeper-3.4.12/bin/../build/lib/*.jar:/opt/local/zookeeper-3.4.12/bin/../lib/slf4j-log4j12-1.7.25.jar:/opt/local/zookeeper-3.4.12/bin/../lib/slf4j-api-1.7.25.jar:/opt/local/zookeeper-3.4.12/bin/../lib/netty-3.10.6.Final.jar:/opt/local/zookeeper-3.4.12/bin/../lib/log4j-1.2.17.jar:/opt/local/zookeeper-3.4.12/bin/../lib/jline-0.9.94.jar:/opt/local/zookeeper-3.4.12/bin/../lib/audience-annotations-0.5.0.jar:/opt/local/zookeeper-3.4.12/bin/../zookeeper-3.4.12.jar:/opt/local/zookeeper-3.4.12/bin/../src/java/lib/*.jar:/opt/local/zookeeper-3.4.12/bin/../conf:.:/Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/lib:/Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/Users/tangxuandong/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/var/folders/yx/g41l6w1n62bgm721hhlr6b1c0000gn/T/
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Mac OS X
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=x86_64
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=10.14.4
2019-04-09 19:00:16,999 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=tangxuandong
2019-04-09 19:00:17,000 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/Users/tangxuandong
2019-04-09 19:00:17,000 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/opt/local/zookeeper-3.4.12/bin
2019-04-09 19:00:17,001 [myid:] - INFO  [main:ZooKeeper@441] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@255316f2
Welcome to ZooKeeper!
2019-04-09 19:00:17,028 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1028] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-04-09 19:00:17,094 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@878] - Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2019-04-09 19:00:17,104 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1302] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x16a01a1936b0007, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper, hbase]
[zk: localhost:2181(CONNECTED) 1] ls /hbase
[meta-region-server, rs, splitWAL, backup-masters, table-lock, flush-table-proc, master-maintenance, online-snapshot, switch, master, running, draining, hbaseid, table]
[zk: localhost:2181(CONNECTED) 2]

浏览器访问
访问：http://localhost:16010/

上一篇：HBase--HMaster的启动流程

下一篇：监控--jmxtrans+influxdb+Grafana