1、源码下载
CDH Hadoop版本:hadoop-2.6.0-cdh5.16.2-src.tar.gzApache Hadoop版本:hadoop-2.7.0-src.tar.gz
2、解压源码压缩包
包的根目录下有BUILDING字样的文本文件,该文档为编译部署文档,你是Liunx编译就找linux编译的部分,你是Windows就按照文档提供的Windows环境编译。
3、本文编译使用Linux编译,环境要求为
JDK 1.7+Maven 3.0或更高版本ProtocolBuffer 2.5.0CMake 2.6 or newerzlib headers(如果需要Hadoop支持压缩格式,博主要让Hadoop支持各种压缩,本文配了)
4、环境部署
(1)下载解压JDK&Maven
[root@izt4n8t6hadahw7jli48l0z software
]$
wget http://mirror.rise.ph/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
[root@izt4n8t6hadahw7jli48l0z software
]$
tar -zxvf jdk-8u141-linux-x64.tar.gz -C
../
[root@izt4n8t6hadahw7jli48l0z software
]$
tar -zxvf apache-maven-3.6.3-bin.tar.gz
-C
../
[root@izt4n8t6hadahw7jli48l0z software
]$ vim ~/.bashrc
export JAVA_HOME
=/root/app/jdk1.8.0_141
export MAVEN_HOME
=/root/app/apache-maven-3.6.3
export PATH
=$JAVA_HOME/bin:
$MAVEN_HOME/bin:
$PATH
export MAVEN_OPTS
="-Xmx2g -XX:ReservedCodeCacheSize=1g"
(2)安装依赖
yum
install -y gcc gcc-c++
make cmake
yum
install -y autoconf automake libtool
curl make g++ unzip
yum
install -y openssl openssl-devel svn ncurses-devel zlib-devel libtool
yum
install -y snappy snappy-devel
bzip2 bzip2-devel lzo lzo-devel lzop autoconf automake
(3)安装ProtocolBuffer 2.5.0
[root@izt4n8t6hadahw7jli48l0z ~
]$
wget https://github.com/protocolbuffers/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
[root@izt4n8t6hadahw7jli48l0z ~
]$
tar -zxvf protobuf-2.5.0.tar.gz -C
../app/
[root@izt4n8t6hadahw7jli48l0z ~
]$
cd /root/app/protobuf-2.5.0
[root@izt4n8t6hadahw7jli48l0z protobuf-2.5.0
]$ ./configure --prefix
=/root/app/protobuf
[root@izt4n8t6hadahw7jli48l0z protobuf-2.5.0
]$
make && make install
[root@izt4n8t6hadahw7jli48l0z protobuf-2.5.0
]$
cd /root/app/protobuf
[root@izt4n8t6hadahw7jli48l0z software
]$ vim ~/.bashrc
export PROTOCOL_HOME
=/root/app/protobuf
export PATH
=$PROTOCOL_HOME/bin:
$PATH
[root@izt4n8t6hadahw7jli48l0z protobuf
]$ protoc --version
libprotoc 2.5.0
5、解压hadoop源码&执行编译命令
[root@izt4n8t6hadahw7jli48l0z software
]$
tar -zxvf hadoop-2.6.0-cdh5.16.2-src.tar.gz -C
../
[root@izt4n8t6hadahw7jli48l0z hadoop-2.6.0-cdh5.16.2
]$ mvn clean package -Pdist,native -DskipTests -Dtar