Otu esi etinye ma hazie Apache Hadoop na Otu Node na CentOS 7

Apache Hadoop bụ ntọala mepere emepe maka nchekwa data buru ibu na nhazi data n'ofe ụyọkọ kọmputa. Ọrụ a dabere na ihe ndị a:

Hadoop Common – o nwere ọba akwụkwọ Java na akụrụngwa chọrọ nke modul Hadoop ndị ọzọ chọrọ.
HDFS – Sistemụ Faịlụ Ekesa Hadoop – Sistemụ faịlụ nwere ike ịbelata Java ekesara n'obere ọnụ ọnụ.
MapReduce – YARN framework for parallel big data processing.
Hadoop YARN: Usoro maka njikwa akụrụngwa.

Edemede a ga-eduzi gị ka ị ga-esi tinye Apache Hadoop na otu ụyọkọ ọnụ na CentOS 7 (na-arụkwa ọrụ maka ụdị RHEL 7 na Fedora 23+). A na-atụkwa aka na ụdị nhazi a dị ka Hadoop Pseudo-Distributed Mode.

Kwụpụ 1: Wụnye Java na CentOS 7

1. Tupu ịga n'ihu na nrụnye Java, buru ụzọ banye na onye ọrụ mgbọrọgwụ ma ọ bụ onye ọrụ nwere ohere mgbọrọgwụ hazie aha nnabata igwe gị na iwu a.

# hostnamectl set-hostname master

Ọzọkwa, tinye ndekọ ọhụrụ na faịlụ ndị ọbịa na igwe FQDN nke gị iji rụtụ aka na adreesị IP sistemụ gị.

# vi /etc/hosts

Tinye ahịrị dị n'okpuru:

192.168.1.41 master.hadoop.lan

Jiri ntọala nke gị dochie aha nnabata dị n'elu yana ndekọ FQDN.

2. Ọzọ, gaa na ibe nbudata Oracle Java wee jide ụdị Java SE Development Kit 8 kachasị ọhụrụ na sistemụ gị site n'enyemaka nke iwu curl:

# curl -LO -H "Cookie: oraclelicense=accept-securebackup-cookie" “http://download.oracle.com/otn-pub/java/jdk/8u92-b14/jdk-8u92-linux-x64.rpm”

3. Mgbe nbudata ọnụọgụ abụọ Java gwụchara, wụnye ngwugwu ahụ site n'inye iwu dị n'okpuru:

# rpm -Uvh jdk-8u92-linux-x64.rpm

Kwụpụ 2: Wụnye Hadoop Framework na CentOS 7

4. Ọzọ, mepụta akaụntụ onye ọrụ ọhụrụ na sistemụ gị na-enweghị ike mgbọrọgwụ nke anyị ga-eji ya maka ụzọ nwụnye Hadoop na gburugburu ebe ọrụ. Akwụkwọ ndekọ aha ụlọ ọhụrụ ga-ebi na ndekọ /opt/hadoop ndekọ.

# useradd -d /opt/hadoop hadoop
# passwd hadoop

5. Na nzọụkwụ ọzọ gaa Apache Hadoop peeji nke iji nweta njikọ maka ụdị kwụsiri ike kachasị ọhụrụ wee budata Archive na sistemụ gị.

# curl -O http://apache.javapipe.com/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz

6. Wepụ Archive oyiri ọdịnaya ndekọ na hadoop akaụntụ home ụzọ. Ọzọkwa, jide n'aka na ị gbanwee ikikere faịlụ ndị e depụtaghachiri n'otu aka ahụ.

#  tar xfz hadoop-2.7.2.tar.gz
# cp -rf hadoop-2.7.2/* /opt/hadoop/
# chown -R hadoop:hadoop /opt/hadoop/

7. Ọzọ, banye na onye ọrụ hadoop wee hazie Hadoop na Java Environment Variables na sistemụ gị site na dezie faịlụ .bash_profile.

# su - hadoop
$ vi .bash_profile

Tinye ahịrị ndị a na njedebe faịlụ:

## JAVA env variables
export JAVA_HOME=/usr/java/default
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar

## HADOOP env variables
export HADOOP_HOME=/opt/hadoop
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_YARN_HOME=$HADOOP_HOME
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

8. Ugbu a, bido mgbanwe gburugburu ebe obibi wee lelee ọkwa ha site n'inye iwu ndị a:

$ source .bash_profile
$ echo $HADOOP_HOME
$ echo $JAVA_HOME

9. N'ikpeazụ, hazie ssh key based authentication maka akaụntụ hadoop site n'ịgbaso iwu ndị dị n'okpuru (dochie aha nnabata ma ọ bụ FQDN megide iwu ssh-copy-id ya mere).

Ọzọkwa, hapụ okwuntughe ahụ edebere oghere ka iji ssh banye na-akpaghị aka.

$ ssh-keygen -t rsa
$ ssh-copy-id master.hadoop.lan