Otu esi etinye Hadoop Single Node Cluster (Pseudonode) na CentOS 7
Hadoop bụ usoro mepere emepe nke a na-ejikarị eme ihe na Bigdata. A na-ewuli ọtụtụ ọrụ nyocha Bigdata/Data n'elu Hadoop Eco-System. Ọ nwere okpukpu abụọ, otu bụ maka ịchekwa data na nke ọzọ bụ maka nhazi data.
A ga-elekọta nchekwa site na sistemụ faịlụ nke ya akpọrọ HDFS (Hadoop Distributed Filesystem) na nhazi ga-elekọta ya site na YARN (Ma Ọzọ Resource Negotiator). Mapreduce bụ igwe nhazi nke Hadoop Eco-System nke ndabara.
Edemede a na-akọwa usoro ịwụnye nrụnye Pseudonode nke Hadoop, ebe daemons niile (JVMs) ga na-agba Single Node Cluster na CentOS 7.
Nke a bụ maka ndị mbido ịmụ Hadoop. Na ezigbo oge, a ga-etinye Hadoop dị ka ụyọkọ multinode ebe a ga-ekesa data n'etiti sava dị ka ngọngọ na a ga-arụ ọrụ ahụ n'otu ụzọ ahụ.
- Obere ntinye nke sava CentOS 7.
- Ntọhapụ Java v1.8.
- Hadoop 2.x ntọhapụ kwụsiri ike.
Na ibe a
- Otu esi etinye Java na CentOS 7
- Halite nbanye na-enweghị paswọọdụ na CentOS 7
- Otu esi etinye Hadoop Single Node na CentOS 7
- Otu esi ahazi Hadoop na CentOS 7
- Ịhazi usoro faịlụ HDFS site na AhaNode
1. Hadoop bụ Eco-System nke mejupụtara Java. Anyị kwesịrị itinye Java na sistemụ anyị iwu ka ịwụnye Hadoop.
# yum install java-1.8.0-openjdk
2. Ọzọ, nyochaa ụdị Java arụnyere na sistemụ.
# java -version
Anyị kwesịrị ịhazi ssh na igwe anyị, Hadoop ga-ejikwa ọnụ na iji SSH. Master node na-eji njikọ SSH jikọọ ọnụ ohu ya wee rụọ ọrụ dịka mmalite na kwụsị.
Anyị kwesịrị ịtọ ssh na-enweghị paswọọdụ ka onye nwe ya nwee ike ịkparịta ụka na ndị ohu site na iji ssh na-enweghị paswọọdụ. Ma ọ bụghị ya maka ntọala njikọ ọ bụla, ịkwesịrị itinye paswọọdụ.
N'ime otu oghere a, ọrụ Master (Namenode, Secondary Namenode & Resource Manager) na ọrụ ohu (Datanode & Nodemanager) ga-agba ọsọ dị ka JVM dị iche iche. N'agbanyeghị na ọ bụ node singe, anyị kwesịrị inwe ssh na-enweghị paswọọdụ iji mee Nna-ukwu ka ọ na-ekwurịta okwu ohu na-enweghị nkwenye.
3. Tọọ nbanye SSH na-enweghị paswọọdụ site na iji iwu ndị a na sava ahụ.
# ssh-keygen # ssh-copy-id -i localhost
4. Mgbe ị ahazi paswọọdụ SSH nbanye, gbalịa ịbanye ọzọ, ị ga-ejikọta na-enweghị paswọọdụ.
# ssh localhost
5. Gaa na ebe nrụọrụ weebụ Apache Hadoop wee budata ntọhapụ kwụsiri ike nke Hadoop site na iji iwu wget na-esonụ.
# wget https://archive.apache.org/dist/hadoop/core/hadoop-2.10.1/hadoop-2.10.1.tar.gz # tar xvpzf hadoop-2.10.1.tar.gz
6. Ọzọ, tinye mgbanwe gburugburu Hadoop na faịlụ ~/.bashrc dị ka egosiri.
HADOOP_PREFIX=/root/hadoop-2.10.1 PATH=$PATH:$HADOOP_PREFIX/bin export PATH JAVA_HOME HADOOP_PREFIX
7. Mgbe ịgbakwunye mgbanwe gburugburu ebe obibi na ~/.bashrc faịlụ, wepụta faịlụ ahụ wee nyochaa Hadoop site na ịme iwu ndị a.
# source ~/.bashrc # cd $HADOOP_PREFIX # bin/hadoop version
Anyị kwesịrị ịhazi faịlụ nhazi Hadoop n'okpuru ka ị daba na igwe gị. Na Hadoop, ọrụ ọ bụla nwere nọmba ọdụ ụgbọ mmiri ya na akwụkwọ ndekọ aha ya iji chekwaa data ahụ.
- Faịlụ nhazi Hadoop – core-site.xml, hdfs-site.xml, mapred-site.xml & yarn-site.xml
8. Nke mbụ, anyị kwesịrị imelite JAVA_HOME na ụzọ Hadoop na faịlụ hadoop-env.sh dị ka egosiri.
# cd $HADOOP_PREFIX/etc/hadoop # vi hadoop-env.sh
Tinye ahịrị na-esonụ na mmalite nke faịlụ.
export JAVA_HOME=/usr/lib/jvm/java-1.8.0/jre export HADOOP_PREFIX=/root/hadoop-2.10.1
9. Ọzọ, gbanwee faịlụ core-site.xml faịlụ.
# cd $HADOOP_PREFIX/etc/hadoop # vi core-site.xml
Tapawa na-eso n'etiti mkpado <configuration> dị ka egosiri.
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
10. Mepụta akwụkwọ ndekọ aha n'okpuru tecmint ndekọ ụlọ onye ọrụ, nke a ga-eji maka nchekwa NN na DN.
# mkdir -p /home/tecmint/hdata/ # mkdir -p /home/tecmint/hdata/data # mkdir -p /home/tecmint/hdata/name
10. Ọzọ, gbanwee faịlụ hdfs-site.xml faịlụ.
# cd $HADOOP_PREFIX/etc/hadoop # vi hdfs-site.xml
Tapawa na-eso n'etiti mkpado <configuration> dị ka egosiri.
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/home/tecmint/hdata/name</value>
</property>
<property>
<name>dfs .datanode.data.dir</name>
<value>home/tecmint/hdata/data</value>
</property>
</configuration>
11. Ọzọ, gbanwee faịlụ mapred-site.xml faịlụ.
# cd $HADOOP_PREFIX/etc/hadoop # cp mapred-site.xml.template mapred-site.xml # vi mapred-site.xml
Tapawa na-eso n'etiti mkpado <configuration> dị ka egosiri.
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
12. N'ikpeazụ, gbanwee faịlụ yarn-site.xml.
# cd $HADOOP_PREFIX/etc/hadoop # vi yarn-site.xml
Tapawa na-eso n'etiti mkpado <configuration> dị ka egosiri.
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
13. Tupu ịmalite ụyọkọ ahụ, anyị kwesịrị ịhazi Hadoop NN na sistemụ mpaghara anyị ebe etinyere ya. Ọtụtụ mgbe, a ga-eme ya na mbido mbụ tupu ịmalite ụyọkọ nke mbụ.
Ịmepụta NN ga-eme ka mfu data na NN metastore, ya mere, anyị ga-akpachara anya karị, anyị ekwesịghị ịhazi NN mgbe ụyọkọ na-agba ọsọ ọ gwụla ma achọrọ ya ụma.
# cd $HADOOP_PREFIX # bin/hadoop namenode -format
14. Malite NameNode daemon na DataNode daemon: (ọdụ ụgbọ mmiri 50070).
# cd $HADOOP_PREFIX # sbin/start-dfs.sh
15. Malite ResourceManager daemon na NodeManager daemon: (ọdụ ụgbọ mmiri 8088).
# sbin/start-yarn.sh
16. Ịkwụsị ọrụ niile.
# sbin/stop-dfs.sh # sbin/stop-dfs.sh
Nchịkọta
N'ime edemede a, anyị agabigala usoro site na nzọụkwụ iji guzobe ụyọkọ Hadoop Pseudonode (Single Node). Ọ bụrụ na ị nwere ihe ọmụma bụ isi nke Linux wee soro usoro ndị a, ụyọkọ ahụ ga-adị elu n'ime nkeji iri anọ.
Nke a nwere ike ịba uru nke ukwuu maka onye mbido ịmalite ịmụ ma na-eme Hadoop ma ọ bụ ụdị vanilla Hadoop a nwere ike iji maka ebumnuche mmepe. Ọ bụrụ na anyị chọrọ inwe ụyọkọ oge, ma anyị chọrọ ma ọ dịkarịa ala sava anụ ahụ 3 n'aka ma ọ bụ nye Cloud maka inwe ọtụtụ sava.