Otu esi etinye ma hazie njikwa Cloudera na CentOS/RHEL 7 - Nkebi 3

N'isiokwu a, anyị kọwara usoro nzọụkwụ site na nzọụkwụ iji wụnye Cloudera Manager dị ka ọrụ mmepụta ihe. Na Nkebi nke 2, anyị agabigalarị Cloudera Pre-requisites, jide n'aka na akwadoro sava niile nke ọma.

Emume kacha mma maka ibunye sava Hadoop na CentOS/RHEL 7 – Akụkụ 1
Ịtọlite ihe ndị a chọrọ na Hadoop chọrọ yana ike nchekwa - Nkebi nke 2

N'ebe a, anyị ga-enwe ụyọkọ ọnụ 5 ebe ndị nna ukwu 2 na ndị ọrụ 3. Ejila m ihe atụ 5 AWS EC2 gosi usoro nrụnye. Akpọla m ndị sava ise ahụ aha dị n'okpuru.

master1.linux-console.net
master2.linux-console.net
worker1.linux-console.net
worker2.linux-console.net
worker3.linux-console.net

Onye njikwa Cloudera bụ ọrụ nhazi na nlekota oru maka CDH dum. Anyị nchịkwa na-akpọkarị ya ngwa njikwa maka Cloudera Hadoop. Anyị nwere ike ibugharị, nyochaa, jikwaa ma mee mgbanwe nhazi site na iji ngwá ọrụ a. Nke a dị oke mkpa iji jikwaa ụyọkọ ahụ dum.

N'okpuru bụ ojiji dị mkpa nke Cloudera Manager.

Kwunye ma hazie ụyọkọ Hadoop n'ụzọ akpaaka.
Nyochaa ahụike ụyọkọ
Hazie ọkwa
Nchọpụta nsogbu
Na-akọ
Na-eme mkpesa ojiji ụyọkọ
Na-ahazi akụrụngwa nke ọma

Kwụpụ 1: Ịwụnye Apache Web Server na CentOS

Anyị ga-eji master1 dị ka sava weebụ maka ebe nchekwa Cloudera. Ọzọkwa, Cloudera Manager bụ WebUI, yabụ anyị kwesịrị itinye Apache. Soro usoro ndị a iji wụnye sava weebụ apache.

# yum -y install httpd

Ozugbo arụnyere httpd, malite ya ma mee ka ọ malite na buut.

# systemctl start httpd
# systemctl enable httpd

Mgbe ịmalite httpd, hụ ọkwa ahụ.

# systemctl status httpd

Mgbe ịmalite httpd, mepee ihe nchọgharị na sistemụ mpaghara gị wee mado adreesị IP nke master1 na nchọta ọchụchọ, ị ga-enweta ibe ule a iji hụ na httpd na-aga nke ọma.

Kwụpụ 2: Hazie DNS mpaghara iji dozie IP na aha nnabata

Anyị kwesịrị inwe ihe nkesa DNS ma ọ bụ hazie /etc/hosts iji dozie IP na aha nnabata. N'ebe a, anyị na-ahazi /etc/hosts, mana n'oge dị adị, ihe nkesa DNS raara onwe ya nye ga-adị maka ebe mmepụta ihe.

Soro usoro ndị a dị n'okpuru ka ịdebanye aha maka sava gị niile na /etc/hosts.

# vi /etc/hosts

Ekwesịrị ịhazi nke a na sava niile.

13.235.27.144   master1.linux-console.net     master1
13.235.135.170  master2.linux-console.net     master2
15.206.167.94   worker1.linux-console.net     worker1
13.232.173.158  worker2.linux-console.net     worker2
65.0.182.222    worker3.linux-console.net     worker3

Kwụpụ 3: Hazie Nbanye na-enweghị paswọọdụ SSH

A na-etinye onye njikwa Cloudera na master1 na ngosipụta a. Anyị kwesịrị ịhazi ssh-enweghị paswọọdụ site na master1 gaa na ọnụ ndị ọzọ niile. N'ihi na Cloudera Manager ga-eji ssh na-ekwurịta okwu ndị ọzọ niile ngwungwu.

Soro usoro ndị a iji hazie ssh paswọọdụ na-enweghị ike sitere na master1 gaa na sava niile fọdụrụnụ. Anyị ga-enwe onye ọrụ 'tecmint' ka ọ gaa n'ihu.

Mepụta onye ọrụ 'tecmint' niile sava 4 site na iji useradd iwu dị ka egosiri.

# useradd -m tecmint

Iji nye onye ọrụ 'tecmint' ohere mgbọrọgwụ, tinye ahịrị dị n'okpuru n'ime faịlụ /etc/sudoers. Ị nwere ike ịgbakwunye akara a n'okpuru mgbọrọgwụ dị ka inye na nseta ihuenyo.

tecmint   ALL=(ALL)    ALL

Banye na onye ọrụ 'tecmint' wee mepụta igodo ssh na master1 site na iji iwu dị n'okpuru.

# sudo su tecmint
$ ssh-keygen

Ugbu a detuo igodo emepụtara na sava 4 niile site na iji iwu ssh-copy-id dị ka egosiri.

$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email 
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email 
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email  
$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email

Ugbu a ị ga-enwe ike ssh site na master1 gaa na sava niile na-enweghị paswọọdụ dịka egosiri.

$ ssh master2
$ ssh worker1
$ ssh worker2
$ ssh worker3

Kwụpụ 4: Wụnye na ịhazi Cloudera Manager

Anyị nwere ike iji ebe nchekwa ihe na-ere ahịa (Cloudera) iji wụnye ngwugwu niile site na iji ngwa njikwa ngwugwu na RHEL/CentOS. N'oge dị adị, ịmepụta ebe nchekwa nke anyị bụ omume kachasị mma n'ihi na anyị nwere ike ọ gaghị enwe ịntanetị na sava mmepụta.

Ebe a anyị na-aga ịwụnye Cloudera Manager 6.3.1 ntọhapụ. Ebe anyị ga-eji master1 dị ka ihe nkesa repo, anyị na-ebudata ngwugwu ahụ n'ụzọ a kpọtụrụ aha n'okpuru.

Mepụta akwụkwọ ndekọ aha ndị a kpọtụrụ aha na sava master1.

$ sudo mkdir -p /var/www/html/cloudera-repos/cm6

Anyị nwere ike iji ngwa wget budata ngwugwu n'elu http. Yabụ, wụnye wget site na iji iwu dị n'okpuru.

$ sudo yum -y install wget

Na-esote, budata faịlụ tar Cloudera Manager site na iji iwu wget na-esote.

$ wget https://archive.cloudera.com/cm6/6.3.1/repo-as-tarball/cm6.3.1-redhat7.tar.gz

Wepụ faịlụ tar n'ime /var/www/html/cloudera-repos/cm6, anyị emeela master1 dị ka sava weebụ site na ịwụnye http ma anyị nwalere na ihe nchọgharị ahụ.

$ sudo tar xvfz cm6.3.1-redhat7.tar.gz -C /var/www/html/cloudera-repos/cm6 --strip-components=1

Ugbu a, nyochaa na faịlụ Cloudera rpm niile dị na /var/www/html/cloudera-repos/cm6/RPMS/x86_64 directory.

$ cd /var/www/html/cloudera-repos/cm6
$ ll

Mepụta faịlụ /etc/yum.repos.d/cloudera-manager.repo na sava niile dị na ụyọkọ ụyọkọ nwere ọdịnaya ndị a, ebe a master1 (65.0.101.148) bụ sava Weebụ.

[cloudera-repo]
name=cloudera-manager
baseurl=http:///cloudera-repos/cm6/
enabled=1
gpgcheck=0

Ugbu a agbakwunyere ebe nchekwa ahụ, mee iwu dị n'okpuru ka ịlele ebe nchekwa ndị enyere.

$ yum repolist

Gbaa iwu dị n'okpuru ka ịlele ngwungwu metụtara Cloudera niile dị na ebe nchekwa.

$ yum list available | grep cloudera*

Wụnye cloudera-manager-server, cloudera-manager-agent, cloudera-manager-daemons cloudera-manager-server-db-2.

$ sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server cloudera-manager-server-db-2

Gbaa iwu dị n'okpuru ka ịlele ngwungwu Cloudera arụnyere.

$ yum list installed | grep cloudera*

Gbaa iwu dị n'okpuru ka ịmalite cloudera-scm-server-db nke bụ nchekwa data dị n'okpuru iji chekwaa Cloudera Manager na metadata ọrụ ndị ọzọ.

Site na ndabara, Cloudera na-abịa na postgre-sql nke agbakwunyere na Cloudera Manager. Anyị na-etinye nke agbakwunyere, na nchekwa data mpụga nke enwere ike iji. Ọ nwere ike ịbụ Oracle, MySQL, ma ọ bụ PostgreSQL.

$ sudo systemctl start cloudera-scm-server-db

Gbaa iwu dị n'okpuru ka ịlele ọnọdụ nchekwa data.

$ sudo systemctl status cloudera-scm-server-db

Hazie db.properties maka sava Cloudera Manager.

$ vi /etc/cloudera-scm-server/db.properties

Hazie uru dị n'okpuru bụ EMBEDED iji mee Cloudera Manager jiri Database Embedded.

com.cloudera.cmf.db.setupType=EMBEDDED

Gbaa iwu dị n'okpuru ka ịmalite sava Cloudera Manager.

$ sudo systemctl start cloudera-scm-server

Gbaa iwu dị n'okpuru ka ịlele ọkwa nke sava Cloudera Manager.

$ sudo systemctl status cloudera-scm-server

Gbaa iwu dị n'okpuru ka ịmalite wee lelee ọkwa nke onye ọrụ njikwa Cloudera.

$ sudo systemctl start cloudera-scm-agent
$ sudo systemctl status cloudera-scm-agent

Ozugbo sava Cloudera Manager nke ọma na-arụ ọrụ nke ọma, ị nwere ike ịlele WebUI (nbanye nbanye) na ihe nchọgharị site na iji adreesị IP na nọmba ọdụ ụgbọ mmiri 7180 nke bụ nọmba ọdụ ụgbọ mmiri nke Cloudera Manager.

https://65.0.101.148:7180

N'isiokwu a, anyị ahụla usoro nzọụkwụ site na ịwụnye Cloudera Manager na CentOS 7. Anyị ga-ahụ CDH na nrụnye ọrụ ndị ọzọ na isiokwu na-esote.