搭建简易的hadoop集群(完全分布式)，金九银十-EW帮帮网

 保存退出
+ 重启hadoop104

 
```
reboot

```

启动hadoop102

Untitled

确保3台节点启动成功

打开xshell，新建两个会话
- hadoop103会话
点击确定
- hadoop104会话
点击确定
- 连接
选中3个会话，点击连接

弹出如下提示，输入yes即可

搭建hadoop3.x完全分布式集群

集群规划

hadoop102	hadoop103	hadoop104
namenode	resourcemanager	datanode
datanode	datanode	nodemanager
nodemanager	nodemanager	secondarynamenode

配置host映射

点击hadoop102选项卡

Untitled

编辑hosts文件

vim /etc/hosts

192.168.138.102 hadoop102
192.168.138.103 hadoop103
192.168.138.104 hadoop104

Untitled

保存退出

配置集群ssh
- hadoop102配置ssh无密登录
```
ssh-keygen -t rsa
```
一直回车即可
```
ssh-copy-id hadoop102
```
输入yes，提示输入密码的地方输入root，这里也是没有回显的！！

如上操作再将公钥复制到hadoop103、hadoop104
```
ssh-copy-id hadoop103
```
```
ssh-copy-id hadoop104
```
- 分发hosts
```
xsync /etc/hosts
```
- hadoop103配置ssh无密登录
点击hadoop103选项卡

生成公钥
```
ssh-keygen -t rsa
```
如hadoop102的操作，将公钥复制到其他节点
```
ssh-copy-id hadoop102
```
```
ssh-copy-id hadoop103
```
```
ssh-copy-id hadoop104
```
- hadoop104配置ssh无密登录
选择hadoop104选项卡

生成公钥
```
ssh-keygen -t rsa
```
复制公钥到其他节点
```
ssh-copy-id hadoop102
```
```
ssh-copy-id hadoop103
```
```
ssh-copy-id hadoop104
```

上传hadoop安装包

回到hadoop102选项卡

创建目录

mkdir /opt/software && cd /opt/software

上传安装包

开启xftp

Untitled

将下载好的hadoop和jdk移动到右边

Untitled

回到xshell

mkdir /opt/module

tar -zxvf /opt/software/hadoop-3.1.3.tar.gz -C /opt/module/

tar -zxvf /opt/software/jdk-8u401-linux-x64.tar.gz -C /opt/module/

改名

mv /opt/module/jdk1.8.0_401/ /opt/module/jdk

添加JAVA_HOME

添加java环境变量

vim /etc/profile

#JAVA\_HOME
export JAVA\_HOME=/opt/module/jdk
export PATH=$PATH:$JAVA\_HOME/bin

添加HADOOP_HOME

添加hadoop环境变量

#HADOOP\_HOME
export HADOOP\_HOME=/opt/module/hadoop
export PATH=$PATH:$HADOOP\_HOME/bin
export PATH=$PATH:$HADOOP\_HOME/sbin

Untitled

保存退出

刷新环境变量

source /etc/profile

分发环境变量

xsync /etc/profile

修改配置文件

改名

mv /opt/module/hadoop-3.1.3/ /opt/module/hadoop

core-site.xml

vim /opt/module/hadoop/etc/hadoop/core-site.xml

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://hadoop102:8020</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>/opt/module/hadoop/data</value>
        </property>
</configuration>

Untitled

保存退出

hdfs-site.xml

vim /opt/module/hadoop/etc/hadoop/hdfs-site.xml

<configuration>
        <property>
                <name>dfs.replication</name>
                <value>3</value>
        </property>
        <property>
                <name>dfs.namenode.http-address</name>
                <value>hadoop102:9870</value>
        </property>
        <property>
                <name>dfs.namenode.secondary.http-address</name>
                <value>hadoop104:9868</value>
        </property>
</configuration>

Untitled