Install Hadoop 2.7 on Ubuntu 14.04

Create Hadoop Account

1
2
3

$ sudo useradd -m hadoop -s /bin/bash
$ sudo passwd hadoop
$ sudo adduser hadoop sudo

Setup ssh environemtn

$ sudo apt-get install openssh-server
$ cd ~/.ssh/ 
$ ssh-keygen -t rsa
$ cat id_rsa.pub >> authorized_keys

Instal Java

1	$ sudo apt-get install openjdk-7-jre openjdk-7-jdk

修改~/.bashrc押入

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

1	$ source ~/.bashrc

Install Hadoop

1
2
3

$ sudo tar -zxvf ./hadoop-2.7.0.tar.gz -C /usr/local
$ sudo mv ./hadoop-2.7.0/ ./hadoop
$ sudo chown -R hadoop:hadoop ./hadoop

單機Hadoop建立

執行測試程, ˊ執行時記得不能有output`目錄存在

$ sudo mkdir input
$ sudo cp README.txt input
$ bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.0-sources.jar  org.apache.hadoop.examples.WordCount input output
$ cat output/*

偽分布式Hadoop建立

修改/usr/local/hadoop/etc/hadoop/core-site.xml

<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/usr/local/hadoop/tmp</value>
        <description>Abase for other temporary directories.</description>
    </property>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

修改/usr/local/hadoop/etc/hadoop/hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/usr/local/hadoop/tmp/dfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/usr/local/hadoop/tmp/dfs/data</value>
    </property>
</configuration>

###請動Hadoop
進行Namenode格式化

1	$ bin/hadoop namenode -format

啟動Guardian process

1	s bin/start-dfs.sh

可以連到 http://localhost:50070 來觀察adoop狀態

俄式偽分布式Hadoop

$ bin/hdfs dfs -mkdir -p /user/hadoop
$ bin/hdfs dfs -put README.txt input
$ bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.0-sources.jar  org.apache.hadoop.examples.WordCount input output
$  bin/hdfs dfs -cat output/*

第十三號艦隊