Python On Google Colaboratory

Python On Google Colaboratory

Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum and first released in 1991, Python has a design philosophy that emphasizes code readability, notably using significant whitespace. It provides constructs that enable clear programming on both small and large scales.

Python is a general purpose programming language. Hence, you can use the programming language for developing both desktop and web applications. Also, you can use Python for developing complex scientific and numeric applications. Python is designed with features to facilitate data analysis and visualization.


Link:-https://colab.research.google.com/notebooks/welcome.ipynb

1):-
            print("Hello, World!")

2):-
            if 5 > 2:
                 print("Five is greater than two!")

3):-

      #This is a comment.      print("Hello, World!")

Variables

x = 5

   y = "John"

print(x)

print(y)

-------------------------------------------------------------------------------------

x = "Python is "
y = "awesome"
z =  x + y
print(z)

                         -------------------------------------------------------------------------------------

x = 5

y = 10

print(x + y)

                         -------------------------------------------------------------------------------------

Configuring Hadoop 2 On Linux(RHEL 7) | Hadoop Admin World

In this tutorial i am going to explain how to configure
hadoop 2 on a linux machine. If you have read my
previous post about What is hadoop then you will be
knowing about the general introduction about the
hadoop, like what are its components what is the
architecture and other things about it. You can

check the previous post here

https://administrationinhadoop.blogspot.com

How to Install JAVA 8 on RHEL

Step 1 – Download Latest Java Archive

Sudo -i

Sudo yum install wget

JAVA DOWNLOAD

link in The description

tar xzf jdk-8u202-linux-x64.tar.gz

Step 1.1 – Install Java 8 with Alter

/opt/jdk1.8.0_202

alternatives --install /usr/bin/java java /opt/jdk1.8.0_202/bin/java 2

alternatives --config java

alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_202/bin/jar 2

alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_202/bin/javac 2

alternatives --set jar /opt/jdk1.8.0_202/bin/jar

alternatives --set javac /opt/jdk1.8.0_202/bin/javac

Step 1.2 – Check Installed Java V

java -version

Step 1.3 – Setup Java Environment Variables

export JAVA_HOME=/opt/jdk1.8.0_202/

export JRE_HOME=/opt/jdk1.8.0_202/jre

export PATH=$PATH:/opt/jdk1.8.0_202/bin/jdk1.8.0_202/jre/bin

vi /etc/bashrc

export JAVA_HOME=/opt/jdk1.8.0_202/

export JRE_HOME=/opt/jdk1.8.0_202/jre

export PATH=$PATH:/opt/jdk1.8.0_202/bin/jdk1.8.0_202/jre/bin

Step 2: Creating Hadoop

adduser hadoop

passwd hadoop

su - hadoop

ssh-keygen -t rsa

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

chmod 0600 ~/.ssh/authorized_keys

ssh localhost

exit

Step 3. Downloading Hadoop

cd ~

HADOOP DOWNLOAD

tar xzf hadoop-2.6.5.tar.gz

mv hadoop-2.6.5 hadoop

Step 4. Configure Hadoop Pseudo-Distributed 

Mode

Vi ~/.bashrc

export HADOOP_HOME=/home/hadoop/hadoop
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

source ~/.bashrc

Now edit $HADOOP_HOME/etc/hadoop/hadoop-env.sh file and set  

 JAVA_HOME environment variable.

Change the JAVA path as per install on your syste

export JAVA_HOME=/opt/jdk1.8.0_202/

cd $HADOOP_HOME/etc/hadoop

vi core-site.xml

<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>

vi hdfs-site.xml

<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
 <name>dfs.name.dir</name>
   <value>file:///home/hadoop/hadoopdata/hdfs/namenode</value>
</property>
<property>
 <name>dfs.data.dir</name>
   <value>file:///home/hadoop/hadoopdata/hdfs/datanode</value>
</property>

Edit mapred-site.xml

<property>
 <name>mapreduce.framework.name</name>
  <value>yarn</value>
</property>

Edit yarn-site.xml

<property>
 <name>yarn.nodemanager.aux-services</name>
   <value>mapreduce_shuffle</value>
</property>

4.3. Format Namenode

hdfs namenode -format

Step 5. Start Hadoop Cluster

cd $HADOOP_HOME/sbin/

start-dfs.sh

start-yarn.sh

Hadoop Single Node Cluster Setup On Ubuntu 14.04

Single Node Cluster
Ubuntu 14.04
AWS EC2






Once you logged in AWS click on EC2

To start using Amazon EC2 you will want to launch a virtual server, known as an Amazon EC2 instance.

Now Sear search for ubuntu server 14.04 LTS

                   connecting to your AWS instance  

cd Downloads/

chmod 400 Your_Key.pem

ssh -i "Your_Key.pem" ubuntu@ec2-127-127-127-127.compute-1.amazonaws.com

                     update ubuntu repository 

sudo apt-get update

                              install java in ubuntu

sudo apt-get install openjdk-7-jdk -y

                create key for client server relationship

ssh-keygen

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

download hadoop for ubuntu 14.04

https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz

http://archive.apache.org/dist/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz

wget http://archive.apache.org/dist/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz

tar -xzvf hadoop-1.2.1.tar.gz

sudo mv hadoop-1.2.1 /usr/local/hadoop

configure bash file

nano ~/.bashrc

export HADOOP_PREFIX=/usr/local/hadoop/
export PATH=$PATH:$HADOOP_PREFIX/bin

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 
export PATH=$PATH:$JAVA_HOME 

Setup Configuration Files

The following files will have to be modified to complete the Hadoop setup:

/usr/local/hadoop/conf/hadoop-env.sh /usr/local/hadoop/conf/core-site.xml

/usr/local/hadoop/conf/mapred-site.xml

/usr/local/hadoop/conf/hdfs-site.xml

hadoop-env.sh

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 
export HADOOP_OPTS=-Djava.net.preferIPV4Stack=true 

core-site.xml

<property> 
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value> 
</property>

<property> 
       <name>hadoop.tmp.dir</name> 
       <value>/usr/local/hadoop/tmp</value> 
</property> 

mapred-site.xml

<property> 
       <name>mapred.job.tracker</name> 
       <value>hdfs://localhost:9001</value>
</property> 

hdfs-site.xml

<property>
     <name>dfs.replication</name> 
     <value>1</value>
</property> 

final step 

mkdir /usr/local/hadoop/tmp 

hadoop namenode -format 

start-all.sh

jps