- Apache Kafka Tutorial
- Apache Kafka - Home
- Apache Kafka - Introduction
- Apache Kafka - Fundamentals
- Apache Kafka - Cluster Architecture
- Apache Kafka - Work Flow
- Apache Kafka - Installation Steps
- Apache Kafka - Basic Operations
- Simple Producer Example
- Consumer Group Example
- Integration With Storm
- Integration With Spark
- Real Time Application(Twitter)
- Apache Kafka - Tools
- Apache Kafka - Applications
- Apache Kafka Useful Resources
- Apache Kafka - Quick Guide
- Apache Kafka - Useful Resources
- Apache Kafka - Discussion
Apache Kafka - Installation Steps
Following are the steps for installing Java on your machine.
Step 1 - Verifying Java Installation
Hopefully you have already installed java on your machine right now, so you just verify it using the following command.
$ java -version
If java is successfully installed on your machine, you could see the version of the installed Java.
Step 1.1 - Download JDK
If Java is not downloaded, please download the latest version of JDK by visiting the following link and download latest version.
http://www.oracle.com/technetwork/java/javase/downloads/index.htmlNow the latest version is JDK 8u 60 and the file is “jdk-8u60-linux-x64.tar.gz”. Please download the file on your machine.
Step 1.2 - Extract Files
Generally, files being downloaded are stored in the downloads folder, verify it and extract the tar setup using the following commands.
$ cd /go/to/download/path $ tar -zxf jdk-8u60-linux-x64.gz
Step 1.3 - Move to Opt Directory
To make java available to all users, move the extracted java content to usr/local/java
/ folder.
$ su password: (type password of root user) $ mkdir /opt/jdk $ mv jdk-1.8.0_60 /opt/jdk/
Step 1.4 - Set path
To set path and JAVA_HOME variables, add the following commands to ~/.bashrc file.
export JAVA_HOME =/usr/jdk/jdk-1.8.0_60 export PATH=$PATH:$JAVA_HOME/bin
Now apply all the changes into current running system.
$ source ~/.bashrc
Step 1.5 - Java Alternatives
Use the following command to change Java Alternatives.
update-alternatives --install /usr/bin/java java /opt/jdk/jdk1.8.0_60/bin/java 100
Step 1.6 − Now verify java using verification command (java -version) explained in Step 1.
Step 2 - ZooKeeper Framework Installation
Step 2.1 - Download ZooKeeper
To install ZooKeeper framework on your machine, visit the following link and download the latest version of ZooKeeper.
http://zookeeper.apache.org/releases.htmlAs of now, latest version of ZooKeeper is 3.4.6 (ZooKeeper-3.4.6.tar.gz).
Step 2.2 - Extract tar file
Extract tar file using the following command
$ cd opt/ $ tar -zxf zookeeper-3.4.6.tar.gz $ cd zookeeper-3.4.6 $ mkdir data
Step 2.3 - Create Configuration File
Open Configuration File named conf/zoo.cfg
using the command vi “conf/zoo.cfg” and all the following parameters to set as starting point.
$ vi conf/zoo.cfg tickTime=2000 dataDir=/path/to/zookeeper/data clientPort=2181 initLimit=5 syncLimit=2
Once the configuration file has been saved successfully and return to terminal again, you can start the zookeeper server.
Step 2.4 - Start ZooKeeper Server
$ bin/zkServer.sh start
After executing this command, you will get a response as shown below −
$ JMX enabled by default $ Using config: /Users/../zookeeper-3.4.6/bin/../conf/zoo.cfg $ Starting zookeeper ... STARTED
Step 2.5 - Start CLI
$ bin/zkCli.sh
After typing the above command, you will be connected to the zookeeper server and will get the below response.
Connecting to localhost:2181 ................ ................ ................ Welcome to ZooKeeper! ................ ................ WATCHER:: WatchedEvent state:SyncConnected type: None path:null [zk: localhost:2181(CONNECTED) 0]
Step 2.6 - Stop Zookeeper Server
After connecting the server and performing all the operations, you can stop the zookeeper server with the following command −
$ bin/zkServer.sh stop
Now you have successfully installed Java and ZooKeeper on your machine. Let us see the steps to install Apache Kafka.
Step 3 - Apache Kafka Installation
Let us continue with the following steps to install Kafka on your machine.
Step 3.1 - Download Kafka
To install Kafka on your machine, click on the below link −
https://www.apache.org/dyn/closer.cgi?path=/kafka/0.9.0.0/kafka_2.11-0.9.0.0.tgzNow the latest version i.e., – kafka_2.11_0.9.0.0.tgz will be downloaded onto your machine.
Step 3.2 - Extract the tar file
Extract the tar file using the following command −
$ cd opt/ $ tar -zxf kafka_2.11.0.9.0.0 tar.gz $ cd kafka_2.11.0.9.0.0
Now you have downloaded the latest version of Kafka on your machine.
Step 3.3 - Start Server
You can start the server by giving the following command −
$ bin/kafka-server-start.sh config/server.properties
After the server starts, you would see the below response on your screen −
$ bin/kafka-server-start.sh config/server.properties [2016-01-02 15:37:30,410] INFO KafkaConfig values: request.timeout.ms = 30000 log.roll.hours = 168 inter.broker.protocol.version = 0.9.0.X log.preallocate = false security.inter.broker.protocol = PLAINTEXT ……………………………………………. …………………………………………….
Step 4 - Stop the Server
After performing all the operations, you can stop the server using the following command −
$ bin/kafka-server-stop.sh config/server.properties
Now that we have already discussed the Kafka installation, we can learn how to perform basic operations on Kafka in the next chapter.
To Continue Learning Please Login
Login with Google