Best Apache Kafka Online Courses
We’ll take a step-by-step approach to learn all the fundamentals of Apache Kafka.
At the end of this course, you’ll be productive and you’ll know the following:
- The Apache Kafka Ecosystem Architecture
- The Kafka Core Concepts: Topics, Partitions, Brokers, Replicas, Producers, Consumers, and more!
- Launch your own Kafka cluster in no time using native Kafka binaries – Windows / MacOS X / Linux
- Learn and Practice using the Kafka Command Line Interface (CLI)
- Code Producer and Consumers using the Java API
- Real world project using Twitter as a source of data for a producer and ElasticSearch as a sink for our consumer
- Overview of Advanced APIs (Kafka Connect, Kafka Streams)
- Real World Case Studies and Use cases
- Overview of Advanced Kafka for Administrators
- Advanced Topic Configurations
- Annexes (starting a Kafka cluster locally, using Docker, etc…)
Overview of the Course Contents –
Section 1 – Course Introduction: In this section, we are going to see the required pre-requisites for this course and for Apache Kafka Connect. We are also going to talk about the objectives and the structure of the course.
Section 2 – Apache Kafka Connect Concepts: In this section, we will learn about what is Kafka Connect, Apache Kafka Connect architecture, we will talk about Connectors, Configuration, Tasks, Workers. We are also going to learn the difference between the standalone vs distributed mode of the Kafka Connect.
Section 3 – Setup and Launch Kafka Connect Cluster: In this sections, we will learn how to install Docker on your machine and get started with Apache Kafka Connect in the simplest way possible using Docker Compose.
Section 4 – Apache Kafka Connect Data Source – Hands-on: In this section, we will gain some practical experience on Kafka Connect Data Source. We will learn the Kafka Connect Data Source architecture and the list of available connectors. We will have some hands-on practice and learning on File Stream Source Connector both standalone and distributed, and on Twitter Source Connectors…!!!
Section 1: Getting Started
- This section explains about the author and the course objectives.
Section 2: Kafka Architecture
- This section explains about the following:
- Why do you need Kafka ?
- What is Kafka ?
- Kafka Architecture and the different components in the Kafka architecture.
- Role of zookeeper, Kafka Broker, Kafka Cluster, Producers and Consumers.
Section 3: Download Kafka
- This sections explored about the different approaches of downloading the Kafka distribution in your local.
Section 4: Core Internals of Apache Kafka + Hands on
- This section shows you the demo on how to start zookeeper and Kafka broker.
- Detailed explanation about Topics, Partitions, Consumers and Producers.
- Hands on creating topic and how to connect producer and consumers to the created topic using Console Producer and Consumer.
- Hands on about Kafka commit log.
- Kafka Streams – First Look: Let’s get Kafka started and run your first Kafka Streams application, WordCount
- End-to-End Kafka Streams Application : Write the code for the WordCount, bring in the dependencies, build and package your application, and learn how to scale it. This is a complete end to end example
- KStream and KTable Simple Operations: Learn all the stateless operations available for the KStream and KTable API
- Practice Exercise – Favourite Colour: Practice your newly acquired skills by writing your own Kafka Streams application, Favourite Colour. It will be challenging! Includes a Scala version of the example
- KStream and KTable Advanced Operations: Learn all the stateful operations available for the KStream and KTable API
- Exactly Once Semantics – Theory: Learn what EOS (Exactly Once Semantics) is, how Kafka 0.11 enables it, and how to activate them in Kafka Streams
- Exactly Once – Practice Exercise – Bank Balance: Practice your newly acquired knowledge by writing your own Kafka Streams Exactly Once application, to compute a running Bank Balance for your customers
- Testing your Kafka Streams Application: Learn how to test the WordCount Kafka Streams Topology with Kafka Streams v1.1.0
Overview of the Course Contents –
Section 1 – Course Introduction: In this section, we will have introduction to the course. We will talk about what are the course pre-requisites, course objectives, who are the targeted students for this course. You will also have course material for download in this section!
Section 2 – Target Architecture: In this section, we will see the cloud providers such as AWS and explore other alternatives for AWS. We will also cover ZooKeeper Quorum Architecture and Kafka Cluster Architecture in this section.
Section 3 – Zookeeper Quorum Setup: In this sections, we will learn what is ZooKeeper, the Zookeeper role in Kafka Cluster, and ZooKeeper Configuration. We will have hands-on labs on AWS Setup, Zookeeper Single Machine Setup and Zookeeper Quorum Setup.
Section 4 – Kafka Cluster Setup: In this section, we will gain some practical hands-on experience on further AWS Setup, Single Kafka Broker Setup, Multi Kafka Broker Setup etc. Before that we will see Apache Kafka Basics on Kafka Configurations. We will have a demonstration on how to change the Kafka Broker configuration and we will also cover advanced Kafka configurations in this section.
After taking this course, you will become proficient in deploying Kafka in production! You will know all the required setups and installations and Kafka production settings and will be able to set-up Kafka in cluster mode in your own environment!
Best Apache Kafka Books:
- Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale 1st Edition by Neha Narkhede & Gwen Shapira & Todd Palino
- Apache Kafka by Nishant Garg
- Learning Apache Kafka, Second Edition by Nishant Garg
- Apache Kafka 1.0 Cookbook: Over 100 practical recipes on using distributed enterprise messaging to handle real-time databy Raul Estrada
- Apache Kafka Cookbook by Saurabh Minni
- Streaming Architecture: New Designs Using Apache Kafka and MapR Streams 1st Edition by Ted Dunning & Ellen Friedman