Confluent Developer Skills for Building Apache Kafka®
Nov 26, 2020 2020-11-30 16:35Confluent Developer Skills for Building Apache Kafka®
Course Description
In this three-day, hands-on Apache Kafka developer training course, you will learn how to build an application that can publish data to, and subscribe data from, a Kafka cluster. You will learn the role of Kafka in the modern data distribution pipeline, discuss core Kafka architectural concepts and components, and review the Kafka developer APIs. The course also covers other components in the broader Confluent Platform such as Kafka Connect and Kafka Streams.
This is an official course by Confluent.
What you’ll learn
By the end of the training, you will learn to:
- Write Producers and Consumers to send data to and read data from Kafka
- Integrate Kafka with external systems using Kafka Connect
- Write streaming applications with Kafka Streams & ksqlDB
- Integrate a Kafka client application with Confluent Cloud
-
1.- Fundamentals of Apache Kafka®
-
Explain the value of a *Distributed Event Streaming Platform*
-
Explain how the “log” abstraction enables a distributed event streaming platform
-
Explain the basic concepts of:
- Brokers, Topics, Partitions, and Segments
- Records (a.k.a. Messages, Events)
- Retention Policies
- Producers, Consumers, and Serialization
- Replication
- Kafka Connect
-
-
2.- Producing Messages to Kafka
-
Sketch the high level architecture of a Kafka producer
-
Illustrate key-based partitioning
-
Explain the difference between `acks=0`, `acks=1`, and `acks=all`
-
Configure `delivery.timeout.ms` to control retry behavior
-
Create a custom `producer.properties` file
-
Tune throughput and latency using batching
-
Create a producer with Confluent REST Proxy
-
-
3.- Consuming Messages from Kafka
-
Illustrate how consumer groups and partitions provide scalability and fault tolerance
-
Tune consumers to avoid excessive rebalances
-
Explain the difference between “range” and “round robin” partition assignment strategies
-
Create a custom `consumer.properties` file
-
Use the Consumer API to manage offsets
-
Tune fetch requests
-
Create a consumer with Confluent REST Proxy
-
-
4.- Schema Management in Apache Kafka®
-
Describe Kafka schemas and how they work
-
Use the Confluent Schema Registry to guide schema evolution
-
Write and read messages using schema-enabled Kafka
-
-
5.- Stream Processing with Kafka Streams
-
Compare KStreams to KTables
-
Create a Custom `streams.properties` file
-
Explain what co-partitioning is and why it is important
-
Write an application using the Streams DSL (Domain-Specific Language)
-
-
6.- Data Pipelines with Kafka Connect
-
Explain the motivation for Kafka Connect
-
List commonly used Connectors
-
Explain the differences between standalone and distributed mode
-
Configure and use Kafka Connect
-
-
7.- Event Streaming Apps with ksqlDB
-
Use ksqlDB to filter and transform a stream
-
Write a ksqlDB query that joins two streams or a stream and a table
-
Write a ksqlDB query that aggregates values per key and time window
-
Write Push and Pull queries and explain the differences between them
-
• Create a Connector with ksqlDB
-
-
8.- Design Decisions
-
List ways to avoid large message sizes
-
Decide when to use ksqlDB vs. Kafka Streams vs. Kafka Connect SMTs
-
Explain differences and tradeoffs between processing guarantees
-
Authenticate a client app with a secure Kafka cluster
-
-
9.- Confluent Cloud
-
Explain what “fully-managed” means in the context of Confluent Cloud
-
Authenticate a Kafka client to Confluent Cloud
-
Do basic operations with the `ccloud` CL
-
