Basics of Kafka - course 42,000 rub. from IBS Training Center, training 24 hours, Date: December 4, 2023.

Topics covered:
1. Review (theory – 1 hour)
What is Kafka.
Examples of successful and unsuccessful use of Kafka.
Kafka and the database: similarities and differences.
Kafka and message queue: similarities and differences.
Kafka storage architecture: Brokers, Topics, Partitions, Segments, Replicas.
Kafka cluster architecture: Zookeeper, partitioning, master and slave replication nodes, ISR.
Clients and APIs.
2. Setting up a Cluster (practice – 1 hour)
Practice 2.1: Running a Kafka cluster using Docker.
Installing Zookeeper.
Installing Kafka brokers.
3. Application development in Java (theory – 2.5 hours, practice – 2.5 hours)
Kafka Producer API: Basics.
Practice 3.1: Sending one message.
Practice 3.2: Sending multiple messages.
Practice 3.3: Sending multiple messages with explicit partitioning.
Practice 3.4: Sending messages without keys.
Kafka Consumer API: basics, consumer groups.
Practice 3.5: One recipient.
Practice 3.6: Multiple recipients in one group.
Additional features: Packets, Repetitions, Idempotency, Transactions, Rebalancing Consumer Groups.

Practice 3.7: Sender without packets and repetitions.
Practice 3.8: Idempotent Sender.
Practice 3.9: Transactional Sender.
Practice 3.10: Sending and Receiving in a Transaction.
Practice 3.11: Receiving-processing-sending in a transaction.
4. External access to the Kafka cluster (theory – 2 hours, practice – 2 hours)
Security: SSL, SASL, Kerberos.
REST Proxy: sending and receiving messages in JSON, binary and Avro formats.
Practice 4.1: Access via REST Proxy.
Schema Registry: schema registry.
Practice 4.2: Using REST Proxy with Schema Registry.
5. Maintenance and monitoring (theory – 1 hour, practice – 1 hour)
Setting up a cluster for different purposes.
Zookeeper: configuration, standalone and quorum mode, CLI.
Practice 5.1: Monitoring a Kafka Cluster with Prometheus.
6. Several clusters (theory – 1 hour, practice – 1 hour)
Why might you need multiple clusters in Kafka?
MirrorMakerAccess.
Practice 6.1: Using MirrorMaker to Synchronize Clusters.
Confluence Replicator.
7. Kafka Connect (theory – 2 hours, practice – 2 hours)
Kafka Connect architecture.
Kafka Connect data sinks.
Practice 7.1: FileStream File Receiver.
Practice 7.2: JDBC Receiver.
Kafka Connect data sources.
Practice 7.3: FileStream Source.
Practice 7.4: JDBC Source.
8. Fundamentals of Kafka Streams (theory – 2 hours, practice – 2 hours)
Kafka Streams: DSL, API, StreamBuilder, creation and implementation of topology.
Demo 8.1: Implementing the Streams topology.
Demo 8.2: Building a Streams application using StreamBuilder.
Kafka Streams device: multi-threading, multiple instances and tasks.
Data schemas in Kafka: serialization to JSON schemas, Avro schemas, schema registry, schema evolution.
Demo 8.3: Using JSON Schemas.
Stateful Processing: State Store, Fault Tolerance for State Stores.
KTable: Data update streams, caching, retrieval speed, aggregation.
Windows in streams: timestamps, floating, jumping, session.
Connection: stream to stream, KTable to KTable, stream to KTable.