| Description |
1 online resource (269 pages) |
| Physical Medium |
polychrome |
| Description |
text file |
| Contents |
Cover; Title Page; Copyright; Credits; About the Authors; About the Reviewer; www.PacktPub.com; Customer Feedback; Table of Contents; Preface; Chapter 1: Introduction to Messaging Systems; Understanding the principles of messaging systems; Understanding messaging systems; Peeking into a point-to-point messaging system; Publish-subscribe messaging system; Advance Queuing Messaging Protocol; Using messaging systems in big data streaming applications; Summary; Chapter 2: Introducing Kafka the Distributed Messaging Platform; Kafka origins; Kafka's architecture; Message topics; Message partitions. |
|
Replication and replicated logsMessage producers; Message consumers; Role of Zookeeper; Summary; Chapter 3: Deep Dive into Kafka Producers; Kafka producer internals; Kafka Producer APIs; Producer object and ProducerRecord object; Custom partition; Additional producer configuration; Java Kafka producer example; Common messaging publishing patterns; Best practices; Summary; Chapter 4: Deep Dive into Kafka Consumers; Kafka consumer internals; Understanding the responsibilities of Kafka consumers; Kafka consumer APIs; Consumer configuration; Subscription and polling; Committing and polling. |
|
Additional configurationJava Kafka consumer; Scala Kafka consumer; Rebalance listeners; Common message consuming patterns; Best practices; Summary; Chapter 5: Building Spark Streaming Applications with Kafka; Introduction to Spark ; Spark architecture; Pillars of Spark; The Spark ecosystem; Spark Streaming ; Receiver-based integration; Disadvantages of receiver-based approach; Java example for receiver-based integration; Scala example for receiver-based integration; Direct approach; Java example for direct approach; Scala example for direct approach. |
|
Use case log processing -- fraud IP detectionMaven; Producer ; Property reader; Producer code ; Fraud IP lookup; Expose hive table; Streaming code; Summary; Chapter 6: Building Storm Applications with Kafka; Introduction to Apache Storm; Storm cluster architecture; The concept of a Storm application; Introduction to Apache Heron; Heron architecture ; Heron topology architecture; Integrating Apache Kafka with Apache Storm -- Java; Example; Integrating Apache Kafka with Apache Storm -- Scala; Use case -- log processing in Storm, Kafka, Hive; Producer; Producer code ; Fraud IP lookup. |
|
Running the projectSummary; Chapter 7: Using Kafka with Confluent Platform; Introduction to Confluent Platform; Deep driving into Confluent architecture; Understanding Kafka Connect and Kafka Stream; Kafka Streams; Playing with Avro using Schema Registry; Moving Kafka data to HDFS; Camus ; Running Camus; Gobblin; Gobblin architecture; Kafka Connect; Flume; Summary; Chapter 8: Building ETL Pipelines Using Kafka; Considerations for using Kafka in ETL pipelines; Introducing Kafka Connect; Deep dive into Kafka Connect; Introductory examples of using Kafka Connect; Kafka Connect common use cases. |
| Note |
Summary. |
| Summary |
Design and administer fast, reliable enterprise messaging systems with Apache Kafka About This Book Build efficient real-time streaming applications in Apache Kafka to process data streams of data Master the core Kafka APIs to set up Apache Kafka clusters and start writing message producers and consumers A comprehensive guide to help you get a solid grasp of the Apache Kafka concepts in Apache Kafka with pracitcalpractical examples Who This Book Is For If you want to learn how to use Apache Kafka and the different tools in the Kafka ecosystem in the easiest possible manner, this book is for you. Some programming experience with Java is required to get the most out of this book What You Will Learn Learn the basics of Apache Kafka from scratch Use the basic building blocks of a streaming application Design effective streaming applications with Kafka using Spark, Storm &, and Heron Understand the importance of a low -latency, high- throughput, and fault-tolerant messaging system Make effective capacity planning while deploying your Kafka Application Understand and implement the best security practices In Detail Apache Kafka is a popular distributed streaming platform that acts as a messaging queue or an enterprise messaging system. It lets you publish and subscribe to a stream of records, and process them in a fault-tolerant way as they occur. This book is a comprehensive guide to designing and architecting enterprise-grade streaming applications using Apache Kafka and other big data tools. It includes best practices for building such applications, and tackles some common challenges such as how to use Kafka efficiently and handle high data volumes with ease. This book first takes you through understanding the type messaging system and then provides a thorough introduction to Apache Kafka and its internal details. The second part of the book takes you through designing streaming application using various frameworks and tools such as Apache Spark, Apache Storm, and more. Once you grasp the basics, we will take you through more advanced concepts in Apache Kafka such as capacity planning and security. By the end of this book, you will have all the information you need to be comfortable with using Apache Kafka, and to design efficient streaming data applications with it. Style and approach A step-by -step, comprehensive guide filled with practical and real- world examples Downloading the example code for this book. You can download the example code f ... |
| Local Note |
eBooks on EBSCOhost EBSCO eBook Subscription Academic Collection - North America |
| Subject |
Apache Kafka.
|
|
Real-time data processing.
|
|
Real-time data processing. |
|
Application software -- Development.
|
|
Application software -- Development. |
|
COMPUTERS -- General. |
| Genre/Form |
Electronic books.
|
| Added Author |
Singh, Chanchal.
|
| Other Form: |
Print version: Kumar, Manish. Building Data Streaming Applications with Apache Kafka. Birmingham : Packt Publishing, ©2017 |
| ISBN |
9781787287631 (electronic book) |
|
1787287637 (electronic book) |
|
1787283984 |
|
9781787283985 |
|
