最新章節
- Summary
- Message processing semantics
- Out-of-order events
- Level of parallelism
- Data serialization
- Data formats
品牌：中圖公司
上架時間：2021-07-02 18:30:53
出版社：Packt Publishing
本書數字版權由中圖公司提供，并由其授權上海閱文信息技術有限公司制作發行
- Summary 更新時間：2022-07-12 10:38:49
- Message processing semantics
- Out-of-order events
- Level of parallelism
- Data serialization
- Data formats
- External data lookups
- Data sources
- Data and state persistence
- Latency and throughput
- Streaming Application Design Considerations
- Summary
- Best practices
- Adding policies
- Adding Kafka Service to Ranger
- Apache Ranger for authorization
- Understanding Zookeeper authentication
- List ACLs
- Common ACL operations
- Understanding ACL and authorization
- Configuring SASL for Kafka client - producer and consumer
- Configuring SASL for Kafka broker
- Steps to enable SASL/GSSAPI - in Kafka
- Kerberos SASL for authentication
- Configuring SSL for Kafka clients
- Configuring SSL for Kafka Broker
- Steps to enable SSL in Kafka
- Wire encryption using SSL
- An overview of securing Kafka
- Securing Kafka
- Summary
- Consumer metrics
- Broker matrices
- Producer matrices
- Useful Kafka matrices
- Alerting and monitoring
- Kafka and data governance
- Big data and Kafka common usage patterns
- Exactly once delivery
- At most once delivery
- At least once delivery
- Kafka message delivery semantics
- Producer read and consumer write choices
- Appropriate hardware choices
- Managing high volumes in Kafka
- Using Kafka in Big Data Applications
- Summary
- Data migration
- Decommissioning brokers
- Multicluster deployment
- Single cluster deployment
- CPU
- Network
- Hard drives
- Memory
- Replication factor
- Capacity planning goals
- Capacity planning
- Consumer request processing
- Producer request processing
- Metadata request processing
- Replication
- Role of Zookeeper
- Kafka cluster internals
- Kafka Cluster Deployment
- Summary
- Fraud detection application
- IP lookup service
- IP record producer
- Property reader
- Maven dependency of Kafka Streams
- Use case example of Kafka Streams
- KTable
- Kafka Stream word count
- Maven dependency
- Understanding tables and Streams together
- Integrated framework advantages
- Kafka Stream architecture
- Kafka Stream - lightweight Stream processing library
- Using Kafka in Stream processing
- Introduction to Kafka Streams
- Building Streaming Applications Using Kafka Streams
- Summary
- Kafka Connect common use cases
- Introductory examples of using Kafka Connect
- Deep dive into Kafka Connect
- Introducing Kafka Connect
- Considerations for using Kafka in ETL pipelines
- Building ETL Pipelines Using Kafka
- Summary
- Flume
- Kafka Connect
- Gobblin architecture
- Gobblin
- Running Camus
- Camus
- Moving Kafka data to HDFS
- Playing with Avro using Schema Registry
- Kafka Streams
- Understanding Kafka Connect and Kafka Stream
- Deep driving into Confluent architecture
- Introduction to Confluent Platform
- Using Kafka with Confluent Platform
- Summary
- Running the project
- Storm application
- Fraud IP lookup
- Producer code
- Producer
- Use case – log processing in Storm Kafka Hive
- Integrating Apache Kafka with Apache Storm - Scala
- Example
- Integrating Apache Kafka with Apache Storm - Java
- Heron topology architecture
- Heron architecture
- Introduction to Apache Heron
- The concept of a Storm application
- Storm cluster architecture
- Introduction to Apache Storm
- Building Storm Applications with Kafka
- Summary
- Streaming code
- Expose hive table
- Fraud IP lookup
- Producer code
- Property reader
- Producer
- Maven
- Use case log processing - fraud IP detection
- Scala example for direct approach
- Java example for direct approach
- Direct approach
- Scala example for receiver-based integration
- Java example for receiver-based integration
- Disadvantages of receiver-based approach
- Receiver-based integration
- Spark Streaming
- The Spark ecosystem
- Pillars of Spark
- Spark architecture
- Introduction to Spark
- Building Spark Streaming Applications with Kafka
- Summary
- Best practices
- Common message consuming patterns
- Rebalance listeners
- Scala Kafka consumer
- Java Kafka consumer
- Additional configuration
- Committing and polling
- Subscription and polling
- Consumer configuration
- Kafka consumer APIs
- Understanding the responsibilities of Kafka consumers
- Kafka consumer internals
- Deep Dive into Kafka Consumers
- Summary
- Best practices
- Common messaging publishing patterns
- Java Kafka producer example
- Additional producer configuration
- Custom partition
- Producer object and ProducerRecord object
- Kafka Producer APIs
- Kafka producer internals
- Deep Dive into Kafka Producers
- Summary
- Role of Zookeeper
- Message consumers
- Message producers
- Replication and replicated logs
- Message partitions
- Message topics
- Kafka's architecture
- Kafka origins
- Introducing Kafka the Distributed Messaging Platform
- Summary
- Using messaging systems in big data streaming applications
- Advance Queuing Messaging Protocol
- Publish-subscribe messaging system
- Peeking into a point-to-point messaging system
- Understanding messaging systems
- Understanding the principles of messaging systems
- Introduction to Messaging Systems
- Questions
- Piracy
- Errata
- Downloading the color images of this book
- Downloading the example code
- Customer support
- Reader feedback
- Conventions
- Who this book is for
- What you need for this book
- What this book covers
- Preface
- Customer Feedback
- Why subscribe?
- www.PacktPub.com
- About the Reviewer
- About the Authors
- Credits
- Building Data Streaming Applications with Apache Kafka
- Copyright
- Title Page
- Coverpage
- Coverpage
- Title Page
- Copyright
- Building Data Streaming Applications with Apache Kafka
- Credits
- About the Authors
- About the Reviewer
- www.PacktPub.com
- Why subscribe?
- Customer Feedback
- Preface
- What this book covers
- What you need for this book
- Who this book is for
- Conventions
- Reader feedback
- Customer support
- Downloading the example code
- Downloading the color images of this book
- Errata
- Piracy
- Questions
- Introduction to Messaging Systems
- Understanding the principles of messaging systems
- Understanding messaging systems
- Peeking into a point-to-point messaging system
- Publish-subscribe messaging system
- Advance Queuing Messaging Protocol
- Using messaging systems in big data streaming applications
- Summary
- Introducing Kafka the Distributed Messaging Platform
- Kafka origins
- Kafka's architecture
- Message topics
- Message partitions
- Replication and replicated logs
- Message producers
- Message consumers
- Role of Zookeeper
- Summary
- Deep Dive into Kafka Producers
- Kafka producer internals
- Kafka Producer APIs
- Producer object and ProducerRecord object
- Custom partition
- Additional producer configuration
- Java Kafka producer example
- Common messaging publishing patterns
- Best practices
- Summary
- Deep Dive into Kafka Consumers
- Kafka consumer internals
- Understanding the responsibilities of Kafka consumers
- Kafka consumer APIs
- Consumer configuration
- Subscription and polling
- Committing and polling
- Additional configuration
- Java Kafka consumer
- Scala Kafka consumer
- Rebalance listeners
- Common message consuming patterns
- Best practices
- Summary
- Building Spark Streaming Applications with Kafka
- Introduction to Spark
- Spark architecture
- Pillars of Spark
- The Spark ecosystem
- Spark Streaming
- Receiver-based integration
- Disadvantages of receiver-based approach
- Java example for receiver-based integration
- Scala example for receiver-based integration
- Direct approach
- Java example for direct approach
- Scala example for direct approach
- Use case log processing - fraud IP detection
- Maven
- Producer
- Property reader
- Producer code
- Fraud IP lookup
- Expose hive table
- Streaming code
- Summary
- Building Storm Applications with Kafka
- Introduction to Apache Storm
- Storm cluster architecture
- The concept of a Storm application
- Introduction to Apache Heron
- Heron architecture
- Heron topology architecture
- Integrating Apache Kafka with Apache Storm - Java
- Example
- Integrating Apache Kafka with Apache Storm - Scala
- Use case – log processing in Storm Kafka Hive
- Producer
- Producer code
- Fraud IP lookup
- Storm application
- Running the project
- Summary
- Using Kafka with Confluent Platform
- Introduction to Confluent Platform
- Deep driving into Confluent architecture
- Understanding Kafka Connect and Kafka Stream
- Kafka Streams
- Playing with Avro using Schema Registry
- Moving Kafka data to HDFS
- Camus
- Running Camus
- Gobblin
- Gobblin architecture
- Kafka Connect
- Flume
- Summary
- Building ETL Pipelines Using Kafka
- Considerations for using Kafka in ETL pipelines
- Introducing Kafka Connect
- Deep dive into Kafka Connect
- Introductory examples of using Kafka Connect
- Kafka Connect common use cases
- Summary
- Building Streaming Applications Using Kafka Streams
- Introduction to Kafka Streams
- Using Kafka in Stream processing
- Kafka Stream - lightweight Stream processing library
- Kafka Stream architecture
- Integrated framework advantages
- Understanding tables and Streams together
- Maven dependency
- Kafka Stream word count
- KTable
- Use case example of Kafka Streams
- Maven dependency of Kafka Streams
- Property reader
- IP record producer
- IP lookup service
- Fraud detection application
- Summary
- Kafka Cluster Deployment
- Kafka cluster internals
- Role of Zookeeper
- Replication
- Metadata request processing
- Producer request processing
- Consumer request processing
- Capacity planning
- Capacity planning goals
- Replication factor
- Memory
- Hard drives
- Network
- CPU
- Single cluster deployment
- Multicluster deployment
- Decommissioning brokers
- Data migration
- Summary
- Using Kafka in Big Data Applications
- Managing high volumes in Kafka
- Appropriate hardware choices
- Producer read and consumer write choices
- Kafka message delivery semantics
- At least once delivery
- At most once delivery
- Exactly once delivery
- Big data and Kafka common usage patterns
- Kafka and data governance
- Alerting and monitoring
- Useful Kafka matrices
- Producer matrices
- Broker matrices
- Consumer metrics
- Summary
- Securing Kafka
- An overview of securing Kafka
- Wire encryption using SSL
- Steps to enable SSL in Kafka
- Configuring SSL for Kafka Broker
- Configuring SSL for Kafka clients
- Kerberos SASL for authentication
- Steps to enable SASL/GSSAPI - in Kafka
- Configuring SASL for Kafka broker
- Configuring SASL for Kafka client - producer and consumer
- Understanding ACL and authorization
- Common ACL operations
- List ACLs
- Understanding Zookeeper authentication
- Apache Ranger for authorization
- Adding Kafka Service to Ranger
- Adding policies
- Best practices
- Summary
- Streaming Application Design Considerations
- Latency and throughput
- Data and state persistence
- Data sources
- External data lookups
- Data formats
- Data serialization
- Level of parallelism
- Out-of-order events
- Message processing semantics
- Summary 更新時間：2022-07-12 10:38:49
