دانلود کتاب Heron Streaming: Fundamentals, Applications, Operations, and Insights

فهرست مطالب :

Foreword
Preface
Who This Book Is For
How This Book Is Organized
What You Need for This Book
Typographical Conventions
Acknowledgments
Contents
About the Authors
Part I Heron Fundamentals
1 Stream Processing
1.1 Big Data Processing
1.1.1 Lambda Architecture
1.1.1.1 Batch Processing Layer
1.1.1.2 Stream Processing Layer
1.1.1.3 Serving Layer
1.1.2 Kappa Architecture
1.2 Big Data Stream Processing
1.3 From Apache Storm to Apache Heron (Incubating)
1.3.1 Motivation for Heron
1.3.2 Heron Design Goals
1.3.3 Join the Apache Heron (Incubating) Community
1.4 Stream Processing Tools
1.5 Summary
References
2 Heron Basics
2.1 Topology Data Model
2.1.1 Topology
2.1.2 Spout
2.1.3 Bolt
2.1.4 Grouping
2.2 Heron Architecture and Components
2.2.1 Cluster-Level Components (Six Components)
2.2.1.1 Scheduler
2.2.1.2 State Manager
2.2.1.3 Uploader
2.2.1.4 Heron CLI
2.2.1.5 Heron Tracker
2.2.1.6 Heron UI
2.2.2 Topology-Level Components (Four Components)
2.2.2.1 Heron Instance
2.2.2.2 Stream Manager
2.2.2.3 Topology Master
2.2.2.4 Metrics Manager
2.3 Submission Process and Failure Handling
2.4 Submit the First Topology
2.4.1 Preparation
2.4.2 Install the Heron Client
2.4.3 Heron Example Topologies
2.4.4 Submit the Topology JAR File
2.4.5 Observe the Running Topology
2.5 Summary
References
3 Study Heron Code
3.1 Code Languages
3.2 Requirements for Compiling
3.3 Prepare the Compiling Environment
3.4 Source Organization
3.4.1 Directory Organization
3.4.2 Bazel Perspective
3.5 Compile Heron
3.6 Examine Compiling Results
3.6.1 Examine the API
3.6.2 Examine Packages
3.7 Run Tests
3.7.1 Unit Test
3.7.2 Integration Test
3.8 Summary
References
Part II Write Heron Topologies
4 Migrate Storm Topology to Heron
4.1 Prepare the Storm Topology Code
4.1.1 Examine the Storm Topology Code
4.1.2 Examine the Storm Flux Code
4.2 Migrate the Storm Topology Code to a HeronTopology Project
4.2.1 Adjust the Topology Java Code
4.2.2 Adjust the Project File pom.xml
4.2.2.1 Add Dependency
4.2.2.2 Build with Dependencies
4.2.3 Compile the Topology JAR File
4.3 Migrate Storm Flux to Heron ECO
4.4 Summary
References
5 Write Topology Code
5.1 Before Writing Code
5.1.1 Design Topology
5.1.2 Choose a Heron API
5.2 Write Topology in Java
5.2.1 Code the Topology
5.2.1.1 Code Main
5.2.1.2 Code Spout
5.2.1.3 Code Bolt
5.2.2 Understand Tuple Flow
5.2.2.1 How Tuple Is Constructed
5.2.2.2 How Tuple Is Routed
5.3 Write Topology in Python
5.3.1 Code Main
5.3.2 Code Spout
5.3.3 Code Bolt
5.3.4 Compile and Run
5.4 Summary
Reference
6 Heron Topology Features
6.1 Delivery Semantics
6.1.1 At-Least-Once
6.1.2 Effectively-Once
6.1.2.1 Requirements for Effectively-Once
6.1.2.2 Exactly-Once Versus Effectively-Once
6.1.2.3 Stateful Topologies
6.1.2.4 Implement Effectively-Once
6.2 Windowing
6.2.1 Windowing Concepts
6.2.2 Windowing Example
6.3 Summary
Reference
7 Heron Streamlet API
7.1 Streamlet API Concepts
7.1.1 Streamlets
7.1.2 Operations
7.2 Write a Processing Graph with the Java Streamlet API
7.2.1 Sources
7.2.2 Sinks
7.2.3 Transform: Filter, Map, FlatMap
7.2.4 Partitioning
7.2.5 Clone and Union
7.2.6 Reduce by Key and Window
7.2.7 Join
7.2.8 Configuration
7.3 Write a Processing Graph with the Python Streamlet API
7.3.1 Source Generator
7.3.2 Processing Graph Construction
7.4 Write a Processing Graph with the Scala Streamlet API
7.4.1 Install sbt
7.4.2 Source Directory
7.4.3 Compose Processing Graph
7.4.4 Examine the JAR File
7.5 Summary
References
Part III Operate Heron Clusters
8 Manage a Topology
8.1 Install Heron Client
8.1.1 What Is Inside heron-core.tar.gz
8.1.2 YAML Configuration
8.2 Run Topology
8.2.1 Topology Life Cycle
8.2.2 Submit Topology
8.2.3 Observe the Topology Running Status
8.3 Explore the ``heron\'\' Command
8.3.1 Common Arguments and Optional Flags
8.3.2 Explore ``heron submit\'\' Options
8.3.3 Kill Topology
8.3.4 Activate and Deactivate Topology
8.3.5 Restart Topology
8.3.6 Update Topology
8.4 Summary
References
9 Manage Multiple Topologies
9.1 Install Heron Tools
9.2 Heron Tracker
9.3 Heron UI
9.4 Heron Explorer
9.5 Summary
Reference
Part IV Heron Insights
10 Explore Heron
10.1 Heron Processes
10.1.1 Java Processes
10.1.2 C++ Processes
10.1.3 Python Processes
10.2 State Manager
10.3 Heron Scheduler
10.3.1 Three Plans: Logical, Packing, and Physical
10.3.2 Restart Dead Processes
10.4 Data Flow
10.4.1 Capture Packets
10.4.2 Communication Primitive
10.5 Metrics System
10.5.1 File Sink
10.5.2 MetricsCache Manager Sink and TopologyMaster Sink
10.6 Summary
11 Extending the Heron Metrics Sink
11.1 Time Series
11.2 Metrics Category
11.3 Customize a Metrics Sink
11.3.1 Metrics Sink SPI
11.3.2 Metrics Sink Configuration
11.4 MySQL Metrics Sink
11.4.1 Implement IMetricsSink
11.4.2 Configure the Metrics Sink
11.4.3 Observe Metrics
11.5 Summary
Reference
12 Extending Heron Scheduler
12.1 Scheduler SPI
12.1.1 ILauncher and IScheduler Work Together
12.1.2 ILauncher
12.1.3 IScheduler
12.2 Timeout Scheduler
12.2.1 Timeout Launcher
12.2.1.1 Prepare Container
12.2.1.2 Launch by Service
12.2.1.3 Launch by Library
12.2.2 Timeout Scheduler
12.2.2.1 Abstract Parent Class
12.2.2.2 Service Mode
12.2.2.3 Library Mode
12.2.2.4 Service Mode Versus Library Mode
12.3 Summary
13 Heron Is Evolving
13.1 Dhalion and Health Manager
13.1.1 Dhalion
13.1.2 Health Manager
13.2 Deploy Mode (API Server)
13.3 Cloud-Native Heron
13.4 Summary
Reference
Index