دانلود کتاب The Tao of Microservices

فهرست مطالب :

The Tao of Microservices Book Description The Tao of Microservices Richard Rodger Copyright Dedication Brief Table of Contents Table of Contents Preface Acknowledgments About this Book Who should read this book Roadmap About the code Other online resources About the Author About the Cover Illustration Part 1. Building microservices Chapter 1. Brave new world 1.1. The technical debt crisis 1.2. Case study: A microblogging startup 1.2.1. Iteration 0: Posting entries Listing 1.1. Posting an entry Listing 1.2. Listing entries for the user alice Table 1.1. Business activities and their associated message flows Table 1.2. Messages that each microservice sends and receives Figure 1.1. Iteration 0: Messages and services that support posting and listing entries 1.2.2. Iteration 1: A search index Table 1.3. Messages that each microservice sends and receives Figure 1.2. Iteration 1: Adding messages and services that support searching entries Table 1.4. A sequence of small deployment steps 1.2.3. Iteration 2: Simple composition Table 1.5. Modifications to the behavior of entry-store over time Figure 1.3. Iteration 2: An entry store cache implemented by message capture 1.2.4. Iteration 3: Timelines Figure 1.4. Iteration 3: Adding social timelines 1.2.5. Iteration 4: Scaling Figure 1.5. Iteration 3: Scaling by sharding at the message level 1.3. How the monolith betrays the promise of components 1.4. The microservice idea 1.4.1. The core technical principles Transport independence Figure 1.6. The underlying infrastructure that moves messages shouldn’t be known to microservices. Pattern matching Figure 1.7. Introducing support for new user profiles Additivity Figure 1.8. Additivity allows a system to change in discrete, observable steps. 1.5. Practical implications 1.5.1. Specification 1.5.2. Deployment 1.5.3. Security 1.5.4. People 1.6. What you get for your money 1.7. Summary Chapter 2. Services 2.1. Defining microservices 2.2. Case study: The digital edition of a newspaper 2.2.1. The business goals 2.2.2. The informal requirements 2.2.3. A functional breakdown Figure 2.1. A possible component architecture for the newspaper system 2.3. Microservice architectures 2.3.1. The mini web servers architecture 2.4. Diagrams for microservices Figure 2.2. Building the article page 2.5. The microservice dependency tree Figure 2.3. Building the full article page Figure 2.4. Letting other services know that an article has been read 2.5.1. The asynchronous message architecture Figure 2.5. The scatter/gather pattern 2.6. Monolithic projects vs. microservice projects 2.6.1. How microservices change project management 2.6.2. Uniformity makes estimation easier 2.6.3. Disposable code makes for friendlier teams 2.6.4. Homogeneous components allow for heterogeneous configuration 2.6.5. There are different types of code 2.7. The unit of software 2.8. Requirements to messages to services Table 2.1. Mapping requirements to messages Table 2.2. article-page Table 2.3. article-list-page Table 2.4. article Table 2.5. cache Table 2.6. api-gateway Table 2.7. user Table 2.8. adverts Table 2.9. target-content 2.9. Microservice architecture diagrams Figure 2.6. The full newspaper system Note Figure 2.7. Service and network element deployment characteristics 2.9.1. Diagramming message flows Figure 2.8. Message interactions 2.10. Microservices are software components 2.10.1. Encapsulated 2.10.2. Reusable 2.10.3. Well-defined interfaces 2.10.4. Composable 2.10.5. Microservices in practice as components Figure 2.9. Extending the article service without modification Figure 2.10. Introduction of new functionality into the live system 2.11. The internal structure of a microservice 2.12. Summary Chapter 3. Messages 3.1. Messages are first-class citizens Table 3.1. Messages representing activities Figure 3.1. Conceptual levels of system design 3.1.1. Synchronous and asynchronous 3.1.2. When to go synchronous 3.1.3. When to go asynchronous 3.1.4. Thinking distributed from day one 3.1.5. Tactics to limit failure 3.2. Case study: Sales tax rules 3.2.1. The wider context 3.3. Pattern matching 3.3.1. Sales tax: starting simple 3.3.2. Sales tax: handling categories Table 3.2. Pattern to service mapping 3.3.3. Sales tax: going global 3.3.4. Business requirements change, by definition 3.3.5. Pattern matching lowers the cost of refactoring 3.4. Transport independence 3.4.1. A useful fiction: the omnipotent observer 3.5. Message patterns 3.5.1. Core patterns: one message/two services 1/2: Request/response Figure 3.2. Request/response pattern: the calculation of sales tax, where microservice A is the shopping-cart service, microservice B is the sales-tax service, and the message is calculate-sales-tax. The shopping-cart service expects an immediate answ... 1/2: Sidewinder Figure 3.3. Sidewinder pattern: in the e-commerce system, the shopping-cart service may send a checkout message to the delivery service. A recommendation service can observe the checkout messages to understand the purchasing behavior of customers and ... 1/2: Winner-take-all Figure 3.4. Winner-take-all pattern: in the e-commerce system, you can configure the delivery service to work in this mode. For redundancy and fault tolerance, you run multiple instances of the delivery service. It’s important that the physical goods ... 1/2: Fire-and-forget Figure 3.5. Fire-and-forget pattern: the general form of this interaction involves a set of different microservices observing an emitted message. The shopping-cart service emits a checkout message, and various other services react to it: delivery, che... 3.5.2. Core patterns: two messages/two services 2/2: Request/react Figure 3.6. Request/react pattern 2/2: Batch progress reporter Figure 3.7. Batch progress reporter 3.5.3. Core patterns: one message/n services 1/n: Orchestra Figure 3.8. Orchestra 1/n: Scatter/gather Figure 3.9. Scatter/gather 1/n: Multiversion deployment Figure 3.10. Multiversion deployment 1/n: Multi-implementation deployment Figure 3.11. Multi-implementation deployment 3.5.4. Core patterns: m messages/n services 3.5.5. m/n: chain Figure 3.12. Chain 3.5.6. m/n: Tree Figure 3.13. Tree 3.5.7. Scaling messages Lowering latency Increasing throughput 3.6. When messages go bad 3.6.1. The common failure scenarios, and what to do about them Note 3.6.2. Failures dominating the request/response interaction Slow downstream Upstream overload 3.6.3. Failures dominating the sidewinder interaction Lost actions 3.6.4. Failures dominating the winner-take-all interaction Poison messages Guaranteed delivery 3.6.5. Failures dominating the fire-and-forget interaction Emergent behavior Catastrophic collapse 3.7. Summary Chapter 4. Data 4.1. Data doesn’t mean what you think it means 4.1.1. Data is heterogeneous, not homogeneous Figure 4.1. Using different databases for different microservices 4.1.2. Data can be private 4.1.3. Data can be local Figure 4.2. Local user profile data stored only on shard instances 4.1.4. Data can be disposable 4.1.5. Data doesn’t have to be accurate 4.2. Data strategies for microservices 4.2.1. Using messages to expose data 4.2.2. Using composition to manipulate data Figure 4.3. Interaction between the article-page and article services Figure 4.4. Composing the article-page and article services together Figure 4.5. CRUD message flow compared to CQRS message flow Figure 4.6. Direct synchronous interaction with the search-engine service Figure 4.7. Reactive asynchronous interaction with the search-engine service 4.2.3. Using the system configuration to control data Figure 4.8. Eventually consistent data interactions Figure 4.9. Immediately consistent data interactions Figure 4.10. Deployment sequence to replace a shard Figure 4.11. Deployment sequence to split a shard 4.2.4. Using weaker constraints to distribute data 4.3. Rethinking traditional data patterns 4.3.1. Primary keys 4.3.2. Foreign keys 4.3.3. Transactions Figure 4.12. Operations performed to allow a user to read an article Table 4.1. Timeline showing data inconsistency Table 4.2. Timeline with transactions Table 4.3. Timeline with reservations Figure 4.13. Trade-offs when choosing conflict resolution strategies 4.3.4. Transactions aren’t as good as you think they are 4.3.5. Schemas draw down technical debt 4.4. A practical decision guide for microservice data 4.4.1. Greenfield Initial development Production phase 4.4.2. Legacy Initial development Production phase 4.5. Summary Chapter 5. Deployment 5.1. Things fall apart 5.2. Learning from history 5.2.1. Three Mile Island Figure 5.1. High-level components of a nuclear reactor Figure 5.2. A small subset of interactions between high and low levels in the reactor The timeline of the accident Learning from the accident 5.2.2. A model for failure in software systems Figure 5.3. A single-component system, where P0 is the failure rate of component C0 A two-component system Figure 5.4. A two-component system, where Pi is the failure rate of component Ci Multiple components Figure 5.5. Probability of system failure against the number of components, where all components are 99% reliable Figure 5.6. A nonlinear four-component system 5.2.3. Redundancy doesn’t do what you think it does 5.2.4. Change is scary Figure 5.7. A skewed estimator of failure probability Figure 5.8. Estimated reliability of the system when four components change simultaneously 5.3. The centre cannot hold Warning 5.3.1. The cost of perfect software 5.4. Anarchy works 5.5. Microservices and redundancy 5.6. Continuous delivery 5.6.1. Pipeline 5.6.2. Process 5.6.3. Protection 5.7. Running a microservice system 5.7.1. Immutability Figure 5.9. Microservice diagram key Rollback pattern Figure 5.10. Rollback pattern sequence Homeostasis pattern Figure 5.11. Homeostasis pattern sequence History pattern Figure 5.12. History pattern sequence 5.7.2. Automation Canary pattern Figure 5.13. Canary pattern sequence Progressive Canary pattern Figure 5.14. Progressive Canary pattern sequence Bake pattern Figure 5.15. Bake pattern sequence Merge pattern Figure 5.16. Merge pattern sequence Split pattern Figure 5.17. Split pattern sequence 5.7.3. Resilience Slow downstream Upstream overload Lost actions Poison messages Guaranteed delivery Emergent behavior Catastrophic collapse Apoptosis pattern Figure 5.18. Apoptosis pattern sequence Mitosis pattern Figure 5.19. Mitosis pattern sequence Kill Switch pattern Figure 5.20. Kill Switch pattern sequence 5.7.4. Validation Version Update pattern Figure 5.21. Version Update pattern sequence Chaos pattern 5.7.5. Discovery Table 5.1. Service-discovery methods 5.7.6. Configuration 5.7.7. Security 5.7.8. Staging 5.7.9. Development 5.8. Summary Part 2. Running microservices Chapter 6. Measurement 6.1. The limits of traditional monitoring 6.1.1. Classical configurations Figure 6.1. Classical response-time charts 6.1.2. The problem with averages Figure 6.2. Histogram chart showing response times Figure 6.3. Histogram chart showing response times, with caching causing two peaks 6.1.3. Using percentiles Figure 6.4. Histogram chart (no cache) showing response times, with 90th percentile Figure 6.5. Histogram chart (with cache) showing response times, with 90th percentile Figure 6.6. Time series chart of average and 90th percentile response times 6.1.4. Microservice configurations 6.1.5. The power of scatterplots Figure 6.7. Scatterplot of weight (in kilograms) versus height (in centimeters) Figure 6.8. Current-behavior scatterplot of microservice response times 6.1.6. Building a dashboard 6.2. Measurements for microservices 6.2.1. The business layer 6.2.2. The message layer Universal measurements Measuring synchronous and consumed messages Note Table 6.1. Outbound message measures Figure 6.9. Synchronous message outbound send and response counts Figure 6.10. Outbound response failure counts over time Figure 6.11. Current response-time behavior by message family Table 6.2. Inbound message measures Figure 6.12. Different message transports have different delivery speeds. Measuring synchronous and observed messages Table 6.3. Inbound message measures Measuring asynchronous and observed messages Table 6.4. Outbound message measures Table 6.5. Inbound message measures Measuring asynchronous and consumed messages 6.2.3. The service layer Where’s the problem? Upstream problems Local problems Downstream problems Service-instance health Figure 6.13. Current behavior of service instances of the same type Figure 6.14. Categorical scatterplot of service-instance response times 6.3. The power of invariants Figure 6.15. Ratio of message-flow rates 6.3.1. Finding invariants from the business logic Figure 6.16. Messages caused by posting an entry 6.3.2. Finding invariants from the system architecture Actor-style invariants Figure 6.17. Actor/service interaction Publish/subscribe-style invariants Figure 6.18. Publish/subscribe service interaction Chain-style invariants Figure 6.19. Service/chain interaction Tree-style invariants Figure 6.20. Service/tree interaction 6.3.3. Visualizing invariants 6.3.4. System discovery Distributed tracing Figure 6.21. Distributed trace of an entry post over microservices Distributed logging Error capture 6.3.5. Synthetic validation 6.4. Summary Chapter 7. Migration 7.1. A classic e-commerce example 7.1.1. The legacy architecture Figure 7.1. The basic legacy e-commerce architecture Table 7.1. External integrations of the e-commerce system 7.1.2. The software delivery process 7.2. Changing the goal posts 7.2.1. The practical application of politics 7.3. Starting the journey 7.4. The strangler tactic 7.4.1. Partial proxying 7.4.2. What to do when you can’t migrate 7.4.3. The greenfield tactic The e-commerce greenfield scenario 7.4.4. The macroservice tactic Macroservice extraction 7.5. The strategy of refinement 7.6. Moving from the general to the specific 7.6.1. Adding features to the product page Figure 7.2. The evolving product microservice family Figure 7.3. Number of product types that each microservice handles 7.6.2. Adding features to the shopping cart Listing 7.1. Add coupon item message 7.6.3. Handling cross-cutting concerns 7.7. Summary Chapter 8. People 8.1. Dealing with institutional politics 8.1.1. Accepting hard constraints 8.1.2. Finding sponsors 8.1.3. Building alliances 8.1.4. Value-focused delivery 8.1.5. Acceptable error rates 8.1.6. Dropping features Figure 8.1. Value versus complexity matrix 8.1.7. Stop abstracting 8.1.8. Deprogramming 8.1.9. External validation 8.1.10. Team solidarity 8.1.11. Respect the organization 8.2. The politics of microservices 8.2.1. Who owns what? 8.2.2. Who’s on call? 8.2.3. Who decides what to code? Autonomy and values Everybody codes The Tower of Babel Big-picture thinking Disposable code Valuable mistakes 8.3. Summary Chapter 9. Case study: Nodezoo.com 9.1. Design 9.1.1. What are the business requirements? 9.1.2. What are the messages? BR1: Performance BR2: Messages relating to search BR3: Messages relating to module information BR3.1: Messages relating to sources BR4: Messages relating to module changes 9.1.3. What are the services? Message pattern: role:search,cmd:search Figure 9.1. The role:search, cmd:search interaction Message pattern: role:search,cmd:insert Figure 9.2. The role:search, cmd:insert interaction Message pattern: role:info,cmd:get Figure 9.3. The role:info,cmd:get interaction Message patterns: role:info,need:part and role:info,collect:part Figure 9.4. The role:info,need/collect:part interactions Message patterns: role:npm,cmd:get and role:npm,cmd:query Figure 9.5. The role:npm,cmd:* interactions Message patterns: role:github,cmd:get and role:github,cmd:query Message patterns: role:info,event:change Figure 9.6. The role:info, event:change interaction 9.2. Deliver 9.2.1. Iteration 1: Local development Service: web Listing 9.1. nodezoo-web/web.js: home page route Listing 9.2. nodezoo-web/web.js: api/query route Listing 9.3. nodezoo-web/srv/web-dev.js: development-service execution Service: search Listing 9.4. nodezoo-search/search.js: message list Listing 9.5. nodezoo-search/search.js: handler for role:search,cmd:search Listing 9.6. nodezoo-search/search.js: handler for role:search,cmd:insert Listing 9.7. nodezoo-search/srv/search-dev.js: development service execution Service: info Listing 9.8. nodezoo-info/info.js: message list Listing 9.9. nodezoo-info/info.js: handler for role:info,cmd:get Listing 9.10. nodezoo-info/info.js: handler for role:info,collect:part Listing 9.11. nodezoo-info/srv/info-dev.js: development service execution Service: npm Listing 9.12. nodezoo-npm/npm.js: message list Listing 9.13. nodezoo-npm/npm.js: handler for role:npm,cmd:get Listing 9.14. nodezoo-npm/npm.js: handler for role:npm,cmd:query Listing 9.15. nodezoo-npm/npm.js: intercept for role:entity,cmd:save,name:npm Listing 9.16. nodezoo-info/srv/npm-dev.js: development service execution Practicalities Figure 9.7. The system after iteration 1 Listing 9.17. nodezoo-system/fuge/fuge.yml: fuge configuration 9.2.2. Iteration 2: Testing, staging, and risk measurement Docker configuration Staged services Listing 9.18. nodezoo-web/srv/web-stage.js: staged service execution Listing 9.19. nodezoo-info/srv/info-stage.js: staged service execution Listing 9.20. nodezoo-npm/srv/npm-stage.js: staged service execution Validation services Listing 9.21. nodezoo-repl/srv/repl-stage.js: staged service execution Figure 9.8. Deployment-risk score 9.2.3. Iteration 3: The path to production The service-execution scripts Listing 9.22. nodezoo-repl/srv/repl-prod.js: production execution script Listing 9.23. nodezoo-system/prod/nodezoo.yml: repl service configuration Listing 9.24. nodezoo-web/srv/web-prod.js: production execution script Listing 9.25. nodezoo-system/prod/nodezoo.yml: Kubernetes configuration Listing 9.26. nodezoo-system/prod/nodezoo.yml: repl service configuration Listing 9.27. nodezoo-repl/srv/search-prod.js: production execution script Listing 9.28. nodezoo-info/srv/info-prod.js: production execution script Listing 9.29. nodezoo-npm/srv/npm-prod.js: production execution script 9.2.4. Iteration 4: Enhancing and adapting Adding a data source Listing 9.30. nodezoo-github/github.js: business logic code Listing 9.31. nodezoo-github/srv/github-prod.js: execution script Adding a feature Listing 9.32. nodezoo-web/web.js: API code Listing 9.33. nodezoo-search/search.js: API code Listing 9.34. nodezoo-suggest/suggest.js: business logic Listing 9.35. nodezoo-suggest/ssrv/suggest-prod.js: production execution script Extending the system Listing 9.36. nodezoo-updater/updater.js: production execution script 9.2.5. Iteration 5: Monitoring and debugging Figure 9.9. Message-flow rates and invariants Figure 9.10. Current behavior for message response times Tracing Figure 9.11. Dynamic service relationships Figure 9.12. Trace of the search message flow over services Logging Figure 9.13. Searching the service logs Debugging 9.2.6. Iteration 6: Scaling and performance The web service The search service The info service The npm and github services The suggest service Everything else 9.3. Brave new world Core message patterns: one message/two services 1/2: Request/Response 1/2: Sidewinder Core message patterns: one message/two services (continued) 1/2: Winner-take-all 1/2: Fire-and-forget Index SYMBOL A B C D E F G H I J K L M N O P Q R S T U V W Z List of Figures List of Tables List of Listings