دانلود کتاب Hands-On Software Engineering with Golang: Move beyond basic programming to design and build reliable software with clean code

فهرست مطالب :

Title Page Copyright and Credits Dedication About Packt Contributors Table of Contents Preface Section 1: Software Engineering and the Software Development Life Cycle Chapter 1: A Bird's-Eye View of Software Engineering What is software engineering? Types of software engineering roles The role of the software engineer (SWE) The role of the software development engineer in test (SDET) The role of the site reliability engineer (SRE) The role of the release engineer (RE) The role of the system architect A list of software development models that all engineers should know Waterfall Iterative enhancement Spiral Agile Lean Eliminate waste Create knowledge Defer commitment Build in quality Deliver fast Respect and empower people See and optimize the whole Scrum Scrum roles Essential Scrum events Kanban DevOps The CAMS model The three ways model Summary Questions Further reading Section 2: Best Practices for Maintainable and Testable Go Code Chapter 2: Best Practices for Writing Clean and Maintainable Go Code The SOLID principles of object-oriented design Single responsibility Open/closed principle Liskov substitution Interface segregation Dependency inversion Applying the SOLID principles Organizing code into packages Naming conventions for Go packages Circular dependencies Breaking circular dependencies via implicit interfaces Sometimes, code repetition is not a bad idea! Tips and tools for writing lean and easy-to-maintain Go code Optimizing function implementations for readability Variable naming conventions Using Go interfaces effectively Zero values are your friends Using tools to analyze and manipulate Go programs Taking care of formatting and imports (gofmt, goimports) Refactoring code across packages (gorename, gomvpkg, fix) Improving code quality metrics with the help of linters Summary Questions Further reading Chapter 3: Dependency Management What's all the fuss about software versioning? Semantic versioning Comparing semantic versions Applying semantic versioning to Go packages Managing the source code for multiple package versions Single repository with versioned folders Single repository – multiple branches Vendoring – the good, the bad, and the ugly Benefits of vendoring dependencies Is vendoring always a good idea? Strategies and tools for vendoring dependencies The dep tool The Gopkg.toml file The Gopkg.lock file Go modules – the way forward Fork packages Summary Questions Further reading Chapter 4: The Art of Testing Technical requirements Unit testing Mocks, stubs, fakes, and spies – commonalities and differences Stubs and spies! Mocks Introducing gomock Exploring the details of the project we want to write tests for Leveraging gomock to write a unit test for our application Fake objects Black-box versus white-box testing for Go packages – an example The services behind the facade Writing black-box tests Boosting code coverage via white-box tests Table-driven tests versus subtests Table-driven tests Subtests The best of both worlds Using third-party testing frameworks Integration versus functional testing Integration tests Functional tests Functional tests part deux – testing in production! Smoke tests Chaos testing – breaking your systems in fun and interesting ways! Tips and tricks for writing tests Using environment variables to set up or skip tests Speeding up testing for local development Excluding classes of tests via build flags This is not the output you are looking for – mocking calls to external binaries Testing timeouts is easy when you have all the time in the world! Summary Questions Further reading Section 3: Designing and Building a Multi-Tier System from Scratch Chapter 5: The Links 'R'; Us Project System overview – what are we going to be building? Selecting an SDLC model for our project Iterating faster using an Agile framework Elephant carpaccio – how to iterate even faster! Requirements analysis Functional requirements User story – link submission User story – search User story – crawl link graph User story – calculate PageRank scores User story – monitor Links 'R' Us health Non-functional requirements Service-level objectives Security considerations Being good netizens System component modeling The crawler The link filter The link fetcher The content extractor The link extractor The content indexer The link provider The link graph The PageRank calculator The metrics store The frontend Monolith or microservices? The ultimate question Summary Questions Further reading Chapter 6: Building a Persistence Layer Technical requirements Running tests that require CockroachDB Running tests that require Elasticsearch Exploring a taxonomy of database systems Key-value stores Relational databases NoSQL databases Document databases Understanding the need for a data layer abstraction Designing the data layer for the link graph component Creating an ER diagram for the link graph store Listing the required set of operations for the data access layer Defining a Go interface for the link graph Partitioning links and edges for processing the graph in parallel Iterating Links and Edges Verifying graph implementations using a shared test suite Implementing an in-memory graph store Upserting links Upserting edges Looking up links Iterating links/edges Removing stale edges Setting up a test suite for the graph implementation Scaling across with a CockroachDB-backed graph implementation Dealing with DB migrations An overview of the DB schema for the CockroachDB implementation Upserting links Upserting edges Looking up links Iterating links/edges Removing stale edges Setting up a test suite for the CockroachDB implementation Designing the data layer for the text indexer component A model for indexed documents Listing the set of operations that the text indexer needs to support Defining the Indexer interface Verifying indexer implementations using a shared test suite An in-memory Indexer implementation using bleve Indexing documents Looking up documents and updating their PageRank score Searching the index Iterating the list of search results Setting up a test suite for the in-memory indexer Scaling across an Elasticsearch indexer implementation Creating a new Elasticsearch indexer instance Indexing and looking up documents Performing paginated searches Updating the PageRank score for a document Setting up a test suite for the Elasticsearch indexer Summary Questions Further reading Chapter 7: Data-Processing Pipelines Technical requirements Building a generic data-processing pipeline in Go Design goals for the pipeline package Modeling pipeline payloads Multistage processing Stageless pipelines – is that even possible? Strategies for handling errors Accumulating and returning all errors Using a dead-letter queue Terminating the pipeline's execution if an error occurs Synchronous versus asynchronous pipelines Synchronous pipelines Asynchronous pipelines Implementing a stage worker for executing payload processors FIFO Fixed and dynamic worker pools 1-to-N broadcasting Implementing the input source worker Implementing the output sink worker Putting it all together – the pipeline API Building a crawler pipeline for the Links 'R' Us project Defining the payload for the crawler Implementing a source and a sink for the crawler Fetching the contents of graph links  Extracting outgoing links from retrieved webpages Extracting the title and text from retrieved web pages Inserting discovered outgoing links to the graph Indexing the contents of retrieved web pages Assembling and running the pipeline Summary Questions Further reading Chapter 8: Graph-Based Data Processing Technical requirements Exploring the Bulk Synchronous Parallel model Building a graph processing system in Go Queueing and delivering messages The Message interface Queues and message iterators Implementing an in-memory, thread-safe queue Modeling the vertices and edges of graphs Defining the Vertex and Edge types Inserting vertices and edges into the graph Sharing global graph state through data aggregation Defining the Aggregator interface Registering and looking up aggregators Implementing a lock-free accumulator for float64 values Sending and receiving messages Implementing graph-based algorithms using compute functions Achieving vertical scaling by executing compute functions in parallel Orchestrating the execution of super-steps Creating and managing Graph instances Solving interesting graph problems Searching graphs for the shortest path The sequential Dijkstra algorithm Leveraging a gossip protocol to run Dijkstra in parallel Graph coloring A sequential greedy algorithm for coloring undirected graphs Exploiting parallelism for undirected graph coloring Calculating PageRank scores The model of the random surfer An iterative approach to PageRank score calculation Reaching convergence – when should we stop iterating? Web graphs in the real world – dealing with dead ends Defining an API for the PageRank calculator Implementing a compute function to calculate PageRank scores Summary Further reading Chapter 9: Communicating with the Outside World Technical requirements Designing robust, secure, and backward-compatible REST APIs Using human-readable paths for RESTful resources Controlling access to API endpoints Basic HTTP authentication Securing TLS connections from eavesdropping Authenticating to external service providers using OAuth2 Dealing with API versions Including the API version as a route prefix Negotiating API versions via HTTP Accept headers Building RESTful APIs in Go Building RPC-based APIs with the help of gRPC Comparing gRPC to REST Defining messages using protocol buffers Defining messages Versioning message definitions Representing collections Modeling field unions The Any type Implementing RPC services Unary RPCs Server-streaming RPCs Client-streaming RPCs Bi-directional streaming RPCs Security considerations for gRPC APIs Decoupling Links 'R' Us components from the underlying data stores Defining RPCs for accessing a remote link-graph instance Defining RPCs for accessing a text-indexer instance Creating high-level clients for accessing data stores over gRPC  Summary Questions Further reading Chapter 10: Building, Packaging, and Deploying Software Technical requirements Building and packaging Go services using Docker Benefits of containerization Best practices for dockerizing Go applications Selecting a suitable base container for your application A gentle introduction to Kubernetes Peeking under the hood Summarizing the most common Kubernetes resource types Running a Kubernetes cluster on your laptop! Building and deploying a monolithic version of Links 'R' Us Distributing computation across application instances Carving the UUID space into non-overlapping partitions Assigning a partition range to each pod Building wrappers for the application services The crawler service The PageRank calculator service Serving a fully functioning frontend to users Specifying the endpoints for the frontend application Performing searches and paginating results Generating convincing summaries for search results Highlighting search keywords Orchestrating the execution of individual services Putting everything together Terminating the application in a clean way Dockerizing and starting a single instance of the monolith Deploying and scaling the monolith on Kubernetes Setting up the required namespaces Deploying CockroachDB and Elasticsearch using Helm Deploying Links 'R' Us Summary Questions Further reading Section 4: Scaling Out to Handle a Growing Number of Users Chapter 11: Splitting Monoliths into Microservices Technical requirements Monoliths versus service-oriented architectures Is there something inherently wrong with monoliths? Microservice anti-patterns and how to deal with them Monitoring the state of your microservices Tracing requests through distributed systems The OpenTracing project Stepping through a distributed tracing example The provider service The aggregator service The gateway Putting it all together Capturing and visualizing traces using Jaeger Making logging your trusted ally Logging best practices The devil is in the (logging) details Shipping and indexing logs inside Kubernetes Running a log collector on each Kubernetes node Using a sidecar container to collect logs Shipping logs directly from the application Introspecting live Go services Building a microservice-based version of Links 'R' Us Decoupling access to the data stores Breaking down the monolith into distinct services Deploying the microservices that comprise the Links 'R' Us project Deploying the link-graph and text-indexer API services Deploying the web crawler Deploying the PageRank service Deploying the frontend service Locking down access to our Kubernetes cluster using network policies Summary Questions Further reading Chapter 12: Building Distributed Graph-Processing Systems Technical requirements Introducing the master/worker model Ensuring that masters are highly available The leader-follower configuration The multi-master configuration Strategies for discovering nodes Recovering from errors Out-of-core distributed graph processing Describing the system architecture, requirements, and limitations Modeling a state machine for executing graph computations Establishing a communication protocol between workers and masters Defining a job queue RPC service Establishing protocol buffer definitions for worker payloads Establishing protocol buffer definitions for master payloads Defining abstractions for working with bi-directional gRPC streams Remote worker stream Remote master stream Creating a distributed barrier for the graph execution steps Implementing a step barrier for individual workers Implementing a step barrier for the master Creating custom executor factories for wrapping existing graph instances The workers' executor factory The master's executor factory Coordinating the execution of a graph job Simplifying end user interactions with the dbspgraph package The worker job coordinator Running a new job Transitioning through the stages of the graph's state machine Handling incoming payloads from the master Using the master as an outgoing message relay The master job coordinator Running a new job Transitioning through the stages for the graph's state machine Handling incoming worker payloads Relaying messages between workers Defining package-level APIs for working with master and worker nodes Instantiating and operating worker nodes Instantiating and operating master nodes Handling incoming gRPC connections Running a new job Deploying a distributed version of the Links 'R' Us PageRank calculator Retrofitting master and worker capabilities to the PageRank calculator service Serializing PageRank messages and aggregator values Defining job runners for the master and the worker Implementing the job runner for master nodes The worker job runner Deploying the final Links 'R' Us version to Kubernetes Summary Questions Further reading Chapter 13: Metrics Collection and Visualization Technical requirements Monitoring from the perspective of a site reliability engineer Service-level indicators (SLIs) Service-level objectives (SLOs) Service-level agreements (SLAs) Exploring options for collecting and aggregating metrics Comparing push versus pull systems Capturing metrics using Prometheus Supported metric types Automating the detection of scrape targets Static and file-based scrape target configuration Querying the underlying cloud provider Leveraging the API exposed by Kubernetes Instrumenting Go code Registering metrics with Prometheus Vector-based metrics Exporting metrics for scraping Visualizing collected metrics using Grafana Using Prometheus as an end-to-end solution for alerting Using Prometheus as a source for alert events Handling alert events Grouping alerts together Selectively muting alerts Configuring alert receivers Routing alerts to receivers Summary Questions Further reading Chapter 14: Epilogue Appendix: Assessments Other Books You May Enjoy Index