Fragmentos indexados
53
Resource
Chapter 2. Defining Nonfunctional Requirements
Data Aplications · Listo
Kind
markdown
Attached
no
Leer recurso
Chapter 2. Defining Nonfunctional Requirements
The Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made. When was the last time a technology with a scale like that was so error-free?
>
— Alan Kay, in interview with Dr. Dobb's Journal (2012)
If you are building an application, you will be driven by a list of requirements. At the top of your list is most likely the functionality that the application must offer: what screens and what buttons you need, and what each operation is supposed to do in order to fulfill the purpose of your software. These are your functional requirements.
In addition, you probably have nonfunctional requirements: for example, the app should be fast, reliable, secure, legally compliant, and easy to maintain. These requirements might not be explicitly written down, because they may seem somewhat obvious, but they are just as important as the app's functionality; an app that is unbearably slow or unreliable might as well not exist.
Many nonfunctional requirements, such as security, fall outside the scope of this book. But we will consider a few, and this chapter will help you articulate them for your own systems. In particular, we will look at the following:
- Defining and measuring the performance of a system
- What it means for a service to be reliable — namely, continuing to work correctly, even when things go wrong
- Allowing a system to be scalable by having efficient ways of adding computing capacity as the load on the system grows
- Making it easier to maintain a system in the long term
The terminology introduced in this chapter will also be useful in the following chapters, when we go into the details of how data-intensive systems are implemented. However, abstract definitions can be quite dry; to make the ideas more concrete, we will start this chapter with a case study of a social networking service, which will provide practical examples of performance and scalability.
---
Case Study: Social Network Home Timelines
Imagine we have been given the task of implementing a social network in the style of X (formerly Twitter), where users can post messages and follow other users. This will be a huge simplification of how such a service actually works, but it will help illustrate some of the issues that arise in large-scale systems.
Let's assume that users make a total of 500 million posts per day, or 5,800 posts per second on average. Occasionally, the rate can spike to as high as 150,000 posts per second. Let's also assume that the average user follows 200 people and has 200 followers (although there is a very wide range: most people have only a handful of followers, and a few celebrities, such as Barack Obama, have over 100 million followers).
Representing Users, Posts, and Follows
We keep all the data in a relational database, as shown in Figure 2-1. We have one table for users, one table for posts, and one table for follow relationships.
Figure 2-1. A simple relational schema for a social network in which users can follow one another
[Currently logged-in]
[ user: 17055506 ]
|
v
+--------------+---------------+
| follows table |
+--------------+---------------+
| follower_id | followee_id |
+--------------+---------------+
| 17055506 | 12 <----------+--------+
+--------------+---------------+ |
|
+-----+------------+------------------+ |
| posts table | |
+-----+------------+------------------+ |
| id | sender_id | text | timestamp |
+-----+------------+------------------+--------------+
| 20 | 12 <-------+ just setting up | 1142974214 |
+-----+ | my twttr | |
+------------------+--------------+
|
v
+----+-------------+---------------+
| users table |
+----+-------------+---------------+
| id | screen_name | profile_image |
+----+-------------+---------------+
| 12 | jack | 1234567.jpg |
+----+-------------+---------------+Let's say the main read operation that our social network must support is the home timeline, which displays recent posts by people the user is following. We could write the following SQL query to get the home timeline for a particular user:
SELECT posts.*, users.* FROM posts
JOIN follows ON posts.sender_id = follows.followee_id
JOIN users ON posts.sender_id = users.id
WHERE follows.follower_id = current_user
ORDER BY posts.timestamp DESC
LIMIT 1000To execute this query, the database will use the follows table to find everybody who current_user is following, look up recent posts by those users, and sort them by timestamp to get the most recent 1,000 posts by any of the followed users.
Posts are supposed to be timely, so let's assume that after somebody makes a post, we want their followers to be able to see it within five seconds. One approach is for the user's client to repeat the preceding query every five seconds while the user is online (this is known as polling). If we assume that 10 million users are online and logged in at the same time, that would mean running the query 2 million times per second. Even if we were to poll less frequently, this is a lot.
This query is also quite expensive: if a user is following 200 people, the query needs to fetch a list of recent posts by each of those 200 people and merge those lists. Two million timeline queries per second times 200 followed accounts makes 400 million lookups per second — a huge number. And that's the average case. Some users follow tens of thousands of accounts; for them, this query is very expensive to execute and difficult to make fast.
Materializing and Updating Timelines
How can we do better? First, instead of polling, it would be better if the server actively pushed new posts to any followers who are currently online. Second, we should precompute the results of the query so that a user's request for their home timeline can be served from a cache.
Imagine that for each user, we store a data structure containing their home timeline (i.e., the recent posts by people they are following). Every time a user makes a post, we look up all their followers and insert that post into the home timeline of each follower — like delivering a message to a mailbox. Now when a user logs in, we can simply give them this precomputed home timeline. Moreover, to receive a notification about any new posts on their timeline, the user's client simply needs to subscribe to the stream of posts being added to their home timeline.
The downside of this approach is that we now need to do more work every time a user makes a post, because the home timelines are derived data that needs to be updated. The process is illustrated in Figure 2-2. When one initial request results in several downstream requests being carried out, we use the term fan-out to describe the factor by which the number of requests increases.
Figure 2-2. Fan-out: delivering new posts to every follower of the user who made the post
Posts for recipient 1 Get home timeline
Fan-out: deliver +--[ T7 | T5 | T3 | T1 ]---> (website, API) [user 1]
post to each |
follower | Posts for recipient 2
[User makes post] +--[ T8 | T6 | T5 ]---------> (website, API) [user 2]
| |
v | Posts for recipient 3
[All posts: T8|T7|T6|T5|T4|T3|T2|T1]-+--[ T8 | T7 | T5 | T4 | T3 ]-> (website, API) [user 3]At a rate of 5,800 posts per second, if the average post reaches 200 followers (i.e., a fan-out factor of 200), we will need to do just over 1 million home timeline writes per second. This is a lot, but it's still a significant saving compared to the 400 million per-sender post lookups per second that we would otherwise have to do.
If the rate of posts spikes because of a special event, we don't have to do the timeline deliveries immediately — we can enqueue them and accept that it will temporarily take a bit longer for posts to show up in followers' timelines. Even during such load spikes, timelines remain fast to load, since we simply serve them from a cache.
This process of precomputing and updating the results of a query is called materialization, and the timeline cache is an example of a materialized view (a concept we will discuss further in later chapters). The materialized view speeds up reads, but in return we have to do more work on writes. The cost of writes for most users is modest, but a social network also has to consider some extreme cases:
- If a user is following a very large number of accounts, and those accounts post a lot, that user will have a high rate of writes to their materialized timeline. However, that user is not likely reading all the posts in their timeline, so it's OK to simply drop some of their timeline writes and show the user only a sample of the posts from the accounts they're following.
- When a celebrity account with a very large number of followers makes a post, we have to do a lot of work to insert that post into the home timelines of each of their millions of followers. In this case, dropping some of those writes is not OK. One way of solving this problem is to handle celebrity posts separately from everyone else's posts: we can save ourselves the effort of adding celebrity posts to millions of timelines by storing them separately and merging them with the materialized timeline when it is read. Despite such optimizations, handling celebrities on a social network can require a lot of infrastructure.
---
Describing Performance
Most discussions of software performance consider two main types of metric:
Response time
The elapsed time from the moment when a user makes a request until they receive the requested answer. The unit of measurement is seconds (or milliseconds, or microseconds).
Throughput
The number of requests per second, or the data volume per second, that the system is processing. For a given allocation of hardware resources, there is a maximum throughput that can be handled. The unit of measurement is "somethings per second."
In the social network case study, "posts per second" and "timeline writes per second" are throughput metrics, whereas "time it takes to load the home timeline" and "time until a post is delivered to followers" are response time metrics.
Throughput and response time are often related. An example of such a relationship for an online service is sketched in Figure 2-3. The service has a low response time when request throughput is low, but response time increases as load increases. This is because of queueing: when a request arrives on a highly loaded system, the CPU is likely already in the process of handling an earlier request, and therefore the incoming request needs to wait until the earlier request has been completed. As throughput approaches the maximum that the hardware can handle, queueing delays increase sharply.
Figure 2-3. As the throughput of a service approaches its capacity, the response time
increases dramatically because of queueing.
Response time
^
| .
| ..
| ...
| ...
| ...
| ...
| ....
| Service time on ......
| unloaded system .....................................
+---------------------------------------------------->
| Throughput
Maximum that hardware can handleWhen an Overloaded System Won't Recover
If a system is close to overload, with throughput pushed close to the limit, it can sometimes enter a vicious cycle where it becomes less efficient and hence even more overloaded. For example, if a long queue of requests is waiting to be handled, response times may increase so much that clients time out and resend their requests. This causes the rate of requests to increase even further, making the problem worse — a retry storm. Even when the load is reduced again, such a system may remain in an overloaded state until it is rebooted or otherwise reset. This phenomenon is called a metastable failure, and it can cause serious outages in production systems.
>
To avoid retries overloading a service, you can increase and randomize the time between successive retries on the client side (exponential backoff) and temporarily stop sending requests to a service that has returned errors or timed out recently (by using a circuit breaker or token bucket algorithm). The server can also detect when it is approaching overload and start proactively rejecting requests (load shedding), or send back responses asking clients to slow down (backpressure). The choice of queueing and load balancing algorithms can also make a difference.
In terms of performance metrics, the response time is usually what users care about the most, whereas the throughput determines the required computing resources (e.g., how many servers you need) and hence the cost of serving a particular workload. If throughput is likely to increase beyond the current hardware's capability, the capacity needs to be expanded; a system is said to be scalable if its maximum throughput can be significantly increased by adding computing resources.
Latency and Response Time
"Latency" and "response time" are sometimes used interchangeably, but in this book we will use these and a few related terms in a specific way (illustrated in Figure 2-4):
- The response time is what the client sees; it includes all delays incurred anywhere in the system.
- The service time is the duration for which the service is actively processing the client's request.
- Queueing delays can occur at several points in the flow — for example, after a request is received, it might need to wait until a CPU is available before it can be processed.
- Latency is a catchall term for time during which a request is not being actively processed — that is, during which it is latent. In particular, network latency or network delay refers to the time that a request and response spend traveling through the network.
Figure 2-4. Response time, service time, network latency, and queueing delay
Make Receive
request response
Client ----+---------------------------------------------------- ---+----> Time
\ /
\ Request Response /
\ /
Service --------+-------+------------------+-------+---------+------> Time
| | | |
|<----->|<---------------->|<----->|
Network Queueing Service Queueing Network
latency delay time (resp) latency
|<------------------------------------------------->|
Response timeThe response time can vary significantly from one request to the next, even if you keep making the same request over and over again. Many factors can add random delays — for example, a context switch to a background process, the loss of a network packet and TCP retransmission, a garbage collection pause, a page fault forcing a read from disk, or mechanical vibrations in the server rack.
Queueing delays often account for a large part of the variability in response times. As a server can process only a small number of things in parallel, it takes only a small number of slow requests to hold up the processing of subsequent requests — an effect known as head-of-line blocking. Even if those subsequent requests have fast service times, the client will see a slow overall response time due to the time waiting for the prior request to complete. The queueing delay is not part of the service time, and for this reason it is important to measure response times on the client side.
Average, Median, and Percentiles
Because the response time varies from one request to the next, we need to think of it not as a single number, but as a distribution of values that we can measure. In Figure 2-5, each gray bar represents a request to a service, and its height shows how long that request took. Most requests are reasonably fast, but occasional outliers take much longer. Variation in network delay is also known as jitter.
Figure 2-5. Illustrating mean and percentiles: response times for a sample of 100 requests
Response time
^
| |
99th |. . . . . . . . . . . . . . . . . . . . .|. . . . . .
perc.| |
| | | |
95th |. . . . . . . . . . . . . | . . . . . . . | . . . | .
perc.| | | |
| | | || | || | | | | | | | | | | | | | || | | ||
| | | || | || | | | | | | | | | | | | | || | | ||
Mean-|--|-|-||--|--||--|-|-|-|-|-|--|-|-|-|-|-|-||--|-|-||-
avg. | | | || | || | | | | | | | | | | | | | || | | ||
Med. |. |.|.||.|.||.|.|.|.|.|.|.|..|.|.|.|.|.|.||..|.|.||.
p50 | || | || | || | | | | | | | | | | | | | || | | ||
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+----->
RequestsIt's common to report the average response time of a service (technically, the arithmetic mean). The mean response time is useful for estimating throughput limits. However, the mean is not a very good metric if you want to know your "typical" response time, because it doesn't tell you how many users actually experienced that delay.
Usually it's better to use percentiles. If you take your list of response times and sort it from fastest to slowest, the median is the halfway point — for example, if your median response time is 200 ms, that means half your requests return in less than 200 milliseconds, and half your requests take longer. This makes the median a good metric if you want to know how long users typically have to wait. The median is also known as the 50th percentile, sometimes abbreviated as p50.
To figure out how bad your outliers are, you can look at higher percentiles: the 95th, 99th, and 99.9th percentiles are common (abbreviated p95, p99, and p999). For example, if the 95th percentile response time is 1.5 seconds, that means 95 out of 100 requests take less than 1.5 seconds, and 5 out of 100 requests take 1.5 seconds or more.
High response-time percentiles, also known as tail latencies, are important because they directly affect users' experience of the service. For example, Amazon describes response time requirements for internal services in terms of the 99.9th percentile, even though this affects only 1 in 1,000 requests. This is because the customers with the slowest requests are often those who have the most data on their accounts — that is, they're the most valuable customers.
Optimizing the 99.99th percentile (the slowest 1 in 10,000 requests) was deemed too expensive and found to not yield enough benefit for Amazon's purposes. Reducing response times at very high percentiles is difficult because they are easily affected by random events outside of your control, and the benefits are diminishing.
The User Impact of Response Times
>
It seems obvious that a fast service is better for users than a slow service. However, it is surprisingly difficult to get hold of reliable data to quantify the effect that latency has on user behavior. Some often-cited statistics are unreliable — for example, Google reported that a slowdown in search results from 400 ms to 900 ms was associated with a 20% drop in traffic and revenue, but other studies from the same era found much smaller effects. A study by Yahoo found 20%–30% more clicks on fast searches when the difference between fast and slow responses is 1.25 seconds or more.
Use of Response Time Metrics
High percentiles are especially important in backend services that are called multiple times as part of serving a single end-user request. Even if you make the calls in parallel, the request still needs to wait for the slowest of the parallel calls to complete. It takes just one slow call to make the entire end-user request slow, as illustrated in Figure 2-6. Even if only a small percentage of backend calls are slow, the chance of getting a slow call increases if an end-user request requires multiple backend calls, so a higher proportion of such end-user requests end up being slow — an effect known as tail latency amplification.
Figure 2-6. When several backend calls are needed to serve a request, just a single slow
call can slow down the entire end-user request.
End-user request
|
+---------+----------+
| Web application |
+---------+----------+
|
+---------+---------+-------+-------+---------+--------+
| | | | | |
v v v v v v
92 ms 76 ms 103 ms 143 ms 86 ms 487 ms 133 ms
+----------+--------+----------+ +----------+--------+--------+--------+
|Backend 1 |Backend2|Backend 3 | |Backend 4 |Backend5|Backend6|Backend7|
+----------+--------+----------+ +----------+--------+--------+--------+
| | | | | | |
v v v v v v v
[db] [db] [db] [db] [db] [db] [db]
Slowest call: Backend 6 @ 487 ms
--> Total end-user response time dominated by this single callPercentiles are often used in service level objectives (SLOs) and service level agreements (SLAs) as ways of defining the expected performance and availability of a service. For example, an SLO may set a target for a service to have a median response time of less than 200 ms and a 99th percentile under 1 second, and a target that at least 99.9% of valid requests result in non-error responses. An SLA is a contract that specifies what happens if the SLO is not met (e.g., customers may be entitled to a refund).
Computing Percentiles
>
If you want to add response time percentiles to the monitoring dashboards for your services, you need to efficiently calculate them on an ongoing basis. For example, you may want to keep a rolling window of response times for requests in the last 10 minutes. The simplest implementation is to keep a list of response times for all requests within the time window and sort that list every minute. Open source percentile estimation libraries include HdrHistogram, t-digest, OpenHistogram, and DDSketch.
>
Beware that averaging percentiles (e.g., to reduce the time resolution or to combine data from several machines) is mathematically meaningless. The right way of aggregating response time data is to add the histograms.
---
Reliability and Fault Tolerance
Everybody has an intuitive idea of what it means for something to be reliable or unreliable. For software, typical expectations include the following:
- The application performs the function that the user expected.
- The application can tolerate the user making mistakes or using the software in unexpected ways.
- Its performance is good enough for the required use case, under the expected load and data volume.
- The system prevents any unauthorized access and abuse.
If all those things together mean "working correctly," then we can understand reliability as meaning, roughly, "continuing to work correctly, even when things go wrong." To be more precise about things going wrong, we will distinguish between faults and failures:
Fault
A fault occurs when a particular part of a system stops working correctly — for example, if a single hard drive malfunctions, or a single machine crashes, or an external service (that the system depends on) has an outage.
Failure
A failure occurs when the system as a whole stops providing the required service to the user — in other words, when it does not meet the SLO.
The distinction between faults and failures can be confusing because they are the same thing, just at different levels. For example, if a hard drive stops working, we say that the hard drive has failed; if the system consists of only that one hard drive, it has stopped providing the required service and thus has also failed. However, if the system consists of multiple hard drives, the failure of a single hard drive is only a fault from the point of view of the bigger system, and the bigger system might be able to tolerate that fault by having a copy of the data on another hard drive.
Fault Tolerance
We call a system fault-tolerant if it continues providing the required service to users in spite of certain faults occurring. If a system cannot tolerate a certain part becoming faulty, we call that part a single point of failure (SPOF), because a fault in that part escalates to cause the failure of the whole system.
Fault tolerance is always limited to a certain number of certain types of faults. For example, a system might be able to tolerate a maximum of two hard drives failing at the same time, or a maximum of one out of three nodes crashing. It would not make sense to tolerate any number of faults; if all nodes crash, nothing can be done.
Counterintuitively, in such fault-tolerant systems, it can make sense to increase the rate of faults by triggering them deliberately — for example, by randomly killing individual processes without warning. This is called fault injection. Many critical bugs are actually due to poor error handling; by deliberately inducing faults, you ensure that the fault-tolerance machinery is continually exercised and tested. Chaos engineering is a discipline that aims to improve confidence in fault-tolerance mechanisms through experiments such as deliberately injecting faults.
Hardware and Software Faults
When we think of causes of system failure, hardware faults quickly come to mind:
- Approximately 2%–5% of magnetic hard drives fail per year; in a storage cluster with 10,000 disks, we should therefore expect on average one disk failure per day.
- Approximately 0.5%–1% of SSDs fail per year. Uncorrectable errors occur approximately once per year per drive, even in fairly new drives.
- Approximately 1 in 1,000 machines has a CPU core that occasionally computes the wrong result, likely because of manufacturing defects.
- Data in RAM can be corrupted by random events such as cosmic rays or permanent physical defects. Even with ECC memory, more than 1% of machines encounter an uncorrectable error in a given year.
- An entire datacenter might become unavailable (e.g., because of a power outage or network misconfiguration) or even be permanently destroyed (e.g., by fire, flood, or earthquake).
Tolerating hardware faults through redundancy
Our first response to unreliable hardware is usually to add redundancy to the individual hardware components in order to reduce the failure rate of the system. Disks may be set up in a RAID configuration, servers may have dual power supplies and hot-swappable CPUs, and datacenters may have batteries and diesel generators for backup power.
Redundancy is most effective when component faults are independent. However, experience has shown significant correlations between component failures. Hardware redundancy increases the uptime of a single machine; however, using a distributed system has additional advantages, such as being able to tolerate a complete outage of one datacenter. Cloud providers use availability zones to identify which resources are physically co-located.
Systems that can tolerate the loss of entire machines also have operational advantages. A single-server system requires planned downtime if you need to reboot the machine, whereas a multi-node fault-tolerant system can be patched by restarting one node at a time. This is called a rolling upgrade.
Software faults
Although hardware failures can be weakly correlated, they are still mostly independent. On the other hand, software faults are often very highly correlated, because it is common for many nodes to run the same software and thus have the same bugs. Such faults are harder to anticipate, and they tend to cause many more system failures than uncorrelated hardware faults. Examples include the following:
- A software bug that causes every node to fail at the same time in particular circumstances. For instance, on June 30, 2012, a leap second caused many Java applications to hang simultaneously because of a bug in the Linux kernel, bringing down several internet services.
- A runaway process that uses up a shared, limited resource, such as CPU time, memory, disk space, network bandwidth, or threads.
- A service that the system depends on slows down, becomes unresponsive, or starts returning corrupted responses.
- Cascading failures, where a problem in one component causes another component to become overloaded and slow down, which in turn brings down another component.
The problem of systematic faults in software has no quick solution. Lots of small things can help: carefully thinking about assumptions and interactions in the system; thorough testing; ensuring process isolation; allowing processes to crash and restart; avoiding feedback loops such as retry storms; measuring, monitoring, and analyzing system behavior in production.
Humans and Reliability
Humans design and build software systems, and the operators who keep the systems running are also human. One study of large internet services found that configuration changes by operators were the leading cause of outages, whereas hardware faults played a role in only 10%–25% of cases.
It is tempting to label such problems as "human error" and to wish that they could be solved by better controlling human behavior. However, blaming people for mistakes is counterproductive. What we call "human error" is not really the cause of an incident, but rather a symptom of a problem with the sociotechnical system in which people are trying their best to do their jobs.
Various technical measures can help minimize the impact of human mistakes, including thorough testing, rollback mechanisms for quickly reverting configuration changes, gradual rollouts of new code, detailed and clear monitoring, and observability tools for diagnosing production issues.
Increasingly, organizations are adopting a culture of blameless postmortems: after an incident, the people involved are encouraged to share full details about what happened, without fear of punishment, since this allows others in the organization to learn how to prevent similar problems in the future.
How Important Is Reliability?
>
Reliability is not just for nuclear power stations and air traffic control; more mundane applications are also expected to work reliably. Bugs in business applications lead to lost productivity, and outages of ecommerce sites can have huge costs in terms of lost revenue and damage to reputation.
>
In many applications, a temporary outage of a few minutes or even a few hours is tolerable, but permanent data loss or corruption would be catastrophic. Consider a parent who stores all their pictures and videos of their children in your photo application. How would they feel if that database was suddenly corrupted?
>
As another example of how unreliable software can harm people, consider the Post Office Horizon scandal. Between 1999 and 2019, hundreds of people managing Post Office branches in Britain were convicted of theft or fraud because the accounting software showed a shortfall in their accounts. Eventually it became clear that many of these shortfalls were due to bugs in the software, resulting in many of these convictions being overturned — probably the largest miscarriage of justice in British history.
---
Scalability
Even if a system is working reliably today, that doesn't mean it will necessarily work reliably in the future. One common reason for degradation is increased load. Perhaps the system has grown from 10,000 concurrent users to 100,000 concurrent users, or from 1 million to 10 million.
Scalability is the term we use to describe a system's ability to cope with increased load. Scalability is not a one-dimensional label — it is meaningless to say "X is scalable" or "Y doesn't scale." Rather, discussing scalability means considering questions like these:
- If the system grows in a particular way, what are our options for coping with the growth?
- How can we add computing resources to handle the additional load?
- Based on current growth projections, when will we hit the limits of our current architecture?
Understanding Load
First, you need a clear understanding of the current load on the system. Often this will be a measure of throughput — for example, the number of requests per second to a service, the number of gigabytes of new data arriving per day, or the number of shopping cart checkouts per hour.
Once you understand the load on your system, you can investigate what happens when the load increases:
- When you increase the load in a certain way and keep the system resources unchanged, how is the performance of your system affected?
- When you increase the load in a certain way, how much do you need to increase the resources if you want to keep performance unchanged?
If doubling the resources will enable you to handle twice the load while keeping performance the same, we say that you have linear scalability. Much more likely is that the cost grows faster than linearly.
Shared-Memory, Shared-Disk, and Shared-Nothing Architectures
Vertical scaling (scaling up) means moving to a more powerful machine with more CPU cores, more RAM, and more disk space. The problem with a shared-memory approach is that the cost grows faster than linearly; a high-end machine with twice the hardware resources typically costs significantly more than twice as much.
The shared-disk architecture uses several machines with independent CPUs and RAM but stores data on an array of disks that is shared among the machines. Contention and the overhead of locking limit its scalability.
The shared-nothing architecture (also called horizontal scaling or scaling out) involves a distributed system with multiple nodes, each of which has its own CPUs, RAM, and disks. The advantages of this approach include:
- Potential to scale linearly
- Can use whatever hardware offers the best price/performance ratio
- Can more easily adjust hardware resources as load changes
- Can achieve greater fault tolerance by distributing across multiple datacenters
The downsides are that it requires explicit sharding and incurs all the complexity of distributed systems.
Principles for Scalability
The architecture of systems that operate at large scale is usually highly specific to the application. There is no such thing as a generic, one-size-fits-all scalable architecture (informally known as magic scaling sauce).
A good general principle for scalability is to break a system into smaller components that can operate largely independently from one another. This is the underlying principle behind microservices, sharding, stream processing, and shared-nothing architectures.
Another good principle is not to make things more complicated than necessary. If a single-machine database will do the job, it's probably preferable to a complicated distributed setup. A system with 5 services is simpler than one with 50. Good architectures usually involve a pragmatic mixture of approaches.
---
Maintainability
Software does not wear out or suffer material fatigue, but the requirements for an application frequently evolve, the environment it runs in changes, and it may have bugs that need fixing.
It is widely recognized that the majority of the cost of software is not in its initial development but in its ongoing maintenance — fixing bugs, keeping systems operational, investigating failures, adapting it to new platforms, modifying it for new use cases, repaying technical debt, and adding new features.
Every system we create today will one day become a legacy system if it is valuable enough to survive for a long time. To minimize the pain for future generations who need to maintain our software, we should design it with maintenance in mind. In this book we will pay attention to several principles:
Operability
Make it easy for the organization to keep the system running smoothly.
Simplicity
Make it easy for new engineers to understand the system, by implementing it using well-understood, consistent patterns and structures and avoiding unnecessary complexity.
Evolvability
Make it easy for engineers to make changes to the system in the future, adapting it and extending it for unanticipated use cases as requirements change.
Operability: Making Life Easy for Operations
Good operability means making routine tasks easy, allowing the operations team to focus on high-value activities. Data systems can help by doing the following:
- Allowing monitoring tools to check the system's key metrics and supporting observability tools to give insights into the system's runtime behavior
- Avoiding dependency on individual machines
- Providing good documentation and an easy-to-understand operational model
- Providing good default behavior, but also giving administrators the freedom to override defaults when needed
- Self-healing where appropriate, but also giving administrators manual control over the system state when needed
- Exhibiting predictable behavior, minimizing surprises
Simplicity: Managing Complexity
When complexity makes maintenance hard, budgets and schedules are often overrun. In complex software, there is also a greater risk of introducing bugs when making a change. Conversely, reducing complexity greatly improves the maintainability of software.
One attempt at reasoning about complexity breaks it into two categories: essential and accidental. Essential complexity is inherent in the problem domain of the application, while accidental complexity arises only because of limitations of our tooling.
One of the best tools we have for managing complexity is abstraction. A good abstraction can hide a great deal of implementation detail behind a clean, simple-to-understand façade. For example, high-level programming languages are abstractions that hide machine code, CPU registers, and system calls. SQL is an abstraction that hides complex on-disk and in-memory data structures, concurrent requests from other clients, and inconsistencies after crashes.
Evolvability: Making Change Easy
It's extremely unlikely that your system's requirements will remain unchanged forever. They are much more likely to be in constant flux: you learn new facts, previously unanticipated use cases emerge, business priorities change, users request new features, new platforms replace old platforms, legal or regulatory requirements change.
In terms of organizational processes, Agile working patterns provide a framework for adapting to change. The ease with which you can modify a data system and adapt it to changing requirements is closely linked to its simplicity and its abstractions. Loosely coupled, simple systems are usually easier to modify than tightly coupled, complex ones. Since this is such an important idea, we will use a different word to refer to agility at a data system level: evolvability.
One major factor that makes change difficult in large systems is irreversibility. For example, say you are migrating from one database to another. If you cannot switch back to the old system in case of problems with the new one, the stakes are much higher than if you can easily go back. Minimizing irreversibility improves flexibility.
---
Summary
In this chapter we examined several examples of nonfunctional requirements: performance, reliability, scalability, and maintainability. Through these topics, we also encountered principles and terminology that will be relevant throughout the rest of the book.
We started with a case study of implementing home timelines in a social network, which illustrated some of the challenges that arise at scale. We then discussed how to measure performance (e.g., using response time percentiles) and the load on a system (e.g., using throughput metrics), and how these metrics are used in SLAs. Scalability is a closely related concept: it focuses on ensuring that performance stays the same when the load grows.
To achieve reliability, you can use fault-tolerance techniques, which allow a system to continue providing its services even if a component is faulty. We saw examples of hardware faults that can occur and distinguished them from software faults, which can be harder to deal with because they are often strongly correlated. Another aspect of achieving reliability is to build resilience against humans making mistakes, and we saw blameless postmortems as a technique for learning from incidents.
Finally, we examined several facets of maintainability, including supporting the work of operations teams, managing complexity, and making it easy to evolve an application's functionality over time. There are no easy answers to how to achieve these goals, but one approach that can help is to build applications using well-understood building blocks that provide useful abstractions.