An idempotent API operation is an operation that produces the same effect no matter how many times it is executed.
For example, an API operation that sets a user's name is a typically idempotent operation. Whether it occurs one time or a thousand times, the effect of the operation is that the user's name will be set to the target value.
Idempotent operations do not necessarily return the exact same response output every time they are invoked, they only need to have the same impact upon the system. Deleting a resource is an example of this distinction; the first time you invoke the delete, the object is deleted. The second time you attempt to delete it, there is no change in the state of the system, but the object remains deleted as requested.
Benefits: HTTP Spec. Conformance, Prevent Conflicting Edits, Reduced Server State, Retry to Recover from Temporary Failure.
Trade-offs: Increased Development Costs, Increased Implementation Complexity.
Conformance to the HTTP 1.1 specification, RFC2616.
Benefits: Prevent Customer Frustration.
Influenced by: Idempotency.
Concurrent "edit" API operations will not have one succeed, and then another succeed, causing the changes in the first operation to be erased by the second operation.
Benefits: Improved Concurrency, Prevent Data Inconsistency, Prevent Data Loss.
Influenced by: Idempotency.
Less information will need to be stored and managed by the application. Reducing server state is particularly relevant to removing information that isn't strictly business oriented, but is kept around only to aid the implementation of services or workflows.
Benefits: Improved Availability, Improved Scalability.
Influenced by: Idempotency.
Increased difficulty in developing the software.
Influenced by: Idempotency.
Is a cause of: Increased Maintenance Costs, Increased Testing Costs.
The ability to safely retry requests that may or may not have been processed. A typical temporary failure that would benefit from this capability is a glitch during a high-availability failover, a temporary networking problem, or a small load spike.
Benefits: Automatically Recover from Failures, Easier to Develop Reliable Client.
Influenced by: Idempotency.
More API operations can be processed simultaneously without them conflicting with each other.
Benefits: Improved Server Utilization, Increased Throughput.
Influenced by: Prevent Conflicting Edits.
Data inconsistency errors are when logical or business constraints are violated by the data already stored and accepted into the system. These types of errors tend to propagate into other aspects of the system where logic was not designed to handle the data in its inconsistent state.
Benefits: Prevent Customer Frustration.
Influenced by: Prevent Conflicting Edits.
Availability is essentially the opposite of downtime. A system is available when clients are able to perform the expected tasks and functions using the system.
Benefits: Prevent Customer Frustration.
Influenced by: Reduced Server State.
Scalability is the ability for a system to handle increased amounts of traffic, customers, and data with linearly increasing costs.
Not all systems are scalable. Unscalable systems have bottlenecks or restrictions that need to be solved, or they exhibit exponential growth in cost in relation to a linear growth in customers. For example, a system is unscalable if adding 1,000 users would result in $10,000 in costs, but the next 1,000 users would result in $100,000 in costs.
Benefits: System Supports More Customers.
Influenced by: Reduced Server State.
The costs associated with testing the system are increased, typically in the area of person time.
Influenced by: Increased Implementation Complexity.
Is a cause of: Increased Costs.
The costs associated with maintaining the system and software are increased, typically in the area of developer time.
Influenced by: Increased Implementation Complexity.
Is a cause of: Increased Costs.
The costs associated with further product development, such as adding new features and fixing bugs, are increased. Typically theses costs are increased in developer time.
Influenced by: Idempotency.
Is a cause of: Increased Costs.
The difficulty of developing an API client that works in the expected manner under a variety of extraneous circumstances is reduced.
Benefits: Improved Mobile Client, Reduced Development Costs.
Influenced by: Retry to Recover from Temporary Failure.
The existing server resources used to run the system are utilized at a higher rate in order to accomplish more work without new resources. This is, of course, only possible if the servers are not already completely utilized to their maximum.
Benefits: Reduced Server Costs.
Influenced by: Improved Concurrency.
Systems can automatically recover from failure if they're capable of classifying the type of the failure, classifying the type of the request that failed, and determining that there is a path of action that will result in a better outcome than returning an error.
As an example, a load balancer like HAProxy is capable of redispatching requests if a backend server disconnects while processing a request.
Benefits: Prevent Data Loss.
Influenced by: Retry to Recover from Temporary Failure.
Increased throughput is serving more API operations within the same period of time.
Benefits: Improved Client Performance.
Influenced by: Improved Concurrency.
The system is capable of supporting more customers and growing beyond its current workload.
Benefits: Increased Revenue.
Influenced by: Improved Scalability.
More money is going out the door.
Influenced by: Increased Development Costs, Increased Maintenance Costs, Increased Testing Costs.
Is a cause of: Reduced Profit.
Less time and money is being spend to develop the application system.
Benefits: Reduced Costs.
Influenced by: Easier to Develop Reliable Client.
The usability or functionality of a potential mobile client, such as an iPhone or Android application, is improved.
Benefits: Prevent Customer Frustration.
Influenced by: Easier to Develop Reliable Client.
Less money is being spent on server hardware and maintenance.
Benefits: Reduced Costs.
Influenced by: Improved Server Utilization.
Data entered or manipulated by the customer does not disappear into the void without anyone knowing about it.
Benefits: Prevent Customer Frustration.
Influenced by: Automatically Recover from Failures, Prevent Conflicting Edits.
API operations performed by the client application take less time to roundtrip and process.
Benefits: Prevent Customer Frustration.
Influenced by: Increased Throughput.
Less money is being made by the business.
Influenced by: Increased Costs.
Less money is going out the door.
Benefits: Increased Profit.
Influenced by: Reduced Development Costs, Reduced Server Costs.
Customers are not pissed off by the way your application works, or doesn't work. Customer frustration is a very common problem with applications, and it can often occur without anyone knowing about it until it reaches a breaking point where a customer cannot stand anymore.
Benefits: Prevent Customer Loss.
Influenced by: HTTP Spec. Conformance, Improved Availability, Improved Client Performance, Improved Mobile Client, Prevent Data Inconsistency, Prevent Data Loss.
Fewer customers leaving your application to find alternatives.
Benefits: Increased Revenue.
Influenced by: Prevent Customer Frustration.
More money is coming in from customers.
Benefits: Increased Profit.
Influenced by: Prevent Customer Loss, System Supports More Customers.
More money is being made by the business.
Influenced by: Increased Revenue, Reduced Costs.
For an HTTP API, idempotency of some HTTP methods is a requirement of RFC2616 § 9.1.2.
The HTTP methods GET, HEAD, PUT, and DELETE are required by the specification to be idempotent. The methods OPTIONS and TRACE are expected to be idempotent by virtue of having no side-effects.
The method POST is commonly used for any operations that are not idempotent.
Links: Idempotency and HTTP Spec. Conformance.
Idempotent operations can be safely retried after any type of failure that might be a temporary failure. If the operation has already been completed successfully but the response was lost, then the idempotency of the operation guarantees that invoking it a second time is safe. If it had not been completed, then it would be processed as expected.
Links: Idempotency and Retry to Recover from Temporary Failure.
By ensuring that it is safe to recover from errors by retrying requests, it becomes far easier to develop a reliable API client. The client can retry requests without user intervention, and without requiring the user to resync their state and start an operation from scratch.
Links: Retry to Recover from Temporary Failure and Easier to Develop Reliable Client.
As long as it is safe to recover from errors by retrying, it's possible to do so automatically.
Links: Retry to Recover from Temporary Failure and Automatically Recover from Failures.
Mobile clients are a notoriously difficult environment to build software in. Limited CPU, memory, and storage capabilities are just a small part of the challenge; the unreliable networking is the biggest difficulty. The easier it is to build a reliable API client, the easier it will be to build a high-quality mobile client.
Links: Easier to Develop Reliable Client and Improved Mobile Client.
If your mobile application doesn't suck, customers won't be as frustrated with it.
Links: Improved Mobile Client and Prevent Customer Frustration.
There will be cases where a request failure occurs that a client cannot retry. If you have an outage that affects your system's upstream networking, then the client issuing requests may not be immediately capable of retrying those requests, and other intervening forces may prevent the client from ever retrying that request.
An automatic failure recovery system can prevent the end-user from losing data that they expected, desired, and required to be entered into your system.
Links: Automatically Recover from Failures and Prevent Data Loss.
Even though you're providing an API as a service to your customers, it's likely that you're going to have to develop clients using it to. In fact, you probably want to, just to "dogfood" your own work.
Designing your API such that it is easier to develop reliable clients is going to end up reducing your own development efforts in building that client software.
Links: Easier to Develop Reliable Client and Reduced Development Costs.
Since development costs are obviously costs, it only follows that reducing them would result in an overall reduction in costs.
Links: Reduced Development Costs and Reduced Costs.
As profit is generally your revenue minus your costs, all other things being equal, reducing costs will increase profit.
Links: Reduced Costs and Increased Profit.
Often a system that's designed for to support idempotent API operations will also prevent conflicting modification operations as well. It is not a strict requirement though, but there is a tendency for one implementation to address both issues.
A common way to implement an idempotent edit operation would include identifying the state of the resource or object before its edited, such as with a version number or a timestamp. In an HTTP API, this can be accomplished with the If-Match header using a resource's ETag. This makes the operation both:
Links: Idempotency and Prevent Conflicting Edits.
Conflicting edits being processed can result in data loss. Preventing conflicting edits will prevent this form of data loss.
An example of the data loss would be two administrators simultaneously editing a user profile. The first admin changes the user's e-mail address, while the second admin changes the user's password. Depending upon the API design, the lack of a mechanism to prevent conflicting edits will cause one of these changes to be disregarded when both updates are submitted to the system simultaneously. One will be applied, then the second will be applied overwriting the first.
Links: Prevent Conflicting Edits and Prevent Data Loss.
Conflicting edits being processed can result in an inconsistent data state. Preventing this requires either preventing conflicting edits, or it would require every API operation to perform validation checks that might be outside the scope of the operation being processed.
Preventing conflicting edits will prevent some possible occurrences of data inconsistency.
Links: Prevent Conflicting Edits and Prevent Data Inconsistency.
An alternative to preventing conflicting edits would be to serialize requests, either on a per-server, per-tenant, or per-user basis depending upon what is appropriate for the application. If this type of serialization can be avoided because conflicting edits are prevented, then request concurrency levels can be higher.
Links: Prevent Conflicting Edits and Improved Concurrency.
Preventing data inconsistency will prevent application errors or confusing, nonsensical output in response to API operations. This will reduce the likelihood of customers being frustrated by these outputs.
Links: Prevent Data Inconsistency and Prevent Customer Frustration.
Customers will get frustrated if the impact of their API operations are silently lost because they'll have difficulty figuring out if they're responsible for the errors, or your system is. Avoiding that scenario is going to reduce the likelihood of customer frustration.
Links: Prevent Data Loss and Prevent Customer Frustration.
The ability to serve more requests simultaneously will, all other things being equal, generally increase the number of requests that can be served in a given time period.
However, there is usually a tipping point at which higher concurrency may result in lower throughput if your system spends too much time managing concurrent requests, or the concurrency level exceeds the threshold of the system.
Links: Improved Concurrency and Increased Throughput.
Most server hardware is optimized for processing multiple operations at one time. By improving the concurrency of your API system, you're able to take better advantage of the hardware you have, leading to increased utilization.
Links: Improved Concurrency and Improved Server Utilization.
All other things being equal, reducing server costs will reduce the overall costs of running a business.
Links: Reduced Server Costs and Reduced Costs.
Improving the throughput of request processing in your system will reduce the amount of time clients are waiting for API operations to complete. This should make the responsiveness of your API better, which will allow client applications to accomplish more work and be more responsive.
Links: Increased Throughput and Improved Client Performance.
Particularly for interactive clients, improving the performance of an API client will reduce the amount of time spent waiting for API requests to be processed, which will alleviate customer frustration. A typical application of this will be a mobile client; the faster it performs, the more confident users feel.
Links: Improved Client Performance and Prevent Customer Frustration.
Frustrated customers are customers that you're more likely to lose, especially if the cost of migrating to an alternative is relatively low. Many customers will actually suffer a lot of frustration before taking on a migration project, especially in the B2B service space, but it's usually not a great idea to test this theory intentionally.
Links: Prevent Customer Frustration and Prevent Customer Loss.
For the vast majority of businesses, customers are the source of revenue. Losing customers means losing revenue, so preventing customer loss will prevent revenue loss.
Links: Prevent Customer Loss and Increased Revenue.
As profit is generally your revenue minus your costs, all other things being equal, increasing revenue will increase profit.
Links: Increased Revenue and Increased Profit.
Designing an API's idempotent operations requires a design where API operations are fully autonomous. The content of the API request must be logically complete and independent from server-side state in order for requests to be reappliable. As a result, this style of API design tends to require less server-side state.
The reduction of server-side state is particularly true when compared to distributed transaction coordination, which is sometimes considered an alternative to an idempotent API design. Distributed transactions require very coordinated shared state between multiple application servers; all components of that state are completely absent in an idempotent API.
Links: Idempotency and Reduced Server State.
Shared server state is a common bottleneck in scaling applications. Instead of scaling one component, such as the system's application servers, there is now a requirement to scale two systems simultaneously, as you'll need to scale your shared state services at the same time.
By reducing the amount of shared server state, you eliminate a scaling hurdle entirely.
Links: Reduced Server State and Improved Scalability.
Shared server state is often stored in a separate system than your traditional data store because it requires different access patterns and a higher write workload. As a result, the use of a shared server state storage is an additional point-of-failure for the system. By reducing or removing shared server state, this point-of-failure is eliminated as well.
Links: Reduced Server State and Improved Availability.
As long as you have the ability to scale your system, you have the ability to support more customers.
Links: Improved Scalability and System Supports More Customers.
As customers are most businesses source of revenue, the ability to support more customers will correspond to the ability to generate more revenue.
Links: System Supports More Customers and Increased Revenue.
Most API operations are easy to design and develop in an idempotent manner. But, there are always exceptions, and some API operations will be more difficult to develop in this way, and therefore will result in a longer implementation time and a higher development cost.
A simple example of a more difficult idempotent operation is the idempotent creation of objects or resources, particularly those that do not have natural primary keys.
Links: Idempotency and Increased Development Costs.
Increasing development costs will increase overall costs.
Links: Increased Development Costs and Increased Costs.
Since revenue is generally profit minus costs, an increase in costs will result in an increase in revenue.
Links: Increased Costs and Reduced Profit.
Although for many API operations idempotency is just a matter of design, for some it will increase the complexity of implementing the operation. The operation may be required to compare the expected state of objects or resources with the current state to verify that the operation has not already been processed, for example.
Links: Idempotency and Increased Implementation Complexity.
Maintenance expenses are usually unavoidable on any development project. Additional features, tweaks, and bug fixes are the obvious maintenance costs, but every line of code in an application is a maintenance cost waiting to happen.
Complex implementations, even if implemented flawlessly, are more difficult and expensive to maintain in the future. Development staff will have increased training and ramp-up time to work on complex systems, and higher levels of quality control will be required for maintenance work relative to a simpler system.
Links: Increased Implementation Complexity and Increased Maintenance Costs.
Complex implementations tend to be more difficult to accurately and completely test. There are additional costs involved in communicating the complexity between departments, and time spent developing, verifying, and implementing test plans.
Links: Increased Implementation Complexity and Increased Testing Costs.
As maintenance costs are still costs, increasing them will increase your overall costs.
Links: Increased Maintenance Costs and Increased Costs.
As testing costs are still costs, increasing them will increase your overall costs.
Links: Increased Testing Costs and Increased Costs.
Some developers using your API will have expectations about its behavior based upon the HTTP specification. When there's a mismatch between your API and their expectations, it can be a source of confusion and frustration.
Specification expectations can also be coded into tools, frameworks, proxies, and other intermediaries between developers and your API. As an example, it would be spec conforming for an API framework to reissue PUT or DELETE requests automatically if they fail, based upon the expectation that these methods will be idempotent as per the HTTP spec.
Links: HTTP Spec. Conformance and Prevent Customer Frustration.
Increasing the utilization of server resources will reduce any premature hardware acquisition costs.
Links: Improved Server Utilization and Reduced Server Costs.
Customers get frustrated when the systems they're trying to use are not available. Increasing availability will reduce the potential for this frustration.
Links: Improved Availability and Prevent Customer Frustration.