Every professional in the software industry knows that building software is a highly unpredictable business. This is why we don't like the question of “How long is it going to take?". The thing that always puzzled me, however, is that that unpredictability always seems to point in the wrong direction: software seems to almost always take longer to build than expected. With practically any other process, one would expect something similar to a normal (binomial) distribution around the expected estimate. Sometimes the project would take more time than the estimate, and sometimes less. But with software, it almost always takes longer. I can't help but ask why that is. I believe the answer has many different components, but one of the most important (and most insidious) ones is implicit requirements.

Every piece of software has at its core a small set of use cases that generate the primary value-add for the end user. Most of us tend to naturally focus on these core set of requirements, especially at the beginning before any of the software has been built. And when called upon to estimate how long the software will take to build, we tend to judge the size and complexity of this set, and then give an estimate based on that. Unfortunately, in most typical software, the core use cases represent only a small fraction of the total set of use cases that need to be built. There is a much larger set of requirements that are required to be in place for the core set to work at all, and these use cases are typically not visible until a much deeper analysis.

Let's illustrate by using an example. Imagine you wish to write a simple messenger application. To keep things simple we will pretend that this is meant to be a web application rather than a mobile application, but the arguments would apply equally well to a mobile application. In this app, you can select a person you know and send them a message. They can respond, and this will show up in the same thread. You can also add people to a thread (but they will not see any messages that were sent before they were added, to keep things simple). This application is not tied to a specific company, and people may send messages to people outside the company as well, but it is meant to be suitable for a commercial environment. So how many core uses cases are there in such an app? Well, there is the actual thread view where I can see the messages in a thread and post messages to the thread. And there is the use case of adding a person to the thread. A new thread is created by adding the first person. So all in all, there are only two core use cases (we did try hard to keep it simple). Almost all of the end-user value is generated by these two core use cases.

These use cases are not viable, however, without a number of additional use cases that we did not describe above. To begin with, finding a user means that they have a user profile of some sort that we can search on; this is required by the core use case of adding a user to a thread. Implicitly, we need a use case for creating a profile, one for editing a profile, and we need a login mechanism. If we have a login mechanism that relies on username and password, we also need use cases for handling forgotten passwords. And the most typical mechanisms for handling forgotten passwords rely on security questions, so now we need a use case for managing those as well. All of a sudden, we went from having the two core use cases to having another three for managing profiles, forgotten passwords, and security questions. Realistically, our core use case of finding a user implies another three use cases we did not foresee until we looked deeper.

Let's take it another step further. If you have ever tried to find someone on Skype you will certainly know the frustration of typing a first and last name in the search bar and getting a large number of returned results, with no clear way of identifying the individual you want. If you are lucky, that user has set a profile photo, but even then there are often people that have multiple accounts with the same photo (I have created one account for each company I worked at, in order to be able to keep my work account and my personal account separate). In short, that profile will need something more than just first and last name in it. On Skype, I often check location as a way to narrow the search results. If you plan to include location in the profile, then you will most likely end up having a list of countries and probably State or Provinces. It would not be a good idea to hardwire these as they do change (what is now the Republic of North Macedonia used to be known as FYROM as recently as January 2019, and did not even exist before September 1991). And it is not just countries and provinces. If you have phone numbers in the profile you will need a list of country codes. If you display flag icons associated with the countries, you will need to manage those as well. So our implicit use case of managing profiles now implies a number of other use cases for managing reference data.

Then there use cases implied by legislation. The profile information we talked about is protected under GDPR in Europe. If you allow access by European citizens (even if your company is not based in Europe), you are obliged to abide by the rules of GDPR. In general, this will imply (at a minimum) obtaining user consent (unless you use some other legal basis for obtaining the data, but that unlikely in the application as we described it). You may also want to cover the right to erasure, the right to access, the right to rectification, and the right to restrict processing. Sounds like we might be adding another five use cases to our total.

And then there are the use cases required by enterprise customers. Generally, companies like having control of any of the accounts that are considered company accounts. This means that your enterprise customers will expect that, at a minimum, they can disable accounts that are considered corporate accounts. This means another use case for administrators to manage such accounts. But it also implies having an entity representing the company (to which the corporate accounts can belong), so you can add another use case for creating and managing the corresponding entity. Oh, and you will need to add a use case for managing the syadmin accounts of a corporate entity. And some of your enterprise customers will insist on having SSO capabilities (since all their other applications work with SSO), so add another use case for the SSO login, and another for managing the SSO certificates, identity providers, etc.

And finally, there are requirements that are implied by simple prudence. The moment you have things like permissions and SSO involved, changes to the configuration in your application become a security concern for your enterprise customer. Most certifications require that they have audit processes in place for changes that impact access control and authentication, so add some infrastructure for audit logs, and a use cases for searching and viewing the logs. You probably will also be asked about your Disaster Recovery and Business Continuity plans - once again, they need to show they have something in place in order to maintain their certification, and by extension they will ask that of you. Then there's things like performance monitoring and availability monitoring that you probably want to put in place both for yourself and your customers. And so on, and so forth.

Hopefully by now you are beginning to see how our two core use cases exploded into a set much bigger than what we originally expected. We admittedly cut a few corners for simplification: not all uses cases are born equal, and some of the implied use cases you might be able to postpone for a while, but that does not change the conclusion: the bulk of the requirements comes from the implicit use cases that are there to support the value-generating core, not the core use cases themselves. The core uses cases really are just the tip of the iceberg. Note that there are two tendencies in Agile that tend to make this trend significantly worse:

• The focus on business value tends to keep the spotlight on core uses cases and increases the probability that implicit requirements will be missed.
• The emphasis on relative estimation means that implicit requirements are never included unless the estimate is relative to an epic which took into account all implicit requirements. This is rarely the case.

In the end, most teams I know of try to paste on a solution on top of the Agile framework, by trying to independently identify all implicit requirements using hard-earned experience, and then entering them into the backlog as independent user stories. Typically these are judged as being of lower priority, and end up being implemented in the later parts of the life-cycle, meaning that the architecture and implementation already in existence (and, often, already deployed to production) does not take them into account. As a result, they are often more costly, and much more risky than they need to be. That is simply one of the shortcomings of Agile - if anyone told you Agile didn't have any, they lied, but that is the topic for yet another, future, post.