The Invisible Interface
Technical Debt seems to be omnipresent in software systems, yet despite our best intentions to not take shortcuts, our systems seem to organically decay over time - it was never anyone’s intent to make a mess, and yet here we are. This entropy comes from many places, and one of the major sources is violating the Invisible Interface.
Software systems are a model of reality. Your Customer database record isn’t actually a Customer - it’s a representation of one. The problem that all models have is that they simply are not base reality. While your model should have high fidelity to reality, the complexity of the real world is far in excess of what you can reasonably represent in code and data. This means that as you define concepts such as “Customer”, you will never be able to reify all of the assumptions and properties you can mentally visualize into code, schemas, and application logic.
This results in two interfaces: the Formal Interface defined in code, and the Invisible Interface defined in your head.
Let’s say I’m building a marketplace where individuals can post listings and sell used goods to other individuals on the platform. Everyone can either buy or sell, so I might choose to represent all of these individuals with a single concept:
case class Customer(name: String, email: String)
This “Formal Interface” covers the tiniest fraction of the mental concept of a “Customer”. As you can see here, the “Invisible Interface” that we have in our minds has data and capabilities far in excess of what has been represented in the Formal Interface.
Indeed, no matter how much data and how many capabilities we add, the “Formal Interface” will always lag behind reality - which is perfectly fine! Many of those gaps are not useful (for example, blood type is something you could capture) but they are still there.
Violating the Invisible Interface
While it’s certainly a problem to have a Formal Interface with major and relevant gaps relative to its Invisible Interface, it turns out that this isn’t usually technical debt - it’s more like feature incompleteness.
Technical debt instead tends to come from the addition of new use cases that violate the Invisible Interface. If you were to imagine a code generator that perfectly mirrors reality and generates all of the code and data that completely captures the essence of the concept being modeled, violations occur when someone uses the existing Formal Interface in a way that breaks functionality on the Invisible Interface. This is a problem for a few reasons:
- Extension: The unviolated boundaries of the Invisible Interface represent future extension possibilities of the Formal Interface.
- Assumption: There may be other systems that rely on the Invisible Interface that are now broken by this violation.
- Union: Your Formal Interface must now satisfy the union of requirements of the different use cases that apply to it, making it more difficult to modify and slim down the existing Formal Interface implementation.
My used goods marketplace is now humming along, and to supercharge the next stage of growth, I want to make it possible for businesses to operate on this platform as well. If you look at the Formal Interface for
Customer above, there would be absolutely no reason I couldn’t just stuff
Business into that entity as well - they have a name and an email, and they’re also customers who need to buy and sell stuff on the platform, right?
var person1 = Customer("Justin", "firstname.lastname@example.org") var business1 = Customer("Acme Supplies", "email@example.com")
Let’s consider the different breakages that may result from violating your invisible interface in this way:
- Extension: Features that you may have desired previously, such as a “Home Mailing Address” or a “Key Lockbox” are now not quite so obvious to implement. Business features that you may wish to add, such as “Designated Agent” may also not be sensible for individual people.
- Assumption: The most common breakage here is for analytics, as that function typically integrates with the data store directly and heavily relies on assumptions about semantics. For example, customers with large sale volumes may previously have been filtered out as buggy outliers.
- Union: Now that you’re supporting businesses and people, deleting fields and functionality becomes more complicated - you may want fields that are more generic so as to generalize across both businesses and people, but when it comes time to deprecate, you have to determine whether either of those entities are using that data, handle deprecation for both, and only then eliminate the field.
It is, of course, possible to remediate some of these concerns with some other changes. A very basic version of polymorphism might look like just adding a
var person1 = Customer("Justin", "firstname.lastname@example.org", CustomerType.PERSON) var business1 = Customer("Acme Supplies", "email@example.com", CustomerType.BUSINESS)
and then change everything that interacts with customers to switch on the CustomerType.
The broader point is not that polymorphism and re-use are bad, but simply that the concept and Invisible Interface of
Customer has evolved despite the Formal Interface not changing one bit.
When this happens, it is extremely important to re-establish a clear Invisible Interface for that new concept. Without that clarity, developers adding new use cases will not understand if and when they should be extending
Customer, and this can result into the
Customer concept devolving into a meaningless container for anything that has a name, an email, and wants to buy or sell stuff on the platform. This is the type of entropy that leads to modeling problems like “Customers form a tree structure since businesses will need dedicated accounts for their sales personnel”.
While simplified, I would wager most businesses have some domain in their ecosystem that exhibits these challenges. These decisions are happening constantly throughout an organization on a variety of scales - and while it may not be of consequence for a sufficiently localized concept, if you stack up enough change without taking care of your Invisible Interface, you’ll find yourself in trouble.
As a parting heuristic, be cautious when someone says something like:
If you really think about it, X is a kind of Y, right?
It’s almost always a violation of the Invisible Interface, and while you may decide to go through with it, make sure that you’re updating your Invisible Interface as well.