class: center, middle # Errors and Partial Functions --- class: center, middle # Disclaimer Beyond my general disclaimer, these are personal notes that may be out of date. Read generously. --- class: center, middle # Day 1: What is an Error? --- class: left, middle ## Wheel of Fortune When someone spins the wheel, which of the following would you consider to be an "Error"? - $500 - $1000 - WEAK_SPIN - JACKPOT - BANKRUPT - WHEEL_ON_FIRE - LOSE_TURN --- class: left, middle Most people will divide this list in the following way: Valid: - $500 - $1000 - BANKRUPT - LOSE_TURN - JACKPOT Errors: - WEAK_SPIN - WHEEL_ON_FIRE --- class: left, middle There's another way you might model this, however: Valid: - $500 - $1000 - BANKRUPT - LOSE_TURN - JACKPOT - WEAK_SPIN Errors: - WHEEL_ON_FIRE The idea here is that `spin-wheel` returns a `SpinWheelResult` instead of a `Wedge`, which could then have `WheelLogicViolation` as a return value, making `WEAK_SPIN` a proper value *instead of an error*. --- class: middle The core idea here is that nothing is *inherently* an error. Rather, we choose to model things certain ways because they modularize our concepts well - we like to think of a `Wedge` as the output of spinning the wheel because it results in a conceptually concise "happy-path", but this is purely an artifact of what is an ergonomic way of *thinking about things*. --- class: middle ## Partial Functions There is, however, a more formal way of thinking about what an error is. --- class: left, middle A function `f: Domain -> Codomain` is simply a mapping from elements of a Domain to elements of a Codomain. A **total** function is simply one that is defined for all elements of the Domain, in contrast to a **partial** function, which is undefined for certain inputs. --- class: left, middle ### Example: Division The division function `/: (R,R) -> R` is undefined for the denominator value of `0`, and is therefore a **partial** function. If we had instead defined it as `/: (R,R\{0}) -> R`, this is now a **total** function. What are some other examples? --- class: left, middle ### Arbitrariness of Totality As we can see from the division example, Totality is mostly a *modeling* property by how you choose to *describe* your function - in a very real sense, both of those functions are division *and* are the division that we're all familiar with. The question of whether or not the function is "total" is simply a matter of how you want to describe its possible inputs and outputs. --- class: left, middle ### Partiality-Error Equivalence The fact that partial functions exhibit the same kind of "modeling arbitrariness" as errors is not a coincidence. Indeed, everything that we think of as an *Error* can be formalized as a *cause of Partiality* - put another way, it's *Errors* that make total functions *partial*. The division function that accepts `0` has a `DivideByZero` error. The function that asks a database to look up a particular value has a `NetworkTimeout` error when the internet connection gets cut. --- class: middle ## Pure & IO-based Partiality --- class: left ### Pure Partiality The "pure" set of concerns revolves around the *input* values that are passed into your functions, of which there are typically two sources of error: -- - **Static Enforcement**: The point of describing your Domain and Codomain is to only have those values enter and exit your function. A dynamic type system, while allowing you to describe your function, does nothing to prevent invalid values from runtime evaluation. - Example: Python will allow you to try and divide a string. -- - **Imprecise Definition**: Your Domain may be specified in a way such that there are values with no corresponding mapped values in the Codomain. - Example: Passing in 0 to the denominator of a division function. - Example: Querying a database for an ID that doesn't exist. -- In today's appendix, we'll discuss Dependent Typing, which is a type system powerful enough to eliminate the divide-by-0 problem, which raises an interesting question: Is it possible to eliminate the second class of errors? -- It's actually a consequence of the *Halting Problem* that type systems cannot be powerful enough to do that. --- class: left, middle ### IO Partiality The second source of Partiality is of fundamental importance, because computing must run on *real world* machines, which has two main sources: - **Physical Layer**: every function ultimately runs against a real machine despite our virtualized programming model. Therefore, *every single function is Partial*, since there are physical constraints (such as `OutOfMemory`) that can cause potentially any function to fail. - **Side-Effect**: Virtually every function that is of *practical use* will perform some side-effect in the real world that *may fail*. --- class: middle ## Practice: Query a Database Let's say you've got a database, and you have to query a table to fetch a particular record by its ID. What are the possible errors? How would you classify each of them by the type of Partiality they introduce? --- class: left ## Error Representations Before concluding, let's do a quick survey of the different kinds of error representations that are commonly used, and discuss the kinds of Partiality they tend to represent. -- - Sentinel Values: Returning `null`, `false`, `-1`, etc. -- - Convention-based Sum-types: Returning `val, err := fn()` as in golang -- - Exceptions and Panics: Java's `throw Exception` or golang's `panic()` -- - Sum-types and Monads: Scala's `Try`, Haskell's `Maybe`, etc. -- When you run through a bunch of different use cases, you start noticing that there doesn't seem to be much of a rhyme or reason for people using a particular error representation for a particular use cases - there's a lot of inconsistency and personal preference/familiarity dictating these choices. --- class: left ## Appendix: Dependent Types As we established previously, it's not possible for a type-system to be strong enough to eliminate the possibility of Pure Partiality errors, but it's nonetheless interesting to explore more powerful type systems as they *can* create tighter bounds on the domains of your functions, which can *reduce* the number of Pure Partiality error cases you must handle. ```idris safe_div : (x : Int) -> (y : Int) -> {auto p : so (y /= 0)} -> Int safe_div x y = div x y ``` --- class: left, middle ## Wrap-up - Today we've equipped a mental model for what an error is: > For every function you write, anything for which an output value isn't defined for a particular input, or an IO error causing incorrect termination, is a cause of partiality and should be modeled as an error. - Work through the exercises at (https://github.com/justin-yan/errorsandpartialfunctions) in order to develop a better feel for the monadic interface and pattern matching. --- class: middle # Day 2: Error Handling --- class: middle ## Good Error Code --- class: left ### First-class Total-ize Your Functions -- In the real world, every function is "total" in the sense that your programs will always be taking *some* action in response to a function. When division by zero occurs, if you do not specify behavior, your language runtime will always do *something* (even if that "something" is "nothing") - whether it be to tear the process down, silently return nothing, or hang forever. -- The essence of robust software is not to *eliminate errors* since errors are simply *branches of other behavior* due to how we have chosen to model our functions, rather, it is to *exhaustively handle those branches in an intentional fashion*. -- > Error handling mechanisms exist in order to help you convert Partial functions into "Total" functions. -- Let's consider: ```java public static Integer divide(Rational numerator, Rational denominator) throws ArithmeticException ``` this signature is essentially `/: (Q,Q) -> Integer || ArithmeticException`, which is Total! --- class: left ### Error Code is still *Code* -- Errors are simply other branches to be dealt with, which means error-handling code is *still code*, which means that *Modularity* matters. -- In particular, there are three guiding heuristics that are particularly worth considering for error handling code: - Authority - Volatility Risk - End-to-End Principle --- class: left ### Authority One of the key questions you should be asking yourself when it comes to handling an error is simply: "Who has enough information to actually handle this error properly?" -- Let's consider a case: If you have a platform that abstracts away integrations with many different partners, how would you use the Authority heuristic to decide which errors the platform should encapsulate and which it should bubble up? --- class: left ### Physical Failures An interesting consequence of the Authority Heuristic is in how to deal with Physical Layer failures. The reality is that these are inevitable and unhandleable - your code cannot remediate failures in the physical machine it runs on - and as such, they are not worth modeling. Process Teardown should be how we proceed, and recovery must be delegated to the *meta-system* (such as a daemon or a human). This then means that *all code* should be written as if it could fail and terminate at *any moment of time*, and that the resulting state should be such that the meta-system's recovery process will result in *correct application state*. --- class: left ### End-to-End The end-to-end principle, in conjunction with Authority, is helpful for understanding Retries. -- Developers incorrectly think of retries as a way to handle errors. The existence of a retry policy might *reduce* the incidence of errors that propagate, but they cannot *eliminate* errors from happening, and therefore cannot save you from writing code to handle error cases. A retry policy will result in either a function that never returns (infinite retries means infinite loops - consider an upstream system dying) or the error propagates more rarely (but still propagates). -- Fundamentally, *some system* has the authority to execute full and total remediation of the error, and no matter how strong of a retry policy or how many domain transformations you insert in between that system and the cause of the error, there is no getting around the fact that that system must understand how to handle the error. Retries are an optimization of *reliability*, not a substitute for *robustness*. --- class: left ### Volatility Risk Another major modularity principle is that of Volatility Risk. -- Code to handle errors is the same as code to handle the happy path - if you have components that are vulnerable to change, then you'll want to decompose your code along those boundaries, so that changes do not risk contaminating the rest of your code. -- A great example of this is database IO. The vast majority of server-side logic allows database-specific exceptions or errors to leak out of the persistence layer adapter. This means that *anyone* in *any other* part of the codebase could `catch (SQLException e)` and write logic that is now dependent on the specific database you are using. If you were to ever try and change the database you were using, you'd now have invisible dependencies all over the codebase that would be very difficult to tease out. --- class: middle ## Ideal Error Handling --- class: left ### Axioms What should an ideal form of error handling do? -- - It should use first-class language branching mechanisms, allowing developers to take control of all causes of partiality with the same code they handle all other branches. - Developers should be forced to model *and* handle errors exhaustively (i.e. write and handle *total* functions). In order of importance: - Statically checked types - *safe* types (no null references) - GADTs with compiler-enforced destructuring. - Statically enforce that return values be used. --- class: left ### Sentinel Values > I call it my billion-dollar mistake. It was the invention of the null reference in 1965. > Tony Hoare Returning sentinel values has had a history of improper use, since you ultimately depend on *documentation* to realize that certain values represent *errors*, and it's very easy to silently fail by simply *using* the sentinel value as if it were a legitimate return. --- class: left ### Exceptions Exceptions were a next step that had two major drivers: 1. A separate value with explicit control flow implications made it very difficult for people to silently and accidentally ignore an error. 2. Unchecked exceptions also allow us to avoid contaminating literally every single function signature with a error return type because of the partiality causes that result from physical-layer failures, such as OutOfMemory. Unchecked exceptions, unfortunately, have the problem that they don't actually statically force developers to handle errors - it's perfectly fine to just run a function and hope that it doesn't blow up. Checked exceptions are a little better, but they require an entirely parallel control-flow handling mechanism and a second "return type" that goes against the idea of this just being "code". --- class: left ### Error Monads I am partial to the monadic style of error modeling. -- - A stdlib PartialFunctionError datatype should exist and be extensible (IOPartial & PurePartial) to describe error values. -- - An Error Monad should exist (and supersede Option/Maybe/Try - covering both Pure and IO partiality) and be constructible from a sum-type with `PartialFunctionError` in order to convert functions from partial to total. -- - Exceptions or the concept of Process Teardown remain in the language to deal with Physical Failures. -- This hits the major points: - You use GADTs and first-class pattern matching, as well as a Monad structure to destructure and handle errors. - Your compiler forces people to use the error values, which are then impossible to misuse due to compiler-enforced destructuring. - Panics are limited to specific use cases and prevent unhandleable partiality from being present in *every* single function signature or type.