Do I need tests when I have the compiler - Andrzej Jóźwiak - TomTom Dev Day 2020

Editor's Notes

• #2: Why such provocative topic? It is a question I asked myself after reading some internet drama. How do you start a drama in the internet? - you might ask. It’s always about what someone said. Let’s first introduce the actors, or an actor …
• #3: Uncle Bob Martin is a well known persona in our world. He wrote the excellent Clean Code book, he is a fervent advocate of Test Driven Development, he is also one of the original signers of Agile Software Development Manifesto. A tutor, a speaker, an experienced programmer. It is hard to discuss with such accomplishments. Nowadays he often joins discussions about software on Twitter where he talks about Clean Code, TDD and Agile Software Development, but is he always correct? This tweet alone caused a literal conflagration. Like you might imagine there are tons of people who agree with him and tons that do not. I presumed that like with every heated debate this will burn out on its own, unless … unless someone pours some more fuel into it. The person I expected the least was ...
• #4: … Uncle Bob himself. I know he is super into Clojure (a Lisp dialect) now but this statement couldn’t be left unnoticed. Like you can imagine the discussion was at square one again...
• #5: I too had to at least read through all the replies that soon followed and at least try to understand what is going on, what is Uncle Bob’s agenda here...
• #6: But maybe just maybe (and bear with me here) Bob is just messing with us as he is also an internet Troll (to add another thing to the long list of his accomplishments) So how is it with types? What other smart individuals have to say?
• #7: Working Effectively with Unit Tests – a nice book about good tests from 2015 One point for Uncle Bob here. Is it true though? In my own experience there is nothing like a good test suite. The majority of tests I’ve seen are fragile and brittle and break on any attempt of refactoring. This is a problem with the tests as a technique or us writing them? Hmm...
• #8: Martin Fowler, probably known as well as Uncle Bob Martin. He wrote several books, from which the best known is "Refactoring: Improving the Design of Existing Code." He also was one of the initial authors of the Agile Software Development Manifesto.​ In several different publications he emphasized that types and typing is important. One point for types?
• #9: Here a not so well known person at least in the Java world. Stuart is a Clojure developer and committer, he wrote several books about the topic. It seems he thinks types and compilation cannot substitute testing in any way. So far it's 2 to 1 for Uncle Bob.​
• #10: Mark is a well known person in .NET world. He is a book author and a speaker, his topic range from C#, Dependency Injection, F# and Haskell. His presentations about architecture are thought provoking and although .NET is not the regular Java cup of … tea? Coffe :) I highly recommend to read his blog and watch his talks. Introductions aside, what does he mean? We will talk about it further down the line but it all boils down to the number of possible implementations that a function can have. Just as an appetizer think about two one argument functions - first takes an Integer and returns an Integer: Integer → Integer - second takes a generic type A and returns a generic type A: A → A What do we know about these two functions? How many implementations of the Integer → Integer functions are there? How many implementations of the A → A functions are there? What do we know about A? We cannot create it, we cannot instantiate it, what else can we do? We can only return it. There is only one valid implementation of an f :: A → A function and it is f x = x So Mark help us to even out the score? It is 2:2 Let’s dig deeper
• #11: You may probably have not heard of a small framework for creating web apps in Ruby called Ruby on Rails? So David is the author and he started a very interesting discussion about the Test Driven Design and its impact on the design of our software. He hosted a very nice discussions about the topic with Martin Fowler, Kent Beck. Kent is the original author of Junit. The idea is simple yet terrifying, all the things we do in the name of testing are causing damage to our architecture. A component that without all these testing facilities would be simple and easy to comprehend, now has a lot of different things that allow injecting dependencies, indirection, argument, params etc. What was once simple now is complex. Very interesting talk I insist you should watch it. This quote is not especially connected with the topic of types vs tests but shows a very interesting point of view, that tests might also be undesirable thing...
• #12: This a quote that is constantly at the back of my head when I code. It is a thing I heard watching a talk about programming in OCaml (Objective Categorical Abstract Machine Language). How many times we write tests that check if our code in not in some illegal state due to the inputs we are passing to it. Would we need these tests if it was not possible to represent an illegal state? Illegal states – nah! that’s not me – you might think! But be perfectly honest now! As our classes grow with new fields, the possible combinations of states increases. I remember few years back I was working with a code base in C# for some audio related stuff. Class responsible for playing the given files like mp3 had over 100 distinct fields. Do you think anyone knew in what state the class was every time when it was used? I believe this small quote is the key to at least lowering the number of tests we have in our code base. This needs some preparation a design a thought. I think I heard this in a talk by Tomek Nurkiewicz “a week of coding will save you 15 minutes spent on design” ;)
• #13: What is a type system for? What issues does it solve? Why do we even bother with something like a type system at all? Wouldn’t it be better to not have types at all. I heared they make your productivity lower! :) All jokes aside type system helps us imensly: - it helps us with checking that the program has the intended properties - it is guiding us towards a correct program When a program typechecks we know that for a certain degree it is correct ;-)
• #14: We often want to check how our code will behave with the inputs we pass to it. Sometimes it is possible to pass inputs we do not know how or want to handle. This way we need to check all manner of things: ranges, constraints, structure.
• #15: Let’s not focus on obvious stuff: - that Integer in Java can be a null - that Integer in Java is finite - that there is something like MAX_INT and MIN_INT The only thing that we should focus on here is the domain of the function. Here we pass two Integers and return an Integer. Is this correct? What happens for the case in which b == 0? How can we solve it? Do we need to add defensive code before each invocation of the divide function?
• #16: We could >>narrow<< the domain. I emphasized the word because it is very important. For the divide function to work properly we need to pass as the b argument all Integers except zero. This way we will guarantee that the divide function will be total. Functions can be total – which means that they return result for every value of their domain. Functions can be partial – which means that there are values in the domain that the function does not (cannot) return a result. Like you can imagine it is much easier to reason about a total function. Narrowing the domain like this forces the caller of the function to supply correct arguments. In most cases programmers have tendency to push such “dependency” backward even more. So I need to call a function that expects a NonZeroInteger then I will ask for NonZeroIntegers also. This will help me to push the responsibility to the edge of our system.
• #17: A few examples of partial functions that can be made total thanks to domain narrowing. getFirst takes a List and returns “first” element of the said list. But what happens if the List is empty. It is clear that the intention was to work with lists that have at least one element. We can narrow the domain and create a NonEmptyList class instead which needs to have at least one element. We fix the issue with another approach, more commonly known which is just passing the default argument, this way anyone calling the function needs to handle the case of the empty list. take function returns given amount of elements from the given list. What should happen for the amount specified as -42. What does a negative amount even mean? We could narrow the domain and instead of all Integers we could pass Natural numbers which are all positive integers and 0. Better? It clearly shows the intent of the code.
• #18: Narrowing the domain helps with our objects aswell. Here we have an Order class with a List of Tickets for a Ticketing portal domain logic. Can we have an Order with an empty List of tickets. Can a buyer specify and empty order, what would be the point of such a thing? We could use NonEmptyList instead. Our intent is clearly visible.
• #20: The same function as in the input example. How can we fix the issue without narrowing the domain? What about the codomain, can we extend it somehow to encode the one possibility that there is no value – the b == 0 case. Let’s use type called Optional. It allows encoding a case where the value is missing. This type has different names in different langs and libraries: - Optional – Java 8 - Optional – Guava library - Option – vavr library - Maybe – Haskell - Option – Scala - Optional – F# - Option – Arrow (Kotlin) library You can think about it as: data Optional a = Some a | None
• #21: We exactly show through the types what values can be expected. Here we could talk about cardinality of a type. In simple words cardinality expresses the amount of possible values that the type can have. For example cardinality of a Boolean is 2 because there are only two possible values True and False. What is the cardinality of Integer? For all intents and purposes we can say it is infinite. What is the cardinality of Optional<Integer>? cardinality(Optional<Integer>) = cardinality(Integer) + 1 Why +1, because Optional can have the None, Nothing, Absent case. Which is only one such case. Returning Optional from the divide function we’ve increased the codomain by 1, to all possible Integer results we’ve added None.
• #22: We can do the similar thing for the getFirst function. To cover the case of empty List we can return an optional of the result. The types clearly shows what one can expect from the code. Similar example is getting a User from a Database by his Id. There might not exist a User with such Id, this is why we can return an Optional of a User. This second example with the database can be also fixed by narrowing the domain. Instead od passing a Long Id, we could have a specialized Id type that could only be created for exisiting Users. We would be certain that the function will always return a result.
• #24: There are some obvious issues with this code:- lack of cohesion between fields, depending on the ConnectionState some of the fields should not be instantiated. Should the fields be part of the enum?- lack of information which fields are optional, can we have a ConnectionInfo with a null session id that is CONNECTED? It seems ridicules but our API allows it. We often lul our selves into false sense of security and just say that "nah, my code will never get to such state, there is no need to test it" but we forget that: -- we do not work alone, there are other people in the team -- we hardly ever work in a non multithreaded environment, some thread races are hard to predict in the beginning -- if we allow for mutability then we invite problems with open arms. How many times we had an object that models a Route Plan that is mutable and has a departure and destination. How many times we allowed to have a null departure and a null destination or a departure without destination? Nothing that we wanted to happen? -- it is not easy to understand such a mess of fields what are the intentions behind them Issues: -- lastPingTimeInMs - if there is no information about time available then what value should we use instead? -1 like in the good ol' C days? Isn't this too dangerous? It's like a forbidden fruit, maybe someday we will send error information through this field -2, -3, -4 etc sky is the limit -- connectionStartTimeInMs and connectionStopTimeInMs has the same problem as the lastPingTimeInMs but also shows a different issue. Can we have a connectionStartTimeInMs without a connectionStopTimeInMs? Surely yes! Does a connectionStopTimeInMs without a connectionStartTimeInMs has sense? Probably depending on the context and the domain. -- sessionId if the information about session id is unavailable what should we put in this field? a null? ok but is any String a valid session id? Is an empty string a valid session id? is a string composed out of only white chars a valid session id? This field also suffers from the same problem that lastPingTimeInMs did, null is super tempting and depending on the situation can mean different things like errors, exceptions, warnings etc. Do we like to decipher the fields meaning and valueseach time we read the code? I definitely do not! What should we do then? The answer is types! - do not use bare long and strings for modeling - show the optionality explicitly - something else?? something better let's check!
• #25: First let's add missing types. For example instead of a bare String for the session Id type we can use an explicit type with a proper name. Instead of long for time in milliseconds we can use a java.time.LocalTime or java.time.DurationEverything that can be absent is wrapped with an java.util.Optional.Looks better but still allows for illegal states:- connected with no connectionStartTime- connected with no sessionId- disconnected with no connectionStopTimeWe could list and list the issues for some timeWhat about our earlier idea to mix the ConnectionState with information that correspond to it? Although its visible which field is optional it is still easy to have an illegal state.
• #26: Remember the cardinalities we were talking about? Let’s talk about different kinds of types: - Sum types – you already know one example, which is Optional. Their cardinality is the sum of cardinalities of cases of the type. - Product type – you already know them. Each normal Java class denotes a product type. An example class User: User { String name; Int age; Boolean isRegistered; } Cardinality of the User is c(User) = c(String) * c(Int) * c(Boolean). User can be any combination of the types it contains. This is why product. Let’s create a sum type denoting the Connection State which will contain fields that are relevant for that particular state. There is no way we will have illegal state happening in such model.
• #31: We can move optionality out of the value into the type, thus easier modeling is possible on the type level, we can express the intent more easily.
• #34: It seems that not only the RIGHT types are needed but also RIGHT tests!
• #35: Property Based Test example Message A → To Protobuf → Protobuf → To Message → Message B Message A == Message B (the property is that we should get the same message as the result if we chain the functions like above)
• #37: What do we know about this function:- it takes some type A as an argument- it returns some type A as a result- we do not know anything about type A- we cannot create type AWhat can we do:- we could return null! It is Java after all but let's not be silly!- we can only return the value we got!
• #42: Similarly to identity function we cannot just create a new A out of thin air but- we can return an empty list- we can return a list of smaller size- we can return a list of larger size- we can return a list with the first element repeated- we can return a list with each element doubled or tripledI guess we could go on. Also you might think that this bares the end ofour journey showing that tests have won and types cannot do anything aboutit.If only there was a way to encode the properties we need into the types:- the returned list should be of the same size- the returned list should have the same elements as the original one- there should be a way to encode the order into the typeSo if we got a list [1, 2, 3] we should return a list [3, 2, 1]Can such magic be done through types? Yes, kinda, but not in Java.
• #43: From wikipedia: In programming language theory and proof theory, the Curry–Howard correspondence (also known as the Curry–Howard isomorphism or equivalence, or the proofs-as-programs and propositions- or formulae-as-types interpretation) is the direct relationship between computer programs and mathematical proofs. In other words, the Curry–Howard correspondence is the observation that two families of seemingly unrelated formalisms—namely, the proof systems on one hand, and the models of computation on the other—are in fact the same kind of mathematical objects. If one abstracts on the peculiarities of either formalism, the following generalization arises: a proof is a program, and the formula it proves is the type for the program. More informally, this can be seen as an analogy that states that the return type of a function (i.e., the type of values returned by a function) is analogous to a logical theorem, subject to hypotheses corresponding to the types of the argument values passed to the function; and that the program to compute that function is analogous to a proof of that theorem. This sets a form of logic programming on a rigorous foundation: proofs can be represented as programs, and especially as lambda terms, or proofs can be run.
• #45: What if we could pass value into the type level. Here we have an example in which we have an Integer List in which all elements are above 10. This is not possible in Java but we could imagine how it works.
• #46: Through dependent types we can generate proofs of our code. There are specialized languages that allow creation of such proofs.
• #47: Most notable and well known proof assistants.
• #48: This all seems like a lot of extra hassle, the question is if anything really useful has been created with the use of proof assistants? And by useful I do not mean someones PhD degree here but something that we might use on a daily basis. Real software with the emphasis on REAL!!! with an upper case R!
• #49: Safe and verified operating system kernel seems like an important thing.
• #50: This often bothers me. We find a lot of bugs in our code but do we ever question the result generated by our tools i.e. compilers? Except the most spectacular of bugs where the compiler does not generate the code or just crashes for what seems a valid code, what about all the silent errors? Where the generated executable is incorrect although the source is? This is not a new question. It was asked several times already. In 1995 an empirical study found integer division errors in 12 out of 20 available commercial compilers. In 2008 another study found issues related to volatile memory access in 13 commercial compilers. This does not sound scary but if we think deeply about it this means that these kinds of issues may occur in many critical embedded systems. What about operating systems? You wouldn’t want an issue in a heart-lung machine? Or a power plant cooling system? A study in 2011 found 325 unknown issues in different compilers. Who knows what will the future bring? CompCert has managed to produce a verified C compiler through the usage of a proof assistant Coq. http://compcert.inria.fr/doc/index.html
• #51: Yes yes, I know what you will immediately say: this project is on github surely this is not anything any enterprise should care about! You couldn’t be further from the truth, this is a project done by several organization one of which is a well known company Microsoft. You might say now that all these examples were not real software. Real software are all those CRUDs, web shops, games! Yes games. We can discuss all about all that academic mambo jambo but where are proofs for games?
• #52: What if I told you there is a guy who classifies languages not as Turing Complete but Pac-Man Complete? Who understands TDD as Type-Driven Development? The screenshot above is from a game written in a dependent-type language Idris. It is also a proof that you can embed a quite complicated logic into the type system and can have it verified on the compilation level. Ofcourse these all seem like academic examples and no REAL programming languages bother with things like this?
• #53: I understand that most people put Haskell, OCaml, F#, Agda, Idris, Coq and others in the same basket as some esoteric languages like Whitespace or Brainfuck. Nothing more then a cute toy to talk about on conferences or make MsC or PhD about. Not enough curly brackets, weird syntax, strange language. These FP weirdos are always talking about morphisms, monoids, monads, functors, applicatives all these complicated words that are alien and scary. You keep thinking like this until you revise the acronyms and words we are using in the Java world: SOLID, DI, Generics, IoC, CQRS, MVC, MVP, MVVM, Polymorphism, Inheritance. You are not afraid because you are used to this already. I know about two more mainstream languages that plan on adding dependent types or proof assistance. Rust has a RFC that attempts to add Pi type support. Kotlin’s library Arrow plans on adding simple proof asistance for its Arrow-Meta compiler plugin. Did you hear about a little language called Ada95? The next generation Ada language has a proover called Spark and they advise programmers to use it. Times change!
• #54: Who knows maybe Java will get dependent types some time in the future just after higher kinds and ad-hoc polymorphism ;-) This does not sound so funny anymore. We already have records at our disposal, they plan to add some form of pattern matching, so we will see who will have the last laugh.
• #55: If you would like to read more about the topic you have this short bibligraphy, also on each slide you have links to useful materials that were mentioned.
• #56: I hope you did not fell asleep! Questions?