George Fairbanks, Google
In this experience report, George Fairbanks discusses his recent experiences from assembling large bits of software. He reminds us of how sneakily dependencies become complicated through the analogy of the frog in a gradually heating pot of water. Architects could solve the complexity problem up front in a waterfall process, but how and when can they architecturally intervene in an incremental development process?
Starting with a simple example of a coffee maker, George gives some examples of testing the pump and heater in which the test results are good because the tester has managed to bypassed the problem. Now, though, the dependencies have dependencies. While each problem is separately solvable, the whole has become a burden. The tester must specify a larger and larger list of things that aren’t actually related to the test under study. So you can shuffle the complexity around from place to place, but it still ends up in a big pile that you have to manage. And when you make it easy to perform unit tests, you’ve taken away all the relationships between them.
George considers the intersections between dependency injection, code modularity, and testing. He describes the software pattern of dependency injection, which allows you to inject objects into a class rather than allowing the class to create the object itself. To create package modularity, you want to group source code into modules, but people tend to inject classes instead of chunked modules. In testing, you may have many fussy, mock, or fake dependencies, and it is not uncommon for test code to be larger than production code. The result is an inability to see the forest for the trees. How do you know you have a real-enough configuration of the software you’re testing?
In George’s photocopier example, once you consider all the parameters – including type of paper feed, copies per minute, and stapling – how many legal configurations are there? This is what architecture is looking for. Architects get intellectual control through concerns about modules and testing. Most projects don’t explicitly think about system configuration. It’s a missing abstraction. What is the configuration, with code collected into modules and their legal arrangements? How is this system allowed to vary? In a hypothetical dependency injection timeline, what starts as a simple system may become a write-only dumping ground, where people put stuff in but are afraid to take anything out because they don’t know what the ripple effects will be.
This is not new. For example, DeRemer and Kron wrote “Programming-in-the-Large Versus Programming-in-the-Small” in 1975, but we still haven’t solved this problem. Refactoring becomes so expensive that it would be easier to rewrite the whole system instead of doing the refactoring.
In an iterative development cycle, when is the time to practice architecture? It’s more difficult today, and there are cultural obstacles as well as process obstacles. The monster of complexity creeps up slowly and must be attacked, but where to inject architecture techniques is unclear. It’s not a matter of not knowing what to do or recognizing that it’s important; it’s a matter of when to do it in a development cycle that focuses on the bare minimum to build the next feature. The future is bright if we can solve this problem.