Earlier in my career I was put on a project from a recently fired developer that was some of most convoluted and buggy code I've ever seen. I and the other developers that were put on the project to save it all lobbied _hard_ to rewrite it from scratch.

We were denied, so we tried to trudge along and ended up rewriting the worst component of the bunch, as it was so buggy that rewriting it and then making facades to maintain the old interface was the only conceivable way to make it work.

It went so well we started doing it to other components, and when two components on each side of the facade had both been rewritten, we found we could get rid of the facade. Without putting the project on stall, over time, we had essentially rewritten the entire thing.

It's a really good way to do things. I've been on rewrites before and they didn't go as well (second syndrome effect, etc).

I started out writing unit tests for it. There weren't any, so I wrote them until I had unit tests that tested everything. Those unit tests also document the behaviour of the module. So then I started asking people what it was really supposed to do. Then I refactored it into something more maintainable. Only after that did I change the unit tests to reflect the behaviour the module was supposed to have, and based on that I finally wrote the code we needed.

The unit tests were my anchor in the storm. They keep me grounded. They mean I can do anything to the code, while keeping tabs on the impact my changes have.

The trouble with unit tests is their small scale. You write 3 Mb of source code filled with unit tests, does that give you any guarantee the program will even start when you call it ? (A point of frustration that I have had with other developers is that they deliver "a product", which immediately sigsevs upon execution, and then claim that's not possible, all tests pass. Same with memory leaks, which are never caught by unit tests, not in C++ and not in Java either).

First thing to do is to get the program having a backend and a frontend separately, then introduce a system test. Something that does what the main customer(s) do with your application, and does it while seriously stressing the system. You need it to send out mails ? Make it send out 10000000 mails while only using the backend calls the webserver calls while activating a sleep(1) in the mock database routines, and impose a memory limit of half a gig, checking if memory usage actually drops when you stop pushing the application. Make it use an actual database, and make it send actual emails to a fake "smarthost" email server. If it doesn't complete in 5 minutes, something is wrong. I've been meaning to implement this test atop docker. I tried to generate VMs to test the full application, start the app in a simulated network and run most of the actual production code, but making the VMs took ~25 minutes (and they weren't even on the correct machine by then), which made it unusable. I wonder if docker can do better.

Yes this test will be flaky, yes if it does indicate failure figuring out what's wrong is bloody hard, yes it means abandoning several holy cows of software development, yes it means some program will tell you where to focus attention, yes it will force you to actually think about debugging production failures BEFORE shipping, it forces you to have test systems for the program's components, it generally makes you deal with reality. You will have discussions with this test if you do it right ("No that hash-table MUST be faster than that linked list ! Aaaargh !"). And that is something sorely needed in software development.

Second thing I use this for. Boss-man wants a new feature. Ok, sure. I add a system test that uses it. Needless to say, it will fail. But it allows me to have a good think about what the high-level changes needed are, without unit tests constantly badgering me that some method now accepts different parameters. It will tell me if I got it right. And when I deliver it, the freaking program will start, and won't crash the first time someone uses this feature. You tell me how to make unit tests that deliver 10% of those guarantees. Forgot the big picture after spending 2-3 hours on getting a module fixed after adding new functionality to it ? Just single step through the system test and see where it blows up, and hey. There's the next thing needed for the high-level change.

Unit test programmers are terminally afraid of changing module interfaces to suit higher-level objectives. Given the amount of unit tests needed to even approach 95% coverage, I fully understand the resistance. But if they choose to write unit tests and I need the module change, I just go in and do it, disabling the unit tests. I tend to get some attitude in code reviews.

It also means that replacing the database with something else is something you can just do in an afternoon. Changing the core logic of the system, from a set of rules to a rule interpreter ? Changing that rule interpreter to jython ? No problem, the test will tell you if common use cases of your product are actually still working. And that's what the business/boss (should) care about.

Ops will love you if you do this. You will consistently deliver a working product that operates well within known parameters, as opposed to a system where the tiny wheels all turn, just not in the same direction.

Unit tests optimize for the wrong thing : they make sure programmers' jobs become easy, because the modules behave the way they "should", and can be changed with "predictable" results, which of course turn out not to be predictable in the real world. Furthermore the interactions between the "should"s of multiple modules ... turn out to be less than simple. Unit tests allow you to say "not my fault" (e.g. if your method is merely slow, but suddenly gets called billions of times because some other system changed and makes the system suddenly crash when too much is going on, but not due to a logic error). They make it easy to change systems by only touching the smallest components, which of course also massively limits the changes possible in a system.

Unit tests impose strong limitations on the changes that you can make to code (in reasonable time). Therefore, for the large majority of code, they are a burden, not a boon. There is one (big) exception. Algorithm and data structure code should be thoroughly unit tested. In most projects however, you have maybe 2-3 methods that fall into that category. If you have more, find an open source project to replace 50% of them, or figure out how to use more general data structures to do your thing.

The best thing I could say about it is that when approaching behind-the-scenes pieces this is my preferred approach, but for customer-facing pieces I prefer (and have had better reception on updates) to update those pieces as a whole. But again, this depends on the complexity, updating a small piece such as a button should not require a large change but updating the layout or styling would require more significant updates.

The way you do that is that you first, piece by piece, split the BFDA into multiple layers, then you make it client-server, split the data-storage into its own server, split the gui of, split the business logic of. Then you rewrite the parts that request stuff for the client into using JSON and rest, then all you have to do is to port the (hopefully) tiny amount of gui interface the user needs to Javascript and the entire app still works.

Is it easy? No. Does it waste some work? Yes.

A few years ago I stopped an impending complete code re-write and a fellow developer was aghast at the very idea of supporting an application that was "only half MVC".

You can replace parts of an application with webservice calls, for instance.

Maybe I can introduce some things like the strangler-pattern in upcoming projects.