Refactoring and performance II

A common concern with refactoring is the effect it has on the performance of a program. To make the software easier to understand, I often make changes that will cause the program to run slower. This is an important issue. I don't belong to the school of thought that ignores performance in favor of design purity or in hopes of faster hardware. Software has been rejected for beign too slow, and faster machines merely move the goalposts. Refactoring can certainly make software go more slowly — but it also make the software more amenable to performance tuning. The secret to fast software, in all but hard-real time contexts, is to write tunable software first and then tune it for sufficient speed.

I've seen three general approaches to writing fast software. The most serious of these is time budgeting, often used in hard-real time systems. As you decompose the design, you give each component a budget for resources — time and footprint. That component must not exceed its budget, although a mechanism for exchanging budgeted resources is allowed. Time budgeting focuses attention on hard performance times. It is essential for systems, such as heart pacemakers, in which late data is always bad data. This technique is inappropriate for other kinds of systems, such as the corporate information systems with which I usually work.

The second approach is the constant attention approach. Here, every programmer, all the time, does whatever she can to keep performance high. This is a common approach that is intuitively attractive — but it does not work very well. Changes that improve performance usually make the program harder to work with. This slows development. This would be a cost worth paying if the resulting software were quicker — but usually it is not. The performance improvements are spread all around the program; each improvement is made with a narrow perspective of the program's behavior, and often with a misunderstanding of how a compiler, runtime, and hardware behaves.

Even if you know exactly what is going on in your system, measure performance, don't speculate. You'll learn something, and nine times out of ten, it won't be that you were right!

Ron Jeffries

The interesting thing about performance is that in most programs, most of their time is spent in a small fraction of the code. If I optimize all the code equally, I'll end up with 90 percent of my work wasted because it's optimizing code that isn't run much. The time spent making the program fast — the time lost because of lack of clarity — is all wasted time.

The third approach to performance improvement takes advantage of this 90-percent statistic. In this approach, I build my program in a well-factored manner without paying attention to performance until I begin a deliberate performance optimization exercise. During this performance optimization, I follow a specific process to tune the program.

I begin by running the program under a profiler that monitors the program and tells me where it is consuming time and space. This way I can find that small part of the program where the performance hot spots lie. I then focus on those performance hot spots using the same optimizations I would use in the constant-attention approach. But since I'm focusing my attention on a hot spot, I'm getting much more effect with less work. Even so, I remain cautious. As in refactoring, I make the changes in small steps. After each step I compile, test, and rerun the profiler. If I haven't improved performance, I back out the change. I continue the process of finding and removing hot spots until I get the performance that satisfies my users.

Having a well-factored program helps with this style of optimization in two ways. First, it gives me time to spend on performance tuning. With well-factored code, I can add functionality more quickly. This gives me more time to focus on performance. (Profiling ensures I spend that time on the right place). Second, with a well-factored program I have finer granularity for my performance analisys. My profiler leads me to smaller parts of the code, which are easier to tune. With clearer code, I have a better understanding of my options and of what kind of tuning will work.

I've found that refactoring helps me write fast software. It slows the software in the short term while I'm refactoring, but makes it easier to tune during optimization. I end up well ahead.

Martin Fowler, "Principles in Refactoring", in Refactoring: Improving the Design of Existing Code (2nd Edition), 64-67.