Lines Of Code - Dispelling The Myths
You see stuff related to Lines Of Code (LOC) everywhere on programming blogs. People discussing it as a metric of project size, or of programmer productivity, often drawing conclusions in absolutist tones such as “fewer lines of code means less bugs”. I’m writing today to dispute the idea that anything related to LOC is absolute. And I’m going to do it one myth at a time.
LOC As A Metric of Accomplishment or Project Size
This one is old, and most experienced programmers now officially reject it outright. But still, I feel it worth repeating: The number of LOC you write in a single day is worthless as a metric of what you accomplished that day. After all- If someone wrote a 1000 line class in a single day, and you wrote the same class in 500 lines (and maybe tossed in a little extra functionality while you were at it), who was more productive?
As a metric of project size, it makes a little more sense, but just barely- The Wikipedia article on LOC mentions that “Many useful comparisons involve only the order of magnitude of lines of code in a project.” That at least makes a little sense. One can reasonably assume that the same set of programmers, using the same language, will be able to accomplish more with 10000 lines of code than with 1000.
Ambiguity: Too Many Ways To Count
The next issue is this: How do you COUNT lines of code? That same Wikipedia article linked above has an interesting example:
for (i=0; i < 100; ++i) printf("hello"); /* How many lines of code is this? */
How many lines is that? One for the line? Two for logical seperation of forloop and print statement? What if you wrap the line in curly braces, one on each line, and move the comment to it’s own line, so it looks more like
for (i=0; i < 100; ++i) { /* How many lines of code is this? */ printf("hello"); }
Did you see that? I just took one line of code, and turned it into 5! Things get so much more confusing than stylistic issues like this, though. What about libraries? What about frameworks? Jeff Atwood wrote a post a couple of months ago wherein he mentioned that the original version of Basecamp had been written in only 600 lines of code. Given that Basecamp was written using Ruby on Rails, which handles all sorts of crazy things for you, I think the more honest metric would have to be the 600 lines of basecamp, plus however large the RoR framework is, plus the code RoR generated for them when they started the project. Money says it’s more than 600 lines.
I’ll give you another example. Currently, the amount of code I’ve written for Migratr is 5000 lines and change. However, Migratr uses FlickrNet, a popular .NET library for interacting with the Flickr API. FlickrNet, if you download the source, is more than 12,371 lines of code (according to the unix util “wc”). I’ve also incorporated a few other libraries, all of which are linked in Migratr’s “about” page. Why? Because I didn’t want to write those 12,000 lines. Someone already had, and they did a fantastic job of it, and I didn’t see any point in re-inventing the wheel. So the total code involved in Migratr is, at this point, more than 17,000 lines. I’ve written 5,000 of them, and I’m the sole developer of Migratr. But I wouldn’t dream of saying “I built this application that interacts with a whole slew of online photo services and migrates all your photos and does all this neat stuff, and it was only 5,000 lines!” No. MY contribution was 5000 lines, but Migratr depends on a lot more than that.
The point being, where do you draw the line? Is Migratr a 5000+ LOC project? or 17,000+? Do you only count LOC if you have access to the source of the library you’re using? Do you only count the code written for your specific project? Could I export Migratr’s backend as a library and let someone write
public void migrate(String filepath) { ImportFromSource(Services.Flickr, filepath); exportToDestination(Services.SmugMug, filepath); }
Did you see that? My whole project was just replaced! Some twerp in a garage somewhere just re-wrote Migratr in 5 lines of code! It doesn’t matter that it depended on my 5,000 line library, which depends on (among others) 12,000 lines of the Flickrnet library. You only saw 5 lines. It was only 5 lines.
But clearly, since 5,000 is more than 5, I can at least say that I was more productive.
The relationship between line count and bug count only goes so far
The idea that “Less lines of code mean less bugs” is some sort of universal absolute is the biggest of all the LOC myths. In a desperate effort to nullify your “oh snap, time for a flamewar” reflex, I’m going to add a couple of disclaimers before I get into this. Any reference I make to a programming language is not an attack on that programming language, just an attack on the abuse of that language. Also, I realize that reducing the amount of code required for a specific task can help reduce the bug count, but only so far. I’m saying there’s a limit to this. Here’s the line I want to draw for you.
Reducing the code involved in a task will only help reduce the bug count IF THE PROCESS MAKES THE CODE MORE READABLE. Past that, you’re probably creating lines of code that do more than one thing per line, and thus increasing the number of bugs possible per line of code.
Perl programmers are especially frightening in this regard. I really love PERL and use it instead of shell scripting languages such as BASH whenever possible, but there’s something in the PERL culture that demands it’s programmers to try and do something in the fewest number of characters possible. There’s even a name for this as a sport- It’s referred to as “Perl Golf”.
Really quick, can you tell me what this is?
-pl s!.!y$IVCXL426(-:$XLMCDIVX$dfor$$_.=5x$&*8%29628;$$$_=$_!egfor-4e3..y/iul-}/-$+ /%s''$';*_=eval
I’ll give you a hint- It’s not a cat walking across my keyboard. If that was your first guess, though, we’re at least on the same page. This was the winning entry in a game of Perl Golf: The problem was to write a Roman Numeral Calculator in the fewest characters possible. Submissions can be found here. Now, this is an amazing piece of code. The person who wrote it is clearly very skilled. The fact that Perl makes code like that possible is pretty impressive. The problem here is that people write code like this outside of PERL golf, and think to themselves, “it’s only one line, so it must have fewer bugs than a 50 line solution”. Really? Because if I had to debug one, I’d have definitely gone with the 50 line solution. And don’t tell me that that one line of code worked on the first, second, or 5th try. The 50+ line solutions might have, but not that catwalk.
Writing unmaintanable code does not reduce the bug count for a project. A friend of mine coined a term for code like that- “Write-Only Code”- a takeoff on the permissions you can set with chmod. Think about it: Readable, writeable, and executable. Shouldn’t code be all three?
Another quick example:
(0/:l)(_+_)
The first time I saw this snippet, I thought someone was, via emoticons, trying to re-enact the facial expression one wore when seeing “Goatse” for the first time. It’s not actually a story told by emoticon. It’s a piece of scala code that sums the elements in a list. And I just don’t see it as having been easier to write than a forloop. It’s definitely not as maintainable. This piece of code, actually, prompted a conversation with a friend about holding a “code or emoticon” contest, but we really couldn’t come up with any serious contenders to this one. Maybe some Perl Golfers could throw their hat in the ring?
I want to state too, that for this section, I’m well aware that I didn’t provide buggy one-liners. I provided working one-liners. This is because I was trying to emphasize, the point isn’t if it works or not- The point is if you can tell by looking at it what it should do. If you can’t, it’s unmaintainable. If it’s unmaintanable, then even if it works the first time, it’ll produce a slew of bugs as soon as anyone tries to modify it.
So, in essence- Stop paying attention to Lines of Code. They’re impossible to count in any useful way. They’re a crappy metric of project size or personal accomplishment. And they don’t control how many bugs your code has. You do.
-Alex
If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.
Comments
You have the phenomenal ability to state the obvious. I mean, hey, it’s not like this conclusions were drawn already some 20 years ago …
The otherwise impeccable spelling of this post only made the Peices more jarring. :(
Other than that, I agree :)
–feep
Your premise is a good start, but you haven’t proposed an alternative. The classic rebuttal to this argument (it’s certainly not a new one) is to borrow from Churchill:
” is the worst form of , except for all those other forms that have been tried from time to time.”
Your conclusion is to throw out / ignore LOC. This is difficult to replace without some other metric.
I agree with you, but I would still like to see some analysis of an alternative to replace LOC.
Oh, huh. Insert:
* lines of code
* methods of measuring software development productivity
respectively.
@Rhab: I’m aware that most of what I’m saying isn’t new. But until the madness ends, I feel it still needs repeating- especially the “fewer lines doesn’t mean fewer bugs” part, which far too many language enthusiasts adhere to as a defense for writing really awful-looking code as proof that their language is superior to all.
@James: Awesome- Thanks!
@FeepingCreature: Thanks, fixed:D
@GWC: Yeah- Part of the problem is that programmers naturally want some sort of objective metric, and even with the stated ambiguities, if they’re handled consistently then LOC can provide that metric. And yeah, that Churchill quote was floating through my head while I was writing this post (I love quoting Churchill).
But there are shifts that can be made today. If you HAVE to quote LOC, for instance, toss in a disclaimer on what that count DOESN’T include. And that “line count vs bug count” thing, out of all of them, is the real soapbox of the entry. People don’t have to squish everything onto one line. It doesn’t make the code more stable. Things just don’t have to be this way:D
“The idea that “Less lines of code mean less bugs” is some sort of universal absolute is the biggest of all the LOC myths”
Who exactly is arguing that? This sounds like a classic strawman. I’ve heard plenty of people argue that in general, reducing the number of lines of code reduces the complexity/# of bugs. But I don’t think I’ve ever heard anyone argue it in absolute terms (ie, a 746 line class always has fewer bugs than a 784 line class).
Alex,
Your arguments are specious. I have used LOC myself to manage projects quite successfully, and my products have been notable for delivering on time, on budget, and with great quality, and these products are large applications: trader workstation and ISP Network management software to name two, both totalling 1 Million+ LOC each. I attribute much of that to the fidelity that lines of code gives me in understanding the progress of development and the scope of the work.
And yes there most definitely is a correlation to more lines of code written and more bugs in the software, so much so that I’m able to predict with high accuracy the number of defects needed to find and fix in our releases to achieve high quality objectives. The fact is that the more code written, the more bugs in the software. 10 LOC in the same language will always have less bugs found in test than 1000 LOC written.
One can create all kinds of fictitious examples that appear to make LOC a silly metric, but in practice, those scenarios never materialize or matter much.
Have you ever attempted to use LOC as a tool for managing your development? If not, I suggest you give it a try. You might like it. Remember the perfect is the enemy of the good. The metric doesn’t have to be perfect to provide great value. LOC isn’t perfect, but it’s the best metric we have for measuring production. I’m doing a whole series on metrics on my blog, and I will eventually cover LOC. Please visit and share your comments.
Bill-
(Shorthand: 10 line solution is X, 1000 line solution is Y)
Does X accomplish the same thing as Y? How well documented are each? Does X depend on library Z to come out 990 lines shorter? If so, how many LOC is Z? Is Z Open-Source? Well documented?
You mentioned fictitious examples in your response. I gotta say, a 10-liner vs a 1000-liner that accomplish the same thing (and let’s say as a parameter, use the same libraries) is pretty far-reaching, unless that 1000-liner is incredibly poorly written, or the 10-liner is a game of Golf (as mentioned above). I didn’t say there’s zero relationship between line count and bug count. I said that it only goes up to a point. Past a floor value for line count, you’re essentially just obfuscating the code - That’s what the Perl Golf sample was meant to illustrate. “printf(”Hello World”)” is obviously going to be less bug-prone than “print(”H”);print(”e”);…print(”d\n”);”, but that’s because it’s not superfluous.
However, I don’t claim to be the final authority on anything (hell, I had to look up ’specious’ on dictionary.com). I’m always happy to be proven wrong as long as I learn something from it, so I’ll definitely be giving your series a read:)
Alex,
I liked your last paragraph. You have a very amiable nature.
This can be a large subject, so I’ll try to be brief. X and Y are 2 different people writing a “C” program for 2 different problems and all the code is code they write themselves: no libraries or borrowing code or anything like that. Let’s also assume they both choose to solve the problem in a way that a reasonable person would agree is efficient and appropriate.
Given that, can we agree that x is likely to take less time and have less bugs than y? If we can agree that this correlation exists between writing more code and taking more time and having more bugs, then you can use that relationship to estimate better and track progress better. Now there’s more to it than that, but that’s the basic relationship that makes LOC useful to managing software projects. And there is no other software metric that I’m aware of that has the comparable fidelity to LOC for this purpose.
BTW, when people use LOC for managing software, they use SLOC which removes comments and white space from the count.
Alex –
Why are you so adamant on including libraries in the calculations? After all, any application will use many libraries to accomplish even basic tasks. Just because we don’t think of device drivers and OS functions as libraries doesn’t make them any less so. And since you’re certainly not advocating that each Internet application poorly re-implement a TCP/IP stack (library) or low-level data structures (java.lang.String), why draw the line higher up the stack? If a library is well-tested in the field, why shouldn’t programmers use it?
Also, if you are managing programmers, how do you estimate the density of defects in your code, so that you have an idea of when all the defects have been located and resolved? LOC may not be an ideal metric, but two programs in the same language by programmers of similar caliber will have similar defect densities. (An analogy: we all make typos with some frequency. Ergo, more letters typed equals more typos.) Also, what is your objection to running code through a code formatter a la Checkstyle to perform LOC metrics?
Your obfuscation vs. 5-line Java example is also specious, and obscures very real differences between languages. I am sure you would agree that a skilled Java programmer could write a Web content management application in fewer lines of code than a skilled assembly programer, in their respective languages. Similar differences exist between C++/Java and some of the dynamic languages (Ruby, Python, etc.) Pretending that the productivity enhancement is composed of obfuscation is probably not something that you want to be posting in public, if you care about accuracy.
The density of defects in codes cannot be measured by LOC.
Defects are “non-deterministic” in nature. Its not something
that you can sample because it doesnt have predictable,
reliable patterns that you can expect from every single
programmer you manage.
For example, Unless you know for a fact that Average
programmer in the US, age between 25-30 makes n% of defects
per million line of codes, this kind of defects measurement is
absurd. You can only define the level of defects only if you
know for certain what is the measuring stick.
Fewer LOC is a good objective that every manager should drive
their team to achieve. Having less code to manage is always a
good thing. But it is superficial to equate LOC to quality.
Increase in LOC is always a result of the increase in the
number of coressponding process, or functions or modules that
the programmer have to write. Sometimes things like defensive programming and reflective programming that wasnt in the official use case scenarios adds more LOC into the software but the software gains tremendous quality gains.
LOC metrics makes sense only if it is analyzed with the number
of process the programmer had to translate into codes.
Better algorithm does not equal less codes.
Does the manager factor in that fact when considering overall
quality of the software?.
Trying to put an “ideal” quantity to a dependant variable Y
(quality / defect) when it is a product of independant
variables Y=ax + bx + Nn which you cant quantify only gives
false sense of quality. How would one know how many LOC is ideal just by looking at the number of SLOC?
One line of text does not equal one line of code or program statement. It is not a measure of “thinking unit” of a programmer. Judging the competency of your programmers using LOC is ill informed and misguided.
@Bill-
Yes, under those circumstances X will probably have fewer bugs than Y. However, that scenario involves the X solving a different problem than Y, which allows X to be as elegant and readable a solution as Y, and kind of skirts my point: That past a certain point, reducing LOC can only be achieved by writing code that does more than one thing per line, at which point you’re keeping the bug count, just upping the number of bugs per line. If Y and X were different implementations of functionally equivalent code (let’s say same function signatures, hypothetically equivalent from a black-box-testing perspective), it wouldn’t be possible for X and Y to both be reasonable solutions. If X was the reasonable solution, Y would have to be written by someone new to the language, or just preferred writing incredibly bloated code. If Y was the reasonable solution, X would have to be some of the most horrendous, unmaintanable C agony I’ve ever encountered.
RG- There’s an implied quality difference between packaged standard libraries for a language (C# depends on the .NET framework, for instance) and a third party library that you lifted off a website because it was GPL and did everything you wanted to do yourself. “Well-Tested in the field” helps keep the bug count in check, but I’ve yet to see a popular library under active development with a bug forum/list that’s been empty for any real period of time.
I don’t have anything against running code through a formatter- It’s the context vacuum that bothers me. 5 lines could mean two lines of “meat” and three lines of organization (brackets, whitespace, comments, whathaveyou). Personally I have nothing against counting whitespace and documentation- If you work somewhere that’s judging you based on LOC, you should still get points for making your code more readable. It’s just that reporting a number like “50 lines of code” without specifying how it’s counted doesn’t tell us anything about the code quality or complexity. Numbers without context are how that famous quote originated- “There are three kinds of lies. Lies, Damned lies, and statistics.”
“a skilled Java programmer could write a Web content management application in fewer lines of code than a skilled assembly programer, in their respective languages” - Definitely. It would, indeed, be ridiculous to dispute that. Also it definitely remains true throughought the higher level languages: Ruby backends will have fewer LOC than C backends, etc. But that’s just another way that LOC can mean less in a vacuum of context. Saying “I wrote this in 600 lines of code” tells us pretty much nothing of the code quality, bug count, or programmer’s skill without knowing at LEAST what language they used, how much of the code was already done for them (again, not counting libraries bundled with the language distribution- those simply establish a benchmark) by third party libraries/frameworks, and maybe actually looking at the code to see whether it was written in an intelligent manner or a clever(e.g Perl Golf) one.
@Alex –
Out of curiosity, would you consider the libraries that come with a GPL/BSD J2EE stack to be in the quality bracket with .NET, or with a random GPL library? What about the libraries that come with the Apache httpd server? And are you really using C# and .NET as examples of robust code, where the bug forums are subject to public inspection?
In either case, if you are evaluating code that I wrote for a project, why on earth would you care how many LOC are in the libraries I use? They are not part of the code you are evaluating, much like the OS, app server stack, etc. Like you said, we need to know context. If I am writing an application for Windows, we know the context is that the OS will slow everything down and make it crash sometimes for no fault of mine. In other words, we know the libraries I use all the way down to the OS level are buggy. But you don’t count that against me. Why not think the same way about OSS libraries? Some are buggy, some aren’t. But here’s another point for you to evaluate: choice of libraries. Did I make a good choice or not?
Also, you may want to familiarize yourself with logical SLOC: http://en.wikipedia.org/wiki/Source_lines_of_code. I think this concept will make you feel a lot better about LOC as a measurement. Note that you can measure logical LOC without obfuscating code. Specifying a method at 50 logical SLOC does tell us something useful about the code vs a method with the same goals that is 500 SLOC written by the same person.
Your arguments against LOC are similar to the arguments against using weight as a partial measure of health. No, one number will not tell you everything you need to know about the situation. But that doesn’t mean you stop measuring it, or ignore the numbers you have. Doctors are professionals who have to measure complex systems, and they work with the metrics that they can gather. Why is this method not appropriate for software developers?
And again, what are you proposing instead?
Alex,
There’s a misunderstanding of the way people who advocate LOC as a metric actually use it. Until that misunderstanding is corrected, this discussion will remain at an impasse.
@RG — Can you contact me via email? I posted my contact info with my comment.
I have a project that I’ve been working on with one other guy for about three months. We had a discussion about how many lines of code we’d produced. He guessed about 3,000. I guessed about 300,000. wc said more than 700,000. So what’s going on?
In this project we are automatically parsing the schema of a large legacy database into a custom XML format, and then using XSLT to generate model and controller classes and Velocity macros for views to create a classic MVC application (in C# and Monorail, but actually that makes little difference). So very few of those lines of code are actually hand-written.
And that provides an interesting sidelight into the lines of code debate in more than one way. What are the ’source lines of code’? The roughly 3,500 lines of sed script and XSL transforms which I actually wrote, the roughly 66,000 lines of the (automatically derived) database schema, or the 700,000 plus lines of C#, Velocity, NHibernate and other stuff which they generate?
If I hadn’t written the auto-generators, that code would still have to have been written (we might have been motivated to write it more tightly, but it would still have been in hundreds of thousands of SLOC). So the number of lines of code you actually write is not a good measure of project size, and the number of lines of code produced is not, either.
And remember, this is happening all down the software stack. I use XSL-T to generate Java, C# or some other high level language. The compiler takes that high level language and in an analogous process generates what in BCPL we used to call ‘CINT Code’ - machine instructions for a virtual machine. The run time system takes the CINT code and generates machine instructions. And inside the processor, the microcode interpreter takes those machine instructions and uses them to generate actual machine operations.
At each stage in the process, the number of discrete statements increases. Which should you count? Why? Even at the level of a binary, a larger binary does not necessarily deliver more functionality than a smaller one.
The other thing is, where a mechanical process is generating code, fixing one bug in the generator will fix hundreds or even thousands of bugs in the generated code. So my hunch is that while functionality scales more-or-less with the number of generated lines of code, bugs scale more or less with the number of lines of code in the generator.
[…] I found a nice article (actually the article found me…) written by Alexander Lucas titled Lines Of Code - Dispelling The Myths about a common project management error, which makes estimations and worse makes the evaluation of […]
LOC of code is always about estimation. An estimation (the name is “estimation”, not “final effort” or something like that) can be Order of Magnitude, Finite, Rough, Wet Finger type. And also it depends on which stage of the project you are in and what expertise you have.
At the same time LOC is never used for estimation in a standalone way. It is via Baselining Techniques or COCOMO model.
For LOC, SEI has specified which will be considered to be part of counting and which can not be. And again, your organization may have some different way of calculation also.
You can have a look at:
http://art-of-project-management.blogspot.com/2008/03/estimation-technique-lines-of-code.html
However, havin said all these, bottom line is what kind of engineers you have, what kind of expertise you have and above all the committment to work. Nothing can beat that.
And may be that is reason why so many IT project (read project management) which are not in good shape even with proliferation of so many tools/techniques.
Another reason for ambiguity is the different tools that are available to count your lines of code. I tried out 2 different ones (both of wich had disclaimers to the effect that LOC was not a good metric) and they gave me 2 different answers. One was about 300,000 and the other told me that I had about 500,000 (this is including frameworks of course). One counted lines that the other did not obviously, but which lines?