My approach and a ‘rockstar’ developer’s approach
I recently wrote a blog piece discussing research that showed that the phenomenon of the 10x[1] developers is a real one. In the research, subjects were asked to develop a program to encode phone numbers as words. When they had completed the task, they had to submit it to the researchers along with the total time they had spent on the task. The requirement specification and example input and output files are publicly available here. The research aimed to control for developer’s experience in that all the participants were graduate Computer Science students.
Even with a homogenous pool of developers, there was a surprisingly large variance in the time taken to complete the task. The C/C++ developers took between 3 to 25 hours and the Java developers took between 4 and 63 hours. Further, the researchers found that neither correctness (measured with an automated acceptance test) nor runtime performance was correlated with development time.
I thought it would be fun to try to implement the program in Clojure, my favourite programming language. The task took me 2 hours 45mins[2] and I wrote 125 NCLOC[3], which included 30 NCLOC of unit tests. It turns out that Lisp wizard and total AI god, Dr Peter Norvig has also implemented the program[4]. His solution was 45 NCLOC and took him 2 hours. Both our programs passed the automated acceptance test and ran efficiently.
Differences in programming style
There is a significant difference in styles between my and Norvig’s programs that reflects both the eras in which the programs were written and the different programming languages. His implementation contains global variables, mutation, imperative code and IO in the middle of the inner loop. It also has no unit tests. His algorithm though is beautiful in its simplicity. My code does include unit tests and avoids global variables and mutations. The style is functional and all IO has been pushed to the outer edges of the program.
As well as style, there is also a difference in the design of our algorithms. To write reliable programs, I believe it is critical to eliminate defects that are serious, but only occur rarely. It is these types of issues that can escape automated and manual testing and only become apparent once a program is in production. One class of issues comes from concurrency and I talk about them in this blog piece. Another comes with algorithms that normally exhibit O(n log n) or better performance but can deteriorate to quadratic or worse. A classic example is the quicksort algorithm whose worse case is quadratic. Simple implementations of quicksort show this worse case behaviour if they sort already sorted data. Far better to my mind is to use introsort or heapsort which are O(n log n) for all inputs.
In the phone number task, I was concerned that pathological worst cases were also possible. For example, the ‘phone number’
20957420957420957420957420957420957420957420957412
is a valid input as it has no more than 50 digits. Norvig’s implementation uses a brute force search and requires at least 218 = 37,822,859,361 possible encodings to be checked[5]. On my computer this took over two hours! If a carefully crafted dictionary is used, then there are some phone numbers for which this brute force search is completely infeasible.
To guard against this type of issue I used dynamic programming. As a result, my program determines there are no encodings for the number above in around 90 milliseconds. Has Norvig not heard of dynamic programming? Of course he has! He just chose an approach that allows for a simpler program, at the risk of pathological behaviour for some edge cases.
Final thoughts
Are we both 10x developers?
I believe that software development magnifies small difference in abilities, sometimes resulting in very large differences in outcomes. However, the concept of a 10x developer is far too simplistic. Even if two developers write programs that pass automated acceptance tests and have similar run-time performance, it does not mean the programs are fungible.
Differences in coding style and handling edge cases affect maintainability and reliability; both are key requirements of well written software.
My code is available here.
Picture of phone book courtesy of Adam Lehman. Used under Creative Commons CC BY 2.0 licence and available from here.
[1] “A 10x developer is an individual who is thought to be as productive as 10 others in his or her field.” From Techopedia.
[2] I looked at the task the day before so I benefited from some unmeasured thinking time; however, more than 2/3 of the participants in the original study had an overnight break or longer which was not included in their times. [3] Non-comment line of code. [4] I didn’t look at Norvig’s solution until I had completed my own! [5] This is because 209574 has 21 possible encoding and 12 has no encodings.
Comentários