The business case for speed
Image credit: lincolnblues
I talk about speed in two senses in this article: (1) the pace of software development, and (2) the speed of software as experienced. I do that, because in modern intelligent systems, quite a bit of data science goes into the end product, and the speed of the science tools determines the pace of development.
It's a truism that time is money. Under the America Invents Act, the U.S. has changed to a "first-to-file" system, so the first organization to file for a patent wins a valuable monopoly on a technology. In social media and P2P commerce (eBay, Etsy,) success involves the creation of a community, the first mover can create a two sided market giving them a persistent advantage. Even without a deep moat, a company that consistently improves products faster than competitors can move ahead.
Speed is often a "non-functional requirement" that doesn't get a user story and isn't on a bulleted list of features that enterprise buyers think they need. Often organizations find out that they product they've bought or are developing is unacceptably slow only after they've spent a lot of money on it. Rollouts of content management systems, issue tracking systems and other tools frequently fail when users when users abandon unacceptable slow and awkward products.
The success of GitHub has been driven by a brand image that revolves around speed -- unlike the average off-brand issue tracker, you don't have time to go out to coffee while creating a new trouble ticket.
Low latency enables new applications
Image credit: http://www.flickr.com/photos/sergesegal/
The kind of speed I'm talking about is not high throughput, but low latency. The first one is prized by the corporate mindset that believes nine women can have a baby in one month, but the latter is something customers can feel. (When you take care of latency, throughput often takes care of itself.)
Tablet computers feel like tablet computers not just because they have a touchscreen but because they lack a spinning hard drive. If your operating system regularly hung up for ten seconds for I/O, the feeling that you can manipulate things with your fingers wouldn't be there.
Early adopters of the Oculus Rift have discovered what academic researchers discovered 20 years ago: Virtual Reality applications require extreme low latency, even latency less than can be consciously perceived in head tracking can have side effects such as nausea and vomiting.
Fast is the opposite of slow
Image Credit: Wendy Seltzer
Low latency is hard to bolt onto a product or a process that already exists, simply because fast is the opposite of slow. Unlike racing video games, auto racing is an unforgiving sport, because time lost by early mistakes can't be gotten back by faster driving later. To set the record for driving across the US you have to maintain an average speed of roughly 90 mph, and that's impossible if you spend much time at all stuck in traffic or stopped by the cops; it's just not possible to catch up by driving at 180 mph.
There's a common canard in computer science, attributed to patron saint Don Knuth, that "Premature Optimization is the root of all evil." It comes down to the belief that any system has one component, one subsystem, one inner loop, where it spends all of it's time. This is a half truth, because once you pick the low hanging fruits of performance improvements for a complex system, you often have several "bottlenecks" that make similar contributions. Real life systems also live in different environments, so the bottleneck is different when your data set is small enough to fit in memory and when it isn't, or if you are using an SSD or a hard drive for storage.
In any process, it's necessary to identify and eliminate all of the bottlenecks. For instance, the most neglected method of schedule compression is to start working on a project earlier. It's a common lament that "Silicon valley would rather work on projects like Instagram than Hyperloop", yet, the real difference is that you have to get permission first if you want to build a Hyperloop or a new kind of nuclear reactor -- and that adds risk and time that investors can't deal with. (If phone companies had their way, it would be the same for mobile apps, but that's another article.)
Getting "shit" done
Image credit: The Knowles Gallery
The dominant paradigm of software management in 2013 is to "get shit done." There's something to say for being focused, pumped, and productive, but this is often an excuse to neglect planning, project management, what Steven Covery calls sharpening the saw, health and sanity. This paradigm appears both in startups and established organizations.
A key malpractice here is setting arbitrary schedules based on perceived business needs or the belief that setting a short schedule will create a "timebox", force developers to stop lollygagging, and focus efforts on a minimal viable product rather than a 'Cadillac' product with unneeded and unused features.
In some organizations, developers are asked to provide estimates, but when management hears them, they say that the estimate are "too much", and force estimates to be revised down.
When a project starts out on an impossible schedule, it's impossible to real project management because you can't put together a story that makes sense. Without project management, you can't make good decisions about what to do, and you waste time, a lot of time, with expensive mistake.
The usual effect of unrealistic deadlines is that projects fail, take longer than necessary, and produce "shit" that disappoints customers, is expensive to maintain and improve, and put you at a disadvantage relative to faster competitors.
The path less taken
Funny enough, if you focus on controlling risk, and think about the variance of the schedule, the mean of the schedule takes care of itself.
The title of Steve McConnell's book, Rapid Development is just the tiniest bit deceptive. In most of the 647 pages, he talks about efficient software development -- which is mainly a matter of being realistic and avoiding mistakes. Organizations that have mastered efficient development can attain rapid development by adding manpower to attain 10-30% schedule compression at progressively increasing expense.
A unified theory of business processes
Diagram by Karn G. Bulsuk
Williams Edward Deming introduced the the plan-do-check-act (PDCA) cycle, a universal model of business process and even the experience that users have with software. The core idea is that a business process doesn't just produce a product or service, it also produces knowledge that can be fed back to make the process faster, better, and cheaper.
The PDCA cycle corresponds, for instance, to a "sprint" or "milestone" in an agile development process.
Assuming a business goal requires multiple cycles, there are two ways to get to the goal faster: (i) complete the work in fewer cycles and (ii) go around the cycles faster. (i) is more about strategy and (ii) is about tactics.
The number of cycles can be decreased by wise planning, learning effectively, and learning from other people's mistakes. (Much software malpractice revolves around "reinventing the wheel" when it comes to deceptively difficult tasks.) Learning from other's mistakes is a matter of hiring experienced people, paying attention to the work of other people in the field, and often book learning. Learning from your own mistakes is a matter of having an open culture where people can be honest with each other and themselves and being smart about when it is right to "play through the pain" and when it's right to stop the production line when things are going wrong.
Speeding Up the Cycle
The 1993 book Fast Cycle Time: How to Align Purpose, Strategy and Structure for Speed is still one of the best on the topic of speeding up development cycles.
The core idea is right in the title: to speed up your cycles, you need to address bottlenecks throughout the process and throughout the organization. A hang-up anywhere causes lost time that can't be gotten back. Everything needs to be retooled for speed, from the management process to the development process to the the tools software developers use. Developers just don't have the time to sit through 40 minute builds, use dual core laptops, or use off-brand project management and documentation tools that are slow to load.
Speed as a competitive advantage
Just as the perceived speed of software depends on the architecture and implementation, the speed of product development depends on every aspect of the development process. Although speed is most often pursued through wishful thinking, organizations can get a competitive edge by bucking the trend, being deliberate, and organizing their process around speed.