IT Performance Improvement
Networking and Telecommunications
Share This Article
Process Improvement Requires a Focus on Productivity
Over my 20 years in the software industry, I've seen process improvement initiatives come and go, as well as countless attempts to find a silver bullet tool that delivers quality with the click of a button (pull of a trigger). Unfortunately, there is no silver bullet. There have been some good ideas in theory, but in practice there have been few real successes.
I think the major problem is that when managers try to improve quality through process improvements, they unintentionally end up piling more work on already time-starved developers. Quality tasks are introduced in such a way that they require developers to adjust their tried-and-true workflow to include yet more work. Not surprisingly, this usually isn't well-received. If developers don't believe that learning and applying this new practice will be worth the effort (e.g., by relieving them from tedious tasks and allowing them to focus on the creative work they enjoy), it will eventually decay. The developers probably won't reject it outright; they will simply do it less and less ... until they stop doing it altogether.
In some sense, what I'm trying to say is that managers forcing quality on developers is like parents forcing vegetables on their kids. Given a choice, most kids won't touch a pile of broccoli sitting on their dinner plate. Try to make them eat it, and you're likely to find it spit out into a napkin later on. But if you blend pureed broccoli into the chocolate brownies they always reach for, the kids will get end up getting the intended nutrients without you ever having to utter "Eat your veggies."
Why Workflow is Essential
I think the problem we have is that we are pushing these process improvement and quality initiatives without really considering how they affect productivity. We take for granted that quality increases productivity, but that's not the case. If you want to introduce a quality initiative into the organization, you need to figure out how to do it in a way that doesn't disrupt or slow down the normal workflow. Otherwise, there's little chance of it resulting in a sustainable quality process.
A good workflow can make or break a group's quality initiative. For example, say the manager requires developers to check coding standards and fix all violations before check in. With most tools, this requires a lot of extra work-each developer has to run the tool, look at a report of problems, sort through false positives, figure out which of the reported violations he is responsible for fixing, and then actually fix them. This extra work slows down the development process and distracts developers from the creative tasks they're hired to do. Eventually, the process will decay, and the team will be back at square one.
The teams that succeed where others fail are the ones that make quality tasks an unobtrusive part of their existing
workflow. Developers just write code as normal and check it into source control. Every night, an automated
infrastructure finds and analyzes the new code. It does this all on its own, without anyone on the team having to think
about it. If it senses that human intelligence is needed, for instance, to fix coding standard violations or review
test failures, the automated infrastructure pushes tasks the appropriate team members, as shown in Figure 1.
Figure 1. Automated process running on integrated infrastructure.
Team members just need to react. When developers arrive to work each morning, they start up their IDE as normal, then they click a button to import any quality tasks (fixing coding standard violations, reviewing test failures, and so on) assigned to them. This way, developers don't need to leave their normal work area to review and perform their assigned tasks. It's much more efficient than working from a report because they can click directly from the task to the related code.
Since developers aren't distracted with unnecessary tasks, they accomplish more and get to focus on the creative tasks they enjoy most. And since the analysis is managed automatically, managers know that code is checked consistently and correctly. It's a win-win situation for both development and management.
For another example, assume that someone in the organization delivers a mandate that "all code must be peer reviewed." If the development manager responsible for implementing the peer code review interprets this mandate as meaning "no developer can check in code before it is reviewed," he is creating a tremendous roadblock in the developers' natural workflow. This is a prime example of a quality initiative that negatively impacts productivity. This process is illustrated in Figure2.
Figure 2. If the development manager responsible for implementing the peer code review interprets this mandate as meaning "no developer can check in code before it is reviewed," he is creating a tremendous roadblock in the developers' natural workflow. This is a prime example of a quality initiative that negatively impacts productivity.
Now, the developer has to do a lot of extra work before adding code to source control. Even with an automated infrastructure managing key tasks, the developer still has to:
- Remember to initiate the process that identifies code which is ready for review.
- Enter a description of the intended changes.
Without an automated infrastructure, he also has to:
- Identify the code that needs to be reviewed.
- Organize it into logical packages.
- Distribute the review packages to the appropriate reviewer.
- Notify the appropriate reviewer.
- Monitor the review status until closure and possibly remind the reviewer of the pending review.
This is tedious and time-consuming work, and decay is inevitable. Developers will probably try to follow this process initially, and project progress will likely slow to a crawl as a result of the additional work and the new roadblock. Once they realize this, they will initiate reviews less and less frequently. Eventually, peer code reviews will be nothing more than a bad memory.
A better approach would be to interpret the mandate as "I don't want to have unreviewed code in any application when it goes to production," which allows code review to occur after check in. This means that code in the source control system is not going to be 100% reviewed all the time. Code will not be reviewed the minute it is checked in... it might take several days. But, this is a small tradeoff for the more productive workflow that it enables.
By removing the roadblock between the developer and his code check in, the mandate becomes much less disruptive to
the development process. Tools exist to set up an automated infrastructure that performs the tasks on a regular
basis; e.g., every 24 hours. Developers don't need to bother with any preparation or notification tasks; they are all
managed automatically. The burden on the developer is significantly reduced because he's not forced to perform
additional work-he just checks in the code as normal. The only added work for the team is the actual
peer review, a creative process that cannot be automated (although it can be greatly optimized with the assistance of
automated code analysis tools). This less disruptive process is illustrated in Figure 3.
Figure 3. With an automated infrastructure that performs the tasks on a regular basis, developers don't need to bother with any preparation or
What's important is that this type of setup actually does not increase the amount of work, but rather reduces it. How? Because the code review actually enables early error detection. This, in turn, reduces the length and difficulty of debugging, which typically consumes tremendous development resources.
The interesting thing here is that the greatest value is achieved by focusing on productivity, not on quality. Focus on quality without considering productivity and you'll likely end up with minimal to no quality improvements, slipped schedules, and a disgruntled team. But focus on productivity by helping the team work smarter and you'll find that the team ends up delivering better software faster.
Industrializing the Process to Foster Creativity
What's the key to making this work? You need to industrialize the development process, relieving humans from any task that does not truly require creativity and human intelligence.
This requires an integrated infrastructure composed of the following elements:
- A requirements management system
- A source control system
- A bug-tracking system
- A staging system you can safely test on
- A production system that runs the live application
- A deployment system that manages deployment to the staging system or production system using a repeatable process
- A monitoring system that provides visibility into the entire process
- A build system that builds the application
This integrated infrastructure is absolutely essential.
On top of this infrastructure sits a backbone process that I call a "nightly run." At the schedule time each night, it automatically…
- Checks out the code.
- Compiles the code.
- Links the code.
- Verifies whether the application is in compliance with the organization's policies (SOA governance, security, SOX, coding policies, etc.).
- Deploys the built application to the staging system.
- Runs the regression test suite to ensure that the team is alerted when modifications impact application behavior.
- Deploys the built application to the deployment system (e.g., a production server or an embedded target) if the specified criteria are satisfied.
- Monitors the entire process and generates reports for process visibility.
This is the basic process that distributes work across the team. This is your production line.
If you think about what tasks in this workflow require human intervention, what do you notice? Nothing repetitive. Nothing that a computer could handle. Only things that truly require human intelligence. This is the key to fostering creativity.
This whole process is what I call Automated Defect Prevention (ADP). ADP is a practical approach to software management through process improvement. This strategy is enabled by an infrastructure that automates repetitive tasks, tracks project status, and provides instant access to the information needed for informed decision making and process improvement. Applying ADP, teams evolve a sustainable quality process that delivers predictable outcomes.
Dr. Adam Kolawa is the co-founder and CEO of Parasoft, a leading provider of Automated Error Prevention (AEP) software solutions. Kolawa's years of experience with various software development processes has resulted in his unique insight into the high-tech industry and the uncanny ability to successfully identify technology trends. As a result, he has orchestrated the development of several successful commercial software products to meet growing industry needs to improve software quality - often before the trends have been widely accepted. Co-author of Bulletproofing Web Applications and Automated Defect Prevention: Best Practices in Software Management, he has contributed to and written over 100 commentary pieces and technical articles for publications such as The Wall Street Journal, CIO, Computerworld, Dr. Dobb's Journal, and IEEE Computer, as well as numerous scientific papers on physics and parallel processing. His recent media engagements include CNN, CNBC, BBC, and NPR. Kolawa holds a Ph.D. in theoretical physics from the California Institute of Technology, and has been granted ten patents for his recent inventions. In 2001, Kolawa was awarded the Los Angeles Ernst & Young's Entrepreneur of the Year Award in the software category.