Congrats to Cybernance and Wellsmith!

Calavista's clients make Built In Austin's 50 Startups to Watch in 2018.

Congrats to Cybernance and Wellsmith for being selected as Austin Built 50 Startups to Watch in 2018! Calavista is proud to be the chosen development team for these two awesome startups solving critical problems in diverse industries. Cybernance's SaaS platform creates a shared visualization of cyber risks for boards, executives, security staff, and all key stakeholders. Wellsmith is revolutionizing the way people manage chronic disease.

At Calavista, we pride ourselves on our passion and high curiosity quotient - key elements to solving complex problems with custom software development. For a startup, working with the Calavista team frees up the CEO to focus intensely on building the business and the culture instead of worrying about whether the product will be delivered on time - or if it will even work.

Our vast experience, Hyper-Agile® approach, and ability to scale a team from zero to dozens in weeks instead of months are the foundations to success when needing to bring an MVP from idea to market in months instead of years. We boast a success rate of over 90% - delivering software that works, on time and on budget. Because of this success rate, investors can have confidence that they will see their product launch successfully in the market, and achieve a rapid return on their investment.

Check out the Cybernance case study.

Check out the Wellsmith case study and podcast.

We are proud to be a part of the Cybernance and Wellsmith teams and look forward to seeing them continue to change their market landscapes in 2018!


How Misconfiguration Can Leave You Vulnerable to Attackers

When deploying an application, make sure to use the correct set of configuration settings in your deployment scripts.

Configuration is an essential part of every application. Misconfiguration can happen at any level of the application stack – from code, to web and application servers, to databases and frameworks. Below, I’ve compiled a list of some of the most common scenarios.

Deployment of development configuration to production

Development configuration can include tracing, unencrypted string connections, test accounts with a weak password, descriptive error messaging, and more. A malicious attacker will be able to use tracing or error messaging to gain access to insecure accounts, compromising the application. When deploying an application, make sure to use the correct set of configuration settings in your deployment scripts.

Failure to secure directories

Protected or private directories are directories that are available only to confirmed application users, admins, or to the application’s code. Protected directories might include sensitive information in the form of files and images, or an account control panel.

Third-party applications installed on a production server

A production server that has additional applications installed on it often poses a security risk. Some applications have their own vulnerabilities and known exploits. For example, some applications might need to use a port in the firewall that otherwise might be blocked. Rather than trying to attack your application directly, a clever attacker may fish for known holes in other applications which they guess might reside on your server. You can do everything right, but your server is only as secure as your least secure application.

Web serving source files

A web server that is not configured to run a technology in a desired endpoint might serve the file back to the client instead of executing it. This can include compiled class files, PHP code and more. Once a hacker has access to your source code, they will be able to access any aspect of your application stack.

You should always modify every default account on all applications and servers.

Directory listings enabled

With directory listing enabled, an attacker can view all of the files on your web application. This can lead into sensitive files that are not linked from the application, viewed by the client. Even if the file itself cannot be accessed, the mere presence of that file – or the file’s name – may give the attacker damaging information about you or your site.

Default accounts are not changed

If your default account is admin or test, and you left your password as password, an attacker can easily guess them and log on to the application. You should always modify every default account on all applications and servers.

Misconfigured firewalls

A firewall that allows more ports than necessary to be open, or allows unauthorized hosts to connect to the server, can result in an attacker gaining control over the server. For example, a database server that requires an open port in order to execute queries from the web server may neglect to restrict access to the open port. In that case, any attacker can then connect to that port and attack the database, using brute force techniques.

Missing OS security patches

Neglecting to update your OS will result in an attacker utilizing security holes to gain control over your server. It is recommended to apply critical security patches immediately and have a regular maintenance interval for all OS updates. Regular updates should be tested in development before deployment to production to insure application compatibility. Remember, if your OS or application vendor sends you a security patch, by definition, it’s known to the world. Leaving a security patch un-applied is an invitation for exploitation.

Misconfiguration is a part of the OWASP top 10 most critical applications security risks.

Conclusion

Misconfiguration is a part of the OWASP top 10 most critical applications security risks. Calavista rigorously manages configuration and security on our projects – as all reputable software development shops should. When hiring a software development group, ask them hard questions about how they manage security issues – both during development and in the deployment phase. A good shop should be able to answer that question without hesitation. Beware of the deer-in-the-headlights look.


5 Best Practices for Embracing Agile

Before jumping into Agile, it is good for organizations to better understand the spirit of Agile and how to use it to make the shift.

More and more companies are making the transition from traditional development methods to Agile. Many times, development organizations are not always clear why and how to make this shift. A common “pro” that I hear from stakeholders and project managers new to Agile is, “We need to move to Agile because we are always changing our minds.”

However, the Agile Manifesto actually values: “Responding to change over following a plan.” It is common for new organizations to misinterpret this as, “we can change our minds without impact to the plan.” Before jumping into Agile, it is good for organizations to better understand the spirit of Agile and how to use it to make the shift.

Agile offers a lightweight and flexible framework that delivers business value much more quickly than traditional development methods. And, its continual iterative process provides an adaptability that ensures that software remains aligned with business needs, which makes it very appealing.

Unlike traditional development methods, Agile is about the conversation of what the business is looking for in a system. It focuses on user experience rather than system experience, and on collaboration and communication at the right levels – not extensive documentation. Having said that, there are some key best practices that can make all the difference in successfully practicing Agile and reaping the many benefits it has to offer.

Get Everyone Aligned

Agile focuses on user experience rather than system experience, and on collaboration and communication at the right levels

The first order of business is to get everyone on board with what Agile is, what it means, and how it works. Agile is as much a philosophy as a methodology, and the way you make it work for your projects and environment will look different for your company than for someone else’s. Right from the start you should establish your approach and ensure you have buy-in from all participants. I have seen this work from both a top-down and bottom-up approach. The key is to prepare Agile training for all levels of the organization prior to the start of the project.

Ensure Participation

Agile absolutely requires participation. But most people involved in a software project have other priorities and responsibilities, which can diminish the participation that’s needed in an Agile environment. It is essential to create a cadence in your meetings that is predictable, measurable, and actionable to keep your users engaged in the conversation. This will require more face-to-face time because nonverbal communication is key. If your teams are distributed, hold regular videoconference for the meetings. Don’t shortcut this − the benefits of seeing someone’s face in a Scrum meeting cannot be overstated. Don’t settle for a conference call if video is a possibility.

Keep your business owners engaged throughout the process − and make sure it’s valuable for them. Business owners do not have to participate in every meeting. Make sure the meetings they’re in are tailored specifically to their concerns and domain knowledge. Most likely, your business owner would not care about the internal data structure being developed, but they would care what type of reports they could get from the system. The bottom line is, if people − from stakeholders to the development team − aren’t engaged or don’t show up, you can do the mechanics of doing Agile, but you’re really not doing Agile.

Collaborate

It is essential to be transparent with all of your work in progress even as your specifications are being created.

Set up a collaboration tool set − a common place for shared docs or wiki page where people can see your work in progress. It is essential to be transparent with all of your work in progress even as your user story and/or specifications are being created. If you don’t have access to an electronic online collaboration tool set, create portable user storyboards out of paper and sticky notes. Always have a place where people can see the work in the progress, and feel comfortable commenting on it. That can be a Wiki or a war room with marker boards and Post-It notes, but there must be a project nexus where all information is available.

Be Transparent

Be as transparent as possible. It’s amazing what feedback and ideas you’ll get from people when they see what you’re doing. You’ll discover things you didn’t even know were an issue all because someone else may have a perspective or information that you don’t. This also allows for the team to make small corrections through the process to keep the project on track, as opposed to a major shift after a completed phase typically done in the traditional waterfall approach. Having a nexus, as described above, is one way to force ourselves to be transparent – all the time.

Be Disciplined

Lastly, don’t fall back into waterfall even when it may seem easier or simpler. Embrace and work through the challenges you may face. Take the time to do an Agile retrospective; not only will it be very helpful, but it is essential to continually improve your process. Agile is a set of values and not a methodology. Remind yourself and your organization of the Agile Manifesto and work in the spirit of it to make the shift.


Don’t Choose An Amateur When You Need a Pro

When a team cannot put the nail in the coffin and finish a project, it’s usually because they’ve accumulated too much technical debt.

If you think it’s expensive to hire a professional, wait until you hire an amateur. – Red Adair*

I recently changed the radiator in my Jeep. I did it myself, in part to save money, in part because it was a project my son and I could do together, and in part just because I wanted to. There’s satisfaction in doing a task yourself, even if – sometimes especially if – it’s not the kind of task you normally do. And with a 20-year-old car that I paid $500 for – what’s the worst that can happen?

On the other hand, there are tasks that I defer to the professionals on. Brain surgery, for one. Another is accounting. My business partner and I are smart guys – we could certainly figure out how to file our own corporate taxes if we wanted, but why? The cost of doing it wrong is huge. If you overpay, you’re out thousands of dollars. If you underpay, you could go to jail. All in all, the cost of doing it right is not much compared to the cost of doing it wrong. So I use a professional.

Sometimes people choose amateurs to do a job that calls for a professional. I see this in software development all the time.

At Calavista, we recently spoke with a prospect (we’ll call him “Mycroft”) who had contacted us after his (outsourced) development team failed to make its third consecutive delivery deadline. His team had been working for nine months on an application, and though they’d done “great work” so far, they had not been able to actually release the project.

When we asked what “great work” meant, he said that the product demoed cleanly, looked good, and clearly worked – it just wasn’t complete. There were a few minor bugs to work out, and he couldn’t understand why it was taking so long.

Accumulating Technical Debt

Personally, my experience is that when a team cannot put the nail in the coffin and finish a project, it’s usually because they’ve accumulated too much technical debt. That is, they might find the quick, “demo-able” solution to some problem is X, while the real, “releasable” solution would be Y. Y is more complex and time-consuming than X, so they do X. Often the reasons for this are valid – e.g., the customer needs to see the functionality working so they can make decisions on other things – and sometimes the team is just lazy. Or ignorant. But regardless, the choice to do the simple thing is made again and again, and ultimately when the time comes to actually deliver, all of those choices now have to be addressed. In some cases, there may be so many things to resolve that it’s an effective re-architecture of the product, and “finishing things off” would really mean re-writing much of the code.

When we warned Mycroft of this, he pooh-poohed the idea (“I’ve seen it work!”), and indicated that the code was solid, and he didn’t think it would take long (or cost much) for us to clean things up and ship his code. He made sure we knew that he was very experienced, having run many development groups and brought lots of products to market. He knew what things should cost in this industry, and (in so many words) put us on notice that he was nobody’s fool.
During the course of this conversation, it became apparent that Mycroft had a team of eight people, for which he was paying an average of $6/hour. Now the warning bells were really going off. But we did agree to look at the code, so that we could give him an estimate.

The Harsh Reality

Account passwords were stored in plain text, credit card CCID numbers were actually stored in the database, and worst of all, there were huge vulnerabilities to SQL injection attack.

The code was frankly shocking. Shortcuts were taken everywhere, crippling the application. Account passwords were stored in plain text, credit card CCID numbers were actually stored in the database, and worst of all, there were huge vulnerabilities to SQL injection attack. Taken together, this meant that a savvy attacker could easily spoof the application into revealing all of the customer names, account info (including credit card numbers and CCIDs), and passwords. These problems – signs of quick and easy implementations in order to get functionality to demo – were systemic and ubiquitous. This was not an enterprise application. Security, scalability, performance… everything had been sacrificed to make a demo work.

The analogy we used was to a Hollywood set. His developers had built a false town. The bank’s façade was complete, but inside, there was no safe – the money was lying around in piles. Same with the hotel, the saloon, the blacksmith’s shop. You could walk an investor through the town on a guided tour and it would look good, but if you opened the doors to the public, it was all over.

The upshot is that Mycroft’s company would need to start over completely. In this case, choosing the cheapest option was a spectacularly bad decision. Six-figures of investment (even at $6/hour), was down the drain. More importantly, nine months of market opportunity was lost.

Hire the Professional

I’m actually not a believer in “you get what you pay for.” I’ve seen large software firms charge exorbitant prices for middling work. But I do believe in hiring the professional. In this case, Mycroft’s team clearly had no experience in producing applications that were enterprise-class, didn’t follow institutional guidelines on credit care security and, in general, didn’t observe commonly-accepted (and necessary) standardized coding practices. They wrote a piece of software that could impress an individual in a demo, but could never actually be used.

There are all sorts of lessons to be learned here. “If it looks too good to be true…,” “always interview your developers,” “always perform code reviews,” “build in specific performance/security/scalability requirements in from the start,” etc. But Red sums it up the best. Hiring an amateur can be the most expensive mistake you’ll ever make.

*Red Adair was a legendary oil well firefighter, whose company was hired to extinguish some of the most catastrophic oil well blazes in history from Texas, to the North Sea, to Kuwait. He charged top dollar for his services, but his customers knew that no one would do the job better or faster.


Who Moved My Team? Part 5 – Techniques and Metrics

The lesson here is that while data is good, it cannot be obtained at the cost of productivity.

Managing Virtual and Distributed Development Groups in an Outsourced World – Part 5 of 5

In Part 4 of “Who Moved My Team,” we discussed how to ensure IP continuity on a distributed project. Today we look at techniques and metrics.

Part 5 – Techniques and Metrics

Techniques for managing virtual and distributed teams can be summed up in four key parts:

Monitor

Strive to build the most complete picture you can of the entire software development process. Don’t rely on milestones or a single metric (or two). Get enough information to paint a complete picture of what’s going on behind the scenes, not just what’s coming out the door. Focus especially on developer load, time spent fixing bugs to time spent writing features, bug recidivism rates, and estimation accuracy. If the first three are going up, things are going badly, no matter how great the “progress” is perceived to be. If the estimates are inconsistent, then there’s no reason to expect that the goals set by the development team are realistic in the first place.

Automate

Assume that developers will spend zero time supporting your instrumentation. Either they won’t support it, and the data will be sketchy, made up, or skewed by their memories – or they will attack it dutifully, but it will severely hamper productivity.  Whatever you install must require virtually no action on the part of the development team to work. Things that absolutely require action on the team’s part, like change requests, should be able flow back and forth with as little required effort as possible.

Manage

You need to act on the information you have. Pay particular attention to changes. If your team’s time spent fixing defects suddenly climbs from 10% to 20%, ask yourself a) why, and b) where is the extra 10% going to come from? Ask “why” before adjusting the schedule – as Steve Maguire says, it’s better to search for the leaks than to make the team bail faster.[1] But if the leaks either can’t be found or fixed, be aware that your team will either have to work longer hours (which will decrease their overall productivity), or you’ll have to slip the schedule. The more time they spend fixing bugs, the less time they spend writing new code. It’s just that simple.

In addition to tracking the progress of a project, use this information to determine the strengths and weaknesses of your staff, or to point out problem areas of the code. Ensure that your best people are being rewarded, and that weaker ones are being trained or culled out.

Finally, don’t assume that your local manager will take care of problems you are observing. Would you make that assumption if the team were local? Local management is there to assist you, not replace you. Besides, he or she is immersed in local politics and pressures. Though they may have their finger on the pulse of the group, you have objectivity on your side, and so you frequently have insights that they don’t have. And it’s ultimately your team.

Plan for Change

Turnover is a fact of life. You need to plan for both the incremental kind and the wholesale kind. For the incremental type, make sure developers continue to cross-pollinate, and that you are actively identifying and encouraging the most important ones.   Bringing key people to the U.S. for “training” is both an excellent opportunity to make sure their knowledge is shared, and to encourage them to stay with their employer (or you) when encouraged to leave by a competitor. You want them to ask a prospective recruiter, “How many times a year will you send me to the U.S.? Can you help me establish relationships with companies there that may want to hire me?”  Paradoxically, if you’re helping them to develop their career, they’ll actually want to stay longer, not leave sooner.

For the wholesale changes, ensure that your code management tools are robust, and that they support issue-based or change-set based code views. Having a clause that lets you hire key employees (see above) in the event of termination of your agreement can be important. If you’ve done the first part well, you’ll know who they are, you will have spent time with them in the U.S., and they’ll already be thinking of this as an opportunity.

When dealing with an offshore team, it’s not necessary to just hire a good manager or partner and hope for the best – offshore software development can achieve similar levels of quality that offshore hardware manufacturing has. By appropriately applying technology and methodology to the offshore development process, managers can gain insights that rival what they could gather by being present.

Key Metrics

I identified some specific metrics targets in the preceding blog posts. But where does this information come from?  There are many tools packages out there that can help, and more are available every day. At our various client sites, we’ve used both open source and commercial tools, depending on the client’s environment and budget. The tools, and functionalities available, change rapidly so rather than list a bunch of companies and products, I’d suggest a Web search on “software development metrics,” or similar, together with the names of some tools you currently use. Sourceforge.net and the other usual suspects have some useful offerings. However, there are a few key points to keep in mind when implementing a data program.

Pain/Value

The problem is that quantitative data is painful to obtain. I once put in place a detailed metrics-gathering program, where developers would track their estimates, their time, causes of defects, etc. I spent two weeks selling it to the development staff.  When I had 100% buy-in and commitment, we began executing the plan.

It lasted one day. That’s how long it took for developers to become overwhelmed by their tasking. It didn’t matter that the data gathering would only take 10-12 minutes on every task – no one had 10-12 minutes. One comment I heard was, “It’s noon, I’ve got one of six tasks done today, and following the plan will take over an hour for the six tasks. The bug is fixed – let it go, and let me get back to my job writing code.”

The thing is, he was right. Keeping track of data wasn’t his job. I didn’t hire him to be a bookkeeper – I hired him to write code. I know of one project where a customer was billed several thousand dollars per month to pay for the time it took people to track their time. Tasking them to do that kind of work is a waste of their time and your money.

What Works?

The lesson here is that while data is good, it cannot be obtained at the cost of productivity. Not only will productivity drop, but developers will resent it, and will often start cutting corners, like waiting until the end of the day, or week, or month, to complete their data sheets. Once this happens, the data’s out the window, because at that point it becomes as much wishful thinking as the estimates themselves were. Data acquisition must be near-painless to the development team. If it’s not, not only will it be resented, but its accuracy will be so compromised as to render it worthless. This implies that metrics must be gathered essentially automatically, with virtually no effort on the part of the staff. Tools that require developers to record when they start coding, run their first compile, begin debugging, etc. must be avoided. About the most you can hope for from developers is for them to click once on something when they start a task, on again when they finish.  If it’s more involved than that, 99 times out of 100, it just won’t work.

 

[1] Steve Maguire, Debugging the Development Process, Microsoft Press, pp 151ff


Who Moved My Team? Part 4 – How Can I Ensure IP Continuity When the Bulk of My Team is Remote?

Attrition doesn’t have to be the huge issue many companies face.

Managing Virtual and Distributed Development Groups in an Outsourced World – Part 4 of 5

In Part 3 of “Who Moved My Team,” we discussed evaluating your teams performance. Today we look at how to ensure IP continuity.

Part 4 – How Can I Ensure IP Continuity When the Bulk of My Team is Remote?

The final pitfall for a manager is the threat of losing key resources. It’s easy to lose sleep over the question, “can a competitor cripple us by hiring away key people half a world away, whom I might not suspect are at risk?”

I have seen turnover rates in India variously quoted by self-serving sources anywhere from 10% (“See, India’s pretty stable.”) to 75% (“Yikes!”) annually. Regardless, it’s my experience, and that of those I’ve worked within the industry, that turnover is becoming a large problem. Other countries are more or less stable, but the key is that you have little direct contact with a distributed team. Watts Humphrey, founder of the Software Process Program at Carnegie Mellon’s Software Engineering Institute, made the claim that developers will stay with a job unless actively pushed, and they can be kept loyal and engaged through rewarding assignments and appreciation of their efforts.[1] That is, you have to know what they’re doing, and encourage them.

But how can you give the appropriate encouragement, or even know whom to give a pat on the back to, without being there? The quick answer is that’s what local management is for – but that’s just dodging the issue. At the end of the day, it’s your team, or else you’d be gone and the local manager would have your job.

Ensure that knowledge is transferred, both within the offsite team, and from the offsite team back to your company.

What Works?

First, attrition doesn’t have to be the huge issue many companies face. Mr Humphrey was correct: Engineers don’t just leave a company – they tend to be pushed. Remote or offshore resources who are treated like fodder – given the worst (most boring or most impossible) tasks to perform, paid a minimum, made to feel not part of the team, and asked to work long hours – will leave. Wouldn’t you? Those who are treated as teammates, whose ideas are listened to, who are given challenging but do-able projects, who are paid fairly and given adequate time off…those people stay. At Calavista, we’ve had remote developers doing great work on projects for 5 years or more. Project attrition just isn’t an issue for us, and it shouldn’t have to be for you, either.

However, since occasional attrition is a reality, you should focus a great deal of effort on retaining the people who matter most. Therefore, the first step is to identify the key people. This may require some actual contact, but some of the principles previously covered in the section on evaluating a team’s performance are germane here. Once they have been identified, it’s important to encourage them in their work. Bonuses, training opportunities (especially if that means travel to the U.S.), and more challenging assignments all make good rewards.

The second aspect is to ensure that knowledge is transferred, both within the offsite team, and from the offsite team back to your company. Much of this is taken care of by the cross-pollination discussed earlier. However, in an offshore world, it’s very possible for a partnership to go south, resulting in the unexpected replacement of your entire team. For this reason, it is extremely important to maintain strong code management capabilities, preferably with the ability to track code changes by issue, or change set. New developers, when assigned a bug fix, should not have to wander aimlessly through the code trying to understand it – they should be able to inspect the work that was done in the issue that is broken. For instance, if there’s a bug in Feature A, a developer should be able to isolate the changes introduced by Feature A as a starting point for her sleuthing. If she can’t, at best she’s wasting time. At worst, the obfuscation could cause enormous problems if the offshore team is changed.

Next Up

We conclude the conversation of managing virtual and distributed development groups in the Part 5 – Techniques for managing virtual and distributed teams

[1] Watts S. Humphrey, Engineers Will Tolerate a Lot of Abuse, IEEE Software, vol. 18, no. 5, pp 13-15


Who Moved My Team? Part 3 – Evaluating Team Performance

Generally, strong developers are characterized by the same thing that characterizes a strong development team: the ability to deliver quality code on time.

Managing Virtual and Distributed Development Groups in an Outsourced World – Part 3 of 5

In Part 2 of “Who Moved My Team,” we discussed how to manage change. Today we look at how to evaluate a team’s performance.

Part 3 – How Can I Evaluate a My Team’s Performance?

I’ve never met a development manager yet who would say that they actually didn’t know who their best developers were. But when asked, “What makes you think she’s your best developer?” most managers will give some pat answer such as, “I’ve been doing this for 20 years – it’s my job to know.”

In fact, we’ve seen again and again that when a team is evaluated, it’s often the “best” developers who are causing some of the most serious problems. That’s not to say that they aren’t “good,” or even the most important developers on the team. They frequently are. But often, it’s the most prolific developer, the one who’s been with the company since the beginning, who knows all the ins and outs of the code and who personally wrote many of the core routines, who is the one that breaks the build most often, or introduces schema changes which cause problems for other developers, or has surprisingly high defect rates. It’s not always clear why this is so, but we have observed that such key developers are frequently given the most difficult tasks, rarely turn to others for help or review, and are usually overloaded. Combine that with the fact that the environment in which they made their first and greatest contributions was one of frantic activity and heroic effort by a few developers, and it may be that they just have difficulty making the transition from a skunk-works mentality to that of a mature, disciplined development shop.

Why Bother?

So, do we really care who the best developers are? Isn’t it a team effort? There are many reasons to try to evaluate individuals, which go beyond the simple, “whom do I keep, and whom do I fire?” questions. The goal is not so much to force-rank your team, but to determine the strengths and weaknesses of individuals so that they can be better managed.

If you can clearly identify these strengths and weaknesses, it becomes much easier to coach these individuals to higher levels of performance. For instance, a developer who consistently misses deadlines may turn into the company’s most valuable asset if just given a private office space with a door, because he is brilliant but easily distracted (see [1] for a discussion of the impact of walled offices on developer productivity). Another may benefit enormously from outside training in a particular subject. Finally, knowing your team’s individual strengths helps you meet their needs better, and thus keep your most valuable employees on hand.

Traits of a Good Developer

What makes a good developer? Generally, strong developers are characterized by the same thing that characterizes a strong development team: the ability to deliver quality code on time. There is also the “technical prowess” aspect, but in my opinion, finding the Alpha Geek is not really the same thing as finding the best developer.

So how can you tell a good developer from a bad one? Part of it is, in fact, the walking around thing; the human touch – the manager’s investment of personal time in her team – which should not be overlooked. But if you’re a world away, what else do you have? You need more quantitative data − and there are several metrics which are particularly revealing:

Estimation Accuracy

Not to beat a dead horse, but it’s all about delivering quality code on time. Developers are notoriously inaccurate when it comes to estimating times, and there is absolutely no substitute for developers estimating all tasks, and then comparing actuals to estimates. And by all tasks, I mean all coding tasks. At one company, we were tracking estimation accuracy. We were pleasantly surprised to discover that the median task was delivered in 80% of its estimated time. Unfortunately, that didn’t explain our problems, so we did a little more digging and discovered something interesting: the average time estimate was just over one hour. What was happening was that people would estimate simple, straightforward (and quick) bug fixes, but when faced with the hairier and more time-consuming bugs and features, they just wouldn’t bother with an estimate. (“But I don’t know how long it’s going to take me – I haven’t even started it yet!” is a reasonable response, but not very helpful when trying to scope work.) When we started mandating estimates on every task, the results quickly began to look much uglier. The bottom line is, if you’re not estimating every task, you’re not getting valid data.

As an aside, time can be measured in different units, e.g. elapsed, active, on-task, etc. (Get the blog post on measuring time here.) In the end, it really doesn’t matter which units developers use to give estimates, but the developers themselves must recognize which time frame they are operating in and be consistent about it. And you, of course, must know how to interpret their estimates.

Defect Rates

The other part of the “ability to deliver quality code on time” mantra is delivering quality code. Measuring defect rates is difficult, but important. With a locally managed team, managers can often come to a sense of this by asking developers (all developers, not just the “good” ones) where they think defects are coming from. Some will cover for their friends and some will pin the blame on someone they don’t like, but if you consistently ask this question of everyone eventually a picture will form.

A more direct way to get this information is through the dreaded “bug bowl” (aka “test fest,” “bug-o-rama,” etc.). I say dreaded, because I don’t like them for many reasons, which I won’t go into here. However, they can serve a purpose, especially if done with a specific goal. One approach is to grant bonuses to developers based on the number of bugs they find, but subtract money from their bonus pool for each bug of theirs that someone finds. There are some keys caveats and rules to be followed here, but it’s possible to quickly generate an awareness of who tends to introduce bugs. This can be done with your outsourced team as easily as with a local group.

When a developer boasts that “no one understands this module but me,” it’s actually a pretty severe self-indictment.

Maintainable Code

Lastly, code must be maintainable by others. When a developer boasts that “no one understands this module but me,” it’s actually a pretty severe self-indictment. A developer who writes code that the other developers (of comparable skill and experience) can’t easily extend, fix, or otherwise recode, is either unskilled, undisciplined, or trying to build job security. No one stays with a company forever, and managers must plan for turnover. In his article, “Extreme Programming Explained,” Kent Beck describes the importance of cross-pollinating developers, so that no one “owns” one section of code. [2] This is important for the reasons he outlines, but from my point of view the most important one is to ensure that code is maintainable by others.

One common measure people apply to maintainable code is the comment ratio. This can be interesting, but comments for their own sake are rarely useful. More valuable methods may be applied, however. First, one can solicit the feedback of developers (directly or indirectly) on the toughest modules to work on, as when looking for defect rates. Or one can actually try to measure the defect rates. In this case, you’re not so interested in who wrote the code that had to be fixed, but who wrote the code that the bug was written on top of. Another way to approach this is, “What is the average defect-introduction by module?” For instance, if your team is actually tracking defects as they are fixed, and determining who wrote the bugs, you can take it one step farther and start tracking who wrote the code that the buggy code was leveraging. If Bob writes buggy code in module A, then his defect rate goes up. But if the average developer is twice as likely to introduce a bug in module A than in any other module, then the writer of module A might actually be a larger part of the problem than Bob.

What Works?

It would be great to keep track of which developers are writing code that others are having trouble extending, and how many defects per 1000 lines of code different developers are introducing, based on the technologies they’re implementing, and all of that. But how do you do this with a remote team? For that matter, how do you do this with your local team?

It’s true that this level of detail is difficult to track without specialized tools, but valuable information can be obtained just by tracking simpler values: estimates to actuals, task completion rates, and bug recidivism, to name three. If the monitoring of these relatively straightforward metrics is coupled with regular polling of the development team about problem areas of the code (“if we could stop for four weeks, which sections of the code would you want to see rewritten?…”), or for that matter, problem developers, some real conclusions can begin to be drawn about both your team, and about individual contributors.

Next Up

We continue the conversation of managing virtual and distributed teams in Part 4 – How Can I Ensure IP Continuity When No One Else Actually Works on the Product I Manage?

[1] Joel Spolsky, The Joel Test: 12 Steps to Better Code, JoelOnSoftware.com

[2] Kent Beck, Extreme Programming Explained, Boston, Addison-Wesley


Who Moved My Team? Part 2 – Managing Change

It is critical that whatever change management tool is used allows developers, managers and testers to share the same specifications and modify them easily.

Managing Virtual and Distributed Development Groups in an Outsourced World – Part 2 of 5

In Part 1  of “Who Moved My Team,” we discussed ways to determine if a release will ship on time, along with the pros and cons of managing to milestones or using a task-based or metrics-based approach. Today, we tackle how to manage change.

Part 2 – How Can I Manage Change?

Change in a software project is inevitable. In fact, Agile software methodologies generally embrace change as a desirable part of the development process. But such methodologies (e.g. Scrum, XP, et al.) generally rely on a very high degree of cooperative effort between development, management and the end user.

One company we worked with ended up shutting down their distributed QA team because they couldn’t manage change. Developers would change product specifications on the fly, and the outsourced QA team would log the new “feature” as a bug. Both teams became frustrated with each other, and ultimately, the testers were brought back in house. This company threw away a sizable commitment of time and money because they couldn’t field a change management system that would allow developers and testers to communicate quickly over changes to their specifications.

What Works?

It is unreasonable to expect that in all cases specifications will be executed as originally implemented. Simply saying, “developers may not change the specification,” does not address the problem. Agile is all about embracing change, after all. Therefore, when managing requirements, especially for a remote team, it is critical that whatever change management tool is used allows developers, managers (including both development and product management) and testers to share the same specifications and modify them easily.

In the most aggressive case, this implies a workflow/authorization process that some high-end requirements management (RM) tools provide. However, in the simplest case, a developer must be able to at least easily document the fact that they deviated from the specification, and it must be easy for Testing and Management to observe that fact.

The simplest answer is to implement an electronic version of the storyboard – one that everyone on the team can easily look at, comment on, or change. There are numerous tools that can facilitate this kind of collaboration. Tools that can do this job – to varying degrees of effectiveness – include:

  • Acunote
  • Scrinch
  • AgileZen
  • TinyPM
  • Jira
  •  Trac
  • Mingle
  • VersionOne
  • Rally
  •  xPlanner
  • Pivotal Tracker
  • 21Scrum

There are many, many more to choose from, and I make no attempt to rank them here – I’d get hundreds of emails about how I “missed the best feature of X”, or I just “didn’t get it” when I said Y about Z… It doesn’t matter – search for “agile collaboration tools” and you can find full-featured reviews galore. The point is, there are an abundance of tools that allow close integration between the people gathering and formalizing requirements, writing code, and testing. Find one you like and put it to use.

Key features should include the ability to easily

  • Generate new requirements
  • Change requirements – and easily see who has made what change…
  • Comment on, or ask clarifying questions about, requirements
  • Indicate the state of a requirement
  • Record testing results

But the most important feature is that the tool must easily integrate into your team’s workflow. It must be the repository of record for all project documentation. It doesn’t matter what someone told you on the phone, or even wrote in an email – it is the collaboration tool that holds the definitive set of notes. Therefore, it’s critical that team members have the tool open all the time, whenever they are working. It should be like an email client – always accessible.

Implicit with that is that there should be one such tool. We had a customer who regularly tried to use multiple channels to convey information. On any given day, you might receive a phone call, an email, a text, a ping on Skype, an update in their issue tracking system, or a note in Hipchat. Any one of those media could be used to change a specification, or to re-prioritize tasks. We had to tell the customer that we would only respond to comments posted in the issue tracking system – and actually turned off Skype and chat, and stopped answering the phone.

The point of all of that is: pick one tool and use it as the source of record. Make sure every piece of code written tracks back to something documented in that tool, and document any changes there. Other media can be used to bring someone’s attention to a change to that source, but may notreplace that source as a record of change. Do all of that, and you’re halfway home…

Next Up

We continue the conversation of managing virtual and distributed teams in the Part 3 – How Can I Evaluate My Team’s Performance?


Who Moved My Team - Part 1 of 5

A software organization is only as good as its ability to ship quality code on time.

Managing Virtual and Distributed Development Groups in an Outsourced World – Part 1 of 5

Agile software processes are difficult to execute well when your team is local, vastly more so when it is not. However, with a few exceptions (namely managing the risk of a wholesale team change) the goals of managing an offshore team are the same as managing a local one:  delivering quality code on time, while ensuring intellectual property security and continuity, and providing for the growth and improvement of your team.  It is the execution of these goals that is necessarily different.

As long as outsourcing managers try to emulate a close contact approach, their results must necessarily suffer. You cannot manage by walking around, and should not try to. The concept of management from a distance without relying on physical bridges must be embraced. That is, rather than a manager flying to India once a month to make sure they “have the big picture,” tools and techniques should be employed to generate the big picture without relying on direct contact. That is, we must construct “virtual bridges.”

There are four major questions that are key in any development organization, but which are particularly difficult to answer in an outsourced environment:

  • How can I know the release will really ship on time?
  • How can I manage change?
  • How can I evaluate my team’s performance?
  • How can I ensure IP continuity, when no one in my physical work center actually works on the product I manage?

Managers who can answer these questions accurately and consistently can write their own ticket in the evolving software development world. Parts 1 through 4 of this blog examine each of these questions in detail, and Part 5 concludes with obtaining key metrics and best practices for managing virtual/distributed teams.

Part 1 – How Can I Know the Release Will Really Ship on Time?

Ultimately, a release shipping on time is the acid test for a software team – because a software organization is only as good as its ability to ship quality code on time. You can have the best developers, the best hardware, and the best processes – but if you consistently miss your ship dates, you are simply not a good software organization. It doesn’t matter how strong your architecture is, how smart your developers are, or how advanced your technology is. If you’re 18 months late with the latest killer app, no one will care. Likewise, if your team is consistently four to six weeks late with every release, you’ll be looking for a new job before long. At least, you would be if you worked for me.

The 80% Rule

So how can you tell? The obvious answer is to ask the developers – but a corollary to the old 80% rule (“80% of the problems come from 20% of the code”) is that 80% of the code is always 80% done. Ask most software developers, “How’s it coming?” and the majority of the time you’ll hear something along the lines of, “nearly done,” or “looking good – a few things left to do,” etc. Obviously, you’ll get the occasional, “I’m concerned – it’s taking much longer than I’d expected,” (the 20%), but most of the time, you’ll get the expected answer. The point is not that software developers deliberately deceive, but that software development is difficult and full of unknowns. The more quantitative, unbiased, data you can accumulate, the better.

Milestones

Consider a manager with a distributed team, starting a large new software project. The project will take a year, and the team is half a world away, so clearly some sort of accountability mechanism must be put in place. A common mechanism is the milestone, so let’s start there. Let’s say there are six two-month milestones (M1-M6) in place. The project begins, and as many do, encounters some growing pains.  Nevertheless, no one wants to start off a project by missing their first milestone, so the team pulls together and works long hours to hit the first set of deliverables, even while still tweaking the infrastructure.

The next milestone is more complex and reflects that the team should be in full swing. Though they hit their first milestone, things are still pretty messy. In fact, M2 is nearly missed when it’s discovered that some of the work in M1 was actually defective, and two weeks are lost correcting the problems. But M2 is met – largely through the heroics of a few key people.

Local management decides that they have to get the problem under control, so they crack down. No vacations, and everyone must work Saturdays until we “catch up” or “get ahead of the problem.” The problem is that people under that kind of pressure actually get less work done than those who work regular hours.1  The developers are tired and distracted, and so introduce more defects.

M3 is a death march for the entire team, but the overtime and extra work are successful, and M3 is met. During M4, the wheels really come off of the cart. It’s been six months of overtime and pressure, cancelled vacations, and six- to seven-day work weeks. At this point, some key employees quit.  Specifically, the ones most confident of their ability to get a new job with a more humane atmosphere – read: the best developers. With fewer people and some key personnel missing, the remaining team members are under enormous pressure.

Near the end, it’s realized that many of the features that had thought to have been completed are actually bug-ridden and will need to be re-implemented. By working frantically, just enough features are completed in time to make the M4 goal – except that new errors are being found every day. Those are put into the backlog to be fixed later, masking the problem.2 Upon “successful” completion of M4, the development team takes a much-needed long weekend. They return to a laundry list of bugs that the testing team has discovered in their absence, which means that nearly one-third of the work will need to be re-implemented. Needless to say, M5 is missed and the project ultimately fails.

Milestones give periodic snapshots of the project, and have three major problems:

1) They tend to spawn “milestone-itis,” where hitting the milestone is seen as an end to itself,
2) A large portion of the available time may be spent preparing (administratively and otherwise) for the milestone meeting, and
3) There’s little visibility into what’s going on the rest of the time.

Task-Based Management

Another way to handle this is the “task-based” approach, where instead of large, discreet milestones, a progress chart is generated with a daily target of completed items. If the project above has 500 subtasks, one might expect to complete an average of two tasks per work day over the year. All tasks aren’t equivalent, but if you pass month three and you’ve only completed 50 tasks, it’s probably time to start asking some hard questions.

The advantages to this method are obvious. First, there’s no “milestone” to prepare for, so there’s less time spent in preparation/administration. Second, there’s a more consistent visibility into the pace of the work. Local management can’t bury a schedule slip under a blizzard of overtime and bonuses – it can be noticed immediately. However, if you look at the scenario above, the schedule was kept, with a few minor blips, right up until the moment M4 was delivered. That is, at M4, the correct number of tasks had been “delivered.” The observant manager might have recognized the bumps along the road for what they were – real problems being dealt with by overwhelming manpower – but she might as easily have attributed them to the standard ups and downs of the development process. It was not until things totally fell apart after the end of M4 in September that the scope of the problem would be apparent.

Metrics-Based Management

What were the warning signs?  Really, there were two. First, people were working longer hours, both during the week and the weekend. Eventually, vacations were cancelled, weekend work was mandated, and developers started leaving in frustration.  Second, as people were overworked, a greater and greater percentage of their time was dedicated to fixing bugs. If either of these metrics had been available, it would have been apparent before the end of M1 – in January/February – that there was a problem with workload.3  Around the end of April, it would have been noticed that the time spent on bug fixes spiked. In July, management could have observed a bunch of senior people stopping work (as they quit) and a bunch of new names show up on the development team. In August, the bug fix numbers spike again. But all along, since January, the average number of hours worked by the developers hovered in the 60-70 hour/week range.

Calavista-software-development-leader-austin-task-progress-vs-developer-hours-chart

This project was in trouble back in January. By managing to milestones, it wasn’t discovered until October – far too late to attempt to fix things. By adopting “task-based” metrics, it was clear at the end of August that things were in trouble, allowing four months to try to correct. But by monitoring more subtle cues – specifically defect rates, workloads, and changes to those metrics over time – it becomes clear 11 months before the end of the project that it is in serious trouble, and clearly not improving.

What Works?

The lesson learned here is that milestones give an incomplete picture and encourage local management to manage to the milestone, rather than the project. Task-based management gives a better view, but can only tell you when things have gone wrong. A better approach is to manage to metrics – what’s going on with the team, as well as what’s going on with the code base.

Another way of thinking about this is from a doctor’s point of view. Running a project by milestones is like someone telling you when the patient’s heart has stopped. You call the crash cart, use the paddles, and pray they can be saved.  If you’ve missed a project milestone, you’re already in extremis, and it’s unlikely the project will be a success. Likewise, using task completion rates is like tracking one vital sign – say, the pulse. Useful in many ways, but ineffective if the patient is suffering from cancer.

As a remote manager, you want the fullest array of vital signs that you can make available for yourself. Some metrics that are of particular importance are:

  • Who’s working on the project?
  • How hard are they working?
  • What are they spending their time on?
  • Are certain areas of the code causing undue problems?
  • How are their estimates holding up?

And, perhaps most important,

  • How are these metrics changing over time?

Next Up

We continue the conversation of managing virtual and distributed teams in the Part 2 – How Can I Manage Change?


 1 Steve Maguire, Debugging the Development Process, Microsoft Press, pp. 151ff
 2 I once worked on a project where some significant piece of functionality wasn’t done by the “feature freeze” deadline. It wasn’t even started. Rather than admitting to having missed the deadline, a developer was instructed to add the widget that would drive the functionality to the screen, and just leave it unimplemented. So then it could be classified as “complete,” and when QA pointed out that it didn’t work, they’d just call it a bug, and the developer could spend a week during the hardening period to actually implement the feature. Swear to God.
3 The progress chart (Fig. 1) shows the relationship to task progress, defect fix time, and average hours spent on a project in trouble. In this example it’s obvious at a glance that this project is in serious trouble early on – even though the task completion rate remains well over 100%.


Calavista Software Moves to Accommodate Significant Business Growth

We recognized the need to provide our employees and customers a new work environment that offered more room for collaboration and innovation

Calavista Software, a custom software development firm founded in 2001 and based in Austin, TX, has seen tremendous growth over the last few years – with increases in both client base and projects. As a result, Calavista doubled in size and eventually outgrew their space in the Hill Country Galleria. In early 2016 they completed a move to a brand new 12,000 sq foot space – located in the Spiceworks Building on 360 in West Austin.

“We pride ourselves in bringing together the right team for each of our clients. A key differentiator for our customers is the strategic leadership we provide to ensure projects are delivered on time and on budget. Over the past year, we recognized the need to provide our employees and customers a new work environment that offered more room for collaboration and innovation,” said Lawrence Waugh, Co-Founder.

Calavista specializes in delivering software that solves complex business problems. Backed by a patented delivery system, and experienced development directors and architects, Calavista has achieved an unparalleled project success rate of over 90% – nearly three times the industry average. “We’re proud to say that we deliver software development without the drama,” said Sandeep Gupta, Co-Founder, “and we’re looking forward to continuing the tradition of contributing to our customers’ successes over the next decade.”

Calavista offers software development services including consulting, architecture, testing, and UX/UI design. Founded in Austin, and with a long list of local and national clients, Calavista continues to evolve its approach with clients all over the world. According to Sandeep, “Our goal is to assemble an agile team that is dedicated to each of our customer’s specific needs – it was time for us to find a space that reflected that growth and success.”