hire professional

Hire The Professional

Written By: Lawrence Waugh, Founder at Calavista

"If you think it's expensive to hire a professional...
...wait until you hire an amateur."
- Red Adair *

I recently changed the radiator in my Jeep.

I did it myself, in part to save money, in part because it was a project my son and I could do together, and in part just because I wanted to. There's satisfaction in doing a task yourself, even if- sometimes especially if- it's not the kind of task you normally do.

And with a 20-year-old car that I paid $500 for- what's the worst that can happen?

On the other hand, there are tasks that I defer to the professionals on. Brain surgery, for one. Another is accounting. My business partner and I are smart guys- we could certainly figure out how to file our own corporate taxes if we wanted- but why? The cost of doing it wrong is huge. If you overpay, you're out thousands of dollars. If you underpay, you could go to jail. All in all, the cost of doing it right is not much compared to the cost of doing it wrong. So I use a professional.

Sometimes people choose amateurs to do a job that calls for a professional. I see this in software development all the time.

At Calavista, we recently spoke with a prospect (we'll call him "Mycroft") who had contacted us after his (outsourced) development team failed to make its 3rd consecutive delivery deadline. His team had been working for 9 months on an application, and though they'd done "great work" so far, they had not been able to actually release the project.

When we asked what "great work" meant, he said that the product demoed cleanly, looked good, and clearly worked- it just wasn't complete. There were a few minor bugs to work out, and he couldn't understand why it was taking so long.

Personally, my experience is that when a team cannot put the nail in the coffin and finish a project, it's usually because they've accumulated too much technical debt. That is they might find the quick, "demo-able" solution to some problem is X, while the real, "releasable" solution would be Y. Y is more complex and time-consuming than X, so they do X. Often the reasons for this are valid- e.g. the customer needs to see the functionality working so they can make decisions on other things- and sometimes the team is just lazy. Or ignorant. But regardless, the choice to do the simple thing is made again and again, and ultimately when the time comes to actually deliver, all those choices now have to be addressed. In some cases, there may be so many things to resolve that it's an effective re-architecture of the product, and "finishing things off" would really mean re-writing much of the code.

When we warned Mycroft of this, he pooh-poohed the idea ("I've seen it work!"), and indicated that the code was solid, and he didn't think it would take long (or cost much) for us to clean things up and ship his code. He made sure we knew that he was very experienced, having run many development groups, and having brought lots of products to market. He knew what things should cost in this industry, and (in so many words) put us on notice that he was nobody's fool.

During the course of this conversation, it became apparent that Mycroft had a team of 8 people, which he'd cobbled together, clearly with price as the driving factor. Now the warning bells were really going off. But we did agree to look at the code, so that we could give him an estimate.

The code was frankly shocking. Shortcuts were taken everywhere, crippling the application. Account passwords were stored in plain text, credit card CCID numbers were actually stored in the DB, and worst of all, there were huge vulnerabilities to SQL injection attack. Taken together, this meant that a savvy attacker could easily spoof the application into revealing all of the customer names, account info (including credit card numbers and CCIDs), and passwords. These problems- signs of quick and easy implementations in order to get functionality to demo- were systemic and ubiquitous. This was not an enterprise application. Security, scalability, performance...everything had been sacrificed to make a demo work.

The analogy we used was to a Hollywood set. His developers had built a false town. The bank's facade was complete, but inside, there was no safe- the money was lying around in piles. Same with the hotel, the saloon, the blacksmith's shop... You could walk an investor through the town on a guided tour and it would look good, but if you opened the doors to the public, it was all over.

The upshot is that Mycroft's company will need to start over completely.

In this case, choosing the cheapest option was a spectacularly bad decision. 6-figures of investment (even given the low hourly rate), down the drain. More importantly, 9 months of lost market opportunity.

I'm not actually a believer in "you get what you pay for"- I've seen large software firms charge exorbitant prices for middling work. But I do believe in hiring the professional. In this case, Mycroft's team clearly had no experience in producing applications that were enterprise-class, followed institutional guidelines on credit card security, and just observing commonly-accepted (and necessary) standardized coding practices. They wrote a piece of software that could impress an individual in a demo, but which could never actually be used.

There are all sorts of lessons learned here. "If it looks too good to be true...," "always interview your developers," "always perform code reviews," "build specific performance/security/scalability requirements in from the start," etc. But Red summed it up. Hiring an amateur can be the most expensive mistake you'll ever make.

*- Red Adair (en.wikipedia.org/wiki/Red_Adair) was a legendary oil well firefighter, whose company was hired to extinguish some of the most catastrophic oil well blazes in history: from Texas, to the North Sea, to Kuwait. He charged top dollar for his services, but his customers knew that no one would do the job better or faster.

Web App

Web App Jump Start Comparison

Written By: Steve Zagieboylo, Senior Architect at Calavista

Presentation1 (dragged)One thing Calavista does very well as a company is building Web Applications from scratch. If you have a great idea for an application and you plan to bet your life savings on building a company around it, a smart move would be to find a development organization with a on-time, within-budget record of greater than 90%, such as Calavista. One way that we achieve this record is by using some great tools to jump start the application.

Quick Comparison

JHipster is more of a one-and-done quickstart -- at least, that’s the way I’ve always used it.  Once we had the initial project with the data model basically defined, we did not use the tool any more.  JHipster does support such a mode, I understand, but it feels more awkward.  Cuba Platform, on the other hand, is completely designed to be used for the lifetime of the project.  There is an exit strategy if you find you are not happy with it, but clearly the intent is that you continue to use it.

Best Practices for a Java Web Application

There are a number of elements that are common too many applications -- web framework, security, build environment, etc.  It doesn’t make any sense to develop them from scratch each time.  Of course, every new project comes with its own different challenges, so we do not want some “one size fits all” framework.  What we need is a great starting point, but one that does not limit where we will eventually end up.  It should provide us exactly what we would have created if we had the time and skill to build it properly from the ground up, but do so in minutes rather than weeks.  All the pieces should use Industry Best Practices, which, for this purpose, I’ll define as follows.

  • Spring Boot Web Framework
  • Clean separation of layers: Data Model, API, Business logic
  • Authentication using Spring Security
  • Database access using JPA
  • REST services defined with JAX-RS annotations
  • Maven or Gradle build system, with proper project structure
  • Unit tests with at least 80% code coverage
  • Use a “best of breed” UI framework

Complete, Functioning Application

The big requirement for our Jump Start is that it should create a complete application, with some easy way to create our data model without having to construct all the pieces by hand.  There should be some meta level at which I describe the Data Model, and the Jump Start tool builds the pieces for me, from the UI to the DTOs to the database entities. 

Presentation1 (dragged) 2I can’t overstate the importance of a complete, functioning application. Consider every project where you spent the first few days, at least, just getting a basic “Hello, World” working. Of course, for a “Hello, World” in a web application, you need at least user identity, database connection, login, basic user management, a UI framework, and a REST framework. Building those from scratch is going to take a few days, at least. Building them with one of these quick starts is under an hour, and you get a lot more functionality besides.

There are a few tools which purport to give such a quick start to your application.  This set of blogs is going to focus on two of them which are very different in approach but both accomplish the goal admirably.  They are:

These other tools were also considered, but discarded for different reasons:

  • Django -- https://www.djangoproject.com/  This also looks promising, but it creates a Python-based application, rather than Java.  While this also is a viable alternative for web applications, we rather arbitrarily decided to limit our choices to Java for now.
  • Several no-code application builders:  bubble.io, kintone.com, journeyapps.com, all discarded because we are planning for an application that is something more than just a glorified CRUD application over a database.  We want the ability to get to the core code, once the basic application has been generated for us, to write custom business logic and to create custom UI.

We’ll consider these two platforms on these criteria:

    Presentation1 (dragged) 3

  • Installation and setup
  • Defining the Data Model
  • Creating and Running the Application
  • Built-in Functionality
  • Analysis of the Generated UI for the Entities
  • Analysis of the Generated Code
  • Analysis of the Generated Unit Tests


I have a lot of experience with JHipster, having used it successfully for three different projects for Calavista customers.  So I plan to spend more time with the Cuba Platform, because I am learning it for this exercise.  (That’s actually the reason for the exercise.)  But I promise to try to look at JHipster as if I were approaching that with equally unfamiliar eyes.

Continuous Deployment

Yes, Virginia, Continuous Deployment Does Have Controls and Approvals: Part 3

Written By: Daniel Kulvicki, Solutions Director at Calavista

In my last two blogs, I went over the specifics of Continuous Deployment and gave some examples of how you can enforce quality and controls even though you are releasing at breakneck speeds.  To finish off my series, we will dive into Dark Launches and Feature Toggling.  These mechanisms allow our teams to deploy software as fast as possible while reducing risk of bugs or issues to our broader customer base. 

Dark Launches

Let us start with the official definition of what it is to provide Dark Launches of software features. 

Dark launching is the process of releasing production-ready features to a subset of your users prior to a full release. This enables you to decouple deployment from release, get real user feedback, test for bugs, and assess infrastructure performance.

Although that is the official definition of Dark Launches, I like to have my own interpretation.  To provide releases as quickly as possible, you can also Dark Launch and not have any users test the new feature(s).  Some companies prefer this as it allows for faster releases and reduces risks of issues.  What you are essentially providing is a “turned off” feature (or features) in production.  There is some risk since you need to ensure even if the feature is turned off that it does not impact other turned-on features in production.  However, that means you are still deploying code at extremely fast speeds and minimizing risk due to being able to turn on the feature later and test it with whatever size audience you need!

Feature Toggle

Feature Toggle is one of the most popular architectures used for Continuous Deployment.  At the beginning, the main usage of feature toggles was to avoid conflict that can arise when merging changes in software at the last moment before release.  However, what quickly came to fruition was a structure by which to introduce new code quickly without causing breaking changes within an application.  Some organizations that adopted Feature Toggles soon realized that they were on the verge of Continuous Deployment.  As their release cadence became sound and fast, these companies were able to change their software product lifecycle to adapt to take advantage of this newfound speed. 

Feature Toggles do have some downsides that will need to be addressed up front for an organization to adopt this style of architecture.  If you are not careful, technical debt can occur since turning on features permanently can cause stagnant code.  However, if you manage this correctly; the speed you gain will outweigh the management of the debt. 


Well, hopefully I have shown you how one implementation of Continuous Delivery can help your organization be even more agile than it currently is with keeping the same level of quality your customers deserve.  I do understand that Continuous Delivery is not a great fit for every organization and sometimes the overhead that it brings can be a bit too much for smaller groups.  Most organizations do not even evaluate if it is a possibility.  Hopefully, this series will help you and your team determine how more Agile you can become! 


Software and Sourdough

Written By: Steve Zagieboylo, Senior Architect at Calavista

My non-computer hobby since quarantine started has been making sourdough bread. I created my own starter and I’ve been making bread almost every weekend since last March. I’ve gotten pretty good at it, such that commercial bread now is unacceptable to anyone in my family. (In other words, I’m not allowed to stop making it every single weekend.)

In a Calavista company meeting last week I joked that I was planning a blog on making sourdough bread. The company president then challenged me to write a blog comparing software development to bread making. So here goes…

Software and Sourdough: StarterTeamSoftware and Sourdough: Plan

Software and Sourdough: ToolsI’d like to point out just how successful our processes are, both my bread making and our software development.  At 90+% on time and within budget, Calavista exceeds the industry norm by a ridiculous margin.  My success with bread is only around 80%; I’ve had a few bricks, but I’ve learned from each one.  (For instance, using whole grain flour means you need more water in the dough, and more time fermenting.)  Comparing the two processes was a bit of a stretch, but I hope it was entertaining and gave a little insight into the software development process we follow in Calavista.

Marten DB With Jeremy Miller

Written & Presented By: Jeremy Miller, Senior Architect at Calavista

The 6 Figure Developer is a website designed to help others reach their potential. They're focused on providing developers tools, tips, and techniques to increase your technical skills, communication skills, and income.




Marten DB with Jeremy Miller


In this podcast, our Senior Architect, Jeremy Miller, is interviewed by The 6 Figure Developer podcast hosts John Callaway, Clayton Hunt and Jon Ash. Jeremy has a technical conversation about Marten DB, a .NET Transactional Document DB and Event Store on PostgreSQL. Tune in to find out more!

irritant removal system

What Can The IRS Do For You?

Written By: Andy Fruhling, Chief Operating Officer at Calavista

As you start the new year and set your goals for 2021, I suggest looking into how the IRS can actually help youI’ve seen many cases where it has improved the user experience for key customers, built product capability beyond your team’s capacity and accomplished so much moreAs we are at the beginning of “tax season” in the United States, I should probably be a little more explicit.  I’m not talking about the Internal Revenue Service.  I don’t know anyone who has benefited from that IRS.  One of our customers recently called us their “Irritant Removal System” (IRS) and, after a brief chuckle, we saw how this concept  could be very beneficial to many companies. 


The Good, The Bad, and The Ugly

During my years of running product management teams at various software companies, I had my teams regularly conduct face-to-face meetings with our top customers.  The purpose of the meetings was to talk through “The Good”, “The Bad”, and “The Ugly”.  In other words, we wanted to hear directly from our top customers about what was working well with our product(s), what was not working so well, and what was driving the users crazy.   

I specifically organized the discussions in this order.  Hearing about what was working well (“The Good”), gave the product managers anecdotes we then shared with other customers and prospects.  We were also able to share “The Good” with the development teams, who too often only heard the negative and rarely heard about what was working well.   

Talking through what was not working well (“The Bad”) provided a good review of the top issues with the software for this customer.  Almost always the customer had already created trouble tickets for these items.  Further discussions during these meetings simply allowed us to get a better understanding of the issues including insight into the real “Why for these specific issues.  This provided a better view of the priorities for the top issues.   


The Golden Opportunity

For me, the real “Golden” opportunity was actually hearing about what was driving the end users crazy (“The Ugly”).  Initially, customers didn’t understand the difference between “The Bad” and “The Ugly” in this context.  The items that drive the end users crazy were irritants but not really bugs in the software.  Yes, they caused issues with the user’s experience, but they were not deemed worthy of a trouble ticket.  Many times, these items were relatively easy to address – perhaps something as simple as: “When I am doing this function on this screen, I need a couple of additional pieces of information from another screen.”    Sometimes they were more complex.  Either way, these items negatively impacted the experience of the end users and needed to be addressed.   

Although these customer discussions provided a lot of good information, they often did not lead to the results we wanted.  Once you have a list of irritants, what do you do with it?  The development capacity is usually fully committed to new features (roadmap) and bug fixes (maintenance).  Anyone who has actually run software development teams will probably tell you that these teams are always over committed on other items and the irritants had to wait unless you were willing to drop some roadmap items and/or maintenance items.  In other words, only a very small portion (if any) of the irritants were ever addressed and the end users see no relief from “The Ugly”. 

Eliminate "The Ugly" with IRS

Unfortunately, I did identify a good solution for this while running these product management teams early in my career or even as I, later, managed very large software development organizations.  It was always the same dilemma positioned around a tradeoff between the already planned roadmap and maintenance items versus the irritants.  Unfortunately, the irritants rarely made it to the list!  

It was not until our customer labeled Calavista as their Irritant Removal System that I made the connection.  As their IRS, we provide a cost-effective full team that delivers high quality code on schedule and can scale up or down as needed.  Let’s break that statement down.  Cost-effective means that our costs are roughly the same and perhaps even less than typical internal development teams.  Full team means the IRS includes more than just a couple of strong developers.  While strong development talent is a good start, you need the full team including management oversight, architecture, testing, DevOps, and possibly even automation of the development processes along with proven methodologyHigh quality code seems self-explanatory but too often this is overlooked, and people believe they can “test in quality” at the end.  We believe delivering high quality code requires a focus on quality across the entire software delivery cycle from initial story points through construction, testing, and production rollout with a consistent, repeatable, proven processes and DevOps automation where possible  We take great pride in our ability to successfully deliver projects “on schedule” for our customers at nearly 3x the industry average.  Like high quality, this comes from many aspects of our approach and has been proven on our projects for nearly 20 years.  Scale up and scale down provides the ability to address more irritants or less irritants as needed, based on business need.  With this model, the IRS can address issues that are impacting your end user’s experience with the software and help you address other items that do not fit within your internal development capacity.  So, ask yourself: “What can the IRS do for you?” 


code software development blue

Technical Debt Patterns: Copied and Pasted Code

Written By: Steve Zagieboylo, Senior Architect at Calavista


This is the sixth part in the series, "Technical Debt Patterns."

There are two (or more!) copies of the same bit of code, because some lazy (or unknowing) programmer copied a whole block and just edited a few of the lines.  Sure, it’s a lot easier than figuring out the right refactoring, and it gets the job done.


Why is it a problem?

I had another developer ask me that question in all seriousness, and I had to pause to answer without just referring to the obvious inelegance of it.  (He and I had debated before over the value of elegance; he didn’t think it was important.)  Fortunately, I knew of a specific example of his own code in which I had fixed a bug recently.  The bug was not an egregious error, just a weird situation not handled correctly, and was in my own code.

The problem was that after I fixed it, the bug didn’t go away!  I was so sure this was causing it, because it fit the symptoms so perfectly.  I finally figured out that actual bug I was trying to fix was in a parallel bit of code, the exact same error for the same reason, because the code where it was actually occurring had been copied and pasted from the place where I assumed it was happening.  In searching the code, I found yet another instance of the same problem, in a nearly identical stretch of code.  The other developer had copied and pasted the block I had originally written, and copied my bug.  It was fortunate that the copy I originally fixed hadn’t triggered the case I was focussed on, or I never would have realized that there were two more copies of the bug.


Cost Comparison

The cost of leaving these copies in place is clear, you can end up deploying code that has bugs in it which you have already found and believe you have fixed.  It is especially embarrassing when you inform your users that a particular bug has been fixed, and then have it show up again in a slightly different incarnation.  At the very least, the cost is that with every bug you fix, you have to double check whether or not that same bit of code appears elsewhere, and possibly make the same fix there, too.

The cost of fixing this particular issue used to be a lot higher, but with modern IDE’s and their powerful refactoring tools, it has become much easier.


How do I fix it?

The modern IDE’s have terrific refactoring tools to fix and to avoid this problem.  They will even point it out to you in many cases, though if there are enough small changes, they might miss it.  The refactor operation names are the same in both IntelliJ and Eclipse (for Java), as well as Rider (for C#).  I know that Visual Studio has the same refactor operations, but I’m not sure of the exact names.

  • Refactor / Extract Method
  • Refactor / Introduce Parameter or Refactor / Introduce Variable and Refactor / Inline Variable

First you extract the common bit of code as a method from one of the places where it appears, ignoring for the moment the bits that are different between the versions.  Consider whether or not this needs to be a static method, or if all the places that need to call it are in the same base class, and make sure you put it in that base class if this is the case.
In the new method, select a part that is different between the copies and use Refactor / Introduce Parameter.  The refactoring tool is clever enough to add the right value for the parameter in the calling code.

Another approach is to start by introducing a new variable for each of the different bits of code, move those above the code you extract as a method, and then inlining the values back into the method call.  (This is what we used to do before Refactor / Introduce Parameter worked so well.)

Finally, go to the other instances of the code and add a call to your new method, copying in the right parameters into the arguments before you wipe out the old code.


Your code has shrunken

If you’ve done the job right, you will have reduced the overall size of your code.  Of course, if anyone is measuring you by the amount of code you write, it is time to get a new job.


Continuous Deployment

Yes, Virginia, Continuous Deployment Does Have Controls and Approvals: Part 2

Written By: Daniel Kulvicki, Solutions Director at Calavista

In Part 1, we talked through some of the misconceptions of Continuous Deployment (CD) and how it does not create a “Wild West” release approach. This time, I want to cover three major flows or processes (using GitHub as an example) that can be implemented to enable delivery of stable, high-quality features while also allowing for code to be deployed continuously. Although most of the new code will have the new features turned off, we can still ensure minimal impact to the code in production. With this approach, releases can occur at a much faster rate as we are isolating areas to test while still allowing the code to propagate through all your different environments (Dev, Test, Prod, etc.).


1. Branching Flow

While there have been many branching approaches over the years, Calavista recommends a Trunk-Based strategy for Continuous Deployment. For Continuous Deployment, you need to minimize the time and effort of merging code
changes. This means your branching strategy cannot have multiple branches that require reconciliation as changes are merged together such as you see with development branching and release branching strategies. There is some expectation of automated testing coverage on every PR due to the fact we will rely on our automated tests to reduce overall manual testing time as noted in the PR Approval Flow section below.

As teams get bigger in size, sometimes short-lived feature branches will need to be introduced. However, they must remain short lived.

trunk based branching
                                           Figure 1: Trunk Based Branching

 2. PR Approval Flow

The PR Approval Flow ensures only high-quality code is merged into the Trunk branch. A Pull Request (or PR) is a
mechanism for a developer to notify team members that a unit of code is complete and ready for review before being merged into the code branch. Every PR that is created for Trunk will be required to go through a PR Approval Flow (such as the one below). While the approval flow may vary based on your specific needs, the flow must include the basics:

a. All PRs should create their own isolated environment for verification by people and/or automated tests

b. Each PR is tested to ensure the feature works as designed or bug fix is resolved as designed before it is merged into Trunk

PR Approval Flow
                      Figure 2: PR Approval Flow

3. Feature Activation Flow

Since features are usually deployed before they are activated, Product Managers will want a process by which to approve when a new feature is activated. New features have the potential to be turned off for extended periods of time before they are turned on. This can be due to Marketing Campaigns, Seasonal Patterns, etc. Below you will find the recommended Calavista Feature Activation Flow for Continuous Deployment.

Feature Activation Flow
                            Figure 3: Feature Activation Flow

While these approval flows provide a more substantial quality mechanism for Continuous Deployment environments, not all organizations are ready for CD. I have seen CD used effectively to create faster feedback cycles which in turn allow for a much faster feature development rate at lower costs and better overall quality. If you have teams that are not rapidly delivering high-quality code on regular, consistent basis, ask us how we can help.

In my final blog on Continuous Deployment, I will focus on Dark Releases (when features are turned off when deployed) and how to implement and maintain them for your organization.

Continuous Deployment

Yes, Virginia, Continuous Deployment Does Have Controls and Approvals: Part 1

Written By: Daniel Kulvicki, Solutions Director at Calavista

Often, Continuous Deployment is considered a step too far for most development organizations. Many people fear that it removes any gates or approval process and puts the release mechanism solely in the development team’s hands. This is incorrect. Continuous Deployment can actually can increase controls and provide better quality code!

I found an article online that described Continuous Deployment as too risky for any organization. Although the author made some great points, a wrong assumption was made that quality gates and even testers would go away. Yes, without quality gates and testers Continuous Deployment would be an absolute disaster. Continuous Deployment does not always mean Continuous Release. With concepts like A/B testing, feature toggling, and dark launches; it is possible to be able to continuously deploy safely, even to production, while being safe to your users. Below is a standard definition for Continuous Deployment.

“Continuous deployment can be thought of as an extension of continuous integration, aiming at minimizing lead time, the time elapsed between development writing one new line of code and this new code being used by live users, in production.
To achieve continuous deployment, the team relies on infrastructure that automates and instruments the various steps leading up to deployment, so that after each integration successfully meeting these release criteria, the live application is updated with new code.

Instrumentation is needed to ensure that any suggestion of lowered quality results in aborting the deployment process, or rolling back the new features, and triggers human intervention.”

Per the definition, instrumentation and automation are very key principles to Continuous Deployment, however; I have found that there are other requirements for Continuous Deployment that are equally as important. Quality gates and work contained in small batches allow for the right mindset to perform more frequent releases to production. You want to build a cadence with your team without quality suffering. At Calavista, we have a good set of principles to help enable that speed of deployment with the right security blanket for your team.

In my opinion, Continuous Deployment will only become more popular in the future and adoption will increase greatly. However, we cannot just jump into it. It needs to be treated the same as an organization moving from Waterfall to Agile development. Many different parts of a company will need to change to adapt to the new release model.

In my next blog, I will address a recommended implementation of a continuous release chain that we support at Calavista.

shopping for quality

Shopping for Quality- How to Ensure You're Hiring a Company That Can Deliver

Written By: Lawrence Waugh, Founder at Calavista

I’ve written before that most people who are looking for a software development firm to deliver code for them tend to really care about 3 things. Those things are:

1) Price
2) Quality
3) Speed of delivery

There’s no surprise there – those are the things we all care a lot about in most purchases. No one wants to overpay, buy inferior goods, or wait interminably (or maybe indefinitely) for most anything they buy. I’ve written previously about ways to minimize cost (spoiler alert: Fixed Fee is not your friend). In this post, I want to focus on quality.

How can you be sure the company you engage will deliver high-quality code at whatever price you’re paying them?

The good news is that it’s easier to tell than you may think.

Don't believe the hype

The first thing to realize is that any company can say anything they want about themselves. It doesn’t matter how nice their website looks – that’s the work of the marketing group and has nothing to do with how well they actually write code. They can speak eloquently and say all the right things. That’s the talent of their salespeople. You may even speak to some of their engineers and think they’re brilliant – and maybe they are. But “brilliance” doesn’t necessarily translate into superior software delivery. Software projects fail over 70% of the time (according to the benchmark 2015 Standish Group Chaos Report) and very rarely do you hear that blamed on the poor quality of the developers. “Well, yes we were late, but what did you expect? All of our developers are below average in skill…,” said no development manager, ever.

There’s only one thing that matters when judging the likelihood of receiving a quality software product on time and on budget: the company’s history of delivering quality software on time and on budget.

Nothing that they say about themselves matter. The only thing that matters is what their past customers say about them. So, ask around.

There are references and then there are references

Well, that’s obvious - asking a prospective vendor for references is de rigueur. Typically, the company will provide 2, or maybe 3, customers for you to speak with. That seems right, right? I mean, how many calls do you really want to make?

It’s not enough. Not nearly.

Every company has 2 or 3 go-to customers that will say nice things about them. So, they get used a lot. Perhaps there are relationships involved that transcend work, or maybe they’ve just made those customers particularly happy. Regardless, even a small, new company can usually come up with a few people for you to talk to.

But think for a minute what that means.

Have they only had 2 customers? Or have they had dozens, but only made 2 of them happy enough to be a reference? In either case, is that someone you want to do business with?

A company that serves, say, 10 customers a year, and has been around for 5 years, will have had about 50 customers. Hopefully, a substantial fraction of those 50 would be happy with the work they did. So then why would they only offer you 2 people to talk to? Or 3? Or 5?

Ask them for 10. Or 12. Or 15. If they can’t provide you with that many, ask “Why not?” Have they not had 10 customers? Or have they not made 10 customers happy? It’s one or the other.

Regardless, if they can’t give you 10 or 15 references, ask them for a list of customers -- a complete list, not just a few. Then – after you have the list, not before – ask them for the name of the person they worked for at each of those customers. There had to be someone responsible for the project. If they say that the person is no longer with the company, that’s OK – get their name anyway. Then look those people up on LinkedIn, Facebook, Twitter, or whatever. Reach out to them and ask about their experience with the company.

You only need a few minutes of conversation to understand how the project went, how the company delivered. Most people will be happy to spend 5 minutes with you on the phone – whether their experience was good or bad. Don’t stop with one – call and talk to at least 5. In about an hour, you can talk to 10 of their customers. You’ll be surprised at the things you hear and then you’ll know the truth – or at least you will have an idea of the quality of work you should expect.

If delivering the project on schedule and on budget is important to you, is it worth an hour of conversations? It is the only way you’ll know the truth. Everything else is just hype.

Calavista claims that over the past 2 decades, we have completed over 100 projects, and delivered well over 90% of them on time and on budget. If that’s true, then there should be scores of customers who are happy with our work, and who should be glad to speak to a prospect.

There are. So, we lead with that. The first thing we’ll do when a prospect indicates serious interest is to give them a list of recent customers who will speak to them about us. To keep relevance, we typically limit that list to customers we’ve worked with in the past 3 years. That list typically has about 20 names on it from maybe 15 companies.

This just seems obvious to us. If you are looking at a company that will not (or cannot) provide this kind of list, you probably really need to understand why.

When you’re looking to spend a large sum of money on a software project, you need to do your homework. It may take a few hours – maybe even a day or two. But in the scheme of things, what’s the cost of that compared to the cost of a failed project?

And finally – once you’ve settled on a company to work with, and you complete the project (or don’t, God forbid) – be open to someone reaching out to you on LinkedIn. Be honest and objective. Because someone may be betting their career on your candor.