code software development blue

Technical Debt Patterns: Bilingual Required

Written By: Steve Zagieboylo, Senior Architect at Calavista


This is the third part in the series, "Technical Debt Patterns."

This problem refers to the developers speaking a different language about the problem domain than anyone else, including the users. It is a lack of what Eric Evans refers to as “Ubiquitous Language,” the term Eric Evans uses in his book Domain Driven Design. He explains (correctly) that the developers should derive their entity names from the subject matter experts (SMEs), so that they are using the same language when they speak to them. This vastly simplifies the conversations with the SMEs and other users, with marketing, with documentation, with your own user interface!

Sadly, in my 35+ years writing software, I have never once been on a project where these problems didn’t arise. Names change, specs change, marketing comes down with a new dictum, there’s always something. What we called a Cube yesterday is now called a box; Filter is now Selector, and so on. So now that it is there, what are you going to do about it?

Step 1: Identify the terms that have multiple names.

Is it just one major object, or is it dozens of little objects strewn throughout? If you haven’t done so already, make a dictionary of all the domain entities, including both (or all!) their names in developer-speak, marketing-speak, and SME-speak. The beauty of this step is that it is going to be used no matter how you end up deciding to tackle the problem. Put your glossary somewhere that everyone knows where to find it, where you point new developers as one of the first things they see. You should be using, for developer documentation, some sort of wiki that ties well into your issue tracking system. Put the glossary front and center, and then link to it frequently.

Step 2: Identify the scope of the problem.

Usually the problem is more than just the class names that refer to these dual-named entities. There are argument names, member names, and local names that are just a lowercase version of the entity name; there are usually multiple occurrences of each named entity, with data transfer objects (DTOs), service classes, unit test classes, and more; there are database tables -- changing those names means a migration; and there may be a public API that would be a huge pain to change.

If you want to be able to follow file history, then good cooperation from your source control system is a must.

Another consideration is your source control system and its ability to track filename changes which is required in most languages if the class name changes. If you want to be able to follow file history, then good cooperation from your source control system is a must.

The last element of scope is the most important: Is your product one that you sell copies of the software to customers or one where the only commercial version is a single instance that you host and sell access to? In the former case, you probably maintain different versions, with point and sub-point releases. If your new development is on version 7.x, but you still support 6.3.x and 5.8.x, then your transition period where developers will have to develop in both languages becomes a huge nightmare. If that’s the case, I would suggest that just good documentation and socialization of the problem is going to be good enough, because actually fixing it is impossible -- those old names will still be in your older versions of the code.

Step 3: Change Ahead!

Plan when it is going to happen, possibly just after a major release when everyone is going to be taking a breath.

If you’ve decided that you do want to fix the problem, the biggest challenge will be timing and branches. Plan when it is going to happen, possibly just after a major release when everyone is going to be taking a breath. Make sure the developers all have their code checked in and merged to the main branch, because these changes are sure to cause merge conflicts with everything. Use the refactoring tools in the IDE just to change the names of all the classes, and don’t bother with the secondary uses, the member names, argument names, and local variables, unless the refactoring tools will do this automatically. Now get it all checked in and run your unit tests. If this goes smoothly, congratulate yourself because you’ve done better than I ever have.

The next step, if you’re going to do it, is to take the database tables. Because this needs a migration it will take longer. If you are changing column names within the tables, it will take longer, still. Fortunately, this should be reasonably isolated, so this can be drawn out and you can let the developers start making branches again. Honestly, in the times I have attempted to move to the Ubiquitous Language, I’ve never changed the database table names. I just documented in the few classes that touch the database directly, and kept my glossary up to date.

Summary

Of all the Technical Debt Patterns, this is the one that I would most likely recommend that you don’t try to fix it, just mitigate it with good documentation. First, it is a minor inconvenience but not really crippling unless you really have a lot of churn on your development staff. It is more of a pain and an embarrassment when you are introducing new developers to the team, and once they are used to it, it doesn’t cost that much. The other consideration is how much of a moving target it is that you are aiming at. If the product is a new, world-changing paradigm (and what isn’t?) then you can count on marketing changing all the names again in another quarter. They might even change back to what you used to have. So, unless you have a pretty stable product, I would recommend just to live with it.


software problem

What's Your Problem?

Written By: Daniel Kulvicki, Solutions Director at Calavista


I know. The title is a little in your face, but I have always asked this question on each of my projects. Instead of focusing on a solution, focus on the problem that you are solving and keep that focus through the entire project lifecycle. Most projects spend 90% of their time on the solution and only 10% on the actual problem space. We should be putting equal weight to the problem space and the solution space. Let us look at the differences of the problem space versus the solution space.

Problem Solution Space
Figure 1- Problem Space vs. Solution Space

Problem Space

"A market is a set of related customer needs, which rests squarely in problem space or you can say “problems” define market, not “solutions”. A market is not tied to any specific solutions that meet market needs. It is a broader space. There is no product or design that exists in problem space." (Ref 1)

What I have found in the past, is that when you ask customers what problem you need to solve; they come back with a solution. Solutions are great for customer feedback to ensure your problem is being solved. However, you must first identify your core problem. My favorite question is “What are you trying to solve?” and it usually takes about 3-4 iterations of answers to that question to get close to the real problem space. Eventually, a “voila!” happens and it just seems easy after that. Until you start working towards the solution.

Solution Space

"If I speak of solution space, any product or the product design — such as mock-ups, wire-frame, prototype, depends on and is built upon problem space, but is in solution space."

Everyone loves the solution space. It is hands on, validates, and is something new! How quickly have meetings turned to whiteboard sketches and tons of ideas that gets everyone excited? All of this excitement is great and is needed when working in the solution space. But, don’t forget the problem you are solving. Everyone might start off with the right core problem to solve, but as the solution progresses; that problem may change, and no one notices it as they drive to their “awesome” solution. That is why you need to have a certain discipline as a project is executing to step back into the problem space in order to validate the solution.

Product Discovery

Product Discovery is essentially an Agile approach to manage your problem space. Product Owners on an Agile project tend to manage the backlog, but not the problem space. Hence, the need to build in Product Discovery as part of backlog grooming. Try it sometime in a future project. With the right discovery process, your projects will start to come in on time and budget like they do at Calavista.


code software development blue

Technical Debt Patterns: Abstraction Overlooked

Written By: Steve Zagieboylo, Senior Architect at Calavista


This is the second part in the series, "Technical Debt Patterns."

The Abstraction Overlooked pattern is where you have some concrete class that has a “type” of some sort, and you find yourself basing some of its behavior on its type, either through if-then-else or switch statements. Often this occurs because the original programmer didn’t realize that there would be more than one type of this object, either because requirements have changed, he just didn’t think about it, or the entire project has grown larger than anyone ever imagined.

Example

Say you’re writing a tool for stock brokers to track the assets of their clients. One aspect of a client is his cash account, which your transactionally-sensitive code draws from on buys and adds to on sells. Your code includes a test on a withdrawal that you aren’t drawing more than is in the account, and rejects the transaction if it is. Your code works great, and the stockbroker is happy. Then he comes back with a new feature, Premium Customers. One aspect of a Premium Customer is that he is allowed to overdraw the cash account by some fixed amount, because we trust him. You edit your withdrawal code to check if this is a Premium Customer, and it now allows the transaction if it is within his overdraw limit. This is literally less than a line of code – just an extra test in the if statement. Then another type of customer is allowed to attach his bank savings account to his cash account, and you’re supposed to change the code to draw from that on an overdraw. And there will be more – you should have made the account an abstraction.

The Cost

The biggest aspect of the cost is not ongoing development, but ongoing quality and testing.
The cost for this pattern is a subtle one for many developers, because the biggest aspect of the cost is not ongoing development, but ongoing quality and testing. Every time there’s a new way a special type of account behaves, the developer is now editing code that is used by all accounts, with some possibility of introducing a bug to one of them. Sure, you have unit tests, and I’d even bet that, for this example, they’ve got enough coverage that you wouldn’t miss something. But can you say that when it is something less carefully tested than monetary transactions? As the cases get more complex and there are three or four places where there’s a behavior change based on the type, are you really sure you haven’t missed one?

The Fix

The fix for this problem is usually well-contained, and, if you do have those unit tests, fairly easy to confirm is correct.

1. Scope the fix first by declaring the class to be abstract and making a single concrete version of the class that has literally nothing in it. Then rebuild and see how many errors pop up. They will be all the places you’re calling the constructor of your now-abstract class. Ideally, your only errors will be in the one place that owns this object and in the unit tests which are specific to it. If there are a lot more, you might want to take a step back and ask yourself if the missing abstraction is a level higher (or at both levels). If you’re satisfied with the number of errors, make a factory to create the correct instance of the class, thinking about what concrete classes you’re going to end up with and what the factory will need to know to create the right one. (I’m fond of making the factory a static method of the abstract class or interface, but there are people who hate this approach and always want a separate class with this responsibility. I’m not quite willing to say they are wrong, but I’m also not going to change the way I do it. It’s a trade-off of clarity vs. simplicity, and I usually go for the latter.)

2. At this point you might decide you want an interface rather than an abstract class; that’s almost never a bad idea, even if there is a single abstract class that all the concrete classes extend. Extracting out the interface forces you really to think about what functionality is fundamental to the concept and what is implementation detail. I like making the abstract class, though, even with the interface. If you found yourself here, there’s probably a lot of functionality still that is common to all the concrete classes, and that code can stay in the abstract base. Let the highest level thing, the interface or the abstract base, keep the name by which the rest of the world knows this concept, and make the concrete classes have new names that describe what they are.

3. Then you have a pretty straightforward process of identifying what the concrete classes should be and moving the code out of the if-then-else or switch blocks into the appropriate location. Often you’ll find that the abstract methods which the concrete classes are overriding are protected – they are not the methods being exposed through the public interface, just a small part of them. There’s nothing wrong with this approach, but don’t overdo it, either. The abstract methods should make sense according to what they accomplish, not according to the details of how they get accomplished in the special cases you care about.

The abstract methods should make sense according to what they accomplish.

4. Once you’re a little way into the process, try to imagine the most extreme concrete class you might ever be expected to make. Ask yourself first whether or not you’ve passed all the information to the factory that you would need in order to know to create this. Then ask yourself if the abstract methods you’ve created would support the special cases that this class represents. Don’t add new abstract methods that exist only to support this imaginary case, but possibly rethink the ones you have, and make sure that the right information is available for them. It’s hard to quantify how, exactly, this experiment will inform the process, but I’ve almost never come out of it without some new bit of data.

5. The last step is to revisit your unit tests. If those tests were creating the original object directly, they need to change to call the factory. Take an inventory of all the unit tests to be sure all your concrete classes are being thoroughly tested, even all the methods that are not specialized in any way. You might want to create a new class AccountTester (for our example) that tests all the methods of any account, with parameters passed in for the expected results. Then your individual tests will consist of calling the factory, creating a helper with all the right parameters, then calling it. Think how happy you’ll be next year when you find yourself writing yet another concrete instance of this class.


The Fundamentals of Continuous Software Design

Written & Presented By: Jeremy Miller, Senior Architect at Calavista


CouchCon Live is a single day virtual technical conference focused on connecting and growing our vibrant tech community! We're bringing quality technical content to a virtually connected community. Whether you're a new developer or have been working in the industry for years, we know you'll meet other great people and learn something new!

 

https://www.youtube.com/watch?v=v9icxKMJ9PA&t=108s

 

The Fundamentals of Continuous Software Design

Scrum completely dominates the landscape of Agile Software Development these days, but it’s always been a little focused on project management and a little bit light on specific software engineering practices and guidance. It’s unfortunately easy for Scrum projects to drown in technical debt while the team tries to keep up with the drumbeat of constant sprint deliverables.

Harking back to earlier Extreme Programming ideas, let’s review some of the fundamentals of continuous software design and how we can keep our codebases from turning into a giant mess as the system evolves. With some concrete examples, let’s dive in and learn more about reversibility, the importance of testability in software design, the ‘last responsible moment’, striving for simplicity now (YAGNI) without cutting off a pathway to more complex behavior later.


Does Your Software Team Deliver as Well As Amazon Delivers Packages?

Written By: Andy Fruhling, Chief Operating Officer at Calavista


Where is my Amazon Package?

As we all learn to adjust to the new normal, many of us are becoming even more dependent on software. Software is responsible for bringing us our news, our entertainment, our meetings, our social interactions, our groceries, and much more. Earlier this year, my mother-in-law ordered groceries online for the first time. In her first experience, she quickly realized online ordering was much easier than going to the store and very reliable. She started with a small order to see how it worked – something like:

  • 1 dozen eggs
  • 1 half gallon of milk
  • 1 box of granola cereal
  • 2 boxes of Angel Hair pasta
  • 1 small bag of baby peeled carrots
  • 1 loaf of French bread
  • 1 package of fresh mushrooms
  • 2 pounds of boneless chicken

She was quoted $43.98 for the order. This price included free same day delivery. As part of the process, she was notified that “Some items may be out of stock due to increased demand. By allowing substitutions, you help us fulfill more of your items.” For each item ordered, she was able to provide some guidance for how to proceed if that item was not available. Over the next few hours, she received three text messages:

  • Message 1: Confirming her order was received,
  • Message 2: Providing her with the expected delivery time for later in the day, and
  • Message 3: Notifying her that the order was delivered successfully.

Her groceries were delivered to her house later that evening with everything she ordered for the price she was quoted. With the three text messages, she knew what to expect and when. There could have been substitutions for some of the products, but in this case there weren’t. Now she orders online on a regular basis and each time the groceries she orders are delivered on schedule at the price she was quoted.

Does Your Software Development Team Deliver as Well as Amazon?

In the example above, my mother-in-law requested some grocery items to be delivered by an agreed upon time and at an agreed upon price. She continues to use the service because it met her expectations. If Amazon (or other delivery service) entirely failed to deliver your order over 20% of the time and delivered only part of your order and/or was late on your order over 45% of the time, would you keep ordering? Most people would not – yet we continue to run software projects with success rates even worse than this. The first Standish Group Chaos Report (published in 1994) and every Standish report published since has shown software development projects are delivered on time and on budget less than 40% of the time each year. Obviously, there is a lot of room for improvement here!

Without going too deep into Agile software development best practices, there are a few comparisons between an online delivery process and a software delivery process that would lead to more successful software delivery projects. The online delivery process above

1. Created a clearly defined and documented list of requirements upfront: Admittedly, selecting products to order on Amazon is a very simplified requirements definition process and some may argue this is an unfair comparison to software development projects. At the same time, many teams cite unclear requirements as a key reason for their project failure. No requirements mean failure since there is no finish line. You would not place an order at Amazon without identifying what you want by when. At the same time, you would not detail your list to say “Whole Grain Rolled Oats, Whole Grain Rolled Wheat, Brown Sugar, Canola Oil, Dried Cranberries, Almonds, Dehydrated Apples, Inulin, Whey, Sugar, Nonfat Dry Milk, Glycerine, Whey Protein Concentrate, Natural Flavor, Honey, Sunflower Oil, Natural Mixed Tocopherols Added to Preserve Freshness” to say you wanted a box of Quaker Simply Granola cereal. Define your project objectives (or OKRs) and high-level requirements to start. Then include a plan to iterate with the product stakeholders to further refine requirements during the project. This is similar to substitution options from Amazon and leads into the next point.

2. Outlined guidelines for what to do if any requirements could not be met: When I shop with Amazon/Whole Foods, I can define three options for substitutions: Best available (based on Amazon’s logic), Don’t substitute, and identify the item you want as an alternate. There are many ways great teams handle “substitution” – too much to cover here. At a very high-level, there are three guidelines to start this step: 1) Define OKRs for the project so that everyone understands and agrees to the “Why.” 2) Categorized your requirements so that everyone knows which requirements can be dropped if time runs short and which cannot. Decide this at the start of the project. 3) Build iteratively with regular reviews with product stakeholders for feedback and make necessary adjustments along the way.

3. Provided regular and accurate communication on progress: This is one of my favorites.  When you order from Amazon, they send you multiple messages to let you know your order is progressing through each step correctly.  Not only does this build trust between you and Amazon, it also allows you to alter plans (when needed) to align with the delivery schedule.  The same is true between a development team delivering projects – it builds trust and allows for adjustments when needed.

As a society, we are becoming more and more dependent on software for our personal lives and our businesses. Whether they realize it or not, most companies rely on software for their very survival. If your software development teams are failing to deliver, how can your company succeed? These three simple steps: clearly defining requirements upfront, outlining guidelines for how to handle unmet requirements before they happen, and ensuring you have regular and accurate communications throughout the process can ensure you deliver software more like Amazon delivers packages.


code software development blue

Technical Debt Patterns: Series Introduction

Written By: Steve Zagieboylo, Senior Architect at Calavista


Whether your software project has been under development for 2 months or 2 decades, you have some technical debt. These are grungy bits of code that you know could be better. They were expedient; your target changed; you had a less-than-stellar coder on the team; or they arose for one of a hundred other reasons. But those ugly warts are sitting in your code base and you either have to allocate some development time to fixing them or you will continue to pay the “interest” on the debt as you continuously work around them, but there’s a cost either way.

Software challenges can frequently be broken down into known patterns with well-understood and straightforward solutions.

Patterns

Software Design Patterns is a popular and very successful concept in software development. The concept was originally suggested by Christopher Alexander in 1977, who was applying to software development concepts he learned as an architect. The gist is software challenges can frequently be broken down into known patterns with well-understood and straightforward solutions. There are hundreds of white papers and books on this subject, and the approach has been proven to be very successful.

This same approach can be applied to technical debt: Most examples of technical debt can be described as one or two of a small number of patterns. By analyzing these patterns, we can hope not only to understand the cost and best approaches to fixing them, but also to understand the “interest,” the cost of living with these issues when the cost of fixing them looks prohibitive. Often the ongoing cost is more than it appears at first glance, and understanding this cost can help in prioritizing the issues. In addition, the fix is frequently easier than appears at first blush, thanks to the refactoring tools in modern development environments. But sometimes the right call is just to live with it, perhaps with a small amelioration. Either way, it is still worth the exercise of recognizing what type of debt you are looking at, and really understanding its cost.

Our List of Technical Debt Patterns

The patterns below will be covered in this series (though more might be added). Each pattern will be described in more detail; then its costs will be discussed, both the costs to fix and the interest cost of just living with the problem; finally steps will be suggested to correct or ameliorate the problem. Often, fixing a problem that you’ve lived with for months or years gives the team such a sigh of relief that they quickly recover the time spent fixing it.

  • Abstraction Overlooked
  • Bilingual Required
  • Tree Rings
  • Overburdened Class
  • Copied and Pasted Code
  • Too Much Overloading
  • One Service to Rule Them All

Common Thread: Stop Digging!

There is one common thread across all these different problems and solutions: Once you realize that you are standing in a hole, the first step is to stop digging. You may look at the problem and think that fixing it cleanly is overwhelming, so you just continue with the paradigm you have, flawed as it is. This only adds to the problem, making any solution seem even more overwhelming later.

The approach we are advocating is to figure out what the right answer looks like, and then implement any new features using the correct approach. Allow the old and new approaches to live side-by-side for a while, and convert the old code to the new code gradually. When there is some change required in any bit of the old code, then convert that one to the new approach, as part of the change. Perhaps convert others as there is time. Always, make sure that there are good unit tests, so you can be sure that your conversions have not broken anything.

For some of the patterns, this approach works very well; for others, not so much. This will also be addressed in the upcoming blogs in this series.