Software engineers — have you ever honestly considered how we approach testing of fully integrated software systems? If we applied our approach to any other mature industry, we would be laughed out of existence.

Can you imagine a car manufacturer, in crash testing their vehicle, using a crash dummy with sensors for one particular body part? They set up the dummy with head sensors, crash a car head-on, then read the results to ensure compliance with regulatory standards. They then swap to torso sensors, crash a second car head-on, then read the results. Next, they refit the dummy with lower body sensors, and crash a third car head-on. Each test destroys a vehicle. Each vehicle could have been sold.

How long do you think that car manufacturer could remain in business? The material cost of destroying a sellable vehicle for each individual sensor reading would be staggering — and every one of those results could have been captured in a single test. The approach of using one sensor per test may be best practice for testing a component — an airbag, for example — pre-assembly. But it is the wrong approach when testing a post-assembly product like a vehicle. Yet this is commonly the approach used in testing fully integrated software systems, each test with verifications for a single, specific requirement. Is it any wonder that the cost of testing fully integrated systems is so high, and why there are so many delays in delivering finished product?

Now imagine applying to software development the approach used in mature industries. Given data in a unique scenario, perform a set of actions to a destination, with sensors — verifications — in place to confirm any number of result states, each responding with feedback regardless of how many indicate failures. A multitude of individual results can now be measured against any number of requirements, in a single test. No verification is dependent on the success of any other verification.

Take the analogy one step further. Every crash dummy is fitted with the same standard harness of sensors — regulatory compliance verified identically across every test, without exception, regardless of what additional sensors a specific scenario requires. The comprehensive baseline is not assembled test by test. It is built into the dummy itself.

The same principle applies to software. A single validation module — shared across every test that validates the same subject — ensures that baseline verification is comprehensive, consistent, and not dependent on the discipline of the individual engineer who wrote the test. The standard sensors are always in place. The scenario-specific verifications are added on top. Every test benefits from both, without the author having to think about either.

This is similar to setting up a crash dummy with head, torso, arm and leg sensors, submitting the vehicle to a head-on collision, then verifying the results of each and every sensor individually. Use the same fully-sensorized dummy in every crash scenario, and the number of unique test executions required is greatly reduced. Doing the same for each software scenario has the same result: a smaller, more manageable test library and greatly reduced number of unique test executions.

This principle does not change in the age of AI-assisted testing. AI generates tests based on what it understands to be important — which is almost always derived from the requirements. It will set up one sensor per test as reliably as any engineer trained to do the same. The approach does not become comprehensive simply because a tool generated it. Comprehensive validation requires deliberate design — the decision to instrument every sensor, not just the one the requirement described.

Reconciling our approach to validation of post-assembly software with that used by other mature industries makes comprehensive, resilient, effective and efficient testing of fully integrated software systems achievable. The result is higher-quality software, delivered in a sustainable release cycle, at greatly reduced cost.

Be reconciled!

Early in my career I was working as an automated test engineer on a web application that had been in production for years. The application included a feature for editing client company data — name, address, contact information, that sort of thing. Routine functionality. Any issues discovered were addressed, and regression testing passed.

The testing framework available to our team was exceptional — one of the best I have ever worked with. Among its capabilities was the ability to query for a suitable test record at runtime, apply logic to determine expected values dynamically, and then validate any number of fields at the end of the test. I set up a test for the company edit feature and validated every column of the database record. On the first execution, the test failed.

I looked at the data. The name was correct. The address was correct. The contact information was correct — everything the requirement cared about looked exactly right. I ran it again, got the same result, and began removing verifications one by one trying to isolate the failure. Then my eyes caught something. The ID value was different.

A query of the database told the rest of the story. Every time a company record was edited through the web application, a brand new row was being created in the database. The original record was not updated. It was orphaned. The company now existed twice — or more, depending on how many times it had been edited. This had been happening in production for quite some time.

When I reported the defect, it was deferred. The tests had never caught it. Production had never complained. The risk was judged acceptable. A couple of months later, production performance degraded badly enough to impact the business. The defect was addressed — along with several other behaviors of the same kind that the investigation surfaced.

The tests had been passing for years. What they were not doing was telling the truth about what the system was actually doing.

The previous approach to validation was the industry standard: verify what the requirement specifies, confirm the visible outcome, move on. By that measure, the company edit feature worked perfectly. The name changed. The address changed. The contact changed. Requirement met. Test passed. What no test had ever asked was: what happened to the rest of the record?

The answer, it turned out, was the most important answer in the test. And no one had ever asked the question.

This is what your tests are not telling you — and the rise of AI-assisted testing does not change it. AI generates tests based on what it understands to be important, which is almost always derived from the requirements. It would have verified the name, the address, the contact. It would not have thought to verify the ID. A developer writing AI-assisted unit tests for this feature faces the same risk — the AI reflects the developer’s intent, not the system’s complete behavior. What gets verified is what was considered. What was never considered remains invisible. Expertise — the kind that knows what a comprehensive validation must include and ensures it is there — is not something AI supplies. It is something the engineer brings to the AI.

A test suite that verifies the requirement and nothing more is producing confidence. It is not producing coverage. And confidence without coverage is not a passing test. It is a deferred defect — one you will find eventually, in the environment where finding it costs the most.

Comprehensive validation means verifying the complete state of the entity under test — not just the fields that changed, but every field that should not have changed. Not just the visible outcome, but the underlying record that outcome is built on. When you validate the entire entity, you are asking the system to account for itself. That is where the lies get exposed.

Being reconciled to a comprehensive validation approach in all testing — application, services, or database — requires a bit more effort up front. But the risks mitigated and cost of correction saved is worth the attention to detail, and provides trustworthy confidence for the business and clients.

Be reconciled!

Nana recently underwent surgery for a large abdominal hernia. Afterward, she shared a picture of the operating room. Hanging from the ceiling was a device best described as remarkable — several articulating arms, each tipped with a precision instrument, positioned over the operating table. There was no large incision. No surgeon reaching directly into her body. Instead, several small punctures, through which each instrument was inserted and guided as needed. The operation was performed entirely through the machine.

I looked at that picture and thought: what an extraordinary tool. And then, almost immediately: imagine handing it to a first-year resident who has never held a scalpel, never lost a patient, and never developed the judgment that only comes from years of training under surgeons who demanded more than competence.

Surgery in the early 1900s was educated guesswork performed with blunt instruments, unreliable anesthesia, and almost no understanding of infection. Patients who survived the procedure frequently did not survive the recovery.

Every advancement that followed — antiseptics, precision instruments, imaging, anesthesia, and eventually robotic-assisted surgery — did not lower the skill requirement. It raised the consequences of lacking it.

The scalpel did not make surgeons interchangeable. It made the gap between a trained surgeon and an untrained hand more dangerous. The tool never replaced judgment. It amplified it — in both directions.

Agentic AI is the articulating arm hanging from the ceiling of software engineering. It is an extraordinary advancement.

In the wrong hands, it produces results that are clean, confident, and completely wrong — at a speed and scale previously impossible.

To business and technical leaders: the assumption that agentic AI produces superior results, or results that are “good enough,” regardless of who is directing it, is the assumption that the robotic surgical system operates itself.

It does not.

Someone trained, experienced, and principled is still standing at the controls. The tool amplifies what that person brings. Nothing more.

If what they bring is shallow, the tool amplifies that too — faster, and at greater cost than before.

The businesses most at risk are not the ones that fail to adopt agentic AI. They are the ones that adopt it without understanding what they are actually putting in charge of their software.

To development engineers: the question is not whether agentic AI will change your career. It will.

The question is what you bring to it.

The engineers who will thrive are not the ones who learned to prompt. They are the ones who understand sound principles and best practices deeply enough to recognize when the tool is right, when it is wrong, and when it is confidently producing something that will cost the business dearly six months from now.

If you cannot reliably make that distinction, the tool is not making you more effective. It is making you more dangerous.

The tool will not tell you. It does not know. It cannot know.

Only you can know — and only if you have done the work to develop the judgment that makes that knowledge possible.

Nana’s surgeon did not walk into that operating room because a remarkable machine was hanging from the ceiling.

The machine was there because a remarkable surgeon was walking into the room.

The advancement of the tool has never, in any industry, reduced the value of the skilled practitioner.

It has always increased it. And made the cost of the unskilled harder to hide.

Reconciling yourself to the value of proven experience — engineers who have built sound principles and best practices through years of deliberate practice — is what keeps businesses from accelerating headlong into disaster, and what keeps engineers relevant in the age of the very tools threatening to replace them.

The businesses that thrive will be the ones who identify and retain true Software Crafters. The engineers who thrive will be the ones who have committed to becoming one.

Be reconciled!

Years ago, while building a business, a colleague recommended a book. I don’t recall the title. What I do recall is a story from its pages — one that stayed with me.

In that story, a young man eager for success is challenged by his mentor. The mentor tells him, plainly: achieving great success will not come from the things you know. Nor will it come from the things you know you don’t know. It is the things you don’t know you don’t know which will hold you back, or enable your success. Discovering and mastering those — that is where success is secured.

Agile Open Northwest draws people at the cutting edge of Agile and AI-assisted software engineering — practitioners actively shaping how software is built and delivered. I go there specifically knowing I will hear and see things I would never think of on my own. I go to share my perspectives and have them challenged or confirmed. I leave every time with new insight and practical vision that makes me better as a Software Crafter.

A graduate just beginning their career opened a session by sharing all the things they knew they didn’t know. They were looking for guidance from experienced peers in the industry.

When my turn came, I referred to that story from the book.

The most experienced engineers in the room nodded, genuinely. The comments that followed confirmed what the nods already said: the engineers who go furthest in this craft are, without exception, the ones who stayed open. Not the ones who accumulated the most answers. The ones who remained most willing to question the answers they already had.

Have you even dismissed something? I know I have. A principle recommended by someone whose work you respect. A practice you half-heartedly attempted. A pattern that keeps appearing in the work of engineers you admire.

You may be right about it. Some things deserve to be dismissed. But here is the question you cannot answer honestly without doing the work: how would you know? Did you consider it thoughtfully? Did you test it yourself?

Seniority makes this worse, not better. Every year of experience adds another layer of confidence between you and the risk of being wrong. By the time you reach senior engineer, staff engineer, or architect — the insulation is considerable. You have earned your opinions. You have the battle scars to prove them. And that is precisely when the unknown unknowns are most dangerous, because you have the most authority to act on them and the least incentive to question them.

The engineer who stopped being challenged is the engineer who stopped growing. The tragedy is they are usually the last to know it.

A sermon I heard as a child has never left me. The preacher said: you can learn from your own thirty years of wisdom — or another’s thirty years of wisdom can give you a head start.

Whose thirty years are you learning from?

Reconciling yourself to continuously discovering what you don’t know you don’t know keeps you fresh. It strengthens sound principles, refines your practices, and sets an example for those who follow.

Be reconciled!