The Obama administration announced Friday that it was putting a private firm in charge of fixing its faulty health insurance Web site and set the end of November as a target date for working out all the bugs, the first indication of how long repairs may take.
I am trying to figure out the timeline. Suppose that we work backwards:
–two weeks of beta testing takes us back to November 16
–three weeks of user testing takes us back to this week
–one week of integration testing takes us back to October 20
–one week of unit testing takes us back to October 13
–one week to code fixes takes us back to October 6
–two weeks to identify problems and specify fixes takes us back to late September
What I infer is that they have had a working version of the system, not ready for deployment but ready for testing, for a few weeks now.
Otherwise, I do not think I would want to be the guy who confidently announced that the system will be functioning smoothly by the end of November.
You can probably toss out most the unit testing in this emergency. For something like this, the individual methods are probably pretty simple and straightforward anyway. By most reports, the problem is accessing all the different external systems (did I hear something like 40?) like credit checks and then coordinating that. One fails, and the whole thing breaks.
That is probably simply a design error, but it is what it is at this point.
As far as unit testing goes, it not only isn’t likely the main issue, it may be very difficult to do usefully without a lot of work. If I need to run a credit check, and then pull insurance info for the region and age group, how do I unit test that? What good is it without credit and insurance data? And if I supply canned data (and so cut out the complicated part of the code that asks for and retrieves it) have I really tested much of anything?
If you had the time, you probably would still have to use canned data (or create it on the fly) but you would write replacements for the credit and insurance sources that /looked/ like the real thing to the code, but weren’t, so you could more realistically test even during unit testing. But that takes time.
There are two sides to this coin:
1. It’s a website. OK, we have to talk to 40 different APIs, but this is not rocket science, even if it is a massive integration challenge. But it’s one that software engineering provides plenty of successful cases and at least one model for
2. Even getting A to talk to B is a challenge. A is capability- and security-deficient, as is B. By the requirements, they are supposed to talk to each other, and when things luck out, they can. But if we ever stray even slightly from the happy path, A and B cannot talk, and sometimes we cannot even detect when this is the case.
(1) is true for a great deal of (successful) private software firms, though, depending on how you measure, it is true for (well) less than half of such.
(2) is true for everyone else.
I think what time will bear out is that this is not a technical exercise, but a political one. Five weeks is enough time not to fix the system, but to mount a media counteroffensive. This administration is competent at very little, but it excels at that.
The system is so apparently broken that it has nowhere to go but up, and given both how many well-credentialed, accomplished tech people are in the bag for this president (and are available to buttress talking points), and how generally friendly legacy media his to him five weeks may be enough time to flip the narrative. It would look something like this: “they haven’t fixed everything, but look how much it’s improved. Republicans should stop being obstructionists. This is the law, and we have to try our best to make it work.”
I’m not saying this will work out for them. A lot can still go wrong that’s out of their control and blow up in their face. However, this is their best bet, and it may not be a bad one.
As it already works in other states, it may be faster to adopt than fix, not that this will be done, but it does demonstrate it is possible.
I think it’s more likely “the end of November” was a politically chosen date rather than an actual project planning date. Not that those are very reliable either.
Attention will be a lot less pointed on this six weeks from now. Maybe some guy will get thrown under the bus, but I bet even that won’t happen. And who knows, maybe the horse will learn to fly.
My bet is that we will see a system that throws up no error messages at all, just plows through the process and dumps the raw info on some hapless interns who will have to sort through it and fill out a paper application. Call it the Mechanical Turk solution.
The sooner that people see what this abortion of a health plan will actually cost them, the better….unfortunately they won’t have a reasonably functioning system for at least 3 months (spin notwithstanding).
Speaking as a web application UX researcher, I don’t think they need or will do 3 weeks of user testing. These days that kind of experience testing is done in rough prototypes, and the changes to this task flow are either just needed (don’t care if it’s hard) or should be tested during the coding process. Ditto with beta testing (they don’t need to test this product concept with the market).
From the state of it, I assume they’ll just be pushing changes directly live. Long timelines of testing are designed to ensure that a working system doesn’t become broken.
With such a broken system, they could iterate very quickly by simply pushing out code and using the users as testers. I think using such a strategy is the only way they could even _think_ it would be possible to meet the deadline they’ve set.