Update: Add to all the big screw ups listed below this doozy:

HHS had forty-two months to build this system after the passage of the Affordable Care Act in March 2010 to hit the October 1 deadline.  Why did they wait more than forty months to begin building the payment system that is absolutely critical to the system’s potential operation? And what can we expect to see from a massive, $12 million system that has to be delivered in less than four months, which to this day is still not complete?

… Now we find out that 40% of the system slipped through the cracks until well past the last moment, and CMS was forced to do a no-bid award to catch up.

The incompetence exposed by this disaster almost escapes the ability of adjectives to adequately describe it.

CMS/HHS did not realize for 3 and 1/2 YEARS they were not going to get done on time? As I illustrate below, this whole government take over of America’s health care services was amateur hour on steroids. – end update

ObamaCare was doomed to extended failure years ago. What we see now is just the initial shudders of a government program so damaged, so poorly thought out, it cannot succeed. It will produce massive insurance loss come January 1, 2014 because the ‘system’ is so broke it cannot recover.

The evidence for this conclusion/prediction is all around, especially to the trained eye of people who have made a career out of federal contracting on large, complex, computer-based programs. I remind people that not just anyone in the government or industry can stand up a large system with lots of interfaces and lots of human users. Private industry is better at popular websites, but websites are not all that are needed for large systems. In fact, in the world of large federal programs, the website is usually an abstracted window through which filtered information can be exposed and limited requests can be made.

The systems behind many websites are enormous, and require thousands of engineers and technical managers (as opposed to financial managers, policy managers, communications managers, etc) to make work. While a diverse management team is required on all programs, large computer-based programs require a strong technical manager and team.

This skill set is rare even today, both in and out of government, 40 years into the computer age. So when the country embark son these challenges, we HAVE to have the right team in place. ObamaCare was an enormous challenge in both the  computer system world and the website world. So it needed special skills to pull off.

What is astounding is how the Democrats’ signature government program of the last half century was so badly botched, and how it all starts with inexperienced leadership. From the White House to CMS, the organization in HHS designated to lead the development and deployment of ObamaCare, for some reason getting the right team to run this was a non-starter.

I predicted back in one of my first posts on this debacle the main culprit was inexperienced bureaucrats pretending to know how to run a large computer-based program:

If you try and jump from managing some health care technology to deploying a system like ObamaCare – you will fall on your face.

NASA, DoD and DoT have the necessary government infrastructure to accurately define, procure and monitor something like ObamaCare. They have this capability because the built up a contractor and civil servant base able to take on the challenge over many decades. HHS and other government entities usually do not.

This suspicion was just confirmed by the New York Times:

… over the past three years five different lower-level managers held posts overseeing the development of HealthCare.gov, none of whom had the kind of authority to reach across the administration to ensure the project stayed on schedule.

As a result, the president’s signature initiative was effectively left under the day-to-day management of Henry Chao, a 19-year veteran of the Medicare agency with little clout and little formal background in computer science.

Emphasis mine. This is like asking a 20-year veteran car service mechanic to perform complicated surgery. Just ridiculous. And criminal in the private sector. We in the private sector would be barred from contracting if we put someone in a lead position without the experience and expertise required.  Sadly, the government does it more and more these days as it takes on more and more large projects without increasing its pool of expertise.

Yet it was not just an incompetent leader at CMS that doomed ObamaCare. The program structure had CMS take on the Systems Engineering/Architecture role, usually left to the rare and skilled contracting team capable of dealing with large computer-based system development:

But the Medicare agency reserved the role of general contractor, or system integrator, for itself, even though it lacked the necessary in-house software engineering resources to handle such a task.

It is no surprise ObamaCare failed then, is it?

Chao and the White House (also completely inept in these things) ran ObamaCare into the ground:

Mr. Chao had to consult with senior department officials and the White House, and was unable to make many decisions on his own. “Nothing was decided without a conversation there,” said one agency official involved in the project, referring to the constant White House demands for oversight.

Step back and think about this for a moment. This was the administrations most important achievement, the fantasy if liberals for decades. And because of this high import, these morons decided to run the development themselves! They really, really cared about this program. So imagine what they will do about you or your family’s health issues – given you are just a statistic in a see of millions?

Fatal decisions came fast and furious from the beginning, as the inept wandered around making snap, stupid decisions based on their ignorance on these matters. I posted recently on some of the more obvious missteps.

First up, the technically naive leaders at CMS cut corners on the required technical documentation. Even experienced technical managers have to tread with care when “tailoring” out required steps. One key item to go away was the data architecture model, which maps how data at IRS and Social Security maps to application and enrollment data inside ObamaCare, and then maps to insurance plan data, etc.

Worse than that, as I noted in my previous post, HHS/CMS decided that this huge, complex effort would best be served by skimping on the required technical design documents – a process called “tailoring”:

First and foremost you need to define all interfaces and get them on a coherent schedule.  Then you have to model all state and federal sources of data (i.e., define how each one labels common information and create a Rosetta stone to relate each existing label in those data sources to a common definition, like “first name”) and then you need to model all data products (each state and each insurer in the state has unique definitions for data as well).

…

And of course, data modeling and translation is one of those things HHS/CMS tailored out to save time. [In the above diagram] R = Recommended, which in tight programs means it won’t get done.

In an information intensive system like Healthcare.gov, the data model is one of the 5 main pillars you cannot succeed without.  The other 4 are:

• The external interface control documents
• The system requirements
• The system architecture
• The system Concept of Operations

The last one is very critical because it provides the human context for the others, which are simply specifications of what the hardware and software are to do. The Concept of Operations gets the people on the same page, and fills in gaps or eliminates the fog found in the specifications. When you deal with diverse organizations each working under diverse contracts, the glue is the Concept of Operations. Without it the ‘team’ will drift and wander.

It also is used to define your testing – so that you test the system the way it will be used. This is important: you don’t want to forget to test something that will produce problems when you “Go Live”.

Think brakes on a car for a good analogy.

But the White House, ignorant of what it means to develop a large computer-based system, apparently panicked when they saw the complexity they were taking on:

According to two former officials,” they write, “CMS staff members struggled at ‘multiple meetings’ during the spring of 2011 to persuade White House officials for permission to publish diagrams known as ‘concepts of operation,’ which they believed were necessary to show states what a federal exchange would look like. The two officials said the White House was reluctant because the diagrams were complex, and they feared that the Republicans might reprise a tactic from the 1990s of then Sen. Bob Dole (R., Kan.), who mockingly brandished intricate charts created by a task force led by first lady Hillary Clinton.”

“In the end…the White House quashed the diagrams,” which prevented states from learning about how the administration wanted the insurance exchanges to be designed.

Talk about hiding your head in the sand!

So two of the critical technical elements of any large, computer-based program were gutted. Failure was becoming the only possible outcome.

But wait, that is not all that happened. Going back to the New York Times article, the result was obvious – confusion. Constant flux in the system plan, which led to lots of talking but no progress:

A pattern of ever-shifting requirements persisted throughout the project, including the administration’s decision late last year to try to redesign the site’s appearance and content to make it more informative to consumers, according to many specialists involved. The administration also decided to reconfigure it as a national site, instead of one where each state had its own front page, after many states decided not to open their own exchanges.

IDEO, a consulting firm based in Palo Alto, Calif., that had done some early design prototypes, was enlisted to help revamp the site’s front end. The team at IDEO ended up as frustrated with the Medicare agency as other vendors.

“It was monstrous, a monstrous impact,” said one specialist about the amount of code that had to be rewritten because of the redesign and other similar changes. Administration officials strongly dispute that, saying the impact was minimal.

Administration officials will never admit they had no idea what they were doing. But we can see clearly from the evidence and the result, they had no clue. Around April 2013 someone finally got the bright idea to bring in  experienced people and ask for an assessment.

The results were brutal. This is one of the summary slides regarding how far afield Healthcare.gov was [click to enlarge]:

Right side is normal, left side was the CMS run disaster. Unstable requirements and conflicting benchmarks for completion (what they refer to as “success”), combined with insufficient time to test pretty much meant everyone with an ounce of experience knew in April Obamacare was going to “auger in” on “Go Live” day.  With this kind of assessment, the chance for success is basically nil.

But the naive CMS technical leadership probably did not even understand what was being told to them. And what was being recommended was tough news. The list if issues was basically insurmountable:

The mind boggles at this assessment. So many problems. Clearly the result of people way in over their heads. Each line a damning statement. I could do a paper on each line and on what went wrong and what it would take to fix.  And we are talking years here to fix – not days.

I could also spend hours going over the September 2013 emails and how CMS and the White House had not learned a damn thing from the April warnings. In perusing these emails I discovered tests being defined on the fly, hot patching going on like crazy without any design thought, and just all around chaos and panic as performance testing had to move off the production system. There was no “factory” test capability that could come close to mimicking the real world. These emails are just another chapter in the story of a program that was run ineptly from the beginning, and the obvious results that had to follow.

There is little point in wallowing in all the mistakes that led to the current mess. Suffice to say the system is over a year from operating to specification. And what we are getting instead is politically stupid spin.

For example, has anyone noticed the difference between “applied”, “enrolled” and “covered”? The reporting on this is curious, as I noted before.  The number of people enrolled (verses applied) is very suspicious:

By October 16, some 3,000 paper applications were in the processing center but only “81 have been successfully entered.”

…

October 9: “A new problem in the system has been identified: for about 30% of the 70,000 applicants, the system has skipped applicants through ‘events’ that are required to complete the application.” In other words, nearly a third of applicants couldn’t fill out the form, because the website was skipping “events” or entries they needed to make.

Emphasis mine. Look at the dates and the numbers. Oct 9: 70,000 applicants, 30% of which (21,000) have massive errors. But one week later we see 3,000 PAPER applications were in processing. Of these, only 81 applicants had become ‘enrolled’. These are different states. Yet we see a mix of numbers being presented as a sign of fixes coming? But are they coming?

Which number are we getting? Which number is being pushed as success? Here is the question I pondered the other day:

So how is it the site can handle 25,000 people at any given moment on any given day, but only 27,000 people are enrolled after 4-6 weeks? What is the bottle neck?

When I see the website capacity is heading towards 50,000 people, I realize that number is basically meaningless. Because the problem is that the system is still spewing out bad applications to insurers. And the only solution has been humans reviewing the applications and cleaning them up by contacting the applicant. A process the April Red Team stated would take 30 days per applicant!

But “enrolled” is still not “covered”. And that is the acid test come January 1.  Who have insurance and who lost insurance. And that is a huge issue brewing underneath the radar:

TWS: If I don’t pick a plan until 12/15, won’t it be too late for my info to go to the insurance company, them to bill me, and me to make a payment by 12/21? Seems pretty tight.

[Healthcare.gov] Rep: You must make your first premium payment by 12/15/13 for your coverage to begin January 1, 2014. If you make your payment by the 21st, your coverage will begin in February 1, 2014.

And right now, enrollees cannot pay for their coverage through Healthcare.gov:

The chief digital architect for the federal health insurance marketplace said Tuesday that 30 percent to 40 percent of the project was still being built. The official, Henry Chao, made the assessment in testimony before a panel of the House Energy and Commerce Committee. Lawmakers expressed surprise that so much work remained to be done seven weeks after the federal website opened to the public. Mr. Chao, the deputy chief information officer at the Centers for Medicare and Medicaid Services, said the government was still working on “back office systems,” including those needed to pay insurance companies that are supposed to provide coverage to millions of people under President Obama’s health care law. “We have yet — we still have to build the financial management aspects of the system, which includes our accounting system and payment system and reconciliation system,” Mr. Chao said. These parts of the system, he said, are “still being developed and tested.”” Robert Pear in The New York Times.

How can all those millions of people who lost their insurance get covered by January 1 when the ObamaCare is only able to process the backlog of applications by hand, and the payment system is not even built? By all measures, ObamaCare is a galactic disaster that is still becoming visible. And as more and more is exposed, one wonders how long the Baghdad Bobs on the left can keep pretending otherwise?