Nov 16 2012
Major Update: More pathetic details here:
As the Web traffic from volunteers attempting to connect to Orca mounted, the system crashed repeatedly because of bandwidth constraints. At one point the network connection to the campaign’s data center went down—apparently because the ISP shut it off. “They told us Comcast thought it was a denial of service attack and shut it down,” Dittuobu recounted. “(Centinello) was giddy about it,” he added—presumably because he thought that so much traffic was sign of heavy system use.
Major Update: More details are emerging on the mess that was Romney’s GOTV debacle:
But Orca turned out to be toothless, thanks to a series of deployment blunders and network and system failures. While the system was stress-tested using automated testing tools, users received little or no advance training on the system. Crucially, there was no dry run to test how Orca would perform over the public Internet.
I surmised as much from seeing the results. Being someone who has worked on some of this nation’s largest command and control systems, it was obvious what happened here. That’s amateur hour, and it cost us all – end update
Sorry for the not posting, but elections matter and I had to take steps to ensure we can ride out the coming economic storm. Besides, I needed some space to analyze the debacle that was this election instead of just reacting. The day after the election the hard truth showed up on what happened, and it explained a lot. Before I get to that, realize that the Absentee/Early vote effort by Romney and the GOP was damn good. So good there should have been no way to lose CO – but they did. So good they should have taken some Senate seats, but they did not. So good that if they won (as usual) the election day GOTV (Get Out The Vote), we would not be here.
But they did not. And we know why.
Here is the sad tale of technical incompetence that led to the 2012 election debacle, and its name is “ORCA“:
The entire purpose of this project was to digitize the decades-old practice of strike lists. The old way was to sit with your paper and mark off people that have voted and every hour or so, someone from the campaign would come get your list and take it back to local headquarters. Then, they’d begin contacting people that hadn’t voted yet and encourage them to head to the polls. It’s worked for years.
From the very start there were warning signs. After signing up, you were invited to take part in nightly conference calls. The calls were more of the slick marketing speech type than helpful training sessions.
On one of the last conference calls (I believe it was on Saturday night), they told us that our packets would be arriving shortly. Now, there seemed to be a fair amount of confusion about what they meant by “packet”. Some people on Twitter were wondering if that meant a packet in the mail or a pdf or what. Finally, my packet arrived at 4PM on Monday afternoon as an emailed 60 page pdf.
OK, full stop. This is to be a massive, coordinated campaign to monitor the voting and get people out. It takes time and preparation for those 100’s of thousands of volunteers to be trained, outfitted and ready to execute. Yet some idiot waits to the bloody last day to get the instructions out? 60 pages on a Monday afternoon for a coordinated plan of attack to be implemented the next morning?
Are you ‘effing kidding me! This is incompetence writ large. That material needed to be out 4 weeks earlier. It required people to be registered and able be vote monitors:
At 6:30AM on Tuesday, I went to the polls. I was immediately turned away because I didn’t have my poll watcher certificate. Many, many people had this problem. The impression I got was this was taken care of because they had “registered me”. Others were as well. But apparently, I was supposed to go on my own to a Victory Center to pick it up, but that was never communicated properly.
While I was home, I took to Twitter and the web to try to find some answers. From what I saw, these problems were widespread. People had been kicked from poll watching for having no certificate. Others never received their pdf packets. Some were sent the wrong packets from a different area. Some received their packet, but their usernames and passwords didn’t work.
So now the GOP had lost ALL insight into what is happening on the ground. The entire project relies on knowing who has and has not voted. They were blind while Team Obama knew what was happening because their people were in place. This explains why GOP vote monitors were being kicked out everywhere.
Again, what moron tries to execute a tactical effort like this at the last second? And if you think this was the worst of it, the technology behind this disaster was not ready for prime time either:
Now a note about the technology itself. For starters, this was billed as an “app” when it was actually a mobile-optimized website (or “web app”). For days I saw people on Twitter saying they couldn’t find the app on the Android Market or iTunes and couldn’t download it. Well, that’s because it didn’t exist. It was a website. This created a ton of confusion. Not to mention that they didn’t even “turn it on” until 6AM in the morning, so people couldn’t properly familiarize themselves with how it worked on their personal phone beforehand.
By 2PM, I had completely given up. I finally got ahold of someone at around 1PM and I never heard back. From what I understand, the entire system crashed at around 4PM. I’m not sure if that’s true, but it wouldn’t surprise me.
You NEVER launch something this complex hours before it must execute. The nimrods who were behind this should go find another career, like ditch digging. You NEVER wait to the night before to send out a complex tactical plan. You leave time for screw ups to be corrected. You rehearse, you checklist. You NEVER deploy a website or app without brutal load testing. Everything should be up, running and working like a fine Swiss Watch.
What we got in the Romney GOTV were technical amateurs and incompetents. They took what was a huge wave of momentum and cut the knees out from it on the day the GOP usually carries the day.
You can point fingers everywhere you want, but the proof is in CO where the early voting had the GOP at historic highs and in the lead. Yet, in the end, Romney’s Orca was still able to grasp defeat from the jaws of victory in Colorado. And since this occurred across the nation, it is also clear it was a systemic problem.
Elections are about having the best team win. And given this royal screw up (which could amount to 2-4% in the vote tallies), it is clear the best team did win.
… the absolute failure of Romney’s get-out-the-vote effort, which underperformed even John McCain’s lackluster 2008 turnout.
Romney volunteers in Virginia confirmed that the campaign had relied entirely on Project Orca to turn out the vote in key areas such as Roanoke, where Romney and Ryan had made appearances. Volunteers who had driven to Virginia from safely-Republican Tennessee were shocked at the disorganization they encountered.
If each of the 37,000 volunteers that had been devoted to Orca had instead brought 20 voters to the polls in those states over the course of the day, Romney would have won the election.
Next Up: The Petraeus War & How Obama’s “Stazi” Tried To Silence The Truth of Benghazi