Long day for me today. We had a consultant come in from out of town ("out of country" is more accurate but less rythmic) and he didn't get to our building until damn near 3pm. So, to make good use of his time, we had to stay late. We only have him to help solve our problems until tomorrow morning, so we are pressed to utilize the opportunity.
On the TV behind me are the talking heads of Obama and McCain, badmouthing each other and talking about "The Surge". The surge, as people will undoubtedly remember, was Bush's last-ditch effort to save the abysmal Iraq war effort, and probably put a good spin on things for the Republicans before elections started. Obama says the surge has failed, McCain says it's a success, and both are insulting each other's opinions on the matter.
The problem is that the surge neither succeeded nor failed. It wasn't part of some comprehensive plan, there was no intended goal and no criteria that defined what "success" or "failure" were. To McCain, what would you have to see right now to decide the surge failed? To Obama, what would you need to see to say that it succeeded? The problem is that both men are going to define it differently because there were no benchmarks laid out beforehand.
When we do an experiment at work, we have numbers and thresholds and goals. We use controls, although we can't run a second war without a surge to compare (note to Bush: this is not a dare). We set specific numbers: We want to see SNR increase by 2 dB, we want QoS reliability to improve by 5%, we want to cut power consumption by 20%, etc. We didn't have goals with the surge, therefore it is impossible to declare it a failure or a success. We can talk about whether it met our personal expectations, or whether we think it's cost to benefit ratio was desirable, or any of other qualitative comparisons, but they don't strictly mean anything.