- Rationalists think that there is a good chance of succeeding without experimentation.
- Empiricists don't.
As a new empiricist (my academic background is pure math and philosophy), I am of course a bit evangelical...and the biggest errors made by the smartest people I read all seem to me to be of the same category: Rationalism.
Hence...a question: Given a new complex product, and no experimentation/iteration, how likely is it that the product succeeds? How likely is it that it is close? My assumption is that the probability is roughly 0%. Given almost as much experimentation as you want...how likely is it that the first prototype you build is roughly good enough? Again...I suggest 0%.
I know that this is true of complex software (more than hello world -- though I once wrote 200 lines to solve a toy problem, compiled it clean and it ran successfully -- once in the 30 years I've been programming.). It is also famously true of Battle plans. It is also true of engineering product development. And it is mostly true of corporate activity (correcting for delusions). I believe this to be true of Artificial Intelligence. And also of government systems.
If I'm right as a hardcore evangelical empiricist, then all the theorizing you wish to do is near-useless until you get on the ground and try a few different things.
Rationalism in government.
Madison and company did an insanely good job. 200 year good run by design, world-leader for at least 100, and growth-driver as well. Wow. The Swiss seem to have a fabulous, 700+ year model as well with magnificent stability, if not the same level of growth-driving. I'm personally fond of Singapore's model, but the country is barely older than I am.
The 1792 French model sucked. The 1921 German attempt at "democracy" failed catastrophically. The 1917 Russian revolution was worse. The Asian Communists were, if anything, even worse (China, Korea, Vietnam, Cambodia).
On systems that were stable for >200 years, we're rather shy of examples. Feudalism in Europe? Roman (+/- Holy) empire. Chinese Empire. Islamic Caliphate. The age of absolutism just wasn't that long. And none of these escaped Clark's Malthusian trap for the majority of the people. One is inclined to believe they all encourage stagnation, or at least the bureaucracy that stifles growth. Only stuff left is the David Friedman / Roderick Long unearthings of Celtic and Norse anarchies, some of which appear to have lasted longer than the States so far, but which didn't encourage growth either.
Is empiricism the wrong approach here? Or is someone going to argue that .. yes, it's true of all other complex systems known to man, but in this particular case, there's a better than 25% chance that Big Design, No Testing would succeed for government?
Posts
6 comments:
It's a false dichotomy. Obviously experimentation is desirable, sometimes it is just not possible.
I work at a software startup. We do test driven development and release daily iterations. We are very heavy on experimentation. But this is allowed because experimentation is so cheap, it's easy to change code, it's easy to push a new update if our code broke something.
But even in this experimentation heavy environment, we still rely mostly on rational logic. If I write 100 lines of code, that's all from logic and rationality. Usually only a couple of those lines will have a bug the first time I run it. Thus even in this highly iterative environment, I'm still relying to a great deal on rationality.
And conversely, if a piece of code passes the tests, but there is something logically wrong with it, the person doing the code review forces the programmer to fix it. That's just good engineering.
But sometimes experimentation is either extremely expensive or not even possible. Building a rocket that goes to the Moon is a good example. Every iteration is super expensive, so extensive engineering effort must be put into the system. Even more extreme is the manned moon lander itself - it had to work the first time it was ever used, or people would die. And it did.
Creating a government is much more like building a rocket. Every iteration is super expensive and mistakes are deadly. Thus you cannot under invest in engineering.
This is my problem with Romer and the "Let a thousand nations bloom" folk. They call for experimentation, but spend little time on the actual design of the experiment they want to implement. That's like saying, "Let's build a rocket by experimenting", and then not spending any time to actually learn rocket science or hire rocket scientists. Nor do they seem to be aware that no rocket scientists currently exist. Thus if an experiment in government was run now, it would probably fail, because no one has put the brainpower into making it work. And actually, there have been previous iterations and data points in government design: the American constitution, the elective monarchy of the Holy Roman Empire, the various republics and oligarchies of Italy and Greece, caliphates, papal states, the monarchies and aristocracies of Belle Epoque Europe, etc. Plus there are the designs of multinational corporations, non-profits, cities, communities, communes, virtual worlds, etc.
Lots of experiments have been run. But the competitive government types are not putting in the effort to actually learn from these experiments, in order to make the next experiment as well designed as humanly possible.
The only people actually working on the problem are Nick Szabo and Moldbug.
And of course, in the mainstream, experimenters are doing the same thing over and over again, expecting different results. The failure has been predictable and tragic. See: Iraq and Afghanistan.
A lot of the experiments you mention - communism in Russia, democracy in 1920's Germany - were examples of people failing to learn from previous experiments. Communism had been tried dozens of times in communes across the United States, and in different parts of the world. It had always failed, for predictable reasons. Yet they kept trying anyways.
The divide between "empricism" and "rationality" is false. What's needed is for people to learn from history, and learn from mistakes, and think through new designs. That way when the opportunity to experiment is presented, or when people are forced to experiment do to terrible failure of existing systems, the resulting attempt can be as good as possible.
Two parts.
1. Not a false dichotomy.
As a developer/systems/process improvement guy in a shop with poor practices that I'm trying to move closer to the ones you get to deal with daily (I could paint a heroic martyr picture, but I'll refrain), I am coming from a similar place.
We agree...experimentation is done when it's cheap. When it's not cheap, we do less experimentation, and it breaks a lot more.
You pull out the standard example of rockets. What percentage of probes (Mars, Venus, etc.) function as expected? Seems to me that we had a boatload of rockets, most of which failed...until we got that working. Then we had a bunch of multi-stagers with the same success profile. Then satellites, again with the same profile. Finally, after we appeared to have roughly 1 zillion attempts that were nearly the same, we tried human flights. Given the best processes money can buy, and tremendous experience with every relevant part, we're only failing at a 5% rate on manned launches and a 20% rate on extraplanetary probes. Any bets on whether that fabulous success rate is caused by using the same, mostly working launch technology for 30 years?
I don't think it's a false dichotomy. I think that without experimentation, you should expect to fail. Repeatedly. Painfully. And I think that your experience as a developer, especially a TDD guy, says that too.
The failures constitute the substitute for experiments. After you fail enough, you may have learned enough to succeed, but radical changes constitute another high likelihood of failure.
If what you're claiming is that Moldbug is proposing an experiment that has a high probability of teaching us something, I am significantly onboard.
If what you're claiming is that Moldbug is proposing an experiment that has a high probability of actually being better than what we've got, I am substantially less onboard.
Rationalism is probably the best option you've got when you have no chance to experiment. But that just ups your chances of success from 3% to 7%...it's not like you've actually made success likely.
2. Most people do same thing, expect new results. This is stupid. Yes. Double plus yes.
I don't see Moldbug's project as trying to design a completely novel system or product from scratch. I see it as taking everything we've learned about government over the past two hundred years (and everything from before then that the founders failed to take into account) in order to come up with the next, better iteration of government.
But it is true, the more radical the iteration, the more chance of introducing new bugs in addition to fixing the bugs of the previous iteration.
Nick Szabo's ideas are a bit more modest, more incremental improvements. See his "Comments on the Constitution Series.
But it's not like our own system was particularly well designed, especially in it's current state. The original design had many mistakes. From that point it evolved, devolved, and decayed.
So I do think that there is a good chance that an improved design, learning from the mistakes of the past, could be a substantial improvement over the current state of affairs.
But all political changes are dangerous, so I'm not sure I'd support experimenting on the national level, unless it was clear that the current government was in a state of irreversible decay.
But I would definitely support trying new ideas on city-state level.
That said, the number of experiments will still be very limited, even if the experiments happen at the city-state level. Thus it is vitally important to learn from history and incorporate lessons from all past designs into the new design.
Devin,
Thanks for the link to Szabo. I'll go looking.
Other than that, we seem to be entirely in agreement. I don't think I can add to or disagree with anything in your last comment.
Thanks for exploring together.
I love to see the agreements, but there is still tension in Aretae's thought?:
Most people do same thing, expect new results. This is stupid. Yes. Double plus yes.
Vs
If what you're claiming is that Moldbug is proposing an experiment that has a high probability of actually being better than what we've got, I am substantially less onboard.
I say: The DOUBLE PLUS YES is why it's SO easy to improve on the status quo. The Iraq example is spot on. Current prevailing practices are retarded. (Yes, surely individual retarded current practices have evolved to support each other and ameliorate each other in complicated beneficial ways. And, perfectly smart adjustments have been found by regular people to also ameliorate bad practices. But still!!)
Moldbug and Szabo do not labor alone. Even Dr Horrible has workable improvements:
"And by the way it's not about making money, [it's about d]estroying the status quo because the status is not quo. The world is a mess and I just need to rule it."
Bonus:
"My application is strong this year. A letter of condemnation from the deputy mayor - that's got to have some weight. So... fingers crossed!"
@Andrew
While it's hard to argue with Dr. Horrible, I will nonetheless endeavor to at least show that my internal tension is much less than it seems
Fundamentally it turns on the question in the post. How hard is new stuff? I am arguing that ALL new experiments are substantially unlikely to succeed, even if 90% of their core points are unqualifiedly better than what we have. Details matter, screwing the details can screw the whole thing, and we don't know which details are the ones that matter.
If you believe in knowledge/reality improvements via plans that succeed you have one position. If you believe that knowledge/reality improvements happen almost exclusively through experiments that mostly fail, but eventually get good enough, you get a different position.
See my next post on Madison, which I am about to write.
Post a Comment