Monday, September 18, 2006

What happens when statistics and masses replace institutions?

Open sourceI've long thought that in open source software we are seeing a trend away from trust in an institution (think: Microsoft) and towards trust in 'good luck'- i.e., in the statistical likelihood that if you fall, someone will catch you. In open source, this is most manifest in support- instead of calling a 1-800 # (where someone is guaranteed to help you, as long as you're willing to be on hold for ages and pay sometimes very high charges), one emails a list, where no one is responsible for you, but yet a great percentage of the time, someone will answer anyway. There is no guarantee, but community practices evolve to make it statistically likely that help (or bug fixing, or whatever) will occur. The internet makes this possible- whereas in the past if you wanted free advice, you had to have a close friend with the right skills free time, you can now draw from a much broader pool of people. If that pool is large enough (and in software, it appears to be) then it is a statistical matter that one of them is likely to have both the right skills and the right amount of free time.

Clay Shirky today makes an argument that this isn't just something that is occurring in open source, but is hitting other fields of expertise as well: "My belief is that Wikipedia's success dramatizes instead a change in the nature of authority, moving from trust inhering in guarantees offered by institutions to probabilities created by processes." Instead of referring to a known expert to get at knowledge, you can ask Wikipedia- which is the output of a dialectic process which may fail in specific instances but which Clay seems to suggest can be trusted more than any one institution's processes in the long run.

So does this have any implications for the law? My sense is that it might. The Berkman Center's OpenLaw project experimented with generating briefs via volunteer input years ago, but never quite got the critical mass needed to take off and become self-sustaining. So it might not be that we're writing briefs by public collaboraiton in a wiki any time soon. But Wikipedia is already an excellent resource for double-checking interpretation of important cases, SSRN downloads are being used to supplement (and perhaps eventually replace?) journal placement as a metric of quality of scholarly output, blogs are being cited in Supreme Court decisions, and we're seeing some of the first computer systems that make legal decisions, and allow external input.

Clay would take from this that our legal orthodoxy and leading institutions are being challenged- in many cases, instead of first looking to what the Leading Journals tell us, we're looking to what blogs, wikis, and other systems of shared expertise tell us. His extrapolation seems to be that this will eventually change how we feel about authority in our field. I'm not sure I'm ready to go that far- but based on some of the data points noted above, I've got a hunch we'll end up a lot closer to his end of the spectrum than we currently are.


Blogger Mike Linksvayer said...

"guaranteed to help"

Not really. Someone is guaranteed to talk to you, eventually. Whether paid support actually solves your problem is another question.

In decentralized scenarios nobody is guaranteed to even talk to you, but I wonder if the chances of actually getting a solution are any less probable, or any more probabalistic. Depends on the problem of course.

Chris Anderson wrote about people's discomfort with probabalistic systems late last year. My riff on that bears directly on the support example:

"Distributed systems sacrifice claims of perfection for optimization at the macroscale."

9/18/2006 10:33 PM  
Blogger Luis Villa said...

Fair. I used 'guaranteed' because I wanted to focus on what people think they are getting (which is usually more important than what people actually get anyway.)

Also, thanks for the links- I had seen the Long Tail one but failed to remember it.

9/18/2006 10:43 PM  
Anonymous Anonymous said...

John Fleck says -

It's worth looking back at basic economics and not thinking we're inventing the wheel here. I think the open source/volunteer model is an interesting and new special case of a more general principle.

"The market" behaves in a quite similar way. There is no one institution in charge of making sure there's bread in the markets of Manhattan. Community practices have evolved to make it statistically *highly* likely that there will be bread there - a whole bunch of suppliers making separate decisions, a whole bunch of stores making separate decisions, etc. The motivation here is different - profit - but the system behavior has a great deal of similarity.

9/19/2006 11:40 AM  
Blogger Luis Villa said...

Sure; nothing is new under the sun :) But as we know, the market is fairly broken in fields that are heavily dependent on reputation, particularly when that reputation can't be directly monetized. Google has provided us with a way to keep replace our old financial record-keeping (he who has the most money, wins) with a new unit of currency- he who has the most links, wins. On the plus size, this allows us to 'monetize' reputation a lot better than we used to, which allows for (say) blogs to supplant journals as reputational measures; on the down side, my landlord does not yet accept links.

9/19/2006 12:01 PM  

Post a Comment

Links to this post:

Create a Link

<< Home