Socially Speaking

The April 5th Waxxi cast with Jimmy Wales is up, and available for your streaming (or downloading) pleasure, here.

At just 32 minutes, this was the shortest cast we’ve done so far, and the first one we haven’t had to cut in two parts. Although the reason for the shortened cast had to do more with Jimmy’s schedule than anything else, it’s a refreshingly nice length.

It’s a fascinating conversation, and while it’s (again, for the first time) just mine and Jimmy’s voices you’ll hear, the interactivity and participation was awesome.
One question, for those of you who prefer text, came from the live chat:

Steve asks: What are the pros and cons of a transparent search algorithm? Are the advantages of having an open book approach to how search is performed worth the risks of some who will use the information to “game” the system?

JW:

Yeah, so that is a really core question. And, in a certain sense, the success or failure of this entire concept hinges on that question: which actually will work better in the long run?

So, there are a couple things that I think are pretty clear. When you have an transparent, open search engine with freely licensed software, when people find that there’s a problem, there’s a potential for people to actually correct it – and actually have oversight into what’s gone wrong and how to fix it – that you really don’t get in a proprietary search engine, unless you hire lots and lots of people.

The political implications are, well, they’re important to me. And I mean political with a small ‘p’, not really talking about government, but talking about the organization of society, and the organization of information in society. I think as citizens and consumers and producers in the world, we should be concerned about secrecy around such a core piece of the infrastructure of the Internet. So that’s one of the major pros.

Now, if you talk to security people – so, people who work in computer security – they’ll always tell you that “security through obscurity is a bad idea.” In other words, if the way you’re keeping something secure is by keeping it secret so people can’t game it, well you’re always subject to people to figuring it out and gaming it without you noticing. You’re subject to that kind of attack all the time.

One of the reasons we trust the encryption algorithms that we use is that they’re published. They’re public, and they’ve been tested by many, many mathematicians and computer programmers. Everybody can throw what they want at it, and try to find a flaw. If you’ve got a secret encryption algorithm, well…you just don’t know: I mean, has it really been tested thoroughly? And so forth.

So, I think the same idea applies to search algorithms. If the only reason it’s good is because it’s secret, well, that never lasts. What you really want is to truly begin to solve the problem in a more systematic way. For that I think the open approach is the best.

During the Jimmy Wales ‘cast, some of the fundamental questions involved his new project, Wikia, the future of human-powered search, and competing with Google. Some fantastic content awaits you in the podcast, which will be up on Waxxi very soon.

Can you tell us more about Wikia?

Basically, Wikia is my new project. We are building thousands of Wiki communities in parallel. The way that we like to describe it is that Wikipedia is the encyclopedia, and Wikia is the rest of the library and the magazine racks. So, it’s a totally new organization completely separate from Wikipedia, and growing really quickly. [We’re] spending a lot of energy in trying to improve the software and make it easier to use, to try to push this whole free culture revolution out to the next wave of participants.

How will human powered search work?

So, the search project is one of the projects of Wikia, and basically what we’re looking at is everything is open source software – all free software. We want to publish all the algorithms; we want to bring some transparency into the search business.

There are a lot of people who are trying to do human powered search, or trying to do new algorithmic search, but I don’t know of anybody who’s really trying to make a radical commitment to being open and transparent in the sense of free software. So, that’s basically what our goal is.

How it’s actually going to work? Well, that’s yet to be determined. I mean, we’re still in the open design stages. It’s not the kind of thing where we labor for twelve months in stealth mode, then build it all out and announce it in a flurry. It’s a project to build a search engine, so anyone can participate and we’re discussing and debating how to go about it.

How do you plan to successfully compete with Google? A question via chat, from the incredibly participative Rick Myers:

Um…I have no idea! I mean, I think the real answer is, if you believe as I do: that quality search is becoming a commodity. So, if you take a look at the results from Yahoo!, look at the search results from Google, from Ask – they’re really quite similar in many respects, and that’s been increasingly so in the last couple of years. And then you look at some of the stuff that’s been going on in the open source world, around search engines. I think we’re in striking distance of having good quality search in a free software package.

If that’s true, then it’s really a matter of just having the servers and people that manage them, and then you can really compete. If that’s true, then competition is not about having the most money and the most rocket scientists. It’s about having open transparency: search results people can trust because they can understand how things are ranked and sorted.

So that’s basically the approach we’re taking. I don’t normally think much in terms of competition, I think more in terms of finding something cool and fun to do, and doing it.

We had another great interactive experience at yesterday’s Waxxi ‘cast. Jimmy Wales, as you might expect, was a fascinating guest. We’re truly grateful that he took the time to join us, and share his thoughts and ideas on Wikipedia, Wikia, search, culture, the web, Google, and more.

It was interesting to see the level of interaction via IM vs. phone. This was the first Waxxi ‘cast where most all of the questions came in via chat or private IM vs. the phone (just two callers who ‘raised their hand’ to ask a quesion, one of whom we couldn’t hear because his headset’s mic was muted – woops!). Since most of the participants were true power-users/early adopters, IM is simply more natural a communications method than the phone.

We had a great showing of NYC’s ‘next generation of digital movers and shakers’ – otherwise known as nextNYers. One of them, David Evans of Civil Netizen, says of Waxxi:

If basic podcasting is like standing, then Waxxi’s interactive podcasts are like flying…

He also blogged about his experience yesterday:

The chat was moderated by Tracy Sheridan of Waxxi, who performed the job masterfully like a great conductor, bridging the worlds of phone callers, online chatters, studio guests, and staff into a fluid experience for both Jimmy and the guests. The event was reminscent of a good talk show where the host and guest interact with both a live studio audience as well as with phone in callers.
…

I’ve never felt so drawn in to an interactive online event like I was today. For an hour, I felt like I wasn’t even sitting at my desk, but rather as a part of a live studio audience. Suddenly, listening to TWiT on the train ride home just doesn’t sound as exciting as it used to…

Wow. We’re absolutely blushing! Thanks to David, to our guest Jimmy Wales, and each person who participated. We had a lot of fun. Till next time!

In a report from Tokyo yesterday, Reuters declared, “the online collaboration responsible for Wikipedia plans to build a search engine to rival those of Google Inc. and Yahoo, Inc.”

Well, we could’ve told you that.

Jimmy Wales came out, fully loaded, with some fighting words — or about as peaceful as fighting words can get:

The idea that Google has some edge because they’ve got super-duper rocket scientists may be a little antiquated now.

He went on to describe Google and Yahoo! as “black boxes” that won’t reveal how they rank search results. And, that collaborative search technology could transform the structure of the Internet.

What this translates to is Wikia, the for-profit sister site to Wikipedia, will take great lessons from the non-profit’s core. Wikia Search, then, is a place where:

…users could work together to improve search engines, just as Wikipedia users had tweaked and rewritten articles on the sprawling encyclopedia.

Wikia as a whole hosts collaborative community publishing sites, and is supported by advertising. Examples of some of these communities include 24, the Muppet Wiki (one of Jimmy’s favorites), and the currently featured collaboration, Gears of War.