Why an Open Source Search Algorithm is the Answer
January 7th, 2008Google is known for their free services, open APIs and their “don’t be evil” motto. For most part they aren’t evil, but keeping their search algorithm a secret is just plain evil. Microsoft is known as a closed source software giant with a desktop OS monopoly. What makes Google any different when they have a proprietary search algorithm and they have a Web monopoly? Why don’t they open their search algorithm? The reasons are very similar to why Microsoft has kept their code closed and ironically Google too can become victim of the power of social computing and open source.
Wikia Search launched today and while there has been much criticism, their principals make total sense:
Transparency - Openness in how the systems and algorithms operate, both in the form of open source licenses and open content + APIs. Community - Everyone is able to contribute in some way (as individuals or entire organizations), strong social and community focus. Quality - Significantly improve the relevancy and accuracy of search results and the searching experience. Privacy - Must be protected, do not store or transmit any identifying data.
Google gives “guidelines” on how to make Web sites more SEO friendly, but why not offer complete transparency to their algorithm? The first reaction is because the algorithm is very, very complex and that an open source algorithm could be abused by those that find loopholes. Well it is already being abused by Black Hat SEO practices and that is exactly why it should be socially driven and open source! It is the very reason why open source solutions like Linux and Apache are a success: Open and engaged communities have much more combined knowledge than closed communities can ever have - even more than the smart Google engineers. For now Google is riding the wave that Microsoft did in the nineties, but sooner or later they will come under pressure and will be faced with the decision to open up their algorithm or not.
To me, the four principals of Internet search that Wikia Search has identified defines what Google means by “don’t be evil” and while their alpha release may be disappointing I applaud Wikia for doing this. Just like open source supporters have fought Microsoft, I believe the social computing community will be fighting Google some day. An ungrateful irony.
January 13th, 2008 at 11:10 pm
>
I totally disagree. It sounds like your defense is that their search offering is a monopoly on the web, but there’s nothing wrong with monopolies. It’s when things start hurting the consumer (or when Google uses that advantage illegally) that it becomes an anti-trust issue.
What secrets should be held by companies today? Any?
January 13th, 2008 at 11:11 pm
Follow up - I was responding to your comment ‘keeping their search algorithm a secret is just plain evil.’ but the plug-in ate the characters I surrounded it with.
January 15th, 2008 at 8:17 am
Hey Matt, thanks for stopping by. My argument is that a search algorithm is a perfect candidate for social computing. I don’t think monopolies are necessarily bad, but as a monopoly they have a level of social responsibility. Secrets are also okay. In Google’s case their lightning fast response times, huge database of indexed pages and their ability to crawl sites frequently is what gives them the competitive edge and the technology behind that can certainly remain a secret.
When you buy a computer, you want to know what components you are getting (Intel 2.6GHz), but don’t care about how the components were manufactured and patented technologies they use. Likewise, from a SEO customer perspective, it will be nice to know exactly which elements of your Web page are important and which aren’t, instead of guessing.
April 18th, 2008 at 11:44 am
If Google were to publish exactly how their search algorithm worked, I wonder if their search results would be less useful. It seems like that could increase the amount of spam as people found ways to take advantage of it.
If someone wanted to create an opensource search engine the question is if anyone would use it. If the results were somehow better than Google, people would eventually switch from Google. However it seems likely that the results wouldn’t be as good as Google because of the reasons mentioned above.
It is an interesting idea though.