Wowd has entered private beta testing, and I got some invite keys for my flickr friends if you want to give it a whirl. Feedback on the concept and implementation are welcomed.

In short, Wowd has built a fully distributed search engine. No server farm. No web crawlers. It’s like the Skype of real-time search.

Besides changing the cost, and data center energy equation for a new entrant, we think this architecture will have some interesting implications. Instead of relying on content or link analysis for page rank, which is architected – and sometimes gamed – by the publishers on the web, Wowd relies on the real-time surfing behavior of real people. It’s hard to game the behavior of the crowd in the cloud as this scales. So, you get the real-time web, the social web, and over time, the deep web. Think of all those sites that block web crawlers or are hopelessly out of date in search results today – from social networks, to eBay and Craigslist to various niches, like used car sites.

It has been quite exciting to see the product in development over the years. When the founder, Boris, first came to us, he had a bold vision and a smart team of young developers in Serbia. It reminded me of the Skype team from Estonia in so many ways. Boris and I spoke with the founders or original programmers at most of the major search companies, and they opined that it couldn’t be done (with reasonable system latency). That made it all the more interesting, and we seed financed the company. In full disclosure, I am also on the Board.

We thought that we would need many thousands of users before the distributed search engine would “work” and that early search results would be skewed by small numbers, biased by a disproportionate number of Serbs =). It is fascinating to watch in the early days, as many of the properties of the system are emergent, and the quality and performance of the system should improve with scale.

So, if you want to play with it, the link below should take you past the gates to the Wowd site. Please do so only if you are curious and willing to play with early code. It is a browser app that works across Windows, Mac and Linux. If you like it, great, if not, please keep it open and check in from time to time to see if it has improved, and of course, constructive feedback is even better. OK, so here’s the private beta Invite Link. Click on the “Get Wowd” button there to join the Wowd cloud.

• Recent article and video in readwriteweb (their invite keys ran out, fyi)
• Blogs: Wowd, Boris’ Distributed Search

30 responses to “Get Wowd”

  1. cluttered sofa Avatar
    cluttered sofa

    thanks for sharing

    Reply
  2. shaky payment Avatar
    shaky payment

    Looks quite interesting. Will sure test it.
    —
    Seen on my Flickr home page. ( ?² )

    Reply
  3. jurvetson Avatar
    jurvetson

    Thanks! Two new nodes just joined the Wowd cloud from here. Cool.

    Reply
  4. Jeff Clow Avatar
    Jeff Clow

    Am having a hard time downloading, Steve…..when I click the button on the site that you linked to, it doesn’t do anything.

    Any ideas? I’m running Windows XP….

    Reply
  5. gdsanders Avatar
    gdsanders

    Cool. I’ll check it out.

    Reply
  6. born1945 Avatar
    born1945

    Thanks Steve. I have downloaded the app and will use it in conjunction with other search engines I use. It certainly is a different idea than Google, etc. It will be interesting to see how it fills in over the next few months.

    To others, I recommend the VIDEO for a quick idea of how it all works.

    Reply
  7. jurvetson Avatar
    jurvetson

    Jeff – if you click the blue button on the top right, does it take you to a page with links for the various computer types?

    Reply
  8. seemly food Avatar
    seemly food

    Wowd reminds me of the first time this was done, back in the late 90s.

    Anyone remember Peer 2 Peer? Infrasearch? Gone Silent?

    Reply
  9. Jeff Clow Avatar
    Jeff Clow

    Yes, but when I click on Windows, nothing happens….

    Reply
  10. angry shock Avatar
    angry shock

    Checking it out. You know this interests me muchly. My first question of the relevance system to crawl and show pages is: how the newcomers get into play if the system only rewards with visibility to sites with an already large audience? My web can be good and relevant to the people searching for related content but if I am new nobody will find me in Wowd? How do I gather that momentum?

    This look very 2.0-type-of-sites-oriented, yet a major part of the web is not 2.0 and will not be (not all websites apply to that format, thinking that is forcing form over substance, which is a mistake, imo).

    Anyway, I look forward to the answer, as a user and as a web designer and a SEO specialist, but I celebrate all the efforts in making a better web by the hand of a better index of it.

    Reply
  11. angry shock Avatar
    angry shock

    FYI, when I click on Windows to download, also nothing happens, like to Jeff. 😉

    Reply
  12. jurvetson Avatar
    jurvetson

    Strange…. Asking them …. That link seems to work on Windows 2000 and Macs… looking for XP…

    12 new cloud members so far today – maybe all Mac users. =)

    To your first question: a new site does not have inbound links either. Human attention might flock to items of value more quickly than new inbound links followed by crawlers. I do admit that it puts the SEO and structural consultants at risk…

    Not sure I understand the second question, but we may find that archival search needs to lean more on contextual cues or personalized search (the next feature set).

    Reply
  13. jurvetson Avatar
    jurvetson

    commence fire drill in Belgrade…..

    "please tell your friends that we’ve been swamped w/ interest, that the web site itself is responding slowly or not at all right now, and that the team behind the site is working furiously to bring it back"

    Reply
  14. Bill York Avatar
    Bill York

    Hey everyone. I am the VP of Engineering for Wowd. As was said, we had some issues with searches timing out on the web site, stemming from the surge of interest from Steve’s post and the ReadWriteWeb post. We think those are resolved now. In any case, if you download and run the local client things should work fine.

    However, I don’t see anything on our side showing the download link failures some people have reported here. If anyone would like to pursue this further or help debug it, please contact me at bill.york at wowd.com.

    Finally, @GiselaGiardino, the way Wowd handles new pages is that the first Wowd user who finds a new public page causes it to be added to the index in a couple of minutes, after which it is available for any future searches. Textual content, inbound links and human visits to the page are all factored in to the ranking. Also, the Wowd Hot List shows the pages that are INCREASING in popularity the fastest right now, not the ones that are already the most popular, so a new page that has some buzz should show up there even if the total visitor count is still low. Hope that helps.

    Reply
  15. angry shock Avatar
    angry shock

    @Steve: True in part.(imo) New sites sometimes *do not have* inbound links, sometimes *they do* to speed up web crawling, yet also Google / Yahoo / Etc allows you to add it manually to their index and you have tools like Webmastertools to help Google trace an categorize/rank your website.

    Wowd doesn’t put SEO consultants at risk, I guess, as per what Bill York mentions as the way to get indexed: Someone had to find your web somewhere else and it’s added… so you still have to be visible first somewhere else or do it yourself the finding to be indexed at wowd.

    OK. I just didn’t knew the procedure. 🙂

    As per the second part, there’s a difference in between "web2.0" kind of sites (blogs, networks, shared content sites, wikipedia…) and normal "window" sites where the owner displays information alone, there’s no user generated content, there’s nothing being asked to the user but to read and if they are interested do something (contact, buy, link, etc). The normal institutional websites work that way.

    I sensed that Wowd works more or less like delicious, technorati and the other crawlers which are fed by user generated actions, like votes, reviews, comments, etc… (am I wrong? correct me please) which belongs to the era 2.0. Ordinary websites tend to stay out of this stream of 2.0 relevancy. Some may say: "then turn your website to a web-2.0 kind of one!" Sure, but my opinion is that, if your website does not fit that format, your service or product, you just shouldn’t use the 2.0 format, and find other means of relevancy (ordinary SEO).

    Anyway, this explanation doesn’t matter much. As long as there are algorithms to follow relevancy based on behaviour, there will be people able to tackle them to their best advantage despite genuine deserving of getting visibility. It’s a double razor. Just like democracy. Just like marketing. It’s an ethical stance and a system which permits ethical detours because it’s naturally not perfect.

    I just read about Google adding a Sidewiki feature, for example… so people can comment on websites as aside notes… I wonder how they are going to filter the comments out, and I also wonder if this comment feature will impact in the rankings (rewarded in someway). If they don’t, comments will be really authentic. If they do, I guess we will just witness another spam trend like that on blog comments, of here in flickr, too.

    ‘Tis a complicated issue this search engine / wbe index thing! And the SEO is one side of it… the other aspects are fastness, integrity, hardware, software, etc. I’m in this since 2000 and still reading and talking about the same problems!

    Wish you much success with Wowd, I’ll stay tuned! 😉

    Reply
  16. NYCandre Avatar
    NYCandre

    FYI – no problem downloading or installing (XP) the local client. That was at 9/23/09 11:19pm EST, the client runs OK but often "timed out" somewhat improved, but still hiccupping 4am EST. Understandable, it ‘s private beta .. Updated 6am EST this is likely due on my end to software protection (firewall, antivirus. antispyware). Will contact the techs to let them know.

    First impressions: generic searches are different enough from Google’s – avoids a lot of repetitive similar links, and sometimes points out to more interesting links earlier in the search. I like the option of ranking search results by either popularity or freshness, and the (as yet untested) search own history capability.

    – the search from own history sounds like a great idea – once it is built up. Not clear if the search history *and* viewed pages are saved locally – if not saved, then the dead link issue is a problem. If saved that would be great, would save filing time.

    -curious if WOWD will index *all* my search history or only searches made from the WOWD interface (a rethorical question, clearly only from the WOWD interface, and only when I have started the WOWD client). This means I’d have to make some commitment to using exclusively W for search, an uncertain proposition, while I know the established G search is OK. Hmm .. An interesting possibility would be to let WOWD collect my non-WOWD searches with Google and even other websites (with appropriate filters/ permissions).

    – if my W history and web recommendations are only saved locally does that means I can only use one computer .. and if it crashes this is all gone.. I know it isnt going to be that black and white, some clarification might help

    – is W twitter like, in the sense that I could specify somehow who my preferred peer searchers are (with their authorization..)…

    – the general idea of using users actual searches for ranking and the cloud of users machines to bypass the ram/ hard drive latency bottlenecks and offload cpu to peer machines sounds pretty smart and ambitious -this must be getting Google’s engineers busy – or maybe they’ll be interested in an acquisition 😉

    Thanks for giving us an early look into this new tech development. Didn’t mean to write so much here, will look for the appropriate forum.

    Reply
  17. Duncan Rawlinson - Duncan.co Avatar
    Duncan Rawlinson – Duncan.co

    *new member!

    Reply
  18. jurvetson Avatar
    jurvetson

    NYC.andre – great questions… group-flavored search makes a lot of sense…

    Alieness: On the SEO comment, yes, it’s by so mean binary, and if one defines the SEO term as “marketing consulting” in general, then sure, it will always be relevant.

    I was referring to the subset of marketing consulting that is aimed at gaming the search engines. And by gaming, I mean purposeful modification of the structure or content of a website to feed the search engines in a way that does not improve the end user experience.

    Ideally, one could modify the content or structure of a site to create a better user experience and thereby get better search rankings. That would be an ideal world of aligned incentives, and core to the Wowd dream. Marketing consulting – like how to write interesting stories or take good pictures – should be universal, and should dominate if the gamers and spammers could be neutralized.

    I was speaking of the SEO consultants that focus on gaming the system, an evolutionary arms race with the search engines that adds no value to the end user and introduces transaction costs and noise to product marketing.

    Back in 1995, when I was the webmaster for our site, it was remarkably easy to game the system. Infoseek published their algorithm – parsing the first 100 words of any page for keywords. So a quick insertion of 100 chosen words at the top of the page in a small font, white letters on a white background, would get you top placement in search results. It was that easy. And in the early days, there was not much competition (most VC firms did not have a website in 1995, much less someone thinking about SEO games). This is a clear example of gaming the content filters. The content of the page that the end user sees is completely decoupled from the search engine food that invisibly rigs search results.

    The games got more complex, but the example of content gaming is similar to URL tagging and link farms – adding no value to the end user while trying to rig results.

    So, my hope is that if you ignore the architects of the web, ignore the links and the content, then the SEO gaming consulting industry is at risk, to the benefit of all. Sure, there are several ways to imagine gaming Wowd – to fake the patterns of human attention – but my hope is that the cost of faking the behavior of millions of users will not scale well as the cloud grows, and thus, we can shift the cost vs. detection/prevention balance against the spammers.

    P.S. I just heard Ballmer at Microsoft say that the next 5 years of search will herald more innovation than the past five years. Especially in the UI. The two hot new areas for them are visualization of results and natural language processing.

    Reply
  19. angry shock Avatar
    angry shock

    Hola Steve. That’s my deepest hope, too. What I don’t see clearly yet is Wowd’s advantage over another existent systems in this regard (referrring to BY explanation).

    It’s a debate around the concept of "Wisdom of Crowds" and also the paradigm of "natural selection". Some think: "let them do and they will organize for their best general interest" (the Hive Mind you know well about), and others think: "the system needs intelligent intervention and rules in some way to bring an expected and useful order out of an otherwise chaotic activity (and perhaps badly intended or with bad results).

    As you see, I deem it as, first of all, a conceptual/epytemological/sociological conundrum rather than a technical one. We have to think what kind of web do we want. (and I know this exceeds our conversation here, and I know this has already been decided as per the results)

    But as I said, a search engine is not only a matter of good indexing and relevancy alone, it has some other features of importance too, so in case Wowd ends up being just as good in this area as others, but they offer fastness, better interface… a better overall user experience, they will surely have their good chunk of the market very soon.

    Good to hear Balmer saying that. I hope so, don’t think it’s that easy tho, when you see the advancement AIs are doing in regard to interpreting and using natural language -as far as i’m concerned-. It’s still pretty discouraging. I don’t think the semantic web will be a reality but after a complete breakthrough in AI technology and/or Cognitive Sciences. And this might not happen so soon. My bet. (Let’s promise to come back to this thread in 5 years!)

    Reply
  20. photon~wave Avatar
    photon~wave

    WOW d!
    Interesting concept: Distributed search.
    Let me tell what I could understand about it.

    With the downloadable software, a browser
    becomes a local search engine which crawls
    only among the visited websites in each
    particular machine. I belive that only
    links to each page plus a set of tags
    are stored in the user’s hard disk.
    The alternative for storing the complete
    history of the browser’s cache is unreasonable.
    The soft simultaneously ranks each link in according
    to the number of visits to the corresponding site.
    Also, the soft behaves like a botnet that responds
    to the central server’s inquiries. (I don’t mean
    to suggest that the botnet behavior be malicious
    but i think that the browser’s app must be open
    source).

    Thus, the searching process is fired by a user
    from his computer and the central server asks
    to all(?) botnets using tags. The botnets answer
    to the server with the top ranked links having
    the required tags. Then, the server make the
    final rank with the most "voted" links and return
    to the user the ranked list.

    Therefore, the system is a full peer democracy,
    and the ranking system seems to be transparent.
    A site will be top-ranked only if it is actually
    visited massively in the world, without any other
    considerations like number of links that point
    to it. Never mind what keywords or metacommands
    have a page in its headers, the page will be top-
    ranked only if it is being seen in the browsers.
    From this point of view, forget SEO. Or perhaps,
    the better strategy for SEO will be hypping up
    the sites by its real merits.

    Of course, distributed search only will take-off
    when the botnet soft will be massively used.
    Here, i found the first problem: How to become
    the concept of distributed search in a meme
    and how to become the soft in a need, given that
    the first users will not find much utility,
    perhaps for months. Additionally, the success of
    the project is based on that the users leave the
    soft running… The only related cases that i
    have in mind are @home projects, but they are
    restringed to particular communities.
    Past experiences like Skype are not the case here.
    Only two people that agree to meet in skype put
    the system operative.

    Another problem i see is the highly heterogeneous
    and punctuated distribution of speed access to the web.
    Thus, if the time out of the server is relatively short
    to be realible, a huge number of botnets will be out
    of game and vast regions of the world will not participate.

    On the other hand, i suggest that proper tagging of links
    with time-stamp into the browsers allows for "time search".
    Thus, we can inquire not only for what people are
    currently seeing (real time searching), but also for
    what they were reading a month or a year ago about
    our interests.

    Moreover, tagging links with the geographical region
    of the browser (for this task IP must be used but not
    necessarily recorded) allows for "local search".
    Thus, we can inquire about our interests only in
    a particular city or country.

    And for last, but not less important, it is the
    issue about how to build a sustainable business
    for distributed search.
    I foresee ads inserted directly in the list of
    search results. But it is the realm of Google!

    I hope the angel investors could resist until
    the business blast-off.
    Good luck.

    Reply
  21. Tanner Powell Avatar
    Tanner Powell

    Yay, I’m glad I stopped by here. After watching you speak about this a while back I’ve thought about the prospect of federated search on a daily basis. Very excited to try this out, thanks.

    Reply
  22. nels1 Avatar
    nels1

    Thanks for the invite Steve:)
    I have been testing it for the last couple of days and here are some comments so far (apologies for the length)…

    It is interesting that the information is crowdsourced although there is no facility to continue the crowd conversation regarding the links, therby losing the potential leverage of the wisdom of crowds. Social Bookmarking sites such as Digg, Stumbleupon and Reddit for instance have a lot of user interaction regarding each link posted which often enhances the value of the link itself in isolation.

    Wowd seems to be more "crowd-clicked" rather than "crowd-sourced". The important difference here is that clicking a link does not necessarily imply any relevance or usefulness, but simply that it was clicked. When we are searching online for the purpose of research (compared to for discovery), we often open multiple links and close them just as quickly when it’s clear that they are not relevant. Unless there are ways to use metrics to evaluate how long we spend on the site or how deep we venture in, or to introduce rating/ranking criteria for the links visited (for both search and discovery purposes), it means that every link clicked is equally relevant, which is most often not the case.

    Compare this to crowd-submitted sites such as the social bookmarking sites mentioned before. Usually for someone to find a link and then take the effort to submit it, it has already gone through 2 levels of self-filtering – ie. 1. it is relevant to me, and 2. it is good and important enough for me to want to share with others, so there is already inherent value in each link, and on top of that, the ranking system and the continuing conversation relating to the link add even further value to the user. There is always the potential for spam but with a robust community, crowd-filtering is usually quite successful.

    One of the unique points stated in the Wowd video is that the search results represent what is being viewed by other users in real-time. For the lay person, many may also assume Google Search is in real time despite not realising that it may take many weeks for a website to appear after being crawled. Real time may have its advantages if what you’re searching for is popular right now on a crowd level. However, if you look at the top 100 current search terms on Google Zeitgeist, it will also show that what the crowd is looking at right now (britney spears, matte nail polish, survivor samoa episode 2:)) is often not interesting or relevant to you at all, either in search or discovery. Most users are looking for content that is relevant and compelling and interesting to *them*, not to the whole internet community. That is also why having categories (such as news, entertainment, science, politics etc) to filter the search results would help with improving relevancy for the individual user. Wowd does state that it is a content discovery and recommendation network rather than a search engine, so it’s main competitors I think are still the social bookmarking sites.

    The challenge for new search applications is that there must be a compelling reason for people to change the way they search and discover as they have been so used to using Google, Yahoo and Digg, Stumbleupon etc. When I tested out Bing, there were some really interesting and compelling features relating to image searches that weren’t available with Google at the time. However, since then, Google have added these features too so there wasn’t enough reason to switch over to Bing). I see lots of potential with Wowd, although I can also imagine potential users asking:

    1. How is it better than searching *within* already well established large social bookmarking sites such as Digg, Stumbleupon and Reddit?
    2. In terms of real time, how is it better than filtering my search results *within* the established large social bookmarking site by filtering the results to show the most recent?
    3. The content is crowdsourced, but how do I filter out the noise?

    Reply
  23. jurvetson Avatar
    jurvetson

    Thanks y’all !!

    Heading out to the desert…. Wont’ be able to compose a thoughtful reply for a few days. One difference is that active voting can be incorporated but is a small subset of the attention data. Yes on categories – even user defined tags.

    Reply
  24. Bill York Avatar
    Bill York

    [Sorry for the long post, but I am really enjoying the public dialog]

    @Gisela
    Wowd is not really in the "Web 2.0" category as you describe it. Unlike systems that require explicit user voting or bookmarking (Delicious, Digg) Wowd powers its search rankings from the statistical aggregation of our users’ implicit actions (e.g. visiting pages). Since this analysis is done by the Wowd software on your computer and only shared with the cloud as anonymous data, your privacy is protected. So webmasters and marketers take heart: build a page or site that attracts users and it WILL be highly-ranked by Wowd, regardless of how it scores with the live-out-loud crowd. On the other hand, some of the current SEO tricks, such as link farm sites, will not have much impact on Wowd rankings, which is bad for the spammers and good for the average user.

    @NYC.andre
    We had some problems with timeouts on cached servers the day Steve posted this invite. We think that has been cleared up, so please let us know if you are still seeing any problems. As for Wowd history search, we are currently storing only the index data from your visited pages, not the full page content. We could store the content, but that would obviously have disk space consequences. The data is saved only on your local disk for privacy reasons, but have a multi-computer Wowd history sync process is on our roadmap. Wowd does, in fact, capture all the pages you visit, not just the ones found via Wowd searches. It is far more useful to be able to search the full history. So feel free to continue to use other search and discovery tools as appropriate. Finally, being able to specify your "peer group" for searches and ranking is a very powerful concept. Stay tuned for further developments on that front.

    @skeptical thinker
    Pretty much right-on. However, there are no "central servers". The cloud forms a single distributed store. Each computer in the peer-to-peer network is responsible for storing some fraction of the data in the whole index (not necessarily the fraction found by that computer’s user, just the fraction assigned to it by the algorithm). Then, when any member computer does a search, it requests data from the nearest (lowest-latency) peers that have relevant index data, and then does the intersection, ranking and display steps on the local system. And of course, it is our job to make this all work in the face of unreliable networks and computers with various connection lags.

    @nels1
    As you say, the world already has plenty of explicit-voting crowsourced ranking sites. And as many have noticed, the flaw is that most ordinary people aren’t interested in "living out loud" and voting and commenting explicitly on sites. So the "democracy" ends up being dominated by a small but vocal minority of the community. While a click is a weaker signal of quality than a vote, it is still a human judgement, and our analysis shows that aggregating these click statistics can be a powerful ranking signal, and one that is harder to game than page linking or explicit voting. And even in plain, ordinary searching there is value in being able to find the new content on any page on the web real time as soon as any member of the community finds it, rather than waiting for the next crawl or vote.

    Reply
  25. born1945 Avatar
    born1945

    I have decided to delete Wowd from my computer at this time. I will check it out in the future, but right now it’s not for me. Steve, thanks for opportunity to look at the program. What follows are the comments I left on the Wowd site:

    Browser: Firefox 3.5.3, Mozilla 5.0, Windows XP Professional.
    At present I do not see the benefit of Wowd for me. Possibly in the future it will be something I can use. Even when my computer sets idle Wowd is using resources, from 1 to 24 % of the processing power. As I was loading a ".pps" file from an email, Wowd was using resources, slowing down the loading of the file. I don’t understand why Wowd is using resources when I am not actually using the program.

    If the program was not grabbing computer resources like it is, I might be inclined to keep it resident on my computer. As it is, I will be deleting the program.

    Best of luck in making this a successful program. I think the idea is a good one, but not for me right now.

    Reply
  26. Bill York Avatar
    Bill York

    @born1945
    Sorry to see you go. As you said, Wowd can run in the background when you are idle. For example, it does this when it is serving up fragments of index data to other nodes in the Wowd Cloud. It is precisely these idle cycles that we are trying to capture and put to use in the peer-to-peer cloud. In this way the Wowd community can create a truly planetary-scale distributed computation and storage engine, and enable a first-class search service without requiring huge server farms with all their attendant power and cooling costs. You can control Wowd’s resource consumption through the Settings panel.

    However, Wowd should never slow down your other activities on your computer. It is only the idle cycles we are after. So if you decide to try Wowd again at some point, I would like to get more detail on your experience. Thanks for trying Wowd.

    Reply
  27. @edvard Avatar
    @edvard

    Thank you for sharing, this is epic.

    Reply
  28. Gregman2 Avatar
    Gregman2

    Thanks Steve. Wowd is fast.

    Reply
  29. Whetstone Multi-Media Artist Avatar
    Whetstone Multi-Media Artist

    I would love to test Wowd. Thanks for the info.

    Reply
  30. photon~wave Avatar
    photon~wave

    Tuesday, October 20, 2009
    Wowd Public Launch

    Reply

Leave a Reply to Gregman2 Cancel reply

Your email address will not be published. Required fields are marked *