IBM Search Engine "Trawls" Web.

If three separate World Wide Web pages contain hyperlinks to three other sites, something is going on. That principle underlies Clever, search technology developed by IBM researchers that might offer a better way of locating information on the Web. Clever brought the research team - R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins - the distinction of best paper at this year_s World Wide Web Conference, taking place here this week.Entitled "Trawling the Web for Emerging Cyber Communities," the paper describes an approach that uses patterns of interconnections among Web sites to discover communities that one might not expect. It can also locate information in ways a conventional search engine would not. One page pointing to two other sites usually does not indicate any important connection among the three sites, Raghavan said. Even two pointing to two is not significant enough. But when three sites all point to three other sites, and the links are not what Raghavan called nepotistic - that is, there is an obvious link among the pages such as the fact that they are part of the same organization_s Web site. One use of this principle could be in discovering unexpected communities on the Web. That could have applications to marketing, among other things. Another application is finding sites that meet certain criteria but that might not be easy to find with the conventional text-searching approach. A Clever search would pick up on the fact that sites pointing to one of these companies_ sites also tend to point to the others, and would discern a community. Given a term such as "mainframe" or the names IBM and Hitachi, say, it would find mainframe-related sites including the vendors_ home pages, even though those pages don_t actually contain the word "mainframe." IBM sees Clever not as a general-purpose consumer search engine but as a tool for more sophisticated searchers. It is currently in pilot use within IBM, and the company will move quickly to bring it to market, possibly by working with one or more marketing partners.