Google joins crowd, adds semantic search capabilities

[dhilipkumar]

Google joins crowd, adds semantic search capabilities
Search market leader takes initial step toward Semantic Web, to help users refine queries

Google Inc. has given its search engine an initial injection of semantic search technology, pushing the market leader into what many observers see as the future of Internet search.

The new technology will enable Google's search engine to identify concepts and associated terms related to queries, thereby improving the list of related search terms that are displayed along with search results, the company said in a blog post today.For example, Google's search engine, upon encountering a query such as "principles of physics," will now be able to understand that "angular momentum," "special relativity," "big bang" and "quantum mechanics" are all related terms, according to the blog post, which was co-written by two of Google's technical staffers.

Ori Allon, technical leader of Google's search quality team and one of the blog post's authors, said in an interview that the added capabilities involve a dollop of semantic search technology mixed with a large helping of on-the-fly data mining, with a primary goal of helping users to refine their queries so they can find the specific information they're looking for.

"This is a new approach to query refinement because we're finding concepts and entities related to queries while you do a search, so everything is happening in real time and not [pre-assembled]," Allon said.He added that the company isn't using semantic technology more broadly at this point because full conceptual analysis of documents would slow down the search and query-refinement process. "If we want to get it all done in a matter of milliseconds [using semantic tools only], there's a lot of innovations we still have to do," Allon said. "A full semantic search would be very hard to do in [such a] limited amount of time."

But offering suggestions for refining queries is only the first of what Google officials "hope will be many other applications" of the new technology, Allon said. For example, users eventually should see improvements in areas such as page ranking."For simple queries like 'Britney Spears' and 'Barack Obama,' it's pretty easy for us to rank the pages," he said. "But when the query is 'What medication should I take after my eye surgery?' -- that's much harder. We need to understand the meaning."

Google has been criticized for using what is considered an aging approach to processing search queries based primarily on analyzing keywords and not on understanding their meaning.

Over the years, Google executives have acknowledged that semantic technology will be an important component of search engines in the future — while also saying that they see semantic capabilities as a part of the algorithmic mix, not as a full replacement for traditional keyword analysis.

computerworld

[dhilipkumar]

Google joins crowd, adds semantic search capabilities

"Right now, Google is really good with keywords, and that's a limitation we think the search engine should be able to overcome with time," Marissa Mayer, Google's vice president of search products & user experience, said in an October 2007 interview with IDG News Service. "People should be able to ask questions and we should understand their meaning, or they should be able to talk about things at a conceptual level."

But, Mayer added, Google already offers "a lot of context around things like acronyms" because of the huge amounts of search data that it processes. "I think the best algorithm for search," she said, "is a mix of both brute-force computation and sheer comprehensiveness, and also the qualitative human component."

In January, during a conference call on Google's fourth-quarter results, CEO Eric Schmidt touched briefly on the topic of the Semantic Web, hinting that the company is getting more serious about semantic technology. "Wouldn't it be nice," Schmidt said, "if Google understood the meaning of your phrase, rather than just the words that are in the phrase? We have made a lot of discoveries in that area that are going to roll out."

Many of Google's competitors already are busy developing or working to perfect semantic search engines, betting that they will be able to deliver on the promise of the technology: to let users type queries in natural language and have a search engine understand their meaning and intent.

Last year, for example, Microsoft Corp. acquired Powerset Inc. to gain access to the start-up vendor's semantic search technology. Also last year, Yahoo Inc. announced plans to support various Semantic Web standards as part of its new open search platform, the latest in a series of moves by that company to embrace the semantic approach. Other vendors that are offering semantic capabilities in their search engines include Ask.com and Hakia Inc.

Google today also added another enhancement to its search engine: longer "snippets," that are text excerpts extracted from Web sites to show within search results where query keywords appear.

Many critics have said that the current excerpts aren't very useful because they often don't provide enough context to help users decide whether to go to a particular Web site. Now, when people enter queries that include three or more keywords, Google's engine will deliver longer snippets designed to give users a better sense of how the keywords are used on Web sites.

It remains to be seen if publishers and other Web site owners will cry foul over the longer snippets. In the past, some publishers have complained that lengthy search engine abstracts give away too much of their content, potentially causing would-be visitors to not click on links because they've already gotten the information they're looking for.

That is an area where search engines have to strike a delicate balance between giving users the most precise information possible related to queries and not violating the copyrights of Web site owners.

[dhilipkumar]

Google 3.0: Semantic search


If you aren't following changes in search technology, you could have almost missed today's announcement from Google – which seems to be insignificant at first, but at a closer look a dramatic improvement that effectively rings in Google 3.0.

The original Google was good enough to fend of rivals until 2007, when the company announced its first major improvement called Universal Search. Back then, the company combined its search silos - text, images, videos, shopping and others – but already hinted at a future feature called contextual search. Contextual search, commonly referred to as semantic search, links related topics to better predict what users may be looking for.

This contextual search is now live and includes algorithms that Google claims understand what other search terms may be useful to understand users, queries and content. At least in theory, search results should be broader and, in Google speak, more universal. There was no information how the company's page ranking will be impacted by the additional search results that will now be provided.

While Google said that semantic search is now available in 37 language versions of Google, a previously described feature the company has been working on is not available – contextual search across different languages, or, as Google calls it, "cross-language information retrieval." In 2007, Google said that this feature will translate search queries in twelve different languages and return search results from websites in those twelve languages to provide a more comprehensive search experience. Google did not say when this feature will become available.

There are, however, two other minor improvements – a better handling of more complex search queries as well as the display of longer snippets in search results.