todddagres:

Real-time is hot right now.  Real-time messaging, Real-time social networking, Real-time search.  These are powerful movements based on the value of what is happening right now.  But, what is more valuable that knowing what is happening now? That’s easy- knowing what WILL happen. Imagine having insight into what will happen tomorrow or next year.  I will argue all day long that nothing is more valuable than being able to predict the future.

Imagine a search engine that helps you to predict what will happen.  I call this an Inference Engine (rather than a Search Engine.)  Of course it’s futuristic and even science-fictional to think we can write software that predicts the future. That said, can we write software that can scan and analyze real-time information, patterns and trends and in so doing can provide insight into what will happen? I think so. To some extent it being done in financial markets. Algorithms and black boxes scour the web and data feeds for an inkling into what’s happening and how it will affect the future results. Data mining platforms strive to crunch massive amounts of information quickly to provide insight into consumer behavior and trends in purchasing. Given the vast amount of real-time and historical information available on the web, isn’t it possible that some events can be inferred? The Hollywood Stock Exchange has done a reasonable job for years of predicting box office results based on a virtual market for movies. This form of prediction is largely based on “crowd-sourcing”.

While Google is the 800 lb Gorilla in search, the service doesn’t do a good job of predictive search. I consider Google good for searching things that have happened. The current state of the art and most promising innovation in search is real-time search. I consider Twitter Search and OneRiot good for real-time information queries.   Rather than indexing the web and returning relevant search results, real-time search brings results as a stream of information based on the search terms. I believe Real-time search is the current next thing. It is available now and the wave is building. (See my previous post below about the value of real-time search.)

As for an Inference Engine, I have found nothing. There are a few companies out there claiming to deliver predictive information based on the mining of mounds of data. However, there is no consumer search service that even comes close to delivering predictive search. By bouncing around Twitter, OneRiot and Google I am able to gather information that can help make predictions.  However, I have to do most of the work. I would much rather type in a phrase of series of key words and see a prediction pertaining to my search language.  I can see a widget or app using Twitter to feed an Inference Engine based on trends.  OneRiot can enable searches based on real-time data that includes the velocity of trends, themes and sharing.  Taken a step further, OneRiot could potentially help predict events into the future.  The question is how far and how accurate.

The challenge with building an Inference Engine is finding, analyzing and presenting predictions based on onforseen future events.  Clearly, searches must be bounded and the results must be limited to situations where the chance of success is greatest.  For example, asking what lottery number to pick is not reasonable because it is purely chance and no information available (legal that is) would help imporve your odds.  A bounded query would have to be based on information available on the web including real-time information.  The Inference Engine could include crowd-sourcing, chaos theory and  quantum physics. I don’t really care as long as it works.  Examples of a inference queries could be- “number one movie next week”, “stock market direction tomorrow” , or “home mortgage rates in one month”.   Each of these predictions would be based on information that is available on the Web up to the second the query is launched.  The search could also be persistent in that it could be updated continually based on new information.  Given that these predictions would be effected by events in the future, the persistent nature of the search would be powerful.

My purpose in writing this post is hoping that it leads to funding an Inference Engine company. I don’t normally signal my investment intentions but in this case I am making an exception. I believe there are people working on this unstructured search problem and want to cast a broad net. So if you are aware of any company or individual working on search technology to predict the future, please pass it along. Thanks.

td