Microsoft Open Source algorithm that gives Bing some of its intelligence

Today's search engines are more than just stupid wordstrippers. You can ask a question – say, "How tall is the tower in Paris?" – and they will tell you that the Eiffel Tower measures 324 meters (1,063 feet) tall, about the same as an 81-story building. They can do it even if the question never names the tower.

How do they do that? Like everything else these days, they use machine learning. The machine learning algorithms are used to create vectors – essentially long lists of numbers – that represent their input data, whether it's text on a web page, images, sound or videos. Bing captures billions of these vectors for all types of media it indexes. To search vectors, Microsoft uses an algorithm called SPTAG ("Space Partition Tree and Graph"). An input query is converted to a vector and SPTAG is used to quickly find "nearest approximate neighbors" (ANN), ie vectors similar to the input.

Here is how we can answer the question of the Eiffel Tower (with a little excitement): "How tall is the tower in Paris?" These will be pages "close" on the towers, Paris and the height of things. It is almost certain that such pages will talk about the Eiffel Tower.

Microsoft today released the SPTAG algorithm as an open source MIT license on GitHub. This code is proven and production quality, used to answer questions in Bing. Developers can use this algorithm to find their own sets of vectors and do it quickly: a single machine can handle 250 million vectors and respond to 1,000 queries per second. There are examples and explanations in Microsoft's AI Lab and Azure will have a service using the same algorithms.

Microsoft CEO Satya Nadella has repeatedly expressed his desire to "democratize artificial intelligence" and make it accessible to all, creating not only a centralized and specialized tool that requires considerable expertise, but also a broad range of developers, solving many problems. range of problems, can use as part of their toolbox. The SPTAG publication is an example of how Microsoft puts these words into practice. the combination of an Azure service and an open source means that developers can start with the service more constrained and easy to use. As their expertise or requirements become more complex, they can use SPTAG to create their own services.