Microsoft wants to use HoloLens, AI to repair videoconferencing



[ad_1]

Two things are said about almost every video conference meeting: "Can you see my screen?" And "Can you get closer to the microphone?"

Teleconferences, especially those with more than five people, are at best clumsy and, at worst, lead to headaches. In a typical scenario, 10 people are gathered in a room in the home office and seven or eight people traveling or working at home have to call, or Skype from their laptop. Some of them keep the microphone muted and turn off the camera, making it a purely passive participant. Because remote people often can not see the faces of other participants, they have no idea when they can talk. When they get into the discussion, it sounds like this: "Hi hey hi hi hello? Hello? It's Ed in Phoenix, can you hear me? And they can not see people's body language at the other end, so they have no idea if their comments are nodding, shrugging, or staring.

It is a miserable experience that many companies have tried to solve over the years without success. Right now, Microsoft thinks it has the solution – with new technology that improves collaboration, instead of creating barriers.

Two participants in the meeting, one of them virtual, use HoloLens to examine a 3D plan of a building. [Photo: Mark Sullivan]

Microsoft introduced its new meeting technology in a reconstructed conference room at one of the Ignite Conference conferences Monday in Orlando. (Until a last-minute change, the demo was originally scheduled to take place at the conference of CEO Satya Nadella.) I was allowed to attend one of the rehearsals of the presentation in one of the laboratories of Microsoft.

Some of the technology that I have seen is now available, some will be available later this year, and still others are still in the experimental stage. Meeting technology draws on the work of many Microsoft groups, including Office 365, Teams, Bing, Cognitive Services (AI), and even HoloLens.

Before the meeting

Good meetings allow good people to talk about the right things at the right time. Microsoft thinks this process starts well before the meeting itself.

The idea of ​​Microsoft is that each meeting is represented by a place in the cloud where all the relevant information about the subject of the meeting and the people involved can be gathered. It is a more advanced and resource-rich version of the old calendar event. This meeting object is used for communication and collaboration before, during, and after the meeting. Prior to the meeting, Microsoft's AI can add or suggest logistical information, participant information, as well as relevant background files and information. New files can be added during the meeting, such as collaboration notes and a real-time transcription of the discussion. Post-meeting notes such as actions can be added, as well as follow-up notes and communications.

Conversations within the Microsoft Teams application can be translated in real time. [Image: courtesy of Microsoft]

The purpose of the meeting is often from Microsoft Teams, the company's Slack-type communication and collaboration environment. The teams have been available for about two years and, according to Microsoft, are used by 329,000 organizations worldwide, including 87 on the Fortune 100.

MacDonald, from Microsoft, who originally proposed the team's ideas to CEO Satya Nadella and now heads the group of teams, told me to consider the teams as the hub of all applications and services of Microsoft 365. Word , Excel, Edge and many others).

"Every time we talk about meetings, most of them will appear on teams," said MacDonald.

This message was not lost on the people who watched the presentation at the Ignite conference. "Microsoft's latest announcements to Ignite underscore ongoing efforts to make teams the communications, collaborative and work center. . "Said CCS Insight analysis Angela Ashenden in a statement. "Microsoft Teams is becoming the focal point for Office 365 employees. "

The teams

Meetings can come from teams in different ways. If a few people are discussing a problem in a Team channel, there is always a button nearby to take the conversation to a live video chat. There is another button in the chat window that initiates a formal meeting and starts a meeting object that begins to attract relevant data for the meeting.

In the center of the simulated conference room of the demo is a mysterious cone-shaped object with a fish-eye lens at its end. In fact, it is a prototype that is not yet on the market. But his job is to watch and listen to people sitting around the table. At the beginning of the demo, you see a screen on the big screen at the front of the room with a band at the top that shows the 360-degree view of the fish-eye lens, with labels next to it four people at the table.

Below, in the left pane, you see the transcript of the meeting being created by the camera and an AI in natural language. The cone-shaped device uses both sight and sound to identify the faces and voices of people sitting around the table. So he knows who's talking at all times (even when people are talking to even time). The system could also translate words heard in another language. In the right pane of the screen was a space reserved for "information and notes" that can be created by anyone at the meeting.

The screen has been used to display a lot of things. Anyone in the meeting could project the screen of his or her laptop (Surfaces all around) on the screen. At one point, a Cortana robot assistant was called to organize a subsequent meeting.

It was mainly to express Microsoft's vision for meetings. These are not things you can buy for your business today. But the demo also showed some things that are ready for prime time.

Search everywhere

For good people to talk about good things in meetings, research is very important. You must be able to capture guest details, logistical information, relevant documents, and background information, such as previous discussions that took place about the meeting. This information usually comes from open websites and digital assets behind the company's firewall.

For example, Microsoft has been working to extend the reach of its search engine on the Web to obtain more general and public information (via Bing), LinkedIn to obtain professional information on internal and external people and on private companies.

A meeting participant, when logged in with an Office 365 account, can search for a person who will also be present to find out who he or she is addressing. A search in Microsoft 365 can find the location of the person's desktop, an organization chart, the files that the person has publicly shared, or a LinkedIn post she has written. A Conversations view shows the discussions the person had in the Microsoft Yammer discussion app and in the teams.

Microsoft has put the search box everywhere, with research suggested by AI. [Image: courtesy of Microsoft]

The search engine can now also return results from third-party applications that the company can use, such as SAP or Adobe.

At one point in the demonstration, one of the participants suddenly asked if he could take his wife and children with him at the annual sale. He typed, "Can I take my wife and kids with me on a business trip?" In a search box at the top of the Powerpoint screen in which he was working (Microsoft has now placed the search field at the top of all screens). its productivity applications). The man has recovered a mix of links to work-related and non-work related content. At the top of the screen, he saw four cards including bookmarks (to company documents), groups (working groups), files and relevant intranet sites of the company. These links have supported the political part of the company's query. Beneath them, Bing returned links to relevant additional information from the open Web, such as information about places, flights and hotels, for example.

"By using the Bing consumer search engine's AI, we can provide more personalized, relevant and consumed search results," said Naomi Moneypenny, Product Management Director for Teamwork and Search.

Build the brain of the company

The AI ​​under Microsoft 365 always learns from the "signals" that it receives from applications used by a client. All of these signals – which could mean the way a company plans meetings, shares documents or secures strategy documents – appear in a chart, which can be considered the brain of the company, according to Microsoft. The AI ​​(borrowed from Bing) relies constantly on the information in the chart to learn, analyze and make recommendations based on the company's work habits. When a user organizes a meeting, he can see partially filled search results in accordance with the organizational habits detected in the past. If, for example, a meeting of a certain group of people has generally taken place in a certain conference room, the IA may suggest that location for future meetings of the same people.

"Microsoft Graph is really the way we learn how businesses use content, and how they meet, and how collaboration works," said Moneypenny. "We can learn their daily work habits."

Microsoft says it will deploy extended search on Microsoft 365, Bing.com, Office.com and the SharePoint mobile app. And it will start appearing in Microsoft Edge, Windows and Office in the future.

Video meetings become smart

Microsoft has added a new feature to video meetings that allows participants to explore the background of their location. This can be used to ensure that other participants can not see that the user is logging in from a Starbucks or that a cat or baby is in the room. A non-commercial or unpredictable background can be a disincentive for users to keep their camera during a meeting, which may prevent the user from fully participating in the discussion.

Blur a background is easy enough for a still image. The blurring of the background involves a pretty serious artificial intelligence, says Lori Wright, general manager of teamwork and Microsoft 365 collaboration tools. "The AI ​​lets us know what you're up against and which is not, "says Wright. "And the AI ​​has to do it every time you move."

Microsoft Product Marketing Manager Raanah Amjadi using the background blur. [Photo: Mark Sullivan]

One person from the demo meeting showed how Microsoft has added a meeting recording feature that allows users to read the content of recorded meetings at any time, including on a mobile. The recordings include the transcript (or translation) of the meeting, which includes timecodes so you can click on a given phrase and go directly to that part of the video. And you can search for words or topics in the meeting, such as your own name or the name of a client.

"It's my Netflix DIY," says Raanah Amjadi, Product Marketing Manager for Teams and Skype.

In the course of the year, Microsoft 365 users will be able to prepare and launch live streaming events from Teams, Microsoft Stream or the Yammer chat application. It's a great way for management and others to reach the most diverse workforce. People in different time zones can use the mobile application Stream (iOS and Android) to replay the event, even offline.

A new surface hub

At one point in the demonstration, two of the team members moved to the new Surface Hub 2 digital whiteboard to work on a project together. Both were able to connect to the device by touching a fingerprint reader on the bottom edge of the screen. As the Hub then had the account IDs of both users, they could both display one of their own documents on the same screen. They were also able to move content from one document to another. So, for example, user B can insert a statistic into his PowerPoint presentation from the Word or Excel document of user A. Microsoft announces the release of Surface Hub 2 in the second quarter of 2019.

The beginnings of HoloMeetings

The most interesting part of the demo was when two of the participants – one of them in another city – examined and discussed a 3D virtual object using HoloLens. In the demo scenario, the remote person was the smart building manager who could see from the sensor data that seven of the conference rooms in his building were not being used extensively. The fictional company of the demo wanted to help the man (their client) to know why. The construction manager and one of the fictitious company engineers therefore attached their HoloLens helmets to a 3D virtual model of the building. They were both able to pinch and drag the model to see it in different ways.

The other participants in the meeting could not control the image of the 3D building, but they could see it on a monitor in the room. To do this, they used third-party camera software that directed HoloLens people and digital objects to the meeting place (seen by a camera in the corner of the room). In the view on the screen, the image of the building manager was at the front of the table in front of the building model. His image is far from being clear; it was heavily pixelated, but it was good enough to act as an avatar.

Looking at the 3D model, the fictional engineer and the fictional construction manager discovered that the seven unpopular pieces were exposed daily to sunlight. One theory was that these rooms were too hot, a problem easily solved with the window coverings and the air conditioner. If people had just looked at the problem represented by numbers in a spreadsheet, they might have seen the problem what but not the Why. Some problems are like that.

The HoloLens experience shown in the demo is not available today. It's still a bit awkward and Microsoft is working on it. For now, HoloLens is intended for front-line workers who need access to information without moving their hands or looking at the topic they are working on. Remote workers may one day attend meetings using virtual reality or augmented reality.

The life cycle of a meeting

Some of us dread meetings because they eat time during the day. But they are always important; they are where decisions are made. This is where action items are created and assigned and people are motivated to achieve them.

Most good meetings come back to the organizer who is getting ready for the meeting, says Moneypenny. "If I am an excellent meeting organizer and I take the time to plan the right participants, and if I have received all the previous meeting notes and I assure myself that everyone has the right ones information, these are all good things. "

During the meeting, Moneypenny explains that the organizer must ensure that everyone can hear, that there are no obstacles to translation or communication and that the right actions are being taken. to be captured. After the meeting, they must ensure that the actions are distributed.

"These are all things that humans need to do today, so there is a lot of variation in the way things are done," said Moneypenny, "because I depend on how and when a human does it. " Robots support much of the planning work for meetings. "The artificial intelligence can enter and remove this variation and perform these tasks for us," she says.

Some of the Microsoft-related meeting issues seem pretty basic. But society is interested in exceeding boundaries and creating tools that make meetings more productive than most people have seen before. This can mean a situation in which you put on a headset and suddenly, you end up in a meeting that seems so real that you forget that you are in a virtual space.

It may be a distant goal, but it shows how meetings can be better. And because of its presence in the workplace, Microsoft is well positioned to continue.

[ad_2]
Source link