Amazon ran a commercial on this year’s Super Bowl that pretended its digital assistant Alexa had temporarily lost her voice. It featured celebrities like Rebel Wilson, Cardi B and even the company’s chief executive, Jeff Bezos.


While the ad riffed on what Alexa can say to users, the more intriguing question may be what she and other digital assistants can hear — especially as more people bring smart speakers into their homes.


Amazon and Google, the leading sellers of such devices, say the assistants record and process audio only after users trigger them by pushing a button or uttering a phrase like “Hey, Alexa” or “O.K., Google.” But each company has filed patent applications, many of them still under consideration, that outline an array of possibilities for how devices like these could monitor more of what users say and do. That information could then be used to identify a person’s desires or interests, which could be mined for ads and product recommendations.


In one set of patent applications, Amazon describes how a “voice sniffer algorithm” could be used on an array of devices, like tablets and e-book readers, to analyze audio almost in real time when it hears words like “love,” bought” or “dislike.” A diagram included with the application illustrated how a phone call between two friends could result in one receiving an offer for the San Diego Zoo and the other seeing an ad for a Wine of the Month Club membership.


Some patent applications from Google, which also owns the smart home product maker Nest Labs, describe how audio and visual signals could be used in the context of elaborate smart home setups.


One application details how audio monitoring could help detect that a child is engaging in “mischief” at home by first using speech patterns and pitch to identify a child’s presence, one filing said. A device could then try to sense movement while listening for whispers or silence, and even program a smart speaker to “provide a verbal warning.”


A separate application regarding personalizing content for people while respecting their privacy noted that voices could be used to determine a speaker’s mood using the “volume of the user’s voice, detected breathing rate, crying and so forth,” and medical condition “based on detected coughing, sneezing and so forth.”


The same application outlines how a device could “recognize a T-shirt on a floor of the user’s closet” bearing Will Smith’s face and combine that with a browser history that shows searches for Mr. Smith “to provide a movie recommendation that displays, ‘You seem to like Will Smith. His new movie is playing in a theater near you.’


MORE https://www.nytimes.com/2018/03/31/b...ssistants.html




...I am hoping someone will say this is an April Fools prank....