Is Alexa Really Always Listening?

If you’ve ever considered buying an Amazon Alexa, you might have been on the receiving end of a skeptical glance or the cautionary warning, “Be careful, Alexa is always listening!” However, Alexa does have the alluring ability to make our days easier—dutifully performing thousands of otherwise time-consuming skills, like these things you didn’t know Alexa could do. Cue the internal conflict between sacrificing personal information for an extra hand (or, better yet, an extra brain) around the house. Whether you already live with an Alexa or are simply considering shacking up with one, everyone should be aware of the impact that this device can have on your privacy. Is Alexa really always listening? What does it mean for users if she is?

When does Alexa listen?

First and foremost, it is important to differentiate between listening and recording. Although the words are often used interchangeably when discussing Alexa’s capabilities, they actually have very different implications. According to Florian Schaub, Assistant Professor in the University of Michigan School of Information, the microphones in these smart speakers “are always listening, but, by default, they are only listening for the ‘wake word’ or the activation keyword.” Since the whole purpose behind the device is to instantly respond to users’ requests, it makes sense that Alexa is constantly scanning audio for its wake word, which can either be “Alexa,” “Computer,” or “Echo.” However, this does not mean that Alexa is always recording.

When does Alexa record?

Amazon’s list of frequently asked questions says that Alexa only begins recording your conversation upon hearing the device’s wake word. “So, when you say ‘Hey, Alexa,’” Schaub explains, all of “the audio gets analyzed and is being listened to by the microphones on the device, and only if the keyword ‘Alexa’ is detected, then everything that you say after that gets” recorded. After the device records, it uploads the audio to Amazon’s cloud, where they “have algorithms in the server that analyze the speech pattern and try to detect and identify the words you are saying.” While Alexa’s response may seem instantaneous, it actually has to work with Amazon’s cloud to comprehend varying accents, speech clarity, and vocabularies. This means that each time you wake up Alexa, the smart speaker is recording your conversations, “creating an automated transcript of what you are saying, and using that to fulfill your request,” says Shaub.

For example, let’s say you want to use Alexa to check the weather—and that you like to goof off in the “privacy” of your own home by asking funny questions to your Alexa. You might say, in a faux British accent, “Your Royal Highness, Queen Alexa, what is the weather?” Since Alexa is always listening, the device picks up and analyzes all of the audio that you just produced. However, it is only programmed to begin recording your words when it detects it’s trigger word, “Alexa.” The recording is then sent to the cloud, your accent is dissected, and the words are transcribed. Since Amazon’s server knows the location of the speaker, it identifies the weather in your area, sends it to the device, and Alexa reads the response aloud.

RELATED: These cool Alexa accessories are must-have items

You may not be surprised to learn that Alexa records your voice when you are specifically speaking to her. This seems reasonable enough, especially considering Amazon allows you to review and delete these recordings through Alexa’s privacy settings. For many of us, the issue arises when we hear that Alexa is always recording our conversations—even when we haven’t “woken” her up. Officially, Amazon advertises that its Alexa and Echo devices are not programmed to record or store information unless they are specifically activated. However, this has not always been the case.

Can Alexa go rogue?

The truth is, “there are all kinds of reasons the device might accidentally activate and record in situations where you’re not expecting it,” warns Schaub. Since Alexa’s “voice recognition is somewhat finicky,” Schaub says, “if you say something that sounds like Alexa, or if you just use Alexa in a conversation, then the device will activate.”

In 2018, the spine-chilling effects of what seemed like smart helper espionage were felt especially fervently by a family in Portland, Oregon. During a private conversation, the family’s Alexa woke up to a sound that resembled its wake word and began recording. Through a series of mistakes, Alexa misinterpreted the family’s conversation as a “send message” request and forwarded the audio recording to someone in the family’s address book—with the family having no idea that Alexa was on. While this may seem like an isolated incident, it is actually extremely common for Alexa to activate accidentally, begin recording, and upload the “eavesdropped” audio to the cloud. According to Bloomberg’s reporting, there are at least 100 transcripts of conversations uploaded to the cloud each day that Alexas have recorded without being purposely activated.

Bottom line? Although Alexa is programmed to only record audio when it is woken up, there is a strong possibility that your Alexa is activating accidentally. The more you use it, though, the better Alexa’s wake word detection will get, per Amazon’s official spokesperson: “As customers use their devices, we optimize far-field performance. [In the first half of 2019], we have improved wake-word performance by an additional 40 percent.” So, when your device is functioning properly, there is no evidence to believe Alexa is always recording. Plus, the smart helper definitely can’t listen when the microphone is manually turned off.

What are the privacy concerns?

You might be wondering why it even matters that Alexa is always listening or accidentally recording your conversations. It’s not like anyone can access these recordings, right? Wrong. Bloomberg reported that Amazon employs hundreds of people to read and annotate the transcripts that our smart devices upload to the cloud, which is intended to improve Alexa’s processing power and enable the device to respond more accurately to requests. While this is “not necessarily surprising to someone who is familiar with how machine learning works,” Schaub reasons, “it’s not something you would expect from a policy statement, such as ‘we may use this data to improve our services.’ That doesn’t really suggest that someone is listening to what you are saying to your Alexa in your bedroom.”

Amazon confirmed to Reader’s Digest that their employees only listen to “a fraction of one percent of interactions” in order to keep improving the Alexa experience. Thankfully, you can opt out of having your recordings viewed through Alexa’s privacy settings, and Amazon insists that those employees “do not have access to personal information including customer names, account numbers, device identifiers or location information.” However, Schaub stresses that “you really need to trust these companies that they are only recording when you say the wake word, because there is a live microphone in these devices. With any software update that comes to the device, that could change.” The expert in managing privacy in complex socio-technological systems fears that one day, these smart speakers “could be recording the whole time.” Whether or not this specific threat actually looms in the near future, we should all be aware of the reduced privacy that we are accepting when we purchase a smart speaker.

Additionally, Schaub highlights the other major privacy concern of having a smart speaker: “Whatever you say to the device might be used for additional information about you,” and “there are not enough limitations on what Amazon or other companies can actually do with this data.” What many of us don’t realize is that “speech is actually a very rich medium. Voice can tell you whether it’s a man or a woman speaking, it can give you an estimate of how old the person is speaking. If there’s background noise, you might know that there are children in the household. All of these things are interesting to companies like Amazon, but also Google and other companies, to get a better or more accurate picture of who you are in order to better target advertising and promotional materials to you,” explains Schaub.

So, before you plug in your new Alexa and train it to recognize your voice, you might want to consider whether you are comfortable revealing your personal information to Amazon.

Next, read about another gadget you can use your home: The Amazon Astro Home Robot.

Sources: