Are your phones listening to you?

Ever had a weird situation where you’ve been talking about something, then shortly after an advert pops up on your phone or web browser relating to something you just said? There’s enough anecdotal evidence to suggest that it could be going on, but we wanted to prove how possible and easy it was.

If you haven’t already seen our film on the BBC web site about mobile apps and other devices listening to you, it’s here http://www.bbc.com/news/technology-35639549.

The BBC came to us as they wanted to investigate the potential. Dave had a think about it and decided the easiest way would be to code up a rogue Android app that had permissions to the microphone.

Do users actually check the permissions when they install an app? Fairly unlikely, hence it would be easy to get an app on to a users phone that could listen.

Facebook, Twitter, Instagram and many other popular mobile apps have the ‘record audio’ permission. We’re not saying that they use it, but mic access is widespread

We had some concerns about this though:

First, we suspected that battery use would be high when constantly listening to the mic and uploading the audio to a voice-to-text service. Actually, this turned out not to be the case

Second, we were interested in ‘positive reinforcement’ – people often focus on the unusual, so coincidental advert displays could be presented as evidence of ‘snooping’

Anyway, my colleague Dave wrote the app, we installed it on to our own phone, hooked up to a service to take voice to text for us and presented the results in real time on screen, for the purposes of filming.

Was it rocket science? No, anyone with a modicum of Android or iOS coding skills could have done this.

It was just about proving a point; that it’s perfectly possible, that numerous mobile apps could snoop on your conversations if they wanted to.

A few more technical details that didn’t make the film

The media stream of the phone had to be muted, to avoid it making sounds whilst recording

Whilst we set keywords using the snooped voice text to try to generate custom adverts within the app, they didn’t actually work! We need to spend more time on this to get it fixed

There’s one point where we’re are all struggling to speak through suppressing laughter. The voice recognition is pretty good, but not perfect. We were in pieces because Zoe (the BBC reporter) was recorded as saying this:

www.hackXXitup.com-access_log-20160228:x.x.x.x – – [23/Feb/2016:15:05:41 +0100] “GET /recorded/wheat+allergy+to+come+up+with+a+wet+dream HTTP/1.1″ 404 312

No, she didn’t actually say that!

Conclusion

We can’t be certain that any apps are actually snooping on your speech, but it’s perfectly possible.

Loads of apps already have the required permission, and users generally blindly accept the permissions anyway.

The next step would be figuring out a way to review large numbers of apps in the stores to see if any are actually taking your voice data.

Should you be concerned? I’m certainly not overly worried about it, but if you do see an advert that relates to something unusual that you just said, do let me know.