Hackathon "Hack The Echo"


Matthieu Boussard
Oct 16, 2015 EventIoT

Hackathon "Hack The Echo"

September, 22th was held at The family in Paris, the meetup "Meet the Echo". We had luck to be introduced to the new AI device from Amazon: Echo.

Introducing Amazon Echo

Amazon Echo is a new tool from Amazon based on voice. At any time, you can give an order to Echo and have it handled with Alexa, the software service around Echo. The promise with Echo is to have an accurate far field voice recognition, allowing interaction with the device (you do not need to be 10cm away from Echo to have your order properly understood). The second interesting point lies in its connection to Amazon Lambda. The orders given to Echo are processed and the results can execute the associated code on Amazon lambda (this code can also calls external APIs). The last point is that Echo is not only a microphone but it also has a speaker to interact with the user.

Pitch Contest

We knew about voice interface, like KITT, but we didn’t know yet much about Echo/Alexa. We've discovered that, after the presentation, was organized a "pitch contest", where we were invited to perform a 90sec pitch about a new application using Echo. Since, at craft ai, we are keen on agents and personal assistant, we've started by asking ourselves this simple question:

"In which application is the voice the only possible interaction with a virtual agent?".

In the following, we present two answers to this question, which we submitted at the event.

Murder party (won most innovative application price!)

Murder party games are party games wherein one of the player is secretly playing a murderer, and the other attendees must determine who among them is the criminal. One of the specificities of voice interaction is that it can be heard by other people around you. So, during the game, you could ask Echo:

  • “Alexa ? What did Stanley Kubic order at the bar at 19h ?” To have this information, you gave the other players the info that you knew that Stanley Kubik was at the bar at that time. How could you know it? Were you also there?

Starting from that, we imagined a kind of murder party game using Echo as central game master. Since it was only a pitch, we have just a few glimpses of what it could looks like, but at least we would like it to include the following points:

  • The voice interaction with Echo forces the players to give information to others.
  • Echo could simulate an interrogation, where it could play the role of witnesses (beware of what you ask, and what you are supposed to know !)
  • Echo could plays real-time sequence, where players could have to listen to a recorded phone call (and take private notes!)
  • It should also provide private info, for instance pushed on cellphone.
  • Echo will act as the game master, hence should: - push information according to the schedule of the game - to avoid a boring game, push hints when needed - ensure the game is been played according to the rules
  • Echo could also be the murderer !

This pitch won the most innovative price at the pitch contest (and the Amazon FireTV \o/)!

Story teller

Possibly oldest voice interaction of all is storytelling. The idea behind this second pitch is to help you tell an engaging story by focusing on your voice and letting Echo create the atmosphere around it, using connected lights, music and sound player etc.

  • Once upon a time (light off), in a dark forest (weak dark green light, play sound of wind in trees)....

If you had to do those tasks alone, you would certainly break the atmosphere you were trying to build, by having to stop (or to lose focus) on your story to select manually the proper soundtrack. Also if you took the (huge amount of) time needed to prepare your show, you will still have to follow the rhythm imposed by your script, and then you can’t slowdown/speedup your flow, you can’t discuss with your kid, and you can’t improvise. Thus we want to use Echo to catch orders from within the tale itself, and those orders will be forwarded to a craft agent, which will decide what to do. In the first version, the craft agent could simply connected multiple services and include basic rules (i.e which orders can be executed in parallel, which ones can be interrupted etc.). The next steps will be discovered during the hackathon !

After this pitch, we met the Chief of Open Innovation at Radio France, who was also interested in playing with interactions between voice, IoT and AI. We decided together to challenge the storyteller pitch during the HackTheEcho hackathon.

HackTheEcho : a 3 days hackathon with amazon Echo

October, 23th is the beginning of the HackTheEcho. We bought a LIFX connected light bulb. This device provides a REST API to change color, intensity and play effects like strobe, and fire. Our partner at Radio France Open Innovation will bring microphone and sound banks.

At craft ai we are currently preparing connectors to LIFX, lambda, and also to sound/video players like VLC. We are looking forward to plugging everything together this week-end and push the storyteller concept as far as possible and, with the help of Radio France, make an awesome demo.