Design and innovation week – day 3

Day 3 

IMG_6020 (2)

We Work entrance

Ben Saurer,  Clearleft – Beyond boxes and arrows. Designing with voice

It was back to the WeWork co working space for a talk from Ben Sauer from Clearleft.. They are a leading design, strategy and education agency. They make ‘Silverback’ the mac based OSX user testing software.

IMG_5980 (2)

 Voice User Interfaces (VUI)

Ben made some key points regarding VUI’s

  • They have no affordance (how they work signaled by what they look like e.g. door handle = pull , push plate on door = push)
  • The burden of input falls on us as the signal generator not an interface. However he says that VUI is an efficient short cut and there is an opportunity for VUI beyond the use case.
  • Still in its infancy, VUI is good at goldfish moments e.g.Do this, set that, add to calendar.

IMG_5982 (2)

What would be really useful is when we can do ‘if this , then that ‘commands. E.g. if I am in the garden and the doorbell rings, notify me that someone is at the door.’

It’s an interface that demands attention. Ben talked about the movie ‘Her’ by Spike Jonez being a great example of what VUI could be like in the future, talking naturally and adding tone and nuance to what could be usually the mechanistic reading of e-mail by a screen reader.

Talking about Apples ear pods, Ben thought the point beyond wireless headphones was having the Siri assistant available all the time just like in ‘Her’.

“We will keep trading privacy for convenience”

There are two modes of VUI, command active (hey Siri), the hound(go find x) and always on (Ben likening this to the Stasi secret police).

Alexa (not in Australia officially yet, like high speed fiber to the door internet friends have here sigh,) is Amazons ‘play for ubiquitous computing.

Ben talked about WeChats apps bots, whilst having a conversational interaction, still had many menus placed consistently for support. Conversational VUI is still evolving and overhyped. You still chat and use the menus but its still another channel to talk to an organization. Choosing tone and voice of your conversation is important as how people gauge the bot is how they gauge your brand.

Ted Livingston; at an event Ben attended, talked about having to take care of the overall experience when you are in chat, AI is over hyped, and so far without domain knowledge. VIU apps failing and are mainly terrible. Ben then played back and example of a search app that read back your search in a terribly mechanistic voice.

Some examples of VIU that were not terrible included multi modal offerings such as Google Home, and a combination of Siri and Remote on newer Apple TVs.

What would make VIU revolutionary?Don’t put a tiller from a boat on a car.

Shift our design mindset Ben advises. The problem, Ben thinks, is that we bring our design mindset, preconceptions and baggage from other platforms, mobile, touch, desktop etc to new ones projecting biases that hamper, hobble or are not applicable to VUI.

Changing of mindset of design and engineering is needed.

Changing society to get used to ubiquitous VUI is staring with Google home advertising showing families in the room together with individuals talking to the VUI. Google realized ‘bringing people together’ was better than being creepy; if it was a loner person talking to google home like their cat.


Google Home Advertisement

Its like the introduction of mobile phones for the first time in the 80’s; people were initally creeped out by yuppie people on phones, everywhere, no longer tied to a ‘land line’.

Going beyond the use case

VUI will thrive when it can connect things together. “I’m driving to Richmond, and I need to buy a bottle of wine”

Connective tissues

Platforms need to be less ‘walled gardens’ and more interconnected. Siri doesn’t talk to Alexa and vice versa. Currently, we can’t connect ‘job doers’ so eco systems don’t play well together. We can’t tell Echo to ‘call mum, on skype, in the lounge’

We need API’s to interact and link these systems together. Connectivity is coming says Ben, for example Siri kit, signaling that it is starting to happen.

We’ll know its there; when Siri can call us an Uber.

Shazam; shouty app – brands and interaction are largely gone with VUI and a bad voice UI experience becomes how you perceive brands and services.

Ben went on to say that ‘brands will not matter in the future’ once they are integrated into VUI experiences, which, before he qualified that statement led to some interesting discussion in the group. Our group thinks that the voice interface is the brand unless things are so well connected we don’t think about or worry who played what part (all the connected services and brands via that connective tissue) of the experience to “call mum, in the lounge, on Skype”.

What we may want to know is who has our data and how secure it is. But it left us thinking when Ben said that; the brands and technological strata we now don’t care about from 20 years ago, will be the ones people 20 years from now (the ones we care about, interact and work with now) won’t care about.

What we lose


We are continually trading off privacy for convenience. Surveillance and co-veillance (active in our self declaration of our personal data. E.g. find friends)

(he referred us to Matt Novak, Kevin Kelly for more on this area)

VUI considerations – screens give us immediate way finding, its built into the mode. Voice is like being in a dark room. You need to ask is if VUI appropriate for the context you are designing in. E.g. an app to help find the right undergarment , or help you in the kitchen with your hands full.

Roles of VUI

  • Secretary  – dates, notes , emails
  • Bouncer – security for our banking for example . HSBC feeling confident they have got VUI security for banking almost nailed.
  • Gopher – go find me (assistants are still gophers)

VUI is still transactional, not conversational

Building VUIs

Define a VUI persona for your brand. It needs to have a tone of voice with you being the script writer.

Generate the dialogue, type it out and write it like a scene. Question it, rehearse it like improve.

Create a flow like your are designing an IVR. Do ‘Wizard of OZ testing’ read the script with a user and a a person being the VUI.

VUI limitations

It is not good at

  • Large amounts of input
  • Presenting choice (screens are better)
  • Natural breaks in language
  • Fuzzy tasks
  • Recording issued commands
  • Security
  • Privacy
  • Noisy environments
  • Follow up qustions

VUI opportunities

  • Apps
  • Platform integration
  • iOS and Android
  • Accepting criteria
  • Delivering search results

ASOS clothing for example, there are so many interactions on screen, an easy verbal choice would help with that interaction. Pennies; the app , telling it what your expenses were rather than typing them in might be easier.

VUI may render some screen based content redundant. (ref Ursula Franklin)

Technology is not about things, but what it does for people and their habits, rituals , practices and outcomes.

Innovation game, something broken, some popular platform. Ben took us through an example of an innovation game he and his colleagues use for problem solving or idea generation.

  • Post its
  • Problem  on one column, popular thing (service, product, tech company, app etc)  on an other column. See and swap , looking and talking about how that thing can help. Its like improve and spark thinking on how that or something related could help with that problem .

They don’t have to make sense. If they make sense or half sense take a pair and pitch, critique it.


  • Steal
  • Turn
  • Iterate
  • Reorder

This game creates an ‘eco system of evolution’, each pitch and feedback loop iterates on the core ideas taking more time to solve the pain points.

Innovation Game Instructions – Credit: Future London Academy – Carmen S


Gaming hackathons used this with game mechanic and platform/game as columns . e.g. poker | Zelda

Innovation is slow

  • Slow user research is needed to discover deep unmet needs and insights
  • Validate problems as you go
  • Product pace is faster, different speed to find good answers.

(Hence UX Research in Digital services = running discovery studies outside the product road map)

‘How AI can help you be more creative’ – talk from Luba Elliott

Luba Elliot @elluba is a creative AI artist. She shared with us several projects she had  been involved with showcasing the participation of AI on several projects.

Beyond the fence – whilst not strictly 100% AI written, the script and music had been co written, an AI generating and people curating dialogue and music. The AI had been fed 200 musicals from 80’s-90’s and had been performed to mixed and lukewarm reviews.  Whilst not terrible, it did not really stand out from other average shows and broke no new ground, but was remarkable in the way it had been constructed.

Another short film ‘Sun spring’ on YouTube written entirely by an AI was intriguing in the sense the dialog almost made sense and had a strange internal logic of its own .it felt like it was going to make sense at any minute but never did. It was produced for the Sci Fi London 48 hour film challenge.


The third project she shared was ‘auto-encoding Bladerunner’, taking the original film and having it reconstructed frame by frame as the AI ‘understood’ it. It was so like the original it had to be taken off the internet for copyright reasons.

Lawrence Lek ‘ Geomancer’ was also another film that was recommended.

Turning Test performance art – people reading to each other . One or both reading AI generated scripts. People pretending to be ‘uncanny valley’ or ‘siri’

Interestingly where it was the Church commissioning art in the past, organisations like Mircosoft and Google in a sense, are.

AI in creative industry example; the Prisma’ app was mentioned.

‘The AI and art aims to achieve an outcome that neither party could have come up with individually’, so we’re in no immediate danger of being replaced for now.


Walking to the next venue we had lunch at the ‘Borough markets’ before proceeding to No 1. Tower Bridge on the banks of the Thames river. I possibly had the best fresh pasta dish I’d had in years. And walking 5 Km per on average this week I could burn it off lol.

IMG_6027 (1)

‘Future of AR’ – talk from Blippar

We walked and went to the No 1. Tower Bridge building and visited the agency ‘Blippar’,  with the augmented reality (AR) platform of the same name’ Blippar. (Sadly this was during the events at Westminster Bridge not too far away eerily reminiscent of the Melbourne Bourke Street Mall event of late.)

AR should be worth 90 Billion (GBP) by 2020 according to Blippar

IMG_6050 (1)

  • Machine learning
  • Hololens
  • Vive

Mentioned as technology Blippar are working with.  Blippar are helping clients with AR content around brands and has an education arm that manages AR content for education and training.

Blippar are looking at phones as the delivery mode until other platforms such as glasses technology catches up. (Hurdle to overcome will be the negative publicity google glasses had (no one wants to be a “Glasshole’)

aCommerce – Blippar see a an opportunity to place advertising opportunities over real places and locations. Walking around you’d see messages from your favorite brands if you pass by and have the camera viewer seeing them.

In a concept video, the phones camera sees for the person walking along offers and content informed by surrounds, brands and context.  Offers, info etc. AI helping platform how to interpret what we see , Blippar ‘Visual browsers’ go walk, see offers will have to be curated by peoples preferences or they could be overwhelmed and distracted by the additional information overlaying reality

At the office I hovered my Blippar app over the Guardians of the Galaxy demonstration page and got some great AR content. It ‘sticks’ to the phone when you move it to a comfortable viewing position so you don’t have to stay in the same fixed hand position. This may be for some people ‘not’ a pure AR experience, but a great one ergonomics and a longer engagement as it  ‘grabbed’ the AR content and allowed it to be used it comfortably.

IMG_6064 (1)

Facial recognition will also play a part, choose your preferences and organize or the danger of cognitive overload and navigating a real space could be a visually crowded proposition.

Blippar are keen to putit out there, seeing how people use it, this addition of a digital layer to our world. We will have to mediate it and curate it ..and there may be modalities;you as a tourist vs shopping vs commuting ..

AR for learning demonstrated adding context and content to posters and learning cards in class rooms . Visualize chemistry, geometry  etc. They see applications in Education and corporate training, a scaleable assistant.

The platform is heavily monetized; people pay per region and platform in a subscription model.

Use this a link to the app, hover the phone camera over some Blippar content behind a picture and interact. It pulls put their demo content online or you can try theirs here at their showcase page. It will be interesting to see if to ‘blip’ something in AR will be as ubiquitous as googling something.

Their next concept is face profiles in AR (see data about a person on your phone by pointing it at them.. Scarily this looks like a precursor the the concept in Black Mirror Season 3 episode one. (See a preview here) compare this the to the Blippar face recognition imagery in this article and you can see the similarities. In Black Mirror, the AR is built into your eyes so no need to hold your phone up to someone. Convenient eh?

Blippar – facial recognition…


…reminded me of – Black Mirror – near future fiction facial recognition

Take away

AR has potential to inform your daily life, I’d hazard you’d need it contextually aware and comfortable to use ergonomically and socially..

Back to Day 2     or    on to Day 4

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s