resonance - when interaction design meets music tech

Resonance Interaction design meets music technology 16 Mar 2016

Jason Mesut Founder, Resonant

Local Leader, IxDA London

@jasonmesut

I believe in…

Designing products and services that

fit our worlds, and enhance our lives

That could be

Cars, transportation, enterprise

software, mobile apps, web sites,

interactive TV, art installations or

retail experiences

and for some…

Music technology

Music technology…

Combines three of my passions:

Design. Technology. Music

My focus is


We’re going to start by going back

Let’s go back in time a little


Herbie Hancock jazz master, music innovator

+ Quincy Jones producer legend

+ CMi Fairlight - Sampler And Audio Workstation

Herbie + Quincy + Fairlight

http://www.youtube.com/watch?v=n6QsusDS_8A


Fairlight

Records

What are the things you ?

People

Activities

Apps?

Technics 1200 SH-101

OP-1Tenori-On

I love music tech

Q-Bert

Technics 1200

Q-Bert

Technics 1200

Roland SH-101

Q-Bert

Technics 1200

Milo Mesut

Yamaha Tenori-On

Teenage Engineering OP-1

Technics 1200 SH-101

OP-1Tenori-On

I love music tech

My music doesn’t pay the bills

But I also love design. It does.

5 years Industrial Design

years Digital User Experience

16

4 years Local leader

Interaction Design Association

L o n d o n

Subjects over 4 years Internet of Things Future of publishing Urban environment Smart materials Mobility / Automotive Clinical applications Smart home

Upcoming… Music interfaces Consumer healthcare

Looking for speakers and demos

April 27th 2016

L o n d o n

UI/UX/SD/DSUser Interface User eXperience Service Design Design Strategy

I could talk about all of the above

User Interface

UI/UX/SD

What is a

User Interface?

UI/UX/SD

User Interface

UI/UX/SD

User Interface

A human that uses a product or service: customer, employee, citizen, parent, child

UI/UX/SD

User Interface

A human that uses something

A medium for a user to understand

and control a product or service

UI/UX/SD

Bill Verplank

Interaction Design pioneer

Classic diagram by Bill Verplank

Assessment

Thought

Comprehension

UI Inputs Outputs

System Processes Rules

Attention

Perception

Intention

Cognition Sensing

Processes Control actions

Assessment

Thought

Comprehension

UI Inputs Outputs


Attention

Perception

Intention

Cognition Sensing


Reaction

Action

See Hear Touch Smell Taste

Icons from Volodin Anton, hunotika, Deivid Sáenz, Jakob Vogel via the Noun Project

Speed of perception

Amount of information

Interpretation accuracy

Ability to ignore

++

++

+

+

++

+

-

-

+++

+

-

- -

-

+

-

- -

-

+

-

- -

See Hear Touch Smell Taste

See Hear Touch Smell Talk

Handles and buttons

Continuous (analogous) and

Discreet (digital)

Inputs

The best interfaces combine

both handles and buttons

eg, Touchscreen, mouse

Bill Verplank

Graphical

User Interface

Allows users to

interact with

electronic systems

Visual Language

Motion Graphics

Information Design

Information Architecture

Interaction Design

Visual language

Motion graphics

Information Design

Information Architecture

Interaction Design

Still relevant

But there’s more to

UI than just GUI

Physical

Telephone

Voice

Embedded

Distributed

Chat

I want to talk you through

Music tech patterns

Future trajectories

Current challenges

Opportunities

Music technology

Patterns

Music technology is one of the

oldest technologies we have

Photos from the

Horniman museum

13 March 2016

Woodwind

Different classifications

of instruments

Chinese classification Based on material

Stone

Metal

Silk

Bamboo

Wood

Skin

Gourd

Earth

String instruments

Chordophones Stringed instruments Bowed, plucked, struck

Aerophones Wind instruments

Percussion Body of instrument vs. membrane

Electrophones Electromagnetic vs. digital vs. electric/acoustic

Hornbostel-Sachs classification Based on how the sound is made

String instrumentsHornbostel-Sachs classification





http://www.scrgeek.com/music/muInstTax2.html

String instruments





Pipe Aerophones (pitch dependent on pipe length)

Free Aerophones (pitch not dependent on pipe length)

Hornbostel-Sachs classification


String instruments





Pipe Aerophones Membrane vibrates

Idiophones Body of instrument or something inside vibrates



String instruments







Electrophones separate

form factor from sound

Assessment

Thought

Comprehension

UI Inputs Outputs


Attention

Perception

Intention

Cognition Sensing


We have more parameters

we can manipulate Changing or sequencing sounds beyond

playing notes

Precise continuous control

Knobs

ROLAND TB-303 BASSLINE

http://www.youtube.com/watch?v=Enyny7vXGi4

http://www.youtube.com/watch?v=Enyny7vXGi4

ALESIS MICRON

ALESIS MICRON - INSIDE OUT

BEHRINGER BCR-2000 IN ACTION

http://www.youtube.com/watch?v=pPsqoUsc9pI

http://www.youtube.com/watch?v=pPsqoUsc9pI

Coarse and precision continuous control

TURNTABLISM Knobs and faders together

CD SCRATCHING - NUMARK VINYL REPLICATION

Personal video (not available online)

Theremin

CD SCRATCHING - NUMARK VINYL REPLICATION


Continuous and discrete control

Strips and pads

http://www.synaptics.com/products/touchpad-family

http://createdigitalmusic.com/app/uploads/2013/01/baseangle.jpg

Discrete control with flexible arrangements

Grids

AKAI MPC

DJ SHADOW, DJ NUMARK, CUT CHEMIST + 3 MPCS

http://www.youtube.com/watch?v=IQcyLMa716k

http://www.youtube.com/watch?v=IQcyLMa716k

TENORI-ON

TENORI-ON - INSPIRATION

FOUR-TET + TENORI-ON

http://www.youtube.com/watch?v=M20ukzn_rsw

http://www.youtube.com/watch?v=M20ukzn_rsw

LITTLE BOOTS + TENORI-ON

http://www.youtube.com/watch?v=N6tLRCDqJ2c

http://www.youtube.com/watch?v=N6tLRCDqJ2c

TEHN + MONOME

http://vimeo.com/11435470


EDISON + MONOME



Rich desktop applications

DAWs

CONTEXTUAL TOOLS JUST AN ESC KEY AWAY


CAPS LOCK KEYBOARD


Discrete and continuous control

Touchscreens

REBIRTH

http://www.youtube.com/watch?v=WtQNazoMEUs

http://www.youtube.com/watch?v=WtQNazoMEUs

Figure

Modstep

Roli Noise app

Role Noise app

Different controls

Popular User Interface controls

StringsButtonsFaders Switches KnobsStrips

PluckPushPull StrumPress Tap Strike Bend

StringsButtonsFaders Switches

Turn

Knobs

Slide

Strips

Actions



Turn

Knobs

Slide

Strips

DiscreteFast, Continuous

Discrete (changing)

Flexible, Continuous

Precise, Continuous

Coarse, Continuous

Characteristics

Compositions of controls

Distribution of controls

KORG IMS-20

http://www.youtube.com/watch?v=p4Djp-dOFrg

http://www.youtube.com/watch?v=p4Djp-dOFrg

Interoperability

IoT interoperability is a mess

Music technology has been interoperable for over 30 years

CV Gate Control Voltage

Gate

1960s

MIDI Musical

Interchange Digital

Interface

1982

OSC Open Sound Control

1997-ish

The standard allowed different instruments to "speak" with each other and with computers, and this spurred a rapid expansion of the sales and production of electronic instruments and music software.

This interoperability allowed one device to be controlled from another, which reduced the amount of hardware musicians needed to own. MIDI's introduction coincided with the dawn of the personal computer era and the introductions of samplers and digital synthesizers.

The creative possibilities brought about by MIDI technology have been credited as having helped to revive the music industry in the 1980s.

INTRODUCING MIDI

http://video.google.com/videoplay?docid=2004591385588234774#

http://video.google.com/videoplay?docid=2004591385588234774#

What are the possible futures?

Future trajectories

Experimental

Theremin

Tiring

Coarse control

Confusion with natural gestures

Action != intention

Gestural

Theremin

We’re

attracted to

magic like

magpies

Physical interfaces

with digital brains

A recipe for connected products?

+ +Physical Product

Physical/ Digital Interface

Digital Services

Connected product

=

Aesthetic Visceral. Visual, sonic, feel.

Interactive Behavioural. Distribution of inputs and controls.

Experiential Reflective. Fit with person’s context and ecosystems.

Key levels of harmony

Components of integrated experience

+ +Physical Product

Physical/ Digital Interface

Digital Services

Connected product

=

Distributed UI

Interoperable components

Ableton Link

The knob is here to stay

Korg Minologue

Distributed controls and displays

Distributed controls

Distributed knobs

Knobs with embedded displays

One thing is for certain, although we

are entering a digital future,

the Knob is here to stay!

http://knobfeel.tumblr.com/

The future is already

here — it's just not very

evenly distributed. William Gibson, 1993

Cliche slide



Turn

Knobs

Slide

Strips

DiscreteFast, Continuous

Discrete (changing)

Flexible, Continuous

Precise, Continuous

Coarse, Continuous

Characteristics

Actions

The future will be like

today with little twists

What’s stopping progress?

Current challenges

Lack of progressive UIs

outside music tech Too much focus on touch screens

Few talking seriously

about music tech

Seen as niche or academic

Hardware is great but

expensive to manufacture

Music tech is hard to

learn + intimidating Tension between power and ease

of learning

Are we designing a product

or a component of a system?

Instrument vs. studio

1. Lack of progressive UIs

2. Few talking seriously about music tech

3. Hardware is great but expensive

4. (Music) tech is hard to learn + intimidating

5. Product vs system tension

Current challenges

Where should we focus?

The opportunities

Almost anything can be done physically or

digitally now. Now it’s about choice.

Understanding industrial Design

Create new instruments

Inspire peripherals

beyond music tech

Make and share what you do

The Internet of

Ubiquitous

Computing

Information

Appliance Things is

growing

Prototype out loud

Tinkertronics

SAM Labs

Create new interactions

out of old UI controls

and patterns

Slick knobs

Loads of buttons

Create more accessible

and usable solutions

Shun the shiny

Don’t be a magpie

to the magical

Adopt some UCD Understand your user’s needs and explore

different iterations with them

Understand

your users Understand their needs,

observe them creating

with their tools

Explore

prototypes Various fidelities, and

continually iterate

Refine based

on feedback Take on the learnings

and refine

Launch

and listen Never stop learning and

improving

1 2 3 4

Peter Kirn, November 2 2015

Enable new sounds,

inspire movements

Integrated automotive interface

ROLAND TB-303 BASSLINE

1. Create new instruments

2. Inspire peripherals beyond music

3. Make and share more

4. Create new interactions out of older UI

5. Improve accessibility and usability

6. Enable new sound movements

Opportunities

Remember…

We should be designing products and services

that fit our worlds, and enhance our lives

That means…

Get out of the labs and bring

your work to the world

Final words… Herbie

Thank you

[email protected]

@jasonmesut

slideshare.net/jasonmesut

meetup.com/ixda-london

e:

t:

s:

ixda:

http://slideshare.net/jasonmesut

http://meetup.com/ixda-london