ces 2017 demo – lessons learned and qt compositor …schd.ws/hosted_files/aglmmwinter2017/fe/ces...

CES 2017 Demo HMI - lessons learnedand

Qt Wayland Compositor API Overview

Tasuku Suzuki @ The Qt Company

tasuku.suzuki@qt.io

9 February 2017

›Born in Kushiro City, Hokkaido in 1979

›Met Qt in 2001

›Joined Trolltech in 2006›Moved to Nokia in 2008

›Left Nokia in 2012

›Had led Japan Qt Users’ Group for 4 years

›Joined The Qt Company in 2016

2

tasuku.suzuki@qt.io

›CES 2017 Demo HMI – lessons learned

›Qt Wayland Compositor API Overview

3

Two topics:

›single process – a monolithic QML application

4

ALS2016 Demo HMI

›We need brand new HMI design for the next demo!

›We want to separate process!›HomeScreen – system UI

›Dashboard app

›HVAC app

›Radio app

›MediaPlayer app

›Phone app

›…5

After ALS2016

›multi-process - HomeScreen + apps

6

CES2017 Demo HMI

›multi-process - HomeScreen + apps

7

CES2017 Demo HMI

›How long did it take?

›How many people had worked on it?

8

We rewrote DEMO HMI for CES 2017

›How long did it take?›Graphics for 1 month + Coding for 1 month!

›How many people had worked on it?

9

We rewrote DEMO HMI for CES 2017

›How long did it take?›Graphics for 1 month + Coding for 1 month!

›How many people had worked on it?›One designer and two engineers did most!

10

We rewrote DEMO HMI for CES 2017

CES 2017 Demo HMIlessons learned

›Done by the web design team in Linux Foundation

›First Specification for the team:

12

Graphic Design

›Done by the web design team in Linux Foundation

›First Specification for the team:

13

Graphic Design

14

The designer knew ALS2016?

›made ALS2016 Demo HMI run on Mac and sent it

15

We solved it in a smart way

›It took one month

›A zip archive - 1.4GB ›Half of them were in a directory called “_old”

›Heavy SVGs

›Same font files in 7 app directories

›What I needed was minimal SVGs…

16

Got new design on 30th Nov. 2016

1. convert to png›HMI_AppLauncher_Phone_Active-01.svg 1.1MB -> 30KB

›HMI_AppLauncher_Navigation_Inactive-01.png 1.1MB -> 32KB

2. Removed unnecessary elements in SVG data›HMI_HVAC_AirDown_Active.svg 765KB -> 4.8KB

›HMI_HVAC_Circulation_Active.svg 768KB -> 7.6KB

›69 pngs (to be replaced with svgs) + 162 svgs17

Reducing SVG file size

New HomeScreen

›Developed by Mentor Graphics›First commit was pushed on 1st June 2016

›UI was written in Qt/C++ & stylesheets

›https://wiki.automotivelinux.org/homescreen

19

New HomeScreen

›I replaced its UI with new QML one›“WIP: add Qt Quick HomeScreen UI” 2nd Dec.

›Jens integrated the new UI in the next week!

20

New HomeScreen

HomeScreen/HomeScreen/qml $ wc -l *.qml

53 Home.qml

57 MediaArea.qml

29 MediaAreaBlank.qml

85 MediaAreaMusic.qml

24 MediaAreaRadio.qml

63 ShortcutArea.qml

54 ShortcutIcon.qml

105 StatusArea.qml

43 TopArea.qml

91 main.qml

604 total ( – 17 * 10 = 434)

22

New HomeScreen UI

New AGL demo applications

›New feature in Qt 5.7

›Control set for embedded UI›Buttons

›Input Controls

›Indicators…

›Easy to use

›Easy to customize

›Good performance24

Qt Quick Controls 2

› /AGL/DemoApps/CES2017/imports/ qtquickcontrols2aglstyle/

›Apply style›export QT_QUICK_CONTROLS_STYLE=AGL

›QQuickStyle::setStyle("AGL");

›./app –style AGL

25

Qt Quick Controls 2 – AGL Style

›Separated from ALS2016 repo›Almost re-written from scratch

›With new graphic design

›Minimal Qt/C++ to run Qt Quick

›Some apps with its binding

›libhomescreen integrated

›$ make generates .wgt

26

New AGL demo applications

Application C++ (LOC) QML/JavaScript (LOC) SVG (files)

apps/controls (not for demo, for devleopers) 55 110 2

AGL/DemoApps/CES2017/Dashboard 46 381 7

AGL/DemoApps/CES2017/Phone 46 410 35

apps/hvac 84 383 22

apps/mediaplayer 344 254 33

apps/radio 62 232 10

apps/settings (ALPS did most of functionalities) 494 1439 25

apps/mixer 369 93 1

(Navi)

(POI)

27

New AGL demo applications

›Replace png files in HomeScreen with SVGs

›Move AGL/DemoApps/CES2017/* to /apps/

›Fix bug:›SPEC-395: Multimedia app does not proceed to next song …

›Virtual Keyboard integration

›MediaArea at the bottom of HomeScreen

28

TODO list:

›New graphic design

›New multi process architecture

›We completed it in a month

29

Summary: CES 2017 Demo HMI

›designers and developers can work together!

30

We can do it more efficiently if

›we can use more advanced technology like…

31

We can do it more efficiently if

Qt Wayland Compositor API

›A new feature in Qt 5.8 just released last month

›Write your own compositor in Qt/QML›No reference implementation

›Qt provides 10+ examples

›Easy to add animations/graphical effects

›Easy to extend wayland protocol

33

Qt Wayland Compositor API

import QtQuick 2.6

import QtQuick.Window 2.2

import QtWayland.Compositor 1.0

WaylandCompositor {

id: wlcompositor

WaylandOutput {

compositor: wlcompositor

window: Window {

width: 1024; height: 768; visible: true

Rectangle {

id: surfaceArea

anchors.fill: parent

}

}

}

Component {

id: chromeComponent

ShellSurfaceItem { onSurfaceDestroyed: destroy() }

}

WlShell {

onWlShellSurfaceCreated:

chromeComponent.createObject(surfaceArea, { "shellSurface": shellSurface } );

}

}

Minimal example

›WaylandCompositor {…}›For a compositor

›WaylandOutput {…}›For a screen output

›ShellSurfaceItem {…}›For a client surface shell

›WlShell {…} , XdgShellV5{…}, …›Shell interface to generate ShellSurfaceItem with wayland surface

35

Qt Wayland Compositor API

›Represents the compositor

›Usually the root object in qml file

›Needs to have an output and a shell extension to be useful

WaylandCompositor {

id: root

WaylandOutput {…}

WlShell {…}

}36

WaylandCompositor Element

›Manages a rectangular part of the compositor’s geometry that can be used for displaying content.

›Required properties: compositor and window

WaylandOutput {

compositor: root

window: Window {

width: 1024; height: 768; visible: true

Rectangle { id: surfaceArea; anchors.fill: parent }

}

}37

WaylandOutput Element

›Standard desktop-like implementation for ShellSurfaces›Maximization, fullscreen, resizing, moving, etc.

›Needs to handle the onSurfaceDestroyed signal

Component {

id: chromeComponent

ShellSurfaceItem {

onSurfaceDestroyed: destroy()

}

}38

ShellSurfaceItem Element

›Describes how surfaces should be shown and managed

›WlShell, XdgShellV5 and IviApplication are supported

›Only one signal you need to handle:›Create a ShellSurfaceItem when a ShellSurface is created

WlShell {

onWlShellSurfaceCreated: {

chromeComponent.createObject(surfaceArea, {“shellSurface”: shellSurface })

}

}

39

*Shell extension Elements

MorphChrome {

onDestroyAnimationFinished: {

destroy()

}

}

Make it fun!

›Show fullscreen

›Remove window decoration

›Add task bar›Thumbnail

›Close button

Make it useful

›Reuse QML files in HomeScreen

›Compositor-ized

›Add animations

›Add graphical effects

Make it AGL

›Allows you to write wayland compositor in QML

›Multi process, Multi view and/or Multi screen

›Build next generation UX w/ animations and effects

›Super easy, fast and fun!

43

Summary: Qt Wayland Compositor API

Thank you!Debugging CES2017 demo

with our CTO

on Shinkansen

44

tasuku.suzuki@qt.io