Window Manager Application Guide

Revision: 0.5
TOYOTA MOTOR CORPORATION
20th/Mar/2018

Table of content

Introduction

This window manager implements simple layout switching of applications on multiple layers and with different layer layouts.

Intended audience

This document is intended for developers and system integrators who need to know, how the window manager works and how it is to be used.

Scope of this Document

This document covers the window manager that was implemented for TMC and delivered to the Automotive Grade Linux (AGL) project. It includes its implementation details, concepts of operation, configuration and usage.

It does not include

It is highly recommended to have a good understanding of these documents and projects before using the window manager.

Known Issues

Currently there is a one known issues:

  • Only single-surface Qt applications are support through the libwindowmanager library. This is a limitation of how Qt creates surface IDs for the ivi-application interface.

External libraries

This project includes a copy of version 2.1.1 the excellent C++11 JSON library by Niels Lohmann.

Client Library

A client library implementation that internally uses the libafbwsc, is provided in the libwindowmanager.

Concepts

The window manager implements a couple of concepts in order to allow efficient implementation.

Layers

Layers are entities that are stacked on top of each other. Each layer has an ID which is used for the ivi-controller interface, but this ID also implicitly specifies its stacking order, from lowest to highest.

Layers are always full-screen. We do not use layer dimensions as a way to setup the scene, rather - each layer has a layout attached to it, which specifies an area that is used by surfaces to draw on.

Additionally, layers will generally leave surfaces on below layers activated, and only disable surfaces on layers the are above the currently used layer.

It is possible to deactivate these surfaces on lower layers explicitly using the DeactivateSurface API call.

Surfaces

Surfaces are placed on layers according to their name. The surface will then be resized to dimensions, according to the layer's layout configuration.

Configuration

The window manager is configured with the layers.json configuration file, by default it is searched in ${AFM_APP_INSTALL_DIR}/etc/layers.json. However, if /etc/xdg/windowmanager/layers.json exists, it will be used instead. The window manager will use a default configuration unless a configuration file is found in one of these locations.

A sample configuration is provided with the window manager implementation, this sample is installed to ${AFM_APP_INSTALL_DIR}/etc/layers.json.

Note: Currently, window manager doesn't block the application displaying because "Fallback" is set by default. If the "Fallback" is not set in layers.json, window manager blocks the application displaying. In such a situation, you have to add your role(application name) at "role" in layers.json.

Configuration Items

This section describes configuration items available through layers.json. It will do this, by first providing an example, and then going into its components.

main_surface

"main_surface": {
   "surface_role": "HomeScreen",
},

The main_surface object describes a surface that will internally be treated as the main surface - usually this mean HomeScreen. The only special handling this surface receives, is that it is not allowed to deactivate it. Placement of this surface on an layer is done by the other configuration described below.

  • surface_role this configuration item specifies the name of the main surface. Set this to e.g. HomeScreen.

mappings

This configuration item is a list of surface-name to layer mappings.

surface to layer mapping

"mappings": [
  {
     "role": "^HomeScreen$",
     "name": "HomeScreen",
     "layer_id": 1000,
     "area": { "type": "full" },
     "comment": "Single layer map for the HomeScreen"
  },
  {
     "role": "MediaPlayer|Radio|Phone|Navigation|HVAC|Settings|Dashboard|POI|Mixer",
     "name": "apps",
     "layer_id": 1001,
     "area": { "type": "rect", "rect": { "x": 0, "y": 218, "width": -1, "height": -433 } },
     "comment": "Range of IDs that will always be placed on layer 1001, negative rect values are interpreted as output_size.dimension - $value",

     "split_layouts": [
        {
           "name": "Navigation",
           "main_match": "Navigation",
           "sub_match": "HVAC|MediaPlayer",
           "priority": 1000
        }
     ]
  },
  {
     "role": "^OnScreen.*",
     "name": "popups",
     "layer_id": 9999,
     "area": { "type": "rect", "rect": { "x": 0, "y": 760, "width": -1, "height": 400 } },
     "comment": "Range of IDs that will always be placed on the popup layer, that gets a very high 'dummy' id of 9999"
  }
]

Each mapping defines the following items to map corresponding surfaces to a layer.

  • role defines a regular expression that application drawing names are matched against. If applications match this regular expression, the surface will be visible on this layer.

  • name is just a name definition for this layer, it has no functional use apart from identifying a layer with a name.

  • layer_id specifies which ID this layer will use.

  • area is an object that defines the area assigned to surfaces.

  • split_layouts is an optional item, that - if present - defines a number of possible split-screen layouts for this layer.

Area

Areas can be either full or rect, whereas full means a full-screen layer, this is mostly useful for the main_surface or HomeScreen layer. rect declares a layer drawing area specified as a rectangle with start coordinates x and y as well as its dimensions width and height.

The dimensions can be specified relative to the screen dimensions. For this negative values for width and height must be used.

For example, a full-screen surface can have the following rect definition:

"rect": { "x": 0,
          "y": 0,
          "width": -1,
          "height": -1 }

A surface that leaves a 200pixel margin on the top and bottom can use the following rect definition:

"rect": { "x": 0,
          "y": 200,
          "width": -1,
          "height": -401 }

So the expression for the actual surface dimensions when using screen-size-relative values will be:

actual_width = screen_width + 1 + width
actual_height = screen_height + 1 + height

Or in other words, to leave an N wide border around a surface, the actual value in the dimension configuration needs to be -N - 1, and appropriate offsets need to be set for x and y.

split_layouts

This configuration item allows the specification of split-screen layouts on layers for certain surfaces.

A split screen layout always has a main surface and a sub surface. In order to enter a split screen layout, first the main surface of the layout must be activated, and then the sub surface. In order to disable the split layout, one of the two participating surface must be deactivated (or a surface on a layer below the current one must be activated).

"split_layouts": [
   {
       "name": "Navigation",
       "main_match": "Navigation",
       "sub_match": "HVAC|MediaPlayer",
   }
]

A split layout object has the following attributes:

  • name defines its name, it has no actual function other then a way to identify this split layout.

  • main_match is a regular expression that matches for the main surface of this split layout.

  • sub_match is a regular expression that matches for the sub surface of this layout.

In the above example only the surface with drawing name Navigation will be used as the main surface, and the surfaces with drawing name HVAC or MediaPlayer can be used as a sub surface for this layout.

The names must still match the layer's role match!

Building and Running

Dependencies

Build dependencies are as follows:

  • afb-daemon >= 1.0

  • libsystemd >= 222

  • wayland-client >= 1.11

  • wayland-ivi-extension >= 2.0.2 (until eel, wayland-ivi-extension >= 1.13)

  • cmake >= 2.8

Supported environment

Item Description
AGL version Electric Eel
Hardware Renesas R-Car Starter Kit Pro(M3)

Build Configuration

Download recipe If repo is already done, please start with git clone

$ mkdir WORK
$ cd WORK
$ repo init -u https://gerrit.automotivelinux.org/gerrit/AGL/AGL-repo
$ repo sync

Then you can get the following recipe.

  • meta-agl-devel/meta-hmi-framework/recipes-graphics/agl-service-windowmanager-2017

  • meta-agl-devel/meta-hmi-framework/recipes-graphics/libwindowmanager

Bitbake

$ source meta-agl/scripts/aglsetup.sh -m m3ulcb agl-demo
$ bitbake agl-demo-platform

Implementation Notes

The window manager is implemented as a app-framework-binder binding. That means, the build produces one shared object that exports a binding interface.

Structure

The implementation is loosely split across the following source files:

  • main.cpp: The program entry point as used by the afb-daemon. This file defines the afbBindingV2 symbol that is used by the afb-daemon in order to load a binding. It also defines the wayland fd event dispatcher and some globals to be used (as context for the afb calls we receive).

  • app.cpp / app.hpp: This is the main window manager logic implementation.

  • config.cpp / config.hpp: Very simple configuration item interface.

  • controller_hooks.hpp: hook functions called by the wayland controller to call into the window manager instance. Only a very limited number of events are passed to the window manager, which allowed the usage of such a simple interface.

  • json_helper.cpp / json_helper.hpp: Smaller json related helper functions.

  • layers.cpp / layers.hpp: Actually hold all the data from layers.json configuration, do some transformations and service the window manager implementation.

  • layout.cpp / layout.hpp: Very simple layout state for the implementation of split layouts and tracking of the surfaces involved.

  • policy.hpp: PolicyManager implementation stub. Gets passed the current and new layout on layout switch and can decide upon it being valid or not.

  • result.hpp: Simple result class around std::experimental::optional that additionally can hold a char const * to describe the error.

  • util.cpp / util.hpp: general utility functions and structs - and preprocessor definitions (e.g. log*() to AFB logging functions.

  • wayland_ivi_wm.cpp / wayland_ivi_wm.hpp: A C++ object-oriented libwayland-client wrapper. It is instanced in main.cpp and handles all our wayland needs. These files are in master. In eel, the name of these files are wayland.cpp / wayland.hpp

Sequence

To understand the sequence between application and window manager, refer to the spec document.

Binding API

Each function returns a reply containing at least a failed or successful result of the call, additionally, when calls return something, it is noted.

LibWindowmanager

This is the public interface of the class LibWindowmanager.

class LibWindowmanager
{
public:
    LibWindowmanager();
    ~LibWindowmanager();

    enum EventType {
       Event_Active = 0,
       Event_Inactive,

       Event_Visible,
       Event_Invisible,

       Event_SyncDraw,
       Event_FlushDraw,
    };

    int init(int port, char const *token);

    // Window manager API
    int requestSurface(json_object *object);
    int requestSurfaceXDG(json_object *object);
    int activateSurface(json_object *object);
    int deactivateSurface(json_object *object);
    int endDraw(json_object *object);
    int getDisplayInfo(json_object *object);
    int getAreaInfo(json_object *in_obj, json_object *out_obj);

    int getAreaInfo(const char *label, json_object *out_obj);

    void set_event_handler(enum EventType et, handler_fun f);

};

Methods

init(int port, char const *token)

Initialize the Binding communication.

The token parameter is a string consisting of only alphanumeric characters. If these conditions are not met, the LibWindowmanager instance will not initialize, i.e. this call will return -EINVAL.

The port parameter is the port the afb daemon is listening on, an invalid port will lead to a failure of the call and return -EINVAL.

requestSurface(json_object *object)

args: { 'kKeyDrawingName': 'application name' } This method requests a surface with the label given from the Window Manager. It will return surface id a client application can use, and -errno on failure. Additionally, on the standard error, messages are logged to help debugging the issue.

requestSurfaceXDG(json_object *object)

args: { 'kKeyDrawingName': 'application name', 'kKeyIviId': 'ivi id' } This method is mainly intended for xdglauncher that controls xdg application such as chromium. It will return surface id xdglauncher uses, and -errno on failure. Additionally, on the standard error, messages are logged to help debugging the issue.

activateSurface(json_object *object)

args: { 'kKeyDrawingName': 'application name', 'kKeyDrawingArea': 'layout' } This method is mainly intended for manager applications that control other applications (think an application manager or the HomeScreen). It instructs the window manager to activate the surface with the given label.

This method only is effective after the actual window or surface was created by the application.

deactivateSurface(json_object *object)

args: { 'kKeyDrawingName': 'application name' } This method is mainly intended for manager applications that control other applications. In adition, this is for applications that overrides other applications such like popup message. In this case, popup surface requests to be hidden. It instructs the window manager to deactivate the surface associated with the given label. Note, that deactivating a surface also means to implicitly activate another (the last active or if not available main surface or HomeScreen.)

This method only is effective after the actual window or surface was created by the application.

endDraw(json_object *object)

args: { 'kKeyDrawingName': 'application name' } This function is called from a client application when it is done drawing its surface content.

It is not crucial to make this call at every time a drawing is finished - it is mainly intended to allow the window manager to synchronize drawing in case of layout switch. The exact semantics are explained in the next Events Section.

getDisplayInfo(json_object *object)

args: { } This function gets the display information as follows: - width[pixel] - height[pixel] - width[mm] - height[mm]

It outputs the display information for json_object in the argument as follows: {"width_pixel": int value of width[pixel], "height_pixel": int value of height[pixel], "width_mm": int value of width[mm], "height_mm": int value of height[mm]}

It should be called after calling init(). It should not be called in the event handler because it occurs hang-up.

NOTE

It uses wl_output::geometry() for getting physical width[mm] and height[mm] of the display, but the value is different with measured value.

  • value from wl_output::geometry(): width:320 height:520
  • measured value : width:193 height:343

getAreaInfo(json_object in_obj, json_object out_obj)

args1: { 'kKeyDrawingName': 'application name' } args2: { } This function gets the information of area drawn by the application as follows: - x-coordinate - y-coordinate - width - height

It outputs the area information for json_object in the 2nd argument as follows: {"x": int value of x-coordinate, "y": int value of y-coordinate, "width": int value of width, "height": int value of height}

It should be called after calling activateSurface(). It should not be called in the event handler because it occurs hang-up.

NOTE

The same information can given by SyncDraw event.

getAreaInfo(const char label, json_object out_obj)

args1: String of application name args2: { } This function is same with getAreaInfo(json_object *in_obj, json_object *out_obj), but only has difference of 1st argument.

set_event_handler(enum EventType et, handler_fun f)

This method needs to be used to register event handlers for the WM events described in the EventType enum. Only one hendler for each EventType is possible, i.e. if it is called multiple times with the same EventType the previous handler will be replaced.

The func handler functions will receive the label of the surface this event is targeted at.

See Section Events for more detailed information about event delivery to client applications.

Errors

Methods returning an int signal successful operation when returning 0. In case of an error, an error value is returned as a negative errno value. E.g. -EINVAL to signal that some input value was invalid.

Additionally, logging of error messages is done on the standard error file descriptor to help debugging the issue.

Usage

Initialization of LibWindowmanager

Before usage of the LibWindowmanager, the method init() must be called once, it will return -errno in case of an error and log diagnostic messages to stderr.

Request a surface

When creating a surface with Qt - it is necessary to request a surface from the WM, internally this will communicate with the window manager binding. Only after requestSurface() was successful, a surface should be created.

This is also true for QML applications, where only after the requestSurface() should the load of the resource be done. The method returns surface id a client application can use after the surface was requested successfully.

Workings of requestSurface()

LibWindowmanager::requestSurface() calls the AFB binding verb requestsurface of the windowmanager API. This API call will return a numeric ID to be used when creating the surface. This ID is never explicitly returned to the client application, instead, it is set in the application environment in order for Qt to then use it when creating the surface.

With the current Qt implementation this means, that only one surface will be available to client applications, as subsequent windows will increment this numeric ID internally - which then will lead to IDs that cannot be known by the window manager as there is no direct communication from Qt to the WM.

Events

Events are a way for the Window Manager to propagate information to client applications. It was vital for the project to implement a number of events, that mirror functionality that is already present in the wayland protocol.

All events have the surface label as argument - a way to enable future multi-surface applications.

As already stated above, this is currently not possible with the way Qt implements its surface ID setting.

Active and Inactive Events

These events signal an application that it was activated or deactivated respectively. Usually this means it was switched visible - which means the surface will now be on the screen and therefor continue to render.

  • Active(json_object *object) args: { 'kKeyDrawingName': 'application name' } Signal that the surface with the name kKeyDrawingName is now active.

  • Inactive(json_object *object) args: { 'kKeyDrawingName': 'application name' } Signal that the surface with the name kKeyDrawingName is now inactive. This usually means, the layout got changed, and the surface is now considered inactive (or sleeping).

Visible and Invisible

These events signal an application that it was switched to be visible or invisible respectively. These events also are handled implicitly through the wayland protocol by means of wl_surface::enter and wl_surface::leave events to the client.

  • Visible(json_object *object) args: { 'kKeyDrawingName': 'application name' } Signal applications, that the surface with name kKeyDrawingName is now visible.

  • Invisible(json_object *object) args: { 'kKeyDrawingName': 'application name' } Signal applications that the surface with name kKeyDrawingName is now invisible.

SyncDraw and FlushDraw

These events instruct applications that they should redraw their surface contents - again, this is handled implicitly by the wayland protocol.

SyncDraw is sent to the application when it has to redraw its surface.

FlushDraw is sent to the application when it should swap its buffers, that is signal the compositor that its surface contains new content.

  • SyncDraw(json_object *object) args: { 'kKeyDrawingName': 'application name', 'kKeyDrawingArea': 'layout', 'kKeyDrawingRect': { "x": int value of x-coordinate, "y": int value of y-coordinate, "width": int value of width, "height": int value of height } } Signal applications, that the surface with name kKeyDrawingArea needs to redraw its content in the layout with name kKeyDrawingArea - this usually is sent when the surface geometry changed. And the area position and size are included with name kKeyDrawingRect.

  • FlushDraw(json_object *object) args: { 'kKeyDrawingName': 'application name' } Signal applications, that the surface with name kKeyDrawingArea can now be swapped to its newly drawn content as the window manager is ready to activate a new layout (i.e. a new surface geometry).

Sample

In order to enable application to use the WM surface registration function the above described steps need to be implemented.

As a minimal example the usage and initialization can look like the following.

Repo: apps/agl-service-homescreen-2017 Path: sample/template/main.c