Multi-Screen Window Placement

Draft Community Group Report,

This version:
https://webscreens.github.io/window-placement
Test Suite:
https://github.com/web-platform-tests/wpt/tree/master/screen_enumeration
Issue Tracking:
GitHub
Inline In Spec
Editors:
(Google Inc.)
(Google Inc.)

Abstract

This document defines a web platform API that allows script to query the device for information about connected displays, and additional APIs to position windows relative to those displays.

Status of this document

This specification was published by the Second Screen Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. Introduction

This section is non-normative.

Operating systems generally allow users to connect multiple screens to a single device and arrange them virtually to extend the overall visual workspace.

As multi-screen devices and applications become a more common and critical part of user experiences, it becomes more important to give developers information and tools to leverage that expanded visual environment. This document describes some possible incremental solutions enabling web application developers to make use of multi-screen devices, in order to facilitate discussion and seek concensus on a path forward.

Write this section.

1.1. Usage Overview

Write this section.

1.2. Motivating Use Cases

Write this section.

2. Concepts

Finish up this section.

2.1. Screen pixel

A screen pixel is the smallest component of a screen that can be programmed directly. A screen pixel shows one color.

2.2. Color depth

A screen pixel's color depth is the number of bits used to represent the color displayed by that pixel.

2.3. Screen area

A screen area is a rectangular two-dimensional grid of screen pixels with the same color depth.

A screen area has a width, which is the number of screen pixels along the main dimension of the screen area's rectangular pixel grid.

A screen area has a height, which is the number of screen pixels along the secondary dimension of the screen area's rectangular pixel grid.

2.4. Connected screen

The computer system hosting the user agent presents information using one or more connected screens.

A computer system’s connected screens may change while a user agent is running.

A connected screen has a screen area, which is used to present information to the user.

2.5. Available screen area

Web applications cannot assume that a connected screen's screen area is entirely available to them. The operating system hosting the user agent may reserve some of the screen area for its own user interface, and the user agent’s chrome may be taking some of the screen area.

The available screen area of a connected screen is a subset of the screen area that is entirely available for web applications. The rectangle’s edges are parallel to the screen area edges.

The available width of a connected screen is the width of the connected screen's available screen area.

The available height of a connected screen is the height of the connected screen's available screen area.

2.6. System screen area

TODO: Describe the virtual screen area made up of the multiple physical screens.

2.7. Internal screen

Each connected screen may be designated as internal or external.

External screens are manufactured separately from the computer systems they are connected to. It is not unusual for an external screen to be disconnected from one computer system and connected to a different computer system.

Internal screens are usually attached to a computer system at manufacturing time. Internal screens and are not intended to be detached by users. However, internal connected screens may still appear and disappear while the user agent is running.

2.8. Primary screen

The computer system hosting the user agent has exactly one primary connected screen. All the other connected screen are considered secondary.

A connected screen's primary / secondary designation may change while the user agent is running.

2.9. Touch support

A connected screen may have touch support.

Touch support is usually implemented by layering a touch sensor over the screen area.

3. API

const multi = await isMultiScreen();

Returns whether the device has multiple connected screens on success. User agents can prompt for permission.

};
partial interface Window {
  [SecureContext]
  Promise<boolean> isMultiScreen();
};

The isMultiScreen() method steps are:

  1. Let promise be a new promise.

  2. Run these steps in parallel:

    1. Resolve promise with true if the computer system has more than one connected screen, and false otherwise.

  3. Return promise.

Permission check for above?

partial interface Window {
  // NEW: Returns a snapshot of information about connected screens on success.
  [SecureContext]
  Promise<sequence<ScreenInfo>> getScreens();  // UAs may prompt for permission.
};


dictionary ScreenInfo {
  // Shape matches https://drafts.csswg.org/cssom-view/#the-screen-interface
  long availWidth;
  long availHeight;
  long width;
  long height;
  unsigned long colorDepth;
  unsigned long pixelDepth;

  // Shape roughly matches https://w3c.github.io/screen-orientation
  OrientationType orientationType;  // Orientation type, e.g. "portrait-primary"
  unsigned short orientationAngle;  // Orientation angle, e.g. 0

  // Shape matches https://developer.mozilla.org/en-US/docs/Web/API/Screen
  // Critical for understanding relative screen layouts for window placement.
  // Distances from a multi-screen origin (e.g. primary screen top left) to the:
  long left;       // Left edge of the screen area, e.g. 1920
  long top;        // Top edge of the screen area, e.g. 0
  long availLeft;  // Left edge of the available screen area, e.g. 1920
  long availTop;   // Top edge of the available screen area, e.g. 0

  // New properties critical for many multi-screen window placement use cases.
  boolean isPrimary;
  boolean internal;
  float scaleFactor;     // Ratio between physical pixels and device
                         // independent pixels for this screen, e.g. 2
                         // Useful for placing windows on screens with optimal
                         // scaling and appearances for a given application.
  DOMString id;          // A temporary, generated per-origin unique ID; resets
                         // when cookies are deleted. Useful for persisting user
                         // window placements preferences for certain screens.
  boolean touchSupport;
};

The getScreens() method steps are:

  1. Let promise be a new promise.

  2. Run the following steps in parallel:

    1. Let screens be a new list.

    2. For each screen connected screen.

      1. Let info be a new ScreenInfo dictionary describing screen.

      2. Append info to screens.

    3. Resolve promise with screens.

  3. Return promise.

Add permission check to the above.

3.1. Events

Define when screens change.

When blah blah blah happens, fire an event with type screenschange at blah blah.

partial interface Window {
  // NEW: An event fired when the connected screens or their properties change.
  [SecureContext]
  attribute EventHandler onscreenschange;
};

The onscreenschange attribute is an event handler IDL attribute whose event handler event type is screenschange.

3.2. Other Specifications

File bug against permissions spec, since partial enums aren’t a thing.

enum PermissionName {
  // ...
  "window-placement",
  // ...
};

Push this into other spec?

dictionary FullscreenOptions {
  FullscreenNavigationUI navigationUI = "auto";

  // NEW: An optional way to request a specific screen for element fullscreen.
  ScreenInfo screen;
};

4. Security Considerations

Write this section.

5. Privacy Considerations

Write this section.

6. Accessibility Considerations

Write this section.

7. Internationalization Considerations

Write this section.

8. Acknowledgements

Many thanks to

Anssi Kostiainen, Chris Terefinko, Domenic Denicola, Jonathan Garbee, Kenneth Rohde Christiansen, L. David Baron, Lukasz Olejnik, Marijn Kruisselbrink, Matt Giuca, Michael Ketting, Michael Wasserman, Nadav Sinai, Peter Linss, Staphany Park, Theresa O’Connor, Thomas Nattestad, Thomas Steiner,

for helping craft this proposal.

Ensure we didn’t forget anyone!

Special thanks to Tab Atkins, Jr. for creating and maintaining Bikeshed, the specification authoring tool used to create this document, and for his general authoring advice.

Conformance

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[FULLSCREEN]
Philip Jägenstedt. Fullscreen API Standard. Living Standard. URL: https://fullscreen.spec.whatwg.org/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119
[SCREEN-ORIENTATION]
Mounir Lamouri; Marcos Caceres; Johanna Herman. The Screen Orientation API. 17 April 2020. WD. URL: https://www.w3.org/TR/screen-orientation/
[WebIDL]
Boris Zbarsky. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/

IDL Index

partial interface Window {
  [SecureContext]
  Promise<boolean> isMultiScreen();
};

partial interface Window {
  // NEW: Returns a snapshot of information about connected screens on success.
  [SecureContext]
  Promise<sequence<ScreenInfo>> getScreens();  // UAs may prompt for permission.
};


dictionary ScreenInfo {
  // Shape matches https://drafts.csswg.org/cssom-view/#the-screen-interface
  long availWidth;
  long availHeight;
  long width;
  long height;
  unsigned long colorDepth;
  unsigned long pixelDepth;

  // Shape roughly matches https://w3c.github.io/screen-orientation
  OrientationType orientationType;  // Orientation type, e.g. "portrait-primary"
  unsigned short orientationAngle;  // Orientation angle, e.g. 0

  // Shape matches https://developer.mozilla.org/en-US/docs/Web/API/Screen
  // Critical for understanding relative screen layouts for window placement.
  // Distances from a multi-screen origin (e.g. primary screen top left) to the:
  long left;       // Left edge of the screen area, e.g. 1920
  long top;        // Top edge of the screen area, e.g. 0
  long availLeft;  // Left edge of the available screen area, e.g. 1920
  long availTop;   // Top edge of the available screen area, e.g. 0

  // New properties critical for many multi-screen window placement use cases.
  boolean isPrimary;
  boolean internal;
  float scaleFactor;     // Ratio between physical pixels and device
                         // independent pixels for this screen, e.g. 2
                         // Useful for placing windows on screens with optimal
                         // scaling and appearances for a given application.
  DOMString id;          // A temporary, generated per-origin unique ID; resets
                         // when cookies are deleted. Useful for persisting user
                         // window placements preferences for certain screens.
  boolean touchSupport;
};

partial interface Window {
  // NEW: An event fired when the connected screens or their properties change.
  [SecureContext]
  attribute EventHandler onscreenschange;
};

enum PermissionName {
  // ...
  "window-placement",
  // ...
};

dictionary FullscreenOptions {
  FullscreenNavigationUI navigationUI = "auto";

  // NEW: An optional way to request a specific screen for element fullscreen.
  ScreenInfo screen;
};

Issues Index

Write this section.
Write this section.
Write this section.
Finish up this section.
Permission check for above?
Add permission check to the above.
Define when screens change.
File bug against permissions spec, since partial enums aren’t a thing.
Push this into other spec?
Write this section.
Write this section.
Write this section.
Write this section.
Ensure we didn’t forget anyone!