Multi-Screen Window Placement

Draft Community Group Report,

This version:
https://webscreens.github.io/window-placement
Test Suite:
https://github.com/web-platform-tests/wpt/tree/master/screen_enumeration
Issue Tracking:
GitHub
Inline In Spec
Editors:
(Google Inc.)
(Google Inc.)

Abstract

This document defines a web platform API that allows script to query the device for information about connected displays, and additional APIs to position windows relative to those displays.

Status of this document

This specification was published by the Second Screen Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. Introduction

This section is non-normative.

Operating systems generally allow users to connect multiple screens to a single device and arrange them virtually to extend the overall visual workspace.

As multi-screen devices and applications become a more common and critical part of user experiences, it becomes more important to give developers information and tools to leverage that expanded visual environment.

More in this section?

1.1. Motivating Use Cases

The aim of this specification is enable better experiences for web application users with multiple screens. Here are some use cases that inform the design:

1.2. Usage Overview

Write this section, using examples from Explainer:

1.2.1. Detecting when screen properties change

Without even considering multiple screens, sites may want to adapt to screen attribute changes. To avoid polling, change events are fired at the Screen object:

screen.addEventListener('change') e => {
  /* display properties have changed for the current screen */
});

1.2.2. Detecting the presence of extended screen areas

The most basic question developers may ask to support multi-screen devices is: "Does this device have multiple screens that may be used for window placement?" This is provided by the isExtended boolean, exposed to secure contexts without a permission prompt.

if (screen.isExtended) {
  /* enable multi-screen options */
}

1.2.3. Requesting extended screen information

After determining that there are multiple screens, information can be requested about them using the getScreens() method. This method may prompt the user for permission. The resulting object lets developers enumerate screens, inspect screen properties, and watch for changes.

try {
  const screensInfo = await window.getScreens();

  // Do something with the screens right now.
  processScreens(screensInfo);

  // And also if the screens change in the future.
  screensInfo.onscreenschange = e => {
    processScreens(screensInfo);
  };
} catch (ex) {
  /* permission denied, or other error */
}

function processScreens(screensInfo) {
  // Build a UI listing screens.
  clearScreenList();
  screensInfo.screens.forEach(screen => {
    addToScreenList({id: screen.id, name: screen.label, screen: screen});
  });

  // Set the selection to the current screen.
  selectCurrentInScreenList(screensInfo.currentScreen.id);
}

1.2.4. Showing elements fullscreen on a specific screen

A common use case is to present a slideshow or other media fullscreen on a particular screen, while showing controls for the content on another screen. Once a screen has been selected, either interactively, using the screen’s properties, or remembering a previous selection, it can be passed to the requestFullscreen() method.

// Call an assumed helper function which asynchronously selects
// a screen, and returns a ScreenAdvanced instance.
const screenAdvanced = await getScreenForSlideshow();

// Request that a particular element go fullscreen on the selected
// screen.
slideshowElement.requestFullscreen({ screen: screenAdvanced });

1.2.5. Placing windows on a specific screen

Windows can be placed on specific screens using the coordinates provided in the ScreenAdvanced interface.

function openAndCenterWindow(url, screenAdvanced, width, height) {

  // Compute coordinates relative to the target screen’s coordinates.
  const left = Math.round(screenAdvanced.width  - width) / 2;
  const top = Math.round(screenAdvanced.height - height) / 2;

  // Open the window with the requested dimensions.
  return window.open(url, '_blank', <code data-opaque bs-autolink-syntax='`left=${left},top=${top},width=${width},height=${height}`'>left=${left},top=${top},width=${width},height=${height}</code>);
}

2. Concepts

Finish up this section.

2.1. Screen pixel

A screen pixel is the smallest component of a screen that can be programmed directly. A screen pixel shows one color.

Note: On a liquid-crystal display (LCD), each screen pixel is made up of three components. Each component is a (red, green, blue) light with variable intensity. Reasoning about pixel components (subpixel rendering) is out of scope for this specification.

A screen pixel's color depth is the number of bits used to represent the color displayed by that pixel.

Note: Some popular rendering systems model screen pixels as having a color depth of 24. The 24 bits are separated into 3 groups of 8 bits used to represent the intensity of the (red, green, blue) subpixels of an LCD screen pixel.

2.2. Screen area

A screen area is a rectangular two-dimensional grid of screen pixels with the same color depth.

A screen area has a width, which is the number of screen pixels along the main dimension of the screen area's rectangular pixel grid.

A screen area has a height, which is the number of screen pixels along the secondary dimension of the screen area's rectangular pixel grid.

Note: The grid size is usually expressed as width x height. For example, a 1920x1080 screen area has a grid with a width of 1920 pixels and a height of 1080 pixels.

2.3. Connected screen

The computer system hosting the user agent presents information using one or more connected screens.

A computer system’s connected screens may change while a user agent is running.

A connected screen has a screen area (a screen area), which is used to present information to the user.

A connected screen has a color depth, which is the color depth of the screen’s pixels.

A connected screen's device pixel ratio is the result of this algorithm:

  1. Let CSS pixel size be the size of a CSS pixel.

  2. Let device pixel size be the vertical size of a screen pixel.

  3. Return the result of dividing CSS pixel size by device pixel size.

A connected screen has an orientation, which is described in [screen-orientation].

A connected screen has a label, which is a string that meaningfully describes the screen to a user.

Note: The label can be an arbitrary string selected by the user agent. It could describe the screen relative to the device, e.g. "internal" vs. "external", it could include the dimensions, e.g. "640x480", it could include a distinguishing number, e.g. "screen 1" vs. "screen 2", or all of the preceeding.

While many properties of screens could be used for active fingerprinting, the strings used as labels in particular should be considered carefully to minimize the uniqueness. For example, it would be a poor choice to include the serial number of the device.

A connected screen has an id, which is a string that uniquely identifies the screen to a web application.

To prevent the id being used to correlate users across sites (active fingerprinting), the id assigned to a screen should be unique to each site. To prevent the id being used as a form of persistent re-identifier (cookie-like fingerprinting), a new id should be assigned to a screen after storage is cleared for the site.

2.3.1. Available screen area

Web applications cannot assume that a connected screen's screen area is entirely available to them. The operating system hosting the user agent may reserve some of the screen area for its own user interface, and the user agent’s chrome may be taking some of the screen area.

The available screen area of a connected screen is a subset of the screen area that is entirely available for web applications. The rectangle’s edges are parallel to the screen area edges.

The available width of a connected screen is the width of the connected screen's available screen area.

The available height of a connected screen is the height of the connected screen's available screen area.

Are these definitions actually needed? Reference CSSOM View Module 1 §2.3 Web-exposed screen information if so.

2.3.2. Pointer type support

Users often interact with content on connected screens using a pointer, an input device that can target coordinates on the screen. Examples include mouse, pen, or touch contact. [PointerEvents]

The supported pointer types for a connected screen is the ordered set of possible pointerType values that can appear in PointerEvents fired for content on the screen. The set is ordered by sorting in ascending order with the code unit less than algorithm.

Note: As an example, the supported pointer types for a device supporting both mouse and touch input would be « "mouse", "touch" ».

2.3.3. Screen position

A computer system has a virtual screen area, with x- and y-coordinates that increase rightwards and downwards, respectively. Every connected screen's screen area is a view onto a rectangular subset of the virtual screen area.

A connected screen has a screen position which is the x- and y-coordinates in the virtual screen area of the top left corner the rectangular subset it is a view onto, in pixels. Coordinates may be negative, and are typically expressed as (x, y).

2.3.4. Observable properties

The basic observable properties of a connected screen are:

The advanced observable properties of a connected screen are:

2.4. Primary screen

The computer system hosting the user agent has exactly one primary connected screen. All the other connected screen are considered secondary.

Note: The primary connected screen typically hosts the operating system’s user interface for task management, such as the Windows task bar and the macOS Dock.

A connected screen's designation as primary or secondary may change while the user agent is running.

The primary screen is the connected screen that is currently primary.

Note: Most operating systems let the user choose the primary connected screen using a management user interface, such as the Windows Control Panel and the macOS Preferences application.

By convention, the primary screen's screen position is (0,0).

Note: As an example, consider a computer system with three 1920x1080 screens arranged horizontally. If the middle screen is the primary screen, then the left screen’s screen position (-1920,0). The middle screen’s screen position (0,0). The right screen’s screen position is (3840,0).

2.5. Internal screen

Each connected screen may be designated as internal or external.

External screens are manufactured separately from the computer systems they are connected to. It is not unusual for an external screen to be disconnected from one computer system and connected to a different computer system.

Internal screens are usually attached to a computer system at manufacturing time. Internal screens and are not intended to be detached by users. However, internal connected screens may still appear and disappear while the user agent is running.

Note: A laptop has an internal screen and an input device. A laptop might be used while opened, with its internal screen powered on. However, a laptop might also be used while closed, where its internal screen is powered off.

A laptop might be opened and closed while a user agent is running. In this case, the internal screen appears as a connected screen when the laptop is opened. The internal connected screen disappears when the laptop is closed.

2.6. Current screen

Scripts executing in a Window context can access the screen property. This Screen object reflects the properties of the current screen, which is a connected screen that presents the window.

Note: On many operating systems, a window can be presented across multiple screens with different properties, or can be in a "hidden" state and not presented on any screen. Operating systems and user agents are assumed to define a canonical screen for a given Window, for example the screen with the largest intersection area with the window.

3. API

3.1. Extensions to the Screen interface

window . screen . isExtended

Returns true if the device’s display is extended across multiple screens.

partial interface Screen /* : EventTarget */ {
  [SecureContext]
  readonly attribute boolean isExtended;

  [SecureContext]
  attribute EventHandler onchange;
};

Screen needs to derived from EventTarget. This will require changes to CSSOM View Module 1 §4.3 The Screen Interface.

The isExtended getter steps are:

  1. Return true if the computer system has more than one connected screen, and false otherwise.

3.1.1. Screen events

The onchange attribute is an event handler IDL attribute whose event handler event type is change.

When any basic observable property of a Window's current screen changes, fire an event with type change at the Window's associated Screen object.

3.2. Extensions to the Window interface

await window . getScreens()

Returns a promise that fulfills with a Screens object with information about all connected screens. The promise will be rejected if permission is denied.

partial interface Window {
  [SecureContext]
  Promise<Screens> getScreens();
};

Each Window object has an associated Screens object.

Each Window object also has an associated ScreenAdvanced object which reflects the properties of the window’s current screen.

The getScreens() method steps are:

  1. Let promise be a new promise.

  2. Run the following steps in parallel:

    1. Let permissionState be the result of requesting permission to use the powerful feature named "window-placement".

    2. If permissionState is "denied" then reject promise with a "NotAllowedError" DOMException and abort these steps.

    3. Resolve promise with this's associated Screens object.

  3. Return promise.

In addition to the partial interface additions defined above, the Window interface described in CSSOM View Module 1 §4 Extensions to the Window Interface are modified:

Merge the above with CSSOM View Module 1 §4 Extensions to the Window Interface.

3.3. The Screens interface

screens . screens

Returns an array of ScreenAdvanced objects that describe each of the connected screens.

screens . currentScreen

Returns a ScreenAdvanced object that describes the current screen. This object describes the same object that the screen property of Window describes, but provides a superset of the information.

[Exposed=Window, SecureContext]
interface Screens {
  readonly attribute FrozenArray<ScreenAdvanced> screens;

  readonly attribute ScreenAdvanced currentScreen;

  attribute EventHandler onscreenschange;
  attribute EventHandler oncurrentscreenchange;
};

The screens getter steps are:

  1. Let screens be a new list.

  2. For each screen of connected screens:

    1. Let a be the ScreenAdvanced object describing screen.

    2. Append a to screens.

  3. Return the result of sorting screens in ascending order with the screen ordering algorithm.

The screen ordering algorithm defines a connected screen a as less than a connected screen b if the following steps return true:

  1. If a’s screen position y-coordinate is less than b’s screen position y-coordinate, then return true.

  2. If b’s screen position y-coordinate is less than a’s screen position y-coordinate, then return false.

  3. If a’s screen position x-coordinate is less than b’s screen position x-coordinate, then return true.

  4. If b’s screen position x-coordinate is less than a’s screen position x-coordinate, then return false.

  5. Return whether or not a’s id is code unit less than b’s id.

Should x or y have sort priority? Fall back to id?

The currentScreens getter steps are to return the ScreenAdvanced object associated with the Window associated with this.

3.3.1. Screens events

The onscreenschange attribute is an event handler IDL attribute whose event handler event type is screenschange.

The oncurrentscreenchange attribute is an event handler IDL attribute whose event handler event type is currentscreenchange.

When the set of connected screens changes, fire an event with type screenschange at every Screens object.

When the current screen of a Window changes from one connected screen to another (e.g. the {{Window has been moved to a different display), fire an event with type currentscreenchange at the Window's associated Screens object.

3.4. The ScreenAdvanced interface

A ScreenAdvanced object represents a connected screen.

screen . left

Returns the distance from the left edge of the primary screen to the left edge of the screen area.

screen . top

Returns the distance from the top edge of the primary screen to the top edge of the screen area.

screen . isPrimary

Returns whether this screen is designated as the 'primary' screen by the OS (otherwise it is a 'secondary' screen).

screen . isInternal

Returns whether this screen is an 'internal' panel built into the device, like a laptop display (otherwise it is 'external', like a wired monitor).

screen . devicePixelRatio

Returns the ratio between physical and logical pixels.

screen . id

Returns a temporary generated per-origin unique ID; reset when cookies are deleted.

screen . pointerTypes

Returns an array of pointer types supported by the screen, e.g. "mouse", "touch", or "pen".

screen . label

A user-friendly label for the screen, determined by the user agent and OS.

[SecureContext] interface ScreenAdvanced : Screen {
  readonly attribute long left;
  readonly attribute long top;
  readonly attribute boolean isPrimary;
  readonly attribute boolean isInternal;
  readonly attribute float devicePixelRatio;
  readonly attribute DOMString id;
  readonly attribute FrozenArray<DOMString> pointerTypes;
  readonly attribute DOMString label;
};

The left getter steps to return this's screen position's x-coordinate.

The top getter steps to return this's screen position's y-coordinate.

The isPrimary getter steps are to return true if this is the primary screen, or false otherwise.

The isInternal getter steps are to return true if this is internal, or false otherwise.

The devicePixelRatio getter steps are to return this's device pixel ratio.

The id getter steps are to return this's id.

The pointerTypes getter steps are to return this's supported pointer types.

The label getter steps are to return this's label.

3.4.1. ScreenAdvanced events

When any basic observable property or advanced observable property of a ScreenAdvanced's associated connected screen changes, fire an event with type change at the ScreenAdvanced object.

3.5. Extensions to FullscreenOptions

partial dictionary FullscreenOptions {
  // An optional way to request a specific screen for element fullscreen.
  ScreenAdvanced screen;
};

Write me.

3.6. Permissions

The "window-placement" powerful feature is a boolean feature.

Call it "multiscreen" instead? <https://github.com/webscreens/window-placement/issues/54>

File bug against [permissions], to add to the registry.

4. Security Considerations

Write this section.

5. Privacy Considerations

Write this section.

6. Accessibility Considerations

Write this section.

7. Internationalization Considerations

Write this section.

8. Acknowledgements

Many thanks to

Anssi Kostiainen, Chris Terefinko, Domenic Denicola, Jonathan Garbee, Kenneth Rohde Christiansen, L. David Baron, Lukasz Olejnik, Marijn Kruisselbrink, Matt Giuca, Michael Ketting, Michael Wasserman, Nadav Sinai, Peter Linss, Staphany Park, Theresa O’Connor, Thomas Nattestad, Thomas Steiner,

for helping craft this specification.

Ensure we didn’t forget anyone!

Special thanks to Tab Atkins, Jr. for creating and maintaining Bikeshed, the specification authoring tool used to create this document, and for his general authoring advice.

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[CSS-VALUES-4]
Tab Atkins Jr.; Elika Etemad. CSS Values and Units Module Level 4. 11 November 2020. WD. URL: https://www.w3.org/TR/css-values-4/
[CSSOM-VIEW-1]
Simon Pieters. CSSOM View Module. 17 March 2016. WD. URL: https://www.w3.org/TR/cssom-view-1/
[DOM]
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
[FINGERPRINTING-GUIDANCE]
Nick Doty. Mitigating Browser Fingerprinting in Web Specifications. 28 March 2019. NOTE. URL: https://www.w3.org/TR/fingerprinting-guidance/
[FULLSCREEN]
Philip Jägenstedt. Fullscreen API Standard. Living Standard. URL: https://fullscreen.spec.whatwg.org/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[PERMISSIONS]
Mounir Lamouri; Marcos Caceres; Jeffrey Yasskin. Permissions. 20 July 2020. WD. URL: https://www.w3.org/TR/permissions/
[PointerEvents]
Jacob Rossi; Matt Brubeck. Pointer Events. 4 April 2019. REC. URL: https://www.w3.org/TR/pointerevents/
[POINTEREVENTS3]
Patrick Lauke; et al. Pointer Events. 2 June 2021. WD. URL: https://www.w3.org/TR/pointerevents3/
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119
[WebIDL]
Boris Zbarsky. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/

Informative References

[SCREEN-ORIENTATION]
Mounir Lamouri; Marcos Caceres; Johanna Herman. The Screen Orientation API. 19 October 2020. WD. URL: https://www.w3.org/TR/screen-orientation/

IDL Index

partial interface Screen /* : EventTarget */ {
  [SecureContext]
  readonly attribute boolean isExtended;

  [SecureContext]
  attribute EventHandler onchange;
};

partial interface Window {
  [SecureContext]
  Promise<Screens> getScreens();
};

[Exposed=Window, SecureContext]
interface Screens {
  readonly attribute FrozenArray<ScreenAdvanced> screens;

  readonly attribute ScreenAdvanced currentScreen;

  attribute EventHandler onscreenschange;
  attribute EventHandler oncurrentscreenchange;
};

[SecureContext] interface ScreenAdvanced : Screen {
  readonly attribute long left;
  readonly attribute long top;
  readonly attribute boolean isPrimary;
  readonly attribute boolean isInternal;
  readonly attribute float devicePixelRatio;
  readonly attribute DOMString id;
  readonly attribute FrozenArray<DOMString> pointerTypes;
  readonly attribute DOMString label;
};

partial dictionary FullscreenOptions {
  // An optional way to request a specific screen for element fullscreen.
  ScreenAdvanced screen;
};

Issues Index

More in this section?
Write this section, using examples from Explainer:
Finish up this section.
Are these definitions actually needed? Reference CSSOM View Module 1 §2.3 Web-exposed screen information if so.
Screen needs to derived from EventTarget. This will require changes to CSSOM View Module 1 §4.3 The Screen Interface.
Merge the above with CSSOM View Module 1 §4 Extensions to the Window Interface.
Should x or y have sort priority? Fall back to id?
Write me.
Call it "multiscreen" instead? <https://github.com/webscreens/window-placement/issues/54>
File bug against [permissions], to add to the registry.
Write this section.
Write this section.
Write this section.
Write this section.
Ensure we didn’t forget anyone!