The US Patent & Trademark Office has today published an application from Apple relating to a head-mounted device that provides an immersive user experience allowing user’s interaction with the system in a natural, familiar, and intuitive manner.
The patent covers the use of a physical or virtual keyboard and the Apple Pencil to draw with a tracking device accessory. The virtual keyboard has the potential of providing many little advantages over its physical counterpart.
The head-mounted device can act as a virtual reality (VR) system, an augmented reality (AR) system, and/or a mixed reality (MR) system. It can be worn by the user to exhibit visual information within her field of view.
She can observe outputs that the head-mounted device provides, for example, the visual information exhibited on a display. Optionally, the display can enable the user to observe the environment outside of the head-mounted device.
The head-mounted device can provide other outputs as well. These include audio output and/or haptic feedback. Further, the user can interact with this device by giving inputs for processing through one or more parts of the head-mounted device. For example, tactile inputs, voice commands, and other inputs can be provided by the user putting the device mounted to her head.
Head-mounted devices, like head-mounted displays, headsets, visors, smartglasses, head-up display, etc., can perform a number of functions managed by the components (e.g., sensors, circuitry, and other hardware) incorporated in the wearable device.
Along with providing outputs to the user in a variety of ways, the head-mounted device can also be helpful in receiving inputs from the user.
Head Mounted Device + Input via Keyboard
According to Apple, a keyboard is an example of a quite familiar input device. While using a head-mounted device, the user’s hands remain free to operate a keyboard or another device, in a manner similar to the use of a keyboard.
The utility of a keyboard, or any other input devices, and/or any surface can be magnified by a head-mounted device, which can show feedback, outputs and other things based on the use of the keyboard.
For example, the head-mounted device can exhibit text generated from user’s keyboard operation. This text can be displayed such that the user can readily see the keyboard, his hands, and the text produced by keyboard operation.
Along with this, the head-mounted device can display suggested text, keystrokes, or other features correlated with keys of the keyboard for selection by a user.
Further, the keyboard can be shown in a position and orientation so that it conforms to the arrangement of user’s hands within the field of view of the head-mounted device.
Apple patent FIG. 1 below gives a view of a head-mounted device and a keyboard; FIG. 2 illustrates a head-mounted device and an input device in use; FIG. 3 illustrates a view of a head-mounted device and a surface in use.
As shown in FIG. 1, the tracking device (#200) can be placed close to the keyboard (#300) for tracking the motion of the user’s hands and/or the keys of the keyboard during the work. The tracking device can be placed in connection with the head-mounted device so that detections by the tracking device can produce signals transmitted from the tracking device to the head-mounted device.
In FIG. 2 , the head-mounted device is used in conjunction with another input device, such as input device #400 which can be a virtual keyboard and the HMD (head-mounted device) gives the visuals of a keyboard. Between the cameras in the HMD with a tracking device, the user’s input can be perfectly understood exactly like a physical keyboard.
In FIG. 3, Apple moves one step further in the sense that the user will be able to use a desk, table, wall to use a keyboard and, along with that, to write or draw using an Apple Pencil.
Apple’s patent FIG. 7 below exhibits a display of a head-mounted device yielding a view of a keyboard, a user’s hands, and text within a window, FIG. 8 exhibits a display of a head-mounted device yielding a view of a keyboard, a user’s hands, and text with available selections;
FIG. 11 exhibits a display of a head-mounted device yielding a view of a keyboard, a user’s hands, and virtual indicators; and FIG. 14 exhibits a display of a head-mounted device yielding a view of keyboard portions which have an arrangement according to the position of the user’s hands.
The patent describes the head-mounted device that includes speakers and a series of sensors including any of the following features: touch, force, temperature, position, motion, and so on.
The sensor may be a photodetector, a temperature sensor, a light or optical sensor, an atmospheric pressure sensor, a humidity sensor, a magnet, a gyroscope, an accelerometer, a chemical sensor, an ozone sensor, a particulate count sensor, and so on.
The sensor can also be a bio-sensor used to track biometric characteristics, like health and activity metrics.
Apple’s patent application (application number 20200326847) published by the U.S. Patent Office today was filed in the first quarter of 2020 and some work dates back a year. As this is a patent application only, the timing of the market release of the product is not presently known.
