DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
3. Claim(s) 1-2, 4-6, and 8-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Da Veiga et al. (US 2018/0348518 A1, hereinafter referred as “Da Veiga”).
Regarding claim 1, Da Veiga discloses an information presentation method comprising:
collecting, as collected information, at least one of (i) information on a posture of a subject or (ii) information on at least one of a face orientation or a direction of a line of sight of the subject (¶0020-¶0021 discloses the system 100 can include a head position tracker 110, integrated with the HMD 102 or provided by resources external to HMD 102, configured to track the position of the user's head, and generate a corresponding head position signal HP);
analyzing an action of the subject based on the collected information (¶0034 and ¶0054 discloses at T10, the head tracker 110 detected the user's head had moved to a new position, i.e., new location and new orientation);
determining, based on an analysis result of the action, a presentation image that is an image to be presented to the subject (¶0034 discloses the drone positioning control 112 can, in response, send a corresponding CPS instructing the drone 106 to reposition the camera 108 to an instructed new camera position aligned with the user's new head position), the presentation image being an image obtained from a camera that is provided at a position different from a viewpoint of the subject and showing a range selected within a capturable range of the camera (Fig. 1 and ¶0018-¶0020 discloses camera 108 is mounted on the remote drone 106 while the user wears the HMD 102 and its capturable range is defined by the location and orientation of the drone 106); and
presenting, by a display device, the presentation image determined to the subject (¶0027 discloses the camera 108 can transmit the frames to the wireless base transceiver 104, for delivery to the image processor 114; and ¶0017 and ¶0059 discloses an HMD interface 912 can be included, coupled for example to the bus 910, to receive HT and HP from the HMD 902 and to provide output from the image processor 900 to the HMD 102 of visual display data).
Regarding claim 2, Da Veiga discloses the information presentation method according to claim 1, wherein in the collecting, an image sensor is used to collect at least one of (i) the information on the posture of the subject or (ii) the information on at least one of the face orientation or the direction of the line of sight of the subject (¶0025 discloses the head position tracker 110 can include an external camera (not visible in FIG. 1) directed at the user's head).
Regarding claim 4, Da Veiga discloses the information presentation method according to claim 1, wherein in the determining, when the analysis result indicates an upward shift of a center of gravity in the posture of the subject (¶0021 discloses the (X, Y) axes can be, for example, a two-dimensional position on a floor space or other area, and (Z) can be an elevation. The ‘Z’ feature can provide the user with head control of an elevation of the drone 106, for example by squatting or standing on his toes), an image corresponding to a field of view above the field of view of the subject is determined to be the presentation image (¶0054-¶0056 discloses render the view corresponding to the new camera position matching the user’s head movement).
Regarding claim 5, Da Veiga discloses the information presentation method according to claim 1, wherein in the determining, when the analysis result indicates that the posture of the subject is a posture of leaning forward (¶0021-¶0022 discloses tacking the head location and orientation including the X/Y/Z position orientation axes), an image corresponding to a field of view in front of the field of view of the subject is determined to be the presentation image (¶0054-¶0056 discloses render the view corresponding to the new camera position matching the user’s head movement).
Regarding claim 6, Da Veiga discloses the information presentation method according to claim 1, wherein the presenting includes outputting the presentation image to smartglasses that include one or more displays positioned in front of eyes of the subject wearing the smartglasses (Fig. 1 and ¶0005 discloses apparatus that can include a head mounted display, configured to be worn by a user), to present the presentation image on the one or more displays (¶0059 discloses an HMD interface 912 can be included, coupled for example to the bus 910, to receive HT and HP from the HMD 902 and to provide output from the image processor 900 to the HMD 102 of visual display data).
Regarding claim 8, Da Veiga discloses the information presentation method according to claim 1, wherein in the collecting, the information on at least one of the face orientation (¶0021-¶0022 discloses for tracking user head orientation, the head position tracker 110 can be configured to track left-to-right head rotation about the FIG. 1 “azimuth” or “AZ” axis, and elevation about the FIG. 1 “EL” axis.) or the direction of the line of sight of the subject is collected, and in the determining, the presentation image to be presented to the subject is determined based on (i) the analysis result of the action (¶0034 and ¶0054 discloses at T10, the head tracker 110 detected the user's head had moved to a new position, i.e., new location and new orientation) and (ii) the information on at least one of the face orientation or the direction of the line of sight of the subject (¶0034 discloses the drone positioning control 112 can, in response, send a corresponding CPS instructing the drone 106 to reposition the camera 108 to an instructed new camera position aligned with the user's new head position).
Regarding claim 9, Da Veiga discloses the information presentation method according to claim 1, wherein the camera is configured to control at least one of a horizontal angle or a vertical angle within the capturable range (¶0023-¶0024 discloses a frame of reference for the orientation of the camera 108, the system 100 can include a yaw axis, labeled “YW,” and a pitch axis, labeled “PT”, the yaw corresponding to the horizontal angle and the pitch corresponding to the vertical angle), and the presenting includes outputting a control signal for controlling at least one of the horizontal angle or the vertical angle of the camera to obtain an image corresponding to the presentation image determined (¶0024 discloses the camera position controller 112 and an image processor 114, described in greater detail later, can store a translation map between the camera location reference and the user location reference, i.e., an X′, Y′, Z′ to X, Y, Z translation, and between the head orientation reference and the camera orientation reference, i.e., an AZ, EL, TL to YW, PT, RL translation), and subsequently presenting the image obtained from the camera to the subject (¶0027 discloses the camera 108 can transmit the frames to the wireless base transceiver 104, for delivery to the image processor 114; and ¶0017 and ¶0059 discloses an HMD interface 912 can be included, coupled for example to the bus 910, to receive HT and HP from the HMD 902 and to provide output from the image processor 900 to the HMD 102 of visual display data).
Regarding claim 10, Da Veiga discloses a non-transitory computer-readable recording medium having recorded thereon a program for causing a computer to execute the information presentation method according to claim 1 (¶0059 discloses instruction memory 906 can include a tangible medium retrievably storing computer-readable instructions that when executed by the data processor 902 cause the processor to perform operations, described in greater detail in the following paragraphs, providing each of a surface map module 914, an image assembly module 916, an adjusted rendering engine 918, a late stage re-projection (LSR) module 920).
Regarding claim 11, Da Veiga discloses an information presentation device (¶0018 discloses HMD 102) comprising:
an information collector that collects, as collected information, at least one of (i) information on a posture of a subject or (ii) information on at least one of a face orientation or a direction of a line of sight of the subject (¶0020-¶0021 discloses the system 100 can include a head position tracker 110, integrated with the HMD 102 or provided by resources external to HMD 102, configured to track the position of the user's head, and generate a corresponding head position signal HP);
an action analyzer that analyzes an action of the subject based on the collected information (¶0034 and ¶0054 discloses at T10, the head tracker 110 detected the user's head had moved to a new position, i.e., new location and new orientation);
a presentation image determiner that determines, based on an analysis result of the action, a presentation image that is an image to be presented to the subject (¶0034 discloses the drone positioning control 112 can, in response, send a corresponding CPS instructing the drone 106 to reposition the camera 108 to an instructed new camera position aligned with the user's new head position), the presentation image being an image obtained from a camera that is provided at a position different from a viewpoint of the subject and showing a range selected within a capturable range of the camera (Fig. 1 and ¶0018-¶0020 discloses camera 108 is mounted on the remote drone 106 while the user wears the HMD 102 and its capturable range is defined by the location and orientation of the drone 106); and
a presenter that presents the presentation image determined to the subject (¶0027 discloses the camera 108 can transmit the frames to the wireless base transceiver 104, for delivery to the image processor 114; and ¶0017 and ¶0059 discloses an HMD interface 912 can be included, coupled for example to the bus 910, to receive HT and HP from the HMD 902 and to provide output from the image processor 900 to the HMD 102 of visual display data).
Regarding claim 12, Da Veiga discloses an information presentation system comprising: the information presentation device according to claim 11 (¶0018 discloses a system 100 that includes HMD 102 on a user); the camera (Fig. 1, camera 108); and a display device that presents the presentation image to the subject (¶0056 discloses to display the information in the user HMD).
Claim Rejections - 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Da Veiga in view of Scally et al. (US 2017/0261610 A1, hereinafter referred as “Scally”).
Regarding claim 3, Da Veiga doesn’t disclose the information presentation method according to claim 1, wherein in the collecting, an ultrasonic sensor is used to collect at least one of (i) the information on the posture of the subject or (ii) the information on at least one of the face orientation or the direction of the line of sight of the subject.
However, in the same field of endeavor, Scally discloses wherein in the collecting, an ultrasonic sensor (¶0004 discloses eye tracking unit includes one or more transmitters that transmit a signal (e.g., a radar signal or an ultrasonic sound) at an eye of the user, one or more receivers that receive a reflection of the signal generated by interaction of the signal with the eye) is used to collect at least one of (i) the information on the posture of the subject or (ii) the information on at least one of the face orientation or the direction of the line of sight of the subject (¶0027 discloses the eye tracking unit 160 estimates an angular orientation of the user's eye. The angular orientation of the eye corresponds to a direction of the user's gaze within the VR headset 105 and is defined herein as the direction of the foveal axis, which is the axis between a fovea (an indentation on the retina of the eye) of the eye and a center of the eye's pupil).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Da Veiga so that when performing eye tracking in a virtual reality environment, using a detection element that is small and relatively close to the user's eye for eye tracking may be preferred (¶0003).
6. Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Da Veiga in view of Hasegawa et al. (US 2016/0274733 A1, hereinafter referred as “Hasegawa”).
Regarding claim 7, Da Veiga doesn’t disclose the information presentation method according to claim 1, wherein the presenting includes outputting the presentation image to a display device that includes a display located at a position around the subject to present the presentation image on the display.
However, in the same field of endeavor, Hasegawa discloses wherein the presenting includes outputting the presentation image to a display device (¶0010 discloses a video projection unit that projects a video) that includes a display located at a position around the subject to present the presentation image on the display (¶0140 discloses FIGS. 15 and 16 are examples of a case where projection is performed for a rectangular desk. As illustrated in FIG. 15, it is photographed and recognized by the camera 100 of the projection-type display device 205 provided with the operation detecting function that a user 3 is present near the projection object 204 that is the rectangular desk).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Da Veiga for the purpose of improving usability and allowing nearby users to view and interact with displayed information without requiring a head-mounted display.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PRIYANK J SHAH whose telephone number is (571)270-3732. The examiner can normally be reached on 10:00 - 6:00 M-F.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ghebretinsae, Temesghen can be reached on (571) 272-3017. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PRIYANK J SHAH/Primary Examiner, Art Unit 2626
Prosecution Insights