Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9, 11-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20210192802 A1 to Nepveu et al. (“Nepveu”) and US 20140160129 A1 to Sako et al. (“Sako”).
As to claim 1, Nepveu teaches a display control apparatus comprising: one or more processors; and a memory storing computer-readable instructions that, upon execution by the one or more processors, configure the display control apparatus to: output data of an image representing a virtual space to a head-mounted display(HMD) (¶0033, head mounted system may include one or more speakers and an opaque display. As another example, an external display (e.g., a smartphone) may be incorporated within a head mounted system. The head mounted system may include microphones for capturing audio of a physical setting , and/or image sensors for capturing images/video of the physical setting. A transparent or semi-transparent display may also be included in the head mounted system head mounted system may include one or more speakers and an opaque display. As another example, an external display (e.g., a smartphone) may be incorporated within a head mounted system. The head mounted system may include microphones for capturing audio of a physical setting, and/or image sensors for capturing images/video of the physical setting. A transparent or semi-transparent display may also be included in the head mounted system) ; (¶0032); and switch, upon occurrence of the event, data to the HMD from the data of the image representing the virtual space to data of a see-through image representing a real world (Fig. 4A-4E, ¶0051-0054, inserting imagery from a real environment into a virtual environment). Sako teaches detect that a predetermined event related to a virtual object in the virtual space has occurred, ¶0048). In view of the teachings of Sako, it would have been obvious before the effective filing date of the invention to modify the teachings of Nepveu. The suggestion/motivation would be possible to perform display control on an image from a third person viewpoint in the real world, from which a user is overlooked.
As to claim 2, Nepveu and Sako teaches the display control apparatus according to wherein the display control apparatus is further configured to detect the occurrence of the predetermined event upon an action of the virtual object manipulated by a user wearing the HMD (Nepeveu, ¶0066, detecting the transition event includes detecting that a proximity of the electronic device 100a (and/or the user) to a physical object (also referred to as a physical element) in real environment 200 is less than a threshold distance (e.g., 1 foot).
As to claim 3, Nepveu and Sako teaches the display control apparatus according to claim 1, wherein the display control apparatus is further configured to output audio data in the virtual space to the HMD and, further configured to change a manner of outputting the audio data while outputting the data of the see-through image (Nepveu, ¶0049).
As to claim 4, Nepveu and Sako teaches the display control apparatus according to claim 3, wherein, upon the occurrence of the event, wherein the display control apparatus is further configured to change the manner of outputting the audio data while outputting the data of the see-through image (Sako, FIG. 3, the HMD 1 includes a display unit 2, a imaging unit 3, an illumination unit 4, an audio output unit 5, an audio input unit 6, a system controller 10, an imaging control unit 11, a displayed image processing unit 12, a display driving unit 13, a display control unit 14, an imaging signal processing unit 15, an audio signal processing unit 16, an image analysis unit 17, an illumination control unit 18, a GPS reception unit 21, a storage unit 25, a communication unit 26, an image input/output control 27, an audio input/output control 28, and an audio synthesis unit 29).
As to claim 5, Nepveu and Sako teaches the display control apparatus according to claim 1, wherein the display control apparatus is further configured to detect that a predetermined event has occurred in the real world, display control apparatus is configured to switch the data to be output back to the data of the image representing the virtual space when the predetermined eve nt occurs in the real world (¶0046, Nepveu, ¶0042).
As to claim 6, Nepveu and Sako teaches the display control apparatus according to claim 5, wherein, according to details of the event in the virtual space, the display control apparatus is configured change the event in the real world (¶0050, the view of virtual environment 300 changes to correspond to the current location and direction of sight of the user. In addition, the mix of audio objects being provided changes to correspond to the current location and position of the user's ears. For example, the volume level, spatial placement, and/or frequency spectrum or each audio object changes such that the sounds appear to be emitting from a consistent virtual location as the user's location and/or position changes).
As to claim 7, Nepveu and Sako teaches the display control apparatus according to claim 5, wherein the display control apparatus is configured to determine that the event has occurred in the real world, when a predetermined condition is met with respect to at least either a state of a body part of the user, a position and a posture of the user, a state of the HMD being worn, or an elapsed time (¶0081, detects the proximity of the device 100a and/or the user (e.g., arms, legs, or head of the user's body) to the physical object 202-a via one or more internal and/or external image sensors. In some embodiments, the threshold distance is predefined (e.g., by an operating system of device 100a or set by a user of device 100a).
As to claim 8, Nepveu and Sako teaches the display control apparatus according to claim 2, wherein the display control apparatus is configured to detect, as the predetermined event, that the virtual object has exited from a door that exists in the virtual space (¶0062, acoustic properties of the real environment 200 are predetermined acoustic properties assigned to a type of environment (e.g., large room, small room, concert hall, cathedral). In some embodiments, the acoustic properties of the real environment 200 are determined by analyzing acoustic and/or physical features of the real environment 200. For example, the acoustic properties of the real environment 200 can be determined by emitting a predetermined test sound (e.g., a chirp or sweep) and analyzing the changes to the test sound caused by the real environment 200 (e.g., the test sound reflects off the physical features of the room, and the reflections are analyzed). As another example, the physical features of the room can be detected (e.g., the layout and material composition of the walls, floor, ceiling, and objects in the room). An acoustic model of the real environment 200 can then be estimated based on the physical features of the real environment 200).
As to claim 9, Nepveu and Sako teaches the display control apparatus according to claim 2, wherein the display control apparatus is configured to detect, as the predetermined event, that the virtual object has grabbed an item placed in the virtual space (¶0026).
As to claim 11, Nepveu and Sako teaches the display control apparatus according to claim 1, the display control apparatus is further configured to receive an operation performed by the user to set details of the predetermined event (¶0032).
As to claim 12, see the rejection of claim 1.
As to claim 13, see the rejection of claim 1.
As to claim 14, see the rejection of claim 2.
As to claim 15, see the rejection of claim 3.
As to claim 16, see the rejection of claim 4.
As to claim 17, see the rejection of claim 5.
As to claim 18 see the rejection of claim 6.
As to claim 19, see the rejection of claim 7.
As to claim 20, see the rejection of claim 8.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nepveu and Sako as applied to claim 1 above, and further in view of US 9696795 B2 to Marcolina et al. (“Marcolina”).
As to claim 10, Nepveu and Sako teaches the display control apparatus according to claim 4, Nepveu and Sako do not teach wherein the display control apparatus is configured to detect that the virtual object has grabbed a mobile phone in the virtual space, and, the display control apparatus is further configured to output audio data of a communicating party on a phone in the real world to the HMD while outputting the data of the see-through image. Marcolina teaches wherein the display control apparatus is configured to detect that the virtual object has grabbed a mobile phone in the virtual space, and, the display control apparatus is further configured to output audio data of a communicating party on a phone in the real world to the HMD while outputting the data of the see-through image (Col. 5, ll. 5-24, capturing an image of the hands in a three-dimensional (3D) sensory space and sensing a location of a first hand, incorporating the image of the first hand into a virtual reality scene that includes a grabable virtual object, sensing a gesture of the first hand and determining whether the gesture is intended to interact with the grabable virtual object by grabbing the virtual object, the determining further including taking into account at least an angular relationship of a normal to a palm of the first hand to a proximate surface of the virtual object, fingertip separations between the fingertips of the first hand and the proximate surface, a gesture rate at which the first hand closes on the virtual object, a hand posture, whether suitable for grasping the virtual object or incompatible with grasping, and linear velocity of the palm of the first hand relative to the virtual object, and responsive to determining that a gesture of the first hand is intended to grab the virtual object, linking motion of the first hand to manipulation of the virtual object. In view of the teachings of Marcolina, it would have been obvious before the effective filing date of the invention to modify the teachings of Nepveu and Sako. The suggestion/motivation would be a need for improved devices with greater portability and techniques for capturing the motion of objects in real time without fixed or difficult to configure sensors or markers.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTINE A KURIEN whose telephone number is (571)270-5694. The examiner can normally be reached M-F; 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nathan Flynn can be reached at 571-272-1915. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTINE A KURIEN/Examiner, Art Unit 2421 /NATHAN J FLYNN/Supervisory Patent Examiner, Art Unit 2421
Prosecution Insights