Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sako et al., US PGPUB 20160140768 hereinafter referenced as Sako, in view of Harrises et al., US PGPUB 20170206691 hereinafter referenced as Harrises.
As to claim 1, the combination of Sako and Harrises discloses a head-mountable electronic device, comprising: a housing comprising an inward facing side and an outward facing side (e.g., the housing of inner display unit 2i and outer display 2O, fig. 6B);
a face engagement structure connected to the housing ([0047] Specifically, the HMD 1 includes, for example, a wearable unit that is configured like a frame surrounding half of the head from both sides of the head to the back of the head);
an inward facing camera positioned on an inward facing side of the housing, the inward facing camera configured to detect an eye feature of a wearer of the head-mountable electronic device ([0039] Alternatively, the HMD 1 can substantially provide the scene 2-1 from a user viewpoint (first person viewpoint) as illustrated in the left of FIG. 1 by displaying, on the display units 2 in real time, a captured image that is captured by an imaging unit (first imaging unit) that is installed on the HMD 1 and configured to capture an image of an area in an eye-gaze direction of the user);
an external camera disposed on the outward facing side of the housing, the external camera configured to detect a body feature of the wearer outside the face engagement structure ([0042] For example, as illustrated in the center of FIG. 1 and the right of FIG. 1, the HMD 1 controls the display units 2 in response to an instruction from the user P to display, in real time, second captured images 2-2 and 2-3 that are captured by an external camera 32 (second imaging unit) including the user P within the angle of view); and
an outward facing display disposed on the outward facing side of the housing, the outward facing display configured to project a representation of the wearer, the representation based on a stored representation of the wearer and at least one of the eye feature or the body feature ([0117] Alternatively, the predetermined image displayed on the outer display unit 2O may also be an image 2O-5 that is generated by use of an image of the user P as illustrated in FIG. 6B and indicates a situation in which the user P is floating).
Sako does not explicitly disclose the external camera included on the outer display.
However, in the same endeavor, Harrises discloses an external camera included on an outer display ([0440] Interactions with user interface may be tracked using one or more cameras external to the system or part of the system (such as, e.g., using the inward-facing imaging system 500 or the outward-facing imaging system 502)).
Therefore, it would have been obvious to one of ordinary skill in the art to modify the disclosure of Sako to further include Harrises’s outer display with external camera, in order to improve users’ experience such as improving their motor skills.
As to claim 7, the combination of Sako and Harrises discloses a head-mountable display, comprising: a housing (e.g., the housing of inner display unit 2i and outer display 2O, fig. 6B);
a first display positioned on a first side of the housing, the first display configured to project light toward an eye of a user wearing the head-mountable display (e.g., the inner display unit 2i, fig. 6B);
a second display positioned on a second side of the housing opposite the first side, the second display configured to project light away from the user, opposite the first display, and configured to display an avatar of the user on an external surface of the head-mountable display ([0117] Alternatively, the predetermined image displayed on the outer display unit 2O may also be an image 2O-5 that is generated by use of an image of the user P as illustrated in FIG. 6B and indicates a situation in which the user P is floating);
an outward facing sensor configured to detect a facial expression of the user ([0042] For example, as illustrated in the center of FIG. 1 and the right of FIG. 1, the HMD 1 controls the display units 2 in response to an instruction from the user P to display, in real time, second captured images 2-2 and 2-3 that are captured by an external camera 32 (second imaging unit) including the user P within the angle of view);
an inward facing sensor configured to detect an eye feature of the user ([0039] Alternatively, the HMD 1 can substantially provide the scene 2-1 from a user viewpoint (first person viewpoint) as illustrated in the left of FIG. 1 by displaying, on the display units 2 in real time, a captured image that is captured by an imaging unit (first imaging unit) that is installed on the HMD 1 and configured to capture an image of an area in an eye-gaze direction of the user); and
a processor electrically coupled to the outward facing sensor and the inward facing sensor ([0078] The displayed image processing unit 12 is, for example, a so-called video processor).
Sako does not explicitly disclose the processor configured to cause the second display to modify the avatar of the user based on at least one of the facial expression or the eye feature of the user.
However, in the same endeavor, Harrises discloses a processor configured to cause the second display to modify the avatar of the user based on at least one of the facial expression or the eye feature of the user ([0384] the processing modules 70 and 72 may communicate with the remote data repository 74 to create an animation of the user's avatar or to synthesize a user's image based on the information of the user stored in the remote data repository and the detected pose).
Therefore, it would have been obvious to one of ordinary skill in the art to modify the disclosure of Sako to further include Harrises’s facial expression detection method, in order to improve users’ experience.
As to claim 14, the combination of Sako and Harrises discloses a head-mountable device, comprising: a housing comprising an outer surface positioned away from a user wearing the head- mountable device (e.g., the housing of the outer display 2O, fig. 6B);
an external camera configured to detect a facial feature of the user wearing the head- mountable device ([0042] For example, as illustrated in the center of FIG. 1 and the right of FIG. 1, the HMD 1 controls the display units 2 in response to an instruction from the user P to display, in real time, second captured images 2-2 and 2-3 that are captured by an external camera 32 (second imaging unit) including the user P within the angle of view);
an inward facing camera configured to detect an eye feature of the user ([0039] Alternatively, the HMD 1 can substantially provide the scene 2-1 from a user viewpoint (first person viewpoint) as illustrated in the left of FIG. 1 by displaying, on the display units 2 in real time, a captured image that is captured by an imaging unit (first imaging unit) that is installed on the HMD 1 and configured to capture an image of an area in an eye-gaze direction of the user);
a display coupled to an outer surface of the housing, the display configured to project light away from the user ([0117] Alternatively, the predetermined image displayed on the outer display unit 2O may also be an image 2O-5 that is generated by use of an image of the user P as illustrated in FIG. 6B and indicates a situation in which the user P is floating); and
a processor electrically coupled to the external camera and the inward facing camera ([0078] The displayed image processing unit 12 is, for example, a so-called video processor),
Sako does not explicitly disclose the processor configured to generate an avatar of the user based on the facial feature and the eye feature and display the avatar on the display.
However, in the same endeavor, Harrises discloses a processor configured to generate an avatar of the user based on the facial feature and the eye feature and display the avatar on the display ([0384] the processing modules 70 and 72 may communicate with the remote data repository 74 to create an animation of the user's avatar or to synthesize a user's image based on the information of the user stored in the remote data repository and the detected pose).
Therefore, it would have been obvious to one of ordinary skill in the art to modify the disclosure of Sako to further include Harrises’ facial expression detection method, in order to improve users’ experience.
As to claim 2, the combination of Sako and Harrises discloses the head-mountable electronic device of claim 1. The combination further discloses the body feature comprises a facial expression of the wearer (Harrises, [0469] For example, particular body gestures, hand gestures, head pose, and/or facial expressions may be programmed into the system as being particular inputs, which cause particular actions by the display system).
As to claim 3, the combination of Sako and Harrises discloses the head-mountable electronic device of claim 1. The combination further discloses a hand gesture ((Harrises, [0469] For example, particular body gestures, hand gestures, head pose, and/or facial expressions may be programmed into the system as being particular inputs, which cause particular actions by the display system).
As to claim 4, the combination of Sako and Harrises discloses the head-mountable electronic device of claim 1. The combination further discloses the outward facing display comprises a cover glass defining an exterior of the head-mountable electronic device ((Harrises, [0440] Interactions with user interface may be tracked using one or more cameras external to the system or part of the system (such as, e.g., using the inward-facing imaging system 500 or the outward-facing imaging system 502)).
As to claim 5, the combination of Sako and Harrises discloses the head-mountable electronic device of claim 1. The combination further discloses the outward facing display comprises a curved cover glass at least partially defining an exterior of the head-mountable electronic device ((Harrises, [0246] The method of any one of embodiments 1-3, wherein determining information associated with the reflective surface comprises determining one or more of the following: size, shape, location, or semantics information of the mirror).
As to claim 6, the combination of Sako and Harrises discloses the head-mountable electronic device of claim 1. The combination further discloses a processor electrically coupled to the inward facing camera and the external camera, the processor configured to generate an avatar of a head of the wearer based on the eye feature and the body feature ((Harrises, [0469] For example, particular body gestures, hand gestures, head pose, and/or facial expressions may be programmed into the system as being particular inputs, which cause particular actions by the display system).
As to claim 8, the combination of Sako and Harrises discloses the head-mountable display of claim 7. The combination further discloses the outward facing sensor is further configured to detect a hand gesture of the user ((Harrises, [0469] For example, particular body gestures, hand gestures, head pose, and/or facial expressions may be programmed into the system as being particular inputs, which cause particular actions by the display system).
As to claim 9, the combination of Sako and Harrises discloses the head-mountable display of claim 7. The combination further discloses the avatar comprises a real-time representation of the eye feature ((Harrises, [0446] It will be appreciated that the augmented reality content may include real-time information and/or historical data for viewing and analysis).
As to claim 10, the combination of Sako and Harrises discloses the head-mountable display of claim 7. The combination further discloses the avatar is displayed in response to a detection by the outward facing sensor ((Harrises, [0381] As an example of imaging the user's environment and the user, the wearable display system 80 may use the outward-facing imaging system 502 and/or the inward-facing imaging system 500 to acquire images of a pose of the user).
As to claim 11, the combination of Sako and Harrises discloses the head-mountable display of claim 7. The combination further discloses the second display is configured to display a real-time video of the eye feature of the user ((Harrises, [0446] It will be appreciated that the augmented reality content may include real-time information and/or historical data for viewing and analysis).
As to claim 12, the combination of Sako and Harrises discloses the head-mountable display of claim 7. The combination further discloses the avatar is animated ((Harrises, [0509] For example, a semi-transparent animation may be overlaid the user's reflection, to allow the user to follow along with a particular movement).
As to claim 10, the combination of Sako and Harrises discloses the head-mountable display of claim 7. The combination further discloses the second display is configured to cause the head-mountable display to appear transparent ((Harrises, [0445] The display area may be at least partially transparent, and sufficiently transparent to provide a view of an ambient environment through the display area).
As to claim 15, the combination of Sako and Harrises discloses the head-mountable device of claim 14. The combination further discloses the processor is configured to generate a videotelephony transmission of the avatar ((Harrises, [0290] An augmented reality system for conducting a teleconference between a first caller and a second caller).
As to claim 16, the combination of Sako and Harrises discloses the head-mountable device of claim 14. The combination further discloses an outward facing display configured to project the avatar ((Harrises, [0381] As an example of imaging the user's environment and the user, the wearable display system 80 may use the outward-facing imaging system 502 and/or the inward-facing imaging system 500 to acquire images of a pose of the user).
As to claim 17, the combination of Sako and Harrises discloses the head-mountable device of claim 14. The combination further discloses the avatar comprises features selectable by the user ((Harrises, [0029] The method of embodiment 1, wherein subsequently displaying augmented reality content comprises displaying a virtual menu with user-selectable options).
As to claim 18, the combination of Sako and Harrises discloses the head-mountable device of claim 14. The combination further discloses the avatar comprises a real-time representation of the eye feature and the facial feature of the user ((Harrises, [0446] It will be appreciated that the augmented reality content may include real-time information and/or historical data for viewing and analysis).
As to claim 19, the combination of Sako and Harrises discloses the head-mountable device of claim 14. The combination further discloses the external camera is further configured to detect an environment of the user, the processor configured virtually generate a visual representation of the avatar in the environment ((Harrises, [0384] In some embodiments, the remote data repository 74 may be configured to store a world map of a user's environment, a user's avatar, or a user's face model).
As to claim 20, the combination of Sako and Harrises discloses the head-mountable device of claim 14. The combination further discloses the external camera is further configured to detect a hand gesture ((Harrises, [0469] For example, particular body gestures, hand gestures, head pose, and/or facial expressions may be programmed into the system as being particular inputs, which cause particular actions by the display system).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hulbert et al., US PGPUB 20220100271 discloses an electronic device, while the electronic device is worn over a predefined portion of the user's body, displays, via a display generation component arranged on the electronic device opposite the predefined portion of the user's body, a graphical representation of an exterior view of a body part that corresponds to the predefined portion of the user's body. The electronic device detects a change in position of the electronic device with respect to the predefined portion of the user's body. The electronic device, in response to detecting the change in the position of the electronic device with respect to the predefined portion of the user's body, modifies the graphical representation of the exterior view of the body part that corresponds to predefined portion of the user's body in accordance with the detected change in position of the electronic device with respect to the predefined portion of the user's body.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAHLU OKEBATO whose telephone number is (571)270-3375. The examiner can normally be reached Mon - Fri 8:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, WILLIAM BODDIE can be reached at 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAHLU OKEBATO/Primary Examiner, Art Unit 2625
Prosecution Insights