DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-11, and 13-20 are pending.
Response to Arguments
Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 are rejected under 35 U.S.C. 103 as being unpatentable over Sako (US 2018/0003983) in view of Wong (US 2017/0262150) and Kehoe (US 2019/0265781). All reference is to Sako unless indicated otherwise.
Regarding Claim 1 (Currently Amended) Sako teaches a head-mounted device configured to operate in an environment [fig. 1], the head-mounted device comprising:
an inner display [fig. 2 @Internal Display Unit] facing an eye box [fig. 2 @ eye position] configured to display first images [¶0090, “The image processing unit 508 includes an internal image generating unit 508-1 that generates an internal image on the basis of an image signal output from the control unit 501”];
a lens [fig. 2 @Virtual Image Optical Unit] through which the first images [¶0092, “A virtual image optical unit 513 is disposed in front of the display screen of the internal image display panel 511. The virtual image optical unit 513 enlarges and projects the display image of the internal image display panel 511, which is viewed by the user as an enlarged virtual image”] are viewable from an eye box [fig. 2 @eye position];
an outer display [fig. 2 @External Display Unit] configured to display second [¶0090, “in the case of externally displaying the same image as the internal image, the external image generating unit 508-2 is omitted”] images [¶0090, “an external image generating unit 508-2 that generates an external image on the basis of an image signal output from the control unit 501”]; and
an image sensor [¶0081, “The environmental information acquisition unit 504 acquires information related to the environment outside the head-mounted image display device 1, and outputs to the control unit 501 … the environmental information acquisition unit 504 may also be equipped … an image sensor (camera)”, ¶0082, “the control unit 501 controls display operation for displaying … an external image seen from the outside of the relevant device according to environmental information acquired by the environmental information acquisition unit 504”];
configured to gather user input while the inner display is off [¶0139, “although omitted from illustration in FIG. 23 … the state transitions to the Only External Image On state”, ¶0142, “ In … the Only External Image On state in which the external image is turned on, the same image as the internal image or an external-only image that differs from the internal image is displayed ... Furthermore, in response to an instruction from the user given via the input operating unit 502 … the information to display as the external image may be changed”], wherein
an image on an external surface in the environment is adjusted [¶0093, “The projection optical unit 514 enlarges and projects a real image of the external image displayed on the external image display panel 512 onto a wall or the like near the head-mounted image display device 1”, ¶0142 teaches changing the external image based on user input when the inner display is off]
Sako does not teach the image sensor faces away from the eye box and is configured to gather user air gesture input; and an image projected on an external surface is adjusted based on a tracked location at which the air gesture input overlaps the image on the external surface
Wong teaches an image sensor facing away from the eye box [¶0044, “Wearable computing device 10 may also include a camera 26 that is configured to capture images of the environment of wearable computing device 10 from a particular point-of-view. The images could be either video images or still images. The point-of-view of camera 26 may correspond to the direction where HMD 20 is facing”] and
configured to detect user air gesture input [¶0049, “Processor 22 may analyze still images or video images obtained by camera 26 to identify any gesture that corresponds to a control instruction … a gesture that does not involve physical contact with the target device, such as a movement of the wearer's finger, hand, or an object held in the wearer's hand, toward the target device or in the vicinity of the target device, could be recognized as a control instruction”]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of detecting air gestures with an image sensor, as taught by Wong, into the head mounted device taught by Sako in order to control the head mounted device without requiring a user to physically contact or manipulate an input device
Sako in view of Wong does not teach an image projected on an external surface is adjusted based on a tracked position at which the air gesture input overlaps the image on the external surface and wherein the air gesture input is detected at a location between the image sensor and the image
Kehoe teaches an image [fig. 6 @1] projected on an external surface [¶0041, “At step 1, a user wearing a display device (e.g., HMD) sees a projected keyboard on a physical surface (e.g., a desk top)”] is adjusted [¶0043, “At step 7, once the “pressed” key is determined, the keypress event can be simulated in a number of ways, including a visual indication of a key press (step 8) showing press/release of a virtual key”] based on a tracked position [¶0041, “At step 3, user finger tracking can be performed using any combination of camera input (e.g., from the HMD)”] at which the air gesture input overlaps the image on the external surface [¶0042, “At step 4, tracking information can then be further processed to determine where, over the virtual keyboard, the user's fingertips are located to identify a corresponding key. At step 6, keypress selection logic can determine which key is pressed based … the determined finger position (active key) (step 4)”] and wherein
the air gesture input [fig. 6 @finger position] is detected at a location [fig. 6 above image] between the image sensor [¶0041 teaches finger detection sensors are on the HMD] and the image [¶0041 teaches image is on surface]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of detecting the positions of an air gesture on a projected image and adjust the projected image to indicate the overlap positions as taught by Kehoe, into the head mounted device taught by Sako in view of Wong in order to provide visual feedback of a control gesture that does not comprise tactile feedback.
Regarding Claim 2 (Original), Sako in view of Wong and Kehoe teaches the head-mounted device defined in Claim 1 wherein
the air gesture input comprises air gesture input from a user’s hand that is not contacting the head-mounted device [Kehoe: fig. 6 and ¶0041 and ¶0042].
Claims 11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Sako (US 2018/0003983) in view of Lyons (US 2017/0255019). All reference is to Sako unless indicated otherwise.
Regarding Claim 11 (Currently Amended), Sako teaches a head-mounted device, comprising:
a head-mounted housing [fig. 1 @frame];
an inner display [fig. 2 @internal display unit] supported by the head-mounted housing, wherein
the inner display comprises left and right displays [¶0069, “FIG. 2 illustrates the overhead appearance of a head-mounted image display device 1 of the opaque type. As illustrated in the drawing, the head-mounted image display device 1 has internal display units for the left and right eyes on the internal side, that is, the side that faces the user's face”] that are separated by an adjustable distance [¶0069, “the head-mounted image display device 1 is equipped with an interpupillary adjustment mechanism that adjusts the interpupillary width between the right-eye display unit and the left-eye display unit”];
a lens through which images on the inner display [fig. 2 @left and right virtual image optical unit] are viewable from an eye box [fig. 2 @ area in front of user’s eyes];
an outer display [fig. 2 @external display unit] supported by the head-mounted housing, wherein
the outer display [¶0070, “In the head-mounted image display device 1 of the example illustrated in FIGS. 1 and 2, the external display units are disposed at positions in a front-and-back relationship with the internal display units … although the head-mounted image display device 1 is equipped with a pair of left and right external display units, a single external display unit … may also be provided”, replacing fig. 2 left and right external displays with a single display unit overlaps left and right internal displays in fig. 2] overlaps the left and right displays
Sako does not teach a head mounted display configured to store a removable handheld electronic device, wherein the head-mounted housing comprises a recess configured to receive and hold the removable handheld device that is non-overlapping with the outer display; and wireless communications circuitry configured to receive user input from the removable handheld electronic device
Lyons teaches a head mounted display configured to store a removable handheld electronic device [fig. 17B @512], wherein
the head-mounted housing [fig. 17A @518] comprises a recess configured to receive [fig. 17A illustrates sidewall 518 built up near 512 attachment point where 512 fits in recess of built up area] and hold [¶0143, “With reference back to the perspective views of FIGS. 17A and 17B, and also the exploded view of FIG. 19, the remote controls 512 desirably attached to side walls 518 of the main body 502 using docking clips 520. …”, ¶0144, “… the inner anchor portions 526 of each of the docking clips 520 can be pushed through vertical slots 528 formed in the side walls 518 until the anchor portions are past the slots and within the interior cavity 508 of the main body 502. That is, the narrow neck portion 522 has a horizontal length that is substantially the same as the thickness of the side walls 518 such that the clips 520 are held firmly with respect to the main body 502 … Although not shown in great detail, the outer clip portion 524 include attachment structure 525 that mates with corresponding attachment structure 527 provided on the bottom faces of the remote controls 512”] that is non-overlapping with the outer display [figs. 1 and 2 @50, fig. 1 @50 is construed as the single external display device taught by ¶0070 of Sako] and
wireless communications circuitry configured to receive user input from the removable handheld electronic device [¶0090, “the remote controller 30 as illustrated in FIG. 7a, FIG. 7b, FIG. 8a, and FIG. 8b, receives input from the user 70 (not shown) and communicates the input to the mobile computing device 50 (not shown) … wireless communication is preferred”]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of incorporating a single external display for the right and left eyes, as taught by Lyons, into the head mounted device taught by Sako in in order to provide an external display device with self-contained image and communications processing that can function as a standalone smart phone, reducing the overall cost of a user’s electronics.
Regarding Claim 13 (Currently Amended), Sako in view of Lyons teaches the head-mounted device defined in Claim 11 further comprising
magnetic structures [Lyons: ¶0083, “the method of attachment can be accomplished by any of various other methods of attachment, such as … Velcro® magnetics”] in the recess [fig. 17A illustrates 512 fitted in recess] configured to couple the wireless handheld controller [Lyons: [fig. 17A @512] to the head-mounted housing.
Claims 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Sako in view of Wong, Kehoe, and Short (US 2014/0139717). All reference is to Sako unless indicated otherwise.
Regarding Claim 3 (Original), Sako in view of Wong and Kehoe teaches the head-mounted device defined in Claim 1 wherein
the air gesture input comprises air gesture input from an accessory that is not contacting the head-mounted device [Wong: ¶0049, “a gesture that does not involve physical contact with the target device, such as a movement of the wearer's finger, hand, or an object held in the wearer's hand, toward the target device or in the vicinity of the target device, could be recognized as a control instruction”]
Sako in view of Wong does not teach the accessory is selected from the group consisting of: a stylus, a finger-mounted device, and a handheld wireless controller
Short teaches the air gesture accessory is selected from the group consisting of: a stylus, a finger-mounted device, and a handheld wireless controller [¶0028, “In the example shown in FIG. 4, user input device 26 includes an infrared digital stylus 28 and an infrared camera 30 for detecting stylus 28 in workspace 12. Although any suitable user input device may be used, a digital stylus has the advantage of allowing input in three dimensions”, ¶0043]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of holding a digital stylus when providing user input by an air gesture, as taught by Short, into the head mounted device taught by Sako in view of Wong and Kehoe in order to generate three-dimensional gestures using a handheld device that is similar to a writing instrument thereby making the movement corresponding to a gesture more intuitive.
Regarding Claim 4 (Original), Sako in view of Wong and Kehoe teaches the head-mounted device defined in Claim 1
Sako in view of Wong and Kehoe does not teach the image sensor comprises a three-dimensional image sensor
Short teaches an image sensor comprises a three-dimensional image sensor [¶0043, “As noted above, a touch-free infrared digital stylus has the advantage of allowing input in three dimensions, including along work surface 24, without a sensing pad or other special surface”]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of a 3D image sensor, as taught by Short, into the head mounted device taught by Sako in view of Wong and Kehoe in order to recognize three-dimensional non-contact gestures.
Regarding Claim 5 (Original), Sako in view of Wong and Kehoe teaches the head-mounted device defined in Claim 1
Sako in view of Wong and Kehoe does not teach the image sensor comprises a depth sensor
Short teaches an image sensor comprises a depth sensor [¶0043, “… Depth cameras using structured light, time-of-flight, disturbed light pattern, or stereoscopic vision might also be used to enable in-air gesturing or limited touch and touch gesture detection without a touch pad”]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of a depth sensor, as taught by Short, into the head mounted device taught by Sako in view of Wong and Kehoe in order to recognize three-dimensional non-contact gestures.
Claims 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Sako in view of Wong, Kehoe, and Lyons (US 2017/0255019). All reference is to Sako unless indicated otherwise.
Regarding Claim 6 (Previously Presented), Sako in view of Wong and Kehoe teaches the head-mounted device defined in Claim 1 further comprising
a head-mounted housing that supports the inner and outer displays [¶0005, “Since the user may view an image both the front side and the back side of the device housing, a double-sided display device can be a good information providing tool”]
Sako in view of Wong and Kehoe does not teach the head mounted housing configured to store a wireless handheld controller for the head-mounted device
Lyons teaches a head mounted housing [fig. 1 @10] configured to store a wireless handheld controller [fig. 1 @30] for the head-mounted device [¶0092]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate a wireless handheld controller removably attached to the HMD frame, as taught by Lyons, into the head-mounted device taught by Sako in view of Wong and Kehoe in order to provide a convenient device with distinctly shaped buttons allowing the user to select a button based on feel without looking (Lyons: ¶0092).
Regarding Claim 7 (Original), Sako in view of Wong, Kehoe, and Lyons teaches the head-mounted device defined in Claim 6 wherein
the head-mounted housing [Lyons: fig. 17A @518] comprises a recess [Lyons: fig. 17A illustrates sidewall 518 built up near 512 attachment point where 512 fits in recess of built up area] configured to receive the wireless handheld controller [Lyons: fig. 17A @512].
Regarding Claim 8 (Original), Sako in view of Wong, Kehoe, and Lyons teaches the head-mounted device defined in Claim 6 further comprising
magnetic structures configured to couple the wireless handheld controller [Lyons: fig. 1 @30] to the head-mounted housing [Lyons: ¶0083, “the method of attachment can be accomplished by any of various other methods of attachment, such as … Velcro® magnetics”].
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Sako in view of Wong, Kehoe, Lyons and Kim (US 2016/0224176). All reference is to Sako unless otherwise indicated.
Regarding Claim 9 (Original), Sako in view of Wong, Kehoe, and Lyons teaches the head-mounted device defined in Claim 8 wherein
the wireless handheld controller [Lyons: fig. 17A @512] is coupled to the head-mounted housing
Sako in view of Wong, Kehoe, and Lyons does not teach the head-mounted device is configured to transfer wireless power to the wireless handheld controller
Kim teaches the head-mounted device is configured to transfer wireless power [¶0280, “The PMIC may have a wired and/or wireless charging scheme … and an additional circuit for wireless charging, such as a coil loop circuit, a resonance circuit, a rectifier circuit, and the like may be added”]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of wireless charging, as taught by Kim, into the head-mounted device, taught by Sako in view of Wong, Kehoe, and Lyons, in order to provide a convenient method of recharging the device that does not require connecting or disconnecting electrical wiring
Sako in view of Wong, Kehoe, Lyons, and Kim does not teach wirelessly charging the wireless handheld controller
One of ordinary skill in the art would understand the advantages of wirelessly recharging a portable remote control, integrated into the HMD system and how to incorporate the additional circuit required for wireless charging, taught by Kim, in order to wirelessly charge the main battery ion the head-up display and the battery on the remote control device with a reasonable expectation of success.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Sako in view of Wong, Kehoe, and Rochford (US 2017/0068500). All reference is to Sako unless indicated otherwise.
Regarding Claim 10 (Original), Sako in view of Wong and Kehoe teaches the head-mounted device defined in Claim 1
Sako in view of Wong and Kehoe does not teach the outer display is configured to receive user input for controlling external electrical equipment
Rochford teaches the outer display [fig. 1 @110] is configured to receive user input [¶0018, “A touch sensitive device (TSD) 116 may be mounted to the headpiece 102 in a position that permits the person other than the user 104 to activate or otherwise use the TSD 116. While the TSD 116 is shown mounted to a front side of the external display 110”] for controlling [¶0038, “the controller 218 has sufficient processing power to execute a visualization program and provide all functionality of a system … the controller 218 may be communicatively coupled to an external device 206”] external electrical equipment [fig. 2 @206 via 218 and 204]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of controlling external electronic equipment by user input on the head-mounted device, as taught by Rochford, into the head-mounted display, taught by Sako in view of Wong and Kehoe, in order to wirelessly control functionality that cannot be incorporated into the head-mounted display form factor.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Sako in view of Lyons and Kim. All reference is to Sako unless indicated otherwise
Regarding Claim 14 (Original), Sako in view of Lyons teaches the head-mounted device defined in Claim 13
Sako in view of Lyons does not teach the head-mounted device is configured to transfer wireless power to the head-mounted housing
Kim teaches a head-mounted device is configured to transfer wireless power to the head-mounted housing [¶0280, “The PMIC may have a wired and/or wireless charging scheme … and an additional circuit for wireless charging, such as a coil loop circuit, a resonance circuit, a rectifier circuit, and the like may be added”]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of wireless charging a head-mounted device, as taught by Kim, into the head-mounted display, taught by Sako in view of Lyons, in order to conveniently recharge head-mounted device batteries without the need for a charging cable
Sako in view of Lyons and Kim does not teach incorporating the additional circuitry necessary to wirelessly charge the wireless handheld controller when coupled
Before the application was filed one of ordinary skill in the art would have recognized the convenience of recharging the integrated wireless remote control while mounted on the HMD system.
One of ordinary skill in the art would also understand how to incorporate the circuitry required for wireless charging, as taught by Sako in view of Lyons and Kim, in order to wirelessly charge the head-up display main battery as well as the battery powering a wireless remote control with a reasonable expectation of success.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Sako in view of Lyons and Rochford (US 2017/0068500). All reference is to Kim unless indicated otherwise.
Regarding Claim 15 (Original), Sako in view of Lyons teaches the head-mounted device defined in Claim 11, wherein
receive user input from the input device [¶0142, “ in response to an instruction from the user given via the input operating unit 502, the information to display as the external image may be changed”] while the inner display is off [Sako: Only External Image on State], the user input device is a removable handheld electronic device [Lyons: fig. 1 @30]
Sako in view of Lyons does not teach the wireless communications circuitry configured to receive user input and the input device is a removable handheld electronic device
Rochford teaches wireless communications circuitry [¶0038, “ … the controller 218 may be communicatively coupled to an external device 206”, fig. 2 @206 via 218 and 204] configured to receive user input [touchscreen (fig. 1 @116 on external display 110)]
Before the application was filed it would have been obvious to one of ordinary skill in the art to incorporate the concept of transmitting user input via wireless communications, as taught by Rochford, into the head-mounted device, taught by Sako in view of Lyons, in order to control electronic devices not connected to the head mounted device with input from the user wearing the head-mounted device.
Allowable Subject Matter
Claims 16-20 are allowed.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Douglas Wilson whose telephone number is (571)272-5640. The examiner can normally be reached 1000-1800 EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patrick Edouard can be reached at 571-272-7603. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Douglas Wilson/Primary Examiner, Art Unit 2622
Prosecution Insights