DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Drawings
The drawings are objected to because:
Shading does not conform to the standards set forth in 37 CFR 1.84(m), which recites “shading in views is encouraged if it aids in understanding the invention and if it does not reduce legibility. ... Spaced lines for shading are preferred. These lines must be thin, as few in number as practicable, and they must contrast with the rest of the drawings. ... Solid black shading areas are not permitted, except when used to represent bar graphs or color.”
Separate views are not separately labeled. See Figs. 4 and 5, for example.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The abstract of the disclosure is objected to because it exceeds a single paragraph. See line 26, for example. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Applicant is reminded of the proper content of an abstract of the disclosure.
A patent abstract is a concise statement of the technical disclosure of the patent and should include that which is new in the art to which the invention pertains. The abstract should not refer to purported merits or speculative applications of the invention and should not compare the invention with the prior art.
If the patent is of a basic nature, the entire technical disclosure may be new in the art, and the abstract should be directed to the entire disclosure. If the patent is in the nature of an improvement in an old apparatus, process, product, or composition, the abstract should include the technical disclosure of the improvement. The abstract should also mention by way of example any preferred modifications or alternatives.
Where applicable, the abstract should include the following: (1) if a machine or apparatus, its organization and operation; (2) if an article, its method of making; (3) if a chemical compound, its identity and use; (4) if a mixture, its ingredients; (5) if a process, the steps.
Extensive mechanical and design details of an apparatus should not be included in the abstract. The abstract should be in narrative form and generally limited to a single paragraph within the range of 50 to 150 words in length.
See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, and 15-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Held et al (WO 2023048822 A1) in view of Bradski et al (WO 2015192117 A1).
As recited in independent claim 1, Held et al show a system 700 for displaying virtual images in an ambient scene (“mixed-reality display system”, see page 10, line 27), the system 700 comprising: a display device (105 including 720) for displaying virtual images (“virtual image source 720 provides virtual image light 725”, see page 11, line 2) to the user 115 in a user field of view (“superimpose virtual images … over the user’s view of light 610 reflected from real-world objects to thus form a mixed-reality display”, see page 10 lines 23-24) at a first Fe or a second depth Fo; a first imaging device 120 for generating first ambient image data (“outward facing cameras 120 that capture images of the surrounding physical environment may be provided, and these captured images may be rendered on the display device 105 along with computer-generated virtual images that augment the captured images of the physical environment”, see page 5, lines 10-13) in a first portion (portion viewed by one camera 120) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13); a second imaging device (insofar as plural “cameras 120” are disclosed, see page 5, line 10) for generating second ambient image data (“outward facing cameras 120 that capture images of the surrounding physical environment may be provided, and these captured images may be rendered on the display device 105 along with computer-generated virtual images that augment the captured images of the physical environment”, see page 5, lines 10-13) in a second portion (portion viewed by another camera 120) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13), the first portion (from one camera 120) and second portion (from another camera 120) overlapping (inherently) to define a common portion (viewed binocularly; see page 8, lines 21-24, “the components can be extended such that separate displays are provided for each eye of the user in binocular implementations. Such arrangement may facilitate, for example, stereoscopic rendering of virtual images in the FOV of the HMD device 100 and enable other features to be realized on a per-eye basis”) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13) imaged by both the first (one camera 120) and second imaging devices (another camera 120); a processor unit 125 for (inherently) receiving virtual image data specifying a virtual image (insofar as “provide display signals to the display device 105” could not possibly happen without the processor first receiving virtual image data specifying a virtual image, see page 5, lines 18-19), (inherently) receiving the first and second ambient image data (from two cameras 120 and 120), and displaying (see Fig. 7) the virtual image (“virtual images can appear to the user 115 at focal distances Fe and Fo that respectively define two different focal planes at distances d1 and d2”, see page 10, lines 30-31) at the selected depth (see Fe and Fo in Fig. 7).
As recited in independent claim 1, Held et al are silent regarding a target real world object or zone to which the virtual image should appear anchored, identifying, from the first ambient image data, the presence of the target real world object or zone, and determining a position of the target real world object or zone within the first portion of the ambient scene, identifying, from the second ambient image data, the presence of the target real world object or zone, and determining a position of the target real world object or zone within the second portion of the scene, calculating an offset between the position within the first portion and the position within the second portion, selecting, dependent on the offset, a depth for the virtual image.
As recited in independent claim 1, Bradski et al show (see Fig. 88A, for example) a target real world object (the user’s hand 8802, for example) or zone (it is noted by the Examiner that “a target real world object or zone” are recited in the alternative, such that the claim limitation is met by a teaching of one alternative, even in the absence of the other) to which a virtual image 8812 should appear anchored (see anchoring in Fig. 88A), (inherently) identifying, from the first ambient image data (captured image of the user’s hand), the presence of the target real world object (the user’s hand) or zone, and (inherently) determining a position (insofar as the virtual object could not possibly become positioned correctly without such a determination) of the target real world object (the user’s hand) or zone within a first portion (portion seen by one eye, for example) of the ambient scene (the environment around the user), (inherently) identifying, from second ambient image data (another captured image of the user’s hand), the presence of the target real world object (the user’s hand) or zone, and (inherently) determining a position (insofar as the virtual object could not possibly become positioned correctly without such a determination) of the target real world object (the user’s hand) or zone within the second portion (portion seen by the other eye, for example) of the scene (the environment around the user), (inherently) calculating an offset (insofar as correct vergence could not possibly be achieved without such a calculation) between the position (location of the hand within one eye’s field of view) within the first portion (portion seen by one eye) and the position (location of the hand within the other eye’s field of view) within the second portion (portion viewed by the other eye), (inherently) selecting, dependent on the offset, a depth (depth of the hand, for example) for the virtual image 8812.
Moreover, the Examiner finds that the recited identifications, determinations, calculations, and selections were predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to perform the recited identifications, determinations, calculations and selections in the device of Held et al as suggested by Bradski et al. The rationale is as follows: one of ordinary skill in the art would have had reason to render a menu on the user’s hand as taught by Bradski et al (“As illustrated in Fig. 88A, in response to detecting a first defined gesture (e.g., user opening or displaying open palm of hand, user holding up hand), the AR system renders a primary navigation menu in a field of view of the user so as to appear to be on or attached to a portion of the user's hand. For instance, a high level navigation menu item, icon or field may be rendered to appear on each finger”, see Fig. 88A and its description in the text at [1450-1451]).
As recited in claim 2, Held et al are silent regarding identifying the presence of the target real world object or zone in the first or second portion of the ambient scene comprises performing object recognition by comparing the respective ambient image data against data representative of a pre-determined target or set of targets.
As recited in claim 2, Bradski et al disclose “one or more images corresponding to a field of view of a user, wherein the image captures at least one gesture created by the user, and a processor communicatively coupled to the image capturing device configured to identify a set of points as associated with the gesture, and to compare the set of points against a database of predetermined gestures, and to recognize the gesture based at least in part on the comparison” [0024], wherein the database of gestures constitutes data representative of a pre-determined target or set of targets.
Moreover, the Examiner finds that comparing ambient image data against data representative of a pre-determined target or set of targets was predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to compare the ambient images of Held et al against data representative of a pre-determined target or set of targets as taught by Bradski et al. The rationale is as follows: one of ordinary skill in the art would have had reason to detect a gesturing hand so as to initiate rendering a menu on said hand as taught by Bradski et al (“As illustrated in Fig. 88A, in response to detecting a first defined gesture (e.g., user opening or displaying open palm of hand, user holding up hand), the AR system renders a primary navigation menu in a field of view of the user so as to appear to be on or attached to a portion of the user's hand. For instance, a high level navigation menu item, icon or field may be rendered to appear on each finger other than the thumb” [1450]).
As recited in claim 3, Held et al are silent regarding identifying the presence of the target real world object or zone in the first and second portion of the ambient scene comprises performing object recognition by comparing the first ambient image data against the second ambient image data, to identify substantially identical image data.
As recited in claim 3, Bradski et al show identifying the presence of the target real world object (the user’s hand) or zone (it is noted by the Examiner that “the target real world object or zone” is recited in the alternative, such that the claim limitation is met by the prior art teaching of one alternative even in the absence of the other) in the first and second portion of the ambient scene (left and right views of the environment around the user) comprises (inherently) performing object recognition by comparing the first ambient image data (from one camera) against the second ambient image data (from the other camera), to identify substantially identical image data (images of the user’s same hand from two different cameras).
Moreover, the Examiner finds that comparing images of the same hand to identify substantially identical images of the user’s hand was predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to identify substantially identical images of the user’s hand. The rationale is as follows: one of ordinary skill in the art would have had reason to recognize the user’s 3-D hand as taught by Bradski et al (“the system may recognize a 3D object in the real world” [1133]).
As recited in claim 4, Held et al show a user-mountable frame 110 for holding the display device (105 including 720) in the user 115 field of view (“display device 105 and a frame 110 that wraps around the head of a user 115 to position the display device near the user’s eyes to provide a mixed-reality experience to the user”, see page 4 lines 30-31).
As recited in claims 5 and 6, Held et al show that first imaging device 120 is mounted at the frame 110. Held et al further disclose a second imaging device (insofar as plural “outward facing cameras 120” are disclosed, see page 5, line 10).
As recited in claims 5 and 6, Held et al are silent regarding whether said second imaging device is mounted at said frame.
There is no invention in relocating known parts, when the functioning of the apparatus is not changed by the relocation. In re Japikse, 181 F.2d 1019, 86 USPQ 70 (CCPA 1950). In this case, the functioning of the camera is not changed by mounting it on said frame.
Moreover, the Examiner finds that the recited second imaging device location was predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to arrive at the recited second imaging device location in the course of routine design choice. The rationale is as follows: one of ordinary skill in the art would have had reason to try any known imaging device location, including the claimed imaging device location, in the absence of criticality as taught by Held et al (see location of camera 120 in Fig. 1).
As recited in claim 6, Held et al are silent regarding the recited location of the first and second imaging devices
There is no invention in relocating known parts, when the functioning of the apparatus is not changed by the relocation. In re Japikse, 181 F.2d 1019, 86 USPQ 70 (CCPA 1950). In this case, the functioning of the cameras is not changed by mounting in the recited locations.
Moreover, the Examiner finds that the recited locations were predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to arrive at the recited locations in the course of routine design choice. The rationale is as follows: one of ordinary skill in the art would have had reason to capture ambient images approximating the user’s view from two eyes as was notoriously well known in the art.
As recited in claim 7, Held et al show that the common portion (inherently) occupies at least substantially all of a user’s binocular field of view.
As recited in claim 8, Held et al are silent regarding whether the offset is calculated by using an angular position of the object or zone from the first imaging device; and an angular position of the object or zone from the second imaging device.
As recited in claim 8, Bradski et al show that the offset is (inherently) calculated by using an angular position of the object or zone from the first imaging device; and an angular position of the object or zone from the second imaging device.
Moreover, the Examiner finds that calculating offset using angular positions was predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to calculate the offset using angular positions in the device of Held et al as taught by Bradski et al. The rationale is as follows: one of ordinary skill in the art would have had reason to determine the 3-D location of the user’s hand for the purpose of anchoring the virtual image to the hand as was known in the art.
As recited in claim 15, Held et al are silent regarding a helmet.
As recited in claim 15, Bradski et al show a helmet 80.
Moreover, the Examiner finds that a helmet was predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to include the device of Held et al in a helmet as was known in the art. The rationale is as follows: one of ordinary skill in the art would have had reason to protect the user’s head from injury as was well known in the art.
As recited in independent claim 16, Held et al show a system 700 for displaying virtual images in an ambient scene (“mixed-reality display system”, see page 10 line 27), the system comprising: a display device (105 including 720) for displaying virtual images (“virtual image source 720 provides virtual image light 725”, see page 11, line 2) in a user 115 field of view (“superimpose virtual images … over the user’s view of light 610 reflected from real-world objects to thus form a mixed-reality display”, see page 10 lines 23-24) at a first Fe or a second depth Fo; a first imaging device 120 for generating first ambient image data (“outward facing cameras 120 that capture images of the surrounding physical environment may be provided, and these captured images may be rendered on the display device 105 along with computer-generated virtual images that augment the captured images of the physical environment”, see page 5, lines 10-13) in a first portion (portion viewed by one camera 120) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13); a second imaging device (insofar as plural “cameras 120” are disclosed, see page 5, line 10) for generating a second ambient image data (“outward facing cameras 120 that capture images of the surrounding physical environment may be provided, and these captured images may be rendered on the display device 105 along with computer-generated virtual images that augment the captured images of the physical environment”, see page 5, lines 10-13) in a second portion (portion viewed by another camera 120) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13), the first portion (from one camera 120) and second portion (from another camera 120) overlapping (inherently) to define a common portion (viewed binocularly; see page 8, lines 21-24, “the components can be extended such that separate displays are provided for each eye of the user in binocular implementations. Such arrangement may facilitate, for example, stereoscopic rendering of virtual images in the FOV of the HMD device 100 and enable other features to be realized on a per-eye basis”) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13) imaged by both the first (one camera 120) and second imaging devices (another camera 120); a processor unit 125 (inherently) configured to receive virtual image data specifying a virtual image (insofar as “provide display signals to the display device 105” could not possibly happen without the processor first receiving virtual image data specifying a virtual image, see page 5, lines 18-19), receive the first and second ambient image data (from two cameras 120 and 120), and display (see Fig. 7) the virtual image (“virtual images can appear to the user 115 at focal distances Fe and Fo that respectively define two different focal planes at distances d1 and d2”, see page 10, lines 30-31) at the selected depth (see Fe and Fo in Fig. 7).
As recited in independent claim 16, Held et al are silent regarding a target real world object or zone to which the virtual image should appear anchored, identify, from the first ambient image data, the presence of the target real world object or zone, and determine a position of the target real world object or zone within the first portion of the ambient scene, identify, from the second ambient image data, the presence of the target real world object or zone, and determine a position of the target real world object or zone within the second portion of the ambient scene, calculate an offset between the position within the first portion and the position within the second portion, select, dependent on the offset, a depth for the virtual image.
As recited in independent claim 16, Bradski et al show (see Fig. 88A, for example) a target real world object (the user’s hand 8802, for example) or zone (it is noted by the Examiner that “a target real world object or zone” are recited in the alternative, such that the claim limitation is met by a teaching of one alternative, even in the absence of the other) to which the virtual image 8812 should appear anchored (see anchoring in Fig. 88A), (inherently) identify, from the first ambient image data (captured image of the user’s hand), the presence of the target real world object (the user’s hand) or zone, and (inherently) determine a position (insofar as the virtual object could not possibly become positioned correctly without such a determination) of the target real world object (the user’s hand) or zone within the first portion (portion seen by one eye, for example) of the ambient scene (the environment around the user), (inherently) identify, from the second ambient image data (another captured image of the user’s hand), the presence of the target real world object (the user’s hand) or zone, and (inherently) determine a position (insofar as the virtual object could not possibly become positioned correctly without such a determination) of the target real world object (the user’s hand) or zone within the second portion (portion seen by the other eye, for example) of the ambient scene (the environment around the user), (inherently) calculate an offset (insofar as correct vergence could not possibly be achieved without such a calculation) between the position (location of the hand within one eye’s field of view) within the first portion (portion seen by one eye) and the position (location of the hand within the other eye’s field of view) within the second portion (portion viewed by the other eye), (inherently) select, dependent on the offset, a depth (depth of the hand, for example) for the virtual image 8812.
Moreover, the Examiner finds that identifying, determining, calculating, and selecting was predictable before the effective filing date.
It would have been obvious to one of ordinary skill in the art prior to the effective filing date to identify, determine, calculate, and select in the system of Held et al as taught by Bradski et al. The rationale is as follows: one of ordinary skill in the art would have had reason to render a menu on the user’s hand as taught by Bradski et al (“As illustrated in Fig. 88A, in response to detecting a first defined gesture (e.g., user opening or displaying open palm of hand, user holding up hand), the AR system renders a primary navigation menu in a field of view of the user so as to appear to be on or attached to a portion of the user's hand. For instance, a high level navigation menu item, icon or field may be rendered to appear on each finger”, see Fig. 88A and its description in the text at [1450-1451]).
Regarding claim 17: See teachings, findings, and rationale above for claim(s) 2 and/or 3.
Regarding claim 18: See teachings, findings, and rationale above for claim(s) 4-6.
As recited in independent claim 19, Held et al show a system 700 comprising: a display device for displaying virtual images in a user field of view (“mixed-reality display system”, see page 10, line 27); a first imaging device 120 configured to generate first ambient image data (“outward facing cameras 120 that capture images of the surrounding physical environment may be provided, and these captured images may be rendered on the display device 105 along with computer-generated virtual images that augment the captured images of the physical environment”, see page 5, lines 10-13) in a first portion (portion viewed by one camera 120) of an ambient scene (“captured images of the physical environment”, see page 5, lines 12-13); a second imaging device (insofar as plural “cameras 120” are disclosed, see page 5, line 10) configured to generate second ambient image data (“outward facing cameras 120 that capture images of the surrounding physical environment may be provided, and these captured images may be rendered on the display device 105 along with computer-generated virtual images that augment the captured images of the physical environment”, see page 5, lines 10-13) in a second portion (portion viewed by another camera 120) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13), the first portion (from one camera 120) and second portion (from another camera 120) overlapping (inherently) to define a common portion (viewed binocularly; see page 8, lines 21-24, “the components can be extended such that separate displays are provided for each eye of the user in binocular implementations. Such arrangement may facilitate, for example, stereoscopic rendering of virtual images in the FOV of the HMD device 100 and enable other features to be realized on a per-eye basis”) of the ambient scene (“captured images of the physical environment”, see page 5, lines 12-13) imaged by both the first (one camera 120) and second imaging devices (another camera 120); a processor unit 125 (inherently) configured to receive virtual image data specifying a virtual image (insofar as “provide display signals to the display device 105” could not possibly happen without the processor first receiving virtual image data specifying a virtual image, see page 5, lines 18-19) and (inherently) receive the first and second ambient image data (from two cameras 120 and 120), and display (see Fig. 7) the virtual image (“virtual images can appear to the user 115 at focal distances Fe and Fo that respectively define two different focal planes at distances d1 and d2”, see page 10, lines 30-31) at the selected depth (see Fe and Fo in Fig. 7).
As recited in independent claim 19, Held et al are silent regarding a target real world object or zone to which the virtual image should appear anchored, identify, from the first ambient image data, the presence of the target real world object or zone, and determine a position of the target real world object or zone within the first portion of the ambient scene, identify, from the second ambient image data, the presence of the target real world object or zone, and determine a position of the target real world object or zone within the second portion of the ambient scene, calculate an offset between the position within the first portion and the position within the second portion, wherein the offset is calculated by using an angular position of the object or zone from the first imaging device, and an angular position of the object or zone from the second imaging device, select, dependent on the offset, a depth for the virtual image.
See teachings, findings, and rationale above for claims 1, 8, and 16.
Allowable Subject Matter
Claim(s) 9-14, and 20 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 9: The prior art of record is silent regarding calculating the offset by subtracting one angular position from another.
Regarding claims 10 and 20 and their dependent claims 11-13: The prior art of record is silent regarding applying a threshold to the offset.
Regarding claim 14: The prior art of record is silent regarding first and second image sources for generating optical signals bearing virtual images at the first and second depths. The closest prior art of record is Held et al (WO 2023048822 A1), wherein polarization of light from a single source is switched by a modulator in order to enable “rapid state switching to construct a temporally multiplexed mixed-reality scene having appropriate focus cues to provide a comfortable visual experience no matter where in the scene the HMD user is accommodating” (see Fig. 2, lines 6-12).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Julie Anne Watko whose telephone number is (571)272-7597. The examiner can normally be reached Monday-Tuesday 9AM-5PM, Wednesday 10:30AM-5PM, Thursday-Friday 9AM-5PM, and occasional Saturdays.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached at 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
JULIE ANNE WATKO
Primary Examiner
Art Unit 2627
/Julie Anne Watko/Primary Examiner, Art Unit 2627
07/02/2026
Prosecution Insights