Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 and 26 of U.S. Patent No. 9928629. Although the claims at issue are not identical, they are not patentably distinct from each other because of the following.
The table below shows the correspondence between claim 1 of the ‘204 Application and claim 1 of the ‘629 patent.
‘204 Application
‘629 Patent
(Claim 1)
A method, comprising:
positioning a retaining structure in proximity to an eye of a subject, the eye of the subject having a pupil with a pupil diameter;
mounting an optical combiner on the structure in front of the eye;
coating the optical combiner with a pixelated screen, comprising an array of variably transparent pixels;
mounting at least one image capturing device on the structure so as to capture an image of a scene viewed by the eye;
mounting a projector on the structure the projector being configured to project at least one of a portion of the captured image and a stored image onto a section of the screen at a selected location thereof;
rendering the section of the screen at least partially opaque;
selecting the location of the section in response to a region of interest in the scene identified by analysis of the captured image;
and determining a dimension of the section in response to the pupil diameter.
(Claim 20)
A method, comprising:
positioning a retaining structure in proximity to an eye of a subject, the eye of the subject having a pupil with a pupil diameter;
mounting an optical combiner on the structure in front of the eye;
coating the optical combiner with a pixelated screen, comprising an array of variably transparent pixels;
mounting at least one image capturing device on the structure so as to capture an image of a scene viewed by the eye;
mounting a projector on the structure the projector being configured to project at least one of a portion of the captured image and a stored image onto a section of the screen at a selected location thereof;
rendering the section of the screen at least partially opaque;
selecting the location of the section in response to a region of interest in the scene identified by analysis of the captured image;
determining a dimension of the section in response to the pupil diameter
….
Claims 2-20 of the ‘204 application correspond to claims 2-6, 1, 7, 8, 26 and 9-18, respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bar-Zeev et al. (PGPUB Document No. US 2012/0068913) in view of Giudicelli et al. (PGPUB Document No. US 2016/0178910) in view of Jagga et al. (PGPUB Document No. US 2017/0076501) in view of Morita (PGPUB Document No. US 2003/0156144).
Regarding claim 1, Bar-Zeev teaches a method, comprising:
A retaining structure, configured to be positioned in proximity to an eye of a subject (frame of FIG.7A corresponds to the claimed "retaining structure" (Bar-Zeev: FIG.7A, 0083)), the eye of the subject having a pupil with a pupil diameter (the eye of the user wearing the wearable head-up display comprise has eyes comprising of pupils. A pupil diameter is an inherent physical property);
Mounting an optical combiner on the structure in front of the eye (optical component 720 such as a beam splitter which mixes an augmented reality image with light from the real-world scene (Bar-Zeev: FIG.7A, 0085));
Coating the optical combiner with a pixelated screen, comprising an array of variably transparent pixels, (light-transmissive opacity filter 723 (Bar-Zeev: 0083) maybe an LCD (Bar-Zeev: 0050));
Mounting a projector mounted on the structure (augmented reality emitter 708, 716 mounted on the HMD frame as shown in FIG.7A (Bar-Zeev: 0085-0086, FIG.7A)) and configured to project at least one of a portion of the captured image and a stored image onto a section of the screen (The augmented projected image as disclosed in 0080 corresponds to the stored image as claimed (Bar-Zeev: FIG.7A) at a selected location thereof (the location of the augmented image is a function of tracking the user's eye/pupil (Bar-Zeev: 0048)),
Rendering the section of the screen at least partially opaque (The opacity filter provides a region 402 of increased opacity (Bar-Zeev: 0018)),
Selecting the location of the section in response to a region of interest in the scene (The location at which the superimposed AR content is a function of tracking the eye of the user (Bar-Zeev: 0066). The location at which the user views correspond to the selected location).
However, Bar-Zeev does not expressly teach mounting at least one image capturing device on the structure configured to capture an image of a scene viewed by the eye (the "scene" of Bar-Zeev is the real world image that is made visible by making the opacity filter completely transparent not a projection of a captured scene as claimed).
Morita teaches, in a similar field of endeavor, mounting at least one image capturing device on the structure configured to capture an image of a scene viewed by the eye (Morita: 0035, 0037).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of Bar-Zeev such as to implement the real scene and virtual scene compositing system/device of Morita, because this is merely one of the many well-known methods (augmenting information in a see-thru-HMD type system (Bar- Zeev), non-see-thru type system (Morita), etc.) of implementing augmented/mixed reality.
Further, the combined teachings as applied above does not expressly teach the region of interest identified by analysis of the captured image. Jagga teaches, in a similar field of endeavor (AR for medical/surgical system), wherein the region of interest is identified by analysis of the captured image (the use of a marker that allows the system to identify the medical instrument (Jagga: 0059), wherein identifying the marker requires analyzing the image of scene).
Therefore, at the time of the invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize a marker as taught by Jagga, because this enables an added level of accuracy when carrying out the AR operations of the combined teachings above.
And further, the combined teachings as applied above does not expressly teach determining a dimension of the section in response to the pupil diameter. Giudicelli teaches determining a dimension of the section in response to the pupil diameter ("From the size of the eye pupil and the focal length of the collimation element 107, the size of the intermediate image formed in the object focal plane of the collimation element 107 is determined" (Giudicelli: 0101)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to utilize the AR information superimposing teachings of Giudicelli to the AR system of the combined teachings above, because this enables an improved method for displaying AR information to the user. Further, this merely is one of the many well-known ways of providing AR information to the user, wherein the combination of teachings yields predictable results.
Regarding claim 19, the combined teachings teach the method according to claim 1, wherein the retaining structure comprises a spectacle frame (the frame is similar to a conventional eyeglasses frame (Bar-Zeev: 0083, FIG.7A)).
Regarding claim 20, the combined teachings teach the method according to claim 1, wherein the retaining structure comprises a helmet having a head-up display (Helmet (Bar-Zeev: 0083)).
Claims 2, 3, 6, 11, 12, 14 and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bar-Zeev in view of Giudicelli in view of Jagga in view of Jagga as applied to the claim(s) above, and further in view of Hoffmann et al. (PGPUB Document No. US 2015/0070347).
Regarding claim 2, the combined teachings as applied above teaches the method according to claim 1, and comprising identifying objects viewed by the camera manipulated by the user ("region of interest") by receiving radiation by the image capturing device from objects within the real world scene (the inherently property of a camera requires light radiated by object to be received at its sensors such as the cameras of Morita (Morita: 0035, 0037)).
However, the combined teachings as applied above does not expressly teach the process of above being applied to identifying the region of interest being at least one marker located at the region of interest. Hoffmann teaches the concept of utilizing markers within an AR system (Hoffmann: 0109) by a processor (Hoffmann: 0054), wherein the user selectively determining to view a particular marker corresponds to the user's region of interest.
Therefore, at the time of the invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the AR marker teachings of Hoffmann, because this enables an added variety of ways to present AR information to the user.
Regarding claim 3, the combined teachings teach the method according to claim 1, and comprising configuring a further image capturing device (camera coupled to the AR camera of Hoffmann (Hoffmann: 0046)) to identify the region of interest in response to received radiation from at least one marker located at the region of interest (refer to the rejection above with regards to claim 2).
Regarding claim 6, the combined teachings above do not expressly teach but Hoffmann teaches the method according to claim 1, and comprising positioning at least one marker in proximity to the region of interest, and detecting the marker in the captured image so as to identify the region of interest (the AR device of Hoffmann recognizing the marker (Hoffmann: 0109)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the AR marker teaching of Hoffmann, because this enables an added variety of ways to present AR information to the user.
Regarding claim 11, the combined teachings above do not expressly teach but Hoffmann teaches the method according to claim 1, wherein the dimension of the section is determined so that the region of interest is occluded (Hoffman teaches displaying AR overlays to the user, wherein the area on which the AR overlay is displayed occludes the corresponding region of the object in the real world (see FIG.9A of Hoffmann where a portion of the book is occluded by AR overlays 905 and 906. The region of book that the user is looking at corresponds to the region of interest.).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the AR overlay teachings of Hoffmann, because this enables an added variety of ways to present AR information to the user..
Regarding claim 12, the combined teachings teach the method according to claim 1, wherein a region surrounding the region of interest is partially occluded (peripheral AR overlays that are not the main focus (region of interest) of the user results in the surrounding region being partially occluded by said peripheral AR overlays).
Regarding claim 14, the combined teachings teach the method according to claim 12, and comprising deriving from the captured image an image corresponding to the region surrounding the region of interest (the AR overlay such as that of Hoffmann in FIG.9A), and projecting the derived image onto an area of the screen surrounding the at least partially opaque section of the screen (overlaying the AR overlay to the user at the screen of the HMD as taught by Bar-Zeev and Morita (see rejection to claim 1 above)).
Regarding claim 16, the combined teachings above do not expressly teach but Hoffman teaches the method according to claim 1, wherein the dimension of the section is determined in response to a size of the region of interest (Hoffman teaches the concept of the size of AR overlays being proportionate to the size of the real world object such as the highlight indicators surrounding the shoe in image 1004 of FIG. 10. At least in this case, the dimension of the AR overlay, which also directly corresponds to its size in the section is determined in response to the size of the region of interest (region at which the user focused AR information is being displayed).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to implement the AR overlaying teachings of Hoffman, because this provides an added variety of applications for AR devices.
Regarding claim 17, the combined teachings above do not expressly teach but Hoffmann teaches the method according to claim 1, wherein the dimension of the section is determined so that an area greater than the region of interest is occluded (Hoffman teaches, wherein the dimension of the section is determined so that an area greater than the region of interest is occluded (AR information displayed in image 1004 of FIG.10 of Hoffmann is greater than the area at which the user focused object (shoe) is located. Again the size of the AR overlay directly correlates to its size at the section is being projected at).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to implement the AR overlaying teaching of Hoffman, because this provides an added variety of applications for AR devices.
Regarding claim 18, the combined teachings above do not expressly teach but Hoffmann teaches the method according to claim 1, wherein the dimension of the section is determined so that an area less than the region of interest is occluded (However, Hoffmann teaches, wherein the dimension of the section is determined so that an area less than the region of interest is occluded (the AR overlays 905 and 906 are smaller than the region at which the book is located within the real world scene.).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to implement the AR overlaying teachings of Hoffman, because this provides an added variety of applications for AR devices.
Claims 4 and 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bar-Zeev in view of Giudicelli in view of Jagga in view of Jagga in view of Hoffmann as applied to the claim(s) above, and further in view of Balan et al. (PGPUB Document No. US 2012/0306850).
Regarding claim 4, the combined teachings as applied above teach the method according to claim 3, wherein the at least one image capturing device is configured to operate in the visible spectrum, (the detection of markers using reflected visible light (Hoffmann: 0046, 0109)),
However, the combined teachings above do not expressly teach but Balan teaches wherein the further image capturing device is configured to operate in the non-visible spectrum (capture device 58 emitting IR light (Balan: 0040) to recognize IR markers within the real world scene (Balan: 0049)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to further utilize the IR markers of Balan, because this enables an added variety of ways of implementing an AR experience to the user.
Regarding claim 5, the method according to claim 4, and comprising radiating radiation in the non- visible spectrum towards the region of interest ("capture device 58 may emit an infrared light onto the capture area" (Balan: 0040)).
Claim 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bar- Zeev in view of Giudicelli in view of Jagga in view of Jagga as applied to the claim(s) above, and further in view of Teiwes et al. (PGPUB Document No. US 2005/0024586).
Regarding claim 9, the combined teachings above do not expressly teach but Teiwes teaches the method according to claim 1, and comprising determining an initial pupil diameter in response to analysis of a reflected image of the subject while the subject gazes into a mirror in a known ambient light brightness (“The Multidimensional Eye Position Measurement Subsystem may observe the eye either through the stabilized mirror or directly along the optical axis and provide a measurement of pupil size" (Teiwes: 0064). Further, the use of brightness threshold (Teiwes: 076) requires knowing the brightness as claimed).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to utilize the pupil diameter measuring teaching of Teiwes to the combined teachings above, because this is merely one of the many well-known methods of determining the pupil diameter. Further, the combination of teachings yields predictable results.
Allowable Subject Matter
Claims 7, 8, 10, 13 and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616
Prosecution Insights