DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims *** rejected on the ground of nonstatutory double patenting as being unpatentable over claims *** of U.S. Patent No. 9928629.
Although the claims at issue are not identical, they are not patentably distinct from each other because of the following.
The table below shows the correspondence between claim 22 of the ‘608 Application and the corresponding claims of the 9928629 patent.
‘608 Application
‘629 Patent
(Claim 22)
A wearable augmented reality assembly comprising:
a retaining structure in the form of spectacles or a head-mounted structure;
a first optical combiner coupled to the retaining structure and configured to be positioned in front of a right eye of a professional;
a second optical combiner coupled to the retaining structure and configured to be positioned in front of a left eye of the professional;
a liquid crystal display screen formed of an array of pixels;
a first camera mounted on the retaining structure and configured to capture images of scenes viewed by the right eye of the professional in the visible spectrum; a second camera mounted on the retaining structure and configured to capture images of scenes viewed by the left eye of the professional in the visible spectrum;
a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of elements of a scene in front of the wearable augmented reality assembly;
and a processor mounted within the retaining structure,
wherein the elements comprise marker elements on one or more region of interest ("ROI") acquisition markers and one or more identifying elements coupled to a surgical device,
and wherein at least some pixels of the array of pixels are variably transparent,
and wherein an opacity of the at least some of the pixels of the array of pixels is configured to be controlled by the processor.
(Claim 1)
Apparatus, comprising:
a retaining structure, configured to be positioned in proximity to an eye of a subject, the eye of the subject having a pupil with a pupil diameter;
an optical combiner mounted on the structure in front of the eye;
(Claim 4)
he apparatus according to claim 3, wherein the at least one image capturing device is configured to operate in the visible spectrum, and wherein the further image capturing device is configured to operate in the non- visible spectrum.
(Claim 1)
a pixelated screen, comprising an array of variably transparent pixels, coating the optical combiner;
at least one image capturing device mounted on the structure configured to capture an image of a scene viewed by the eye; a projector mounted on the structure and configured to project at least one of a portion of the captured image and a stored image onto a section of the screen at a selected location thereof;
(Claim 5)
The apparatus according to claim 4, and comprising a radiator configured to radiate radiation in the non-visible spectrum towards the region of interest.
(Claim 1)
and a processor, configured to render the section of the screen at least partially opaque, to select the location of the section in response to a region of interest in the scene identified by analysis of the captured image,
(Claim 2)
The apparatus according to claim 1, wherein the processor is configured to identify the region of interest in response to radiation received by the image capturing device from at least one marker located at the region of interest.
(Claim 1)
a pixelated screen, comprising an array of variably transparent pixels, coating the optical combiner;
and a processor, configured to render the section of the screen at least partially opaque, to select the location of the section in response to a region of interest in the scene identified by analysis of the captured image,
The table below shows the correspondence between claims 23, 29-34 and 36-40 of the ‘608 application and the corresponding claims of the ‘629 patent.
‘608
23
29
30
31
32
33
34
36
37
38
39
40
‘629
5
17
18
1
1
1
1
1
12
4
3
1
Claims 22, 23, 26, 27 and 39 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1 and 6 of U.S. Patent No. 10382748 in view of Balan et al. (PGPUB Document No. US 2012/0306850).
Although the claims at issue are not identical, they are not patentably distinct from each other because of the following.
The table below shows the correspondence between claim 22 of the ‘608 Application and claims 1 and 6 of the ‘748 patent.
‘608 Application
‘748 Patent
(Claim 22)
A wearable augmented reality assembly comprising:
a retaining structure in the form of spectacles or a head-mounted structure;
a first optical combiner coupled to the retaining structure and configured to be positioned in front of a right eye of a professional;
a second optical combiner coupled to the retaining structure and configured to be positioned in front of a left eye of the professional;
a liquid crystal display screen formed of an array of pixels;
a first camera mounted on the retaining structure and configured to capture images of scenes viewed by the right eye of the professional in the visible spectrum;
a second camera mounted on the retaining structure and configured to capture images of scenes viewed by the left eye of the professional in the visible spectrum;
and a processor mounted within the retaining structure, wherein the elements comprise marker elements on one or more region of interest ("ROI") acquisition markers and one or more identifying elements coupled to a surgical device,
and wherein at least some pixels of the array of pixels are variably transparent,
and wherein an opacity of the at least some of the pixels of the array of pixels is configured to be controlled by the processor.
(Claim 1)
A method, comprising:
positioning a retaining structure in proximity to a left eye and a right eye of a subject;
mounting a first optical waveguide on the structure in front of the left eye
and a second optical waveguide on the structure in front of the right eye;
coating the first optical waveguide with a first pixelated screen comprising a first array of variably transparent pixels; coating the second optical waveguide with a second pixelated screen comprising a second array of variably transparent pixels;
mounting at least one image capturing device on the structure, the device being configured to capture an image of a scene viewed by the left and right eyes;
mounting at least one image capturing device on the structure, the device being configured to capture an image of a scene viewed by the left and right eyes;
(Claim 6)
positioning a marker in proximity to the region of interest, and using the marker to estimate a distance from region of interest to the left and right eyes.
(Claim 1)
coating the first optical waveguide with a first pixelated screen comprising a first array of variably transparent pixels; coating the second optical waveguide with a second pixelated screen comprising a second array of variably transparent pixels;
However, the ‘608 application does not expressly teach but Balan teaches a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of elements of a scene in front of the wearable augmented reality assembly (capture 58 emitting IR light (Balan: 0040) to recognize IR markers within the real-world scene (Balan: 0049));
And a processor mounted within the retaining structure, wherein the elements comprise marker elements on one or more region of interest ("ROI") acquisition markers and one or more identifying elements coupled to a surgical device (“one or more objects being tracked may be augmented with one or more markers such as an IR retroreflective marker” (Balan: 0049)),
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the ‘608 patent such as to utilize the IR markers of Balan, because this enables an added variety of ways of implementing AR experience to the user.
Regarding claim 23, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the sensor comprises a radiator configured to project the radiation in the near infra-red spectrum (“the IR light component 34 of the capture device 58 may emit an infrared light onto the capture area” (Balan: 0040)).
Regarding claim 26, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the marker elements on the one or more ROI acquisition markers comprise retro-reflectors that reflect the radiation projected by the sensor (Balan teaches an image camera component 32 that includes an IR light component 34 (Balan: 0040) and wherein the one or more device identifying elements and the ROI marker comprises infrared reflective elements (Balan: 0049)).
Regarding claim 27, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the one or more identifying elements comprise reflectors and/or radiators (obtain location and orientation information by detecting, tracking, and triangulating the position of physical tags (e.g., reflective markers) or emitters (e.g., LEDs) (Balan: 0064, 0049)).
Regarding claim 39, the combined teachings teach the wearable augmented reality assembly of claim 38, further comprising a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of the one or more marker elements in front of the wearable augmented reality assembly (“The image camera component 32 may include an IR light component 34…the IR light component 34 of the capture device 58 may emit an infrared light onto the capture area” (Balan: 0040)).
Claims 28, 29, 30, 33-37 and 41 are rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘748 Patent in view of Balan as applied to the claim(s) above, and further in view of Bar-Zeev et al. (PGPUB Document No. US 2012/0068913).
Regarding claim 28, the combined teachings teach the wearable augmented reality assembly of claim 22, further comprising a second sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of accurately overlaying AR information.
Regarding claim 29, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the retaining structure is in the form of spectacles (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because AR spectacles enable an improved AR experience to the user.
Regarding claim 30, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the retaining structure is in the form of a head-mounted structure (“FIG. 1 depicts an example embodiment of an optical see-through HMD device with an augmented reality capability” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because an AR HMD device enables an improved AR experience to the user.
Regarding claim 33, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen is coated on at least one optical combiner (Bar-Zeev teaches the opacity filter can be provided in or on another light-transmissive lens material, which corresponds to being “coated on” as presently recited (Bar-Zeev: 0083). Further, Bar-Zeev teaches the opacity filter can be placed physically behind an optical display component which introduces the augmented reality image to the user's eye (Balan: 0042)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 34, the combined teachings teach the wearable augmented reality assembly of claim 33, wherein the at least one optical combiner comprises a pair of see through glasses, with an optical combiner adapted to be positioned in front of each eye of the professional (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because the see-through lens enables an improved AR experience to the user.
Regarding claim 35, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen comprises a liquid crystal display (light-transmissive opacity filter 723 and 721 can be a see-through LCD panel (Bar-Zeev: 0083, 0085, 0053, 0050, FIG.7C)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 36, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen comprises a pixelated screen comprised of an array of pixels (“The opacity filter can include a dense grid of pixels” (Bar-Zeev: 0051)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing an AR screen.
Regarding claim 37, the combined teachings teach the wearable augmented reality assembly of claim 36, wherein the processor is configured to control an opacity of the array of pixels such that at least the section of the display screen configured to be rendered at least partially opaque is fully occluded (“opacity filter driver can drive pixels in the opacity filter 106 such as by addressing each pixel by a row and column address and a voltage which indicates a desired degree of opacity, from a minimum level which is most light-transmissive level to a maximum level which is most opaque or least light-transmissive” (Bar-Zeev: 0063)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 41, the combined teachings teach the wearable augmented reality assembly of claim 32, further comprising a sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of accurately overlaying AR information.
Claims 31, 38 and 40 are rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘748 Patent in view of Balan as applied to the claim(s) above, and further in view of Morita (PGPUB Document No. US 2003/0156144).
Regarding claim 31, the combined teachings teach the wearable augmented reality assembly of claim 22, further comprising one or more micro-projectors configured to display images captured by the first camera and the second camera (“the actually taken images photo-taken by the video cameras 203L, 203R can be converted to forms which can be synthesized with the virtual space image in the processing apparatus 307. Further, the graphic boards 353, 352 for left and right eyes are connected to the displaying portions (display devices) 201L, 201R for left and right eyes, respectively, thereby to effect display control for these displaying portions 201L, 201R for left and right eyes” (Morita: 0047)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
Regarding claim 38, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the at least one image capturing device comprises a camera configured to operate in the visible spectrum to capture the images of the scenes viewed by the professional (The video camera 111 photo-takes a real space image in a view position and a view direction of the observer directed by the optical prism 115 (Morita: 0026)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
Regarding claim 40, the combined teachings teach the wearable augmented reality assembly of claim 32, further comprising one or more micro-projectors configured to display the images captured by the at least one image capturing device (“the actually taken images photo-taken by the video cameras 203L, 203R can be converted to forms which can be synthesized with the virtual space image in the processing apparatus 307. Further, the graphic boards 353, 352 for left and right eyes are connected to the displaying portions (display devices) 201L, 201R for left and right eyes, respectively, thereby to effect display control for these displaying portions 201L, 201R for left and right eyes” (Morita: 0047)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
Claims 24 and 42 are rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘748 Patent in view of Balan as applied to the claim(s) above, and further in view of Tobita (PGPUB Document No. US 2015/0235423).
Regarding claim 24, the combined teachings above do not expressly teach but Weller teaches the wearable augmented reality assembly of claim 22, wherein the processor is configured to analyze the images captured by the first camera and the second camera and to identify the marker elements on the one or more ROI acquisition markers to determine an orientation and a distance of the ROI from the wearable augmented reality assembly (“wherein the visual rendering unit is configured to adapt a size, orientation and position of the virtual information mapped on the unmanned moving object according to the detected motion, the spatial orientation and the distance of the augmented reality marker relative to the observer.” (Tobita: 0037)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the marker teaching of Tobita, because this enables an effective method of overlaying AR objects.
Claim 42 is similar in scope to claim 24. Therefore, the rejection to claim 24 similarly applies to claim 42.
Claims 25 is rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘748 Patent in view of Balan as applied to the claim(s) above, and further in view of Jagga et al. (PGPUB Document No. US 2017/00076501).
Regarding claim 25, the combined teachings above do not expressly teach but Jagga teaches the wearable augmented reality assembly of claim 22, wherein the processor is configured to analyze the images captured by the first camera and the second camera and to identify the one or more identifying elements coupled to the surgical device to track the surgical device (Jagga teaches an AR marker located on a medical instrument (Jagga: 0057)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to the marker as suggested by Jagga, because this enables an added level of accuracy when carrying out AR operations. Further, the application to the medical field enables extending the benefits of AR to other fields.
Claims 23, 25-27, 31, 32, 39 and 40 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 2, 4, 6 and 10 of U.S. Patent No. 11750794 in view of Bar-Zeev et al. (PGPUB Document No. US 2012/0068913) in view of Morita (PGPUB Document No. US 2003/0156144)
Although the claims at issue are not identical, they are not patentably distinct from each other because of the following.
The table below shows the correspondence between claim 22 of the ‘608 Application and claims 1, 2, 4 and 10 of the ‘794 patent.
‘608 Application
‘794 Patent
(Claim 22)
A wearable augmented reality assembly comprising:
a retaining structure in the form of spectacles or a head-mounted structure;
a liquid crystal display screen formed of an array of pixels;
a first camera mounted on the retaining structure and configured to capture images of scenes viewed by the right eye of the professional in the visible spectrum;
a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of elements of a scene in front of the wearable augmented reality assembly;
and a processor mounted within the retaining structure, wherein the elements comprise marker elements on one or more region of interest ("ROI") acquisition markers and one or more identifying elements coupled to a surgical device,
and wherein at least some pixels of the array of pixels are variably transparent, and wherein an opacity of the at least some of the pixels of the array of pixels is configured to be controlled by the processor.
(Claim 1)
A system for conducting an augmented reality assisted medical procedure on a patient, comprising:
and an augmented reality assembly comprising a retaining structure configured to be mounted on the medical professional,
(Claim 10)
adjusting an opacity of pixels of an array of pixels of the augmented reality assembly.
and a capturing device configured to be operated by the one or more processors so as to image the ROI;
(Claim 2)
the capturing device comprises an infra-red camera and wherein the one or more device identifying elements and the marker elements comprise infra-red reflective elements.
wherein: the capturing device is configured to image the one or more device identifying elements and the marker elements of the ROI marker, the one or more processors use the imaging of the identifying elements to track the surgical device,
(Claim 4)
wherein the one or more processors are configured to occlude at least a portion of the ROI by adjusting the opacity of selected pixels of the array, and to overlay an augmented reality portion of a scene viewed by the medical professional on the occluded portion.
However, the ‘608 application does not expressly teach but Bar-Zeev teaches
A first optical combiner coupled to the retaining structure and configured to be positioned in front of a right eye of a professional (optical component 722 (Bar-Zeev: 0086, FIG.7A, 7B));
A second optical combiner coupled to the retaining structure and configured to be positioned in front of a left eye of the professional (optical component 720 (Bar-Zeev: 0086, FIG.7A, 7B));;
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of the ‘608 application such as to utilize the optical components of Bar-Zeev, because this enables an effect see-through AR HMD device.
Further, the combined teachings above does not expressly teach but Morita teaches a second camera mounted on the retaining structure and configured to capture images of scenes viewed by the left eye of the professional in the visible spectrum (“The HMD photo-taking portion 203 has two photo-taking portions, i.e. a photo-taking portion 203R for right eye and a photo-taking portion 203L for left eye, and each photo-taking portion is constituted by a small-sized camera of NTSC type” (Morita: 0041, FIG.4)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
The table below shows the correspondence between claims 23, 25-27, 31, 32, 39 and 40 of the ‘608 application and the corresponding claims of the ‘794 patent.
‘608
23
25
26
27
31
32
39
40
‘794
12
1
2
2
1
1, 2, 4 ,10
2
1, 6
Claims 24 and 42 are rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘794 Patent in view of Balan as applied to the claim(s) above, and further in view of Tobita (PGPUB Document No. US 2015/0235423).
Regarding claim 24, the combined teachings above do not expressly teach but Weller teaches the wearable augmented reality assembly of claim 22, wherein the processor is configured to analyze the images captured by the first camera and the second camera and to identify the marker elements on the one or more ROI acquisition markers to determine an orientation and a distance of the ROI from the wearable augmented reality assembly (“wherein the visual rendering unit is configured to adapt a size, orientation and position of the virtual information mapped on the unmanned moving object according to the detected motion, the spatial orientation and the distance of the augmented reality marker relative to the observer.” (Tobita: 0037)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the marker teaching of Tobita, because this enables an effective method of overlaying AR objects.
Claim 42 is similar in scope to claim 24. Therefore, the rejection to claim 24 similarly applies to claim 42.
Claims 28, 29, 30, 33-37 and 41 are rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘748 Patent in view of Balan as applied to the claim(s) above, and further in view of Bar-Zeev et al. (PGPUB Document No. US 2012/0068913).
Regarding claim 28, the combined teachings teach the wearable augmented reality assembly of claim 22, further comprising a second sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of accurately overlaying AR information.
Regarding claim 29, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the retaining structure is in the form of spectacles (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because AR spectacles enable an improved AR experience to the user.
Regarding claim 30, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the retaining structure is in the form of a head-mounted structure (“FIG. 1 depicts an example embodiment of an optical see-through HMD device with an augmented reality capability” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because an AR HMD device enables an improved AR experience to the user.
Regarding claim 33, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen is coated on at least one optical combiner (Bar-Zeev teaches the opacity filter can be provided in or on another light-transmissive lens material, which corresponds to being “coated on” as presently recited (Bar-Zeev: 0083). Further, Bar-Zeev teaches the opacity filter can be placed physically behind an optical display component which introduces the augmented reality image to the user's eye (Balan: 0042)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 34, the combined teachings teach the wearable augmented reality assembly of claim 33, wherein the at least one optical combiner comprises a pair of see through glasses, with an optical combiner adapted to be positioned in front of each eye of the professional (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because the see-through lens enables an improved AR experience to the user.
Regarding claim 35, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen comprises a liquid crystal display (light-transmissive opacity filter 723 and 721 can be a see-through LCD panel (Bar-Zeev: 0083, 0085, 0053, 0050, FIG.7C)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 36, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen comprises a pixelated screen comprised of an array of pixels (“The opacity filter can include a dense grid of pixels” (Bar-Zeev: 0051)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing an AR screen.
Regarding claim 37, the combined teachings teach the wearable augmented reality assembly of claim 36, wherein the processor is configured to control an opacity of the array of pixels such that at least the section of the display screen configured to be rendered at least partially opaque is fully occluded (“opacity filter driver can drive pixels in the opacity filter 106 such as by addressing each pixel by a row and column address and a voltage which indicates a desired degree of opacity, from a minimum level which is most light-transmissive level to a maximum level which is most opaque or least light-transmissive” (Bar-Zeev: 0063)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 41, the combined teachings teach the wearable augmented reality assembly of claim 32, further comprising a sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of accurately overlaying AR information.
Claims 22, 28, 29, 33-35, 38 and 41 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-3, 5 and 6 of U.S. Patent No. 12206837 in view of Bar-Zeev in view of Morita.
Although the claims at issue are not identical, they are not patentably distinct from each other because of the following.
The table below shows the correspondence between claim 22 of the ‘608 Application and claims 1, 2, 3, 5 and 6 of the ‘837 patent.
‘608 Application
‘837 Patent
(Claim 22)
A wearable augmented reality assembly comprising:
a retaining structure in the form of spectacles or a head-mounted structure;
a liquid crystal display screen formed of an array of pixels;
a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of elements of a scene in front of the wearable augmented reality assembly;
and a processor mounted within the retaining structure, wherein the elements comprise marker elements on one or more region of interest ("ROI") acquisition markers and one or more identifying elements coupled to a surgical device,
and wherein at least some pixels of the array of pixels are variably transparent, and wherein an opacity of the at least some of the pixels of the array of pixels is configured to be controlled by the processor.
(Claim 1)
A system for conducting an augmented reality assisted medical procedure on a patient, comprising:
and an augmented reality assembly comprising a head-up display configured to be mounted on the medical professional
(Claim 6)
an array of pixels, an opacity of each of the pixels being adjustable by the system processor.
(Claims 2, 5)
wherein the capturing device comprises an infra-red camera.
(Claim 3)
wherein the one or more device identifying elements and the ROI marker comprise infra-red reflective elements.
(Claim 1)
a surgical device for use by a medical professional in the procedure, the device comprising one or more device identifying elements; a region of interest (ROI) marker positioned in proximity to an ROI;
(Claim 6)
comprising an array of pixels, an opacity of each of the pixels being adjustable by the system processor.
However, the ‘837 does not expressly teach but Bar-Zeev teaches,
A first optical combiner coupled to the retaining structure and configured to be positioned in front of a right eye of a professional (optical component 722 (Bar-Zeev: 0086, FIG.7A, 7B));
A second optical combiner coupled to the retaining structure and configured to be positioned in front of a left eye of the professional (optical component 720 (Bar-Zeev: 0086, FIG.7A, 7B)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the teachings of Bar-Zeev, because this enables an effect see-through AR HMD device.
However, the combined teachings above do not expressly teach but Morita teaches,
A first camera mounted on the retaining structure and configured to capture images of scenes viewed by the right eye of the professional in the visible spectrum; a second camera mounted on the retaining structure and configured to capture images of scenes viewed by the left eye of the professional in the visible spectrum (“The HMD photo-taking portion 203 has two photo-taking portions, i.e. a photo-taking portion 203R for right eye and a photo-taking portion 203L for left eye, and each photo-taking portion is constituted by a small-sized camera of NTSC type” (Morita: 0041, FIG.4));;
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
The table below shows the correspondence between claims 23, 25-27, 30-32, 36, 37 and 40 of the ‘608 application and the corresponding claims of the ‘837 patent.
‘608
23
25
26
27
30
31
32
36
37
40
‘837
2
1
3
3
1
1
1, 2, 4 ,10
6
7
1
Regarding claim 28, the combined teachings teach the wearable augmented reality assembly of claim 22, further comprising a second sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of accurately overlaying AR information.
Regarding claim 29, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the retaining structure is in the form of spectacles (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because AR spectacles enable an improved AR experience to the user.
Regarding claim 33, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen is coated on at least one optical combiner (Bar-Zeev teaches the opacity filter can be provided in or on another light-transmissive lens material, which corresponds to being “coated on” as presently recited (Bar-Zeev: 0083). Further, Bar-Zeev teaches the opacity filter can be placed physically behind an optical display component which introduces the augmented reality image to the user's eye (Balan: 0042)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 34, the combined teachings teach the wearable augmented reality assembly of claim 33, wherein the at least one optical combiner comprises a pair of see through glasses, with an optical combiner adapted to be positioned in front of each eye of the professional (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because the see-through lens enables an improved AR experience to the user.
Regarding claim 35, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen comprises a liquid crystal display (light-transmissive opacity filter 723 and 721 can be a see-through LCD panel (Bar-Zeev: 0083, 0085, 0053, 0050, FIG.7C)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of implementing a variably transparent screen.
Regarding claim 38, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the at least one image capturing device comprises a camera configured to operate in the visible spectrum to capture the images of the scenes viewed by the professional (The video camera 111 photo-takes a real space image in a view position and a view direction of the observer directed by the optical prism 115 (Morita: 0026)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
Regarding claim 41, the combined teachings teach the wearable augmented reality assembly of claim 32, further comprising a sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to apply the teachings of Bar-Zeev, because this enables an effective method of accurately overlaying AR information.
Claims 24 and 42 are rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘837 Patent in view of Bar-Zeev in view of Morita as applied to the claim(s) above, and further in view of Tobita (PGPUB Document No. US 2015/0235423).
Regarding claim 24, the combined teachings above do not expressly teach but Weller teaches the wearable augmented reality assembly of claim 22, wherein the processor is configured to analyze the images captured by the first camera and the second camera and to identify the marker elements on the one or more ROI acquisition markers to determine an orientation and a distance of the ROI from the wearable augmented reality assembly (“wherein the visual rendering unit is configured to adapt a size, orientation and position of the virtual information mapped on the unmanned moving object according to the detected motion, the spatial orientation and the distance of the augmented reality marker relative to the observer.” (Tobita: 0037)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the marker teaching of Tobita, because this enables an effective method of overlaying AR objects.
Claim 42 is similar in scope to claim 24. Therefore, the rejection to claim 24 similarly applies to claim 42.
Claims 24 and 42 are rejected on the ground of nonstatutory double patenting as being unpatentable over the ‘837 Patent in view of Bar-Zeev in view of Morita as applied to the claim(s) above, and further in view of Balan.
Regarding claim 39, the combined teachings teach the wearable augmented reality assembly of claim 38, further comprising a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of the one or more marker elements in front of the wearable augmented reality assembly (“The image camera component 32 may include an IR light component 34…the IR light component 34 of the capture device 58 may emit an infrared light onto the capture area” (Balan: 0040)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the IR markers of Balan, because this enables an added variety of ways of implementing AR experience to the user.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 22, 23, 26 and 27-41 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bar-Zeev et al. (PGPUB Document No. US 2012/0068913) in view of Morita (PGPUB Document No. US 2003/0156144) in view of Balan et al. (PGPUB Document No. US 2012/0306850).
Regarding claim 22, Bar-Zeev teaches a wearable augmented reality assembly comprising:
A retaining structure in the form of spectacles or a head-mounted structure (frame of FIG.7A (Bar-Zeev: 0083));
A first optical combiner coupled to the retaining structure and configured to be positioned in front of a right eye of a professional (optical component 722 (Bar-Zeev: 0086, FIG.7A, 7B));
A second optical combiner coupled to the retaining structure and configured to be positioned in front of a left eye of the professional (optical component 720 (Bar-Zeev: 0086, FIG.7A, 7B));
A liquid crystal display screen formed of an array of pixels (light-transmissive opacity filter 723 and 721 can be a see-through LCD panel (Bar-Zeev: 0083, 0085, 0053, 0050, FIG.7C));
And wherein at least some pixels of the array of pixels are variably transparent (the opacity filter provides a region 402 of increased opacity whereas other regions pass through images of the real-world scene (Bar-Zeev: 0077-78));
And wherein an opacity of the at least some of the pixels of the array of pixels is configured to be controlled by the processor (“opacity filter driver can drive pixels in the opacity filter 106 such as by addressing each pixel by a row and column address and a voltage which indicates a desired degree of opacity, from a minimum level which is most light-transmissive level to a maximum level which is most opaque or least light-transmissive” (Bar-Zeev: 0063)).
However, Bar-Zeev does not expressly teach but Morita teaches, in a similar field of endeavor,
A first camera mounted on the retaining structure and configured to capture images of scenes viewed by the right eye of the professional in the visible spectrum; a second camera mounted on the retaining structure and configured to capture images of scenes viewed by the left eye of the professional in the visible spectrum (“The HMD photo-taking portion 203 has two photo-taking portions, i.e. a photo-taking portion 203R for right eye and a photo-taking portion 203L for left eye, and each photo-taking portion is constituted by a small-sized camera of NTSC type” (Morita: 0041, FIG.4));;
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of Bar-Zeev such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
However, the combined teachings above do not expressly teach but Balan teaches a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of elements of a scene in front of the wearable augmented reality assembly (capture 58 emitting IR light (Balan: 0040) to recognize IR markers within the real-world scene (Balan: 0049));
And a processor mounted within the retaining structure, wherein the elements comprise marker elements on one or more region of interest ("ROI") acquisition markers and one or more identifying elements coupled to a surgical device (“one or more objects being tracked may be augmented with one or more markers such as an IR retroreflective marker” (Balan: 0049)),
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the IR markers of Balan, because this enables an added variety of ways of implementing AR experience to the user.
Regarding claim 23, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the sensor comprises a radiator configured to project the radiation in the near infra-red spectrum (“the IR light component 34 of the capture device 58 may emit an infrared light onto the capture area” (Balan: 0040)).
Regarding claim 26, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the marker elements on the one or more ROI acquisition markers comprise retro-reflectors that reflect the radiation projected by the sensor (Balan teaches an image camera component 32 that includes an IR light component 34 (Balan: 0040) and wherein the one or more device identifying elements and the ROI marker comprises infrared reflective elements (Balan: 0049)).
Regarding claim 27, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the one or more identifying elements comprise reflectors and/or radiators (obtain location and orientation information by detecting, tracking, and triangulating the position of physical tags (e.g., reflective markers) or emitters (e.g., LEDs) (Balan: 0064, 0049)).
Regarding claim 28, the combined teachings teach the wearable augmented reality assembly of claim 22, further comprising a second sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Regarding claim 29, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the retaining structure is in the form of spectacles (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Regarding claim 30, the combined teachings teach the wearable augmented reality assembly of claim 22, wherein the retaining structure is in the form of a head-mounted structure (“FIG. 1 depicts an example embodiment of an optical see-through HMD device with an augmented reality capability” (Bar-Zeev: 0043)).
Regarding claim 31, the combined teachings teach the wearable augmented reality assembly of claim 22, further comprising one or more micro-projectors configured to display images captured by the first camera and the second camera (“the actually taken images photo-taken by the video cameras 203L, 203R can be converted to forms which can be synthesized with the virtual space image in the processing apparatus 307. Further, the graphic boards 353, 352 for left and right eyes are connected to the displaying portions (display devices) 201L, 201R for left and right eyes, respectively, thereby to effect display control for these displaying portions 201L, 201R for left and right eyes” (Morita: 0047)).
Regarding claim 32, Bar-Zeev teaches a wearable augmented reality assembly comprising:
A retaining structure adapted to be worn on a head of a professional (frame of FIG.7A (Bar-Zeev: 0083));
A display screen extending from the retaining structure and adapted to be positioned in front of one eye or both eyes of the professional (light-transmissive opacity filter 723 and 721 can be a see-through LCD panel (Bar-Zeev: 0083, 0085, 0053, 0050, FIG.7C)),
Wherein the display screen comprises a section configured to be rendered at least partially opaque while a remaining portion of the display screen is transparent (the opacity filter provides a region 402 of increased opacity whereas other regions pass through images of the real-world scene (Bar-Zeev: 0077-78));
And a processor mounted within the retaining structure (opacity filter control circuit 100 can be a micro-processor (Bar-Zeev: 0054, FIG.1)),
Wherein the processor is configured to control the opacity of the section of the display screen configured to be rendered at least partially opaque (“opacity filter driver can drive pixels in the opacity filter 106 such as by addressing each pixel by a row and column address and a voltage which indicates a desired degree of opacity, from a minimum level which is most light-transmissive level to a maximum level which is most opaque or least light-transmissive” (Bar-Zeev: 0063)),
However, Bar-Zeev does not expressly teach but Morita teaches, in a similar field of endeavor,
At least one image capturing device mounted on the retaining structure configured to capture images of scenes viewed by the professional (“The HMD photo-taking portion 203 has two photo-taking portions, i.e. a photo-taking portion 203R for right eye and a photo-taking portion 203L for left eye, and each photo-taking portion is constituted by a small-sized camera of NTSC type” (Morita: 0041, FIG.4));
Wherein the at least one image capturing device comprises at least one camera configured to operate in the visible spectrum to capture the images of the scenes viewed by the one eye or both eyes of the professional (each photo-taking portion is constituted by a small-sized camera of NTSC type (Morita: 0041, FIG.4));
And wherein the processor is configured to display images of scenes captured by the at least one image capturing device on the section of the display screen configured to be rendered at least partially opaque (the photo-taken real space image is overlapped with the virtual space image to generate the compound real space image.” (Morita: 0041, FIG.4). Applying the teaching of Morita to Bar-Zeev enables an HMD system where the real-world scene is displayed to the user based on images captured by the photo-taking portions 203R, 203L of Morita, and the AR image is displayed at a region of the LCD display with increased opacity as stated in the rejection above (Bar-Zeev: 0077)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the teachings of Bar-Zeev such as to utilize the HMD camera teaching of Morita, because this enables an effective method of overlaying images within an AR system.
However, the combined teachings above do not expressly teach but Balan teaches
A sensor configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of one or more marker elements in front of the wearable augmented reality assembly (capture 58 emitting IR light (Balan: 0040) to recognize IR markers within the real-world scene (Balan: 0049)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the IR markers of Balan, because this enables an added variety of ways of implementing AR experience to the user.
Regarding claim 33, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen is coated on at least one optical combiner (Bar-Zeev teaches the opacity filter can be provided in or on another light-transmissive lens material, which corresponds to being “coated on” as presently recited (Bar-Zeev: 0083). Further, Bar-Zeev teaches the opacity filter can be placed physically behind an optical display component which introduces the augmented reality image to the user's eye (Balan: 0042)).
Regarding claim 34, the combined teachings teach the wearable augmented reality assembly of claim 33, wherein the at least one optical combiner comprises a pair of see through glasses, with an optical combiner adapted to be positioned in front of each eye of the professional (“The display device can include a see-through lens 108 which is placed in front of a user's eye, similar to an eyeglass lens” (Bar-Zeev: 0043)).
Regarding claim 35, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen comprises a liquid crystal display (light-transmissive opacity filter 723 and 721 can be a see-through LCD panel (Bar-Zeev: 0083, 0085, 0053, 0050, FIG.7C)).
Regarding claim 36, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the display screen comprises a pixelated screen comprised of an array of pixels (“The opacity filter can include a dense grid of pixels” (Bar-Zeev: 0051)).
Regarding claim 37, the combined teachings teach the wearable augmented reality assembly of claim 36, wherein the processor is configured to control an opacity of the array of pixels such that at least the section of the display screen configured to be rendered at least partially opaque is fully occluded (“opacity filter driver can drive pixels in the opacity filter 106 such as by addressing each pixel by a row and column address and a voltage which indicates a desired degree of opacity, from a minimum level which is most light-transmissive level to a maximum level which is most opaque or least light-transmissive” (Bar-Zeev: 0063)).
Regarding claim 38, the combined teachings teach the wearable augmented reality assembly of claim 32, wherein the at least one image capturing device comprises a camera configured to operate in the visible spectrum to capture the images of the scenes viewed by the professional (The video camera 111 photo-takes a real space image in a view position and a view direction of the observer directed by the optical prism 115 (Morita: 0026)).
Regarding claim 39, the combined teachings teach the wearable augmented reality assembly of claim 38, further comprising a sensor mounted on the retaining structure and configured to operate in the near infra-red spectrum, the sensor configured to project radiation in the near infra-red spectrum and to capture non-visible images of the one or more marker elements in front of the wearable augmented reality assembly (“The image camera component 32 may include an IR light component 34…the IR light component 34 of the capture device 58 may emit an infrared light onto the capture area” (Balan: 0040)).
Regarding claim 40, the combined teachings teach the wearable augmented reality assembly of claim 32, further comprising one or more micro-projectors configured to display the images captured by the at least one image capturing device (“the actually taken images photo-taken by the video cameras 203L, 203R can be converted to forms which can be synthesized with the virtual space image in the processing apparatus 307. Further, the graphic boards 353, 352 for left and right eyes are connected to the displaying portions (display devices) 201L, 201R for left and right eyes, respectively, thereby to effect display control for these displaying portions 201L, 201R for left and right eyes” (Morita: 0047)).
Regarding claim 41, the combined teachings teach the wearable augmented reality assembly of claim 32, further comprising a sensor configured to measure an inclination of the wearable augmented reality assembly with respect to a direction of gravity (“determined by tracking a position of the user's head using a combination of motion tracking techniques and an inertial measure unit which is attached to the user's head” (Bar-Zeev: 0046)).
Claim(s) 24 and 42 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bar-Zeev in view of Morita in view of Balan as applied to the claim(s) above, and further in view of Tobita (PGPUB Document No. US 2015/0235423).
Regarding claim 24, the combined teachings above do not expressly teach but Weller teaches the wearable augmented reality assembly of claim 22, wherein the processor is configured to analyze the images captured by the first camera and the second camera and to identify the marker elements on the one or more ROI acquisition markers to determine an orientation and a distance of the ROI from the wearable augmented reality assembly (“wherein the visual rendering unit is configured to adapt a size, orientation and position of the virtual information mapped on the unmanned moving object according to the detected motion, the spatial orientation and the distance of the augmented reality marker relative to the observer.” (Tobita: 0037)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to utilize the marker teaching of Tobita, because this enables an effective method of overlaying AR objects.
Claim 42 is similar in scope to claim 24. Therefore, the rejection to claim 24 similarly applies to claim 42.
Claim(s) 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bar-Zeev in view of Morita in view of Balan as applied to the claim(s) above, and further in view of Jagga et al. (PGPUB Document No. US 2017/00076501).
Regarding claim 25, the combined teachings above do not expressly teach but Jagga teaches the wearable augmented reality assembly of claim 22, wherein the processor is configured to analyze the images captured by the first camera and the second camera and to identify the one or more identifying elements coupled to the surgical device to track the surgical device (Jagga teaches an AR marker located on a medical instrument (Jagga: 0057)).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of an ordinary skill in the art to modify the combined teachings above such as to the marker as suggested by Jagga, because this enables an added level of accuracy when carrying out AR operations. Further, the application to the medical field enables extending the benefits of AR to other fields.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to David H Chu whose telephone number is (571)272-8079. The examiner can normally be reached M-F: 9:30 - 1:30pm, 3:30-8:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H CHU/Primary Examiner, Art Unit 2616
Prosecution Insights