DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1-19 have been considered but are moot because the new ground of rejections.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5, 9, 13-14, 16-17, 19 rejected under 35 U.S.C. 103 as being unpatentable over Makino [US 20100057059 A1] in view of Florea [US 20130329079 A1].
As per claim 1, Makino teaches a method for enhancing a digital image of a patient’s eye during an ophthalmic procedure, comprising:
illuminating the patient’s eye with light from a modulable lighting source (Makino Fig 1 ¶0017, lighting units 18, 19);
collecting Makino Fig 1 ¶0017, image pick-up camera 24);
receiving input signals during the ophthalmic procedure, the input signals including a request to enhance an area-of-focus of the Makino ¶0047 “he surgeon (or an assistant) manipulates a light-amount adjustment switch mounted on the controller 31 to increase the illumination light amount” This switching is a request)
in response to the input signals, selectively adjusting respective characteristics of the modulable lighting source and constituent pixels of the Makino ¶0024, ¶0038, automated contrast adjustment is performed by increasing or decreasing the light amount of the illumination light source 18. Adjustments of luminance difference VS and VB implies adjustment of intensity (characteristics) of pixels) of iris and pupil, areas of illumination in image); and
transmitting display control signals to one or more display screens to thereby present an enhanced Makino Fig 1, item 32b, Fig 3).
The only differences from claim is that Makino does not expressly teach collecting digital images, and digitally adjusting characteristics only of constituent pixels of the digital images.
Florea, in a field of image processing, particularly of detected eye regions, teaches collecting digital images and digitally adjusting characteristics only of constituent pixels (Florea ¶0035 “capturing digital images, ”¶0033 “One or more groups of pixels are identified that each include a pupil region of an eye region within the face in the digital image. A border between a pupil region and an iris or a sclera is identified using luminance information (and/or optionally chrominance information, and/or size and/or shape information). The digital image is adjusted by adding one or more glint pixels at a pupil side of the border between the iris and the pupil to generate an enhanced image,”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the method in Makino, by digital processing as in Florea. As per MPEP 2143. example of rationales that may support a conclusion of obviousness include (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results. In this case, digital enhancement of individual pixels in or outside an area of focus (iris or pupil for example), is known as evidenced by Florea. The methods disclosed in Florea provides for improving image characteristics like smoothness, blur, brightness, resolution etc. Hence the improvement to images produced in Makino, by applying known digital techniques would have been predictable.
As per claim 2, Makino in view of Florea further teaches wherein the modulable lighting source is connected to or integral with an ophthalmic microscope, and wherein illuminating the patient’s eye with light from the modulable lighting source includes illuminating the patient’s eye with white light (Makino ¶0018 “The visible-light illumination unit 19 arranged to project a visible light beam to the patient's eye is connected to the control unit”).
As per claim 5, Makino in view of Florea further teaches further comprising: identifying a stage of the ophthalmic procedure via the ECU as an identified stage (Makino ¶0021 “The image processing unit 25 determines whether or not a contrast of a pupil portion and a contrast of the iris pattern are appropriate.” This is an identified stage); and autonomously generating the input signals via the ECU during the ophthalmic procedure based on the identified stage (Makino ¶0021 “Based on the determination result by the image processing unit 25, the control unit 30 adjusts a set value of the light amount of the illumination light source 18”).
As per claim 9, Makino in view of Florea further teaches wherein selectively adjusting the respective characteristics of the modulable lighting source and digitally adjusting characteristics the constituent pixels of the digital image located within the area-of-focus includes: increasing a red light component of the modulable lighting source (Florea ¶0093 “redness may be increased. In YUV color space this may be done by multiplying the V value inside the mouth area by a predefined factor”); and/or
digitally decreasing a brightness and/or color temperature of the constituent pixels of the digital image located outside of the area-of-focus (Makino ¶0025 “in a case where the amount of reflected light from the anterior segment is gradually decreased” , ¶0027 “illumination light source 18 is decreasingly changed from an initial value continuously or step-by-step” ).
As per claims 13-14, 16-17 have limitations similar to claims 1-2, 5, 9 and are rejected for same reasons as above.
Makino in view of Florea further teaches a system comprising a modulable lighting source (Makino Fig 1 items 18, 19), a digital camera operable for collecting the digital image (Garber ¶0038, Florea ¶0065 “a digital camera or real-time imaging appliance” ) and an electronic control unit (ECU) (Makino Fig 1 item 30).
As per claim 19, it has limitations similar to claim 15 and is rejected for same reasons as above. Claim 19 further require a “a computer-readable storage medium on which is recorded instructions”. Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to embody method of Makino in view of Florea in a “a computer-readable storage medium on which is recorded instructions” so that the instructions can be automatically executed.
Claims 3-4, 15 rejected under 35 U.S.C. 103 as being unpatentable over Makino in view of Florea as applied to claims 1, 13 above, and further in view of Kaul [US 20190302460 A1].
As per claims 3-4, Makino in view of Florea does not expressly teach wherein receiving input signals includes receiving voice commands from a surgeon while the surgeon performs ophthalmic procedure, wherein the voice commands include an utterance or statement of a desired region of the patient’s eye, the desired region including a pupil, an iris, a sclera, or a limbus region.
Kaul, in a related field of biomedical applications, teaches wherein receiving input signals includes receiving voice commands from a surgeon while the surgeon performs ophthalmic procedure, wherein the voice commands include an utterance or statement of a desired region of the patient’s eye, the desired region including a pupil, an iris, a sclera, or a limbus region (Kaul ¶0101 “adjusted by the physician by various commands, such as voice command, haptic input, and the like. For example, the physician can look at the patient's eyes, simultaneously look at a computer generated representation of the iris and pupal in the augmented reality display, and issue various commands such as “iris increase, pupil decrease”…”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the method in Makino in view of Florea, by integration of use of voice commands so that semi-automated but computer enhanced assessment of the patient's pupils (or iris) can automatically be brought up (Kaul ¶0086, ¶0101).
As per claim 15, it has limitations similar to claim 4 and is rejected for same reasons as above. Makino in view of Florea and Kaul further teaches receive voice commands via the microphone (Kaul ¶0050).
Claim 6 rejected under 35 U.S.C. 103 as being unpatentable over Makino in view of Florea as applied to claim 1 above, and further in view of Hallen [US 20190099226 A1].
As per claim 6, Makino in view of Florea does not expressly teach wherein collecting the digital image of the patient’s eye is performed using a high-dynamic range (HDR) digital camera.
Hallen, in a related field of ophthalmic surgery teaches wherein collecting the digital image of the patient’s eye is performed using a high-dynamic range (HDR) digital camera (Hallen ¶0035 “an HDR camera used to resolve high quality, three-dimensional views of an eye”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify method in Makino in view of Florea, by using HDR for resolving high quality images of the eye (Hallen ¶0035).
Claims 7-8 rejected under 35 U.S.C. 103 as being unpatentable over Makino in view of Florea as applied to claim 1 above, and further in view of Garber [US 20150324974 A1]
As per claim 7, Makino in view of Florea does not expressly teach further teaches wherein identifying the area-of-focus via the AI logic in response to input signals, wherein identifying the area-of-focus via the AI logic includes processing the digital image via a neural network and/or a trained model (Garber ¶0043 “machine learning techniques, … to segment areas of interest and anatomical zones in the images according to the one or more iridology maps, … segmentation of the area of interest: pupil, iris and sclera of the eye”. ¶0041 “the professionals identify the markings by eyeballing, and their identification is used as training input into a machine learning algorithm.”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the method in Makino in view of Florea, by utilizing machine learning techniques as in Garber. The motivation for artificial intelligence is that is that different regions of the eye like pupil and iris, characterize by attributes such as color, shape and size can be extracted and displayed diagnostics and/or recommendations.
Claim 10 rejected under 35 U.S.C. 103 as being unpatentable over Makino in view of Florea as applied to claim 1 above, and further in view of Hewlett [US 20230152567 A1].
As per claim 10, Makino in view of Florea does not expressly teach wherein the modulable lighting source includes a coaxial lighting source and an oblique lighting source, and wherein selectively adjusting the respective characteristics of the modulable lighting source and the constituent pixels of the digital image located within the area-of-focus includes blending lighting from the coaxial lighting source and the oblique lighting source to optimize a red reflex of the patient’s eye.
Hewlett, in a related field of surgical microscopes, teaches wherein the modulable lighting source includes a coaxial lighting source and an oblique lighting source, and wherein selectively adjusting includes blending lighting from the coaxial lighting source and the oblique lighting source to optimize a red reflex of the patient’s eye (Hewlett ¶0087 “adjust the color temperature and/or spectral makeup of the total light generated by the coaxial light source 321 and the pair of oblique light sources 323 to generate a desired spectrum of incident light in order to obtain a level of desired red reflex from the light illumination”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the method in Makino in view of Florea, by utilizing illumination as in Hewlett so as to clear and semi-clear tissues in the eye (Hewlett ¶0086).
Claims 11, 18 rejected under 35 U.S.C. 103 as being unpatentable over Makino in view of Florea as applied to claims 1, 13 above, and further in view of Bednarik [US 20200257113 A1].
As per claims 11, 18, Makino in view of Florea does not expressly teach detecting a pigmentation color of an iris of the patient’s eye, wherein selectively adjusting the respective characteristics of the modulable lighting source and the constituent pixels of the digital image located within the area-of-focus includes adjusting spectral characteristics of the modulable lighting source based on the pigmentation color.
Bednarik, in a field of eye-tracking and gaze detection, teaches detecting a pigmentation color of an iris of the patient’s eye, wherein selectively adjusting includes adjusting spectral characteristics of the modulable lighting source based on the pigmentation color (Bednarik ¶0261 “dynamic light beam is dependent on each eye optical property …user may just provide selection information for the type of the eye e.g., eye pigmentation and optimized spectral power distribution (SPD) data is loaded”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the method in Makino in view of Florea, by utilizing dynamic light beam so as to optimized spectral power distribution (Bednarik ¶0261).
Claim 12 rejected under 35 U.S.C. 103 as being unpatentable over Makino in view of Florea as applied to claim 1 above, and further in view of Watanabe [US 20090247997 A1].
As per claim 12, Makino in view of Florea does not expressly teach wherein selectively adjusting the respective characteristics of the modulable lighting source and the constituent pixels of the digital image located within the area-of-focus includes adjusting image gamma of the digital image via the ECU to reduce intensity of specular reflectance in the area-of-focus.
Watanabe, in a related field of ophthalmic surgery, teaches adjusting the respective characteristics of the modulable lighting source and the constituent pixels of the digital image located within the area-of-focus includes adjusting image gamma of the digital image via the ECU to reduce intensity of specular reflectance in the area-of-focus (Watanabe ¶0048 “…desire a greater contrast between the pupil and the iris… image sensor 57 has a gamma function, which controls the intensity or brightness of each pixel of the digital image displayed by the imaging interface 59 based on the image detected by the image sensor 57. The typical default is a linear gamma function”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the method in Makino in view of Florea, by integrating gamma function processing so as to provide contrast on a pixel basis (Watanabe ¶0048).
Allowable Subject Matter
Claim 20, not addressed in the art rejection above, is considered free of art. Examiner does not find any references anticipating the claim nor find it obvious to combine references of record to show all limitations as recited, including the four steps for image enhancement.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OOMMEN JACOB whose telephone number is (571)270-5166. The examiner can normally be reached 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANNE M KOZAK can be reached at 571-270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Oommen Jacob/Primary Examiner, Art Unit 3797
Prosecution Insights