The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Response to Arguments
Applicant’s arguments with respect to the claims have been considered but are moot in view of the new grounds of rejection, which are necessitated by the amendments to the claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7-10 and 13-19 are rejected under 35 U.S.C. 103 as being unpatentable over Camus et al. (US Patent No. 6,088,470) in view of Zickler (US Patent Pub. No. 2009/0275929) and Allred et al. (US Patent Pub. No. 2010/0110380).
Regarding claim 1, Camus discloses a “method and apparatus for illuminating and imaging eyes uses multiple light sources producing multiple images of a subject each created under illumination by different illuminators” (see Abstract). Figure 7 illustrates the method of Camus, which includes:
capturing, using a … camera, a first image of the eye while the eye is illuminated with a first light source (see box 32);
within a subsequent time period from the capturing of the first image, capturing a second image of the eye while the eye is illuminated with a second light source, different from the first light source (see box 34);
processing the first and second images into a third image to remove specular reflections associated with the first light source and the second light source (see boxes 36, 38, 40 and 42).
With regard to these steps above, it is noted that “FIG. 3 shows a left-illuminated eye image, a right-illuminated image of the same eye, and a third image formed by fusion of the other two images” (see column 4, lines 18-20). Additionally, Figure 1 illustrates the structural configuration of a system used for this method (see column 4, lines 37-57).
However, it is noted that Camus does not explicitly state that the camera is a “torsion camera” or that this method is used in a surgical procedure. Additionally, the specific processing steps of “processing the first image…”, “processing the second image…”, and “combining…” are not taught by Camus, since Camus teaches a processing scheme in which the combining of the two images into the third image comprises choosing the lower pixel value of the first image and the second image to create the corresponding pixel of the third image.
Regarding a “torsion camera” or the method being used in a surgical procedure, Zickler teaches systems and methods for controlling measurement in an eye during ophthalmic procedure(s) (see Title). In paragraph 10, Zickler teaches that “A variety of eye tracking and beam scanning techniques have been proposed to align the customized laser ablation pattern with the eye. Some of these techniques rely upon imaging the eye. Specular reflections from natural (e.g., tear film, epithelium, stroma, sclera, and the like) or artificially created (e.g., from a corneal flap incision, surgical instrument, intraocular lens, implant, and the like) refractive index discontinuities of the human eye may decrease image detail, which would interfere with position detection and motion compensation.” Zickler provides for a manner to reduce or eliminate specular reflection, but does so utilizing polarization of light (see Figures 12-14). Paragraph 108 teaches that “The image [shown in Figure 12] is captured by an image capture device, such as a camera, eye tracker, or torsional tracker.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the techniques of Camus within an ophthalmic surgical setting to remove specular reflection within images of the eye and to acquire those images with a camera, which may include a torsion camera, as explicitly taught by Zickler, because both Camus and Zickler are directed to ocular imaging with the intention of removing specular reflections within the images that degrade the ability to track the eye, and the use of either one to perform the method taught by of specular reflection removal within a surgical setting via a torsion camera versus any other camera with comparable frame rates would amount to choosing from a finite number of imaging devices and techniques available in the art at the time of the invention, which has previously been held as unpatentable (KSR v. Teleflex).
Regarding the processing steps of “processing the first image…”, “processing the second image…”, and “combining…”, Allred teaches the following in paragraphs 43-44:
An example of a comparison and combination technique is given as follows. First, corresponding pixels are identified (step 670) (i.e., pixels having a coordinate (x,y) in a same perspective are identified). Intensity values of the corresponding pixels are compared (step 680). For example, if the intensity of a given pixel in the first image I(x,y) is greater than the intensity of a corresponding pixel in the second image I'(x,y) by greater than a factor T then I(x,y) is assumed to be the result of specular reflection (690) and the intensity of the corresponding pixel in the third image R(x,y) is selected to be equal to I'(x,y) (step 699). Similarly, if the intensity of a given pixel in the second image I'(x,y) is greater than the intensity of a corresponding pixel in the first image I(x,y) by greater than a factor T, then I'(x,y) is assumed to be the result of specular reflection (690) and the intensity of the corresponding pixel in the third image R(x,y) is selected to be equal to I(x,y) (step 699). Factor T can be chosen to have any suitable value for example T ≥ 1.25 or T ≥ 2.0.
If neither the intensity of the pixel in the first image nor the intensity of the pixel in the second image is greater by a factor T (as determined in step 695), a selection of an intensity value R(x,y) for the corresponding pixel is performed. The corresponding pixel R(x,y) may be chosen using any of a number of different techniques (step 699). For example, the intensity R(x,y) can be chosen to be equal to the lesser of I(x,y) and I'(x,y). In some embodiments, the intensity is chosen to be the average of I(x,y) and I'(x,y). However, the intensity may be chosen to be the greater intensity of I(x,y) and I'(x,y) or any other suitable combination of intensity values I(x,y) and I'(x,y).
In the first paragraph above (i.e., paragraph 43), the method is similar to that taught by Camus. That is, when the first image is determined to have a specular reflection, the pixel from the second image is used to create a pixel for the third image; when the second image is determined to have a specular reflection, the pixel from the first image is used to create a pixel for the third image. However, the second paragraph above (i.e., paragraph 44) teaches that when it is determined that neither image contains a specular reflection, then “In some embodiments, the intensity is chosen to be the average of” the pixel of the first image and the second image. In other words, this teaches that for regions in which neither the first image nor the second image contain a specular reflection (i.e., the portions of these images that equate to the claimed first and second regions of interest), the pixel of the third image will contain an average of the pixel value of the first and second image, thereby reading on “the third image contains information from both the first and second … images”, as claimed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize an averaging technique, as taught by Allred, within the system and methods of Camus, because utilizing an averaging technique as taught by Allred will ensure that the third, combined image is more contrasted than the third image of Camus. By utilizing the lower greyscale pixels in situations when neither image contains a specular reflection, Camus’ third image will be of more washed out (lesser contrast), whereas by averaging the two pixels from the first and second image as in Allred, it will increase the contrast of the third image, thereby creating a better visual final product. Additionally, Allred teaches the exact method taught by Camus as another example (i.e., paragraph 44 teaches that “For example, the intensity R(x,y) can be chosen to be equal to the lesser of I(x,y) and I'(x,y)”), while also teaching a functional equivalent/alternative of averaging (i.e., paragraph 44 teaches that “In some embodiments, the intensity is chosen to be the average of I(x,y) and I'(x,y)”). As such this modification amounts to substitution of known equivalents for generating a third image from two other images to yield predictable results (KSR v. Teleflex).
Regarding the fact that there are three processing steps of:
processing the first image by removing… a first region of interest that contains a specular reflection…
processing the second image by removing… a second region of interest that contains a specular reflection…
combining the processed first image and the processed second image to generate the third image, wherein in regions not corresponding to the first and second regions of interest, the third image contains information from both the first and second processed images;
It is noted that the combination of Camus with Zickler and Allred performs all of these steps in a single process. In other words, rather than actively removing pixel data from the first image, and actively removing pixel data from the second image to then combine those processed images, the method taught by Camus and Allred performs pixel-by-pixel processing in which, if and when the first or second image’s current pixel contains specular reflection data, that pixel is disregarded (i.e., akin to removing from the first/second image), and then a third image pixel is generated by taking the pixel data of the other of the second/first image data. In other words, the process taught by Camus and Allred presents a streamlined approach to that of the claimed invention. In other words, the process performed by Camus and Allred re-orders the steps claimed to ultimately end with the same third image, to which is MPEP 2144.04(IV)(C) states “See also In re Burhans, 154 F.2d 690, 69 USPQ 330 (CCPA 1946) (selection of any order of performing process steps is prima facie obvious in the absence of new or unexpected results)”. It is noted that there is no disclosed use for the first processed image and the second processed image other than to combine them to form the third image. Therefore, there is no criticality to the creation of these two processed images (other than to combine them to form the third image), for which this same outcome is acquired by the methods of Allred with a streamlined processing scheme.
Regarding claim 2, it is noted that Figure 1 of Camus illustrates a first light source 2 and a second light source 4. While either one of these light sources may be interpreted to read on, under a broadest reasonable interpretation, “an auxiliary light emitting diode”, the other of the two is interpreted to read on, under a broadest reasonable interpretation, “OD LED or OS LED for respective right eye or left eye imaging.” It is noted that “an auxiliary” LED does not explicitly impart any limitations to the LED, other than giving it a name – auxiliary. Additionally, it is noted that Camus teaches that “The light sources 2 and 4 may be implemented with one or more high-power light-emitting diodes” (see column 6, lines 59-60).
Regarding claim 3, it is noted that Camus teaches that “As long as the two light sources 2 and 4 are sufficiently separated so that the virtual image 23 and its associated blooming does not overlap virtual image 21 and its associated blooming, every portion of the subject is clearly visible in at least one of the first image or the second image. Hence, in one or both of the left image and center image of FIG. 3 every portion of a person's iris can be clearly seen” (see column 6, lines 50-58). Additionally, it is clearly illustrated in Figure 1 that the light sources 2 and 4 are on opposite sides of the imager.
Regarding claims 4-5, Camus teaches that “the light sources 2 and 4 are pulsed or flashed in synchronization with the exposure times of the camera 11. This can be done using a strobing device 12 and an illumination controller 14 connected to the strobing device 12 and the camera 11” (see column 4, lines 51-55). It is noted that Figure 7 illustrates that box 32 is distinct from box 34, which teaches that there is distinction between the acquisition of the right-illuminated image and the left-illuminated image, thereby reading on the additional limitations in claim 5.
Regarding claim 7, Camus teaches that “Our method and apparatus are particularly useful for creating images that are used to identify a person from that person's iris” (see column 3, lines 42-44 and claim 7). Additionally, Zickler teaches in paragraph 104 that the improvement of removing reflections from images can be helpful in iris detection, as well as paragraph 96 discussing the use of iris tracking for torsion offset determination.
Regarding claim 8, Camus teaches an imaging light includes a first light source (see numeral 2) and a second light source (see numeral 4);
a synchronization circuit configured to receive or generate a pulsed signa to illuminate the first light source for a first time period and illuminate the second light source for a second time period, different than the first time period (see column 4, lines 51-55: Camus teaches that “the light sources 2 and 4 are pulsed or flashed in synchronization with the exposure times of the camera 11. This can be done using a strobing device 12 and an illumination controller 14 connected to the strobing device 12 and the camera 11”), thereby making it obvious to one of ordinary skill in the art that there is a circuit or equivalent in the controller performing the function of this claimed circuit;
a… camera configured to capture a first image of the area during the first time period, the first image having a first light reflection from the first light source (see box 32 in Figure 7), and capture a second image of the area during the second time period, the second image having a second light reflection from the second light source (see box 34 in Figure 7);
a processor configured to generate a third image based on the captured first image and second image, the third image having no specular light reflections (see boxes 36, 38, 40 and 42; see “image processor” 16 in Figure 1; see Figure 3 illustrating the images acquired via box 32 and 34 followed by the third image with no specular reflection).
However, it is noted that Camus does not explicitly state that the camera is a “torsion camera” or that this method is used in a surgical procedure.
Zickler teaches systems and methods for controlling measurement in an eye during ophthalmic procedure(s) (see Title). In paragraph 10, Zickler teaches that “A variety of eye tracking and beam scanning techniques have been proposed to align the customized laser ablation pattern with the eye. Some of these techniques rely upon imaging the eye. Specular reflections from natural (e.g., tear film, epithelium, stroma, sclera, and the like) or artificially created (e.g., from a corneal flap incision, surgical instrument, intraocular lens, implant, and the like) refractive index discontinuities of the human eye may decrease image detail, which would interfere with position detection and motion compensation.” Zickler provides for a manner to reduce or eliminate specular reflection, but does so utilizing polarization of light (see Figures 12-14). Paragraph 108 teaches that “The image [shown in Figure 12] is captured by an image capture device, such as a camera, eye tracker, or torsional tracker.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize the techniques of Camus within an ophthalmic surgical setting to remove specular reflection within images of the eye and to acquire those images with a camera, which may include a torsion camera, as explicitly taught by Zickler, because both Camus and Zickler are directed to ocular imaging with the intention of removing specular reflections within the images that degrade the ability to track the eye, and the use of either one to perform the method taught by of specular reflection removal within a surgical setting via a torsion camera versus any other camera with comparable frame rates would amount to choosing from a finite number of imaging devices and techniques available in the art at the time of the invention, which has previously been held as unpatentable (KSR v. Teleflex).
Regarding claim 9, Figure 3 illustrates that the area comprises an iris. Also, Camus teaches that “Our method and apparatus are particularly useful for creating images that are used to identify a person from that person's iris” (see column 3, lines 42-44 and claim 7). Additionally, Zickler teaches in paragraph 104 that the improvement of removing reflections from images can be helpful in iris detection, as well as paragraph 96 discussing the use of iris tracking for torsion offset determination.
Regarding claim 10, it is noted that Figure 3 of Camus illustrates that the different light sources are arranged to produce reflections captured in images at different locations of the area.
Regarding claim 13, Camus teaches that “The light sources may produce visible light or non-visible light such as infrared” (see column 6, lines 66-67; also see claims 3, 12 and 22 of Camus).
Regarding claim 14, it is noted that Zickler teaches that “While the system and methods of the present invention are described primarily in the context of improving a laser eye surgery system, it should be understood the techniques of the present invention may be adapted for use in alternative eye treatment procedures and systems such as femtosecond lasers and laser treatment, infrared lasers and laser treatments, radial keratotomy (RK), scleral bands, follow up diagnostic procedures, and the like” (see paragraph 18). Therefore, the combination of Camus with the teachings of Zickler make it obvious to utilize eye tracking via specular reflection reduction within ophthalmic surgeries that include those using femtosecond lasers.
Regarding claim 15, Camus states that “FIG. 3 shows a left-illuminated eye image, a right-illuminated image of the same eye, and a third image formed by fusion of the other two images” (see column 4, lines 18-20).
Regarding claims 16 and 18, the following is re-iterated from the rejection of claim 1 above: regarding the fact that there are three processing steps of:
processing the first image by removing… a first region of interest that contains a specular reflection…
processing the second image by removing… a second region of interest that contains a specular reflection…
combining the processed first image and the processed second image to generate the third image, wherein in regions not corresponding to the first and second regions of interest, the third image contains information from both the first and second processed images;
It is noted that the combination of Camus with Zickler and Allred performs all of these steps in a single process. In other words, rather than actively removing pixel data from the first image, and actively removing pixel data from the second image to then combine those processed images, the method taught by Camus and Allred performs pixel-by-pixel processing in which, if and when the first or second image’s current pixel contains specular reflection data, that pixel is disregarded (i.e., akin to removing from the first/second image), and then a third image pixel is generated by taking the pixel data of the other of the second/first image data. In other words, the process taught by Camus and Allred presents a streamlined approach to that of the claimed invention. In other words, the process performed by Camus and Allred re-orders the steps claimed to ultimately end with the same third image, to which is MPEP 2144.04(IV)(C) states “See also In re Burhans, 154 F.2d 690, 69 USPQ 330 (CCPA 1946) (selection of any order of performing process steps is prima facie obvious in the absence of new or unexpected results)”. It is noted that there is no disclosed use for the first processed image and the second processed image other than to combine them to form the third image. Therefore, there is no criticality to the creation of these two processed images (other than to combine them to form the third image), for which this same outcome is acquired by the methods of Allred with a streamlined processing scheme.
Regarding claims 17 and 19, Allred teaches “If neither the intensity of the pixel in the first image nor the intensity of the pixel in the second image is greater by a factor T (as determined in step 695), a selection of an intensity value R(x,y) for the corresponding pixel is performed. … the intensity is chosen to be the average of I(x,y) and I'(x,y).” This teaches that for areas in which there is no specular reflection, then the intensity of the pixels for both images is averaged.
Claims 6 and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Camus in view of Zickler as applied to claims 5, 8 and 10 above, and further in view of Haven et al. (US Patent Pub. No. 2004/0170304).
Camus in combination with Zickler is described above with respect to claims 5, 8 and 10. However, Camus only states that “If a first illuminator and a second illuminator are alternately illuminated at the same frame rate as the video camera, then one field for a single image will have been created with the first illuminator activated and the other field for that image will have been created with the second illuminator activated. Consequently, if video images are available we can select the even field as our first image and the odd field as our second image or vice versa. If this is done we assume that adjacent pairs of pixels in the video image, one pixels from an odd raster line and the other pixel from an even raster line, correspond to a single unique point on the subject” (see column 6, lines 1-14). However, there is no explicit frame rates mentioned.
Haven teaches an apparatus and method for detecting pupils (see Title). Figure 4A illustrates a timing chart between frame acquisitions and pulses emitted by “the on-axis and off-axis light sources (e.g., first light source 103 and second light source 105, respectively, of FIG. 1)” (see paragraph 51-52). It is noted that Figure 1 of Haven illustrates a similar structural set up to that of Figure 1 of Camus. As shown in Figure 4A, an image frame is acquired during each pulse of on-axis and off-axis illumination. Paragraph 47 of Haven teaches an exemplary frame rate of 30 frames per second (fps). However, paragraph 48 teaches that “It is appreciated that frame rates other than 30 fps may be used; for example, higher frame rates are expected to reduce artifacts resulting from the motion of subject 120 (e.g., the vehicle operator) or from background motion.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize frame rates higher than 30 fps, thereby including 60 fps and higher, with in ocular imaging of the eye for tracking (i.e., pupil detection) via a system utilizing multiple illumination angles, as taught by Haven, and to do so within the system and methods of Camus as combined with Zickler, since it has been held that discovering the optimum value of a result effective variable involves only routine skill in the art (In re Boesch, 617 F.2d 272, 205 USPC 215 (CCPA 1980) - see MPEP 2144.05(II)), and Haven explicitly teaches that high frame rates are beneficial in reducing artifacts.
Regarding claim 12, it is noted that Haven teaches multiple structural set ups for the detector and light source(s). These include the configuration shown in Figure 1 and 3, the configuration shown in Figure 5 which most resembles that of Camus, the configuration shown in Figures 8-13. It is noted that Figures 11 and 12 illustrate configurations that read on the subject matter of claim 12, for incorporation of a third light source. Also, Haven teaches that “It is appreciated that both eyes may be monitored” (see paragraph 40).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES KISH whose telephone number is (571)272-5554. The examiner can normally be reached M-F 10:00a - 6p EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Unsu Jung can be reached at (571) 272-8506. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES KISH/ Primary Examiner, Art Unit 3792
Prosecution Insights