Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/31/2024 was filed and is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
Claims are non-statutory under the most recent interpretation of the Interim Guidelines regarding 35 U.S.C. 101 because: the computer-readable storage medium claimed, stated in claim 10, is not positively disclosed in the specification as a statutory only embodiment (para. [0090-0092]). The broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) transitory embodiments are not directed to statutory subject matter) and Interim Examination Instructions for Evaluating Subject Matter Eligibility Under 35 U.S.C. § 101, Aug. 24, 2009; p. 2. To overcome this rejection, the claim may be amended to recite "a non-transitory...".
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 9, 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimoto (“Eyelid Measurements Using Digital Video Processing”, 2008, also cited in IDS filed 10/31/2024), in view of Valenzuela (“Towards an Efficient Segmentation Algorithm for Near-Infrared Eyes Images”, 2020).
Regarding claims 1 and 9, Morimoto teaches A method for measuring palpebral fissure height (Morimoto, Section 1, last paragraph: “Section 4 introduces our computer vision system that automatically measures the Palpebral Fissure and the Marginal Reflex Distance from the video camera”. “measures the Palpebral Fissure” is being interpreted as involving “palpebral fissure height”), using a camera to obtain an eye image (Morimoto, Section 4, first two paragraphs, reproduced below:
PNG
media_image1.png
1018
1093
media_image1.png
Greyscale
. “When the patient is looking at the camera” is being interpreted as involving obtaining an eye image), comprising:
acquiring a first eye position image of a user in front of an eye (Morimoto, see nearest image above, “The first step to capture a good image of the eye is to control the illumination conditions”. Which is being interpreted as involving “a first eye position image”), looking straight ahead (Morimoto, see nearest image above, “when the patient is looking at the camera”) in a near-infrared light field of 700-1200 nm (Morimoto, see nearest image above, “We use two near infrared (NIR) light sources” which is being interpreted as involving “near-infrared light field of 700-1200 nm”);
segmenting a background (Morimoto, see nearest image above, “eyelids are segmented”), an iris (Morimoto, see nearest image above, “iris…segmented), a sclera (Morimoto, pg 1371, Figure 7, the right images show the sclera is segmented) and a pupil (Morimoto, see nearest image above, “pupil is computed”) from the first eye position image (Morimoto, see nearest image above, “The first step to capture a good image of the eye is to control the illumination conditions”. Which is being interpreted as involving “a first eye position image”) …
extracting a center of the pupil from the segmented pupil (Morimoto, pg 1372, Section 4.2, ¶2, reproduced below: “This situation can be automatically detected by the system since the corneal reflection, as shown in Figure 8 appears in the center of the pupil”. “Center of the pupil”; See Figure 9 below, or the color version, which shows the segmented pupil) and obtaining a center line of the pupil in a vertical direction (Morimoto, pg 1372, Figure 9, reproduced below:
PNG
media_image2.png
640
602
media_image2.png
Greyscale
. The vertical line that includes the center of the pupil is being interpreted as involving “obtaining a center line of the pupil in a vertical direction”); and
determining a distance between a junction point of the sclera, the iris or the pupil on the center line of the pupil and the background (Morimoto, pg 1373, Figure 10, which shows the red vertical line going through the center of the pupil, and through the iris. Further, the background, or the eyelid segmented in black, is being interpreted as part of the red line measurement of the palpebral fissure. This sentence phrase is being interpreted as an “or” statement.), and calculating the palpebral fissure height using the distance (Morimoto, pg 1372, Section 4.2, ¶3: “Figure 10 shows some results for the segmentation of the eyelids for the PF and MRD measurements”. Which is being interpreted as involving palpebral fissure height).
However, Morimoto does not appear to explicitly teach by a training method of a neural network.
Pertaining to the same field of endeavor, Valenzuela teaches
by a training method of a neural network (Valenzuela, Abstract, “This article explores alternatives to improve the efficiency of the state of the art method, namely DenseNet Tiramisu, when applied to NIR image segmentation”; see Figure 3 for segmentation of iris, pupil, and sclera. “DenseNet” is being interpreted as involving training of a neural network);
Morimoto and Valenzuela are considered to be analogous art because they are directed to image analysis of the human eye. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for eyelid measurements using digital video processing (as taught by Morimoto) to try out using a neural network (as taught by Valenzuela) because the combination provides an improvement to eye structure detection (Valenzuela, Abstract).
Regarding claim 10, Morimoto teaches A computer-readable storage medium (Morimoto, pg 1371, column 1, last paragraph, “the algorithm searches” is being interpreted as involving a computer-readable storage medium), configured to have at least one program code (Morimoto, pg 1371, column 1, last paragraph, “the algorithm searches”. “Algorithm” is being interpreted as involving “at least one program code”) that is loaded and executed by a processor (Morimoto, pg 1371, column 1, last paragraph, “the algorithm searches”, which is being interpreted to involve “a processor”) to perform the method described in Claim 1 (Morimoto, pg 1369, Section 1, last paragraph: “In this paper we introduce a computer vision technique to automate this process.”).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimoto, as modified by Valenzuela, in view of De Santis (“Optimal segmentation of pupillometric images for estimating pupil shape parameters”, 2006).
Regarding claim 2, Morimoto teaches The method according to Claim 1, wherein the pupil center is an average value of X and Y coordinates (De Santis, see nearest image below, “Matlab instruction regionprops” is being interpreted as obtaining the centroid through the average value of X and Y coordinates. One with ordinary skill would be able to see the source code using the “open regionprops” and observe this to be true where the function starts around line 501. See informational reference: https://www.mathworks.com/matlabcentral/answers/243958-how-does-regionprops-compute-for-the-centroid) of all pixels of the pupil segmented (De Santis, see nearest image below, “same pupil gray level (….segmented image)”, which shows the pupil is segmented) from the first eye position image (De Santis, see nearest image below, “eye elements” shows “the first eye position image” is involved), and the center line of the pupil is obtained by drawing a vertical line through the average value of the X coordinates (De Santis, pgs 180-181, Section 3, ¶4, reproduced below:
PNG
media_image3.png
1023
755
media_image3.png
Greyscale
. Regionprops is being interpreted as involving “the center line of the pupil is obtained by drawing a vertical line through the average value of the X coordinates” as indicated at the “minor and major ellipse axes”. Further, the regionprops instruction contains Major and Minor Axis length, which involve a vertical line through the center).
Morimoto, Valenzuela, and De Santis are considered to be analogous art because they are directed to image analysis of the human eye. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for eyelid measurements using digital video processing, modified to involve a neural network (as taught by Morimoto and Valenzuela) to include average X and Y to obtain the center (as taught by De Santis) because the combination provides an improvement to eye structure detection and segmentation (De Santis, Abstract). Further, this is common mathematics, as one with ordinary skill in the art would know.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimoto, as modified by Valenzuela, in view of Wu (“Study on Iris Segmentation Algorithm Based on Dense U-Net”, 2019).
Regarding claim 3, Morimoto teaches The method according to Claim 1, wherein the training method of the neural network segments the background, the iris, the sclera and the pupil from the first eye position image by:
adopting a combination of UNet and DenseNet neural network models as a model for the neural network (Wu, pg 123960, Section II, last paragraph, reproduced below:
PNG
media_image4.png
444
750
media_image4.png
Greyscale
“We combine Densenet with U-Net”),
taking the first eye position image as an input to the model for the neural network (Wu, see nearest image above, “Iris segmentation” is being interpreted as involving “first eye position”),
adopting the UNet neural network model to reduce (Wu, see nearest image above, “contraction…paths”) and increase a dimension of the input (Wu, see nearest image above, “expansion paths”, which is being interpreted to involve the input),
transmitting an output of a dimensionality reducing block to a corresponding dimensionality increasing block by a skip connection (Wu, pg 123962, Figure 4, reproduced below:
PNG
media_image5.png
612
1128
media_image5.png
Greyscale
. “Skip/concatenation connection”. Examiner recommends viewing the color version. The yellow skip connection arrow is being interpreted as involving “transmitting an output of a dimensionality reducing block to a corresponding dimensionality increasing block”), and
each dimensionality decreasing block and upsampling each dimensionality increasing block using the DenseNet neural network model (Wu, see Section II image above, “Dense U-Net integrates dense connectivity into U-Net's contraction and expansion paths.” “Contraction” is being interpreted as involving “each dimensionality decreasing block”, and “expansion paths” are being interpreted as ”upsampling each dimensionality increasing block”).
Morimoto, Valenzuela, and Wu are considered to be analogous art because they are directed to image analysis of the human eye. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for eyelid measurements using digital video processing, modified to involve a neural network (as taught by Morimoto and Valenzuela) to include a combination of DenseNet and U-Net (as taught by Wu) because the combination provides an improvement to Iris segmentation accuracy (Wu, Section II, last paragraph).
Claim(s) 4 and 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimoto, as modified by Valenzuela, in view of JunMa (“Loss Functions for Medical Image Segmentation: A Taxonomy”, 2019), further in view of Chaudhary (“RITnet: Real-time Semantic Segmentation of the Eye for Gaze Tracking”, 2019).
Regarding claim 4, Morimoto teaches The method according to Claim 3, further comprising verifying a training effect (Valenzuela, pg 171602, Section IV A, line 1: “For training and testing”) of the neural network (Valenzuela, Abstract, “DenseNet”) by a loss function (Valenzuela, Abstract, “DenseNet”, which is being interpreted to have a loss function for training),
wherein the loss function is a compound loss (JunMa, pg 5, last paragraph, reproduced below:
PNG
media_image6.png
148
770
media_image6.png
Greyscale
) that comprises a
focal loss (JunMa, pg 2, 3rd bullet point, reproduced below:
PNG
media_image7.png
114
746
media_image7.png
Greyscale
),
However, JunMa does not appear to explicitly teach surface loss, but may teach a component of it, “Hausdorff Distance” on pg 5. Though does teach generalized dice loss (pg 4) and boundary aware loss (pg 4).
Pertaining to the same field of endeavor, Chaudhary teaches
a generalized dice loss (Chaudhary, Section 3.1, reproduced below:
PNG
media_image8.png
820
482
media_image8.png
Greyscale
. “Generalized Dice Loss”),
a surface loss (Chaudhary, see nearest image above, “Surface Loss”) and
a boundary aware loss (Chaudhary, see nearest image above, “Boundary Aware Loss”).
Morimoto, Valenzuela, JunMa, and Chaudhary are considered to be analogous art because they are directed to medical image segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for eyelid measurements using digital video processing, modified to involve a neural network (as taught by Morimoto and Valenzuela to include compound loss functions (as taught by JunMa and Chaudhary) because the combination provides an improvement to segmentation of eye images (Chaudhary, Section 3.1; Abstract).
Regarding claim 5, Morimoto teaches The method according to Claim 4, wherein the compound loss is calculated as follows: compoundloss=a1×surfaceloss+a2×focalloss+a3×generalizeddiceloss+a4×boundaryawareloss×focalloss (Chaudhary, Section 3.1, last paragraph, reproduced below:
PNG
media_image9.png
64
628
media_image9.png
Greyscale
; JunMa, pg 3, “Focal Loss”; JunMa, pg 5, “By summing over different types of loss functions, we can obtain several compound loss functions”) wherein a1, a2, a3 and a4 are hyperparameters (Chaudhary, see nearest image above, the lamba symbols are being interpreted as hyperparameters.),
a1 is related to a number of duration(s) during training, a2 = 1, a3 = 1 - a1, and a4 = 20. (Chaudhary, pg 3700, Section 4.2, ¶1, reproduced below:
PNG
media_image10.png
392
632
media_image10.png
Greyscale
. “Epoch” is being interpreted as “related to number of duration(s) during training”. One with ordinary skill in the art would know that hyperparameters are used in neural network training),
Morimoto, Valenzuela, JunMa, and Chaudhary are considered to be analogous art because they are directed to medical image segmentation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for eyelid measurements using digital video processing, modified to involve a neural network (as taught by Morimoto and Valenzuela to include compound loss functions (as taught by JunMa and Chaudhary) because the combination provides an improvement to segmentation of eye images (Chaudhary, Section 3.1; Abstract). Further, JunMa demonstrates that compound losses are obvious to try with the predictable result of improving, worsening, or maintaining status quo to neural network training accuracy and other parameters.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimoto, as modified by Valenzuela, in view of Leorna (“Estimating animal size or distance in camera trap images: Photogrammetry using the pinhole camera model”, May 2022).
Regarding claim 6, Morimoto teaches The method according to Claim 1, wherein the palpebral fissure height is:
…wherein the pixel distance (A) of the palpebral fissure height is a distance (Morimoto, pg 1372, column 2, ¶1: “The distance from the corneal reflection to the upper and lower lids define the uMRD and lMRD respectively, in image pixels.” In total, this involves the pixel distance of the palpebral fissure height) between the junction point of the sclera, the iris, or the pupil on the center line of the pupil and the background (Morimoto, pg 1372, Figure 9, the cross hairs show the junction point of the sclera [white part of the eye], iris, and pupil).
However, Morimoto does not appear to explicitly teach pinhole camera model and triangulation. Although, these mathematics are common enough that one with ordinary skill in the art would be aware of, or easily discover, these equations and apply them for the field of image analysis for the eyes.
Pertaining to the same field of endeavor, Leorna teaches
Palpebral fissure height (B) (Leorna, pg 1709, Figure 1, s_0, object size) = a pixel distance (A) of the palpebral fissure height (Leorna, pg 1709, Figure 1, s_i, size of object on image, in pixels) × a single pixel width or length (Leorna, pg 1709, Figure 1, a single pixel would be 1, as one with ordinary skill in the art would know, all numbers have a 1 multiplied to it by default, this is where the identity rule comes in for multiplication) × a distance (D) from the palpebral fissure to a lens of the camera (Leorna, pg 1709, Figure 1, d_0, distance to object) ÷ a distance (C) from a film in the camera to the lens of the camera (Leorna, pg 1709, Figure 1, d_i, distance to image, focal length in pixels),
Morimoto, Valenzuela, and Leorna are considered to be analogous art because they are directed to image analysis involving pixels and real world measurements. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for palpebral fissure height calculation using a camera and segmentation, which gets pixel dimensions (as taught by Morimoto and Valenzuela) to include pinhole camera model and triangulation with pixel to real world conversion (as taught by Leorna) because the combination provides an improvement to using cameras in novel ways (Leorna, Abstract). Further, the mathematics would be obvious to try since one with ordinary skill in the art would easily research methods on how to convert pixel measurements into real world measurements using camera parameters and known distances.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimoto, as modified by Valenzuela and Leorna, in view of Pereira (“A Comparative Study of Clinical vs. Digital Exophthalmometry Measurement Methods”, May 2022).
Regarding claim 7, Morimoto teaches The method according to Claim 6, wherein the distance (D) from the palpebral fissure to the lens of the camera is the distance from a corner of the eye to the lens of the camera (1), minus an ocular prominence or exophthalmia (2) (Pereira, pg 2, Section 2.3: “exophthalmos was defined as the distance from the lateral orbital rim to the corneal vertex”. “Corneal vertex” is being interpreted as involving “exophthalmia”. “lateral orbital rim” is being interpreted as involving “corner of the eye”).
Morimoto, Valenzuela, Leorna, and Pereira are considered to be analogous art because they are directed to image analysis of the human eye. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for eyelid measurements using digital video processing, modified to have a neural network, along with pinhole camera model and triangulation (as taught by Morimoto, Valenzuela, and Leorna) to include exophthalmia distance (as taught by Pereira) because the combination provides an avenue for testing in Digital photography exophthalmometry for accuracy, or obtaining eye measurements (Pereira, Section 5). Further, as Pereira demonstrates, clinical exophthalmometry without images already exists (Section 2.1), demonstrating that one with ordinary skill in the art would know the existence of the distance of exophthalmia. Further, the pinhole camera model for triangulation and distance gathering would know that distance affects accuracy and would test out subtracting the distance of the exophthalmia, or the eye protrusion.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morimoto, as modified by Valenzuela, Leorna and Pereira, in view of Ibraheem (“Exophthalmometric value and palpebral fissure dimension in an African population”, 2014).
Regarding claim 8, Morimoto teaches The method according to Claim 7, wherein the ocular prominence or exophthalmia (2) is an average of a normal eye exophthalmia (Ibraheem, Abstract, “These healthy adult subjects had their exophthalmometric values and palpebral fissure dimensions (horizontal and vertical palpebral fissure, lateral and medial canthal distances, inter-outer canthal distance, inter-inner canthal distance, margin reflex distance, and lid crease) measured with a Hertel’s exophthalmometer and a plain non-stretchable plastic ruler, respectively.” Which shows normal eye exophthalmia data was collected).
Morimoto, Valenzuela, Leorna, Pereira, and Ibraheem are considered to be analogous art because they are directed to image analysis of the human eye. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for eyelid measurements using digital video processing, modified to have a neural network, along with pinhole camera model and triangulation, further modified to finding the distance of exophthalmia (as taught by Morimoto, Valenzuela, Leorna, and Pereira) to include average of a normal eye exophthalmia (as taught by Ibraheem) because the combination provides an avenue for testing in Digital photography exophthalmometry for accuracy, or obtaining eye measurements (Pereira, Section 5). Further, as Pereira demonstrates, clinical exophthalmometry without images already exists (Section 2.1), demonstrating that one with ordinary skill in the art would know the existence of the distance of exophthalmia. Further, the pinhole camera model for triangulation and distance gathering would know that distance affects accuracy and would test out subtracting the distance of the exophthalmia, or the eye protrusion. Additionally, one with ordinary skill in the art would experiment with taking the averages for the estimates, as a way to compare the results with digital exophthalmometry vs clinical exophthalmometry.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Shin et al (US 2025/0182281 A1, Foreign Priority of 08/09/2022) discloses image segmentation of sclera (interpreted from eyeball area), pupil, and iris.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY B DUONG whose telephone number is (571)272-1358. The examiner can normally be reached Monday - Thursday 10a-9p (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.B.D./Examiner, Art Unit 2667 /MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights