DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
During a telephone conversation with Attorney Andreanne Auger on September 9, 2026, the examiner indicated that the claims were being restricted to Group I, consisting of claims 1-12, 14 and 26 and Group II, consisting of claims 29-34. Attorney Auger elected, without traverse, the Group I claims for prosecution on the merits. Accordingly, claims 1-12, 14 and 26 have been examined on the merits and claims 29-34 have been withdrawn from consideration.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 27, 2024 is in compliance with 37 CFR 1.97 and 1.98 and therefore has been considered by the examiner and placed in the file.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation (BRI) using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The BRIs are used for purposes of searching for prior art, but cannot be incorporated into the claims. Claim limitations must be given their plain meaning unless such meaning is inconsistent with the specification. MPEP 2111.01. BRIs for some of the claim limitations are provided below. Should Applicant believe that other interpretations are warranted, Applicant should point to the portions of the present disclosure that clearly show that a different interpretation is appropriate.
Regarding the use of optional or alternative claim language, under MPEP 2111.04, claim scope is not limited by claim language that suggests or makes optional but does not require steps to be performed. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claim Objections
Claim 6 is objected to because of the following informalities: in line 2, the term “calculation” should be changed to - -calculations- -.
Claim 26 is objected to because of the following informalities: in line 5, the phrase “light the M respective wavelengths” should be changed to - -light of the M respective wavelengths- -.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 6, 12 and 14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Pat. No. 10,964,036 B2 to Sylvestre et al. (hereinafter referred to as “Sylvestre”).
Regarding claim 1, Sylvestre discloses a method for detecting biomarkers in a biological tissue (Col. 7, lines 8-11: “[w]ithout limitation, the present technology may be used to detect the presence of biomarkers, for example biomarkers correlated with cerebral amyloids, in an ROI of the subject”), the method
comprising:
obtaining, at a receiver, M images of the biological tissue, each of the M images containing light at one of M respective wavelengths (Fig. 2, Col. 3, lines 47-51: “[a]n organ or tissue 50 of a subject contains a ROI 52 from which a plurality of images 541, 542 . . . 54j are obtained at j distinct wavelengths to generate a hyperspectral image 56 of the ROI 52.” In Sylvestre, the letter “j” corresponds to the number of images and the number of respective wavelengths, and therefore corresponds to the letter “M” in claim 1);
applying, to each of the M images, L masks for obtaining MxL pixel groups (the BRI for the term “mask” is based on its plain meaning since the present specification does not explicitly define the term. In the context of image processing, a mask is applied to image data to control which pixels of the image data will and will not be selected for and/or affected by a subsequent process. This is consistent with the present disclosure in that it describes the masks as being used to sub-sample the pixels comprising each image to obtain groups of pixel values. In Sylvestre, a sliding window 62 acts as L masks in that it is applied L times to each of the images to obtain, for each image, L groups of pixels, each group being k pixels in width and l pixels in height, i.e., to obtain L kxl groups of pixels for each image. This is shown in Figs. 2 and 3 of Sylvestre and described in Col. 3, line 56-Col. 4, line 3: “[a] portion of the hyperspectral image 56, in a window 62, contains spatial information over a width of k pixels and a height of l pixels, in which each of k and l are greater than or equal to one (1) pixel, this window 62 also containing spectral information 64 defined over the j distinct wavelengths. A texture analysis of the hyperspectral image 56 is performed based on spatial information contained in the k·l pixels of the window 62, the texture analysis being resolved over the j distinct wavelengths. By moving the window 62 over the area of the ROI 52, the texture analysis provides a texture image 20B of the ROI 52. The texture image 20B contains information describing the ROI 52, for example normalised contrast image, normalised homogeneity image, normalised correlation image and/or normalised energy image of the ROI 52.”);
applying K statistical calculations to each of the MxL pixel groups (Sylvestre discloses that a texture analysis corresponding to block 110 of Fig. 3 is performed on each of the MxL pixel groups to generate a texture image 20B. The K statistical analyses include a Gray level co-occurrence matrix (GLCM) analysis and/or a Gray level run length matric (GLRLM) analysis and/or a Markov Random Field matrix (MRFM) analysis (Col. 3, line 39-Col. 4, line 23, and Col. 4, line 56-Col. 5, line 8), each of which is considered in the art to be a statistical analysis (GLCM and GLRLM analysis are considered second order statistical analyses and MRFM is a model-based probabilistic analysis). Once the texture image 20B has been obtained, further statistical analysis of the pixel groups are performed, Fig. 3, step 114, Col. 5, lines 48-54: “[f]irst order statistics (FOS) based on a histogram of the texture image may be calculated at operation 114. The FOS may include any one or more of intensity mean, intensity skewness, intensity variance and intensity kurtosis of the texture image.”);
assembling results of the statistical calculations in M feature vectors, each feature vector containing LxK results for a corresponding one of the M wavelengths (As discussed below in regard to the following limitation of claim 1, the classification operation that uses the results of the statistical calculations is performed by a machine learning system 412 shown in Fig. 5. As is known in the art, machine learning operations process feature vectors, also commonly referred to as embeddings, and therefore the LxK results of the statistical calculations are necessarily assembled into feature vectors before being input to the classifier of the machine learning system 412. Also, block 115 of Fig. 3 indicates that feature vectors are assembled before inputting the results into the classifier in that it indicates that dimensional reduction may or may not be performed on the results, which indicates the results are vectors); and
using a machine learning system to extract, from the M feature vectors, a positive indication or a negative indication of a presence of a given biomarker in the biological tissue (The classification operation 116 of Fig. 3 is an operation that is performed by a machine learning system, Fig. 5, block 412, which uses the results of the statistical calculations to extract a positive indication or a negative indication of a presence of a given biomarker in the biological tissue, Col. 5, line 65-Col. 6, line 2: “[t]he classification may lead to a negative result 118, in which case the biological tissue is classified as normal, or to a positive result 120, in which case the biological tissue is classified as abnormal”. Also discussed in Col. 7, lines 8-12: “[w]ithout limitation, the present technology may be used to detect the presence of biomarkers, for example biomarkers correlated with cerebral amyloids, in an ROI of the subject.” Fig. 5 shows the machine learning model 412 that performs the classification operation, as discussed in Col. 8, lines 67-68, “[t]he processor 404 implements an image analysis module 410 and a machine learning module 412.” ).
Regarding claim 2, Sylvestre discloses that the M images of the biological tissue are images of a retina of a subject (Col. 4, lines 9-10 “the biological tissue may comprise the retina of a subject, or a part thereof….”).
Regarding claim 3, Sylvestre discloses that M is an integer number at least equal to 2 (Sylvestre uses the letter “j” in the same way that the present disclosure uses the letter “M” and indicates that j is at least equal to 2 (Col. 3, lines 48-51 discloses that j>=2); L is an integer number at least equal to 1 (Sylvestre discloses that the mask is a sliding window. Each position of the window corresponds to one of the L masks, and therefore L is at least equal to 1); and K is an integer number at least equal to 1 (As indicated above, Sylvestre discloses that a Gray level co-occurrence matrix (GLCM) analysis and/or a Gray level run length matric (GLRLM) analysis and/or a Markov Random Field matrix (MRFM) analysis is applied, and therefore K is equal to at least 1).
Regarding claim 6, Sylvestre discloses that each of the K statistical calculation is selected from an average, a variance, a skewness, a kurtosis, a standard deviation, a median, a smallest value, a largest value, a first, second or third quartile, and any combination thereof (As is known in the art, a GLCM analysis involves both averaging and variance calculations. Additionally, Col. 5, lines 48-50 discusses “[f]irst order statistics (FOS) based on a histogram of the texture image may be calculated at operation 114. The FOS may include any one or more of intensity mean, intensity skewness, intensity variance and intensity kurtosis of the texture image.”).
Regarding claim 12, Sylvestre discloses:
using a multispectral light source to illuminate the biological tissue (Col. 3, line 42-51 discusses the M images obtained at respective wavelengths based on reflectance or fluorescence emitted from the tissue, which means the system includes a multispectral light source for generating the multiple wavelengths);
using a multispectral camera positioned in view of the biological tissue to acquire the M images of the biological tissue (Fig. 5, client devices 3021-302n include cameras that can be multispectral for acquiring images of biological tissue illuminated by a multispectral light source, Col. 8, lines 57-60: “[e]ach client device 302 comprises an image acquisition device (not specifically shown), for example a camera, a plurality of monochromatic images and/or an hyperspectral image of an ROI of a subject.”); and
transferring the M images of the biological tissue from the multispectral camera to the receiver (Figs. 5 and 6, data transfer connection 304 transfers data to receiver 400, which includes the image analysis module 410 and the machine learning module 412: “[e]ach client device 302 comprises an image acquisition device (not specifically shown), for example a camera, a plurality of monochromatic images and/or an hyperspectral image of an ROI of a subject. The client devices 302.sub.1, 302.sub.2 . . . 302.sub.n, use a data transfer connection 304 to forward images of biological tissues from their image acquisition devices to the device 400.”).
Regarding claim 14, to the extent that claim 14 recites the same limitations that are recited in claim 1, the rejection of claim 1 applies mutatis mutandis to claim 14. As indicated above in the rejection 12, Fig. 6 shows the receiver 400 that receives the M images transferred to the receiver 400 via the data transfer connection 304 shown in Fig. 5. The processor 404 “implements an image analysis module 410 and a machine learning module 412. The device 400 outputs, for a given biological tissue, one of the negative result 118 or positive result 120….” (Col. 8, line 66-Col. 9, line 2).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 4, 5 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Sylvestre in view of U.S. Pat. No. 10,957,041 B2 to Yip et al. (hereinafter referred to as “Yip”).
Regarding claim 4, the BRI for this limitation is that the results obtained by applying the K statistical calculations to each pixel group is used to classify the presence or absence of a biomarker. As indicated above in the rejection of claim 1, Sylvestre discloses that the results of the statistical calculations applied to the pixel groups of the sliding window mask are used by the classifier (Fig. 3, 116; Fig. 5, block labeled “Classification (negative vs positive)” to classify the presence or absence of a biomarker). However, Sylvestre does not explicitly disclose that the L masks are anatomical masks. The BRI for the term “anatomical mask” is based on para. [59] of the present specification, which indicates that they are masks used for extracting particular anatomical features.
Yip, in the same field of endeavor, discloses applying tiling masks to received images to sub-sample the images into “small sub-images” for analysis, where the tiling masks that are used during the tiling process are optimized to extract different biomarkers (Col. 22, lines 45-53: “[t]he pipeline 315 may store a plurality of different tiling masks and select a tiling mask. In some examples, the image tiling process selects one or more tiling masks optimized for different biomarkers, i.e., in some examples, image tiling is biomarker specific. This allows, for example, to have tiles of different pixel sizes and different pixel shapes that are selected specifically to increase accuracy and/or to decrease processing time associated with a particular biomarker.”).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system and method of Sylvestre to replace the sliding window process of Sylvestre with the image tiling process of Yip such that L anatomical masks are applied to each of the M images prior to performing the statistical analysis. One of ordinary skill in the art would have been motivated to make the modification to more accurately classify the presence or absence of different biomarkers by using masks that are optimized for the biomarkers. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying the software executed by processor 404 of Sylvestre to replace the sliding window masking process or Sylvestre with the image tiling masking process of Yip).
Regarding claim 5, as indicated above, Sylvestre does not explicitly disclose using anatomical mask. Yip discloses that each anatomical mask is configured to highlight, in the M images of the biological tissue, a corresponding pixel group defining an element selected from an optic nerve hypoplasia, a blood vessel, an optic nerve head, a vessel inside the optic nerve head, a contour of a blood vessel, pigment spots, a drusen, and a retinal background (the tissue classification model 320 of Yip classifies the tiles resulting from the tile masking process into tissue classes including “but are not limited to tumor, stroma, normal, lymphocyte, fat, muscle, blood vessel, immune cluster, necrosis, hyperplasia/dysplasia, red blood cells, and tissue classes or cell types that are positive (contain a target molecule of an IHC stain, especially in a quantity larger than a certain threshold) or negative for an IHC stain target molecule (do not contain that molecule or contain a quantity of that molecule lower than a certain threshold)”, Col. 24, line 25-Col. 25, line 8). This means that each anatomical mask highlights, in the images, a corresponding pixel group defining an element of at least blood vessels and hyperplasia.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system and method of Sylvestre to replace the sliding window process of Sylvestre with the image tiling process of Yip such that the L anatomical masks applied to each of the M images prior to performing the statistical analysis highlight corresponding pixel groups in the anatomically masked images, including at least vessels and hyperplasia. One of ordinary skill in the art would have been motivated to make the modification to more accurately classify the presence or absence of different biomarkers by using masks that are optimized for the biomarkers. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying the software executed by processor 404 of Sylvestre to replace the sliding window masking process or Sylvestre with the image tiling masking process of Yip).
Regarding claim 26, Sylvestre discloses that the process described with reference to Fig. 2 of receiving and processing M images (“M” corresponds to “j” in Sylvestre) is performed for groups of images acquired for patients A, B, C and D, where the biological tissue for patients A and B is amyloid negative and the biological tissue for patients C and D is amyloid positive, and where each group of images corresponds to one of a plurality of biological tissues (Col. 9, line 63-Col. 10, line 18). Thus, the receiver 400 shown in Fig. 6 of Sylvestre receives the groups of images corresponding to the patients and performs the process described with reference to Fig. 2 for each group of images to classify the presence or absence of a biomarker. However, Sylvestre does not explicitly describe the manner in which the machine learning system of Sylvester is trained.
Yip discloses, for group of images (Fig. 15B, PD-L1+ images and PD-L1- images), calculating a respective model loss by comparing a respective label contained in the group of images with a respective positive or negative indication of the presence of a biomarker and using the calculated loss to train a machine learning system (Fig. 15B, Col. 61, lines 26-39, Col. 65, lines 33-41, the group of images labeled PD-L1+ are indicative of the presence of the PD-L1 biomarker and the group of images labeled PD-L1- are indicative of the absence of the PD-L1 biomarker. The PD-L1+ and PDL1_ images are compared to ground truths to generate a loss that is used to train the tissue class locator 1408).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the present disclosure, to modify the system and method of Sylvestre to use the training process of Yip to train the machine learning system of Sylvestre by comparing images having biomarker labels with ground truth indications of the presence or absence of the biomarker to calculate a loss and then using the loss to train the classifier of the machine learning system of Sylvestre. One of ordinary skill in the art would have been motivated to make the modification to ensure that the machine learning system of Sylvestre is properly trained to accurately classify the presence or absence of biomarkers. The modification could have been made by one of ordinary skill in the art before the effective filing date of the present disclosure with a reasonable expectation of success because making the modification merely involves combining prior art elements according to known methods to yield predictable results (modifying the software executed by processor 404 of Sylvestre to perform the training process of Yip).
Allowable Subject Matter
Claims 7-11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 7, none of the prior art teaches of suggests that a machine learning system extracts the positive or negative indication of the presence of the given biomarker in the biological tissue from the M feature vectors by:
prepending a 0th feature vector to the M feature vectors to form a group of M+1 feature vectors, the 0th feature vector identifying a 0th position outside of the M wavelengths, each of the other M feature vectors identifying a position of the respective wavelength among the M wavelengths;
inputting the M+1 feature vectors in a sequential information analysis model; outputting at least one class embedding from the sequential information analysis model; and
applying the at least one class embedding to a classification head, the classification head outputting the positive or negative indication of the presence of the given biomarker in the biological tissue.
Regarding claims 8-11, these claims recite allowable subject matter due to their dependencies from claim 7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Publ. Appl. No. 2023/0206613 A1 discloses machine learning processes for object detection in which the feature vector generated by the flattening layer 610 is appended with additional features generated from temporal feedback and/or meta data 650, and providing the appended feature vector to the linear layer 612 for generating the fully-connected layer that provides the object data 614, the class data 616, and the bounding box data 618.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL J SANTOS whose telephone number is (571)272-2867. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL J. SANTOS/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights