Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments and amendments have persuasively overcome the: claim objections, most of the 112 rejections, and the 101 rejection. The remaining issues are addressed below.
As to the prior art, Applicant argues “Specifically, Ramirez explicitly discloses that an unidentified image is not transmitted to a second level classifier or second model for classification. In Ramirez, if input image was not classified by any of classifiers, then the system merely labels the image as unidentified.”
However, Applicant is overlooking that, as noted by Applicant, Ramirez teaches more than one classifier. Ramirez’s second classifier teaches the claimed second trained model. Additionally, the summary of Ramirez (i.e., [0004]) states “the cascade classifier architecture used in the determination step may include a two-level analysis.” Additionally, the examiner understands UNCL to be a classification.
Claim Objections
Claim 23 is objected to because of the following informalities:
Claim 23 recites the wrong preamble for parent claim 14.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-6 and 14-24 (all claims) are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 14, and 15 recite “using a second trained model which is different from the first trained model,” but it is unclear what weight to give this limitation because these computations are performed outside of the claimed processor. MPEP 2113(I). In other words, if the claimed invention only receives output, there is not a difference from how that output was developed.
Claims 4, 17, and 20 recite “wherein the selection is based on,” but this is unclear because when Markush groups recite a “selection” this is not generally understood as a required method step. See, e.g., MPEP 2117(I).
Claims 22-24 recite “where the respective classified material component image corresponds to a material component not previously trained by the first trained model,” but this is unclear because the antecedent basis is “respective classified material component image.” MPEP 2173.05(e).
Claims 22-24 recite “corresponds to,” but this is subjective. MPEP 2173.05(b)(IV). Reciting an objective standard, such as “is of” is expected to overcome this rejection.
Dependent claims are likewise rejected.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-6 and 14-24 (all claims) are rejected under 35 U.S.C. 102(a)(1) and/or (a)(2) as being anticipated by U.S. Pat. Pub 20190228527 (“Ramirez”). References are listed in a PTO-892 from the Office Action in which they are first used.
A measurement system comprising:
a camera configured to obtain plural images of a sample fluid flowing in a flow cell; and (Ramirez, claim 57, “wherein the method uses a digital microscope camera”)
an image processing device comprising:
a memory, (Ramirez, Fig. 1A, memory 140)
a processor coupled to the memory; (Ramirez, Fig. 1A, processor 120)
a camera operatively connected to the processor; and a (Ramirez, Fig. 1A, analyzer 115 and [0051] “the system may include an analyzer for collecting or acquiring images of the particles. … camera”)
wherein the processor is configured to:
extract, by a convolutional neural network, a plurality of material component images including a plurality of material components contained in the sample fluid from the plural images; (Ramirez, [0004] “To analyze the cells and/or particles contained within a biological sample, images of the cells or particles may first be collected or acquired.”)
classify, using a first trained model in which the convolutional neural network is employed, the plurality of material component images such that the plurality of material components in the sample fluid are detected, (Ramirez, abstract “In a majority of cases, the first level model architecture provides an accurate identification of the cell or particle.”)
the material components including:
red blood cell (RBC), (Ramirez, [0046] “erythrocytes (RBCs)”)
white blood cell (WBC), (Ramirez, [0046] “leukocytes (WBCs)”)
non-squamous epithelial cell (NSE), (Ramirez, [0046] “non-squamous epithelial cells”)
squamous epithelial cell (SQEC), (Ramirez, [0046] “squamous epithelial cells”)
non-hyaline cast (NHC), (Ramirez, [0046] “casts”)
bacteria (BACT), (Ramirez, [0046] “bacteria”)
crystal (CRYS), (Ramirez, [0046] “crystals”)
yeast (YST), (Ramirez, [0046] “yeast”)
hyaline cast (HYST), (Ramirez, [0046] “casts”)
mucus (MUCS), (Ramirez, [0046] “mucus”)
spermatozoa (SPRM), (Ramirez, [0046] “spermatozoa”)
white blood cell clump (WBCC) and (Ramirez, [0046] “cell clumps”)
unclassified material (UNCL), (Ramirez, [0046] “unclassified cast”)
compute a goodness of fit is a value specifying classification certainty for each of the classified plurality of material component images using a pattern matching between a respective classified material component image and a correct answer image and designate material component images including classified material component images having goodness of fits higher than or equal to a first threshold and one or more UNCL component images having goodness of fits higher than a second threshold; (Ramirez, [0004] “The system may then use hierarchical or cascaded classification architecture in analysis of the extracted features. According to various embodiments, the cascade classifier architecture used in the determination step may include a two-level analysis. If the outcome of the first level analysis is inconclusive, … .” Ramirez’s conclusiveness teaches the claimed goodness of fit. Ramirez’s inconclusiveness teaches the claimed designating (see the wherein clause of this claim). See also the mapping of claim 13.)
control a transmission of the designated material component images from among the plurality of material component images by the processor via a network line to an external device, and (Ramirez, Fig. 1A, network components 190. See also [0054] “Additionally, the storage medium 180 may be located in a first computer in which the programs may be executed, or may be located in a second different computer which connects to the first computer over a network 190.”)
receive a classification result of the designated material component images from the external device, wherein the designated material component images are classified by using a second trained model which is different from the first trained model; (Ramirez, abstract “In a minority of cases, the classification of the cell or particle requires a second level step requiring the use of numerical or categorical values from the first level in combination with a second level model.” See also Fig. 1B)
update a previous classification result associated with the designated material component images with the received classification result and configure a user interface including graphical elements displaying the received classification result. (Ramirez, Fig. 1A, display 160. See also, Fig. 7, step 790 “Determine a classification.”)
2. The measurement system of claim 1, wherein: the first trained model is a first trained model generated based on a first machine learning technique applied to training data and the second trained model is a second trained model generated based on a second machine learning technique applied to the same training data. (Ramirez, [0052] “In some embodiments, the reference images may be used as training data for a neural network implementation of the cascade classifier architecture.” Ramirez’s first and second models are different levels of the cascade architecture, and thus are trained on the one set of references images used as training data.)
3. The measurement system of claim 2, wherein:
wherein the training data is obtained by associating components of the classifications with material component images obtained in the past, and (Ramirez, [0052] “In some embodiments, the reference images may be used as training data for a neural network implementation of the cascade classifier architecture.”)
wherein the first trained model receives the plurality of material component images as an input of the first trained model and outputs the detected plurality of material components of the sample fluid. (Ramirez, abstract “In a majority of cases, the first level model architecture provides an accurate identification of the cell or particle.”)
4. The measurement system of claim 3, wherein the processor receives from the data management device a classification result obtained by classifying the designated material component images as detected components. (Ramirez, abstract “In a majority of cases, the first level model architecture provides an accurate identification of the cell or particle.” See also [0054] “Additionally, the storage medium 180 may be located in a first computer in which the programs may be executed, or may be located in a second different computer which connects to the first computer over a network 190.” Ramirez’s networked computer teaches the claimed data management device.)
wherein the first machine learning technique and the second machine learning technique are selected from the group consisting of: convolutional neural network, linear regression, regularization, decision tree, random forest, k-nearest neighbors algorithm (k-NN), logistic regression, support vector machine (SVM), wherein the selection is based on a classification performance showing classification precision, and (Ramirez, [0084] “Examples of machine learning models suitable for this architecture [“first level model”] may be Random Forest, multiclass Support Vector Machines (SVMs), Feedforward Neural Networks (FNNs), etc.” [0098] “Examples of machine learning models suitable under this architecture [“level two classifier”] may be Support Vector Machines (SVMs), Feedforward Neural Networks (FNNs), Random Forest, etc.”)
wherein the classification performance shows that the second machine learning technique provides a higher classification performance than a classification performance by the first machine learning technique. (Ramirez, [0005] “the classification of the cell or particle requires a further step (a “second level model”) to classify the cell or the particle.”)
5. The measurement system of claim 1 wherein:
the processor is configured to select one or more groups from among a plurality of designated material component images, select a representative image of each group, and (Ramirez, [0074] “FIG. 4 shows an example of the clustering process for a given ring mask in the HSV color space, where the X 400 in the chart represents the center of the clusters and each color is associated to pixels belonging to a cell category.” Ramirez’s associated pixels teach the claimed images.)
transmit the selected representative image to the external device. (Ramirez, Fig. 1A, network components 190. See also [0054] “Additionally, the storage medium 180 may be located in a first computer in which the programs may be executed, or may be located in a second different computer which connects to the first computer over a network 190.” Ramirez’s networked computer teaches the claimed external device.)
6. The measurement system of claim 4, wherein the first machine learning technique and the
second machine learning technique are different. (Ramirez, abstract “In a minority of cases, the classification of the cell or particle requires a second level step requiring the use of numerical or categorical values from the first level in combination with a second level model.” Ramirez’s different results teach the claimed different techniques.)
Claim 14 is rejected as per claim 1.
Claim 15 is rejected as per claim 1. See also, Ramirez, claim 61 “A non-transitory computer-readable storage medium.”
Claims 16-21 are rejected as per their counterpart claims.
22. The measurement system of claim 1, wherein the value increases with an increasing proportion of the pattern matching and decreases in at least one of cases where the respective classified material component image is blurred, (Ramirez, [0083] “In the first level model 600, a general classifier is trained to match the training data.” See also [0107] that “out of focus” (i.e., blurring) can reduce classification.)
where plural material components are superimposed, or
where the respective classified material component image corresponds to a material component not previously trained by the first trained model.
Claims 23 and 24 are rejected as per claim 22.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 11494905 B2 – Medical Image Recognition
US 20200134287 A1 – Material Component Images
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID ORANGE whose telephone number is (571)270-1799. The examiner can normally be reached Mon-Fri, 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ORANGE/ Primary Examiner, Art Unit 2663
Prosecution Insights