DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-4, 7-9, 11-13, and 16-18 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Arcadu (US 20230047100).
Regarding claim 1:
Arcadu discloses a method for estimating ulcerative colitis severity depicted in a frame of an endoscopic video (Arcadu ¶[0001]: “computer-implemented methods for analysing colonoscopy videos”; “clinical assessment of inflammatory bowel diseases such as ulcerative colitis”; Arcadu ¶[0005]: “analysing raw colonoscopy videos or portions thereof using a deep neural network classifier”).
receiving the frame of the endoscopic video (Arcadu ¶[0046]: “analysing the colonoscopy video or portion thereof comprises using the first … deep neural network classifier to individually classify the multiple frames”; ¶[0166]: “The SSN takes as input individual frames 3001-300n”; ¶[0177]: “for each frame of each of the 104 videos, the following 3 annotations are available”).
applying a first machine-learned model to the frame of the endoscopic video to estimate a first binary probability that the frame is indicative of ulcerative colitis of greater than a first severity level on a baseline severity scale (Arcadu ¶[0034]: an ordinal classification model may be implemented by “training multiple instances” of a DNN classifier, where “each instance … is a binary classifier”; the first classifier provides “the probability of image data belonging to any of the severity classes other than the lowest severity class”; ¶[0029]: MCES has four levels, MCES=0, 1, 2, and 3. Under Arcadu’s four-class MCES embodiment, the first binary classifier estimates whether the frame is greater than the lowest severity level, i.e., greater than MCES=0).
wherein the first machine-learned model is trained from a set of annotated training endoscopic videos (Arcadu ¶[0005]: the DNN classifier is “trained using raw colonoscopy video data, where entire videos or segments thereof in the training data are associated with the same class label”; ¶[0033]: a predicted MCES classifier can be obtained “using ‘raw’ colonoscopy videos as both training data and subject data”; ¶[0177]: “104 raw colonoscopy videos were selected” and annotated by expert gastroenterologists; ¶[0179]: selected frames were used to train the SSNs).
and wherein each of the set of annotated training endoscopic videos has a respective single label representing a maximum severity of ulcerative colitis observed with respect to the baseline severity scale (Arcadu ¶[0005]: “entire videos or segments thereof in the training data are associated with the same class label”; ¶[0177]: MCES annotations are weak labels and “for MCES scoring, an anatomical section of colon is assigned the score that corresponds to the most severe lesions seen in the section”; ¶[0054]: “a colonoscopy video or portion thereof is commonly assigned to the most severe category (highest severity score) that has been identified in the video”. Therefore, Arcadu teaches single video/segment labels and expressly teaches that MCES scoring assigns the label corresponding to the most severe lesions seen. Applicant’s specification in ¶ [0004] confirms the same conventional scoring rule: “gastroenterologists attribute a single MES to a video based upon the maximum disease severity observed in the video.”).
applying a second machine-learned model to the frame of the endoscopic video to estimate a second binary probability that the frame is indicative of ulcerative colitis greater than a second severity level on the baseline severity scale, the second severity level indicative of more severe ulcerative colitis than the first severity level, wherein the second machine-learned model is trained from the set of annotated training endoscopic videos (Arcadu ¶[0034]: the second binary classifier provides “the probability of image data belonging to the third or higher severity classes,” i.e., greater than the second severity class; ¶[0034] further states the four-class model corresponds to the four MCES levels; ¶[0039] states that the multiple binary instances may be trained simultaneously. Therefore, Arcadu’s second binary classifier estimates whether the frame is greater than a second, more severe severity threshold and is trained from the same annotated training video data).
generating an output severity score for the frame based on at least the first binary probability and the second binary probability (Arcadu ¶[0034]: “the probability of belonging to each of three or more severity classes … can be obtained based on the combined output” of the multiple binary classifiers; ¶[0038]: ordinal class probabilities are calculated from the combined binary outputs; ¶[0166]: for multilevel classifiers, “a frame may be assigned to the class that has the highest probability”).
and outputting the output severity score for the frame of the endoscopic video (Arcadu ¶[0166]: the SSN “produces … as output a severity class prediction for each frame that is analysed”; ¶[0067]: the method may output “the classification from the first deep neural network classifier for each of the multiple frames”).
Regarding claim 2:
applying a third machine-learned model to the frame of the endoscopic video to estimate a third binary probability that the frame is indicative of ulcerative colitis greater than a third severity level on the baseline severity scale, the third severity level being indicative of more severe ulcerative colitis than the second severity level, wherein the third machine-learned model is trained from the set of annotated training endoscopic videos; and wherein generating the output severity score is further based on the third binary probability (Arcadu ¶[0034]: the four-class MCES ordinal model is obtained by combining three binary DNN classifiers; the third classifier (iii) provides “the probability of image data belonging to the fourth severity class,” equivalent to class 4, where class 4 corresponds to MCES=3. Therefore, Arcadu’s third binary (iii) classifier is similar to the claimed third machine-learned model and provides a third probability for the highest MCES severity threshold).
Regarding claim 3:
wherein generating the output severity score comprises: applying a mapping function to at least the first binary probability and the second binary probability to generate respective ordinal class probabilities for a set of discrete severity levels of the baseline severity scale (Arcadu ¶[0038]: “Based on these combined outputs,” the probability of the lowest severity class is calculated as 1 - P(data in classes >1), the second class as P(data in class >1) - P(data in class >2), and the third class as P(data in class >2) - P(data in class >3)); and
selecting from the set of discrete severity levels, the output severity score that corresponds to a maximum of the ordinal class probabilities (Arcadu ¶[0166]: for multilevel classifiers, “a frame may be assigned to the class that has the highest probability”; ¶[0182]: “A single predicted MCES score was assigned to each segment as the MCES score that has the highest average probability”. Arcadu’s probability-difference equations are the claimed mapping function, and Arcadu selects the severity class/MCES score with the highest probability).
Regarding claim 4:
wherein generating the output severity score comprises: comparing the first binary probability to a threshold to generate a first binary value; comparing the second binary probability to the threshold to generate a second binary value (Arcadu ¶[0166]: a discrete classification label may be obtained by applying a threshold to the probability output by the SSN; Arcadu ¶[0179]: a frame is assigned to the first severity class by the first and second SSNs if P(MCES>1)>0.5 and P(MCES>2)>0.5, respectively);
determining the output severity score as a combination of at least the first binary value and the second binary value (Arcadu ¶[0168]: summarized severity may be obtained directly from probabilities or from discrete class assignments derived from those probabilities; Arcadu ¶[0060]: summarized severity can be assigned as the highest severity class having a proportion of frames above a threshold).
Regarding claim 7:
storing the output severity score as an entry in a set of frame-level severity scores for the endoscopic video (Arcadu ¶ [0166] discloses generating frame-level severity outputs for multiple frames of a colonoscopy/endoscopic video. Arcadu teaches that the trained severity scoring network (“SSN”) “takes as input individual frames 3001-300n” and “produces … as output a severity class prediction for each frame that is analysed”. Arcadu further teaches that the SSN outputs probabilities for individual frames, including P(S1)1 to P(S1)n, and that discrete classification labels 340B/350B may be derived from those frame-level probabilities.
Arcadu ¶ [0167] further discloses using those frame-level predictions to obtain summarized severity information for a video segment. Arcadu teaches that a summarized severity class prediction 340C/350C is obtained “based on the predictions … for each of the frames that make up the segment”. Arcadu’s frame-indexed outputs P(S1)1 to P(S1)n and discrete frame labels 340B/350B, which are then used to compute summarized severity 340C/350C, teach maintaining the frame-level severity outputs as a set of frame-level entries. Storing the frame-level outputs as entries in a set is a routine computer implementation needed to summarize, display, or otherwise process the frame-level severity predictions).
determining a maximum severity score from the set of frame-level severity scores (Arcadu ¶ [0168] discloses highest-severity summarization from frame-level predictions. Arcadu teaches that a summarized severity class for a segment may be obtained by assigning “the highest severity class (i.e. the most severe class) that is represented amongst the discrete class assignments … for the frames of the segment above a threshold for the respective severity class”. Arcadu ¶ [0054]] also teaches more generally that “a colonoscopy video or portion thereof is commonly assigned to the most severe category (highest severity score) that has been identified in the video”. Applicant’s specification admits the same conventional MES video-scoring rule: “gastroenterologists attribute a single MES to a video based upon the maximum disease severity observed in the video,” and, if a single frame shows severe UC while the remainder is normal, “the entire video is reported with an MES=3” (Applicant Spec. ¶[0004]).
and outputting the maximum severity score for the endoscopic video (Arcadu ¶ [0051] discloses outputting summarized severity information derived from frame-level predictions. Arcadu teaches that “a clinically relevant summary metric for a colonoscopy video could be obtained based on classification results from individual frames”. Arcadu ¶ [0067] further teaches outputting “the summarised severity class” and/or “the classification from the first deep neural network classifier for each of the multiple frames”).
Regarding claim 8:
wherein the first machine-learned model and the second machine- learned model are each trained using a multi-instance learning algorithm (Arcadu ¶[0061]: assigning a summarized severity class may use a first DNN classifier “trained using multiple instances learning”; Arcadu ¶[0061] further teaches combining predictions with attention-based pooling).
Regarding claim 9:
wherein the baseline severity scale comprises a Mayo Endoscopic Subscore (MES) scale having discrete integer severity levels ranging from 0 to 3 (Arcadu ¶[0029]: MCES comprises four levels referred to as MCES=0, MCES=1, MCES=2, and MCES=3; Arcadu ¶[0042]: the endoscopic severity score may be MCES with four severity classes).
Regarding claim 10:
A non-transitory computer-readable storage medium storing instructions for estimating ulcerative colitis severity depicted in a frame of an endoscopic video is disclosed by Arcadu (Arcadu ¶[0139]: “a non-transitory computer readable medium for assessing the severity of ulcerative colitis in a subject from a colonoscopy video”; Arcadu ¶[0184]: methods may be provided as computer programs or computer-readable media; Arcadu ¶[0185]: “computer readable media” includes non-transitory media). The functional limitations of claim 10 mirror claim 1 and rejected in the same manner as applied above.
Regarding claims 11-13 and 16-18: the claims limitations are similar to claims 2-4 and 7-9, respectively; therefore, rejected in the same manner as applied above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5-6, 14-15, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Arcadu (US 20230047100) in view of Diaz (“Soft Labels for Ordinal Regression” 2019).
Regarding claim 5:
Arcadu teaches the limitations of claim 1 as applied above.
wherein generating the output severity score comprises: combining at least the first and second binary probabilities (Arcadu in ¶[0179] discloses first and second binary probability outputs from severity scoring networks. Specifically, Arcadu discloses training two deep neural networks, i.e., severity scoring networks (“SSNs”), where a first SSN is trained using a first binary severity scheme MCES > 1 / MCES ≤ 1, and a second SSN is trained using a second binary severity scheme MCES > 2 / MCES ≤ 2. Arcadu further discloses that the first SSN outputs, for each frame, a probability P(MCES > 1), and the second SSN outputs, for each frame, a probability corresponding to MCES > 2; Arcadu ¶[0034] also discloses combining outputs of multiple binary classifiers to obtain ordinal severity information. Specifically, Arcadu teaches that an ordinal classification model may be implemented by training multiple binary DNN classifiers, each computing a probability for a respective severity-threshold classification, and that “the probability of belonging to each of three or more severity classes … can be obtained based on the combined output” of the multiple binary classifiers. Arcadu ¶ [0038] further teaches equations for deriving ordinal severity-class probabilities from the combined binary outputs, including calculating intermediate severity-class probabilities by subtracting cumulative binary probabilities, e.g., P(class > 1) − P(class > 2) and P(class > 2) − P(class > 3).
Arcadu does not expressly state that the combined binary probabilities are used to generate a continuous severity score.
However, in a related field, Diaz is directed to ordinal regression, which it describes as a task involving ordered categories and as resembling real-valued regression and classification. In Section 1, Diaz explains that ordinal regression can be treated as “mapping the inputs to a real line” and using “boundaries between ordinal categories” to define the final class. Diaz also explains that K-rank ordinal methods use K−1 binary classifiers trained to determine whether an input has a response y > k, and that the rank prediction typically involves the “accumulation of positive responses” from those binary classifiers.
Diaz further teaches, in Section 3.3, converting ordinal probability outputs into a scalar prediction by using “a simple expected value formula like ∑K k=1 rkpk.” Diaz also states that the ordinal probability method can handle a “continuous domain,” rather than requiring hard assignment to the closest rank.
Accordingly, Arcadu teaches the binary severity probabilities and ordinal MCES probability derivation, while Diaz teaches converting ordinal probability outputs into a continuous scalar prediction using an expected value calculation.
comparing the continuous severity score to a set of thresholds to map the continuous severity score to a discrete severity level of the baseline severity scale (Arcadu ¶ [0179] discloses threshold-based mapping of probability-derived severity information to discrete severity classes. Arcadu teaches that a frame is assigned to a severity class by comparing probability outputs to a threshold, including P(MCES>1)>0.5 and P(MCES>2)>0.5. Arcadu ¶ [0181] also teaches varying classification thresholds between 0 and 1 in 0.05 increments for ROC evaluation. Diaz in section (1. Introduction) also supports thresholding a continuous ordinal prediction into a discrete class because Diaz describes ordinal-regression methods that map inputs to a real line and use boundaries between ordinal categories to define the final output class).
outputting the discrete severity level as the output severity score (Arcadu in ¶ [0166] discloses outputting a discrete severity prediction derived from probability outputs. Arcadu teaches that a discrete classification label may be obtained by applying a threshold to the probability output by the SSN, and that for multilevel classifiers, a frame may be assigned to the class having the highest probability. Arcadu ¶ [0182] further teaches assigning a single predicted MCES score from the probability outputs).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Aracdu to incorporate the teachings of Diaz because Arcadu and Diaz are both directed to ordinal prediction using neural network probability outputs. Arcadu expressly teaches that MCES is an ordinal severity scale and that ordinal classification models are appropriate for such severity scales (Arcadu ¶[0032]). Diaz teaches that ordinal regression systems may produce ordered probability outputs and then use either argmax or an expected value formula at inference time. A POSITA would have been motivated to apply Diaz’s expected value scoring to Arcadu’s MCES ordinal probability outputs to preserve more severity granularity before discretizing the result to the known MCES levels. The combination would have yielded the predictable result of a continuous frame level MCES severity score that is then thresholded to a discrete MCES output.
Regarding claim 6:
Arcadu teaches the limitations of claim 1 as applied above.
wherein generating the output severity score comprises: combining at least the first and second binary probabilities (Arcadu in ¶[0179] discloses first and second binary probability outputs from severity scoring networks. Specifically, Arcadu discloses training two deep neural networks, i.e., severity scoring networks (“SSNs”), where a first SSN is trained using a first binary severity scheme MCES > 1 / MCES ≤ 1, and a second SSN is trained using a second binary severity scheme MCES > 2 / MCES ≤ 2. Arcadu further discloses that the first SSN outputs, for each frame, a probability P(MCES > 1), and the second SSN outputs, for each frame, a probability corresponding to MCES > 2; Arcadu ¶[0034] also discloses combining outputs of multiple binary classifiers to obtain ordinal severity information. Specifically, Arcadu teaches that an ordinal classification model may be implemented by training multiple binary DNN classifiers, each computing a probability for a respective severity-threshold classification, and that “the probability of belonging to each of three or more severity classes … can be obtained based on the combined output” of the multiple binary classifiers. Arcadu ¶ [0038] further teaches equations for deriving ordinal severity-class probabilities from the combined binary outputs, including calculating intermediate severity-class probabilities by subtracting cumulative binary probabilities, e.g., P(class > 1) − P(class > 2) and P(class > 2) − P(class > 3).
Arcadu does not expressly state that the combined binary probabilities are used to generate a continuous severity score.
However, in a related field, Diaz is directed to ordinal regression, which it describes as a task involving ordered categories and as resembling real-valued regression and classification. In Section 1, Diaz explains that ordinal regression can be treated as “mapping the inputs to a real line” and using “boundaries between ordinal categories” to define the final class. Diaz also explains that K-rank ordinal methods use K−1 binary classifiers trained to determine whether an input has a response y > k, and that the rank prediction typically involves the “accumulation of positive responses” from those binary classifiers.
Diaz further teaches, in Section 3.3, converting ordinal probability outputs into a scalar prediction by using “a simple expected value formula like ∑K k=1 rkpk.” Diaz also states that the ordinal probability method can handle a “continuous domain,” rather than requiring hard assignment to the closest rank.
Accordingly, Arcadu teaches the binary severity probabilities and ordinal MCES probability derivation, while Diaz teaches converting ordinal probability outputs into a continuous scalar prediction using an expected value calculation.
Regarding claims 14-15: the claims limitations are similar to claims 5-6; therefore, rejected in the same manner as applied above.
Regarding claim 19:
Arcadu discloses: a method for estimating ulcerative colitis severity depicted in an endoscopic video, the method comprising: receiving the endoscopic video (Arcadu ¶[0001]: “computer-implemented methods for analysing colonoscopy videos”; “clinical assessment of inflammatory bowel diseases such as ulcerative colitis”; Arcadu ¶[0005]: “analysing raw colonoscopy videos or portions thereof using a deep neural network classifier”);
applying a Arcadu ¶ [0166] discloses applying a machine-learned severity model to individual frames. Arcadu teaches that the trained severity scoring network (“SSN”) “takes as input individual frames 3001-300n” and “produces … as output a severity class prediction for each frame that is analysed”. Arcadu ¶ [0032] also discloses an ordinal MCES neural-network model. Arcadu teaches that MCES is an ordinal severity scale where “1 being more severe than 0, 2 being more severe than 1 and 3 being more severe than 2,” and that ordinal classification models are appropriate for such severity scales” Arcadu ¶ [0182] further teaches that all frames passing quality control and their MCES annotations were used to train an SSN to output probabilities of each frame belonging to one of four classes corresponding to the four MCES levels, and that “an ordinal classification model was used as described in Cao et al. (Rank-consistent Ordinal Regression for Neural Networks)”);
Arcadu does not expressly characterize the frame level output as a regression based severity.
Diaz teaches the regression based continuous score aspect of ordinal prediction. Diaz explains that ordinal regression may be approached as “mapping the inputs to a real line” and predicting “boundaries between ordinal categories to define the final output class” (Diaz section 1). Diaz further teaches that ordinal probability outputs may be used at inference time by “a simple expected value formula like ∑K k=1 rkpk,” and that SORD can “encapsulate data from a continuous domain” (Diaz section 3.3).
wherein the first machine-learned model is trained from a set of annotated training endoscopic videos (Arcadu ¶[0005]: the DNN classifier is “trained using raw colonoscopy video data, where entire videos or segments thereof in the training data are associated with the same class label”; ¶[0033]: a predicted MCES classifier can be obtained “using ‘raw’ colonoscopy videos as both training data and subject data”; ¶[0177]: “104 raw colonoscopy videos were selected” and annotated by expert gastroenterologists; ¶[0179]: selected frames were used to train the SSNs).
wherein each of the set of annotated training endoscopic videos has a respective single label representing a maximum severity of ulcerative colitis observed with respect to a baseline severity scale comprising an ordinal set of discrete severity levels (Arcadu ¶[0005]: “entire videos or segments thereof in the training data are associated with the same class label”; ¶[0177]: MCES annotations are weak labels and “for MCES scoring, an anatomical section of colon is assigned the score that corresponds to the most severe lesions seen in the section”; ¶[0054]: “a colonoscopy video or portion thereof is commonly assigned to the most severe category (highest severity score) that has been identified in the video”. Therefore, Arcadu teaches single video/segment labels and expressly teaches that MCES scoring assigns the label corresponding to the most severe lesions seen. Arcadu ¶ [0029] discloses the baseline ordinal scale: the Mayo Clinic Endoscopic Subscore has levels MCES=0, MCES=1, MCES=2, and MCES=3, and Arcadu ¶ [0032] expressly treats MCES as an ordinal severity scale. Applicant’s specification in ¶ [0004] confirms the same conventional video level rule: “gastroenterologists attribute a single MES to a video based upon the maximum disease severity observed in the video.”);
determining a maximum frame-level severity score from the respective frame-level severity scores (Arcadu ¶ [0168] teaches highest-severity summarization from frame-level predictions. Arcadu teaches that a summarized severity class may be obtained by assigning “the highest severity class (i.e. the most severe class) that is represented amongst the discrete class assignments … for the frames of the segment above a threshold for the respective severity class”. Applicant discloses that conventional MES video scoring is based on the maximum disease severity observed in the video (Applicant Spec. ¶[0004]));
comparing the maximum frame-level severity score to a set of thresholds to select a discrete severity level from the baseline severity scale (Arcadu ¶ [01379] teaches threshold-based severity assignment. Arcadu teaches that a frame is assigned to severity classes by comparing probability outputs to thresholds, including P(MCES>1)>0.5 and P(MCES>2)>0.5. Arcadu ¶ [0181] further teaches ROC evaluation by varying thresholds between 0 and 1 in increments of 0.05. Diaz also supports thresholding a continuous ordinal prediction because Diaz explains that ordinal regression may map inputs to a real line and use boundaries between ordinal categories to define the final class (Diaz section 1);
and outputting the discrete severity level (Arcadu ¶ [0166] discloses outputting discrete severity predictions. Arcadu teaches that a discrete classification label may be obtained by applying a threshold to the probability output by the SSN, and that for multilevel classifiers a frame may be assigned to the class having the highest probability. Arcadu ¶ [0182] also teaches assigning a single predicted MCES score from the probability outputs).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Aracdu to incorporate the teachings of Diaz because Arcadu and Diaz are both directed to ordinal prediction using neural network probability outputs. Arcadu expressly teaches that MCES is an ordinal severity scale and that ordinal classification models are appropriate for such severity scales (Arcadu ¶[0032]). Diaz teaches that ordinal regression systems may produce ordered probability outputs and then use either argmax or an expected value formula at inference time. A POSITA would have been motivated to apply Diaz’s expected value scoring to Arcadu’s MCES ordinal probability outputs to preserve more severity granularity before discretizing the result to the known MCES levels. The combination would have yielded the predictable result of a continuous frame level MCES severity score that is then thresholded to a discrete MCES output.
Regarding claim 20:
wherein the regression-based machine-learned model is trained using a multi-instance learning algorithm (Arcadu ¶[0061]: assigning a summarized severity class may use a first DNN classifier “trained using multiple instances learning”; Arcadu ¶[0061] further teaches combining predictions with attention-based pooling. Also see Arcadu ¶ [0046] – ¶ [0051]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Najarian (US 20200364859) teaches: a processor to generate scores indicating severity levels of a disease for a plurality of the informative frames, estimate locations of the plurality of the informative frames in the colon, and generate an output indicating a distribution of the scores over one or more segments of the colon by combining the scores generated for the plurality of the informative frames and the estimated locations of the plurality of the informative frames in the colon.
Kumar (US 20120316421) teaches: processing each of the endoscopic images with the image processing system to determine whether at least one attribute of interest is present in each image that satisfies a predetermined criterion, and classifying the endoscopic images into a reduced set of images each of which contains at least one attribute of interest and a remainder set of images each of which is free from the attribute, and registration selection may be treated as an ordinal regression problem.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WASSIM MAHROUKA whose telephone number is (571)272-2945. The examiner can normally be reached Monday-Thursday 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WASSIM MAHROUKA/Primary Examiner, Art Unit 2665
Prosecution Insights