Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 4/16/2024 and 3/9/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant's arguments filed 3/31/2026 have been fully considered but they are not persuasive. The applicant argues that the prior art of record Wu fails to teach the step of “determining a second cross entropy loss corresponding to the sample image patch based on a difference between a patch label of the sample image patch and the corresponding patch analysis result of the sample image patch”. After an updated search, new prior art was found to reject the amended claims. Even though the emphasis was placed on the training of image recognition model using three types of losses, the solution to the claimed invention is still found in prior arts Sharma et. al. in combination with Wu, Ping, and Jewsbury. The second cross entropy loss is discussed in the prior art Sharma et. al. Thus, all of the claims have been rejected on the basis of the prior art of record.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3-4. 6-8, 10, 12-13, 15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Sharma, Yash et al. “Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification.” International Conference on Medical Imaging with Deep Learning (2021).
Regarding claim 1, Sharma et. al. discloses a method for training an image recognition model, performed by a computer device, the method comprising (Sharma et. al. Figure 1): obtaining a sample image and a corresponding sample label; obtaining a sample image patch bag of sample image patches corresponding to the sample image (Sharma et. al., Fig.1(a), section 3.1 “For digital pathology classification problems, WSIs (W) of patients are available along with their disease labels.”), the sample patch bag having a bag label corresponding to the sample label of the sample image (Sharma et. al. Fig.1(b)-(c), section 3.1 “Hence, using the Otsu thresholding approach and sliding window approach, patches containing substantial tissue area (>50%) of desirable size are extracted. Given a WSI W (bag) with label y, we extract w1, w2, w3, …, wn patches (instances) from it for training.”); determining a relative entropy loss corresponding to the sample patch bag based on a difference between an attention distribution predicted based on image content in the sample patch bag and an expected distribution corresponding to the bag label of the sample patch bag, the attention distribution being a distribution obtained by predicting image content in the sample patch bag, and the expected distribution being a distribution of the sample patch bag indicated by the bag label (Sharma et. al. Fig. 1(d)-(f), section 3.2 “We used the weighted-average aggregation approach proposed in Ilse et. al. (2018) for aggregating the patch-level representation to obtain WSI-level representations.”, section 3.4 “Using the aggregated representation of the WSI and representation of patches (instances), end-to-end training is performed using cross-entropy and KL-divergence loss. Along with WSI and patch cross-entropy loss, for each cluster, KL-divergence loss between the patches’ attention weight and a uniform distribution is included.”); performing feature analysis on the sample patch bag by using an image recognition model to obtain a bag feature analysis result of the sample patch bag; determining a first cross entropy loss corresponding to the sample patch bag based on a difference between the bag label and the corresponding bag feature analysis result of the sample patch bag indicating a recognition result of the image content in the sample patch bag; for each of the sample image patches in the sample patch bag: performing feature analysis on the sample image patch by using the image recognition model to obtain a patch feature analysis result of the sample image patch; determining a second cross entropy loss corresponding to the sample image patch based on a difference between a patch label of the sample image patch and the corresponding patch analysis result of the sample image patch (Sharma et. al. Fig.1 (g), section 3.4 “…the instance representation are passed through Gy’: h [Wingdings font/0xE0] y’ to obtain patches prediction probability. Instance loss is included with weak supervision assumption. Along with WSI and patch cross-entropy loss, for each cluster, KL-divergence loss between the patches’ attention weight and a uniform distribution is included.”); and training the image recognition model based on the relative entropy loss, the first cross entropy loss, and the plurality of second cross entropy losses, the trained image recognition model being configured to recognize image content in an image (Sharma et.al. loss L(Gy, Gy’, Ga, Ge) in section 3.4).
Regarding claim 8, which is a computer device, comprising a processor and a memory, the memory having at least one program stored therein that, when executed by the processor, causes the computer device to implement a method for training an image recognition model of claim 1, which the rejection analysis is incorporated herein.
Regarding claim 15, which is a non-transitory computer-readable storage medium, having at least one program stored thereon that, when executed by a processor of a computer device, causes the computer device to implement a method for training an image recognition model of claim 1, which the rejection analysis is incorporated herein.
Regarding claim 3, Sharma et. al. discloses the method according to claim 1, wherein the determining the first cross entropy loss corresponding to the sample patch bag based on a difference between the bag label and the corresponding bag feature analysis result of the sample patch bag indicating the recognition result of the image content in the sample patch bag comprises: performing first feature processing on the bag feature by a first fully connected layer in the image recognition model to obtain a first fully connected feature; performing second feature processing on the first fully connected feature and the attention distribution corresponding to the sample patch bag by a second fully connected layer in the image recognition model, to obtain a second fully connected feature as a recognition result; an determining, based on a difference between the second fully connected feature and the bag label, the first cross entropy loss corresponding to the sample patch bag (Sharma et. al. Fig. 1 (d)-(g), section 3.2 “weighted-average aggregation approach…to obtain WSI-level representations.” And section 3.4 “KL-divergence loss between the patches’ attention weight and a uniform distribution is included. The aggregated representation is passed through Gy: y[Wingdings font/0xE0] z to obtain WSI prediction probability”).
Regarding claim 4, Sharma et. al. discloses the method according to claim 1, wherein the determining the second cross entropy loss corresponding to the sample image patch based on a difference between the patch label of the sample image patch and the corresponding patch feature analysis result of the sample image patch comprises: performing feature extraction on the sample image patch by using the image recognition model to obtain a patch feature; performing first feature processing on the patch feature by the first fully connected layer in the image recognition model to obtain a third fully connected feature; performing second feature processing on the third fully connected feature by a second fully connected layer in the image recognition model, to obtain a fourth fully connected feature as a patch analysis result; and determining, based on a difference between the fourth fully connected feature and the patch label of the sample image patch, the second cross entropy loss corresponding to the sample image patch, the patch labels being label corresponding to the sample image patch being determined based on the sample label (Sharma et. al. section 3.4 “Along with WSI and patch cross-entropy loss, for each cluster, KL-divergence loss between the patches’ attention weight and a uniform distribution is included. The inclusion of KL-divergence loss regularizes the same-cluster patches’ attention distribution and allows the attention module to weight all the positive class patches uniformly.”).
Regarding claim 6, Sharma et. al. discloses the method according to claim 5, wherein the method further comprises: using, in response to the sample image patches in the sample patch bag belonging to a same sample image, a sample label corresponding to the sample image as a bag label corresponding to the sample patch bag; and determining, in response to the sample image patches in the sample patch bag belonging to different sample images, the bag label corresponding to the sample patch bag based on the patch labels corresponding to the sample image patches (Sharma et. al. section 3.4 “The aggregated representation is passed through Gy: z[Wingdings font/0xE0]y to obtain WSI prediction probability).
Regarding claim 7, Sharma et. al. discloses the method according to claim 1, wherein the training the image recognition model based on the relative entropy loss, the first cross entropy loss, and the plurality of second cross entropy losses comprises: performing weighted fusion on the relative entropy loss, the first cross entropy loss, and the plurality of second cross entropy losses to obtain a total loss value; and training the image recognition model based on the total loss value (Sharma et. al. section 3.4 “The instance representation are passed through Gy’: h[Wingdings font/0xE0]y’ to obtain patches prediction probability…Instance loss is included with weak supervision assumption”).
Regarding claim 10, Sharma et. al. discloses the computer device according to claim 8 wherein the determining the first cross entropy loss corresponding to the sample patch bag based on a difference between the bag label and the corresponding bag feature analysis result of the sample patch bag indicating the recognition result of the image content in the sample patch bag comprises: performing first feature processing on the bag feature by a first fully connected layer in the image recognition model to obtain a first fully connected feature; performing second feature processing on the first fully connected feature and the attention distribution corresponding to the sample patch bag by a second fully connected layer in the image recognition model, to obtain a second fully connected feature as a recognition result; and determining, based on a difference between the second fully connected feature and the bag label, the first cross entropy loss corresponding to the sample patch bag (Sharma et. al. section 3.4, loss L(Gy, Gy’, Ga, Ge)).
Regarding claim 12, Sharma et. al. discloses the computer device according to claim 8, wherein the obtaining a sample image patch bag of sample image patches corresponding to the sample image comprises: segmenting an image region of the sample image to obtain the sample image patches; and allocating sample image patches belonging to a same sample image to a same bag to obtain the sample patch bag (Sharma et. al. Fig. 1 (d)-(g), section 3.2 “weighted-average aggregation approach…to obtain WSI-level representations.” And section 3.4 “KL-divergence loss between the patches’ attention weight and a uniform distribution is included. The aggregated representation is passed through Gy: y[Wingdings font/0xE0] z to obtain WSI prediction probability”).
Regarding claim 13, Sharma et. al. discloses the computer device according to claim 12, wherein the method further comprises: using, in response to the sample image patches in the sample patch bag belonging to a same sample image, a sample label corresponding to the sample image as a bag label corresponding to the sample patch bag; and determining, in response to the sample image patches in the sample patch bag belonging to different sample images, the bag label corresponding to the sample patch bag based on the patch labels corresponding to the sample image patches (Sharma et. al. Fig. 1 (d)-(g), section 3.2 “weighted-average aggregation approach…to obtain WSI-level representations.” And section 3.4 “KL-divergence loss between the patches’ attention weight and a uniform distribution is included. The aggregated representation is passed through Gy: y[Wingdings font/0xE0] z to obtain WSI prediction probability”).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Sharma, Yash et al. “Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification.” International Conference on Medical Imaging with Deep Learning (2021). in view of Jewsbury, Robert et al. “A QuadTree Image Representation for Computational Pathology.” 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) (2021): 648-656. (Year: 2021).
Regarding claim 5, Sharma et. al. discloses the method according to claim 1. However, Sharma et. al. fails to disclose wherein the obtaining a sample image patch bag of sample image patches corresponding to the sample image comprises: segmenting an image region of the sample image to obtain the sample image patches; and allocating sample image patches belonging to a same sample image to a same bag to obtain the sample patch bag.
Jewsbury et. al. teaches wherein the obtaining a sample image patch bag of sample image patches corresponding to the sample image comprises: segmenting an image region of the sample image to obtain the sample image patches; and allocating sample image patches belonging to a same sample image to a same bag to obtain the sample patch bag (Jewsbury et. al. 2.2 MIL framework, where generated collection of down-sampled image regions are given labels either positive or negative based on categories of features).
PNG
media_image1.png
533
1142
media_image1.png
Greyscale
Separating the sample images into different categories based on features that are extracted are critical to the claimed invention. This defines the solution of the problem. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Sharma et. al. and Jewsbury so that this solution is implemented fully.
Regarding claim 19, Sharma et. al. discloses the non-transitory computer-readable storage medium according to claim 15. However, Sharma et. al. fails to disclose wherein the obtaining a sample image patch bag of sample image patches corresponding to the sample image comprises: segmenting an image region of the sample image to obtain the sample image patches; and allocating sample image patches belonging to a same sample image to a same bag to obtain the sample patch bag.
Jewsbury et. al. teaches wherein the obtaining a sample image patch bag of sample image patches corresponding to the sample image comprises: segmenting an image region of the sample image to obtain the sample image patches; and allocating sample image patches belonging to a same sample image to a same bag to obtain the sample patch bag (Jewsbury et. al. 2.2 MIL framework, where generated collection of down-sampled image regions are given labels either positive or negative based on categories of features).
PNG
media_image1.png
533
1142
media_image1.png
Greyscale
Separating the sample images into different categories based on features that are extracted are critical to the claimed invention. This defines the solution of the problem. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Sharma et. al. and Jewsbury so that this solution is implemented fully.
Claim(s) 11, 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Sharma, Yash et al. “Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification.” International Conference on Medical Imaging with Deep Learning (2021). In view of Wu (Chinese Patent CN 111612733 A).
Regarding claim 17, Sharma et. al. discloses the non-transitory computer-readable storage medium according to claim 15, wherein the determining the first cross entropy loss corresponding to the sample patch bag based on a difference between the bag label and the corresponding bag feature analysis result of the sample patch bag indicating the recognition result of the image content in the sample patch bag comprises. However, Sharma et. al. fails to disclose performing first feature processing on the bag feature by a first fully connected layer in the image recognition model to obtain a first fully connected feature; performing second feature processing on the first fully connected feature and the attention distribution corresponding to the sample patch bag by a second fully connected layer in the image recognition model, to obtain a second fully connected feature as a recognition result; and determining, based on a difference between the second fully connected feature and the bag label, the first cross entropy loss corresponding to the sample patch bag.
Wu further teaches performing first feature processing on the bag feature by a first fully connected layer in the image recognition model to obtain a first fully connected feature; performing second feature processing on the first fully connected feature and the attention distribution corresponding to the sample patch bag by a second fully connected layer in the image recognition model, to obtain a second fully connected feature as a recognition result; and determining, based on a difference between the second fully connected feature and the bag label, the first cross entropy loss corresponding to the sample patch bag (Wu [0014]-[0019] where the fifth part of the convolutional layer is a fully-connected layer comprising 32 neurons, and the loss function of the convolutional neural network adopts cross entropy loss for each ith data sample). The use of fully connected layers in the convolutional neural network is important because it provides advantages such as integration of complex features, flexibility, and also simplicity in the learning framework. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Sharma et. al. and Wu such that the method fully utilizes the fully connected layers of the neural network.
PNG
media_image2.png
588
760
media_image2.png
Greyscale
Regarding claim 11 and 18, Sharma et. al. discloses the computer device according to claim 8, and the non-transitory computer-readable storage medium according to claim 15. However, Sharma et. al. fails to disclose wherein the determining the second cross entropy loss corresponding to the sample image patch based on a difference between the patch label of the sample image patch and the corresponding patch feature analysis result of the sample image patch comprises: performing feature extraction on the sample image patch by using the image recognition model to obtain a patch feature; performing first feature processing on the patch feature by the first fully connected layer in the image recognition model to obtain a third fully connected feature; performing second feature processing on the third fully connected feature by a second fully connected layer in the image recognition model, to obtain a fourth fully connected feature as a patch analysis result; and determining, based on a difference between the fourth fully connected feature and the patch label of the sample image patch, the second cross entropy loss corresponding to the sample image patch, the patch labels being label corresponding to the sample image patch being determined based on the sample label.
Wu teaches wherein the determining the second cross entropy loss corresponding to the sample image patch based on a difference between the patch label of the sample image patch and the corresponding patch feature analysis result of the sample image patch comprises: performing feature extraction on the sample image patch by using the image recognition model to obtain a patch feature; performing first feature processing on the patch feature by the first fully connected layer in the image recognition model to obtain a third fully connected feature; performing second feature processing on the third fully connected feature by a second fully connected layer in the image recognition model, to obtain a fourth fully connected feature as a patch analysis result; and determining, based on a difference between the fourth fully connected feature and the patch label of the sample image patch, the second cross entropy loss corresponding to the sample image patch, the patch labels being label corresponding to the sample image patch being determined based on the sample label (Wu [0014]-[0019] where the fifth part of the convolutional layer is a fully-connected layer comprising 32 neurons, and the loss function of the convolutional neural network adopts cross entropy loss for each ith data sample).
The use of fully connected layers in the convolutional neural network is important because it provides advantages such as integration of complex features, flexibility, and also simplicity in the learning framework. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Sharma et. al. and Wu such that the method fully utilizes the fully connected layers of the neural network.
Claim(s) 14 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sharma, Yash et al. “Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification.” International Conference on Medical Imaging with Deep Learning (2021). In view of Ping (Chinese Patent CN 113920109 A).
Regarding claim 14 and 20, Sharma et. al. discloses the computer device according to claim 8, and the non-transitory computer-readable storage medium according to claim 15. However, Sharma et. al. fails to disclose wherein the training the image recognition model based on the relative entropy loss, the first cross entropy loss, and the plurality of second cross entropy losses comprises: performing weighted fusion on the relative entropy loss, the first cross entropy loss, and the plurality of second cross entropy losses to obtain a total loss value; and training the image recognition model based on the total loss value.
Ping teaches wherein the training the image recognition model based on the relative entropy loss, the first cross entropy loss, and the plurality of second cross entropy losses comprises: performing weighted fusion on the relative entropy loss, the first cross entropy loss, and the plurality of second cross entropy losses to obtain a total loss value; and training the image recognition model based on the total loss value (Ping [0048]-[0052] the weighted cross entropy loss function trains the deep learning model).
PNG
media_image3.png
614
1656
media_image3.png
Greyscale
Applying the weighted cross entropy loss function improves the identification accuracy of the medical image identification model. Thus, it would have been obvious to one skilled in the art prior to the effective filing date of the claimed invention to have combined the teachings of Sharma et. al. and Ping so that the weighted cross entropy loss function is applied to the training model.
Conclusion
Response to Amendment
Examiner acknowledges the amendments made to the specification so that the objections to informalities are overcome. However, the prior art of record previously used to reject the claims in the previous non-final rejection are still applicable to rejecting the amended claims for the reasons stated above.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA YIFANG LIN whose telephone number is (571)272-6435. The examiner can normally be reached M-F 7:00am-6:15pm, with optional day off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at 571-272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSICA YIFANG LIN/Examiner, Art Unit 2668 April 17, 2026
/VU LE/Supervisory Patent Examiner, Art Unit 2668
Prosecution Insights