DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending.
Response to Arguments (Election of Species Requirement)
Applicant's election with traverse of Species 1 (claim 2, together with generic claims 1 and 16) in the reply filed on 5/1/2026 is acknowledged. Applicant’s argument regarding the Election of Species Requirement mailed on 13/3/2026 is found persuasive. Therefore, the requirement for species election is withdrawn, and all claims, i.e., claims 1-20, will be examined in this office action.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claim(s) 1-3, 6-9, 13 and 15-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al (US20220207718A1) in view of Li (US20220208355A1).
Regarding claims 1 and 16, Wang teaches a method of training a model to generate predictions using medical images, the method comprising:
(Wang, "Knowledge distillation method for fracture detection includes obtaining medical images", [abstract]; "a neural network may be trained to produce a probability map that indicates the location of the detected fractures.", [0025]; a method of training a model (neural network) to generate predictions (probability maps indicating locations) using medical images)
receiving a plurality of medical imaging data
(Wang, "obtaining medical images including region-level labeled images, image-level diagnostic positive images, and image-level diagnostic negative images, in chest X-rays", [0004]; receiving a plurality of medical imaging data)
wherein the plurality of medical imaging data includes a first set of medical imaging data comprising frame-level annotations and a second set of medical imaging data comprising weakly-labeled data;
(Wang, "Region-level labels may be manually annotated by experts and are more costly to obtain. For example, image-level diagnostic positive CXRs may be annotated by experts to provide region-level labels, e.g., in a form of bounding-box.", [0020]; "Image-level labels may be obtained efficiently at a large scale, e.g., by mining a hospital's image archive and clinical records. In one example, image-level labels may be obtained by finding matching diagnosis code and/or keyword in the clinical records. The image-level labels may include positive labels (e.g., for positive diagnosis) and negative labels (e.g., for negative diagnosis).", [0019]; a first set of imaging data comprising region-level annotations (expert bounding-boxes) and a second set comprising weakly-labeled data (image-level labels derived from clinical records); Li, "The output of each 1D dilated convolution time t is the temporal feature convolved with the frame of time t and the earlier time in the previous layer.", [0101]; medical images comprise temporal frames)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to apply Wang's region-level annotations and weakly-labeled data to Li's medical image frames in order to result in a first set comprising frame-level annotations and a second set comprising weakly-labeled data. The combination of Wang and Li also teaches other enhanced capabilities.
The combination of Wang and Li further teaches:
generating a training dataset, wherein the training dataset comprises frame-level ground truth data; and
(Wang, "For example, for region-level labeled images (R) in CXRs, ground-truths (GT) masks are generated by assigning “one” (“1”) to the pixels within the bounding-boxes (e.g., as region-level labels) and assigning “zero” (“0”) elsewhere.", [0027]; "A teacher model is employed to produce pseudo ground-truths (GTs) on the image-level diagnostic positive images for supervising training of a student model", [0004]; Li, "frame of time t", [0101]; COMMENTS: Wang teaches generating a training dataset comprising ground truth data (GT masks from the labels and pseudo GTs from the teacher model). Combined with Li 's teaching of medical image frames, the generated dataset comprises frame-level ground truth data)
training, using the generated training dataset, a model to generate predictions based on new medical imaging data,
(Wang, "performing a semi-supervised training process on the image-level diagnostic positive images using the pre-trained weights. A teacher model is employed to produce pseudo ground-truths (GTs) on the image-level diagnostic positive images for supervising training of a student model", [0004]; "The trained model was evaluated on the validation set after every training epoch, and the one with the highest validation an area under receiver operating characteristic curve (AUROC) is selected as the best model for inference.", [0056]; training a model (student model) using the generated training dataset (pseudo GTs and pre-trained weights) to generate predictions (best model selected for inference) based on new medical imaging data)
wherein the generated predictions include frame-level feature localizations.
(Wang, "a neural network may be trained to produce a probability map that indicates the location of the detected fractures.", [0025]; "produce sharpened pseudo GTs to provide positive detection responses on the image-level diagnostic positive images.", [0004]; "a neural network may be trained to produce a probability map that indicates the location of the detected fractures."; "ƒθ(x) denotes probability map output of the neural network parameterized by θ.", [0029]; the generated predictions include frame-level feature localizations (probability map output indicating locations of detected fractures, which are feature localizations at the frame/image level; Li, "frame of time t", [0101]; Wang teaches the generated predictions include feature localizations (probability maps indicating the location of fractures / positive detection responses); combined with Li 's context of medical image frames, this teaches frame-level feature localizations)
Regarding claim 2, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the weakly-labeled data comprises at least one of unlabeled data or video-level labeled data.
(Wang, "the baseline knowledge distillation methods treat the image-level positive images as unlabeled data.", [0067]; the image-level positive images (which serve as the weakly-labeled data in the parent claim) are treated as unlabeled data; Li, " Weakly-supervised learning uses input having a corresponding weak label. The weak label means it provides less information compared with the label that would be used in supervised learning. Weakly-supervised learning algorithms can involve mapping the input to a more specific label. For example, in a pixel-level object segmentation task, the provided weak label is bounding-boxes of desirable objects, and in this example the weak label is not so accurate as to indicate the class of every pixel, but it indicates the object location and size. Thus, the weakly-supervised learning algorithms try to use weak labels to exploit the input inherent features, thereby accomplishing a desired task ", [0272]; no all of the pixels in the bounding box are labeled with a class)
Regarding claim 3, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the generated predictions include video-level annotations.
(Wang, "Both fracture classification and localization performances were evaluated using the disclosed fraction detection model. The widely used classification metric AUROC (area under receiver operating characteristic curve) was used to assess classification performance. For object detection, the maximum classification score of all predicted bounding-boxes is taken as the classification score"; [0059]; "image-level CXRs (e.g., labeled/non-labeled) and region-level labeled CXRs may be used to develop a fracture detection model, to identify classification and localization of fractures (including e.g., rib and clavicle fractures, and spine bone fractures) based on CXRs", [0017]; Wang's model generates image-level (i.e., the CXR as a whole) classification predictions, specifically, a predicted label for the whole image (positive/negative, fracture type), which constitutes a "video-level annotation" in the analogous sense of a whole-study or whole-image-level prediction; the maximum classification score of all predicted bounding boxes is used as the classification score for the entire image, representing an image/study-level output)
Regarding claim 6, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 1,
wherein the new medical imaging data comprises an ultrasound video loop, and
wherein the plurality of medical imaging data comprises ultrasound videos, ultrasound frames, or both.
(Li, "not limited to MR scanning, and can readily be adapted to other imaging modalities that have sufficient spatial resolution for diagnostic imaging, including Ultrasound... Medical sonographic examination is an ultrasound-based diagnostic medical imaging technique", [0259]; adapt the models to various available medical imaging modalities; the plurality of medical imaging data comprising ultrasound frames or videos)
Regarding claim 7, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the model is trained to generate a bounding box indicating a location of a target feature or delineate the location of the target feature.
(Wang, "For object detection, the maximum classification score of all predicted bounding-boxes is taken as the classification score.", [0059]; the detection model is trained to predict/generate bounding boxes that indicate the location of the target features (fracture sites))
Regarding claims 8 and 17, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 1:
wherein generating the training dataset includes pre-training a teacher model, using the first set of the medical imaging data comprising the frame-level annotations, to generate pseudo-labels; and
(Wang, “For example, as shown in FIG. 2, the teacher model is employed to produce pseudo ground-truths (GTs) on the image-level diagnostic positive images (P) for supervising training of the student model ... The teacher and student models share the same network architecture (including, e.g., ResNet-50 with feature pyramid network (FPN)), and are both initialized using the pre-trained weights obtained from the exemplary supervised learning step at 120 of FIG. 1.", [0033]; the teacher model is initialized using pre-trained weights obtained from supervised pre-training on region-level (frame-level) data, and this pre-trained teacher model is then used to generate pseudo-labels (pseudo GTs))
wherein training the model includes jointly training the teacher model and a student model using the second set of the medical imaging data comprising the weakly-labeled data, wherein the generated pseudo-labels are used as a ground truth for training the student model.
(Wang, "In one embodiment, the student model is trained via back propagation and iteratively update the teacher model using the exponential moving average (EMA) of the student model weights during training, as also shown in FIG. 2.", [0034]; "For example, as shown in FIG. 2, the teacher model is employed to produce pseudo ground-truths (GTs) on the image-level diagnostic positive images (P) for supervising training of the student model. The student model learns from the pseudo GTs produced from the teacher model on the image-level diagnostic positive images (P).", [0033]; jointly training the student and teacher models using the image-level positive images (weakly-labeled data), where the student model uses the generated pseudo-labels as ground truth for its learning)
Regarding claims 9 and 18, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 8, further comprising:
transferring weights from the trained student model to the trained teacher model based on a transferring rate determined using an exponential moving average function.
(Wang, "the student model is trained via back propagation and iteratively update the teacher model using the exponential moving average (EMA) of the student model weights during training", [0034]; "The weights of the teacher model are updated as follows... α is a smoothing coefficient to control the pace of knowledge update.", eq. (2), [0035]; transferring weights to the teacher model based on an exponential moving average (EMA) function and a transferring rate/smoothing coefficient)
Regarding claim 13, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, further comprising:
applying the trained model to the new medical imaging data to generate the predictions.
(Wang, "All images were padded to square and resized to 1024×1024 for network training and inference. Rotation, horizontal flipping, intensity and contrast jittering were randomly performed to augment the training data. The trained model was evaluated on the validation set after every training epoch, and the one with the highest validation an area under receiver operating characteristic curve (AUROC) is selected as the best model for inference.", [0056]; applying the trained model to new data for inference, which intrinsically constitutes generating predictions based on the input)
Regarding claim 15, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination further teaches the method of claim 1, wherein the model includes a baseline segmentation model or a baseline detection model.
(Wang, "a neural network may be trained to produce a probability map that indicates the location of the detected fractures. Since the shape and scale of fractures can vary significantly, feature pyramid network (FPN) with a ResNet-50 backbone may be employed", [0025]; " The teacher and student models share the same network architecture (including, e.g., ResNet-50 with feature pyramid network (FPN))", [0033]; D2, ¶0206: " The detector is the first time combined with the regular GAN in an end-to-end framework for tumor detection. FIG. 22 shows the architecture of the detector, which is a customized Faster R-CNN", [0206]; using a detection model (ResNet-50 + FPN) as the underlying network architecture, which is a baseline detection model; Li, "WSTS employs a weakly-supervised teacher-student framework (TCH-ST).", [0279]; "The Student Module employs a Student DDRL (SDDRL) and a Student DenseUNet (SDUNet) to learn tumor segmentation under the guidance of the Teacher Module in the non-enhanced image", [0280]; a Faster R-CNN-based detector as well as a segmentation component (DenseUNet))
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al (US20220207718A1) in view of Li (US20220208355A1) and further in view of Liu et al (Unbiased Teacher, 2021) and Parameswaran et al (US20230334291A1).
Regarding claim 4, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination does not expressly disclose but Liu and Parameswaran teaches the method of claim 3, wherein the model is trained to determine a category for the new medical imaging data, and wherein the category is selected from at least two categories.
(Liu, "For each labeled image x^s, the annotations y^s contain locations, sizes, and object categories of all bounding boxes.", [Section 3], p3; Parameswaran, "the image receives only one label indicating the class of the object of interest", [0004]; Liu and Parameswaran teach training models to classify data into defined categories or classes. Applying this known classification method to determine a category from at least two options for medical imaging data yields a predictable result)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporating Liu and Parameswaran into Wang and Li in order to enable the medical imaging model to accurately classify new imaging data into predefined target categories, thereby improving the diagnostic or analytical utility of the generated predictions. The combination of Wang, Li, Liu and Parameswaran also teaches other enhanced capabilities.
Claim(s) 5, 12, 14 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al (US20220207718A1) in view of Li (US20220208355A1) and further in view of Parameswaran et al (US20230334291A1).
Regarding claim 5, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination of Wang, Li and Parameswaran further teaches the method of claim 3, wherein the video-level annotations are generated using a frame-to-video feature encoder.
(Parameswaran, "By selecting one of the frames of the snippet for labeling as shown at step 210, the location of the object in the remaining frames of the snippet can be automatically calculated and the object labeled by the tracking process 270.", [0076]; automatically generating labels for an entire video snippet (video-level annotations) by calculating the locations based on an initial frame using a tracking process)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporating Parameswaran into Wang and Li in order to enable the medical imaging model to implement this tracking calculation using a frame-to-video feature encoder to achieve the predictable result of propagating frame-level features across video sequences. The combination of Wang, Li and Parameswaran also teaches other enhanced capabilities.
Regarding claims 12 and 20, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination of Wang, Li and Parameswaran further teaches the method of claim 8, further comprising:
evaluating quality of frame-level pseudo-labels included in the generated pseudo-labels based on video-level ground truth annotations or video-level pseudo-labels; and
(Parameswaran, "In order to track multiple objects in the scene, a set of these detectors is used to individually track the bounding boxes and drop the detections when the detection score is lower than some threshold.", [0076]; tracking bounding boxes (pseudo-labels) across a video scene and evaluating their detection scores. Evaluating frame-level labels based on the broader video-level context and tracking sequence to determine quality is a predictable application of Parameswaran's video tracking framework to ensure labeling accuracy)
filtering the frame-level pseudo-labels based on the quality.
(Parameswaran, "drop the detections when the detection score is lower than some threshold.", [0076]; filtering out (dropping) detections based on their evaluated quality (detection score))
Regarding claim 14, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination of Wang, Li and Parameswaran further teaches the method of claim 1, further comprising:
evaluating an accuracy of the trained model using a testing dataset; and
(Parameswaran, "The fourth step is to evaluate the model using images that weren't used in the training phase. By doing so, the accuracy of the training model can be tested.", [0005]; Parameswaran teaches the fundamental step of evaluating the accuracy of a trained machine learning model against a separate validation/testing dataset)
retraining the trained model using a different training dataset when the accuracy does not exceed a threshold accuracy.
(Parameswaran, "The model is then retrained using all of the labeled data, yielding an improved result. This cyclic process of labeling, training and querying is continued until the model converges or the validation accuracy is deemed satisfactory by the user", [0099]; iteratively retraining the model on an updated/expanded (different) training dataset if the evaluated validation accuracy does not meet a satisfactory threshold, rendering this limitation an obvious application of known model optimization techniques)
Claim(s) 10-11 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al (US20220207718A1) in view of Li (US20220208355A1) and further in view of Liu et al (Unbiased Teacher, 2021).
Regarding claims 10 and 19, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination of Wang, Li and Liu further teaches the method of claim 9, wherein the transferring rate is adjusted based on evaluating performance of the student model using validation data.
(Liu, "We also evaluate the model using various EMA rate a from 0.5 to 0.9999 and present the mAP result of the Teacher model in Figure 10", [Section A.2.3], p15; Figure 10, "Validation AP on the Teacher model with various MMA rates α.", Fig. 10 caption, p15; evaluating the model's validation performance (Validation AP) across different EMA rates (transferring rates) to find the optimal rate. Doing so would allow dynamically adjusting the transferring rate during training based on this validation performance evaluation to optimize accuracy)
Regarding claim 11, the combination of Wang and Li further teaches its/their respective base claim(s).
The combination of Wang, Li and Liu further teaches the method of claim 8, wherein a frame included in the weakly-labeled data is weakly augmented for training of the teacher model and the frame is strongly augmented for training of the student model.
(Liu, "we thus use the strongly augmented images as as input of the Student, but we use the weakly augmented images as input of the Teacher to provide reliable pseudo-labels.", [Section 3.2], p4-p5; applying weak augmentation to the inputs of the Teacher model and strong augmentation to the inputs of the Student model within a semi-supervised architecture to ensure reliable pseudo-labels during training)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANXUN YANG whose telephone number is (571)272-9874. The examiner can normally be reached on MON-FRI: 8AM-5PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272- 1000.
/JIANXUN YANG/
Primary Examiner, Art Unit 2662 6/27/2026
Prosecution Insights