DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 01/26/2024 and 05/23/2024 are in compliance with the provisions of 37 CFR 1.97 and have been considered by the examiner.
Claim Objections
Claim 1 is objected to because of the following informalities: in claim 1 line 12, “a trained machine learning model” should read “a trained machine learning model, wherein the trained machine learning model”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 9-10 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Azizi et al. (NPL "Big Self-Supervised Models Advance Medical Image Classification").
Regarding claim 9, Azizi discloses a computer-implemented method comprising: receiving a first trained machine learning model, wherein the first trained machine learning model is configured to receive an image as an input (Azizi Figure 1: “Our approach comprises three steps: (1) Self-supervised pretraining on unlabeled ImageNet using SimCLR”) and to output, based on the input image, an output vector that represents an embedding of the input image into a first multi- dimensional embedding space (Azizi page 3462, left-hand column [LHC], first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); generating a second trained machine learning model by using a generic medical training data set (Azizi Figure 1: “(2) Additional self-supervised pretraining using unlabeled medical images. If multiple images of each medical condition are available, a novel Multi-Instance Contrastive Learning (MICLe) is used to construct more informative positive pairs based on different images”) to further train the first trained machine learning model (Azizi page 3462, right-hand column [RHC], first full paragraph: “For MICLe pretraining we initialize our model using SimCLR pretrained weights”), wherein the second trained machine learning model is configured to receive an image as an input and to output, based on the input image, an output vector that represents an embedding of the input image into a second multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); and using a specific medical training data set and the second trained machine learning model (Azizi page 3462, RHC, first paragraph under 4.3. Fine-tuning protocol: “We train the model end-to-end during fine-tuning using the weights of the pretrained network as initialization for the downstream supervised task dataset”), generating a third trained machine learning model (Azizi Figure 1: “(3) Supervised fine-tuning on labeled medical images”), wherein the specific medical training data set includes a plurality of medical diagnostic images that are associated with a specific condition or diagnosis and a plurality of diagnostic labels associated therewith (Azizi Figure 1: the labeled dermatology images and/or labeled chest x-rays; see also pages 3461-3462 that describe the dermatology dataset and chest x-ray dataset), wherein the third trained machine learning model is configured to receive an image as an input and to output, based on the input image, an output that is representative of a property or presence of the specific condition or diagnosis (Azizi pages 3461-3462, paragraphs under Dermatology and Chest X-rays: “The ground truth labels were…differential diagnosis of skin conditions in each case” and “We train the model to predict the five pathologies”). However, Azizi in the main embodiment fails to disclose the generic medical training data set includes a plurality of medical diagnostic images and a plurality of diagnostic labels associated therewith. Nonetheless, in one embodiment, Azizi discloses the generic medical training data set includes a plurality of medical diagnostic images and a plurality of diagnostic labels associated therewith (Azizi page 3464, LHC, paragraphs under 5.3. Comparison with supervised transfer learning; Tables 3 and 4: the supervised pretraining baselines use labeled dermatology images and/or labeled chest x-rays). A person of ordinary skill has good reason to pursue the known options within his or her technical grasp. Azizi acknowledges using supervised pretraining is a common pretraining approach that is ubiquitous in medical image analysis (Azizi page 3458, LHC, last paragraph). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi in the main embodiment to incorporate supervised pretraining to lead to the anticipated success of improving transfer performance on downstream tasks (Azizi page 3464, LHC, last paragraph).
Regarding claim 10, Azizi discloses the computer-implemented method of claim 9, wherein generating the third trained machine learning model comprises using the specific medical training data set to further train the second trained machine learning model, thereby generating the third trained machine learning model from the second trained machine learning model (Azizi page 3462, RHC, first paragraph under 4.3. Fine-tuning protocol: “We train the model end-to-end during fine-tuning using the weights of the pretrained network as initialization for the downstream supervised task dataset”), and wherein the third trained machine learning model is configured to receive an image as an input and to output, based on the input image, an output vector that represents an embedding of the input image into a third multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”).
Regarding claim 17, Azizi discloses the computer-implemented method of claim 9, wherein the diagnostic labels of the plurality of diagnostic labels indicate whether their associated medical diagnostic images are normal or abnormal (Azizi pages 3461-3462, paragraphs under Dermatology and Chest X-rays: “The ground truth labels were…differential diagnosis of skin conditions in each case” and “We train the model to predict the five pathologies”).
Regarding claim 18, Azizi discloses the computer-implemented method of claim 17, further comprising: generating the plurality of diagnostic labels based on medical records associated with the plurality of medical diagnostic images, wherein the medical records include free text notes (Azizi page 3462, LHC, under Chest X-rays: “The ground truth labels were automatically extracted from radiology reports and correspond to a label space of 14 radiological observations”).
Regarding claim 19, Azizi discloses the computer-implemented method of claim 9, wherein the first trained machine learning model comprises a machine learning model that has been trained based on a plurality of natural images (Azizi Figure 1: “(1) Self-supervised learning on unlabeled natural images”).
Claim(s) 1-2, 5, 8, 12-13, 23 and 25-27 are rejected under 35 U.S.C. 103 as being unpatentable over Azizi in view of Bruce et al. (US 2015/0193579 A1).
Regarding claim 1, Azizi discloses a computer-implemented method comprising: receiving, by a first computing system, a specific medical training data set that includes a plurality of medical diagnostic images that are associated with a specific condition or diagnosis and a plurality of diagnostic labels associated therewith (Azizi Figure 1: the labeled dermatology images and/or labeled chest x-rays; see also pages 3461-3462 that describe the dermatology dataset and chest x-ray dataset); receiving, by the first computing system, a plurality of output vectors, wherein each output vector of the plurality of output vectors represents an embedding of a respective one of the plurality of medical diagnostic images into a multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); and training, by the first computing system using the plurality of diagnostic labels and the plurality of output vectors, a trained machine learning model (Azizi page 3462, RHC, first paragraph under 4.3. Fine-tuning protocol: “We train the model end-to-end during fine-tuning using the weights of the pretrained network as initialization for the downstream supervised task dataset”), wherein the trained machine learning model is configured to receive a target output vector that represents an embedding of a target input image into the multi-dimensional embedding space and to output, based on the target output vector, an indication of at least one of a presence, degree of severity, or type of the specific condition or diagnosis (Azizi page 3463, LHC, first paragraph under 4.4. Evaluation methodology: “Our primary metrics for the dermatology task are top-1 accuracy and Area Under the Curve (AUC)…For the chest X-ray task, given the multi-label setup, we report mean AUC averaged between the predictions for the five target pathologies”). However, Azizi fails to explicitly disclose transmitting, by the first computing system to a second computing system, the plurality of medical diagnostic images; and receiving, by the first computing system from the second computing system, an output. In the related art of transmitting medical image data, Bruce discloses transmitting, by the first computing system to a second computing system, the plurality of medical diagnostic images (Bruce paragraphs 0004-0005: “a first computer software application to exchange data with a first server computer system…Each functionality is executable to provide additional data to the server computer system” wherein “The data can include medical imaging data”); and receiving, by the first computing system from the second computing system, an output (Bruce paragraph 0004: “an identifier that identifies the computer session in which the data is being exchanged between the first computer system and the first server computer system is received at the first computer system and from the first server computer system”). Distributing computing is a known technique where processing and data storage are distributed across multiple devices or systems, rather than being handled by a single central device. One of ordinary skill in the art would have been capable of applying distributing computing to enable efficient utilization of diverse resources across multiple locations. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi to incorporate the teachings of Bruce to yield the predictable result of processing medical images more efficiently and with better collaboration.
Regarding claim 2, Azizi, modified by Bruce, discloses the computer-implemented method of claim 1, further comprising: transmitting, by the first computing system to the second computing system, a target medical diagnostic image (Bruce paragraphs 0004-0005: “a first computer software application to exchange data with a first server computer system…Each functionality is executable to provide additional data to the server computer system” wherein “The data can include medical imaging data”); receiving, by the first computing system from the second computing system, an indication (Bruce paragraph 0004: “an identifier that identifies the computer session in which the data is being exchanged between the first computer system and the first server computer system is received at the first computer system and from the first server computer system”) of a target output vector that represents an embedding of the target medical diagnostic image into the multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); and applying, by the first computing system, the target output vector to the trained machine learning model (Azizi Table 1: “Performance of dermatology skin condition and Chest X-ray classification model measured by top-1 accuracy (%) and area under the curve (AUC) across different architectures”).
Regarding claim 5, Azizi discloses a computer-implemented method comprising: receiving, by the first computing system, a target output vector that represents an embedding of the target medical diagnostic image into a multi- dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); and applying, by the first computing system, the target output vector to a trained machine learning model to generate a target indication (Azizi Table 1: “Performance of dermatology skin condition and Chest X-ray classification model measured by top-1 accuracy (%) and area under the curve (AUC) across different architectures”) of at least one of a presence, degree of severity, or type of the specific condition or diagnosis represented in the target medical diagnostic image (Azizi pages 3461-3462, paragraphs under Dermatology and Chest X-rays: “The ground truth labels were…differential diagnosis of skin conditions in each case” and “We train the model to predict the five pathologies”). However, Azizi fails to explicitly disclose transmitting, by a first computing system to a second computing system, a target medical diagnostic image; and receiving, by the first computing system from the second computing system, an output. In related art, Bruce discloses transmitting, by a first computing system to a second computing system, a target medical diagnostic image (Bruce paragraphs 0004-0005: “a first computer software application to exchange data with a first server computer system…Each functionality is executable to provide additional data to the server computer system” wherein “The data can include medical imaging data”); and receiving, by the first computing system from the second computing system, an output (Bruce paragraph 0004: “an identifier that identifies the computer session in which the data is being exchanged between the first computer system and the first server computer system is received at the first computer system and from the first server computer system”). Distributing computing is a known technique where processing and data storage are distributed across multiple devices or systems, rather than being handled by a single central device. One of ordinary skill in the art would have been capable of applying distributing computing to enable efficient utilization of diverse resources across multiple locations. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi to incorporate the teachings of Bruce to yield the predictable result of processing medical images more efficiently and with better collaboration.
Regarding claim 8, Azizi, modified by Bruce, discloses the computer-implemented method of claim 5, further comprising: receiving, by the first computing system from the second computing system, an indication (Bruce paragraph 0004: “an identifier that identifies the computer session in which the data is being exchanged between the first computer system and the first server computer system is received at the first computer system and from the first server computer system”) of the trained machine learning model (Azizi Table 1: “Performance of dermatology skin condition and Chest X-ray classification model measured by top-1 accuracy (%) and area under the curve (AUC) across different architectures”).
Regarding claim 12, Azizi discloses the computer-implemented method of claim 9, wherein the third trained machine learning model comprises the second trained machine learning model and a fourth trained machine learning model, wherein the fourth trained machine learning model is configured to receive an output vector from the second trained machine learning model that represents an embedding of an input image into the second multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”) and to output, based on the output vector from the second trained machine learning model (Azizi page 3459, RHC, last paragraph: “classifier operating on these embeddings could achieve high classification accuracy”), an indication of at least one of a presence, degree of severity, or type of the specific condition or diagnosis (Azizi pages 3461-3462, paragraphs under Dermatology and Chest X-rays: “The ground truth labels were…differential diagnosis of skin conditions in each case” and “We train the model to predict the five pathologies”); and wherein the method further comprises: applying, by a first computing system, the target medical diagnostic image to the second trained machine learning model to generate a target output vector that represents an embedding of the target medical diagnostic image into the second multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”). However, Azizi fails to explicitly disclose receiving, by the first computing system from a second computing system, a target medical diagnostic image; and transmitting, by the first computing system to the second computing system, an indication of an output. In related art, Bruce discloses receiving, by the first computing system from a second computing system, a target medical diagnostic image (Bruce paragraphs 0004-0005: “a first computer software application to exchange data with a first server computer system…Each functionality is executable to provide additional data to the server computer system” wherein “The data can include medical imaging data”); and transmitting, by the first computing system to the second computing system, an indication of an output (Bruce paragraph 0004: “an identifier that identifies the computer session in which the data is being exchanged between the first computer system and the first server computer system is received at the first computer system and from the first server computer system”). Distributing computing is a known technique where processing and data storage are distributed across multiple devices or systems, rather than being handled by a single central device. One of ordinary skill in the art would have been capable of applying distributing computing to enable efficient utilization of diverse resources across multiple locations. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi to incorporate the teachings of Bruce to yield the predictable result of processing medical images more efficiently and with better collaboration.
Regarding claim 13, Azizi discloses the computer-implemented method of claim 9, wherein the third trained machine learning model comprises the second trained machine learning model and a fourth trained machine learning model, wherein the fourth trained machine learning model is configured to receive an output vector from the second trained machine learning model that represents an embedding of an input image into the second multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”) and to output, based on the output vector from the second trained machine learning model (Azizi page 3459, RHC, last paragraph: “classifier operating on these embeddings could achieve high classification accuracy”), an indication of at least one of a presence, degree of severity, or type of the specific condition or diagnosis (Azizi pages 3461-3462, paragraphs under Dermatology and Chest X-rays: “The ground truth labels were…differential diagnosis of skin conditions in each case” and “We train the model to predict the five pathologies”); and wherein the method further comprises: applying, by the first computing system, the target medical diagnostic image to the second trained machine learning model to generate a target output vector that represents an embedding of the target medical diagnostic image into the second multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); and applying, by the first computing system, the target output vector to the fourth trained machine learning model to generate a target indication (Azizi page 3459, RHC, last paragraph: “classifier operating on these embeddings could achieve high classification accuracy”) of at least one of a presence, degree of severity, or type of the specific condition or diagnosis represented in the target medical diagnostic image (Azizi pages 3461-3462, paragraphs under Dermatology and Chest X-rays: “The ground truth labels were…differential diagnosis of skin conditions in each case” and “We train the model to predict the five pathologies”). However, Azizi fails to explicitly disclose receiving, by the first computing system from the second computing system, a target medical diagnostic image; and transmitting, by the first computing system to the second computing system, the output. In related art, Bruce discloses receiving, by the first computing system from the second computing system, a target medical diagnostic image (Bruce paragraphs 0004-0005: “a first computer software application to exchange data with a first server computer system…Each functionality is executable to provide additional data to the server computer system” wherein “The data can include medical imaging data”); and transmitting, by the first computing system to the second computing system, the output (Bruce paragraph 0004: “an identifier that identifies the computer session in which the data is being exchanged between the first computer system and the first server computer system is received at the first computer system and from the first server computer system”). Distributing computing is a known technique where processing and data storage are distributed across multiple devices or systems, rather than being handled by a single central device. One of ordinary skill in the art would have been capable of applying distributing computing to enable efficient utilization of diverse resources across multiple locations. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi to incorporate the teachings of Bruce to yield the predictable result of processing medical images more efficiently and with better collaboration.
Regarding claim 23, Azizi discloses a computer-implemented method comprising: receiving a first trained machine learning model, wherein the first trained machine learning model is configured to receive an image as an input (Azizi Figure 1: “Our approach comprises three steps: (1) Self-supervised pretraining on unlabeled ImageNet using SimCLR”) and to output, based on the input image, an output vector that represents an embedding of the input image into a first multi- dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); generating a second trained machine learning model by using a generic medical training data set (Azizi Figure 1: “(2) Additional self-supervised pretraining using unlabeled medical images. If multiple images of each medical condition are available, a novel Multi-Instance Contrastive Learning (MICLe) is used to construct more informative positive pairs based on different images”) to further train the first trained machine learning model (Azizi page 3462, RHC, first full paragraph: “For MICLe pretraining we initialize our model using SimCLR pretrained weights”), wherein the second trained machine learning model is configured to receive an image as an input and to output, based on the input image, an output vector that represents an embedding of the input image into a second multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”); and applying, by the first computing system, the target medical diagnostic image to the second trained machine learning model to generate a target output vector that represents an embedding of the target medical diagnostic image into the second multi-dimensional embedding space (Azizi page 3462, LHC, first paragraph under 4.2. Pretraining protocol: “we investigate ResNet-50 (1x), ResNet-50 (4x), and ResNet-152 (2x) architectures as our base encoder networks. Following SimCLR, two fully connected layers are used to map the output of ResNets to a 128-dimensional embedding, which is used for contrastive learning”). However, Azizi in the main embodiment fails to disclose the generic medical training data set includes a plurality of medical diagnostic images and a plurality of diagnostic labels associated therewith; receiving, by a first computing system from a second computing system, a target medical diagnostic image; and transmitting, by the first computing system to the second computing system, an indication of the output.
Nonetheless, in one embodiment, Azizi discloses the generic medical training data set includes a plurality of medical diagnostic images and a plurality of diagnostic labels associated therewith (Azizi page 3464, LHC, paragraphs under 5.3. Comparison with supervised transfer learning; Tables 3 and 4: the supervised pretraining baselines uses labeled dermatology images and/or labeled chest x-rays). A person of ordinary skill has good reason to pursue the known options within his or her technical grasp. Azizi acknowledges using supervised pretraining is a common pretraining approach that is ubiquitous in medical image analysis (Azizi page 3458, LHC, last paragraph). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi in the main embodiment to incorporate supervised pretraining to lead to the anticipated success of improving transfer performance on downstream tasks (Azizi page 3464, LHC, last paragraph). However, Azizi still fails to disclose receiving, by a first computing system from a second computing system, a target medical diagnostic image; and transmitting, by the first computing system to the second computing system, an indication of the output.
In related art, Bruce discloses receiving, by a first computing system from a second computing system, a target medical diagnostic image (Bruce paragraphs 0004-0005: “a first computer software application to exchange data with a first server computer system…Each functionality is executable to provide additional data to the server computer system” wherein “The data can include medical imaging data”); and transmitting, by the first computing system to the second computing system, an indication of the output (Bruce paragraph 0004: “an identifier that identifies the computer session in which the data is being exchanged between the first computer system and the first server computer system is received at the first computer system and from the first server computer system”). Distributing computing is a known technique where processing and data storage are distributed across multiple devices or systems, rather than being handled by a single central device. One of ordinary skill in the art would have been capable of applying distributing computing to enable efficient utilization of diverse resources across multiple locations. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Azizi to incorporate the teachings of Bruce to yield the predictable result of processing medical images more efficiently and with better collaboration.
Regarding claim 25, Azizi, modified by Bruce, discloses the computer-implemented method of claim 23, wherein the diagnostic labels of the plurality of diagnostic labels indicate whether their associated medical diagnostic images are normal or abnormal (Azizi pages 3461-3462, paragraphs under Dermatology and Chest X-rays: “The ground truth labels were…differential diagnosis of skin conditions in each case” and “We train the model to predict the five pathologies”).
Regarding claim 26, Azizi, modified by Bruce, discloses the computer-implemented method of claim 25, further comprising: generating the plurality of diagnostic labels based on medical records associated with the plurality of medical diagnostic images, wherein the medical records include free text notes (Azizi page 3462, LHC, under Chest X-rays: “The ground truth labels were automatically extracted from radiology reports and correspond to a label space of 14 radiological observations”).
Regarding claim 27, Azizi, modified by Bruce, discloses the computer-implemented method of claim 23, wherein the first trained machine learning model comprises a machine learning model that has been trained based on a plurality of natural images (Azizi Figure 1: “(1) Self-supervised learning on unlabeled natural images”).
Claim(s) 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Azizi in view of Khosla et al. (NPL "Supervised Contrastive Learning").
Regarding claim 20, Azizi discloses the computer-implemented method of claim 9, wherein generating the second trained machine learning model comprises using a contrastive loss function to further train the first trained machine learning model (Azizi page 3460, RHC, first paragraph: “Multi-Instance Contrastive Learning (MICLe) is used for additional self-supervised pretraining”). However, Azizi fails to disclose using a supervised contrastive loss function. In the related art of training machine learning models, Khosla discloses using a supervised contrastive loss function (Khosla page 2, second paragraph: “we propose a loss for supervised learning that builds on the contrastive self-supervised literature by leveraging label information”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi to incorporate the teachings of Khosla to achieve increased top-1 accuracy with increased robustness (Khosla page 2, third paragraph).
Regarding claim 21, Azizi, modified by Khosla, discloses the computer-implemented method of claim 20, wherein using the supervised contrastive loss function to further train the first trained machine learning model comprises using the supervised contrastive loss function with a temperature parameter greater than 0.5 (Azizi page 3462, LHC, last paragraph: “we pretrain with…temperature in {0.1, 0.5, 1.0}”).
Claim(s) 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over Azizi and Bruce in view of Khosla.
Regarding claim 28, Azizi, modified by Bruce, discloses the computer-implemented method of claim 23, wherein generating the second trained machine learning model comprises using a contrastive loss function to further train the first trained machine learning model (Azizi page 3460, RHC, first paragraph: “Multi-Instance Contrastive Learning (MICLe) is used for additional self-supervised pretraining”). However, Azizi fails to disclose using a supervised contrastive loss function. In related art, Khosla discloses using a supervised contrastive loss function (Khosla page 2, second paragraph: “we propose a loss for supervised learning that builds on the contrastive self-supervised literature by leveraging label information”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified Azizi to incorporate the teachings of Khosla to achieve increased top-1 accuracy with increased robustness (Khosla page 2, third paragraph).
Regarding claim 29, Azizi, modified by Bruce and Khosla, discloses the computer-implemented method of claim 28, wherein using the supervised contrastive loss function to further train the first trained machine learning model comprises using the supervised contrastive loss function with a temperature parameter greater than 0.5 (Azizi page 3462, LHC, last paragraph: “we pretrain with…temperature in {0.1, 0.5, 1.0}”).
Claim(s) 22 is rejected under 35 U.S.C. 103 as being unpatentable over Azizi in view of Patil et al. (US 2023/0252305 A1).
Regarding claim 22, Azizi discloses the computer-implemented method of claim 9. However, Azizi fails to disclose receiving an updated specific medical training data set, wherein the updated specific medical training data set includes an additional plurality of medical diagnostic images that are associated with the specific condition or diagnosis and a plurality of diagnostic labels associated therewith; and using the updated specific medical training data set and the third trained machine learning model, generating an updated trained machine learning model by updating the third trained machine learning model, wherein the updated trained machine learning model is configured to receive an image as an input and to output, based on the input image, an output that is representative of a property or presence of the specific condition or diagnosis. In the related art of training machine learning models, Patil discloses receiving an updated specific medical training data set (Patil paragraph 0005: “training data at the respective clinical site”), wherein the updated specific medical training data set includes an additional plurality of medical diagnostic images that are associated with the specific condition or diagnosis (Patil paragraph 0040: “medical data may comprise any type of data that can be used, produced and/or obtained in a medical setting, including but not limited to…medical images”) and a plurality of diagnostic labels associated therewith (Patil paragraph 0050: “training data associated with some labels compared to other labels”); and using the updated specific medical training data set and the third trained machine learning model, generating an updated trained machine learning model by updating the third trained machine learning model (Patil paragraph 0005: “train the respective local copy of the model on training data at the respective clinical site…updating the parameter in the global model, based on the received local updates to the parameter and the received metadata”), wherein the updated trained machine learning model is configured to receive an image as an input and to output, based on the input image, an output that is representative of a property or presence of the specific condition or diagnosis (Patil paragraph 0041: “In embodiments where the medical data comprises a medical image, the model may output…a diagnosis based on the medical image”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Azizi to incorporate the teachings of Patil to result in improved models and thus improved clinical outcomes for clinical processes that use the models (Patil paragraph 0006).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Li et al. (US 2021/0201701 A1) discloses a second server may be configured to manage training materials received from a first server and implement medical diagnosis training processes. The second server may extract contextual semantic features from the vector or the sequence of vectors using, for example, a recurrent neural network (RNN), a convolutional neural network (CNN), along short term memory (LSTM) model, a transformer, or the like.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTINE ZHAO whose telephone number is (703)756-5986. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.Z./Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677