DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Comments
The Preliminary Amendment filed on November 13, 2025 has been entered and made of record.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21-26, 28-34 and 36-40 are rejected under 35 U.S.C. 103 as being unpatentable over Evron (WO 2021/044431) in view of Askarian et al. (“Novel Image Processing Method for Detecting Strep Throat (Streptococcal Pharyngitis) Using Smartphone”, Sensors 2019, 19(15), 3307, 27 July 2019, pp. 1-17).
As to claims 21, 29 and 37, Evron teaches a non-transitory computer readable medium (i.e., “machine-readable medium”, page 50 lines 4-22) configured to store program code instructions that, when executed by a processor, cause the processor to perform steps comprising/a computer-implemented method (i.e., “system for diagnosing or predicting a medical condition 100”, page 18 lines 16-18) comprising/a A computer-implemented system comprising: a processor (i.e., “processor”, page 49 lines 10-16); and a non-transitory computer-readable storage medium (i.e., “machine-readable medium”, page 50 lines 4-22) storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
obtaining a plurality of training throat images including a set of positive images depicting throats with a bacterial infection and a set of negative images (i.e., “In an obtaining training images and/or metadata step 202, images and/or metadata associated with a specific disease, disorder or condition are obtained. For example, in diagnosing a strep throat, images of throats, previously determined by laboratory tests to be with a streptococcal throat infection (e.g. Fig. 3B) and images without a streptococcus-infected throat, but with a viral pharyngitis (e.g. Fig. 3A) are obtained. The images may undergo processing to enhance the areas of interest and to better differentiate between the positive images and "negative" images”, page 21 lines 11-23);
obtaining bacterial labels for each of the plurality of training throat images to label each training throat image as positive or negative (i.e., “in diagnosing a strep throat, images of throats, previously determined by laboratory tests to be with a streptococcal throat infection (e.g. Fig. 3B) and images without a streptococcus- infected throat, but with a viral pharyngitis (e.g. Fig. 3A) are obtained. The images may undergo processing to enhance the areas of interest and to better differentiate between the positive images and "negative" images”, page 21 lines 11-23);
applying a machine-learned model (i.e., “The metadata of the metadata and/or image sets obtained in obtaining step 202 are analyzed in an analyzing step 204 to form positive condition data and/or image sets 203 and negative condition data and/or image sets 205. The sets of metadata and/or images may undergo processing using AI, to enhance the areas of interest and to better differentiate between the positive sets 203 and negative sets 205”, page 22 line 26 through page 23 line 1); and
outputting the machine-learned model (i.e., “After creating a training model for a particular medical application, the diagnostic engine will use this model to diagnose a patient's illness in real time”, page 37 lines 23-25).
However, Evron does not explicitly disclose the set of negative images depicting healthy throats; extracting features from the plurality of training throat images; and the applying the machine-learned model to learn a mapping between the features and the bacterial labels.
Askarian et al. teaches obtaining a plurality of training throat images including a set of positive images depicting throats with a bacterial infection and a set of negative images depicting healthy throats (i.e., “We recruited 56 subjects following the Texas Tech University Institutional Review Board (IRB) (IRB#: IRB 2018-701). The subjects (56) consisted of 28 healthy and 28 strep throat-diagnosed subjects whose ages were in the range of 20 to 38 years old. Among 56 subjects, 31 were male and 25 were female. Subjects were asked to sit in a relaxed position without any movement and instructed to open their mouths widely. At that moment, experimenters captured subjects’ throat images using a smartphone camera”, 2.1. Data Acquisition, page 3);
obtaining bacterial labels for each of the plurality of training throat images to label each training throat image as positive or negative (See for example, Figure14, “(a) Original image of a healthy throat” and “(c) original image of a diseased throat”, 3. Results, pages 13-14);
extracting features from the plurality of training throat images (i.e., “Strep throat symptoms include red spots on the roof of the mouth, red and swollen tonsils, and white and yellow dots on the tonsils and the back of the mouth. These symptoms are the indications of bacterial inflammation. Hence, our proposed method extracts these features to detect strep throat symptoms”, 2.3. Feature Extraction, page 8);
applying a machine-learned model to learn a mapping between the features and the bacterial labels (i.e., “The proposed algorithm acquires throat image using a smartphone with a gadget, processes the acquired images using color transformation and color correction algorithms, and finally classifies streptococcal pharyngitis (or strep) throat from healthy throat using machine learning techniques”, Abstract, page 1; and “We applied the k-NN classifier to distinguish strep throats from healthy throats”, 2.4. Classification, page 9).
Evron and Askarian et al. are analogous art because they are from the field of digital image processing for medical diagnosis.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Evron by incorporating the set of negative images depicting healthy throats, extracting features from the plurality of training throat images, and applying the machine-learned model to learn a mapping between the features and the bacterial labels, as taught by Askarian et al.
The suggestion/motivation for doing so would have been to provide a more reliable diagnosis by providing a unified color space for the images being applied to the machine learned model.
Therefore, it would have been obvious to combine Askarian et al. with Evron to obtain the invention as specified in claims 21, 29 and 37.
As to claims 22, 30 and 38, Askarian et al. teaches wherein the bacterial infection comprises a strep throat infection (i.e., “The proposed algorithm acquires throat image using a smartphone with a gadget, processes the acquired images using color transformation and color correction algorithms, and finally classifies streptococcal pharyngitis (or strep) throat from healthy throat using machine learning techniques”, Abstract, page 1).
As to claims 23 and 31, Evron teaches wherein the program code instructions further cause the processor to perform steps comprising: receiving, by an application server from a mobile application executing on a device of a subject (i.e., “The photos can be sent from the mobile phone or other device via the internet cloud 122 and stored on the cloud and/or at an application server 132 and/or database server”, page 18 lines 23-26), one or more throat images capturing an inside of the subject’s throat, the throat images captured using one or more user interfaces that instruct the subject to use a camera of the device to capture the throat images (See for example, “Camera guidance networks”, and “The second neural network is classification-based and is set up to classify each frame obtained from the camera with the at least one of the following outputs: move camera right, move camera left, move camera up, move camera down, open mouth, lower tongue, perfect placement”, page 25 lines 13-23); receiving, by the application server from the mobile application, a set of clinical factors (i.e., “metadata”, page 21 line 15 through page 23 line 1) associated with the subject that were input by the subject in one or more user interfaces (See for example, “The user uploads the photo/images to the cellphone App, as well as the answers to the questions in a capturing metadata”, page 23 lines 2-6) that instructed subject to input the set of clinical factors (See for example, page 21 line 15 through page 23 line 1); and inputting the throat images into the machine-learned model (i.e., “analyzing said image and/or said metadata of said patient using a machine learning system previously trained with said training samples”, claim 1 at page 55).
As to claims 24 and 32, Evron teaches wherein the set of clinical factors comprises at least one from a group consisting of: age (i.e., “By “metadata” is meant all the non-visuals signs and symptoms of the disease or disorder, such as, but not limited to … patient’s age”, page 18 lines 13-15); a presence or absence of swollen lymph nodes (i.e., “enlarged lymph nodes”, page 18 lines 13-15); subject temperature (i.e., “3) How high is your temperature?”, page 22 line 14); a presence or absence of fever (i.e., “fever”, page 18 lines 13-15); a presence or absence of a cough (i.e., “coughing”, page 18 lines 13-15); a presence or absence of a runny nose (i.e., “runny nose”, page 18 lines 13-15); a presence or absence of a headache (i.e., “headache”, page 18 lines 13-15); a presence or absence of body aches; a presence or absence of vomiting; a presence or absence of diarrhea; a presence or absence of fatigue; a presence or absence of chills; and a duration of pharyngitis.
As to claims 25 and 33, Askarian et al. teaches wherein at least one of the throat images is pre-processed before being input into the machine-learned model, the pre-processing comprising at least one from a group consisting of: uniform aspect ratio correction; rescaling; normalization; object detection; segmentation; cropping; dimensionality reduction; dimensionality increment; brightness adjustment; image shifting; image flipping; zoom in or out; image rotation; image quality filtering; and image pixel correction (See for example, “The preprocessing step is needed for accurate and effective feature extraction in throat images. Two main parts of the preprocessing steps are (1) color correction and (2) image segmentation”, 2.2. Preprocessing, pages 3-8).
As to claims 26 and 34, Evron teaches determining, at the application server (i.e., “Servers 130, 132 are configured to perform the method(s) of the present invention”, page 19 lines 15-16), a disease state prediction for the subject based on at least an output of the machine-learned model, wherein the disease state prediction indicates whether a bacterial pathogen is present in the subject (i.e., “distinguish between viral and bacterial”, page 27 lines 26-29); transmitting, from the application server to the device via a network connection, the disease state prediction for the subject (See for example, “The input for the application is obtained from the mobile phone or computer camera in addition to the metadata that is also fed by the user. The diagnostic algorithm is a software code that sits in the cloud to which the input is transferred, consisting of the image and the metadata, and where the input processing takes place. The output of the processing software is a medical diagnosis of the disease”, page 34 lines 4-9); and presenting, via the mobile application executing on the device, a user interface including the disease state prediction for the subject (i.e., “The outputs of the software may be displayed, such as on the App on cellphone/mobile 110, or on smart device 112”, page 19 lines 17-18).
As to claims 28 and 36, Evron teaches wherein the plurality of training throat images further include a second set of positive images depicting throats with a viral infection and a second set of negative images depicting throats without the viral infection (i.e., “images of viral and non-viral pharyngitis (viral and bacterial)”, page 10 lines 11-14), and wherein the program code instructions further cause the processor to perform steps comprising: obtaining viral labels for each of the plurality of training throat images in the second set of positive images and the second set of negative images to label each training throat image as positive or negative (i.e., page 10 lines 11-14); extracting features from the plurality of training throat images in the second sets (i.e., “extract discriminative features based on finding appropriate weights (kernels) to convolve with the input in the first layer and then convolve with feature maps in later layers”, page 10 lines 17-19); applying a second machine-learned model to learn a mapping between the features and the viral labels (i.e., “In the inference stage: The input is the image and its metadata, and the output is probability of viral and bacterial pharyngitis”, page 10 lines 11-23); and outputting the second machine-learned model (i.e., “In the inference stage: The input is the image and its metadata, and the output is probability of viral and bacterial pharyngitis”, page 10 lines 11-23).
As to claim 39, Evron teaches wherein the stored instructions further cause the one or more processors to perform operations comprising: receiving, by an application server from a mobile application executing on a device of a subject, one or more throat images capturing an inside of the subject’s throat (i.e., “The photos can be sent from the mobile phone or other device via the internet cloud 122 and stored on the cloud and/or at an application server 132 and/or database server”, page 18 lines 23-26), the throat images captured using one or more user interfaces that instruct the subject to use a camera of the device to capture the throat images (See for example, “Camera guidance networks”, and “The second neural network is classification-based and is set up to classify each frame obtained from the camera with the at least one of the following outputs: move camera right, move camera left, move camera up, move camera down, open mouth, lower tongue, perfect placement”, page 25 lines 13-23);
receiving, by the application server from the mobile application, a set of clinical factors (i.e., “metadata”, page 21 line 15 through page 23 line 1) associated with the subject that were input by the subject in one or more user interfaces (See for example, “The user uploads the photo/images to the cellphone App, as well as the answers to the questions in a capturing metadata”, page 23 lines 2-6) that instructed subject to input the set of clinical factors (See for example, page 21 line 15 through page 23 line 1);
inputting the throat images into the machine-learned model (i.e., “analyzing said image and/or said metadata of said patient using a machine learning system previously trained with said training samples”, claim 1 at page 55);
determining, at the application server (i.e., “Servers 130, 132 are configured to perform the method(s) of the present invention”, page 19 lines 15-16), a disease state prediction for the subject based on at least an output of the machine-learned model, wherein the disease state prediction indicates whether a bacterial pathogen is present in the subject (i.e., “distinguish between viral and bacterial”, page 27 lines 26-29);
transmitting, from the application server to the device via a network connection, the disease state prediction for the subject (See for example, “The input for the application is obtained from the mobile phone or computer camera in addition to the metadata that is also fed by the user. The diagnostic algorithm is a software code that sits in the cloud to which the input is transferred, consisting of the image and the metadata, and where the input processing takes place. The output of the processing software is a medical diagnosis of the disease”, page 34 lines 4-9); and
presenting, via the mobile application executing on the device, a user interface including the disease state prediction for the subject (i.e., “The outputs of the software may be displayed, such as on the App on cellphone/mobile 110, or on smart device 112”, page 19 lines 17-18).
As to claim 40, Evron teaches wherein the set of clinical factors comprises at least one from a group consisting of: age (i.e., “By “metadata” is meant all the non-visuals signs and symptoms of the disease or disorder, such as, but not limited to … patient’s age”, page 18 lines 13-15); a presence or absence of swollen lymph nodes (i.e., “enlarged lymph nodes”, page 18 lines 13-15); subject temperature (i.e., “3) How high is your temperature?”, page 22 line 14); a presence or absence of fever (i.e., “fever”, page 18 lines 13-15); a presence or absence of a cough (i.e., “coughing”, page 18 lines 13-15); a presence or absence of a runny nose (i.e., “runny nose”, page 18 lines 13-15); a presence or absence of a headache (i.e., “headache”, page 18 lines 13-15); a presence or absence of body aches; a presence or absence of vomiting; a presence or absence of diarrhea; a presence or absence of fatigue; a presence or absence of chills; and a duration of pharyngitis.
Claims 27 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Evron in view of Askarian et al. as applied to claims 26 and 34 above, and further in view of Feng et al. (U.S. Pub. No. 2021/0134405). The teachings of Evron and Askarian et al. have been discussed above.
As to claims 27 and 35, Evron and Askarian do not explicitly disclose wherein the disease state prediction comprises at least one of: a probability of a viral pathogen infection; a probability of a bacterial pathogen infection; and a probability of no pathogen infection.
Feng et al. teaches a disease state prediction comprises at least one of: a probability of a viral pathogen infection; a probability of a bacterial pathogen infection; and a probability of no pathogen infection (See for example, “The likelihood score generator or classifier comprises a ranked listing of pathogen or pathogen categories 232. As shown in FIG. 2, in just one embodiment the ranked listing may be a graph of likelihood, percentages, likelihood ratios, or other indication of ranking. In the image, bacterial, viral, and fungal infections are identified”, Paragraph [0069]).
Evron, Askarian et al. and Feng et al. are analogous art because they are from the field of analyzing patient data to generate a diagnosis.
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to further modify Evron and Askarian et al. by incorporating the disease state prediction comprises at least one of: a probability of a viral pathogen infection, a probability of a bacterial pathogen infection, and a probability of no pathogen infection, as taught by Feng et al.
The suggestion/motivation for doing so would have been to provide an additional probabilistic approach for infection diagnosis.
Therefore, it would have been obvious to combine Feng et al. with Evron and Askarian et al. to obtain the invention as specified in claims 27 and 35.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSE M TORRES whose telephone number is (571)270-1356. The examiner can normally be reached Monday thru Friday; 10:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSE M TORRES/Examiner, Art Unit 2664 09/25/2026
