DETAILED ACTION
This office action is in response to the communication received on September 4, 2025 concerning application No. 17/796,087 filed on July 18, 2022.
Claims 1-2, 5-7, 9-12, 14-15, and 17-19 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 09/04/2025 regarding the claim objections have been fully considered. The amendments to the claims have been entered and overcome the claim objection of claims 5, 8-9, and 13 previously set forth. Examiner notes the claim amendments have led to further claim objections.
Applicant's arguments filed 09/04/2025 regarding the 35 USC 103 rejection have been fully considered. The amendments to the claims have been entered and overcome the 35 USC 112b rejection of claim 15 previously set forth. Examiner notes the claim amendments have led to further 35 USC 112b issues.
Applicant's arguments filed 12/27/2024 regarding the claim interpretations have been fully considered, and are not persuasive. Regarding applicant’s argument that the claims are being limited to the specific examples in the specification, examiner notes that the interpretation of the claim limitations under 35 USC 112(f) includes a software for performing the limitation or an equivalent thereof. By having the interpretation include “an equivalent thereof”, the interpretation is not limited to only the specific examples in the specification. Further the claims do not recite sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Applicant’s arguments with respect to claim(s) 1 regarding the newly filed claim amendments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The rejection relies on new prior art references to teach the claim amendments. Please see the rejection below for further details.
Applicant's arguments filed 09/04/2025 regarding the 35 USC 103 rejection of claims 6 and 14 have been fully considered but they are not persuasive. In response to the applicant’s arguments that the prior art fails to teach “the support mask and how it is disposed on smartphone screen. The framing guides are displayed close to the corners of the display”, examiner respectfully disagrees. As set forth in the previous office action and discussed by Applicant Chandler is being relied upon to teach these limitations. Specifically, [0023]-[0025] of Chandler discloses displaying on an interface screen 40 a capture outline 50 (framework support mask) in fig. 3 of a body area being imaged. The capture outline is a transparent mask used to guide the operator in what body area is to be captured and the proper orientation thereof. Fig. 3 shows the support mask includes framing guides (outline) to help frame the region being captured on the screen of the device, where the outline extends to the corners of the screen. Applicant further argues, “the solution proposed by Chandler ‘406 needs the perfect match between the image to be captured by the camera and the mask…the claimed invention is applied to mobile/smartphone devices and should be easy-to-use…the breast size and formats could be completely different from each other…to be implemented in a smartphone, the Chandler ‘406 solution would require heavy image processing’ the claimed invention provides a simple support mark”. However, as the claims are currently written they do not go into detail about the size/shape of the framing guides or how the framing guides are actually being used to assist the user in obtaining the appropriate image. Applicant is reminded that although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Applicant further argues the prior art of record fails to teach “the thermographic image is divided into two halves; in both images and the hypo-radiant and hyper-radiant areas are identified by color analysis; the pre-processing algorithm applies the horizontal flip technique to one of the images for comparing to the other; and the pre-processing algorithm extracts the one or more regions of interest related to the one or more hypo-radiant areas, one or more hyper-radiant area, or areas with higher concentrations of hyper-radiant areas” and “using these regions as an input of an artificial intelligence tool”, examiner respectfully disagrees. As set forth in the previous office action [0066]-[0067] of Cantu discloses comparing the images to highlight points (regions) of interest within the image for quantitative analysis. [0036]-[0037] discloses the module 118 processes the images to perform qualitative and quantitative analysis and generates a report 126, in order to process the images the module must first receive them. Therefore since the module 118 performs the quantitative analysis it receives the one or more regions of interest. This reads on identifying one or more regions of interest and inputting the one or more regions into an AI tool. Naimi is then relied upon for teaching the deficiencies of Cantu. Specifically, figs. 1A-C shows the thermographic image as a whole and figs. 3A-3B and 4A-4B show the thermographic image split into two halves where the left breast represents image A and the right breast image represents image B. [0118] discloses the preprocessing includes a flip of a coordinate system for a thermospatial representation of a left breast (image A) and a non-flipped coordinate system for a thermospatial representation of a right breast (image B). [0097] discloses the images are color-coded according to their thermal data, meaning existing hypo-radiant areas and hyper-radiant areas within the images are identified by color analysis. [0150] and [0158] further disclose using the similarities/dissimilarities (comparison) of the flipped and non-flipped images to distinguish (identify) the regions as being a tumor (region of interest). Where the presence of dissimilar regions that represent hyper-radiant areas are used to distinguish regions that are a tumor, thereby identifying one or more regions of interest. Additionally, as set forth previously it would have been obvious to one of ordinary skill in the art to combine the teachings of Cantu and Naimi because of simple substitution. See the rejection of claim 14 below for further details.
Applicant further argues that it would not have been obvious to combine the prior art references relied upon to teach the claims of the present application, examiner respectfully disagrees. Examiner notes that applicant reference Chandler ‘406 as being the base reference, however, Cantu is applied as the base reference in each of the rejections, as previously set forth. As previously set forth each of the references relate to similar fields of endeavor and the combination of these references takes into account only knowledge of which was within the level of one of ordinary skill in the art at the time the claimed invention was filed and does not include knowledge gleaned only from applicant’s disclosure. For at least these reasons the combination of references is proper.
Claim Objections
Claims 1, 6, 10, 14-15, and 18-19 are objected to because of the following informalities:
Claim 1, line 28, “said repository” should read “said data repository”,
Claim 1, line 29, “the history” should read “the data history”,
Claim 6, lines 21 and 25, “an interface” should read “the interface”,
Claim 10, line 4, “the server repository” should read “the server”,
Claim 14, line 35, “the user” should recite “a user”,
Claim 15, line 3, “the percentage” should read “the percentages”,
Claims 18 and 19, lines 4 and 4 respectively, each instance of “the three images” should recite “the three thermographic breast images”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1, 2, 5, 11, 12, 14-15, and 17-18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the exam" in line 30. There is insufficient antecedent basis for this limitation in the claim. The claim does not previously recite an exam is being performed.
Claim 14 recites the limitation “compares each other” in line 25 which is considered indefinite. It is unclear what is represented by the word each in the claim. For example each can reference at least each of the image A and image B separately or image A or image B and the flipped image. For the purpose of examination and this office action the term “each” is being interpreted as the flipped image and the non flipped image A or B.
Claim 14 recites the limitation "the one or more regions of interest" in line 27. There is insufficient antecedent basis for this limitation in the claim. The claim does not previously recite one or more regions of interest.
Claims dependent upon the rejected claims above, but not directly addressed, are also rejected because they inherit the indefiniteness of the claim(s) they respectively depend upon.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“an intermediate module” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. For the purpose of examination and this office action “an intermediate module” will be interpreted as any software structure/component mediating and targeting information between an electronic application and a server or equivalent thereof (see [0043] of the present applications specification).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cozzie (US 20100312136) in view of Cantu et al. (US 20200258219, hereinafter Cantu), Al Anezi (US 20220087620) and Kakileti (US 20180005085).
Regarding claim 1, Cozzie teaches an auxiliary system for evaluating breast thermographic images (Abstract teaches a system for analyzing thermographic imaging of a breast), comprising:
an electronic device provided with a screen ([0080] discloses the processor makes information on a display device which is considered a screen) and connected with at least one thermal imager ([0032] and fig. 1, the image processing device 110 which is connected to thermographic imaging device 105), the electronic device being provided with at least one embedded electronic application which is installed in the electronic device and is provided with a interface ([0080] discloses the processor 110 is configured to send the information and receive the report and make the report available on a display device. The display device is considered the interface. The information on the processor that instructs the processor is considered the embedded electronic application);
an intermediate module communicating with the embedded electronic application ([0080] discloses the processor is configured to forward information to authority 103, the part of the processor that receives the information and forwards it and the part of authority 103 that receives the data is considered the intermediate module), the intermediate module being provided with a data driver, in which the intermediate module receives thermal image data from the embedded electronic application ([0033] discloses the captured thermographic images are transmitted to the processor 110, the part of the processor that receives the images is considered the data driver);
a server ([0036] and fig. 1, authority 103 which comprises the server), communicating with the intermediate module ([0036] discloses the authority receives images from the processor 110, meaning the authority is in communication with the intermediate module), provided with an artificial intelligence tool ([0126] discloses the authority includes a neural network module 118) and a data repository ([0036] storage 116);
wherein,
the thermal image data is pre-processed by a pre-processing algorithm to identify color patterns similarity and/or disparity ([0066] discloses a comparison of all points in the series 1 thermograph with the same points in the corresponding series 2 thermograph is performed. The comparison includes determining whether the temperature of the corresponding points are similar or different. Fig. 3 shows the temperature points correspond to a specific color shade. Therefore by comparing the temperatures, color pattern similarity and/or disparity is being identified. Claim 12 further discloses the analysis performed on the images is performed after the comparing, meaning the thermal image data is pre-processed. Additionally, [0106] discloses the authority 103 is used to compare the images and [0036] discloses the authority includes algorithms. The algorithm of authority 103 used to perform the comparison is considered the pre-processing algorithm) and, if a disparity in the color patterns is detected, the thermal image data are submitted to a process of color identification by pixel to extract the region of interest from the thermal image data ([0067] discloses that the hottest region of the breast is selected as the region of interest. Fig. 3 shows the temperature points correspond to a specific color shade, therefore by selecting the hottest region, the pixel with the lightest color is being identified and extracted as the region of interest from the thermal image);
the intermediate module sends the captured thermal image data to the server ([0032] discloses the processor 110 forwards the images to authority 103);
the artificial intelligence tool receives the thermal image data and returns an index of suspicion ([0036]-[0037] discloses the module 118 processes the images and generates a report 126, in order to process the images the module must first receive them. [0080] discloses the report includes TH values which [0083] discloses is indicative of whether cancer is present. The report is considered the index of suspicion), which is received by the intermediate module and directed to the embedded electronic application running in the electronic device ([0083] discloses the report 126 is sent back to the processor 110 which makes the report available for output on a display device); and
the intermediate module (2) stores the index of suspicion received in the data repository of the server (1) ([0106] discloses the collected images and scores that make up report 126 are stored in the database 116. Also, [0130] “reports can be stored in database 116”), said repository comprises a data history of at least one patient, wherein the history is updated with patient information and the index of suspicion ([0106] discloses the database 116 contains previously collected images and scores (index) which means the database stores a data history of at least one patient. Further [0080] discloses the report includes patient medical history, therefore by storing the currently generated report in database the history is being updated with patient information and the index of suspicion).
Cozzie does not specifically teach the system is a mobile system that includes a mobile electronic device and the artificial intelligence tool is trained with a sequence of thermal mammographic images wherein cancer was previously diagnosed by a physician.
However,
Cantu in a similar field of breast cancer screening teaches the system is a mobile system that includes a mobile electronic device that is provided with at least one embedded electronic application ([0081] discloses executing instructions using computer-executable components integrated with the application such as a mobile device or smartphone. The integrated application is considered the embedded electronic application).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie to have the system be a mobile system that comprises a mobile electronic device in order to be able to use the system in multiple different locations, thereby making the system more versatile.
Cantu further teaches training an artificial intelligence tool with a sequence of thermal mammographic images wherein cancer was previously diagnosed by a physician ([0009] “training a breast cancer screening model includes: accessing a set of thermographic images of a torso of a patient, the set of thermographic images associated with a pathology label in block S110…inputting the input vector and the pathology label as a training example in breast cancer screening model”. [0021] discloses the pathology label includes a label that indicate the presence of cancerous masses. [0051] further teaches the breast cancer screening model is implemented as an artificial neural network (intelligence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie to have the artificial intelligence tool be trained with a sequence of thermal mammographic images wherein cancer was previously diagnosed by a physician in order to generate a more accurate model because it is using previously obtained data as training material.
Cozzie in view of Cantu does not specifically teach the data history includes temperature of the day and time of the exam.
However,
Al Anezi in a similar field of thermal imaging teaches the data history includes temperature of the day and time of the exam ([0044]-[0045] disclose obtaining the temperature history of the day and the time of day which is accessed from a database/repository).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie in view of Cantu to have the data history include temperature of the day and time of the exam in order to ensure the findings of the exam are not solely based on the environmental factors, thereby making the system more accurate, as recognized by Al Anezi ([0044]).
Cozzie in view of Cantu and Al Anezi does not specifically teach the patient information includes date of last period and use or not of hormones.
However,
Kakileti in a similar field of breath thermography teaches the patient information includes date of last period and use or not of hormones ([0091]-[0100] disclose the patient information includes hormone levels and estimated last using date which corresponds to whether the user uses hormones or not. The information also includes menstrual period and the time period of the menstrual cycle).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the repository disclosed by Cozzie in view of Cantu and Al Anezi to have the patient information include date of last period and use or not of hormones in order to help improve the sensitivity/specificity of the screening, as recognized by Kakileti ([0091]).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cozzie in view of Cantu, Al Anezi, and Kakileti as applied to claim 1 above, and further in view of Kandikar et al. (US 20190385308).
Regarding claim 2, Cozzie in view of Cantu, Al Anezi, and Kakileti teaches the system of claim 1, as set forth above. Cozzie in view of Cantu, Al Anezi, and Kakileti does not specifically teach the artificial intelligence tool comprises a convolutional neural network.
However,
Kandikar in a similar field of screening for cancer discloses an artificial intelligence tool comprising a convolution neural network for obtaining the location of a tumor ([0086] discloses the artificial intelligence algorithm is a convolutional neural network and obtains the location of a tumor).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the artificial intelligence tool of Cozzie in view of Cantu, Al Anezi, and Kakileti for the convolutional neural network of Kandikar because it amounts to simple substitution of one known element for another to obtain the predictable results of evaluating the breast thermographic images
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cozzie in view of Cantu, Al Anezi, and Kakileti as applied to claim 1 above, and further in view of Chandler (US 20110243406).
Regarding claim 5, Cozzie in view of Cantu, Al Anezi, and Kakileti teaches the system of claim 1, as set forth above. Cozzie in view of Cantu, Al Anezi, and Kakileti does not specifically teach the embedded electronic application user interface comprises at least one framework support mask in capturing the breast thermographic image, said framework support mask comprising framing guides to help frame breasts captured in the breast thermographic image onto the screen of the mobile electronic device.
However,
Chandler in a similar field of capturing medical images teaches an electronic application interface comprises at least one framework support mask in capturing the breast thermographic image, said framework support mask comprising framing guides to help frame breasts captured in the breast thermographic image onto the screen of the electronic device ([0023]-[0025] discloses displaying on an interface screen 40 a capture outline 50 (framework support mask) in fig. 3 of a body area being imaged. The capture outline is a transparent mask used to guide the operator in what body area is to be captured and the proper orientation thereof. Fig. 3 shows the support mask includes framing guides (outline) to help frame the region being captured on the screen of the device. [0027] discloses a breast region is a category that is imaged and [0017] discloses the obtained images are thermal images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the embedded electronic application disclosed by Cozzie in view of Cantu, Al Anezi, and Kakileti to comprises at least one framework support mask in capturing the breast thermographic image, said framework support mask comprising framing guides to help frame breasts captured in the breast thermographic image onto the screen of the mobile electronic device in order to help guide the operator in obtaining the correct image of the patient, as recognized by Chandler ([0024]).
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cozzie in view of Cantu, Al Anezi, and Kakileti as applied to claim 1 above, and further in view of Guyon et al. (WO2011087807A2, hereinafter Guyon).
Regarding claim 12, Cozzie in view of Cantu, Al Anezi, and Kakileti teaches the system of claim 1, as set forth above. Cozzie in view of Cantu, Al Anezi, and Kakileti does not specifically teach the index of suspicion is a percentage of chance of having a pathological pattern, wherein a next step message is displayed on the screen based on the index of suspicion returned by the artificial intelligence tool.
However,
Guyon in a similar field of cancer screening teaches the index of suspicion is a percentage of chance of having a pathological pattern, wherein a next step message is displayed on the screen based on the index of suspicion returned by the artificial intelligence tool (pg. 27, para. 4, “the bottom of the exemplary display window includes a statement describing the basis for the assessment and provides a recommendation that the user consult a physician if any of the features is symptomatic of melanoma with more than 50% confidence”, the percent confidence corresponds to the percentage of change of having a pathological pattern and the percent confidence is determined using an ensemble of classifiers. Pg. 24, para. 9, discloses the ensemble classifiers are machine classifiers which are an example of an artificial intelligence tool).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie in view of Cantu, Al Anezi, and Kakileti to have the index of suspicion is a percentage of chance of having a pathological pattern, wherein a next step message is displayed on the screen based on the index of suspicion returned by the artificial intelligence tool in order to reduce the need for an operator to analyze the index of suspicion to determine the next steps, thereby allowing for a more streamlined and efficient process.
Claim(s) 6-7, 10-11, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cozzie (US 20100312136) in view of Cantu et al. (US 20200258219, hereinafter Cantu) and Chandler (US 20110243406).
Regarding claim 6, Cozzie teaches an auxiliary process for evaluation of breast thermographic images (Abstract and title, teach a method for analyzing thermographic imaging of a breast), comprising the steps of:
collecting at least one breast thermographic image by an embedded electronic application installed and running in an electronic device ([0096] step 405 discloses capturing thermographic images of the breast. [0023] discloses transferring the thermographic images to image processing device 110. The process stored in processor 110 which performs the transferring is considered the embedded electronic application), wherein the at least one breast thermographic image is collected by a thermal imager in communication with the electronic device ([0023] discloses the images are captured using thermographic imaging device 105 that is in communication to processor 110 (electronic device)), being the at least one breast thermographic image displayed in an interface of the embedded electronic application ([0080] discloses the processor 110 is configured to send the information and receive the report and make the report available on a display device, where the report includes the series 1 and 2 images used. [0079] discloses the series 1 and 2 images are the thermographic images);
pre-processing the breast thermographic image by a pre-processing algorithm to identify color patterns similarity and/or disparity ([0066] discloses a comparison of all points in the series 1 thermograph with the same points in the corresponding series 2 thermograph is performed. The comparison includes determining whether the temperature of the corresponding points are similar or different. Fig. 3 shows the temperature points correspond to a specific color shade. Therefore by comparing the temperatures, color pattern similarity and/or disparity is being identified. Claim 12 further discloses the analysis performed on the images is performed after the comparing, meaning the thermal image data is pre-processed. Additionally, [0106] discloses the authority 103 is used to compare the images and [0036] discloses the authority includes algorithms. The algorithm of authority 103 used to perform the comparison is considered the pre-processing algorithm) and, if a disparity in the color patterns is detected, the breast thermographic image is submitted to a process of color identification by pixel to extract the region of interest from the thermal image data ([0067] discloses that the hottest region of the breast is selected as the region of interest. Fig. 3 shows the temperature points correspond to a specific color shade, therefore by selecting the hottest region, the pixel with the lightest color is being identified and extracted as the region of interest from the thermal image);
sending, by an intermediate module (2), the breast thermographic image to a server ([0036] discloses the authority receives images from the processor 110), which is provided with an artificial intelligence tool ([0126] discloses the authority includes a neural network module 118) evaluating the breast thermographic image and returning an index of suspicion ([0036]-[0037] discloses the module 118 processes the images and generates a report 126. [0080] discloses the report includes TH values which [0083] discloses is indicative of whether cancer is present. The report is considered the index of suspicion); and
receiving the index of suspicion by the embedded electronic application and making available the index of suspicion in an interface of the embedded electronic application ([0083] discloses the report 126 is sent back to the processor 110 which makes the report available for output on a display device (interface)).
Cozzie does not specifically teach the system is a mobile system that includes a mobile electronic device and the artificial intelligence tool is trained with a sequence of thermal mammographic images wherein cancer was previously diagnosed by a physician.
However,
Cantu in a similar field of breast cancer screening teaches the system is a mobile system that includes a mobile electronic device that is provided with at least one embedded electronic application ([0081] discloses executing instructions using computer-executable components integrated with the application such as a mobile device or smartphone. The integrated application is considered the embedded electronic application).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie to have the system be a mobile system that comprises a mobile electronic device in order to be able to use the system in multiple different locations, thereby making the system more versatile.
Cantu further teaches training an artificial intelligence tool with a sequence of thermal mammographic images wherein cancer was previously diagnosed by a physician ([0009] “training a breast cancer screening model includes: accessing a set of thermographic images of a torso of a patient, the set of thermographic images associated with a pathology label in block S110…inputting the input vector and the pathology label as a training example in breast cancer screening model”. [0021] discloses the pathology label includes a label that indicate the presence of cancerous masses. [0051] further teaches the breast cancer screening model is implemented as an artificial neural network (intelligence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie to have the artificial intelligence tool be trained with a sequence of thermal mammographic images wherein cancer was previously diagnosed by a physician in order to generate a more accurate model because it is using previously obtained data as training material.
Cozzie in view of Cantu does not specifically teach the interface of the embedded electronic application (3) displays a support mask overlapping in real-time the at least one breast thermographic image to be collected and displayed in an interface of the embedded electronic application (3), said support mask comprising framing guides to help frame the breast thermographic image onto a screen comprised in the mobile electronic device, wherein the framed guides are displayed close to corners of the screen comprised in the mobile electronic device.
However,
Chandler in a similar field of capturing medical images teaches the interface of the embedded electronic application (3) displays a support mask overlapping in real-time the at least one breast thermographic image to be collected and displayed in the interface of the embedded electronic application (3), said support mask comprising framing guides to help frame the breast thermographic image onto a screen comprised in the electronic device, wherein the framed guides are displayed close to corners of the screen comprised in the electronic device ([0023]-[0025] discloses displaying on an interface screen 40 a capture outline 50 (framework support mask) in fig. 3 of a body area being imaged. The capture outline is a transparent mask used to guide the operator in what body area is to be captured and the proper orientation thereof. Fig. 3 shows the support mask includes framing guides (outline) to help frame the region being captured on the screen of the device, where the outline extends to the corners of the screen. [0027] discloses a breast region is a category that is imaged and [0017] discloses the obtained images are thermal images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method disclosed by Cozzie in view of Cantu to have the interface of the embedded electronic application (3) display a support mask overlapping in real-time the at least one breast thermographic image to be collected and displayed in the interface of the embedded electronic application (3), said support mask comprising framing guides to help frame the breast thermographic image onto a screen comprised in the electronic device, wherein the framed guides are displayed close to corners of the screen comprised in the electronic device in order to help guide the operator in obtaining the correct image of the patient, as recognized by Chandler ([0024]).
Regarding claim 7, Cozzie in view of Cantu and Chandler teaches the method of claim 6, as set forth above. Cantu further teaches the process is implemented in an auxiliary mobile system, as defined in claim 1 ([0081] discloses executing instructions using computer-executable components integrated with the application such as a mobile device or smartphone. The mobile device or smartphone is considered the auxiliary mobile system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie in view of Cantu and Chandler to have the process be implemented in an auxiliary mobile system in order to be able to use the system in multiple different locations, thereby making the system more versatile.
Regarding claim 10, Cozzie in view of Cantu and Chandler teaches the method of claim 6, as set forth above. Cozzie further teaches a step of sending and updating patient history data, wherein the intermediate module sends patient information and/or index of suspicion to the server repository ([0106] discloses the database 116 contains previously collected images and scores (index) which means the database stores a data history. Further [0080] discloses the report includes patient medical history, therefore by storing the currently generated report in the database the history is being updated with patient information and the index of suspicion).
Regarding claim 11, Cozzie in view of Cantu and Chandler teaches the system of claim 1, as set forth above. Cantu further teaches the mobile electronic device is a smartphone ([0081] discloses executing instructions using computer-executable components integrated with the application such as a mobile device or smartphone, therefore the mobile electronic device is a smartphone).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie in view of Cantu and Chandler to have the mobile electronic device is a smartphone in order to be able to use the system in multiple different locations, thereby making the system more versatile.
Regarding claim 19, Cozzie in view of Cantu and Chandler teaches the process of claim 6, as set forth above. Cozzie further teaches three breast thermographic images are acquired by the thermal imager, being a frontal view, a right-side view, and a left-side view ([0041] discloses capturing a first series of thermograph images which include an anterior (front) view, left view, and right view), wherein the embedded electronic application (3) combines the three images and composes the three images into a single image ([0080] discloses the processor 110 is configured to send the information and receive the report and make the report available on a display device, where the report includes the series 1 and 2 images used. [0041] discloses the series 1 images include an anterior (front) view, right view, and left view images. Therefore the displayed series includes a combination of the three views as a single image).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cozzie in view of Cantu and Chandler as applied to claim 6 above, and further in view of Harrison (EP0885587A1).
Regarding claim 9, Cozzie in view of Cantu and Chandler teaches the method of claim 6, as set forth above. Cozzie further teaches the step of sending the breast thermographic image to a server comprises the sub-steps of:
receiving evaluation request by the intermediate module ([0034] discloses the processor is used to upload the images to the authority 103, the process of obtaining the images needed to upload is considered the evaluation request. The part of the processor that transmits/receives data and the part of the authority that transmits/receives data is considered the intermediate module), communicating with the electronic application and with the server, wherein the intermediate module receives the data from the breast thermographic image and directs it to the server ([0032] discloses the processor 110 forwards the images to authority 103, therefore the processor communicates with the application and the server);
wherein, the intermediate module is responsible for receiving the index of suspicion evaluated by the artificial intelligence tool from the server and sending it to the electronic application ([0083] discloses the report 126 is sent back to the processor 110 and is therefore received by the intermediate module and sends it to the electronic application).
Cozzie in view of Cantu and Chandler does not specifically teach converting the breast thermographic image into a data string on the mobile electronic device.
However,
Harrison in a similar field of thermographic imaging teaches converting the breast thermographic image into a data string on the electronic device (pg. 8, lines 1-10 disclose converting the thermographic images into binary files, which is an example of a character chain which is a data string as discussed in [0048] of the present applications specification).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method disclosed by Cozzie in view of Cantu and Chandler to convert the breast thermographic image into a data string on the mobile electronic device in order to improve the speed and efficiency of the image processing, as recognized by Harrison (pg. 8, lines 35-40).
Claim(s) 14-15 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cozzie (US 20100312136) in view of Nam et al. (WO2018070760A1, hereinafter Nam), Chandler (US 20110243406), Cantu et al. (US 20200258219, hereinafter Cantu), Guyon et al. (WO2011087807A2, hereinafter Guyon), and Naimi et al. (US 20110243409, hereinafter Naimi).
Regarding claim 14, Cozzie teaches an auxiliary system for evaluating breast thermographic images (Abstract teaches a system for analyzing thermographic imaging of a breast), comprising:
a device ([0032] the combination of the processor 110 and imaging device 105) being provided with an electronic application and comprising a thermal camera capable of acquiring breast thermographic image ([0032] and fig. 1, the image processing device 110 which is connected to thermographic imaging device 105. [0025] discloses the imaging device is used for capturing thermographic images), said electronic application being a software provided with a user interface and displaying acquired breast thermographic image on a screen of the device ([0080] discloses the processor 110 is configured to send the information and receive the report and make the report available on a display device, where the report includes the series 1 and 2 images used. [0079] discloses the series 1 and 2 images are the thermographic images. The display device is considered the user interface and the information on the processor that instructs the processor is considered the electronic application that is software);
a remote server ([0036] and fig. 1, authority 103 which comprises the remote server) configured to receive the acquired breast thermographic image ([0023] and [0034], [0036] disclose the thermographic images are transferred/uploaded to the authority 103), said remote server provided with an artificial intelligence tool ([0126] discloses the authority includes a neural network module 118) and a data repository ([0036] storage 116), wherein the breast thermographic image is pre-processed by a pre-processing algorithm, executed in the remote server, to identify color patterns similarity and/or disparity ([0066] discloses a comparison of all points in the series 1 thermograph with the same points in the corresponding series 2 thermograph is performed. The comparison includes determining whether the temperature of the corresponding points are similar or different. Fig. 3 shows the temperature points correspond to a specific color shade. Therefore by comparing the temperatures, color pattern similarity and/or disparity is being identified. Claim 12 further discloses the analysis performed on the images is performed after the comparing, meaning the thermal image data is pre-processed. Additionally, [0106] discloses the authority 103 is used to compare the images and [0036] discloses the authority includes algorithms. The algorithm of authority 103 used to perform the comparison is considered the pre-processing algorithm) and extracting one or more regions of interest based on the color patterns ([0066]-[0067] discloses that the hottest region of the breast is selected as the region of interest. Fig. 3 shows the temperature points correspond to a specific color shade, therefore by selecting the hottest region, the pixel with the lightest color is being identified and extracted as the region of interest from the thermal image); and wherein the artificial intelligence tool receives one or more regions of interest and returns an index of suspicion ([0036]-[0037] discloses the module 118 processes the images to perform qualitative and quantitative analysis and generates a report 126, in order to process the images the module must first receive them. [0066] further discloses comparing the thermographs to highlight points of interest for quantitative analysis, therefore since the module 118 performs the quantitative analysis it receives the one or more regions of interest. [0080] discloses the report includes TH values which [0083] discloses is indicative of whether cancer is present. The report is considered the index of suspicion), which is displayed on the screen of the device by the electronic application ([0083] discloses the report 126 is sent back to the processor 110 which makes the report available for output on a display device);
Cozzie does not specifically teach a smartphone device being provided with an electronic application and comprising a thermal camera capable of acquiring breast thermographic image.
However,
Nam in a similar field of thermographic breast imaging discloses a smartphone device being provided with an electronic application and comprising a thermal camera capable of acquiring breast thermographic image (pg. 4, para. 4, “the present invention can diagnose breast cancer directly by a mobile terminal such as a smart phone having only one per person and having a thermal imaging camera”. Additionally, pg. 5, para. 2 discloses the apparatus includes an image preprocessing unit which is considered an electronic application and pg. 5, para. 3 discloses an infrared camera (thermal imaging) which photographs a part of the body (breast)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie to be a smartphone device being provided with an electronic application and comprising a thermal camera capable of acquiring breast thermographic image in order to provide a simple and inexpensive primary breast cancer diagnostic means, as recognized by Nam (pg. 4, para. 4).
Cozzie in view of Nam does not specifically teach wherein during the breast thermographic image acquisition, a support mask is displayed on the screen by the electronic application, said support mask comprises framing guides helping frame the breasts onto the screen.
However,
Chandler in a similar field of capturing medical images teaches during the breast thermographic image acquisition, a support mask is displayed on the screen by the electronic application, said support mask comprising framing guides to help frame breasts onto the screen ([0023]-[0025] discloses displaying on an interface screen 40 a capture outline 50 (framework support mask) in fig. 3 of a body area being imaged. The capture outline is a transparent mask used to guide the operator in what body area is to be captured and the proper orientation thereof. Fig. 3 shows the support mask includes framing guides (outline) to help frame the region being captured on the screen of the device. [0027] discloses a breast region is a category that is imaged and [0017] discloses the obtained images are thermal images).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the embedded electronic application disclosed by Cozzie in view of Nam to have during the breast thermographic image acquisition, a support mask is displayed on the screen by the electronic application, said support mask comprises framing guides helping frame the breasts onto the screen in order to help guide the operator in obtaining the correct image of the patient, as recognized by Chandler ([0024]).
Cozzie in view of Nam and Chandler does not specifically teach the artificial intelligence tool is previously trained and fed with a sequence of thermal mammographic images where cancer was previously diagnosed by a physician.
However,
Cantu in a similar field of breast cancer screening teaches an artificial intelligence tool is previously trained and fed with a sequence of thermal mammographic images where cancer was previously diagnosed by a physician ([0009] “training a breast cancer screening model includes: accessing a set of thermographic images of a torso of a patient, the set of thermographic images associated with a pathology label in block S110…inputting the input vector and the pathology label as a training example in breast cancer screening model”. [0021] discloses the pathology label includes a label that indicate the presence of cancerous masses. [0051] further teaches the breast cancer screening model is implemented as an artificial neural network (intelligence)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie in view of Nam and Chandler to have the artificial intelligence tool be previously trained and fed with a sequence of thermal mammographic images where cancer was previously diagnosed by a physician in order to generate a more accurate model because it is using previously obtained data as training material.
Cozzie in view of Nam, Chandler, and Cantu does not specifically teach the electronic application displays on the screen of the device a next step message for the user based on the index of suspicion.
However,
Guyon in a similar field of screening for cancer discloses an electronic application displays on the screen of the device a next step message for the user based on the index of suspicion (pg. 27, para. 4, “the bottom of the exemplary display window includes a statement describing the basis for the assessment and provides a recommendation that the user consult a physician if any of the features is symptomatic of melanoma with more than 50% confidence”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system disclosed by Cozzie in view of Nam, Chandler, and Cantu to have the electronic application display on the screen of the device a next step message for the user based on the index of suspicion in order to reduce the need for an operator to analyze the index of suspicion to determine the next steps, thereby allowing for a more streamlined and efficient process.
Cozzie in view of Nam, Chandler, Cantu, and Guyon does not specifically teach the pre-processing algorithm divides the breast thermographic image into two halves, image A and image B, wherein both images A and B are submitted to a process of identifying colors per pixels in such a way that existing one or more hypo-radiant areas and one or more hyper-radiant areas are represented in a predetermined color scale, and the pre-processing algorithm applies a horizontal flip technique to either image A or image B and compares each other to identify areas with highest concentrations of hyper-radiant areas, and then the pre-processing algorithm extracts the one or more regions of interest related to the one or more hypo-radiant areas, one or more hyper-radiant areas, or areas with higher concentrations of hyper-radiant areas.
However,
Naimi in a similar field of thermal breast imaging teaches a pre-processing algorithm divides the breast thermographic image into two halves, image A and image B (figs. 1A-C shows the thermographic image as a whole and figs. 3A-3B and 4A-4B show the thermographic image split into two halves), wherein both images A and B are submitted to a process of identifying colors per pixels ([0097] discloses the representations are color-coded, therefor the images are submitted to a process of identifying colors per pixels) in such a way that existing one or more hypo-radiant areas and one or more hyper-radiant areas are represented in a predetermined color scale ([0097] discloses the images are color-coded according to their thermal data, meaning existing hypo-radiant areas and hyper-radiant areas are represented in a predetermined color scale), and the pre-processing algorithm applies a horizontal flip technique to either image A or image B and compares each other ([0118] discloses the preprocessing includes a flip of a coordinate system for a thermospatial representation of a left breast (image A) and a non-flipped coordinate system for a thermospatial representation of a right breast (image B)) to identify areas with highest concentrations of hyper-radiant areas, and then the pre-processing algorithm extracts the one or more regions of interest related to the one or more hypo-radiant areas, one or more hyper-radiant areas, or areas with higher concentrations of hyper-radiant areas ([0150] and [0158] discloses using the comparison to distinguish (identify) the regions as being a tumor (region of interest) which is an area within the image that has a the highest concentration of hyper-radiant areas).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the pre-processing algorithm of Cozzie in view of Nam, Chandler, Cantu, and Guyon for the pre-processing algorithm of Naimi because it amounts to simple substitution of one known element for another to obtain the predictable results of extracting one or more regions of interest in the images to be analyzed.
Regarding claim 15, Cozzie in view of Nam, Chandler, Cantu, Guyon and Naimi teaches the system of claim 14, as set forth above. Guyon further teaches the index of suspicion is a percentage of chances of having a pathological pattern, wherein if the percentage of chances of having a pathological pattern indicates a potential positive, the electronic device displays a message instructing the user to find a specialized physician (pg. 27, para. 4, “the bottom of the exemplary display window includes a statement describing the basis for the assessment and provides a recommendation that the user consult a physician if any of the features is symptomatic of melanoma with more than 50% confidence”, the percent confidence corresponds to the percentage of changes of having a pathological pattern. Pg. 8, para. 2 discloses the physician being recommended is a particular (specialized) physician).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have substituted the index of suspicion disclosed by Cozzie in view of Nam, Chandler, Cantu, Guyon and Naimi for the index of suspicion comprising a percentage of chances of having a pathological pattern of Guyon because it amounts to simple substitution of one known element for another to obtain the predictable results of determining the change of having a disease.
Regarding claim 17, Cozzie in view of Nam, Chandler, Cantu, Guyon and Naimi teaches the system of claim 14, as set forth above. Chandler further teaches the framing guides are displayed close to corners of the screen of the device (fig. 4 shows the outline of the mask 50 (framing guides) extends to the corners of the screen on the device).
Regarding claim 18, Cozzie in view of Nam, Chandler, Cantu, Guyon and Naimi teaches the system of claim 14, as set forth above. Cozzie further teaches three breast thermographic images are acquired by the thermal camera, being a frontal view, a right-side view, and a left-side view ([0041] discloses capturing a first series of thermograph images which include an anterior (front) view, left view, and right view), wherein the embedded electronic application (3) combines the three images and composes the three images into a single image ([0080] discloses the processor 110 is configured to send the information and receive the report and make the report available on a display device, where the report includes the series 1 and 2 images used. [0041] discloses the series 1 images include an anterior (front) view, right view, and left view images. Therefore the displayed series includes a combination of the three views as a single image).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW BEGEMAN whose telephone number is (571)272-4744. The examiner can normally be reached Monday-Thursday 8:30-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Keith Raymond can be reached at 5712701790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW W BEGEMAN/Examiner, Art Unit 3798
Prosecution Insights