DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/08/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112a
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 22-25, 28, 31-34, 37, 39, and 40 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 22 recites “retrain a common base model architecture with asymmetric loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model.”
Examiner notes that Paragraph 00159 of Applicant’s specification recites “the first prediction model 154 may be retrained with a different objective function or a different loss function.” And Paragraph 0023 recites “adjusting a prior trained machine learning prediction model provided by the first prediction model to optimize a quantile loss function defining the updated regression loss function” However, Applicant’s original disclosure does not support “retrain a common base model architecture with asymmetric loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model.” For examination purposes, Examiner is interpreting that the quantile loss function is an asymmetric loss function that is used to retrain the common based model that is a different type of loss function used to train the model.
Claim 31 recite a similar limitation and is rejected for the same reasons.
Claim 24 recites “suppress transmission of the determined response to a software application when the confidence score does not exceed the threshold.”
Examiner notes that Paragraph 0007 of Applicant’s specification recites “Such triggering of computerized actions based on one or more conditions being met (e.g. meeting and/or exceeding a desired confidence threshold in the prediction) may include generation of content” And Paragraph 00121 recites “via the action generation module 120 that the confidence score output is below a defined threshold, as retrieved from the confidence score repository, may trigger, the content delivery system 100 (e.g. the model reconfiguration module 115) to gather additional data” However, Applicant’s original disclosure does not support “suppress transmission of the determined response to a software application when the confidence score does not exceed the threshold.” For examination purposes, Examiner is interpreting the claims as not sending the determined response when the confidence score does not exceed the threshold.
Claim 33 recite a similar limitation and is rejected for the same reasons.
Claim 25 recites “determine the threshold based on historical behavior associated with prior predicted responses generated by the AI model.”
Examiner notes that Paragraph 00169 of Applicant’s specification recites “performing dynamic threshold adjustment by adjusting decision thresholds for the models based on the detected confidence score such that if a high confidence score is detected, the model may be more lenient in accepting the prediction” And Paragraph 0011 recites “there is provided a computer tool and interface in a networked environment that assesses historical transaction behaviors and activities associated with one or more computing devices and predicts future digital events in a networked computing environment,” However, Applicant’s original disclosure does not support “determine the threshold based on historical behavior associated with prior predicted responses generated by the AI model.” For examination purposes, Examiner is interpreting the claims the threshold is determined based on historical data that have been used by the model to predict an output.
Claim 34 and 40 recite a similar limitation and is rejected for the same reasons.
Claim 28 recites “at least one of enable and disable use of the determined response by downstream software processes based on the confidence score.”
Examiner notes that Paragraph 0007 of Applicant’s specification recites “Such triggering of computerized actions based on one or more conditions being met (e.g. meeting and/or exceeding a desired confidence threshold in the prediction) may include generation of content” And Paragraph 00121 recites “via the action generation module 120 that the confidence score output is below a defined threshold, as retrieved from the confidence score repository, may trigger, the content delivery system 100 (e.g. the model reconfiguration module 115) to gather additional data” However, Applicant’s original disclosure does not support “at least one of enable and disable use of the determined response by downstream software processes based on the confidence score.” For examination purposes, Examiner is interpreting the claims as sending or not sending the determined response based on the confidence score.
Claim 37 recite a similar limitation and is rejected for the same reasons.
Claim Rejections - 35 USC § 112b
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 22, 31, and 39 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “differently/different” in claims 21-22 and 30-31 makes the scope of the claim unclear. The term “different” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It is unclear if the loss function is a different type of loss function or if the parameters of the loss function is different. For examination purposes, Examiner will interpret the claims as a different type of loss function.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 21, 23-24, 26-30, 32-33, 35-38, and 40 are rejected under 35 U.S.C. 103 as being unpatentable over Giovanni John Jacques Palma et al; US 20220138931 A1 filed on Oct 30, 2020 (hereinafter “Palma”) in view of Meghan Frances Fotak et al; US 20220200878 A1 filed on Jul 21, 2021 (hereinafter “Fotak”) in further view of Man Mohan Sugathan et al; US 12321868 B1 filed on Oct 29, 2021.
Regarding claim 21, Palma teaches An apparatus comprising: a memory; and at least one processor communicatively coupled to the memory, the at least one processor configured to: (Palma Paragraph 0006; “a method is provided, in a data processing system, comprising at least one processor and at least one memory, the at least one memory comprising instructions executed by the at least one processor to implement a lesion detection and classification artificial intelligence (AI) pipeline”)
train an artificial intelligence (Al) model to generate a response based on historical data; (Palma Paragraph 0079; “the listing of lesions, their classifications, and contours may be stored in a history data structure associated with a patient with which the volume of input CT medical images correspond such that multiple executions of the AI pipeline on different volumes of input CT medical images associated with that patient may be stored and evaluated over time.” Palma Paragraph 0081; “an automated AI pipeline comprising a plurality of configured and trained ML/DL computer models that implement various artificial intelligence operations for various stages of the AI pipeline… and generate a listing of such lesions as well as the contours of the lesions and the anatomical structures for further downstream computer processing of the AI generated information from the AI pipeline” Examiner notes that an artificial intelligence (Al) model is trained to generate a response (generate a listing of such lesions as well as the contours of the lesions and the anatomical structures) based on historical data (history data structure is used in AI pipeline))
train additional instances of the Al model with different loss functions (Palma Paragraph 0015; “a first machine learning model of the ensemble is trained with a first loss function that penalizes errors in false negative classifications of lesions, and a second machine learning model of the ensemble is trained with a second loss function, different from the first machine learning model…By having multiple machine learning models implementing different loss functions, each machine learning model may compensate for the weaknesses of the other machine learning models with regard to sensitivity and specificity.” Examiner notes that additional instances of the AI model (second machine learning model) is trained with different loss functions (a second loss function))
receive an input at runtime; (Palma Fig 1 and Paragraph 0026; “FIG. 1 is an example block diagram of an AI pipeline implementing multiple specifically configured and trained ML/DL computer models to perform anatomical structure identification and lesion detection in input medical image data in accordance with one illustrative embodiment;” Examiner notes that an input (input volume of medical images) is received in the AI pipeline at runtime (performing anatomical structure identification and lesion detection))
PNG
media_image1.png
731
534
media_image1.png
Greyscale
process the input with the Al model and the additional instances of the Al model to generate a control signal that comprises a determined response, [a determined upper bound, and a determined lower bound]; (Examiner references to previous mapping and Palma Paragraph 0113; “This AI pipeline 100 generated output may be provided to further downstream computing systems 180 for further processing and generation of representations of the anatomical structure of interest and any detected lesions present in the anatomical structure.” to show that the input (input volume of medical images) is processed (through the AI pipeline) with the AI model and the additional instances of the AI model (trained ML/DL computer models to perform) to generate a control signal (output 170 that may be provided to further downstream computing systems) that comprises a determined response (list of lesions with contours and categorization))
Palma does not teach to configure the additional instances of the Al model to generate an upper bound and a lower bound for the response;
[process the input with the Al model and the additional instances of the] Al model to generate a control signal that comprises [a determined response,] a determined upper bound, and a determined lower bound;
However, Fotak does teach to configure the additional instances of the Al model to generate an upper bound and a lower bound for the response; (Fotak Paragraph 0049; “the model (for a given model) to generate (create) a confidence interval for a prediction used for determining whether an actual value (the measured value) in the data is (or is not) an anomaly.” Examiner notes that each additional instances of the AI model (a given model) is used to generate an upper and lower bound (confidence interval) for the response (for a prediction))
[process the input with the Al model and the additional instances of the] Al model to generate a control signal that comprises [a determined response,] a determined upper bound, and a determined lower bound; (Fotak Paragraph 0049; “the model (for a given model) to generate (create) a confidence interval for a prediction used for determining whether an actual value (the measured value) in the data is (or is not) an anomaly.” Examiner notes that each Al model and the additional instances of the AI model (a given model) is used to generate an upper and lower bound (confidence interval))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma and Fotak. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. One of ordinary skill would have motivation to combine Palma and Fotak to reduce processing time for the detection of an anomaly “The improvement is a relatively faster processing time realized for the detection of an anomaly contained in the data (a volume of data, such as a vast quantity of data to be accumulated over a prolonged period of time, time-series data).” (Fotak Paragraph 0008).
Palma in view of Fotak does not teach determine a confidence score based on the control signal; and
output the determined response when the confidence score exceeds a threshold
However, Sugathan does teach determine a confidence score based on the control signal; and (Sugathan Column 5 Line 11; “the natural language processor 118 can calculate confidence scores for the one or more registered intents 128… The confidence score can be a value within a range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.” Examiner notes that a confidence score is determined/calculated based on the control signal comprising a determined response (one or more registered intents), upper, and lower bounds (lower bound of this range and upper bound of this range that the confidence score can be))
output the determined response when the confidence score exceeds a threshold. (Sugathan Column 5 Line 36; “the natural language processor 118 can return an indication of any registered intents 128 having confidence scores that meet or exceed a predefined threshold, as discussed above.” Examiner notes that the determined response (registered intents) is output/returned when the confidence score exceeds a threshold (confidence scores that meet or exceed a predefined threshold))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 23, Palma does not teach The apparatus of claim 21, wherein the at least one processor is configured to determine the confidence score based on a relationship between the determined response and a range defined by the determined upper bound and the determined lower bound.
However, Sugathan does teach The apparatus of claim 21, wherein the at least one processor is configured to determine the confidence score based on a relationship between the determined response and a range defined by the determined upper bound and the determined lower bound. (Sugathan Column 5 Line 11; “the natural language processor 118 can calculate confidence scores for the one or more registered intents 128… The confidence score can be a value within a range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.” Examiner notes that a confidence score is determined/calculated based on a relationship (range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.) between the determined response (one or more registered intents) and a range defined by upper, and lower bounds (lower bound of this range and upper bound of this range that the confidence score can be))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 24, Palma does not teach The apparatus of claim 21, wherein the at least one processor is further configured to suppress transmission of the determined response to a software application when the confidence score does not exceed the threshold.
However, Sugathan does teach The apparatus of claim 21, wherein the at least one processor is further configured to suppress transmission of the determined response to a software application when the confidence score does not exceed the threshold. (Sugathan Fig 2 and Column 17 line 38; “If the confidence score fails to meet or exceed the predefined threshold, the process can proceed to block 218.” Examiner notes that transmission of the determined response to a software application (provide identifier to first virtual agent) is suppressed/blocked when the confidence score does not exceed the threshold (If the confidence score fails to meet or exceed the predefined threshold))
PNG
media_image2.png
792
568
media_image2.png
Greyscale
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 26, Palma teach The apparatus of claim 21, wherein the at least one processor is configured to concurrently process the input based on execution of the AI model and the additional instances of the AI model to generate the control signal within a single runtime execution cycle. (Palma Fig 7 and Paragraph 0093; “The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention… two blocks shown in succession may, in fact, be executed substantially concurrently” Examiner notes that the input is processed (Step 710) concurrently based on execution of the AI model and the addition instances of the AI model (Fig 7 shows a first ML/DL model and second ML/DL model processing input) to generate the control signal (output final lesion prediction output) within a single runtime execution cycle (Fig 7 shows a start and end of execution cycle))
PNG
media_image3.png
761
351
media_image3.png
Greyscale
Regarding claim 27, Palma teaches The apparatus of claim 21, wherein the at least one processor is configured to transmit the determined response to a software application configured to perform an automated action based on the determined response. (Palma Paragraph 0058; “With regard to the illustrative embodiments, the ML/DL computer models of the AI pipeline are executed, after configuration and training through ML/DL training processes, and perform complex computer medical imaging analysis to detect anatomical structures in input medical images and generate outputs… the outputs can be provided to other downstream computer systems to perform additional artificial intelligence operations, such as treatment recommendations and other decision support operations based on the classifications, contours, and the like.” Palma Paragraph 0247; “The configuring of the computing device may also, or alternatively, comprise the providing of software applications stored in one or more storage devices and loaded into memory of a computing device” Examiner notes that the determined response (output of classifications, contours, and the like) is transmitted/provided downstream to a software application (software application in computing system downstream) to perform an automated action (to perform additional artificial intelligence operations) based on the determined response (based on the classifications, contours, and the like))
Regarding claim 28, Palma does not teach The apparatus of claim 21, wherein the at least one processor is further configured to at least one of enable and disable use of the determined response by downstream software processes based on the confidence score.
However, Sugathan does teach The apparatus of claim 21, wherein the at least one processor is further configured to at least one of enable and disable use of the determined response by downstream software processes based on the confidence score. (Sugathan Column 5 Line 36; “the natural language processor 118 can return an indication of any registered intents 128 having confidence scores that meet or exceed a predefined threshold, as discussed above.” Examiner notes that the determined response (registered intents) is output/enabled for use when the confidence score exceeds a threshold (confidence scores that meet or exceed a predefined threshold))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 29, Palma teaches The apparatus of claim 21, wherein the at least one processor is configured to process the input with an artificial intelligence (AI) agent that receives the input and outputs the control signal in response. (Palma Fig 1 and Paragraph 0026; “FIG. 1 is an example block diagram of an AI pipeline implementing multiple specifically configured and trained ML/DL computer models to perform anatomical structure identification and lesion detection in input medical image data in accordance with one illustrative embodiment;” Palma Paragraph 0113; “This AI pipeline 100 generated output may be provided to further downstream computing systems 180 for further processing and generation of representations of the anatomical structure of interest and any detected lesions present in the anatomical structure.” Examiner notes that the input (input medical image data) is processed with an artificial intelligence agent (ML/DL computer models in AI pipeline) that receives the input and outputs the control signal in response (generates output))
Regarding claim 30, Palma teaches A method comprising: training an artificial intelligence (AI) model to generate a response based on historical data; (Palma Paragraph 0079; “the listing of lesions, their classifications, and contours may be stored in a history data structure associated with a patient with which the volume of input CT medical images correspond such that multiple executions of the AI pipeline on different volumes of input CT medical images associated with that patient may be stored and evaluated over time.” Palma Paragraph 0081; “an automated AI pipeline comprising a plurality of configured and trained ML/DL computer models that implement various artificial intelligence operations for various stages of the AI pipeline… and generate a listing of such lesions as well as the contours of the lesions and the anatomical structures for further downstream computer processing of the AI generated information from the AI pipeline” Examiner notes that an artificial intelligence (Al) model is trained to generate a response (generate a listing of such lesions as well as the contours of the lesions and the anatomical structures) based on historical data (history data structure is used in AI pipeline))
training additional instances of the Al model with different loss functions (Palma Paragraph 0015; “a first machine learning model of the ensemble is trained with a first loss function that penalizes errors in false negative classifications of lesions, and a second machine learning model of the ensemble is trained with a second loss function, different from the first machine learning model…By having multiple machine learning models implementing different loss functions, each machine learning model may compensate for the weaknesses of the other machine learning models with regard to sensitivity and specificity.” Examiner notes that additional instances of the AI model (second machine learning model) is trained with different loss functions (a second loss function))
receiving an input at runtime; (Palma Fig 1 and Paragraph 0026; “FIG. 1 is an example block diagram of an AI pipeline implementing multiple specifically configured and trained ML/DL computer models to perform anatomical structure identification and lesion detection in input medical image data in accordance with one illustrative embodiment;” Examiner notes that an input (input volume of medical images) is received in the AI pipeline at runtime (performing anatomical structure identification and lesion detection))
PNG
media_image1.png
731
534
media_image1.png
Greyscale
processing the input with the Al model and the additional instances of the Al model to generate a control signal that comprises a determined response, [a determined upper bound, and a determined lower bound]; (Examiner references to previous mapping and Palma Paragraph 0113; “This AI pipeline 100 generated output may be provided to further downstream computing systems 180 for further processing and generation of representations of the anatomical structure of interest and any detected lesions present in the anatomical structure.” to show that the input (input volume of medical images) is processed (through the AI pipeline) with the AI model and the additional instances of the AI model (trained ML/DL computer models to perform) to generate a control signal (output 170 that may be provided to further downstream computing systems) that comprises a determined response (list of lesions with contours and categorization))
Palma does not teach to configure the additional instances of the Al model to generate an upper bound and a lower bound for the response;
[processing the input with the Al model and the additional instances of the] Al model to generate a control signal that comprises [a determined response,] a determined upper bound, and a determined lower bound;
However, Fotak does teach to configure the additional instances of the Al model to generate an upper bound and a lower bound for the response; (Fotak Paragraph 0049; “the model (for a given model) to generate (create) a confidence interval for a prediction used for determining whether an actual value (the measured value) in the data is (or is not) an anomaly.” Examiner notes that each additional instances of the AI model (a given model) is used to generate an upper and lower bound (confidence interval) for the response (for a prediction))
[processing the input with the Al model and the additional instances of the] Al model to generate a control signal that comprises [a determined response,] a determined upper bound, and a determined lower bound; (Fotak Paragraph 0049; “the model (for a given model) to generate (create) a confidence interval for a prediction used for determining whether an actual value (the measured value) in the data is (or is not) an anomaly.” Examiner notes that each Al model and the additional instances of the AI model (a given model) is used to generate an upper and lower bound (confidence interval))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma and Fotak. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. One of ordinary skill would have motivation to combine Palma and Fotak to reduce processing time for the detection of an anomaly “The improvement is a relatively faster processing time realized for the detection of an anomaly contained in the data (a volume of data, such as a vast quantity of data to be accumulated over a prolonged period of time, time-series data).” (Fotak Paragraph 0008).
Palma in view of Fotak does not teach determining a confidence score based on the control signal; and
outputting the determined response when the confidence score exceeds a threshold
However, Sugathan does teach determining a confidence score based on the control signal; and (Sugathan Column 5 Line 11; “the natural language processor 118 can calculate confidence scores for the one or more registered intents 128… The confidence score can be a value within a range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.” Examiner notes that a confidence score is determined/calculated based on the control signal comprising a determined response (one or more registered intents), upper, and lower bounds (lower bound of this range and upper bound of this range that the confidence score can be))
outputting the determined response when the confidence score exceeds a threshold. (Sugathan Column 5 Line 36; “the natural language processor 118 can return an indication of any registered intents 128 having confidence scores that meet or exceed a predefined threshold, as discussed above.” Examiner notes that the determined response (registered intents) is output/returned when the confidence score exceeds a threshold (confidence scores that meet or exceed a predefined threshold))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 32, Palma does not teach The method of claim 30, wherein the determining the confidence score comprises determining the confidence score based on a relationship between the determined response and a range defined by the determined upper bound and the determined lower bound.
However, Sugathan does teach The method of claim 30, wherein the determining the confidence score comprises determining the confidence score based on a relationship between the determined response and a range defined by the determined upper bound and the determined lower bound. (Sugathan Column 5 Line 11; “the natural language processor 118 can calculate confidence scores for the one or more registered intents 128… The confidence score can be a value within a range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.” Examiner notes that a confidence score is determined/calculated based on a relationship (range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.) between the determined response (one or more registered intents) and a range defined by upper, and lower bounds (lower bound of this range and upper bound of this range that the confidence score can be))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 33, Palma does not teach The method of claim 30, further comprising suppressing transmission of the determined response to a software application when the confidence score does not exceed the threshold.
However, Sugathan does teach The method of claim 30, further comprising suppressing transmission of the determined response to a software application when the confidence score does not exceed the threshold. (Sugathan Fig 2 and Column 17 line 38; “If the confidence score fails to meet or exceed the predefined threshold, the process can proceed to block 218.” Examiner notes that transmission of the determined response to a software application (provide identifier to first virtual agent) is suppressed/blocked when the confidence score does not exceed the threshold (If the confidence score fails to meet or exceed the predefined threshold))
PNG
media_image2.png
792
568
media_image2.png
Greyscale
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 35, Palma teach The method of claim 30, wherein the processing comprises concurrently processing the input based on execution of the AI model and the additional instances of the AI model to generate the control signal within a single runtime execution cycle. (Palma Fig 7 and Paragraph 0093; “The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention… two blocks shown in succession may, in fact, be executed substantially concurrently” Examiner notes that the input is processed (Step 710) concurrently based on execution of the AI model and the addition instances of the AI model (Fig 7 shows a first ML/DL model and second ML/DL model processing input) to generate the control signal (output final lesion prediction output) within a single runtime execution cycle (Fig 7 shows a start and end of execution cycle))
PNG
media_image3.png
761
351
media_image3.png
Greyscale
Regarding claim 36, Palma teaches The method of claim 30, wherein the outputting the determined response comprises transmitting the determined response to a software application configured to perform an automated action based on the determined response. (Palma Paragraph 0058; “With regard to the illustrative embodiments, the ML/DL computer models of the AI pipeline are executed, after configuration and training through ML/DL training processes, and perform complex computer medical imaging analysis to detect anatomical structures in input medical images and generate outputs… the outputs can be provided to other downstream computer systems to perform additional artificial intelligence operations, such as treatment recommendations and other decision support operations based on the classifications, contours, and the like.” Palma Paragraph 0247; “The configuring of the computing device may also, or alternatively, comprise the providing of software applications stored in one or more storage devices and loaded into memory of a computing device” Examiner notes that the determined response (output of classifications, contours, and the like) is transmitted/provided downstream to a software application (software application in computing system downstream) to perform an automated action (to perform additional artificial intelligence operations) based on the determined response (based on the classifications, contours, and the like))
Regarding claim 37, Palma does not teach The method of claim 30, further comprising at least one of enabling and disabling use of the determined response by downstream software processes based on the confidence score.
However, Sugathan does teach The method of claim 30, further comprising at least one of enabling and disabling use of the determined response by downstream software processes based on the confidence score. (Sugathan Column 5 Line 36; “the natural language processor 118 can return an indication of any registered intents 128 having confidence scores that meet or exceed a predefined threshold, as discussed above.” Examiner notes that the determined response (registered intents) is output/enabled for use when the confidence score exceeds a threshold (confidence scores that meet or exceed a predefined threshold))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 38, Palma teaches A computer program product, comprising: at least one computer-readable storage media; and program instructions stored on the at least one computer-readable storage media to perform operations comprising: (Palma Paragraph 0006; “a method is provided, in a data processing system, comprising at least one processor and at least one memory, the at least one memory comprising instructions executed by the at least one processor to implement a lesion detection and classification artificial intelligence (AI) pipeline”)
training an artificial intelligence (Al) model to generate a response based on historical data; (Palma Paragraph 0079; “the listing of lesions, their classifications, and contours may be stored in a history data structure associated with a patient with which the volume of input CT medical images correspond such that multiple executions of the AI pipeline on different volumes of input CT medical images associated with that patient may be stored and evaluated over time.” Palma Paragraph 0081; “an automated AI pipeline comprising a plurality of configured and trained ML/DL computer models that implement various artificial intelligence operations for various stages of the AI pipeline… and generate a listing of such lesions as well as the contours of the lesions and the anatomical structures for further downstream computer processing of the AI generated information from the AI pipeline” Examiner notes that an artificial intelligence (Al) model is trained to generate a response (generate a listing of such lesions as well as the contours of the lesions and the anatomical structures) based on historical data (history data structure is used in AI pipeline))
training additional instances of the Al model with different loss functions (Palma Paragraph 0015; “a first machine learning model of the ensemble is trained with a first loss function that penalizes errors in false negative classifications of lesions, and a second machine learning model of the ensemble is trained with a second loss function, different from the first machine learning model…By having multiple machine learning models implementing different loss functions, each machine learning model may compensate for the weaknesses of the other machine learning models with regard to sensitivity and specificity.” Examiner notes that additional instances of the AI model (second machine learning model) is trained with different loss functions (a second loss function))
receiving an input at runtime; (Palma Fig 1 and Paragraph 0026; “FIG. 1 is an example block diagram of an AI pipeline implementing multiple specifically configured and trained ML/DL computer models to perform anatomical structure identification and lesion detection in input medical image data in accordance with one illustrative embodiment;” Examiner notes that an input (input volume of medical images) is received in the AI pipeline at runtime (performing anatomical structure identification and lesion detection))
PNG
media_image1.png
731
534
media_image1.png
Greyscale
process the input with the Al model and the additional instances of the Al model to generate a control signal that comprises a determined response, [a determined upper bound, and a determined lower bound]; (Examiner references to previous mapping and Palma Paragraph 0113; “This AI pipeline 100 generated output may be provided to further downstream computing systems 180 for further processing and generation of representations of the anatomical structure of interest and any detected lesions present in the anatomical structure.” to show that the input (input volume of medical images) is processed (through the AI pipeline) with the AI model and the additional instances of the AI model (trained ML/DL computer models to perform) to generate a control signal (output 170 that may be provided to further downstream computing systems) that comprises a determined response (list of lesions with contours and categorization))
Palma does not teach to configure the additional instances of the Al model to generate an upper bound and a lower bound for the response;
[processing the input with the Al model and the additional instances of the] Al model to generate a control signal that comprises [a determined response,] a determined upper bound, and a determined lower bound;
However, Fotak does teach to configure the additional instances of the Al model to generate an upper bound and a lower bound for the response; (Fotak Paragraph 0049; “the model (for a given model) to generate (create) a confidence interval for a prediction used for determining whether an actual value (the measured value) in the data is (or is not) an anomaly.” Examiner notes that each additional instances of the AI model (a given model) is used to generate an upper and lower bound (confidence interval) for the response (for a prediction))
[processing the input with the Al model and the additional instances of the] Al model to generate a control signal that comprises [a determined response,] a determined upper bound, and a determined lower bound; (Fotak Paragraph 0049; “the model (for a given model) to generate (create) a confidence interval for a prediction used for determining whether an actual value (the measured value) in the data is (or is not) an anomaly.” Examiner notes that each Al model and the additional instances of the AI model (a given model) is used to generate an upper and lower bound (confidence interval))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma and Fotak. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. One of ordinary skill would have motivation to combine Palma and Fotak to reduce processing time for the detection of an anomaly “The improvement is a relatively faster processing time realized for the detection of an anomaly contained in the data (a volume of data, such as a vast quantity of data to be accumulated over a prolonged period of time, time-series data).” (Fotak Paragraph 0008).
Palma in view of Fotak does not teach determining a confidence score based on the control signal; and
outputting the determined response when the confidence score exceeds a threshold
However, Sugathan does teach determining a confidence score based on the control signal; and (Sugathan Column 5 Line 11; “the natural language processor 118 can calculate confidence scores for the one or more registered intents 128… The confidence score can be a value within a range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.” Examiner notes that a confidence score is determined/calculated based on the control signal comprising a determined response (one or more registered intents), upper, and lower bounds (lower bound of this range and upper bound of this range that the confidence score can be))
outputting the determined response when the confidence score exceeds a threshold. (Sugathan Column 5 Line 36; “the natural language processor 118 can return an indication of any registered intents 128 having confidence scores that meet or exceed a predefined threshold, as discussed above.” Examiner notes that the determined response (registered intents) is output/returned when the confidence score exceeds a threshold (confidence scores that meet or exceed a predefined threshold))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Regarding claim 40, Palma does not teach The computer program product of claim 38, wherein the determining the confidence score comprises determining the confidence score based on a relationship between the determined response and a range defined by the determined upper bound and the determined lower bound.
However, Sugathan does teach The computer program product of claim 38, wherein the determining the confidence score comprises determining the confidence score based on a relationship between the determined response and a range defined by the determined upper bound and the determined lower bound. (Sugathan Column 5 Line 11; “the natural language processor 118 can calculate confidence scores for the one or more registered intents 128… The confidence score can be a value within a range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.” Examiner notes that a confidence score is determined/calculated based on a relationship (range of values, where a lower bound of this range indicates a certainty that the request fails to match the registered intent 128, and an upper bound of the range indicates a certainty that the request does match the intent 128.) between the determined response (one or more registered intents) and a range defined by upper, and lower bounds (lower bound of this range and upper bound of this range that the confidence score can be))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, and Sugathan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. One of ordinary skill would have motivation to combine Palma, Fotak, and Sugathan to be more efficient in fulfilling requests “any particular virtual agent can allow a user's request to be fulfilled, even if the particular virtual agent is itself incapable of fulfilling the request.” (Sugathan Column 2 Line 14).
Claim(s) 22, 31, and 39 are rejected under 35 U.S.C. 103 as being unpatentable over Giovanni John Jacques Palma et al; US 20220138931 A1 filed on Oct 30, 2020 (hereinafter “Palma”) in view of Meghan Frances Fotak et al; US 20220200878 A1 filed on Jul 21, 2021 (hereinafter “Fotak”) in further view of Man Mohan Sugathan et al; US 12321868 B1 filed on Oct 29, 2021 in further view of Brian Teixeira et al; US 20190057515 A1 filed on Aug 7, 2018 (hereinafter “Teixeira”) in further view of Hossein Honarvargheitanbaf et al; US 20250022596 A1 filed on Nov 28, 2022 (hereinafter “Honarvargheitanbaf”).
Regarding claim 22, Palma does not teach The apparatus of claim 21, wherein the at least one processor is configured to retrain a common base model architecture with [asymmetric] loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model.
However, Teixeira does teach The apparatus of claim 21, wherein the at least one processor is configured to retrain a common base model architecture with [asymmetric] loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model. (Teixeira Paragraph 0055; “This model is thus trained once with MSE, then with joint loss, and then with MSE.” Teixeira Paragraph 0056; “MSE and joint loss are sequentially used to re-train. The initial and final training may use MSE but joint loss for both initial and final or joint for one and MSE for another may be used in other approaches. Other standard losses may be used. Additional or different losses may be used, such as iterating training through three different losses including joint loss in any pattern.” Examiner notes that a common base model architecture (this model) is retrained with loss function (different losses may be used) different than a loss function used to train the AI model (mode is trained with MSE); loss functions penalize overestimation and underestimation)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, and Teixeira. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Teixeira teaches using machine learning to predict the location of an internal body marker from surface data. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, and Teixeira to improve model prediction from using other standard loss “Even though this joint loss results in better prediction, pretraining with MSE or other standard loss may further improve prediction.” (Teixeira).
Palma in view of Teixeira does not teach asymmetric loss functions
However, Honarvargheitanbaf does teach asymmetric loss functions (Honarvargheitanbaf Paragraph 0199; “the training of the neural network comprises adjusting one or more parameters in the plurality of parameters by back-propagation through a loss function... Non-limiting examples of loss functions suitable for the regression task… quantile loss function.” Examiner notes that loss function used for retraining can be asymmetric loss functions (quantile loss function))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, Teixeira, and Honarvargheitanbaf. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Teixeira teaches using machine learning to predict the location of an internal body marker from surface data. Honarvargheitanbaf teaches a method for determining whether a subject has a cardiovascular abnormality are provided. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, Teixeira, and Honarvargheitanbaf to improve neural network model by substituting/adjusting loss functions “the training of the neural network comprises adjusting one or more parameters in the plurality of parameters by back-propagation through a loss function. In some embodiments, the loss function is a regression task and/or a classification task. Non-limiting examples of loss functions suitable for the regression task include, but are not limited to, a mean squared error loss function, a mean absolute error loss function, a Huber loss function, a Log-Cosh loss function, or a quantile loss function.” (Honarvargheitanbaf Paragraph 0199).
Regarding claim 31, Palma does not teach The method of claim 30, wherein the at least one processor is configured to retrain a common base model architecture with [asymmetric] loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model.
However, Teixeira does teach The method of claim 30, wherein the at least one processor is configured to retrain a common base model architecture with [asymmetric] loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model. (Teixeira Paragraph 0055; “This model is thus trained once with MSE, then with joint loss, and then with MSE.” Teixeira Paragraph 0056; “MSE and joint loss are sequentially used to re-train. The initial and final training may use MSE but joint loss for both initial and final or joint for one and MSE for another may be used in other approaches. Other standard losses may be used. Additional or different losses may be used, such as iterating training through three different losses including joint loss in any pattern.” Examiner notes that a common base model architecture (this model) is retrained with loss function (different losses may be used) different than a loss function used to train the AI model (mode is trained with MSE); loss functions penalize overestimation and underestimation)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, and Teixeira. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Teixeira teaches using machine learning to predict the location of an internal body marker from surface data. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, and Teixeira to improve model prediction from using other standard loss “Even though this joint loss results in better prediction, pretraining with MSE or other standard loss may further improve prediction.” (Teixeira).
Palma in view of Teixeira does not teach asymmetric loss functions
However, Honarvargheitanbaf does teach asymmetric loss functions (Honarvargheitanbaf Paragraph 0199; “the training of the neural network comprises adjusting one or more parameters in the plurality of parameters by back-propagation through a loss function... Non-limiting examples of loss functions suitable for the regression task… quantile loss function.” Examiner notes that loss function used for retraining can be asymmetric loss functions (quantile loss function))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, Teixeira, and Honarvargheitanbaf. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Teixeira teaches using machine learning to predict the location of an internal body marker from surface data. Honarvargheitanbaf teaches a method for determining whether a subject has a cardiovascular abnormality are provided. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, Teixeira, and Honarvargheitanbaf to improve neural network model by substituting/adjusting loss functions “the training of the neural network comprises adjusting one or more parameters in the plurality of parameters by back-propagation through a loss function. In some embodiments, the loss function is a regression task and/or a classification task. Non-limiting examples of loss functions suitable for the regression task include, but are not limited to, a mean squared error loss function, a mean absolute error loss function, a Huber loss function, a Log-Cosh loss function, or a quantile loss function.” (Honarvargheitanbaf Paragraph 0199).
Regarding claim 39, Palma does not teach The computer program product of claim 38, wherein the training the additional instances of the AI model comprises retraining a common base model architecture with [asymmetric] loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model.
However, Teixeira does teach The computer program product of claim 38, wherein the training the additional instances of the AI model comprises retraining a common base model architecture with [asymmetric] loss functions that penalize overestimation and underestimation differently than a loss function used to train the Al model. (Teixeira Paragraph 0055; “This model is thus trained once with MSE, then with joint loss, and then with MSE.” Teixeira Paragraph 0056; “MSE and joint loss are sequentially used to re-train. The initial and final training may use MSE but joint loss for both initial and final or joint for one and MSE for another may be used in other approaches. Other standard losses may be used. Additional or different losses may be used, such as iterating training through three different losses including joint loss in any pattern.” Examiner notes that a common base model architecture (this model) is retrained with loss function (different losses may be used) different than a loss function used to train the AI model (mode is trained with MSE); loss functions penalize overestimation and underestimation)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, and Teixeira. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Teixeira teaches using machine learning to predict the location of an internal body marker from surface data. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, and Teixeira to improve model prediction from using other standard loss “Even though this joint loss results in better prediction, pretraining with MSE or other standard loss may further improve prediction.” (Teixeira).
Palma in view of Teixeira does not teach asymmetric loss functions
However, Honarvargheitanbaf does teach asymmetric loss functions (Honarvargheitanbaf Paragraph 0199; “the training of the neural network comprises adjusting one or more parameters in the plurality of parameters by back-propagation through a loss function... Non-limiting examples of loss functions suitable for the regression task… quantile loss function.” Examiner notes that loss function used for retraining can be asymmetric loss functions (quantile loss function))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, Teixeira, and Honarvargheitanbaf. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Teixeira teaches using machine learning to predict the location of an internal body marker from surface data. Honarvargheitanbaf teaches a method for determining whether a subject has a cardiovascular abnormality are provided. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, Teixeira, and Honarvargheitanbaf to improve neural network model by substituting/adjusting loss functions “the training of the neural network comprises adjusting one or more parameters in the plurality of parameters by back-propagation through a loss function. In some embodiments, the loss function is a regression task and/or a classification task. Non-limiting examples of loss functions suitable for the regression task include, but are not limited to, a mean squared error loss function, a mean absolute error loss function, a Huber loss function, a Log-Cosh loss function, or a quantile loss function.” (Honarvargheitanbaf Paragraph 0199).
Claim(s) 25 and 34 are rejected under 35 U.S.C. 103 as being unpatentable over Giovanni John Jacques Palma et al; US 20220138931 A1 filed on Oct 30, 2020 (hereinafter “Palma”) in view of Meghan Frances Fotak et al; US 20220200878 A1 filed on Jul 21, 2021 (hereinafter “Fotak”) in further view of Man Mohan Sugathan et al; US 12321868 B1 filed on Oct 29, 2021 in further view of Tristan Jehan et al; US 20180129659 A1 filed on Jun 8, 2017 (hereinafter “Jehan”).
Regarding claim 25, Palma does not teach The apparatus of claim 21, wherein the at least one processor is further configured to determine the threshold based on historical behavior associated with prior predicted responses generated by the AI model.
However, Jehan does teach The apparatus of claim 21, wherein the at least one processor is further configured to determine the threshold based on historical behavior associated with prior predicted responses generated by the AI model. (Jehan Paragraph 0151; “The Bayesian model may comprise a probabilistic graphical model comprising joint probabilities based on various prior knowledge and the output of the statistical model. The joint probability may then be used to, for example, determine appropriate threshold values for the scores from the statistical model based on the prior knowledge.” Examiner notes that a threshold (appropriate threshold values) is determined based on historical behavior associated with prior predicted responses (based on various prior knowledge and the output of the statistical model) generated by the AI model (Bayesian model))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, and Jehan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Jehan teaches methods for identifying media content using indirect qualities are provided. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, and Jehan to determine appropriate threshold values to improve performance of the model “model parameters in either the statistical model or the Bayesian model are adjusted to improve performance on the validation set.” (Jehan Paragraph 0162).
Regarding claim 34, Palma does not teach The method of claim 30, further comprising determining the threshold based on historical behavior associated with prior predicted responses generated by the AI model.
However, Jehan does teach The method of claim 30, further comprising determining the threshold based on historical behavior associated with prior predicted responses generated by the AI model. (Jehan Paragraph 0151; “The Bayesian model may comprise a probabilistic graphical model comprising joint probabilities based on various prior knowledge and the output of the statistical model. The joint probability may then be used to, for example, determine appropriate threshold values for the scores from the statistical model based on the prior knowledge.” Examiner notes that a threshold (appropriate threshold values) is determined based on historical behavior associated with prior predicted responses (based on various prior knowledge and the output of the statistical model) generated by the AI model (Bayesian model))
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Palma, Fotak, Sugathan, and Jehan. Palma teaches A lesion detection and classification artificial intelligence (AI) pipeline comprising a plurality of trained machine learning (ML) computer models is provided. Fotak teaches a method for detecting at least one anomaly contained in the data that was received. Sugathan teaches embodiments for fulfilling requests through a registry of virtual agents. Jehan teaches methods for identifying media content using indirect qualities are provided. One of ordinary skill would have motivation to combine Palma, Fotak, Sugathan, and Jehan to determine appropriate threshold values to improve performance of the model “model parameters in either the statistical model or the Bayesian model are adjusted to improve performance on the validation set.” (Jehan Paragraph 0162).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL DUC TRAN whose telephone number is (571)272-6870. The examiner can normally be reached Mon-Fri 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.D.T./Examiner, Art Unit 2147 /ERIC NILSSON/Primary Examiner, Art Unit 2151
Prosecution Insights