DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/23/2026 has been entered. Claims 1-8 and 10-21 remain pending in the current application.
Claim Objections
Claims 1, 13, and 15 are objected to because of the following informalities: claims 1, 13, and 15 recite the limitation “the plurality of recommended next steps based on the number of configurable option” should read “the plurality of recommended next steps based on a number of the plurality of configurable option”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8 and 10-21 are rejected under 35 U.S.C. 103 as being unpatentable over Canfield et al. (UA 2021/0137416) in the view of Zaho et al. (NPL: “ChatCAD+: towards a universal and reliable interactive CAD using LLMs”) and Huang et al. (CN 116580801).
Regarding claim 1, Canfield teaches an ultrasound imaging system comprising (figure 1, para. 0027):
an image processing circuit configured to receive image data obtained using an ultrasound probe, and perform an image recognition process to identify an image characteristic based on the image data (para. 0027; The system can also include a data processor 126, e.g., a computational module or circuity, configured to implement a first neural network 128. The first neural network 128 can be configured to receive the image frames 124, either directly from the signal processor 122 or via the data processor 126, determine a presence, absence and/or identity of at least one anatomical feature within each frame, and classify an image view, e.g., a current image view, based on this determination.);
an entity recognition circuit configured to convert the identified image characteristic into an artificial intelligence (AI) model input, the Al model input including a written description of the image characteristic and a current status of a workflow process (paras. 0032-0033; The data processor 126 can be configured to perform multiple functions. As mentioned above, the data processor 126 can be configured to implement a neural network 128, which can be configured to classify images into distinct categories, for example “full view,” “head,” “abdominal,” “chest,” or “extremities.” Sub-categories can include, for example, “stomach,” “bowel,” “umbilical cord,” “kidney,” “bladder,” “legs,” “arms,” “hands,” “femur,” “spine,” “heart,” “lungs,” “stomach,” “bowel,” “umbilical cord,” “kidney,” or “bladder.” Classification results determined by the neural network 128 can be adaptive to a current ultrasound region of interest and/or the completed measurements within the prenatal assessment protocol. For example, if the region of interest is large and includes multiple sub-categories of anatomical features, such as the kidney, liver and umbilical cord, the neural network 128 may classify the current image as “abdominal,” along with an indication of suggested features and/or measurements thereof to be obtained within the abdominal region. Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The examiner notes that the first neural network inputs the image and output a classification of the image based on the anatomical feature, scan view, and current completed measurement within the assessment protocol. The neural network outputs a written description of the organ and the current view and current completed step in the protocol. Thus, the current status of the workflow is outputted along with a description of the image to determine the recommended next step to be followed in accordance with the protocol.);
an AI circuit configured to receive contextual information, the contextual information comprising procedure-specific information (para. 0033; For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. The examiner notes that the AI circuit receives procedure specific information such as a list of measurements to be obtained in accordance with a stored scan protocol.);
the AI circuit configured to receive the AI model input and a prompt, the prompt including a plurality of configurable options (figure 6, paras. 0033 and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600, input requesting additional instructions or assistance for performing an ultrasound scan, and/or input requesting that one or more ultrasound images be saved and/or transmitted to a remote receiver. The examiner notes that the neural network receives inputs from the first neural network and user input prompting the neural network to generate additional recommendations. The second neural network output recommended next steps based on the first neural network input and the user request);
apply the AI model input and the contextual information to an Al model, and output an Al model output corresponding to a plurality of recommended next steps of the workflow process, the plurality of recommended next steps based on the number of configurable options (paras. 0033, 0038-0039, and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The second neural network 130 can be configured to provide suggestions for the next measurement that can or should be obtained. The network 130 can implement a recommender system, such as that shown in FIG. 4. The next measurement can be recommended based on one or more factors. For example, the next recommended measurement may be the measurement that is obtainable by implementing the smallest or most minor adjustments to the ultrasound transducer used to obtain the ultrasound images. The adjustments can include operating parameters, e.g., focal depth, or the position and/or orientation of the ultrasound transducer. By recommending measurements in this manner, a user may quickly and efficiently progress through a scan by minimizing the extent of imaging adjustments required to obtain successive measurements. In addition or alternatively, the next recommended measurement can be the measurement that is obtainable at or above an accuracy threshold. The examiner notes that the second neural network output suggestions for the recommended next measurements based on the input received by the first neural network and any additional requests received by the user.);
the entity recognition circuit configured to convert the AI model output into a control signal corresponding to the plurality of recommended next steps (para. 0029; In examples where a mechanical adjustment mechanism 142, e.g., a robotic arm, is used to control the position and/or orientation of the data acquisition unit 110, the instructions for acquiring image data may be communicated to a controller 144 configured to cause the adjustment mechanism to make the necessary adjustments automatically, without user input. The controller 144 may be configured to adjust the position, orientation and/or operational settings of the data acquisition unit 110 as part of an automatic feedback loop with the data processor 126. In some examples, the controller 144 can receive information regarding a current position of the ultrasound sensor array 112 and a direction the array needs to move or turn, which may be based at least in part on information received from the first neural network 128 and/or second neural network 130, along with information regarding a current fetal position (FIG. 2).); and
a control circuit configured: to receive the control signal (para. 0029; In examples where a mechanical adjustment mechanism 142, e.g., a robotic arm, is used to control the position and/or orientation of the data acquisition unit 110, the instructions for acquiring image data may be communicated to a controller 144 configured to cause the adjustment mechanism to make the necessary adjustments automatically, without user input. The controller 144 may be configured to adjust the position, orientation and/or operational settings of the data acquisition unit 110 as part of an automatic feedback loop with the data processor 126. In some examples, the controller 144 can receive information regarding a current position of the ultrasound sensor array 112 and a direction the array needs to move or turn, which may be based at least in part on information received from the first neural network 128 and/or second neural network 130, along with information regarding a current fetal position (FIG. 2).);
receive an operator preference including a preference related to the plurality of recommended next steps (para. 0034; The user input 140 received at the user interface 134 can be in the form of a manual confirmation that a particular measurement has been obtained. In some embodiments, the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings.);
update the plurality of recommended next steps based on the operator preference (paras. 0027, 0029, and 0034; the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. In some examples, the scan protocol 138 may convey the presence of an anatomical feature within a current image frame 124, and in some embodiments, whether the image of such feature is sufficient to accurately measure the feature or whether additional images of the feature are needed. The user interface 134 can be configured to display the ultrasound images 136 of the region 116 in real time as an ultrasound scan is being performed, along with the adaptive scan protocol 138. The user interface 134 may also be configured to receive a user input 140 at any time before, during, or after an ultrasound scan. For example, the user interface 134 may be interactive, receiving user input 140 indicating confirmation that an anatomical feature has been assessed, and adapting a display responsive to the input. As further shown, the scan protocol 138 may also be input into the user interface 134. The user input 140 received at the user interface 134 can be in the form of a manual confirmation that a particular measurement has been obtained. In some embodiments, the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings. The examiner notes that the displayed recommended next steps are updated based on user input such as the user rejecting the recommended step.); and
control a display of the ultrasound imaging system, wherein controlling the display comprises causing the display to display the updated plurality of recommended next steps for an operator of the ultrasound imaging system to perform (paras. 0027, 0029, and 0034; the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. In some examples, the scan protocol 138 may convey the presence of an anatomical feature within a current image frame 124, and in some embodiments, whether the image of such feature is sufficient to accurately measure the feature or whether additional images of the feature are needed. The user interface 134 can be configured to display the ultrasound images 136 of the region 116 in real time as an ultrasound scan is being performed, along with the adaptive scan protocol 138. The user interface 134 may also be configured to receive a user input 140 at any time before, during, or after an ultrasound scan. For example, the user interface 134 may be interactive, receiving user input 140 indicating confirmation that an anatomical feature has been assessed, and adapting a display responsive to the input. As further shown, the scan protocol 138 may also be input into the user interface 134. The user input 140 received at the user interface 134 can be in the form of a manual confirmation that a particular measurement has been obtained. In some embodiments, the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings. The examiner notes that the displayed recommended next steps are updated based on user input such as the user rejecting the recommended step or based on the completion of the current recommended step.); and
the Al circuit being further configured to receive a second model input including an updated status of the workflow process based on a performed next step and output a second Al model output corresponding to an updated plurality of recommended next steps, such that the control circuit is configured to cause the display to display the updated plurality of recommended next steps based on the updated status (paras. 0027, 0029, and 0034; some examples, the system 100 also includes a display processor 132 coupled with the data processor and a user interface 134. The display processor 132 can link the neural networks 128, 130 to the user interface 134, enabling the neural network outputs to be displayed on or modify the information displayed on the user interface. In various embodiments, the user interface 134 may receive the outputs directly from the second neural network 130, or after additional processing via the data processor 132. In some examples, the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. The systems herein overcome this problem by employing an imaging system coupled with deep learning and an intuitive user interface configured to provide an adaptive scan protocol that is responsive to image data acquired during a scan and the movement and current position of the fetus being evaluated. With reference to FIG. 1, the system 100 enables users to perform effective prenatal assessments by identifying anatomical features within acquired ultrasound image frames 124 and providing instructions to the users for acquiring image data for the next required anatomical feature specified in a worklist, which may be updated and displayed on the user interface 134. The user interface 134 may be configured to display and update the adaptive scan protocol 138 in real time as an ultrasound scan is being performed. In some examples, the user interface 134 may be further configured to display instructions 139 for adjusting the data acquisition unit 110 in the manner necessary to obtain the next recommended measurements. The examiner notes that the system runs in a loop where it receives images and process them and feed the output of the first neural network to the second neural network which outputs an updated recommended steps based on the completion of the previous step or a user input.).
However, Canfield fails to explicitly teach AI circuit comprising a retrieval model configured to receive contextual information, the contextual information comprising patient medical history.
However, Canfield fails to teach an AI circuit comprising a retrieval model configured to receive contextual information comprising patient medical history.
Zaho, in the same field of endeavor, teaches an AI circuit comprising a retrieval model configured to receive contextual information comprising medical information (page 2, left col; ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently. (2) Hierarchical in-context learning for enhanced report generation. Top-k reports that are semantically similar to the LLM-generated report are retrieved from a clinic database via the proposed retrieval module (c.f. Fig. 3. The retrieved k reports then serve as in-context examples to refine the LLM-generated report. (3) Knowledge-based reliable interaction. As illustrated in Fig. 1(b), ChatCAD+ does not directly provide medical advice. Instead, it first seeks help via our proposed knowledge retrieval module for obtaining relevant knowledge from professional sources, e.g. Merck Manuals, Mayo Clinic, and Cleveland Clinic. Then, the LLM considers the retrieved knowledge as a reference to provide reliable medical advice.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a retrieval-augmented generation model because it will help retrieve clinically-sound knowledge from a professional knowledge database, thereby serving as a reliable reference for the AI model to enhance the reliability of its response as disclosed within Zaho in page 3. Additionally, doing so would enhance the accuracy of the output of the AI model.
However, Canfield in the view of Zaho fail to disclose that the contextual information include patient medical history.
Huang, in the same field of endeavor, teaches retrieving contextual information include patient medical history (paras. 59-62; For the patient to be examined, extract the self-report information and historical medical treatment information of the patient to be examined as pre-diagnosis information; Process the pre-diagnosis information to obtain input information. Input information to the external large-scale language model to obtain scan suggestions fed back by the large-scale language model. The scan suggestions are used to obtain scan images by performing ultrasound scans on patients to be examined.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the contextual information of Canfield in the view of Zaho with the contextual information of Huang to provide contextual information include patient medical history because it will help understand the actual situation of the patient and obtain more accurate feedback information to achieve better quality control of the scanning process as disclosed within Huang in para. 63.
Regarding claim 2, Canfield teaches the ultrasound imaging system of claim 1, wherein the plurality of recommended next steps comprise at least one of a recommendation to rotate or move the ultrasound probe a certain way, a recommendation to activate a particular mode, or a recommendation to take a particular measurement (para. 0048; The guidance can be generated by the second neural network 130 in the form of one or more instructions 518 for adjusting an ultrasound transducer in a manner necessary to obtain the next recommended image and/or measurement. For instance, if the head of a fetus has been most recently measured, the next recommended measurement may be of the abdominal region. To comply with this recommendation, the user interface 500 may display instructions 518 for adjusting an ultrasound transducer in a manner that enables images of the abdominal region to be obtained. Instructions may include directional commands, e.g., “Move ultrasound probe laterally,” and/or technique-based commands, e.g., “Move ultrasound probe slower”; “Slow down”; “Stop”; “or “Continue.” In some embodiments, the instructions may comprise modifications of one or more image acquisition parameters. For example, the user interface 500 may provide instructions 518 to alter the imaging plane, the focal depth and/or the pulse frequency of the transducer. In the event that an abnormality is detected, the user interface 500 may provide an instruction to hold the transducer steady at one location, thereby allowing further analysis. Slight adjustments in the imaging angle may also be recommended to more thoroughly characterize a detected abnormality.).
Regarding claim 3, Canfield teaches the ultrasound imaging system of claim 1, where the control circuit is configured to change an operating characteristic of the ultrasound imaging system based on receiving a user input selecting one of the plurality of recommended next steps (paras. 0029 and 0034, 0048, and 0051; the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings. the user interface 500 may display instructions 518 for adjusting an ultrasound transducer in a manner that enables images of the abdominal region to be obtained. Instructions may include directional commands, e.g., “Move ultrasound probe laterally,” and/or technique-based commands, e.g., “Move ultrasound probe slower”; “Slow down”; “Stop”; “or “Continue.” In some embodiments, the instructions may comprise modifications of one or more image acquisition parameters. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600. The examiner notes that the recommendation includes a list of option that include probe movement, parameter adjustment, measurements to obtain, etc. The user selects one of the recommended options using the user interface and a prompt is sent to the system to adjust parameters based on the user selection.).
Regarding claim 4, Canfield teaches the ultrasound imaging system of claim 3, wherein changing the operating characteristic of the ultrasound imaging system comprises changing at least one of a mode, a view, an acquisition parameter, or a measurement setting (paras. 0034, 0048, and 0051; the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings. the user interface 500 may display instructions 518 for adjusting an ultrasound transducer in a manner that enables images of the abdominal region to be obtained. Instructions may include directional commands, e.g., “Move ultrasound probe laterally,” and/or technique-based commands, e.g., “Move ultrasound probe slower”; “Slow down”; “Stop”; “or “Continue.” In some embodiments, the instructions may comprise modifications of one or more image acquisition parameters. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600. The examiner notes that the recommendation includes a list of option that include probe movement, parameter adjustment, measurements to obtain, etc. The user selects one of the recommended options using the user interface and a prompt is sent to the system to adjust parameters based on the user selection.).
Regarding claim 5, Canfield teaches the ultrasound imaging system of claim 1, wherein the control circuit is configured to display the plurality of recommended next steps in real-time with respect to the image processing circuit receiving the image data (paras. 0027, 0029, and 0034; The system 100 may also include a display processor 132 coupled with the data processor 126 and the user interface 134. In various embodiments, the display processor 132 can be configured to generate ultrasound images 136 from the image frames 124 and an adaptive scan protocol 138 that includes a list of required fetal measurements, each of which may be accompanied by a status indicator showing whether or not each measurement has been obtained. The user interface 134 may be configured to display and update the adaptive scan protocol 138 in real time as an ultrasound scan is being performed. In some examples, the user interface 134 may be further configured to display instructions 139 for adjusting the data acquisition unit 110 in the manner necessary to obtain the next recommended measurements.).
Regarding claim 6, Canfield teaches the ultrasound imaging system of claim 1, wherein the image characteristic comprises at least one of a mode, a view, an identification of an imaged structure, an identification of a pathology, an acquisition parameter, or a measurement setting (para. 0027; The first neural network 128 can be configured to receive the image frames 124, either directly from the signal processor 122 or via the data processor 126, determine a presence, absence and/or identity of at least one anatomical feature within each frame, and classify an image view, e.g., a current image view, based on this determination.).
Regarding claim 7, Canfield teaches the ultrasound imaging system of claim 1, however, fails to explicitly teach wherein the AI model input comprises a natural language input.
Zaho, in the same field of endeavor, teaches wherein the AI model input comprises a natural language input (page 2, left col; ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI input of Canfield with the AI input of Zaho to provide a natural language input because it provides clinically relevant information in a manner that is more easily interpretable by AI model and humans as disclosed within Zaho in page 4.
Regarding claim 8, Canfield teaches the ultrasound imaging system of claim 1, however, fails to explicitly teach wherein the AI model comprises a large language model.
Zaho, in the same field of endeavor, teaches the AI model comprises a large language model (pages 2-3; ChatCAD [9] linked existing CAD models with LLMs to enhance diagnostic accuracy and improve patient care.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a large language model because it demonstrates extraordinary capabilities in understanding and generating human-like text as disclosed within Zaho in page 1. Additionally, doing so would help in translating image features into clear context aware and adaptive guidance that a user can easily understand and follow.
Regarding claim 10, Canfield teaches the ultrasound imaging system of claim 8, however, fails to explicitly teach wherein applying the AI model input to the AI model comprises: querying the retrieval model configured to search a medical information database for data relating to the AI model input; combining the data relating to the AI model input with the AI model input to create an augmented AI model input; applying the augmented AI model input to the large language model; and generating the AI model output.
Zaho, in the same field of endeavor, teaches applying the AI model input to the AI model comprises: querying the retrieval model configured to search a medical information database for data relating to the AI model input; combining the data relating to the AI model input with the AI model input to create an augmented AI model input; applying the augmented AI model input to the large language model; and generating the AI model output (pages 2-4; (1) Universal image interpretation. Due to the difficulty in obtaining a unified CAD network tackling various images currently, ChatCAD+ incorporates a domain identification module to work with a variety of CAD models (c.f. Fig. 2(a)). ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently. (2) Hierarchical incontext learning for enhanced report generation. Top-k reports that are semantically similar to the LLM-generated report are retrieved from a clinic database via the proposed retrieval module (c.f. Fig. 3. The retrieved k reports then serve as in-context examples to refine the LLM-generated report. (3) Knowledge-based reliable interaction. As illustrated in Fig. 1(b), ChatCAD+ does not directly provide medical advice. Instead, it first seeks help via our proposed knowledge retrieval module for obtaining relevant knowledge from professional sources, e.g. Merck Manuals, Mayo Clinic, and Cleveland Clinic. Then, the LLM considers the retrieved knowledge as a reference to provide reliable medical advice. The examiner notes that the system first converts image features to text description which will then be inputted into large language model, the LLM uses retrieval module to retrieve medical information relative to the identified feature and use it as an additional input to generate a report).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a retrieval-augmented generation model and a large language model because it will help retrieve clinically-sound knowledge from a professional knowledge database, thereby serving as a reliable reference for the AI model to enhance the reliability of its response as disclosed within Zaho in page 3. Additionally, doing so would enhance the accuracy of the output of the AI model.
Regarding claim 11, Canfield teaches the ultrasound imaging system of claim 10, however, fails to explicitly teach wherein the medical information database comprises at least one of clinical guidelines, standard practices, medical literature, medical textbooks, published research, or previous case studies related to the AI model input.
Zaho, in the same field of endeavor, teaches wherein the medical information database comprises at least one of clinical guidelines, standard practices, medical literature, medical textbooks, published research, or previous case studies related to the AI model input (page 2, (3) Knowledge-based reliable interaction. As illustrated in Fig. 1(b), ChatCAD+ does not directly provide medical advice. Instead, it first seeks help via our proposed knowledge retrieval module for obtaining relevant knowledge from professional sources, e.g. Merck Manuals, Mayo Clinic, and Cleveland Clinic. Then, the LLM considers the retrieved knowledge as a reference to provide reliable medical advice. The examiner notes that the system first converts image features to text description which will then be inputted into large language model, the LLM uses retrieval module to retrieve medical information relative to the identified feature and use it as an additional input to generate a report).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a retrieval-augmented generation model because it will help retrieve clinically-sound knowledge from a professional knowledge database, thereby serving as a reliable reference for the AI model to enhance the reliability of its response as disclosed within Zaho in page 3. Additionally, doing so would enhance the accuracy of the output of the AI model.
Regarding claim 12, Canfield teaches the ultrasound imaging system of claim 1, wherein the AI model output comprises a set of natural language instructions, the set of natural language instructions including the plurality of recommended next steps (para. 0051; Graphic overlays may also include visual instructions, e.g., text and/or symbols, for guiding a user of the system 600 through an adaptive ultrasound scan in a manner necessary to obtain images and/or measurements required for a prenatal assessment.).
Regarding claim 13, Canfield teaches an ultrasound imaging system comprising (figure 1, para. 0027):
a processing circuit having a processor coupled to a memory device storing instructions thereon that, when executed, cause the processing circuit to perform operations comprising (para. 0026; a software-based neural network may be implemented using a processor (e.g., single or multi-core CPU, a single GPU or GPU cluster, or multiple processors arranged for parallel-processing) configured to execute instructions, which may be stored in computer readable medium, and which when executed cause the processor to perform a machine-trained algorithm for identifying various anatomical features of a fetus within ultrasound images and, in some examples, output an indication of the presence or absence of such features.):
receiving image data obtained using an ultrasound probe and performing an image recognition process to identify an image characteristic based on the image data (para. 0027; The system can also include a data processor 126, e.g., a computational module or circuity, configured to implement a first neural network 128. The first neural network 128 can be configured to receive the image frames 124, either directly from the signal processor 122 or via the data processor 126, determine a presence, absence and/or identity of at least one anatomical feature within each frame, and classify an image view, e.g., a current image view, based on this determination.);
converting the identified image characteristic into an artificial intelligence (AI) model input, the Al model input including a written description of the image characteristic and a current status of a workflow process (paras. 0032-0033; The data processor 126 can be configured to perform multiple functions. As mentioned above, the data processor 126 can be configured to implement a neural network 128, which can be configured to classify images into distinct categories, for example “full view,” “head,” “abdominal,” “chest,” or “extremities.” Sub-categories can include, for example, “stomach,” “bowel,” “umbilical cord,” “kidney,” “bladder,” “legs,” “arms,” “hands,” “femur,” “spine,” “heart,” “lungs,” “stomach,” “bowel,” “umbilical cord,” “kidney,” or “bladder.” Classification results determined by the neural network 128 can be adaptive to a current ultrasound region of interest and/or the completed measurements within the prenatal assessment protocol. For example, if the region of interest is large and includes multiple sub-categories of anatomical features, such as the kidney, liver and umbilical cord, the neural network 128 may classify the current image as “abdominal,” along with an indication of suggested features and/or measurements thereof to be obtained within the abdominal region. Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The examiner notes that the first neural network inputs the image and output a classification of the image based on the anatomical feature, scan view, and current completed measurement within the assessment protocol. The neural network outputs a written description of the organ and the current view and current completed step in the protocol. Thus, the current status of the workflow is outputted along with a description of the image to determine the recommended next step to be followed in accordance with the protocol.);
receiving contextual information, the contextual information comprising procedure-specific information (para. 0033; For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. The examiner notes that the AI circuit receives procedure specific information such as a list of measurements to be obtained in accordance with a stored scan protocol.);
receiving the AI model input and a prompt, the prompt including a plurality of configurable options (figure 6, paras. 0033 and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600, input requesting additional instructions or assistance for performing an ultrasound scan, and/or input requesting that one or more ultrasound images be saved and/or transmitted to a remote receiver. The examiner notes that the neural network receives inputs from the first neural network and user input prompting the neural network to generate additional recommendations. The second neural network output recommended next steps based on the first neural network input and the user request);
applying the Al model input and the contextual information to an Al model configured to generate an Al model output corresponding to at least one plurality of recommended next steps of the workflow process, a total plurality of recommended next steps based on the number of configurable options (paras. 0033, 0038-0039, and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The second neural network 130 can be configured to provide suggestions for the next measurement that can or should be obtained. The network 130 can implement a recommender system, such as that shown in FIG. 4. The next measurement can be recommended based on one or more factors. For example, the next recommended measurement may be the measurement that is obtainable by implementing the smallest or most minor adjustments to the ultrasound transducer used to obtain the ultrasound images. The adjustments can include operating parameters, e.g., focal depth, or the position and/or orientation of the ultrasound transducer. By recommending measurements in this manner, a user may quickly and efficiently progress through a scan by minimizing the extent of imaging adjustments required to obtain successive measurements. In addition or alternatively, the next recommended measurement can be the measurement that is obtainable at or above an accuracy threshold. The examiner notes that the second neural network output suggestions for the recommended next measurements based on the input received by the first neural network and any additional requests received by the user.);
converting the Al model output into at least one control signal, wherein operations of the ultrasound imaging system associated with each of the plurality of recommended next steps are embedded into the at least one control signal (para. 0029; In examples where a mechanical adjustment mechanism 142, e.g., a robotic arm, is used to control the position and/or orientation of the data acquisition unit 110, the instructions for acquiring image data may be communicated to a controller 144 configured to cause the adjustment mechanism to make the necessary adjustments automatically, without user input. The controller 144 may be configured to adjust the position, orientation and/or operational settings of the data acquisition unit 110 as part of an automatic feedback loop with the data processor 126. In some examples, the controller 144 can receive information regarding a current position of the ultrasound sensor array 112 and a direction the array needs to move or turn, which may be based at least in part on information received from the first neural network 128 and/or second neural network 130, along with information regarding a current fetal position (FIG. 2).); and
controlling a display of the ultrasound imaging system based on the at least one control signal, wherein controlling the display comprises causing the display to display the plurality of recommended next steps for an operator of the ultrasound imaging system to perform (paras. 0027, 0029, and 0034; the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. In some examples, the scan protocol 138 may convey the presence of an anatomical feature within a current image frame 124, and in some embodiments, whether the image of such feature is sufficient to accurately measure the feature or whether additional images of the feature are needed. The user interface 134 can be configured to display the ultrasound images 136 of the region 116 in real time as an ultrasound scan is being performed, along with the adaptive scan protocol 138. The user interface 134 may also be configured to receive a user input 140 at any time before, during, or after an ultrasound scan. For example, the user interface 134 may be interactive, receiving user input 140 indicating confirmation that an anatomical feature has been assessed, and adapting a display responsive to the input. As further shown, the scan protocol 138 may also be input into the user interface 134. The user input 140 received at the user interface 134 can be in the form of a manual confirmation that a particular measurement has been obtained. In some embodiments, the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings. The examiner notes that the displayed recommended next steps are updated based on user input such as the user rejecting the recommended step or based on the completion of the current recommended step.); and
However, Canfield fails to explicitly teach a retrieval model configured to receive contextual information, the contextual information comprising patient medical history.
However, Canfield fails to teach a retrieval model configured to receive contextual information comprising patient medical history.
Zaho, in the same field of endeavor, teaches a retrieval model configured to receive contextual information comprising medical information (page 2, left col; ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently. (2) Hierarchical in-context learning for enhanced report generation. Top-k reports that are semantically similar to the LLM-generated report are retrieved from a clinic database via the proposed retrieval module (c.f. Fig. 3. The retrieved k reports then serve as in-context examples to refine the LLM-generated report. (3) Knowledge-based reliable interaction. As illustrated in Fig. 1(b), ChatCAD+ does not directly provide medical advice. Instead, it first seeks help via our proposed knowledge retrieval module for obtaining relevant knowledge from professional sources, e.g. Merck Manuals, Mayo Clinic, and Cleveland Clinic. Then, the LLM considers the retrieved knowledge as a reference to provide reliable medical advice.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a retrieval-augmented generation model because it will help retrieve clinically-sound knowledge from a professional knowledge database, thereby serving as a reliable reference for the AI model to enhance the reliability of its response as disclosed within Zaho in page 3. Additionally, doing so would enhance the accuracy of the output of the AI model.
However, Canfield in the view of Zaho fail to disclose that the contextual information include patient medical history.
Huang, in the same field of endeavor, teaches retrieving contextual information include patient medical history (paras. 59-62; For the patient to be examined, extract the self-report information and historical medical treatment information of the patient to be examined as pre-diagnosis information; Process the pre-diagnosis information to obtain input information. Input information to the external large-scale language model to obtain scan suggestions fed back by the large-scale language model. The scan suggestions are used to obtain scan images by performing ultrasound scans on patients to be examined.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the contextual information of Canfield in the view of Zaho with the contextual information of Haung to provide contextual information include patient medical history because it will help understand the actual situation of the patient and obtain more accurate feedback information to achieve better quality control of the scanning process as disclosed within Haung in para. 63.
Regarding claim 14, Canfield teaches the ultrasound imaging system of claim 13, however, fails to explicitly teach wherein the AI model comprises a large language model, and wherein applying the AI model input to the AI model comprises: querying the retrieval model configured to search a medical information database for data relating to the AI model input; combining the data relating to the AI model input with the AI model input to create an augmented AI model input; applying the augmented AI model input to the large language model; and generating the AI model output.
Zaho, in the same field of endeavor, teaches the AI model comprises a large language model, and wherein applying the AI model input to the AI model comprises: querying the retrieval model configured to search a medical information database for data relating to the AI model input; combining the data relating to the AI model input with the AI model input to create an augmented AI model input; applying the augmented AI model input to the large language model; and generating the AI model output (c.f. Fig. 2(a)). ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently. (2) Hierarchical incontext learning for enhanced report generation. Top-k reports that are semantically similar to the LLM-generated report are retrieved from a clinic database via the proposed retrieval module (c.f. Fig. 3. The retrieved k reports then serve as in-context examples to refine the LLM-generated report. (3) Knowledge-based reliable interaction. As illustrated in Fig. 1(b), ChatCAD+ does not directly provide medical advice. Instead, it first seeks help via our proposed knowledge retrieval module for obtaining relevant knowledge from professional sources, e.g. Merck Manuals, Mayo Clinic, and Cleveland Clinic. Then, the LLM considers the retrieved knowledge as a reference to provide reliable medical advice. The examiner notes that the system first converts image features to text description which will then be inputted into large language model, the LLM uses retrieval module to retrieve medical information relative to the identified feature and use it as an additional input to generate a report).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a retrieval-augmented generation model and a large language model because it will help retrieve clinically-sound knowledge from a professional knowledge database, thereby serving as a reliable reference for the AI model to enhance the reliability of its response as disclosed within Zaho in page 3. Additionally, doing so would enhance the accuracy of the output of the AI model.
Regarding claim 15, Canfield teaches a method comprising (figure 1, para. 0027):
receiving, by an image processing circuit, image data obtained using an ultrasound probe (para. 0027; The system can also include a data processor 126, e.g., a computational module or circuity, configured to implement a first neural network 128. The first neural network 128 can be configured to receive the image frames 124, either directly from the signal processor 122 or via the data processor 126, determine a presence, absence and/or identity of at least one anatomical feature within each frame, and classify an image view, e.g., a current image view, based on this determination.);
performing, by the image processing circuit, an image recognition process to identify an image characteristic based on the image data (para. 0027; The system can also include a data processor 126, e.g., a computational module or circuity, configured to implement a first neural network 128. The first neural network 128 can be configured to receive the image frames 124, either directly from the signal processor 122 or via the data processor 126, determine a presence, absence and/or identity of at least one anatomical feature within each frame, and classify an image view, e.g., a current image view, based on this determination.);
converting, by an entity recognition circuit, the identified image characteristic into an artificial intelligence (AI) model input, the AI model input including a written description of the image characteristic and a current status of a workflow process (paras. 0032-0033; The data processor 126 can be configured to perform multiple functions. As mentioned above, the data processor 126 can be configured to implement a neural network 128, which can be configured to classify images into distinct categories, for example “full view,” “head,” “abdominal,” “chest,” or “extremities.” Sub-categories can include, for example, “stomach,” “bowel,” “umbilical cord,” “kidney,” “bladder,” “legs,” “arms,” “hands,” “femur,” “spine,” “heart,” “lungs,” “stomach,” “bowel,” “umbilical cord,” “kidney,” or “bladder.” Classification results determined by the neural network 128 can be adaptive to a current ultrasound region of interest and/or the completed measurements within the prenatal assessment protocol. For example, if the region of interest is large and includes multiple sub-categories of anatomical features, such as the kidney, liver and umbilical cord, the neural network 128 may classify the current image as “abdominal,” along with an indication of suggested features and/or measurements thereof to be obtained within the abdominal region. Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The examiner notes that the first neural network inputs the image and output a classification of the image based on the anatomical feature, scan view, and current completed measurement within the assessment protocol. The neural network outputs a written description of the organ and the current view and current completed step in the protocol. Thus, the current status of the workflow is outputted along with a description of the image to determine the recommended next step to be followed in accordance with the protocol.);
receiving contextual information, the contextual information comprising procedure-specific information (para. 0033; For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. The examiner notes that the AI circuit receives procedure specific information such as a list of measurements to be obtained in accordance with a stored scan protocol.);
receiving, by an AI circuit, the AI model input and a prompt, the prompt including a plurality of configurable options (figure 6, paras. 0033 and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600, input requesting additional instructions or assistance for performing an ultrasound scan, and/or input requesting that one or more ultrasound images be saved and/or transmitted to a remote receiver. The examiner notes that the neural network receives inputs from the first neural network and user input prompting the neural network to generate additional recommendations. The second neural network output recommended next steps based on the first neural network input and the user request);
applying, by the AI circuit, the AI model input and the contextual information to an AI model; outputting, by the AI circuit, an AI model output corresponding to a plurality of recommended next steps of the workflow process, the plurality of recommended next steps based on the number of configurable options (paras. 0033, 0038-0039, and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The second neural network 130 can be configured to provide suggestions for the next measurement that can or should be obtained. The network 130 can implement a recommender system, such as that shown in FIG. 4. The next measurement can be recommended based on one or more factors. For example, the next recommended measurement may be the measurement that is obtainable by implementing the smallest or most minor adjustments to the ultrasound transducer used to obtain the ultrasound images. The adjustments can include operating parameters, e.g., focal depth, or the position and/or orientation of the ultrasound transducer. By recommending measurements in this manner, a user may quickly and efficiently progress through a scan by minimizing the extent of imaging adjustments required to obtain successive measurements. In addition or alternatively, the next recommended measurement can be the measurement that is obtainable at or above an accuracy threshold. The examiner notes that the second neural network output suggestions for the recommended next measurements based on the input received by the first neural network and any additional requests received by the user.);
converting, by the entity recognition circuit, the AI model output into a control signal corresponding to the plurality of recommended next steps (para. 0029; In examples where a mechanical adjustment mechanism 142, e.g., a robotic arm, is used to control the position and/or orientation of the data acquisition unit 110, the instructions for acquiring image data may be communicated to a controller 144 configured to cause the adjustment mechanism to make the necessary adjustments automatically, without user input. The controller 144 may be configured to adjust the position, orientation and/or operational settings of the data acquisition unit 110 as part of an automatic feedback loop with the data processor 126. In some examples, the controller 144 can receive information regarding a current position of the ultrasound sensor array 112 and a direction the array needs to move or turn, which may be based at least in part on information received from the first neural network 128 and/or second neural network 130, along with information regarding a current fetal position (FIG. 2).); and
receiving, by a control circuit, the control signal (para. 0029; In examples where a mechanical adjustment mechanism 142, e.g., a robotic arm, is used to control the position and/or orientation of the data acquisition unit 110, the instructions for acquiring image data may be communicated to a controller 144 configured to cause the adjustment mechanism to make the necessary adjustments automatically, without user input. The controller 144 may be configured to adjust the position, orientation and/or operational settings of the data acquisition unit 110 as part of an automatic feedback loop with the data processor 126. In some examples, the controller 144 can receive information regarding a current position of the ultrasound sensor array 112 and a direction the array needs to move or turn, which may be based at least in part on information received from the first neural network 128 and/or second neural network 130, along with information regarding a current fetal position (FIG. 2).);
controlling, by the control circuit, a display of an ultrasound imaging system based on the control signal, wherein controlling the display comprises causing the display to display the plurality of recommended next steps for an operator of the ultrasound imaging system to perform (paras. 0027, 0029, and 0034; the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. In some examples, the scan protocol 138 may convey the presence of an anatomical feature within a current image frame 124, and in some embodiments, whether the image of such feature is sufficient to accurately measure the feature or whether additional images of the feature are needed. The user interface 134 can be configured to display the ultrasound images 136 of the region 116 in real time as an ultrasound scan is being performed, along with the adaptive scan protocol 138. The user interface 134 may also be configured to receive a user input 140 at any time before, during, or after an ultrasound scan. For example, the user interface 134 may be interactive, receiving user input 140 indicating confirmation that an anatomical feature has been assessed, and adapting a display responsive to the input. As further shown, the scan protocol 138 may also be input into the user interface 134. The user input 140 received at the user interface 134 can be in the form of a manual confirmation that a particular measurement has been obtained. In some embodiments, the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings. The examiner notes that the displayed recommended next steps are updated based on user input such as the user rejecting the recommended step or based on the completion of the current recommended step.); and
receiving, by the AI circuit, a second model input, the second model input including an updated status of the workflow process based on a performed next step, and outputting, by the AI circuit, a second AI model output corresponding to an updated plurality of recommended next steps, such that the control circuit is configured to cause the display to display the updated plurality of recommended next steps based on the updated status (paras. 0027, 0029, and 0034; n some examples, the system 100 also includes a display processor 132 coupled with the data processor and a user interface 134. The display processor 132 can link the neural networks 128, 130 to the user interface 134, enabling the neural network outputs to be displayed on or modify the information displayed on the user interface. In various embodiments, the user interface 134 may receive the outputs directly from the second neural network 130, or after additional processing via the data processor 132. In some examples, the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. The systems herein overcome this problem by employing an imaging system coupled with deep learning and an intuitive user interface configured to provide an adaptive scan protocol that is responsive to image data acquired during a scan and the movement and current position of the fetus being evaluated. With reference to FIG. 1, the system 100 enables users to perform effective prenatal assessments by identifying anatomical features within acquired ultrasound image frames 124 and providing instructions to the users for acquiring image data for the next required anatomical feature specified in a worklist, which may be updated and displayed on the user interface 134. The user interface 134 may be configured to display and update the adaptive scan protocol 138 in real time as an ultrasound scan is being performed. In some examples, the user interface 134 may be further configured to display instructions 139 for adjusting the data acquisition unit 110 in the manner necessary to obtain the next recommended measurements. The examiner notes that the system runs in a loop where it receives images and process them and feed the output of the first neural network to the second neural network with outputs an updated recommended steps based on the completion of the previous step or a user input.).
However, Canfield fails to explicitly teach AI circuit comprising a retrieval model configured to receive contextual information, the contextual information comprising patient medical history.
However, Canfield fails to teach an AI circuit comprising a retrieval model configured to receive contextual information comprising patient medical history.
Zaho, in the same field of endeavor, teaches an AI circuit comprising a retrieval model configured to receive contextual information comprising medical information (page 2, left col; ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently. (2) Hierarchical in-context learning for enhanced report generation. Top-k reports that are semantically similar to the LLM-generated report are retrieved from a clinic database via the proposed retrieval module (c.f. Fig. 3. The retrieved k reports then serve as in-context examples to refine the LLM-generated report. (3) Knowledge-based reliable interaction. As illustrated in Fig. 1(b), ChatCAD+ does not directly provide medical advice. Instead, it first seeks help via our proposed knowledge retrieval module for obtaining relevant knowledge from professional sources, e.g. Merck Manuals, Mayo Clinic, and Cleveland Clinic. Then, the LLM considers the retrieved knowledge as a reference to provide reliable medical advice.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a retrieval-augmented generation model because it will help retrieve clinically-sound knowledge from a professional knowledge database, thereby serving as a reliable reference for the AI model to enhance the reliability of its response as disclosed within Zaho in page 3. Additionally, doing so would enhance the accuracy of the output of the AI model.
However, Canfield in the view of Zaho fail to disclose that the contextual information include patient medical history.
Huang, in the same field of endeavor, teaches retrieving contextual information include patient medical history (paras. 59-62; For the patient to be examined, extract the self-report information and historical medical treatment information of the patient to be examined as pre-diagnosis information; Process the pre-diagnosis information to obtain input information. Input information to the external large-scale language model to obtain scan suggestions fed back by the large-scale language model. The scan suggestions are used to obtain scan images by performing ultrasound scans on patients to be examined.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the contextual information of Canfield in the view of Zaho with the contextual information of Haung to provide contextual information include patient medical history because it will help understand the actual situation of the patient and obtain more accurate feedback information to achieve better quality control of the scanning process as disclosed within Haung in para. 63.
Regarding claim 16, Canfield teaches the method of claim 15, wherein the plurality of recommended next steps comprise at least one of a recommendation to rotate or move the ultrasound probe a certain way, a recommendation to activate a particular mode, or a recommendation to take a particular measurement (para. 0048; The guidance can be generated by the second neural network 130 in the form of one or more instructions 518 for adjusting an ultrasound transducer in a manner necessary to obtain the next recommended image and/or measurement. For instance, if the head of a fetus has been most recently measured, the next recommended measurement may be of the abdominal region. To comply with this recommendation, the user interface 500 may display instructions 518 for adjusting an ultrasound transducer in a manner that enables images of the abdominal region to be obtained. Instructions may include directional commands, e.g., “Move ultrasound probe laterally,” and/or technique-based commands, e.g., “Move ultrasound probe slower”; “Slow down”; “Stop”; “or “Continue.” In some embodiments, the instructions may comprise modifications of one or more image acquisition parameters. For example, the user interface 500 may provide instructions 518 to alter the imaging plane, the focal depth and/or the pulse frequency of the transducer. In the event that an abnormality is detected, the user interface 500 may provide an instruction to hold the transducer steady at one location, thereby allowing further analysis. Slight adjustments in the imaging angle may also be recommended to more thoroughly characterize a detected abnormality.).
Regarding claim 17, Canfield teaches the method of claim 15, further comprising changing, by the control circuit, an operating characteristic of the ultrasound imaging system based on receiving a user input selecting one of the plurality of recommended next steps, and wherein changing the operating characteristic of the ultrasound imaging system comprises changing at least one of a mode, a view, an acquisition parameter, or a measurement setting (paras. 0034, 0048, and 0051; the user input 140 may comprise agreement or disagreement with a next recommended measurement. In this manner, a user may override a recommended measurement. In some examples, the user input 140 can include instructions for implementing particular operational parameters necessary for imaging and/or measuring specific anatomical features, e.g., biparietal diameter, occipito-frontal diameter, head circumference, abdominal circumference, femur length, amniotic fluid index, etc. The operational parameters can include focal depths, pulse frequencies, scan line numbers, scan line densities, or other settings. the user interface 500 may display instructions 518 for adjusting an ultrasound transducer in a manner that enables images of the abdominal region to be obtained. Instructions may include directional commands, e.g., “Move ultrasound probe laterally,” and/or technique-based commands, e.g., “Move ultrasound probe slower”; “Slow down”; “Stop”; “or “Continue.” In some embodiments, the instructions may comprise modifications of one or more image acquisition parameters. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600. The examiner notes that the recommendation includes a list of option that include probe movement, parameter adjustment, measurements to obtain, etc. The user selects one of the recommended options using the user interface and a prompt is sent to the system to adjust parameters based on the user selection.).
Regarding claim 18, Canfield teaches the method of claim 15, wherein the image characteristic comprises at least one of a mode, a view, an identification of an imaged structure, an identification of a pathology, an acquisition parameter, or a measurement setting (para. 0027; the first neural network 128 can be configured to receive the image frames 124, either directly from the signal processor 122 or via the data processor 126, determine a presence, absence and/or identity of at least one anatomical feature within each frame, and classify an image view, e.g., a current image view, based on this determination.).
Regarding claim 19, Canfield teaches the method of claim 15, however, fails to explicitly teach wherein the AI model comprises a large language model and wherein the AI model input comprises a natural language input.
Zaho, in the same field of endeavor, teaches the AI model comprises a large language model (pages 2-3; ChatCAD [9] linked existing CAD models with LLMs to enhance diagnostic accuracy and improve patient care.), and wherein the AI model input comprises a natural language input (page 2, left col; ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently.).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a large language model and AI model input comprising natural language because it demonstrates extraordinary capabilities in understanding and generating human-like text as disclosed within Zaho in page 1. Additionally, doing so would help in translating image features into clear context aware and adaptive guidance that a user can easily understand and follow.
Regarding claim 20, Canfield teaches the method of claim 19, however, fails to explicitly teach wherein applying the AI model input to the AI model comprises: querying, by the AI circuit, the retrieval model configured to search a medical information database for data relating to the AI model input; combining, by the AI circuit, the data relating to the AI model input with the AI model input to create an augmented AI model input; applying, by the AI circuit, the augmented AI model input to the large language model; and generating, by the AI circuit, the AI model output.
Zaho, in the same field of endeavor, teaches wherein applying the AI model input to the AI model comprises: querying, by the AI circuit, a retrieval model configured to search a medical information database for data relating to the AI model input; combining, by the AI circuit, the data relating to the AI model input with the AI model input to create an augmented AI model input; applying, by the AI circuit, the augmented AI model input to the large language model; and generating, by the AI circuit, the AI model output (c.f. Fig. 2(a)). ChatCAD+ can adaptively select a corresponding model given the input medical image. The tentative output of the CAD network is converted into text description to reflect image features, making it applicable for diagnostic reporting subsequently. (2) Hierarchical incontext learning for enhanced report generation. Top-k reports that are semantically similar to the LLM-generated report are retrieved from a clinic database via the proposed retrieval module (c.f. Fig. 3. The retrieved k reports then serve as in-context examples to refine the LLM-generated report. (3) Knowledge-based reliable interaction. As illustrated in Fig. 1(b), ChatCAD+ does not directly provide medical advice. Instead, it first seeks help via our proposed knowledge retrieval module for obtaining relevant knowledge from professional sources, e.g. Merck Manuals, Mayo Clinic, and Cleveland Clinic. Then, the LLM considers the retrieved knowledge as a reference to provide reliable medical advice. The examiner notes that the system first converts image features to text description which will then be inputted into large language model, the LLM uses retrieval module to retrieve medical information relative to the identified feature and use it as an additional input to generate a report).
It would have been obvious to an ordinary skilled in the art before the invention was made to modify the AI model of Canfield with the AI model of Zaho to provide an AI model comprising a retrieval-augmented generation model and a large language model because it will help retrieve clinically-sound knowledge from a professional knowledge database, thereby serving as a reliable reference for the AI model to enhance the reliability of its response as disclosed within Zaho in page 3. Additionally, doing so would enhance the accuracy of the output of the AI model.
Regarding claim 21, Canfield teaches the ultrasound imaging system of claim 1, wherein the AI model input comprises a current status and the AI circuit is further configured to (paras. 0032-0033; The data processor 126 can be configured to perform multiple functions. As mentioned above, the data processor 126 can be configured to implement a neural network 128, which can be configured to classify images into distinct categories, for example “full view,” “head,” “abdominal,” “chest,” or “extremities.” Sub-categories can include, for example, “stomach,” “bowel,” “umbilical cord,” “kidney,” “bladder,” “legs,” “arms,” “hands,” “femur,” “spine,” “heart,” “lungs,” “stomach,” “bowel,” “umbilical cord,” “kidney,” or “bladder.” Classification results determined by the neural network 128 can be adaptive to a current ultrasound region of interest and/or the completed measurements within the prenatal assessment protocol. For example, if the region of interest is large and includes multiple sub-categories of anatomical features, such as the kidney, liver and umbilical cord, the neural network 128 may classify the current image as “abdominal,” along with an indication of suggested features and/or measurements thereof to be obtained within the abdominal region. Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The examiner notes that the first neural network inputs the image and output a classification of the image based on the anatomical feature, scan view, and current completed measurement within the assessment protocol. The neural network outputs a written description of the organ and the current view and current completed step in the protocol. Thus, the current status of the workflow is outputted along with a description of the image to determine the recommended next step to be followed in accordance with the protocol.):
receive a natural language prompt including a number of configurable options (figure 6, paras. 0033 and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600, input requesting additional instructions or assistance for performing an ultrasound scan, and/or input requesting that one or more ultrasound images be saved and/or transmitted to a remote receiver. The examiner notes that the neural network receives inputs from the first neural network and user input prompting the neural network to generate additional recommendations. The second neural network output recommended next steps based on the first neural network input and the user request);
combine the natural language prompt including the number of configurable options with the AI model input to create an augmented AI model input (figure 6, paras. 0033 and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600, input requesting additional instructions or assistance for performing an ultrasound scan, and/or input requesting that one or more ultrasound images be saved and/or transmitted to a remote receiver. The examiner notes that the neural network receives inputs from the first neural network and user input prompting the neural network to generate additional recommendations. The second neural network output recommended next steps based on the first neural network input and the user request);
apply the augmented AI model input to the AI model to output an AI model output corresponding to the plurality of recommended next steps, the plurality of recommended next steps based on the received number of configurable options (paras. 0033, 0038-0039, and 0051; Output generated by the neural network 128 can be input into a second neural network 130, which in some examples, comprises a convolutional neural network (CNN) configured to receive multiple input types. For example, input to the second neural network 130 can include the organ/view classification received from the first neural network 128, along with binary classifications of motion detection, approximated fetal position data, and/or a list of measurements to be obtained in accordance with a stored scan protocol. From these inputs, the second neural network 130 can determine and output a suggested next measurement to be obtained in accordance with the required measurements of a prenatal assessment protocol. The second neural network 130 can be configured to provide suggestions for the next measurement that can or should be obtained. The network 130 can implement a recommender system, such as that shown in FIG. 4. The next measurement can be recommended based on one or more factors. For example, the next recommended measurement may be the measurement that is obtainable by implementing the smallest or most minor adjustments to the ultrasound transducer used to obtain the ultrasound images. The adjustments can include operating parameters, e.g., focal depth, or the position and/or orientation of the ultrasound transducer. By recommending measurements in this manner, a user may quickly and efficiently progress through a scan by minimizing the extent of imaging adjustments required to obtain successive measurements. In addition or alternatively, the next recommended measurement can be the measurement that is obtainable at or above an accuracy threshold. The examiner notes that the second neural network output suggestions for the recommended next measurements based on the input received by the first neural network and any additional requests received by the user.);
receive the updated status based on a performed next step from the number of configurable options of the plurality of recommended next steps (paras. 0027, 0029, and 0034; some examples, the system 100 also includes a display processor 132 coupled with the data processor and a user interface 134. The display processor 132 can link the neural networks 128, 130 to the user interface 134, enabling the neural network outputs to be displayed on or modify the information displayed on the user interface. In various embodiments, the user interface 134 may receive the outputs directly from the second neural network 130, or after additional processing via the data processor 132. In some examples, the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. The systems herein overcome this problem by employing an imaging system coupled with deep learning and an intuitive user interface configured to provide an adaptive scan protocol that is responsive to image data acquired during a scan and the movement and current position of the fetus being evaluated. With reference to FIG. 1, the system 100 enables users to perform effective prenatal assessments by identifying anatomical features within acquired ultrasound image frames 124 and providing instructions to the users for acquiring image data for the next required anatomical feature specified in a worklist, which may be updated and displayed on the user interface 134. The user interface 134 may be configured to display and update the adaptive scan protocol 138 in real time as an ultrasound scan is being performed. In some examples, the user interface 134 may be further configured to display instructions 139 for adjusting the data acquisition unit 110 in the manner necessary to obtain the next recommended measurements. The examiner notes that the system runs in a loop where it receives images and process them and feed the output of the first neural network to the second neural network with outputs an updated recommended steps based on the completion of the previous step or a user input.); and
output an updated AI model output based on the updated status, such that the control circuit is configured to cause the display to display an updated plurality of recommended next steps based on the performed next step (paras. 0027, 0029, and 0034; some examples, the system 100 also includes a display processor 132 coupled with the data processor and a user interface 134. The display processor 132 can link the neural networks 128, 130 to the user interface 134, enabling the neural network outputs to be displayed on or modify the information displayed on the user interface. In various embodiments, the user interface 134 may receive the outputs directly from the second neural network 130, or after additional processing via the data processor 132. In some examples, the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. The systems herein overcome this problem by employing an imaging system coupled with deep learning and an intuitive user interface configured to provide an adaptive scan protocol that is responsive to image data acquired during a scan and the movement and current position of the fetus being evaluated. With reference to FIG. 1, the system 100 enables users to perform effective prenatal assessments by identifying anatomical features within acquired ultrasound image frames 124 and providing instructions to the users for acquiring image data for the next required anatomical feature specified in a worklist, which may be updated and displayed on the user interface 134. The user interface 134 may be configured to display and update the adaptive scan protocol 138 in real time as an ultrasound scan is being performed. In some examples, the user interface 134 may be further configured to display instructions 139 for adjusting the data acquisition unit 110 in the manner necessary to obtain the next recommended measurements. The examiner notes that the system runs in a loop where it receives images and process them and feed the output of the first neural network to the second neural network with outputs an updated recommended steps based on the completion of the previous step or a user input.).
Response to Arguments
Applicant's arguments filed 06/23/2026 have been fully considered but they are not persuasive. The applicant argues that Canfield fails to disclose or suggest “an entity recognition circuit configured to convert the identified image characteristic into an artificial intelligence (AI) model input, the Al model input including a written description of the image characteristic and a current status of a workflow process”. The examiner respectfully disagrees. Canfield teaches a processor that uses a first neural network model to analyze and classify images based on anatomical features, scan views, fetus motion and position, current completed measurements within the prenatal assessment protocol. For example, in the image includes an abdominal area, the neural network will classify the image as abdomen in addition to the measurements to be obtained based on the scan protocol. These outputs get inputted into a second neural network model to help the model recommend what is the next step to recommend to follow the workflow process. Thus, in order for the neural network to recommend an accurate next step that corresponds to the order of the workflow process, the status of the current step has to be provided to the model as an input. Additionally, Canfield teaches that a list of list of measurements to be obtained in accordance with a stored scan protocol is also provided to the second neural network model which is outputted by the first neural network model based on the current characteristics of the image (See paras. 0032-0033).
The applicant argues that Canfield fails to suggest or disclose “output an Al model output corresponding to a plurality of recommended next steps of the workflow process, the plurality of recommended next steps based on the number of configurable options”. The examiner respectfully disagrees. Canfield teaches in figure 6, para. 0051; In some examples, the graphics processor may receive input from the user interface 624, such as a typed patient name or confirmation that an instruction displayed or emitted from the interface has been acknowledged and/or implemented by the user of the system 600. The user interface 624 may also receive input regarding the selection of particular imaging modalities and the operating parameters included in such modalities, input prompting adjustments to the settings and/or parameters used by the system 600, input requesting additional instructions or assistance for performing an ultrasound scan, and/or input requesting that one or more ultrasound images be saved and/or transmitted to a remote receiver. Canfield teaches that the user selectable inputs including requests for additional workflow guidance, image modality selections, and operating parameter selections. As shown in figure 6, the processor receives these user inputs and supplies them to the neural network. Therefore, the neural network receives a prompt comprising plurality of configurable options in addition to the image classification, and generates a workflow guidance based on the received inputs. Additionally, Canfield teaches that the outputted recommended steps can be configured by the system without user manually performing them as disclosed in paras. 0027 and 0029.
The applicant argues that Canfield fails to suggest or disclose iteratively generating next steps based on an updated status from the outcome of a recommended next step. The examiner respectfully disagrees. Canfield teaches The display processor 132 can link the neural networks 128, 130 to the user interface 134, enabling the neural network outputs to be displayed on or modify the information displayed on the user interface. In various embodiments, the user interface 134 may receive the outputs directly from the second neural network 130, or after additional processing via the data processor 132. In some examples, the display processor 132 may be configured to generate ultrasound images 136 from the image frames 124, received either directly or indirectly from the data acquisition unit 110, and generate an adaptive scan protocol 138 that can include a worklist updated to reflect the images and/or measurements obtained and the images and/or measurements that have yet to be obtained. In some examples, the scan protocol 138 may convey the presence of an anatomical feature within a current image frame 124, and in some embodiments, whether the image of such feature is sufficient to accurately measure the feature or whether additional images of the feature are needed. The user interface 134 can be configured to display the ultrasound images 136 of the region 116 in real time as an ultrasound scan is being performed, along with the adaptive scan protocol 138. The system 100 enables users to perform effective prenatal assessments by identifying anatomical features within acquired ultrasound image frames 124 and providing instructions to the users for acquiring image data for the next required anatomical feature specified in a worklist, which may be updated and displayed on the user interface 134. Because the system 100 is responsive to the anatomical features detected in a current image view and any current movement and/or position of the fetus, the user can be prompted to obtain images of certain features and, in some examples, required measurements of such features, in a manner that is adaptive to the physical status of the fetus with respect to the position and angular orientation of the data acquisition unit 110. Thus, Canfield disclosed that the display of recommended next steps gets updated based on the received images that gets fed to the neural network and the neural network updates the list of recommended steps. The list of recommended steps and the status of the steps are displayed and updated in real time as disclosed in paras. 0027, 0029, and 0034.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
MacNeil et al.: “Prompt Middleware: Mapping Prompts for Large Language Models to UI Affordances”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZAINAB M ALDARRAJI whose telephone number is (571)272-8726. The examiner can normally be reached Monday-Thursday7AM-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carey Michael can be reached at (571) 270-7235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZAINAB MOHAMMED ALDARRAJI/Patent Examiner, Art Unit 3797
Prosecution Insights