DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/06/2026 has been entered.
Response to Amendment
Currently claims 1-24 are pending.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes – the claim recites a method for computing a 3D position and orientation of an ultrasound probe, and is therefore a process.
Step 2A, Prong 1, Judicial Exception: Yes – The claim recites the limitations “detecting a two-dimensional (2D) location of an ultrasound probe visible in a 2D laparoscopic (LP) image; estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ”.
These limitations, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitations “wherein the 2D location of the ultrasound probe is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface; and computing a 3D position and orientation of the ultrasound probe based on the back- projected points on the surface.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No- The claim recites the following additional elements “acquired by an LP camera inserted into a patient's body during a laparoscopic ultrasound (LPUS) procedure, the LP camera calibrated in a three-dimensional (3D) space prior to insertion into the patient's body”. Images acquired by the camera and calibration are steps that merely amounts to pre-solution insignificant activities.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No- Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.
Claim 2 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 1.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation “ extracting, from the 2D LP image, a 2D region corresponding to the ultrasound probe; determining the 2D location of the ultrasound probe based on the 2D region.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 3 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 2.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation “ wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe obtained via machine learning based on training data; and the 2D location of the ultrasound probe is defined as a centroid of the 2D region”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 4 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 1.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with an ultrasound model for the ultrasound probe, the ultrasound model modeling physical characteristics of the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “estimating a 3D orientation of the ultrasound probe in accordance with an ultrasound model for the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 5 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 4.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “generating multiple virtual ultrasound probe images by projecting the ultrasound model from the 3D coordinate using different 3D orientations, wherein each of the multiple virtual ultrasound probe images includes a defined feature of the ultrasound probe”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “detecting a corresponding defined 2D feature of the ultrasound probe as observable in the 2D LP image; comparing the defined feature in each of the multiple virtual ultrasound probe images with the corresponding defined 2D feature detected from 2D LP image to obtain a comparison result; and selecting one of the different 3D orientations as the estimated 3D orientation based on the comparison result” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 6 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 4.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “generating multiple virtual ultrasound images based on slices of a 3D model, which are obtained based on the 3D position, different possible 3D orientations of the ultrasound probe, and the ultrasound model”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “comparing each of the multiple virtual ultrasound images with a 2D ultrasound image acquired by the ultrasound probe to obtain a comparison result; and selecting one of the different possible 3D orientations as the estimated 3D orientation based on the comparison result” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 7 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 1.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 8 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 7.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “detecting 2D structures in the 2D ultrasound image; providing labels to the 2D structures that correspond to blood vessels” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further the claim recites the limitation “determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the an ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: The claim additionally recites the limitation “ accessing a 3D model for a target, representing a target organ and related anatomical structures including blood vessels; generating a virtual 2D blood vessel image based on each blood vessel present in the slice”.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.
Claim 9 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes – the claim recites a machine that performs the step for computing a 3D position and orientation of an ultrasound probe, and is therefore a process.
Step 2A, Prong 1, Judicial Exception: Yes – The claim recites the limitations “detecting a two-dimensional (2D) location of an ultrasound probe visible in a 2D laparoscopic (LP) image; estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ”.
These limitations, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitations “wherein the 2D location of the ultrasound probe is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface; and computing a 3D position and orientation of the ultrasound probe based on the back- projected points on the surface.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No- The claim recites the following additional elements “A machine readable and non-transitory medium having information recorded thereon, wherein the information, when ready by a machine, causes the machine to perform steps comprising: acquired by an LP camera inserted into a patient's body during a laparoscopic ultrasound (LPUS) procedure, the LP camera calibrated in a three-dimensional (3D) space prior to insertion into the patient's body”. Images acquired by the camera and calibration are steps that merely amounts to pre-solution insignificant activities.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No- Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.
Claim 10 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 9.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation “ extracting, from the 2D LP image, a 2D region corresponding to the ultrasound probe; determining the 2D location of the ultrasound probe based on the 2D region.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 11 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 10.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation “ wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe obtained via machine learning based on training data; and the 2D location of the ultrasound probe is defined as a centroid of the 2D region”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 12 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 9.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with an ultrasound model for the ultrasound probe, the ultrasound model modeling physical characteristics of the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “estimating a 3D orientation of the ultrasound probe in accordance with an ultrasound model for the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 13 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 12.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “generating multiple virtual ultrasound probe images by projecting the ultrasound model from the 3D coordinate using different 3D orientations, wherein each of the multiple virtual ultrasound probe images includes a defined feature of the ultrasound probe”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “detecting a corresponding defined 2D feature of the ultrasound probe as observable in the 2D LP image; comparing the defined feature in each of the multiple virtual ultrasound probe images with the corresponding defined 2D feature detected from 2D LP image to obtain a comparison result; and selecting one of the different 3D orientations as the estimated 3D orientation based on the comparison result” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 14 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 12.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “generating multiple virtual ultrasound images based on slices of a 3D model, which are obtained based on the 3D position, different possible 3D orientations of the ultrasound probe, and the ultrasound model”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “comparing each of the multiple virtual ultrasound images with a 2D ultrasound image acquired by the ultrasound probe to obtain a comparison result; and selecting one of the different possible 3D orientations as the estimated 3D orientation based on the comparison result” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 15 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 9.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 16 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 15.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “detecting 2D structures in the 2D ultrasound image; providing labels to the 2D structures that correspond to blood vessels” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further the claim recites the limitation “determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the an ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: The claim additionally recites the limitation “ accessing a 3D model for a target, representing a target organ and related anatomical structures including blood vessels; generating a virtual 2D blood vessel image based on each blood vessel present in the slice”.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.
Claim 17 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes – the claim recites a system for computing a 3D position and orientation of an ultrasound probe, and is therefore a process.
Step 2A, Prong 1, Judicial Exception: Yes – The claim recites the limitations “an LP U-probe location detector implemented by a processor and configured for detecting a two-dimensional (2D) location of an ultrasound probe visible in a 2D laparoscopic (LP) image; a 3D U-probe pose estimator implemented by a processor and configured for estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ”.
These limitations, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitations “wherein the 2D location of the ultrasound probe is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface; and computing a 3D position and orientation of the ultrasound probe based on the back- projected points on the surface.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No- The claim recites the following additional elements “A machine readable and non-transitory medium having information recorded thereon, wherein the information, when ready by a machine, causes the machine to perform steps comprising: acquired by an LP camera inserted into a patient's body during a laparoscopic ultrasound (LPUS) procedure, the LP camera calibrated in a three-dimensional (3D) space prior to insertion into the patient's body”. Images acquired by the camera and calibration are steps that merely amounts to pre-solution insignificant activities.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No- Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.
Claim 18 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 17.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation “ extracting, from the 2D LP image, a 2D region corresponding to the ultrasound probe; determining the 2D location of the ultrasound probe based on the 2D region.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 19 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 18.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation “ wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe obtained via machine learning based on training data; and the 2D location of the ultrasound probe is defined as a centroid of the 2D region”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 20 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 17.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with an ultrasound model for the ultrasound probe, the ultrasound model modeling physical characteristics of the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “estimating a 3D orientation of the ultrasound probe in accordance with an ultrasound model for the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation”.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 21 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 20.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “generating multiple virtual ultrasound probe images by projecting the ultrasound model from the 3D coordinate using different 3D orientations, wherein each of the multiple virtual ultrasound probe images includes a defined feature of the ultrasound probe”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “detecting a corresponding defined 2D feature of the ultrasound probe as observable in the 2D LP image; comparing the defined feature in each of the multiple virtual ultrasound probe images with the corresponding defined 2D feature detected from 2D LP image to obtain a comparison result; and selecting one of the different 3D orientations as the estimated 3D orientation based on the comparison result” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 22 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 20.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “generating multiple virtual ultrasound images based on slices of a 3D model, which are obtained based on the 3D position, different possible 3D orientations of the ultrasound probe, and the ultrasound model”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the limitation “comparing each of the multiple virtual ultrasound images with a 2D ultrasound image acquired by the ultrasound probe to obtain a comparison result; and selecting one of the different possible 3D orientations as the estimated 3D orientation based on the comparison result” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 23 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 17.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No, the claim does not recite additional claim limitations, and therefore does not integrate them into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the claim does not recite additional claim limitations.
Claim 24 is rejected under 35 U.S.C. 101 because the claim invention is directed to an abstract idea without significantly more.
Step 1, Statutory Category: Yes – for the same reasons put forth in claim 23.
Step 2A Prong 1, Judicial Exception: Yes, the claim recites the limitation: “detecting 2D structures in the 2D ultrasound image; providing labels to the 2D structures that correspond to blood vessels” is recited.
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further the claim recites the limitation “determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the an ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: The claim additionally recites the limitation “ accessing a 3D model for a target, representing a target organ and related anatomical structures including blood vessels; generating a virtual 2D blood vessel image based on each blood vessel present in the slice”.
These elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application.
Step 2B Inventive Concept: No, Similar to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 9, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Schneider et al., (US20130338505A1) in view of Hasser et al., (US20070021738A1) and in further view of Kruecker et al., (WO2021099171A1), Sabina et al., (US20190231492A1), and Lee et al., “3D Pose Tracking Using Particle Filter with Back Projection-Based Sampling” International Journal of Control, Automation, and Systems (2012) 10(6):1232-1239 DOI 10.1007/s12555-012-0618-8 ISSN:1598-6446 eISSN:2005-4092 (hereinafter “Lee”).
Regarding claim 1, Schneider teaches a method comprising:
detecting a two-dimensional (2D) location of an ultrasound probe visible in a 2D laparoscopic (LP) image acquired by an LP camera (claim 8 the ultrasound probe 100 comprises an ultrasonic transducer with markers that are captured by the laparoscopic camera 720 and the location of the probe is determined with respect to a coordinate system(see fig. 7)) and inserted into a patient's body during a laparoscopic ultrasound (LPUS) procedure (fig.7 the probe 100 and camera 720 are inserted into the body); and estimating a location of the ultrasound probe deployed in the LPUS procedure (claim 8 the ultrasound probe 100 comprises an ultrasonic transducer with markers that are captured by the laparoscopic camera 720 and the location of the probe is determined with respect to a coordinate system; the image would have a 2D location on the coordinate system and would be a model of the probe).
However, Schneider is silent regarding the LP camera calibrated in a three-dimensional (3D) space prior to insertion into the patient’s body.
In the same laparoscopy field of endeavor, Hasser teaches the LP camera calibrated in a three-dimensional (3D) space prior to insertion into the patient’s body ([0055]-[0056] the camera is calibrated in a 3D coordinate space).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Schneider with the calibration of Hasser, as this would assist surgeons in accurately guiding a tool to a target (see Hasser Abstract).
However, the combination of references fails to explicitly disclose estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ.
In the same localization field of endeavor, Kruecker teaches estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ ([042] the pose of the ultrasound probe can be determined based on the pose of the acquired image;[003] the pose of the image is determined based on the organ model);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the pose system of Kruecker, as this would improve workflow by directing the operator to any coverage caps (see Kruecker [006]).
However, the combination of references are silent regarding the 2D location of an image is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface.
In the same imaging field of endeavor, Sabina teaches the 2D location of an image is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface ([0107]) the images are back projected to the a 3D model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to use back scattered points, as this would improve volumetric results and model quality (see Sabina [0227]).
However, the combination of references are silent regarding computing a 3D position and orientation based on the back projected points on the surface.
In the same back projecting field of endeavor, Lee teaches computing a 3D position and orientation (pg. 1233 the pose is estimated) based on the back projected points on the surface (pg. 1233 the pose of several objects is estimated using back projection based sampling; fig. 6 all points with the depth property are located on the surface of the model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the pose estimation of Lee, as this would track target objects more robustly (see Lee pg. 1232). One of ordinary skill would understand this would apply to the back projected points of the human organ taught by modified Schneider, and thus read upon the limitation of “wherein the 2D location of the ultrasound probe is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface; and computing a 3D position and orientation of the ultrasound probe based on the back- projected points on the surface.”
Regarding claim 9, Schneider teaches a machine readable and non-transitory medium having information recorded thereon, wherein the information, when ready by a machine, causes the machine to perform the steps comprising ([0052] the images of the laparoscope can be view by the surgeon, so therefore there must be a machine readable and non-transitory medium having information recorded thereon, wherein the information, when ready by the machine, causes the machine to perform the following steps that can process the data from the laparoscope):
detecting a two-dimensional (2D) location of an ultrasound probe visible in a 2D laparoscopic (LP) image acquired by an LP camera (claim 8 the ultrasound probe 100 comprises an ultrasonic transducer with markers that are captured by the laparoscopic camera 720 and the location of the probe is determined with respect to a coordinate system(see fig. 7)) and inserted into a patient's body during a laparoscopic ultrasound (LPUS) procedure (fig.7 the probe 100 and camera 720 are inserted into the body); and estimating a location of the ultrasound probe deployed in the LPUS procedure (claim 8 the ultrasound probe 100 comprises an ultrasonic transducer with markers that are captured by the laparoscopic camera 720 and the location of the probe is determined with respect to a coordinate system; the image would have a 2D location on the coordinate system and would be a model of the probe).
However, Schneider is silent regarding the LP camera calibrated in a three-dimensional (3D) space prior to insertion into the patient’s body.
In the same laparoscopy field of endeavor, Hasser teaches the LP camera calibrated in a three-dimensional (3D) space prior to insertion into the patient’s body ([0055]-[0056] the camera is calibrated in a 3D coordinate space).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Schneider with the calibration of Hasser, as this would assist surgeons in accurately guiding a tool to a target (see Hasser Abstract).
However, the combination of references fails to explicitly disclose estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ.
In the same localization field of endeavor, Kruecker teaches estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ ([042] the pose of the ultrasound probe can be determined based on the pose of the acquired image;[003] the pose of the image is determined based on the organ model);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the pose system of Kruecker, as this would improve workflow by directing the operator to any coverage caps (see Kruecker [006]).
However, the combination of references are silent regarding the 2D location of an image is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface.
In the same imaging field of endeavor, Sabina teaches the 2D location of an image is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface ([0107]) the images are back projected to the a 3D model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to use back scattered points, as this would improve volumetric results and model quality (see Sabina [0227]).
However, the combination of references are silent regarding computing a 3D position and orientation based on the back projected points on the surface.
In the same back projecting field of endeavor, Lee teaches computing a 3D position and orientation (pg. 1233 the pose is estimated) based on the back projected points on the surface (pg. 1233 the pose of several objects is estimated using back projection based sampling; fig. 6 all points with the depth property are located on the surface of the model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the pose estimation of Lee, as this would track target objects more robustly (see Lee pg. 1232). One of ordinary skill would understand this would apply to the back projected points of the human organ taught by modified Schneider, and thus read upon the limitation of “wherein the 2D location of the ultrasound probe is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface; and computing a 3D position and orientation of the ultrasound probe based on the back- projected points on the surface.”
Regarding claim 17, Schneider teaches a system comprising:
an LP U-probe location detector implemented by a processor ([0052] the images of the laparoscope can be view by the surgeon, so therefore there must be a processor that can process the data from the laparoscope) and configured for detecting a two-dimensional (2D) location of an ultrasound probe visible in a 2D laparoscopic (LP) image (claim 8 the ultrasound probe 100 comprises an ultrasonic transducer with markers that are captured by the laparoscopic camera 720 and the location of the probe is determined with respect to a coordinate system(see fig. 7)) and inserted into a patient's body during a laparoscopic ultrasound (LPUS) procedure (fig.7 the probe 100 and camera 720 are inserted into the body); and estimating a location of the ultrasound probe deployed in the LPUS procedure (claim 8 the ultrasound probe 100 comprises an ultrasonic transducer with markers that are captured by the laparoscopic camera 720 and the location of the probe is determined with respect to a coordinate system; the image would have a 2D location on the coordinate system and would be a model of the probe).
However, Schneider is silent regarding the LP camera calibrated in a three-dimensional (3D) space prior to insertion in the patient’s body.
In the same laparoscopy field of endeavor, Hasser teaches the LP camera calibrated in a three-dimensional (3D) space prior to insertion in the patient’s body ([0055]-[0056] the camera is calibrated in a 3D coordinate space).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Schneider with the calibration of Hasser, as this would assist surgeons in accurately guiding a tool to a target (see Hasser Abstract).
However, the combination of references fails to explicitly disclose estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ.
In the same localization field of endeavor, Kruecker teaches estimating a 3D pose of the ultrasound probe deployed in the LPUS procedure based on the 2D location of the ultrasound probe, an estimated position of a 3D human organ, and an estimated orientation of the 3D human organ ([042] the pose of the ultrasound probe can be determined based on the pose of the acquired image;[003] the pose of the image is determined based on the organ model);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the pose system of Kruecker, as this would improve workflow by directing the operator to any coverage caps (see Kruecker [006]).
However, the combination of references are silent regarding the 2D location of an image is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface.
In the same imaging field of endeavor, Sabina teaches the 2D location of an image is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface ([0107]) the images are back projected to the a 3D model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to use back scattered points, as this would improve volumetric results and model quality (see Sabina [0227]).
However, the combination of references are silent regarding computing a 3D position and orientation based on the back projected points on the surface.
In the same back projecting field of endeavor, Lee teaches computing a 3D position and orientation (pg. 1233 the pose is estimated) based on the back projected points on the surface (pg. 1233 the pose of several objects is estimated using back projection based sampling; fig. 6 all points with the depth property are located on the surface of the model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the pose estimation of Lee, as this would track target objects more robustly (see Lee pg. 1232). One of ordinary skill would understand this would apply to the back projected points of the human organ taught by modified Schneider, and thus read upon the limitation of “wherein the 2D location of the ultrasound probe is back-projected onto a surface of the 3D human organ, resulting in back-projected points on the surface; and computing a 3D position and orientation of the ultrasound probe based on the back- projected points on the surface.”
Claims 2, 3, 10, 11, 18, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Schneider as modified by Hasser, Kruecker, Sabina, and Lee as applied to claim 1, 9, and 17 above, and further in view of Shekhar et al., (US20200197102A1).
Regarding claim 2 modified Schneider teaches the method of claim 1, wherein Schneider further teaches the 2D LP image (fig. 7 laparoscope 720 [0049]) but fails to explicitly disclose extracting a 2D region corresponding to the ultrasound probe; determining the 2D location of the ultrasound probe based on the extracted 2D region/.
However in the same ultrasound field of endeavor, Shekhar teaches extracting a 2D region corresponding to the ultrasound probe ([0039] the system directly tracks the laparoscopic ultrasound transducer in a video by detecting lines describing the outer contours of the transducer probe); determining the 2D location of the ultrasound probe based on the extracted 2D region ([0055] a region of interest (ROI) can be generated for each frame of a first imaging modality using fast visual tracking based on robust discriminative correlation filters such that subsequent processing focuses on the imaging tip. Based on this coarse estimate of the probe's location, the bounding box surrounding the imaging tip can be intended to include at least some portion of the top, middle, and tip of the transducer as seen by the camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the extraction of Shekhar, as this would improve surgical procedure planning and tool navigation (see Shekhar [0005]).
Regarding claim 3, modified Schneider teaches the method of claim 2, but fails to explicitly disclose wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe obtained via machine learning based on training data; and the 2D location of the ultrasound probe is defined as a centroid of the 2D region.
However in the same ultrasound field of endeavor, Shekar teaches wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe ([0041computer vision-based tracking (e.g., augmented reality and/or tool segmentation) obtained via machine learning based on training data ([0056] machine learning-based binary segmentation of the tool; [0065] the machine learning method may be trained on a training database); and the 2D location of the ultrasound probe is defined as a centroid of the 2D region ([0055] Based on this coarse estimate of the probe's location, the bounding box surrounding the imaging tip can be intended to include at least some portion middle of the transducer as seen by the camera).
Regarding claim 10 modified Schneider teaches the medium of claim 9, wherein Schneider further teaches the 2D LP image (fig. 7 laparoscope 720 [0049]) but fails to explicitly disclose extracting a 2D region corresponding to the ultrasound probe; determining the 2D location of the ultrasound probe based on the extracted 2D region/.
However in the same ultrasound field of endeavor, Shekhar teaches extracting a 2D region corresponding to the ultrasound probe ([0039] the system directly tracks the laparoscopic ultrasound transducer in a video by detecting lines describing the outer contours of the transducer probe); determining the 2D location of the ultrasound probe based on the extracted 2D region ([0055] a region of interest (ROI) can be generated for each frame of a first imaging modality using fast visual tracking based on robust discriminative correlation filters such that subsequent processing focuses on the imaging tip. Based on this coarse estimate of the probe's location, the bounding box surrounding the imaging tip can be intended to include at least some portion of the top, middle, and tip of the transducer as seen by the camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the extraction of Shekhar, as this would improve surgical procedure planning and tool navigation (see Shekhar [0005]).
Regarding claim 11, modified Schneider teaches the medium of claim 9, but fails to explicitly disclose wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe obtained via machine learning based on training data; and the 2D location of the ultrasound probe is defined as a centroid of the 2D region.
However in the same ultrasound field of endeavor, Shekar teaches wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe ([0041computer vision-based tracking (e.g., augmented reality and/or tool segmentation) obtained via machine learning based on training data ([0056] machine learning-based binary segmentation of the tool; [0065] the machine learning method may be trained on a training database); and the 2D location of the ultrasound probe is defined as a centroid of the 2D region ([0055] Based on this coarse estimate of the probe's location, the bounding box surrounding the imaging tip can be intended to include at least some portion middle of the transducer as seen by the camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the extraction of Shekhar, as this would improve surgical procedure planning and tool navigation (see Shekhar [0005]).
Regarding claim 19, modified Schneider teaches the system of claim 18, but fails to explicitly disclose wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe obtained via machine learning based on training data; and the 2D location of the ultrasound probe is defined as a centroid of the 2D region.
However in the same ultrasound field of endeavor, Shekar teaches wherein the step of extracting is based on a segmentation model for identifying an ultrasound probe ([0041computer vision-based tracking (e.g., augmented reality and/or tool segmentation) obtained via machine learning based on training data ([0056] machine learning-based binary segmentation of the tool; [0065] the machine learning method may be trained on a training database); and the 2D location of the ultrasound probe is defined as a centroid of the 2D region ([0055] Based on this coarse estimate of the probe's location, the bounding box surrounding the imaging tip can be intended to include at least some portion middle of the transducer as seen by the camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Schneider with the extraction of Shekhar, as this would improve surgical procedure planning and tool navigation (see Shekhar [0005]).
Claims 4, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Schneider as modified by Hasser, Kruecker, Sabina, and Lee as applied to claim 1, 9, and 17 above, and further in view of Lachaine et al., (US20060036170A1).
Regarding claim 4, modified Schneider teaches the method of claim 1, but fails to explicitly disclose transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe, the ultrasound model modeling physical characteristics of the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.
However in the same surgical field of endeavor, Lachaine teaches transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space ([0045] the image to device transformation is used to convert pixel coordinates of two-dimensional image space to three dimensional coordinate system) based on a transformation matrix obtained in calibrating the camera ([0045] the imager is calibrated to the room coordinate system, which is calibrated); estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe, the ultrasound model modeling physical characteristics of the ultrasound probe ([0046] the orientation of the probe can be determined); and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation ([0046] the 3D position and orientation of the probe can be determined in the tracker coordinate system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the tracking system of Lachaine, as this would facilitate the convenient, rapid, and accurate calibration of ultrasound images and registration of the device to a fixed coordinate system (see Lachaine [0006]). One of ordinary skill would be able to apply this to the laparoscopic camera and ultrasound probe system of modified Schneider, and would result in the teaching of the limitation “wherein the step of estimating a 3D pose of the ultrasound probe comprises: transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.”
Regarding claim 12, modified Schneider teaches the medium of claim 9, but fails to explicitly disclose transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe, the ultrasound model modeling physical characteristics of the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.
However in the same surgical field of endeavor, Lachaine teaches transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space ([0045] the image to device transformation is used to convert pixel coordinates of two-dimensional image space to three dimensional coordinate system) based on a transformation matrix obtained in calibrating the camera ([0045] the imager is calibrated to the room coordinate system, which is calibrated); estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe, the ultrasound model modeling physical characteristics of the ultrasound probe ([0046] the orientation of the probe can be determined); and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation ([0046] the 3D position and orientation of the probe can be determined in the tracker coordinate system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the tracking system of Lachaine, as this would facilitate the convenient, rapid, and accurate calibration of ultrasound images and registration of the device to a fixed coordinate system (see Lachaine [0006]). One of ordinary skill would be able to apply this to the laparoscopic camera and ultrasound probe system of modified Schneider, and would result in the teaching of the limitation “wherein the step of estimating a 3D pose of the ultrasound probe comprises: transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.”
Regarding claim 20, modified Schneider teaches the system of claim 17, but fails to explicitly disclose transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.
However in the same surgical field of endeavor, Lachaine teaches transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space ([0045] the image to device transformation is used to convert pixel coordinates of two-dimensional image space to three dimensional coordinate system) based on a transformation matrix obtained in calibrating the camera ([0045] the imager is calibrated to the room coordinate system, which is calibrated); estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe ([0046] the orientation of the probe can be determined); and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation ([0046] the 3D position and orientation of the probe can be determined in the tracker coordinate system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Schneider with the tracking system of Lachaine, as this would facilitate the convenient, rapid, and accurate calibration of ultrasound images and registration of the device to a fixed coordinate system (see Lachaine [0006]). One of ordinary skill would be able to apply this to the laparoscopic camera and ultrasound probe system of modified Schneider, and would result in the teaching of the limitation “wherein the step of estimating a 3D pose of the ultrasound probe comprises: transforming the 2D location into a 3D coordinate of the ultrasound probe in the 3D space based on a transformation matrix obtained in calibrating the LP camera; estimating a 3D orientation of the ultrasound probe in accordance with the ultrasound model for the ultrasound probe; and generating an estimated 3D pose of the ultrasound probe based on the 3D coordinate and the 3D orientation.”
Claims 7, 15, and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Schneider as modified by Hasser, Kruecker, Sabina, and Lee as applied to claim 1, 9, and 17 above, and further in view of Yagi et al., (US20170323587A1).
Regarding claim 7, modified Schneider teaches the method of claim 1, but fails to explicitly disclose labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.
In the same medical imaging field of endeavor, Yagi teaches labeling, based on the estimated 3D pose, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe ([0050] the blood vessel is identified and labeled based on the position and the orientation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the labeling of Yagi, as this would generate a more precise and reliable shape model of the vessel (see Yagi [0005]). One of ordinary skill would understand how to apply this labeling to the 3D pose of the ultrasound probe, and result in the teaching of labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.
Regarding claim 15, modified Schneider teaches the medium of claim 9, but fails to explicitly disclose wherein, the information, when read by the machine, further causes the machine to perform the step of labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.
In the same medical imaging field of endeavor, Yagi teaches wherein, the information, when read by the machine, further causes the machine to perform the step of labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe ([0050] the blood vessel is identified and labeled based on the position and the orientation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the labeling of Yagi, as this would generate a more precise and reliable shape model of the vessel (see Yagi [0005]). One of ordinary skill would understand how to apply this labeling to the 3D pose of the ultrasound probe, and result in the teaching of labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.
Regarding claim 23, modified Schneider teaches the system of claim 17, but fails to explicitly disclose a 2D vessel label generator implemented by a processor and configured for labeling, based on the 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.
In the same medical imaging field of endeavor, Yagi teaches a 2D vessel label generator implemented by a processor and configured for labeling, based on the 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe. ([0050] the blood vessel is identified and labeled based on the position and the orientation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the labeling of Yagi, as this would generate a more precise and reliable shape model of the vessel (see Yagi [0005]). One of ordinary skill would understand how to apply this labeling to the 3D pose of the ultrasound probe, and result in the teaching of labeling, based on the estimated 3D pose of the ultrasound probe, at least one blood vessel in a 2D ultrasound image acquired by the ultrasound probe.
Claims 8, 16, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Schneider as modified by Hasser, Hoffman, and Yagi as applied to claim 7, 15, and 23 above, and further in view of Tan et al., (US20150359517A1), and Kornblau (US20110105895A1).
Regarding claim 8, modified Schneider teaches the method of claim 7, but fails to explicitly disclose detecting 2D structures in the 2D ultrasound image.
In the same laparoscopy field of endeavor, Hasser teaches detecting 2D structures in the 2D ultrasound image ([0043] the laparoscopic ultrasound probe provides two-dimensional ultrasound image slices of the anatomic structure).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Schneider with the imaging of Hasser, as this would assist surgeons in accurately guiding a tool to a target (see Hasser Abstract).
However, the combination of references still fails to teach accessing a 3D model for a target, representing a target organ and related anatomical structures including blood vessels; determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the ultrasound model; generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice; and providing labels to the 2D structures that correspond to blood vessels.
However in the same ultrasound field of endeavor, Tan teaches accessing a 3D model for a target (fig. 3 3D model of the region of interest [0032]), representing a target organ and related anatomical structures including blood vessels (fig. 3 The processing unit includes an adaptive penetration algorithm that highlights layers within the three-dimensional model that include rich structures (e.g., blood vessels, surfaces of internal organs, etc. [0032]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the modeling of Tan, as this would enable surgeons to visualize surgical sites during minimally invasive surgical procedures (See Tan [0008]).
However, the combination of references still fails to teach determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe; generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice.
In the same surgical field of endeavor, Kornblau teaches determining a slice of the 3D model based on the 3D pose of the device and the ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe ([0070] a 2d slice of the 3D image may be viewed that is based on the orientation and position of the 3D image, the image is of the imaging device and its model); generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice ([0070] a perspective view of the blood vessels are show, and the view would be a virtual view of the vessels as it would be viewed through a display screen).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the viewing of Kornblau, as this minimize or reduce unnecessary trauma to the patient (see Kornblau [0070]). One of ordinary skill would understand that this modification would result in the teaching of determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the ultrasound model; generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice.
However, the combination of references are still silent regarding providing labels to the 2D structures that correspond to blood vessels.
In the same medical imaging field of endeavor, Yagi teaches providing labels to the structures that correspond to blood vessels ([0050] the blood vessel is identified and labeled based on the position and the orientation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Schneider with the labeling of Yagi, as this would generate a more precise and reliable shape model of the vessel (see Yagi [0005]). One of ordinary skill would understand that this modification would result in the teaching of providing labels to the 2D structures that correspond to blood vessels.
Regarding claim 16, modified Schneider teaches the method of claim 15, but fails to explicitly disclose detecting 2D structures in the 2D ultrasound image.
In the same laparoscopy field of endeavor, Hasser teaches detecting 2D structures in the 2D ultrasound image ([0043] the laparoscopic ultrasound probe provides two-dimensional ultrasound image slices of the anatomic structure).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of Schneider with the imaging of Hasser, as this would assist surgeons in accurately guiding a tool to a target (see Hasser Abstract).
However, the combination of references still fails to teach accessing a 3D model for a target, representing a target organ and related anatomical structures including blood vessels.
However in the same ultrasound field of endeavor, Tan teaches accessing a 3D model for a target (fig. 3 3D model of the region of interest [0032]), representing a target organ and related anatomical structures including blood vessels (fig. 3 The processing unit includes an adaptive penetration algorithm that highlights layers within the three-dimensional model that include rich structures (e.g., blood vessels, surfaces of internal organs, etc. [0032]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the modeling of Tan, as this would enable surgeons to visualize surgical sites during minimally invasive surgical procedures (See Tan [0008]).
However, the combination of references still fails to teach determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe; generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice.
In the same surgical field of endeavor, Kornblau teaches determining a slice of the 3D model based on the 3D pose of the device and the ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe ([0070] a 2d slice of the 3D image may be viewed that is based on the orientation and position of the 3D image, the image is of the imaging device and its model); generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice ([0070] a perspective view of the blood vessels are show, and the view would be a virtual view of the vessels as it would be viewed through a display screen).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the viewing of Kornblau, as this minimize or reduce unnecessary trauma to the patient (see Kornblau [0070]). One of ordinary skill would understand that this modification would result in the teaching of determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the ultrasound model; generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice.
However, the combination of references are still silent regarding providing labels to the 2D structures that correspond to blood vessels.
In the same medical imaging field of endeavor, Yagi teaches providing labels to the structures that correspond to blood vessels ([0050] the blood vessel is identified and labeled based on the position and the orientation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the medium of modified Schneider with the labeling of Yagi, as this would generate a more precise and reliable shape model of the vessel (see Yagi [0005]). One of ordinary skill would understand that this modification would result in the teaching of providing labels to the 2D structures that correspond to blood vessels.
Regarding claim 24 modified Schneider teaches the system of claim 23, but fails to explicitly disclose detecting 2D structures in the 2D ultrasound image.
In the same laparoscopy field of endeavor, Hasser teaches detecting 2D structures in the 2D ultrasound image ([0043] the laparoscopic ultrasound probe provides two-dimensional ultrasound image slices of the anatomic structure).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of Schneider with the imaging of Hasser, as this would assist surgeons in accurately guiding a tool to a target (see Hasser Abstract).
However, the combination of references still fails to teach accessing a 3D model for a target, representing a target organ and related anatomical structures including blood vessels.
However in the same ultrasound field of endeavor, Tan teaches accessing a 3D model for a target (fig. 3 3D model of the region of interest [0032]), representing a target organ and related anatomical structures including blood vessels (fig. 3 The processing unit includes an adaptive penetration algorithm that highlights layers within the three-dimensional model that include rich structures (e.g., blood vessels, surfaces of internal organs, etc. [0032]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Schneider with the modeling of Tan, as this would enable surgeons to visualize surgical sites during minimally invasive surgical procedures (See Tan [0008]).
However, the combination of references still fails to teach determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe; generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice.
In the same surgical field of endeavor, Kornblau teaches determining a slice of the 3D model based on the 3D pose of the device and the ultrasound model, the ultrasound model modeling physical characteristics of the ultrasound probe ([0070] a 2d slice of the 3D image may be viewed that is based on the orientation and position of the 3D image, the image is of the imaging device and its model); generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice ([0070] a perspective view of the blood vessels are show, and the view would be a virtual view of the vessels as it would be viewed through a display screen).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Schneider with the viewing of Kornblau, as this minimize or reduce unnecessary trauma to the patient (see Kornblau [0070]). One of ordinary skill would understand that this modification would result in the teaching of determining a slice of the 3D model based on the 3D pose of the ultrasound probe and the ultrasound model; generating a virtual 2D blood vessel image based on each blood vessel present in the 2D slice.
However, the combination of references are still silent regarding providing labels to the 2D structures that correspond to blood vessels.
In the same medical imaging field of endeavor, Yagi teaches providing labels to the structures that correspond to blood vessels ([0050] the blood vessel is identified and labeled based on the position and the orientation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the system of modified Schneider with the labeling of Yagi, as this would generate a more precise and reliable shape model of the vessel (see Yagi [0005]). One of ordinary skill would understand that this modification would result in the teaching of providing labels to the 2D structures that correspond to blood vessels.
Response to Arguments
Applicant’s arguments, see pg. 10-13, filed 01/06/2026, have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Change et al., “Spaceborne pose determination based on image to 3D digital surface model matching” 2022 The Authors. IET Image Processing published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology teaches of pose determination based on 3D model matching
Drover et al., "Can 3D Pose be Learned from 2D Projections Alone?" ECCVW 2018 teaches of 3D pose estimation from a single image
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL Y FANG whose telephone number is (571)272-0952. The examiner can normally be reached Mon - Friday 9:30 am - 6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at 5712722714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL YIMING FANG/Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights