DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 22, and 42 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 2, line 5 recites “generating a fragment based upon differences between the two features”. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, it shall be considered as “generating a fragment based upon differences between the features”.
Regarding claim 22, line 5 recites “generating a fragment based upon differences between the two features”. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, it shall be considered as “generating a fragment based upon differences between the features”.
Regarding claim 42, line 5 recites “generating a fragment based upon differences between the two features”. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, it shall be considered as “generating a fragment based upon differences between the features”.
Claims that are not discussed above but are cited to be rejected under 35 U.S.C. 112(b) are also rejected because they inherit the indefiniteness of the claims they respectively depend upon.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8, 21-28, and 41-44 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- The claim recites a method for assessing surgical instrument progress and is therefore a process.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining pose data associated with a surgical instrument; determining depth data associated with the pose data; and determining a position of a centerline associated with the at least the portion of the three-dimensional model.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitation “constructing at least a portion of a three-dimensional model of at least a portion of the patient interior based upon the pose data and the depth data”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim recites the following addition elements “A computer-implemented method for assessing surgical instrument progress within a patient interior, the method comprising:”. These steps merely amount to pre-solution insignificant activities.
This additional element, taken individually or in combination, merely amounts to insignificant extra solution activities because they pertain to a generic computer implementation.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. Ma et al., RNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy. Med Image Anal. 2021 Aug;72:102100. doi: 10.1016/j.media.2021.102100. Epub 2021 May 19. PMID: 34102478; PMCID: PMC8316389. (hereinafter “Ma”) teaches of a computer implement method for accessing surgical instrument progress within a patient’s interior. As such, this limitation is insignificant extra-solution activity, and there is no inventive concept in the claim. In light of the above, claim 1 is ineligible.
Claim 2 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 1.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining features between two images; generating a fragment based upon differences between the two features”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitation “consolidating the fragments to form the at least the portion of the three-dimensional model.”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 2 is ineligible.
Claim 3 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 1.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “providing the depth data to a neural network configured to in-fill portions of the at least the portion of the three-dimensional model”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation that is performed in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 3 is ineligible.
Claim 4 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 1.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining a kinematics threshold, wherein, the kinematics threshold is one of: a speed threshold for motion of at least a portion of the surgical instrument projected upon the centerline; a speed threshold for motion of at least a portion of the surgical instrument projected radially from the centerline; and a distance of at least a portion of the surgical instrument from the centerline. ”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 4 is ineligible.
Claim 5 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 4.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “kinematics threshold is a speed threshold for motion of at least the portion of the surgical instrument projected upon the centerline, and wherein, the method further comprises: determining that the surgical instrument is being withdrawn; and determining that a speed of the surgical instrument projected upon the centerline exceeds the speed threshold”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 5 is ineligible.
Claim 6 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 5.
Step 2A, Prong 1, Judicial Exception: Yes- the claim recites “determining the kinematics threshold based upon kinematics data values in the database corresponding to times when surgical instruments were being withdrawn.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim recites the additional claim elements “ consulting a database comprising surgical instrument kinematics data projected upon reference geometries for a plurality of surgical operations”. This step merely amounts to pre-solution insignificant activities.
This additional element, taken individually or in combination, merely amounts to insignificant extra solution activities because they pertain to merely data gathering in a generic computer implementation.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. Yoshie et al., (US20160081531A1) teaches of consulting past procedures to determine a threshold. As such, this limitation is insignificant extra-solution activity, and there is no inventive concept in the claim. In light of the above, claim 6 is ineligible.
Claim 7 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 6.
Step 2A, Prong 1, Judicial Exception: Yes- the claim recites “filtering the at least the portion of the three-dimensional model to produce a filtered portion”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the claim recites “determining centerline endpoints based upon the filtered portion; generating a new local centerline from poses of the surgical instrument; and extending the centerline with the new local centerline.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite further claim elements
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the claim does not recite further claim elements and is thus ineligible.
Claim 8 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 7.
Step 2A, Prong 1, Judicial Exception: Yes- the claim recites “determining a weighted average between pairs of points in the first array and in the second array”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the claim recites “determining a first array of points on the centerline; determining a second array of points on the new local centerline”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite further claim elements
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the claim does not recite further claim elements and is thus ineligible.
Claim 21 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- The claim recites a method for assessing surgical instrument progress and is therefore a process.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining pose data associated with a surgical instrument; determining depth data associated with the pose data; and determining a position of a centerline associated with the at least the portion of the three-dimensional model.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitation “constructing at least a portion of a three-dimensional model of at least a portion of the patient interior based upon the pose data and the depth data”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim recites the following addition elements “A non-transitory computer readable medium, the transitory computer-readable medium comprising instructions configured to cause a computer system to perform a method for assessing surgical instrument progress within a patient interior, the method comprising:”. These steps merely amount to pre-solution insignificant activities.
This additional element, taken individually or in combination, merely amounts to insignificant extra solution activities because they pertain to a generic computer implementation.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. Ma et al., RNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy. Med Image Anal. 2021 Aug;72:102100. doi: 10.1016/j.media.2021.102100. Epub 2021 May 19. PMID: 34102478; PMCID: PMC8316389. (hereinafter “Ma”) teaches of a computer implement method for accessing surgical instrument progress within a patient’s interior. As such, this limitation is insignificant extra-solution activity, and there is no inventive concept in the claim. In light of the above, claim 21 is ineligible.
Claim 22 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 21.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining features between two images; generating a fragment based upon differences between the two features”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitation “consolidating the fragments to form the at least the portion of the three-dimensional model.”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 22 is ineligible.
Claim 23 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 21.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “providing the depth data to a neural network configured to in-fill portions of the at least the portion of the three-dimensional model”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation that is performed in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 23 is ineligible.
Claim 24 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 21.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining a kinematics threshold, wherein, the kinematics threshold is one of: a speed threshold for motion of at least a portion of the surgical instrument projected upon the centerline; a speed threshold for motion of at least a portion of the surgical instrument projected radially from the centerline; and a distance of at least a portion of the surgical instrument from the centerline. ”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 24 is ineligible.
Claim 25 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 24.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “kinematics threshold is a speed threshold for motion of at least the portion of the surgical instrument projected upon the centerline, and wherein, the method further comprises: determining that the surgical instrument is being withdrawn; and determining that a speed of the surgical instrument projected upon the centerline exceeds the speed threshold”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 25 is ineligible.
Claim 26 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 25.
Step 2A, Prong 1, Judicial Exception: Yes- the claim recites “determining the kinematics threshold based upon kinematics data values in the database corresponding to times when surgical instruments were being withdrawn.”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim recites the additional claim elements “ consulting a database comprising surgical instrument kinematics data projected upon reference geometries for a plurality of surgical operations”. This step merely amounts to pre-solution insignificant activities.
This additional element, taken individually or in combination, merely amounts to insignificant extra solution activities because they pertain to merely data gathering in a generic computer implementation.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. Yoshie et al., (US20160081531A1) teaches of consulting past procedures to determine a threshold. As such, this limitation is insignificant extra-solution activity, and there is no inventive concept in the claim. In light of the above, claim 26 is ineligible.
Claim 27 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 26.
Step 2A, Prong 1, Judicial Exception: Yes- the claim recites “filtering the at least the portion of the three-dimensional model to produce a filtered portion”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the claim recites “determining centerline endpoints based upon the filtered portion; generating a new local centerline from poses of the surgical instrument; and extending the centerline with the new local centerline.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite further claim elements
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the claim does not recite further claim elements and is thus ineligible.
Claim 28 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 27.
Step 2A, Prong 1, Judicial Exception: Yes- the claim recites “determining a weighted average between pairs of points in the first array and in the second array”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Further, the claim recites “determining a first array of points on the centerline; determining a second array of points on the new local centerline”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite further claim elements
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the claim does not recite further claim elements and is thus ineligible.
Claim 41 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- The claim recites a computer system for assessing surgical instrument progress and is therefore a machine.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining pose data associated with a surgical instrument; determining depth data associated with the pose data; and determining a position of a centerline associated with the at least the portion of the three-dimensional model.”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitation “constructing at least a portion of a three-dimensional model of at least a portion of the patient interior based upon the pose data and the depth data”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim recites the following addition elements “A computer system comprising: at least one processor; and at least one memory, the at least one memory comprising instructions configured to cause the computer system to perform a method for assessing surgical instrument progress within a patient interior, the method comprising”. These steps merely amount to pre-solution insignificant activities.
This additional element, taken individually or in combination, merely amounts to insignificant extra solution activities because they pertain to a generic computer implementation.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. Ma et al., RNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy. Med Image Anal. 2021 Aug;72:102100. doi: 10.1016/j.media.2021.102100. Epub 2021 May 19. PMID: 34102478; PMCID: PMC8316389. (hereinafter “Ma”) teaches of a computer implement method for accessing surgical instrument progress within a patient’s interior. As such, this limitation is insignificant extra-solution activity, and there is no inventive concept in the claim. In light of the above, claim 41 is ineligible.
Claim 42 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 41.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining features between two images; generating a fragment based upon differences between the two features”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Further, the claim recites the limitation “consolidating the fragments to form the at least the portion of the three-dimensional model.”.
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 42 is ineligible.
Claim 43 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 41.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “providing the depth data to a neural network configured to in-fill portions of the at least the portion of the three-dimensional model”
This limitation, as drafted, is a process step, that, under its BRI, covers an abstract idea, more specifically a mathematical concept. That is, the listed claim recites a mathematical calculation that is a mathematical operation or an act of calculating using mathematical methods to determine a variable or number, e.g., performing an arithmetic equation such as exponentiation that is performed in a computer environment. Accordingly, the claim recites a mathematical concept-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 43 is ineligible.
Claim 44 is rejected under 35 U.S.C. 101 because the invention is directed to an abstract idea without significantly more.
Step 1: Statutory category: Yes- for the same reasons put forth in claim 41.
Step 2A, Prong 1, Judicial Exception: Yes- The claim recites the limitation “determining a kinematics threshold, wherein, the kinematics threshold is one of: a speed threshold for motion of at least a portion of the surgical instrument projected upon the centerline; a speed threshold for motion of at least a portion of the surgical instrument projected radially from the centerline; and a distance of at least a portion of the surgical instrument from the centerline. ”
This limitation, as drafted, is a process step that, under its broadest reasonably interpretation (BRI), covers an abstract idea, more specifically a process that can be performed in the human mind. That is, nothing in the listed claim elements precludes the step from practically being performed in the mind and/or performed with the aid of a pen and paper, and/or performing the process in a computer environment. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A Prong 2, Integrated into Practical Application: No- The claim does not recite additional claim elements.
Step2B Inventive Concept: No- Similarly to Step2A Prong 2, there are no additional claim elements. In light of the above, claim 44 is ineligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 21, and 41 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ma et al., RNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy. Med Image Anal. 2021 Aug;72:102100. doi: 10.1016/j.media.2021.102100. Epub 2021 May 19. PMID: 34102478; PMCID: PMC8316389. (hereinafter “Ma”).
Regarding claim 1, Ma teaches A computer-implemented method for assessing surgical instrument progress within a patient interior (Section 1. Introduction, 1.2 Solution: Reconstruction 3D Colon by SLAM: an endoscopists can visualize the missing parts during an colonoscopy through reconstruction), the method comprising: determining pose data associated with a surgical instrument (Section 4. Method , 4.1 Pipeline Step 1 Predict Pose for each input frame, see figure 4; 4.2 RNN Training the camera pose is outputted); determining depth data associated with the pose data (Section 4. Method , 4.1 Pipeline Step 1 Predict Depth for each input frame, see figure 4; 4.2 RNN Training the Depth map is outputted); constructing at least a portion of a three-dimensional model of at least a portion of the patient interior based upon the pose data and the depth data (4.1 Pipeline Step 6 Fusion: fuse the RGB values, the RNN-predicted depth map and optimized camera poses into a global mesh; Section 5 Experiments; 5.1 Qualitative Results “—Fig. 10 shows the incremental fusion process of a chunk of colon. The camera is moving along with the latest keyframe. The snapshots were captured in real time.” Fig. 10-14 show reconstruction examples with missing regions); and determining a position of a centerline associated with the at least the portion of the three-dimensional model (Section 5 Experiments; 5.3.1 Quantify Unsurveyed Regions for each 3D colon chuck surface, a straight centerline is established).
Regarding claim 41, Ma teaches A non-transitory computer-readable medium, the non-transitory computer- readable medium comprising instructions configured to cause a computer system to perform a method for assessing surgical instrument progress within a patient interior, the method comprising (Section 1. Introduction, 1.2 Solution: Reconstruction 3D Colon by SLAM: an endoscopists can visualize the missing parts during an colonoscopy through reconstruction, and the reconstruction would be done on a computer), the method comprising: determining pose data associated with a surgical instrument (Section 4. Method , 4.1 Pipeline Step 1 Predict Pose for each input frame, see figure 4; 4.2 RNN Training the camera pose is outputted); determining depth data associated with the pose data (Section 4. Method , 4.1 Pipeline Step 1 Predict Depth for each input frame, see figure 4; 4.2 RNN Training the Depth map is outputted); constructing at least a portion of a three-dimensional model of at least a portion of the patient interior based upon the pose data and the depth data (4.1 Pipeline Step 6 Fusion: fuse the RGB values, the RNN-predicted depth map and optimized camera poses into a global mesh; Section 5 Experiments; 5.1 Qualitative Results “—Fig. 10 shows the incremental fusion process of a chunk of colon. The camera is moving along with the latest keyframe. The snapshots were captured in real time.” Fig. 10-14 show reconstruction examples with missing regions); and determining a position of a centerline associated with the at least the portion of the three-dimensional model (Section 5 Experiments; 5.3.1 Quantify Unsurveyed Regions for each 3D colon chuck surface, a straight centerline is established).
Regarding claim 41, Ma teaches A computer system comprising: at least one processor; and at least one memory, the at least one memory comprising instructions configured to cause the computer system to perform a method for assessing surgical instrument progress within a patient interior, the method comprising (Section 1. Introduction, 1.2 Solution: Reconstruction 3D Colon by SLAM: an endoscopists can visualize the missing parts during an colonoscopy through reconstruction, and the reconstruction would be done on a computer, which would inherently have a processor, and memory for the instructions), the method comprising: determining pose data associated with a surgical instrument (Section 4. Method , 4.1 Pipeline Step 1 Predict Pose for each input frame, see figure 4; 4.2 RNN Training the camera pose is outputted); determining depth data associated with the pose data (Section 4. Method , 4.1 Pipeline Step 1 Predict Depth for each input frame, see figure 4; 4.2 RNN Training the Depth map is outputted); constructing at least a portion of a three-dimensional model of at least a portion of the patient interior based upon the pose data and the depth data (4.1 Pipeline Step 6 Fusion: fuse the RGB values, the RNN-predicted depth map and optimized camera poses into a global mesh; Section 5 Experiments; 5.1 Qualitative Results “—Fig. 10 shows the incremental fusion process of a chunk of colon. The camera is moving along with the latest keyframe. The snapshots were captured in real time.” Fig. 10-14 show reconstruction examples with missing regions); and determining a position of a centerline associated with the at least the portion of the three-dimensional model (Section 5 Experiments; 5.3.1 Quantify Unsurveyed Regions for each 3D colon chuck surface, a straight centerline is established).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 22, and 42 is rejected under 35 U.S.C. 103 as being unpatentable over Ma as applied to claim 1 above, and further in view of Freedman et al., "Detecting Deficient Coverage in Colonoscopies," in IEEE Transactions on Medical Imaging, vol. 39, no. 11, pp. 3451-3462, Nov. 2020, doi: 10.1109/TMI.2020.2994221. (hereinafter “Freedman”).
Regarding claim 2, Ma teaches the method of claim 1, wherein Ma further teaches wherein constructing the at least the portion of the three-dimensional model of the at least the portion of the patient interior based upon the pose data and the depth data, comprises (Section 5 Experiments; 5.1 Qualitative Results “—Fig. 10 shows the incremental fusion process of a chunk of colon. The camera is moving along with the latest keyframe. The snapshots were captured in real time.” Fig. 10-14 show reconstruction examples): generating a fragment (4.5 Fusion SurfelMeshing creates a surfel cloud that is used to generate a triangle mesh from the surfel cloud); consolidating the fragments to form the at least the portion of the three-dimensional model (4.5 the triangle mesh is generated from the surfel cloud).
However, Ma fails to explicitly disclose determining features between two images; and differences between two the features.
In the same image processing field of endeavor, Freedman teaches determining features between two images (page 4 the current and previous RGB frames It and It-1 are used to get the pose; image metric δ is the loss between two images, which is governed by L1 difference and structural similarity); differences between the two feature (page 4 the current and previous RGB frames It and It-1 are used to get the pose; image metric δ is the loss between two images, which is governed by L1 difference and structural similarity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to apply the technique of using difference features as taught by Freedman to the features of Ma, as both inventions relate to endoscopic image processing, and would yield the predictable result of an image reconstruction method that utilizes image differences when reconstructing the model of a colon to one ordinary skill. One of ordinary skill would be able to perform such a technique, and the results of Ma utilizing the difference features to generate fragments and then consolidating those fragments to form a portion of the model are reasonably predictable. This would result in improved depth estimation and reconstruction consistency. One of ordinary skill would understand that the difference features would apply to the fragments taught by Ma, and result in the reading of the limitation “determining features between two images; generating a fragment based upon differences between the two features; and consolidating the fragments to form the at least the portion of the three-dimensional model.”
Claims 22 and 42 are rejected for substantially the same reasons as claim 2.
Claims 3, 23, and 43 are rejected under 35 U.S.C. 103 as being unpatentable over Ma as applied to claim 1 above, and further in view of Song et al., “Semantic Scene Completion from a Single Depth Image” arXiv:1611.08974v1 [cs.CV] 28 Nov 2016 (hereinafter “Song”).
Regarding claim 3, Ma teaches the method of claim 1, wherein Ma teaches determining the position of the centerline comprises(Section 5 Experiments; 5.31 Quantify Unsurveyed Regions for each 3D colon chuck surface, a straight centerline is established); but is silent regarding providing the depth data to a neural network configured to in-fill portions of the at least the portion of the three-dimensional model.
In a reference pertinent to the problem of 3D reconstruction, Song teaches providing the depth data to a neural network configured to in-fill portions of the at least the portion of the three-dimensional model (page 1-2, given a single-view depth map as input, the semantic scene completion network produces labels for all voxels in the view frustrum, and the prediction extends beyond the projection surface implied by the depth map, and thus providing occupancy information for the entire scene.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to apply the technique of providing depth data to a neural network as taught by Song to the method of Ma, as both inventions relate to 3D reconstruction from images, and would yield the predictable result of a reconstruction process that includes utilizing depth data to one of ordinary skill in the art. One of ordinary skill would be able to perform such a technique, and the results of Ma utilizing depth data as in input for a neural network are reasonably predictable. This would allow for enhanced visualization as the missing geometry may be predicted beyond the observed surface.
Claims 23 and 43 are rejected for substantially the same reasons as claim 3.
Claims 4, 5, 24, 25, and 44, are rejected under 35 U.S.C. 103 as being unpatentable over Ma as applied to claim 1 above, and further in view of Kamijo et al., (US20250139769A1) and in further view Liu et al., (US20140005530A1)
Regarding claim 4, Ma teaches the method of claim 1, but fails to explicitly disclose determining a kinematics threshold, wherein, the kinematics threshold is one of: a speed threshold formation of at least a portion of the surgical instrument.
However in the same endoscopic field of endeavor, Kamijo teaches determining a kinematics threshold, wherein, the kinematics threshold is one of: a speed threshold formation of at least a portion of the surgical instrument ([0049] there is a predetermined threshold value for withdrawal speed that is determined in advance).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to apply the technique of implementing a speed threshold for the instrument as taught by Kamijo to the method of Ma, as both inventions relate to intraluminal devices, and would yield the predictable result of the movement of an instrument having a speed threshold to one of ordinary skill. One of ordinary skill would be able to apply such a technique, and the results of Ma implementing a speed threshold for the camera are reasonably predictable. This would result in minimizing the risk of damage done by the camera to the patient.
However, the combination of references are still silent regarding the surgical instrument projected upon the centerline.
In the same intraluminal device field of endeavor, Liu teaches the surgical instrument projected upon the centerline ([0024] the catheter is inserted along the target vessel’s centerline).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application apply the technique of a surgical instrument projected upon the centerline as taught by Liu to the method of modified Ma, as both inventions relate to intraluminal devices, and would yield the predictable result of reconstruction method that has a speed threshold of a surgical instrument that is projected along the centerline to one of ordinary skill. One of ordinary skill would be able to perform such an application, and the results of the determined speed threshold of modified Ma being applied to an instrument that is on the centerline of a vessel are reasonably predictable.
Regarding 5, modified Ma teaches the method of claim 4, but fails to explicitly disclose wherein, the kinematics threshold is a speed threshold for motion of at least the portion of the surgical instrument, and wherein, the method further comprises: determining that the surgical instrument is being withdrawn; and determining that a speed of the surgical instrument projected upon the centerline exceeds the speed threshold.
However in the same intraluminal field of endeavor, Kamijo teaches wherein, the kinematics threshold is a speed threshold for motion of at least the portion of the surgical instrument([0049] there is a predetermined threshold value that is determined in advance), and wherein, the method further comprises: determining that the surgical instrument is being withdrawn ([0032] the endoscopic examination system 100 determines if the withdraw speed is equal or higher to the threshold value); and determining that a speed of the surgical instrument projected upon the centerline exceeds the speed threshold ([0032] the endoscopic examination system 100 determines if the withdraw speed is equal or higher to the threshold value, and this would be along the vessels center line ([0024])).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Ma with the determination of Kamijo, as this would allow examiners to easily grasp the accuracy of the exanimation, specifically where the withdraw speed was fast so that the examiner can check the region where there is a high possibility of overlooking something (see Kamijo [0063]).
Claims 24, 25, and 44 are rejected for substantially the same reasons as above.
Claims 6 and 26 rejected under 35 U.S.C. 103 as being unpatentable over Ma as modified by Kamijo and Liu applied to claim 5 above, and further in view of Yoshie et al., (US20160081531A1).
Regarding claim 6, modified Ma teaches the method of claim 5, but fails to explicitly disclose consulting a database comprising surgical instrument data projected upon reference for a plurality of surgical operations; and determining the threshold based upon data in the data corresponding to times when the instrument is being used.
In the same endoscope field of endeavor, Yoshi teaches consulting a database comprising surgical instrument data projected upon reference for a plurality of surgical operations; and determining the threshold based upon data in the data corresponding to times when the instrument is being used ([0008] a threshold setting section that sets a threshold on the basis of measurement results that were acquired for the same serial number in the past stored in the storing section).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application apply the known technique of setting a threshold based on past measurements as taught by Yoshie to the method of modified Ma, as both inventions relate to endoscopic procedures, and would yield the predictable result of a speed threshold being based on past operations to one of ordinary skill. One of ordinary skill would be able to apply such a technique, and the results of modified Ma using past operations to determine a threshold are reasonably predictable.
However, the combination of references are silent regarding the kinematics data values and when the surgical instruments were being withdrawn.
In the same intraluminal field of endeavor, Kamijo teaches the kinematics data projected upon reference geometries ([0049] there is a predetermined threshold value for withdrawal speed that is determined in advance; [0024] the instrument is inserted along the centerline of the vessel, and therefore the data would be along the centerline); and when the surgical instruments were being withdrawn([0049] there is a predetermined threshold value for withdrawal speed that is determined in advance).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to apply the technique of implementing a speed threshold for the instrument as taught by Kamijo to the method of Ma, as both inventions relate to intraluminal devices, and would yield the predictable result of the movement of an instrument having a speed threshold to one of ordinary skill. One of ordinary skill would be able to apply such a technique, and the results of Ma implementing a speed threshold for the camera are reasonably predictable. This would result in minimizing the risk of damage done by the camera to the patient. One of ordinary skill in the art would understand that implementing the speed threshold of Kamijo through the use of past data of modified Ma would result in the reading upon the limitation of “consulting a database comprising surgical instrument kinematics data projected upon reference geometries for a plurality of surgical operations; and determining the kinematics threshold based upon kinematics data values in the database corresponding to times when surgical instruments were being withdrawn.”
Claim 26 is rejected for substantially the same reasons as claim 6
Claims 7 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Ma as modified by Kamijo, Liu, and Yoshie applied to claim 6 above, and further in view of Goel et al., (US20190279367A1) and Mao (US20230214952A1).
Regarding claim 7, modified Ma teaches the method of claim 6, wherein Ma teaches generating a new local centerline from poses of the surgical instrument (5.3.1 Quantify Unsurveyed Regions the centerline for each 3D colon chunk is established, and these 3D chunks are modeled from the depth and pose; 4.1 Pipeline Step 6 Fusion: fuse the RGB values), but fails to explicitly disclose filtering the at least the portion of the three-dimensional model to produce a filtered portion.
In the same image processing field of endeavor, Goel teaches filtering the at least the portion of the three-dimensional model to produce a filtered portion (fig. 1 the image is preprocessed at step 104 and is then the centerline is identified 132).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Ma with the preprocessing of Goel, as this would allow in the facilitation of repairing a variety of anatomical structures by providing an accurate representation of those structures (see Goel [0034]).
However, the combination are still silent regarding determining centerline endpoints based upon the filtered portion; and extending the centerline with the new local centerline.
In the same image processing field of endeavor, Mao teaches determining centerline endpoints based upon the filtered portion ([0157] the processing device 130 determines a first and second endpoint, and determines a centerline between the points); and extending the centerline with the new local centerline ([0157] the processing device may also connect the centerlines of adjacent sub fragments).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of modified Ma with the centerline generation process of Mao, as this would help improve the extraction of centerlines from images (see Mao [0004]).
Claim 27 is rejected for substantially the same reasons as claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL Y FANG whose telephone number is (571)272-0952. The examiner can normally be reached Mon - Friday 9:30 am - 6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at 5712722714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL YIMING FANG/ Examiner, Art Unit 3798
/PASCAL M BUI PHO/ Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights