Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant’s Response
In Applicant’s Response dated 4/22/26, the Applicant amended Claims 1, 2, 7, 8-10, 14-17, 20-22, added Claims 25, 26 and argued Claims previously rejected in the Office Action dated 1/22/26. Claims 1-22, 25 and 26 are pending examination.
In light of the Applicant’s amendments and remarks, the 35 USC 101 rejections have been withdrawn.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 7-9, 13-22 and 25-26 rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kyperountas, United States Patent Publication 20220071711.
Claim 1:
Kyperountas discloses:
An information processing apparatus comprising:
one or more processors that operate to (see paragraph [0032]). Kyperountas teaches processors:
obtain a three dimensional image or image model of an object, target, or sample (see paragraph [0041]). Kyperountas teaches obtaining a 3D images of image during a medical procedure;
acquire positional information of an image capturing tool inserted in or into the object, target, or sample (see paragraph [0050]). Kyperountas teaches acquiring positional information of the tool inserted into the body;
determine, based on the positional information of the image capturing tool and the relative position of the image capturing tool to the object, target, or sample, portions of the object, target, or sample that are a first portion and a second portion of the three dimensional image or image model, where the first portion corresponds to a portion of the object, target, or sample where the image capturing tool has captured or inspected and the second portion corresponds to a portion of the object, target, or sample that remains to be captured or inspected or where the image capturing tool still is such that the second portion is uncaptured or uninspected (see paragraphs [0015], [0016] and [0050]-[0056], [0082]). Kyperountas teaches determining with the image capturing tool and the model an area/target that has been examined and an area that has not been examined yet, Kyperountas teaches displaying visual cues such as color or shapes; and
update, based on the acquired positional information, (i) the first portion of the three dimensional image or image model of the object, target, or sample with a first indicator or color expression for an area and/or border of the first portion and representing image data indicating how much of the object, target, or sample has been captured or inspected and (iii) the second portion of the three dimensional image or image model of the object, target, or same with a second indicator or color expression for an area and/or border of the second portion and representing image data indicating how much of the object, target, or sample remains to be captured or inspected (see paragraph [0015], [0016], [0063] and [0066]). Kyperountas teaches displaying portions of the images showing what areas were and were not yet examined in color by an imaging sensor.
display the updated first and second portions of the three dimensional image or image model of the object, target, or sample along with the first and second indicators or color expressions to indicate whether a set or predetermined amount, or all, of the areas or the portions of the object, target, or sample have been captured or inspected by the image capturing tool (see paragraph [0055], [0073] and [0074]). Kyperountas teaches display the updated portions of the image indicating whether all the portions of the object have been captured. It is displayed which portions are imaged and which portions are missing.
Claim 2:
Kyperountas discloses:
wherein the one or more processors further operate to one or more of the following:
(i) update and/or display the first portion using a line having a first pattern as the first indicator or color expression, and update and/or display the second portion using another line having a second pattern as the second indicator or color expression; and/or
(ii) update and/or display the first portion using a first color as the first indicator or color expression, and update and/or display the second portion and/or for a second color as the second indicator or color expression, wherein the second color is different from the first color (see paragraphs [0066] and [0082]). Kyperountas teaches a display displaying the second portion of the image that was not examined along with other portions. The colors of the unexamined image are different than the examined region.
Claim 3:
Kyperountas discloses:
wherein one or more of the following: (i) the object, target, or sample is one or more of the following: an anatomy, a kidney, a urinary system, or a portion of a kidney or urinary system; and/or (ii) CT data is obtained and/or used to segment the object, target, or sample (see paragraph [0030] and [0056]). Kyperountas teaches anatomy of the body and a CT data obtained to view the anatomy.
Claim 4:
Kyperountas discloses:
the first portion corresponds to a portion that the image capturing tool has captured or inspected in a Field-of-View of the image capturing tool, and the second portion corresponds to a portion that remains to be captured or inspected in the Field-of-View of the image capturing tool (see paragraphs [0012] and [0019]). Kyperountas teaches having an inspected area of the image in the Field of View and having a portion of the image in the field-of-view that has not been inspected yet.
Claim 5:
Kyperountas discloses:
wherein the captured or inspected first portion represents or corresponds to an overlap between an imaging Field-of-View (FOV) or a view model of the image capturing tool and the surfaces of the object, target, or sample being imaged (see paragraphs [0048] and [0106]). Kyperountas teaches the first portion corresponding to an overlap between the field-of-view.
Claim 7:
Kyperountas discloses:
receive a predetermined or set acceptable size, or a size within a predetermined or set range, of a missed or uninspected/uncaptured area, the missed or uninspected/uncaptured area being an area or portion that is not captured or inspected, or remains to be captured or inspected, by the image capturing tool and/or display the predetermined or set acceptable size along with the displayed updated fir and second portions (see paragraph [0054]). Kyperountas teaches receiving a size of a region within a threshold of an unexamined area; and
update and display a third portion of the three dimensional image or image model of the object, sample, or target with a third indicator or color expression which is different from both of the expressions of the first portion and the second portion of the three dimensional image or image model, wherein the third portion corresponds to, and the third indicator or color expression represents image data representing, the missed or uninspected/uncaptured area of which size is equal to or less than the predetermined or set acceptable size (see paragraph [0054]-[0055], and [0082]). Kyperountas teaches displaying a 3rd portion that is unexamined but also not a region of interest based on the threshold size. It is also taught the colors or shape of the arrows change depending on the parameters.
the third indicator or color expression is shown as a third indicator line or a color that is different from any line pattern or color used for the first indicator or color expression and from any line pattern or color used for the second indicator or color expression (see paragraph [0054]-[0055], and [0082]). Kyperountas teaches the colors or shape of the arrows change depending on the parameters and the thresholds.
Claim 8:
Kyperountas discloses:
receive a predetermined or set acceptable percentage of a completion of a capturing or inspection of the object, target, or sample and/or display the predetermined or set acceptable percentage along with the displayed updated first and second portions (see paragraph [0016]). Kyperountas teaches an indication of an amount that was examined vs unexamined and determining if an alert is sent based on the amount; and
display a percentage of an area or portion captured or inspected by the image capturing tool, and/or indicate a completion of the capturing or inspection of the object, target, or sample, in a case where the percentage of an area or portion captured or inspected by the image capturing tool is equal to or more than the predetermined or set acceptable percentage (see paragraph [0016], [0076]). Kyperountas determines whether the amount of acceptable examined area is received and determining if an alert is to be sent. A measure is provided to indicate how much of the overall region was left unexamined. Kyperountas teaches display to the user and provides the clinician or other user the ability to provide input for selecting a region of interest and/or for double checking the clinician's belief that the region of interest has been fully examined
Claim 9:
Kyperountas discloses:
store time information corresponding to a length of time that a particular portion or area is within the Field-of-View of the image capturing tool (see paragraph [0080] and [0081]). Kyperountas teaches storing time stamps to indicate the amount of time spent in each area of the field of view; and
display the three dimensional image or image model of the anatomy with the first indicator or color expression of the first portion after the image capturing tool has captured or inspected the first portion for a period of time indicated by the stored time information (see paragraphs [0080]-[0082]). Kyperountas teaches displaying the 3D image and model indicating the examined and unexamined portions and the corresponding time stamps.
Claim 13:
Kyperountas discloses:
wherein the one or more processors further operate to: acquire the positional information of the image capturing tool based on a positional information detected by an electromagnetic sensor, the positional information including orientation and position or location information (see paragraph [0050]). Kyperountas teaches magnetometers, accelerometers, and/or the like, which can used to estimate the position and/or direction of orientation of the medical instrument.
Claim 14:
Kyperountas discloses:
wherein the one or more processors further operate to: display, based on a depth map from or based on a captured image captured by the image capturing tool, the three dimensional image or image model of the object, sample, or target (see paragraph [0053]). Kyperountas teaches displaying a depth map based on captured image.
Claim 15:
Kyperountas discloses:
wherein the one or more processors further operate to: display, based on the acquired positional information, the first portion of the three dimensional image or image model of the object, sample, or target with a first color for the first indicator or color expression, the first color being different from a second color of or used for the second indicator or color expression for the second portion of the three dimensional image or image model (see paragraphs [0081] and [0082]). Kyperountas teaches displaying the position of the first portion and second portion with different colors.
Claim 16-21:
Although Claims 16-21 are method claims, they are interpreted and rejected for the same reasons as the apparatus of Claims 1-4, 14, 15.
Claim 22:
Although Claim 22 is a non-transitory storage medium claim, it is interpreted and rejected for the same reason as the apparatus of Claim 1.
Claim 25:
Kyperountas discloses:
wherein one or more of the following:
(i) in a case where the first portion is updated and/or displayed using the first line
pattern and the second portion is updated and/or displayed using the second line pattern, the first line pattern is a solid line and the second line pattern is a dashed line, and/or
(ii) in a case where the first portion is updated and/or displayed using the first color and the second portion is updated and/or displayed using the second color, the first color is a set or predetermined color or red and the second color is another set or predetermined color or yellow (see paragraph [0082]). Kyperountas teaches display the second portion in a different color or length of an arrow based on the region being examined and unexamined.
Claim 26:
Kyperountas discloses:
wherein one or more of the following:
(i) in a case where the first portion is updated and/or displayed using the first line
pattern and the second portion is updated and/or displayed using the second line pattern, the first line pattern is a solid line and the second line pattern is a dashed line, and/or
(ii) in a case where the first portion is updated and/or displayed using the first color and the second portion is updated and/or displayed using the second color, the first color is a set or predetermined color or red and the second color is another set or predetermined color or yellow (see paragraph [0082]). Kyperountas teaches display the second portion in a different color or length of an arrow based on the region being examined and unexamined.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Kyperountas, in view of Tam et al., United States Patent Publication 20230190394 (hereinafter “Tam”).
Claim 6:
Kyperountas fails to teach a view model for viewing during the procedure for the camera tool.
Tam discloses:
wherein the view model is a cone or other geometric shape being used for the model, and the portion that the image capturing tool has captured or inspected includes inner surfaces of a kidney that are located within the FOV or the view model (see paragraph [0196]). Tam teaches a rectangular geometric shape for the model for the image capturing tool of the kidney.
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention was filed to modify the method disclosed by Kyperountas to include view model for the image capturing tool for the purpose of defining a field of view for effectively capturing images, as taught by Tam.
Claim 12:
Kyperountas fails to teach a shape sensor.
Tam discloses:
wherein the one or more processors further operate to: acquire the positional information of the image capturing tool based on a shape sensor of the image capturing tool, the positional information including orientation and position or location information (see paragraph [0152]). Tam teaches a number of other input data can be used by the localization module. An instrument utilizing shape-sensing fiber can provide shape data that the localization module can use to determine the location and shape with the instrument.
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention was filed to modify the method disclosed by Kyperountas to include a shape sensor for the purpose of efficiently defining a shape with the image capturing tool , as taught by Tam.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Kyperountas, in view of Kristensen al., United States Patent Publication 20230346199 (hereinafter “Kristensen”).
Claim 10:
Kyperountas fails to teach the time information is the accumulated during to have overlap between an imaging Field-of-View of the image capturing tool and the surfaces being imaged.
Kristensen discloses:
wherein the stored time information is the accumulated time while having an overlap between an imaging Field-of-View of the image capturing tool and the surfaces of the object, target, or sample being imaged (see paragraphs [0027] and [0040]). Kristensen teaches combining left and right images from overlapping fields of view of an image sensor from a stereo camera, it should be understood that the disclosed technology may also be used in embodiments which provide reconstructions based on images extending beyond individual fields of view of a stereo camera’s image sensors, such as images captured over time. Time is stored for all images and fields of view.
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention was filed to modify the method disclosed by Kyperountas to include the time information is the accumulated during to have overlap between an imaging Field-of-View of the image capturing tool and the surfaces being imaged for the purpose of effectively storing times for overlapping fields of view, as taught by Kristensen.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Kyperountas, in view of Ayvali al., United States Patent Publication 20210196399 (hereinafter “Ayvali”).
Claim 11:
Kyperountas fails to teach obtaining position and location information using a forward kinematics model.
Ayvali discloses:
wherein the one or more processors further operate to: acquire the positional information of the image capturing tool based on a shape of the image capturing tool calculated based on a forward kinematics model, the positional information including orientation and position or location information (see paragraph [0232]). Ayvali teaches using a forward kinematic model to obtain position information during a kidney procedure.
Accordingly, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention was filed to modify the method disclosed by Kyperountas to include a forward kinematics model for the purpose of effectively receiving position and location data from the procedure, as taught by Ayvali.
Response to Arguments
Applicant's arguments filed 4/22/26 have been fully considered but they are not persuasive.
Claim Rejection – 35 USC 101:
Applicant argues Applicant submits that the subject § 101 claim rejection is
deficient and/or has been rendered as moot, and respectfully requests that the subject §
101 claim rejection be withdrawn accordingly.
The Examiner agrees and the rejection has been withdrawn.
Claim Rejection – 35 USC 102
Applicant argues However, there is no allegation as to what the "first expression"
is and the "second expression" is of previously presented claim 1. For at least this reason, the pending rejection has failed to present a prima facie case of anticipation, and consequently, the obviousness rejections relying of the Kyperountas reference have likewise failed to present prima facie cases of obviousness.
The Examiner disagrees.
Kyperountas causes a display to display the composite model and information relating to the section of the internal region. The information may include a visualization of the section of the internal region on the composite model. In the event that the section of the internal region on the composite model has been determined to be a region of interest in operation 320, then the information may include audio and/or visual cues and directions for the clinician to navigate the medical instrument 140 to the section of the internal region (see paragraph [0066]). Kyperountas teaches visual cues to distinguish the examiner and unexamined portions. It is also taught that different colors or shapes. Thus, Kyperountas teaches the color expressions.
Kyperountas displaying different indicators of unexamined areas and examined areas. Any color can be chosen to be the indicating color of the different areas. Kyperountas also teaches different shapes/lengths of the arrows based on if the areas examined/unexamined (see paragraph [0086). It is a design choice of the user to use a patterned line to distinguish areas that have been examined. Thus, Kyperountas teaches the limitations of the claim.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIONNA M BURKE whose telephone number is (571)270-7259. The examiner can normally be reached M-F 8a-4p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at (571)272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIONNA M BURKE/ Examiner, Art Unit 2178 6/27/26
/STEPHEN S HONG/ Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights