DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims.
The arguments and further explanations in the Response to Arguments section above are hereby incorporated by reference.
Fig. 1 is functional block diagram/flowchart that merely show blank boxes/blocks for reference numbers 100-170 which do not show the features of the invention as recited in the claims. In other words, raw reference numbers and otherwise blank boxes are simply not sufficient to show the claimed method features or high-level structural elements performing method steps.
Therefore, the recited method of claims 1-6 and the functional limitations of claims 7-113 must be shown or the features canceled from the claims. No new matter should be entered. It is suggested that English language labels be applied to Fig. 2 which would resolve this objection.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
the receiving unit, processing unit, and module for estimating position of trocar from plural images of claims 7-13; the sub-modules of claim 8; the module for determining from the images the position of at least one organ of claim 9; module for determining organ frame of reference of claim 10; sub-module of claim 11; and module for defining an operating zone of claim 12. Each of these claim elements utilizes a nonce term (unit, module, sub-module) without reciting structure sufficient to perform the recited functions; moreover none of these elements is recognized by those of ordinary skill in the art as denoting structure.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 7, 8, and 13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Rosa {B. Rosa, C. Gruijthuijsen, B. Van Cleynenbreugel, J. Vander Sloten, D. Reynaerts, and E. Vander Poorten, “Estimation of optimal pivot point for remote center of motion alignment in surgery,” International journal of computer assisted radiology and surgery, vol. 10, no. 2, pp. 205–215, 2015}.
Claim 1
In regards to claim 1, Rosa discloses a method for processing images for the automatic estimation of the position of at least one trocar (58), each trocar (58) being configured to permit the passage of a substantially straight instrument (54) therethrough
{See Introduction and cites below. It is noted that the trocar is positioned at the insertion point of the instruments into a body which also corresponds to the optimal pivot point (OPP), PI (pivot point in image), and the position of a remote center of motion (RCM) for surgical robotic manipulators performing laparoscopic procedures. See, for example, Rosa Introduction section “an important requirement is to respect the motion constraint imposed by the trocar point and, therefore, limit the stress upon the body wall”.
Rosa further states that the OPP is within the entry port into the body and does not necessarily coincide with the outer body surface; as such the OPP may be defined as “a point along the axis of the incision at a specific depth which leads to minimal interaction forces upon the body wall when instruments pivot around this point, pg. 206”.
Moreover, a trocar is three-dimensional object extending, in use, inwardly from the outer body surface to a depth/extent that encompasses the OPP. In other words, the OPP/PI/RCM lies within the trocar’s inner volume and corresponds to or is at least indicative of the claimed estimation of the trocar position}, characterized in that it comprises:
- a step (110) of receiving at least two images (118a, 118b), each of the images showing at least one instrument (54) passing through each trocar (58), said instrument being in a different orientation and/or position in at least two of the images (118a, 118b)
{see Introduction, OPP estimation using image processing, Proposed Method pgs. 207-208 including ex vivo images taken with a pre-calibrated camera, Estimation of the Optimal Pivot Point From Camera Images, Assumption 2 Teaching of OPP in which a surgeon moves an instrument freely and gently in the trocar point such that images may be taken by the camera and received by a processor},
- a step (130) of determining, from the images and in at least said two images, the position and the orientation of each instrument (54) with respect to a reference position
{Proposed Method pgs. 207-209 including extraction of straight lines from the camera images to determine the position and orientation of the instrument at the instrument with respect to a reference position for each of the images, Step 1 Line extraction}
- a step (160) of estimating, from the images, the position of each trocar (58) with respect to the reference position, from at least two positions and orientations of the instrument (54) passing through this trocar (58)
{pg. 209 including Step 3 RANSAC PI estimation which estimates the PI (pivot point in image) and equation (2). See also Expansion to 3D and Metric distances, pgs. 209-210 which estimates the OPP location in 3D space based on the PI}.
Claim 2
In regards to claim 2, Rosa discloses characterized in that the step (160) of estimating the position of each trocar (58) comprises for each trocar:
- a sub-step (160a) of defining a main axis (60a, 60b) of the instrument (54) passing through the trocar (58)
{pg. 209 Step 1 line extraction employs Canny edge detection to extract straight lines corresponding to the main axis of the instrument passing through the trocar},
- a sub-step (160b) of determining, from each image, a group of main axes of the instrument in each position and orientation of the instrument
{pg. 209, Step 2 line pruning keeps parallel lines for each instrument pose thus determining, for the set of instrument poses, a group of main axes of the instrument in each position and orientation of the instrument},
- a sub-step (160c) of determining the intersection (62) of the main axes of the group in each position and orientation of the instrument
{pg. 209, Step 3 RANSAC PI estimation that determines the intersection of the main axes},
- a sub-step (160d) of estimating the position of the trocar (58) by matching said position with the intersection (62) of the main axes corresponding to the instrument (54) passing through said trocar (58)
{Step 3.3 and equation (2). See also Expansion to 3D and Metric distances, pgs. 209-210 which estimates the OPP location (trocar position) in 3D space by matching said position with the intersection (62) of the main axes corresponding to the instrument (54) passing through said trocar (58)}.
Claims 7 and 8
The rejection of method claims 1 and 2 above applies mutatis mutandis to the corresponding limitations of device claims 7 and 8 while noting that the rejection above cites to both device and method disclosures.
Claim 13
In regards to claim 13, Rosa discloses an endoscopic imaging system, characterized in that it comprises an image-processing device as claimed in claim 7 {see above mapping) and a viewing screen (18) which is configured to display images received by the image-processing device and additional information supplied by the processing unit (16) depending on the estimated position of each trocar (58) {see pgs. 211-212 Experimental Protocol in which the RANSAC estimation of trocar position(s) and ground truth information (additional information) is displayed on the screen and which depends on the estimated position of each trocar}.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Rosa and Begg (US 20190209241 A1).
Claim 3
In regards to claim 3, Rosa discloses an image processing method and processor but does not include a step (120) of determining, from the images, the position of at least one organ visible in at least one of the images, referred to as target organ (52), with respect to the reference position.
Begg is analogous art from the same field of image-processing enhanced laparoscopic surgery using a trocar and also determines a position of a trocar. See abstract, [0001]-[0003] and cites below.
Begg teaches position sensors 34 and a sensor array 32 of a tracking system 30 that establishes a patient frame of reference and optically tracks the endoscope 40 and trocar 50 as per [0035]-[0042].
Begg also teaches determining, from the images, the position of at least one organ visible in at least one of the images, referred to as target organ (52), with respect to the reference position {see Fig. 3 including images of the patient’s abdominal cavity, surgical site, etc. are used to create a pre-procedure anatomical map that provides 3D location and identification of target tissue structures.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Rosa laparoscopic surgical imaging method and device to include a step (120) of determining, from the images, the position of at least one organ visible in at least one of the images, referred to as target organ (52), with respect to the reference position as taught by Begg because doing so usefully expands the range of information available to the surgeon including locating target organs for surgical procedures, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 9
The rejection of method claim 3 above applies mutatis mutandis to the corresponding limitations of device claims 9 while noting that the rejection above cites to both device and method disclosures.
Claim 4-6 and 10-12 are rejected under 35 U.S.C. 103 as being unpatentable over Rosa and Begg as applied to claim 3 above, and further in view of Kumar (US 2021/0128243 A1).
Claim 4
In regards to claim 4, Rosa is not relied upon to disclose but Begg teaches
a step (140) of determining an organ frame of reference,
Kumar is analogous art from the same field of image-processing enhanced laparoscopic surgery and determining a position of a various items including endoscope and organs and more specifically teaches a step (140) of determining an organ frame of reference, centered on the target organ (52), and a step (150) of determining, for each image, the position of the instrument (54) with respect to the organ frame of reference {as to organ frame of reference see [0049] discussing a first (local) reference frame of an organ centered (original point) on the organ. As to determining instrument position see position tracking device 103 that uses the images to track instruments as per Fig. 2, [0042]-[0049], [0051]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Rosa laparoscopic surgical imaging method and device to include determining an organ frame of reference as taught by Begg and such that the method and device also includes that the organ frame of reference is centered on the target organ and a step (150) of determining, for each image, the position of the instrument (54) with respect to the organ frame of reference as taught by Kumar because doing so usefully expands the range of information available to the surgeon including a conveniently centered frame of reference on the target organ and determining instruments positions in this frame because a common frame of reference more accurately represents the surgical site and relative positions of instruments and target organs; because there is a reasonable expectation of success; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 5
In regards to claim 5, Rosa discloses a sub-step (160e) of defining said position of the trocar (58)
Begg establishes an organ frame of reference and locates/defines the position of the trocar in the organ frame of reference. See [0042]-[0046].
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Rosa laparoscopic surgical imaging method and device which already determines the location of the trocar to also include determining an organ frame of reference as taught by Begg and such that the method and device also includes defining said position of the trocar (58) in the organ frame of reference as also taught by Begg because doing so usefully expands the range of information available to the surgeon including a conveniently centered frame of reference on the target organ and determining trocar and instrument positions in this frame because a common frame of reference more accurately represents the surgical site and relative positions of instruments, trocar and target organs; because there is a reasonable expectation of success; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 6
In regards to claim 6, Rosa is not relied upon to disclose but Begg teaches a step (170) of defining an operating zone (64) of the instrument on the target organ (52) depending on the position of the trocar (58) {see [0002] discussing a common problem with laparoscopic surgery which is that the position and placement of the access port (e.g. trocar) is important because it defines the surgical field and that incorrect trocar placement/position results in a limited surgical field in which the surgeon may not be able to access the target organ efficiently or even at all thus requiring the surgeon to reposition the trocar and instruments passing therethrough. See also [0021]-[0022] overlaying the projected path of each trocar and endoscope onto the surgical field thus further defining the operating zone of Begg’s real-time imaging system. See also Figs. 4-5, [0027]- [0036]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Rosa laparoscopic surgical imaging method and device which already determines the location of the trocar to also include defining an operating zone (64) of the instrument on the target organ (52) depending on the position of the trocar (58) as taught by Begg because the surgical field is geometrically constrained by the relative positions of the trocar(s), endoscope, and instruments such that determining the position of the trocar necessarily also determines/defines the operation zone; because there is a reasonable expectation of success; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 10-12
The rejection of method claims 4-6 above applies mutatis mutandis to the corresponding limitations of device claims 10-12 while noting that the rejection above cites to both device and method disclosures.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Doignon (Doignon, Christophe, Pierre Graebling, and Michel De Mathelin. "Real-time segmentation of surgical instruments inside the abdominal cavity using a joint hue saturation color feature." Real-Time Imaging 11.5-6 (2005): 429-442}. Doignon, section 4.3 stating that in robotic laparoscopy it is of prime importance to predict the trocar position because it is a crucial parameter in the planning of instrument displacements so as to, for example, constrain a needle to follow an acceptable path during stitching. In other words, the center point of the trocar’s location at the surgical entrance portal ideally serves as the fulcrum point for instruments passing through the trocar so as to prevent injury to the patient while also providing an important robotic control parameter guiding said instruments during a surgical procedure.
Wilson J, Tsao TC, Hubschman J, Schwartz S (2010) Evaluating remote centers of motion for minimally invasive surgical robots by computer vision. In: Advanced Intelligent Mechatronics (AIM), 2010 IEEE/ASME International Conference on, pp 1413–1418 doi:10.1109/AIM.2010.5695924 discloses an RCM (trocar) position estimation using image processing and centerline triangulation of the main axes of the instruments passing through the trocar. See section III. THREE DIMENSIONAL CENTERLINE RECONSTRUCTION FROM TWO IMAGES.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/Primary Examiner, Art Unit 2667