Application Status
This non-final action is in response to continuation reissue application, filed Feb. 26, 2024, that is a reissue of U.S. Patent 10499996 (hereafter “the ’996 Patent”). A preliminary amendment, filed Aug. 7, 2024, amends the specification for priority claim benefit and to provide notice of related reissue application to same patent, cancels original claims 1-33 and adds new claims 34-41; while, preliminary amendment, filed Dec. 17, 2024, further amends new claim 34. Claims 34-41 are pending.
Notice of Pre-AIA or AIA Status
The present application is being examined under the AIA first to file provisions.
Prior or Concurrent Proceedings
Applicant is reminded of the continuing obligation under 37 CFR 1.178(b), to timely apprise the Office of any prior or concurrent proceed-ing in which the ’996 patent is or was involved. These proceedings would include interferences, reissues, reexaminations, and litigation.
Information Material to Patentability
Applicant is further reminded of the continuing obligation under 37 CFR 1.56, to timely apprise the Office of any information which is mate-rial to patentability of the claims under consideration in this reissue appli-cation.
These obligations rest with each individual associated with the filing and prosecution of this application for reissue. See also MPEP §§ 1404, 1442.01 and 1442.04.
Amendment
Applicant is notified that any subsequent amendment to the specification and/or claims must comply with 37 CFR 1.173(b)-(g).
(b) Making amendments in a reissue application. An amendment in a reissue application is made either by physically incorporating the changes into the specification when the application is filed, or by a separate amendment paper. If amendment is made by incorporation, markings pursuant to paragraph (d) of this section must be used. If amendment is made by an amendment paper, the paper must direct that specified changes be made, as follows:
(1) Specification other than the claims. Changes to the specification, other than to the claims, must be made by submission of the entire text of an added or rewritten paragraph, including markings pursuant to paragraph (d) of this section, except that an entire paragraph may be deleted by a statement deleting the paragraph without presentation of the text of the paragraph. The precise point in the specification must be identified where any added or rewritten paragraph is located. This paragraph applies whether the amendment is submitted on paper or compact disc (see §§ 1.52(e)(1) and 1.821(c), but not for discs submitted under § 1.821(e)).
(2) Claims. An amendment paper must include the entire text of each claim being changed by such amendment paper and of each claim being added by such amendment paper. For any claim changed by the amendment paper, a parenthetical expression “amended,” “twice amended,” etc., should follow the claim number. Each changed patent claim and each added claim must include markings pursuant to paragraph (d) of this section, except that a patent claim or added claim should be canceled by a statement canceling the claim without presentation of the text of the claim….
(c) Status of claims and support for claim changes. Whenever there is an amendment to the claims pursuant to paragraph (b) of this section, there must also be supplied, on pages separate from the pages containing the changes, the status (i.e., pending or canceled), as of the date of the amendment, of all patent claims and of all added claims, and an explanation of the support in the disclosure of the patent for the changes made to the claims.
(d) Changes shown by markings. Any changes relative to the patent being reissued which are made to the specification, including the claims, upon filing, or by an amendment paper in the reissue application, must include the following markings:
(1) The matter to be omitted by reissue must be enclosed in brackets; and
(2) The matter to be added by reissue must be underlined, except for amendments submitted on compact discs (§§ 1.96 and 1.821(c)). Matter added by reissue on compact discs must be preceded with “<U>” and end with “</U>” to properly identify the material being added…
(g) Amendments made relative to the patent. All amendments must be made relative to the patent specification, including the claims, and drawings, which are in effect as of the date of filing of the reissue application.
The Amendment filed Dec. 17, 2024 proposes amendments to claims that do not comply with 37 CFR 1.173 (b)-(g), which sets forth the manner of making amendments in reissue applications. The cited amendment fails to use proper claim status of “New, Amended” for claim 34 and to use “New” for new claims 35-41 since “Currently Amended” and “Previously Presented” are not a recognized status of a claim in a reissue application, as required by 37 CFR 1.17 (b)(2). Also, the Dec. 2024 amendment fails to use proper claim marking to underline added limitations as recited in new claims 34-41, per 37 CFR 1.173(b)(2) and (d)(2).
Consent of Assignee
This application is objected to under 37 CFR 1.172(a) as lacking the written consent of all assignees owning an undivided interest in the patent. The consent of the assignee must be in compliance with 37 CFR 1.172. See MPEP § 1410.01. In this case, although there is a Consent of Assignee co-filed with this application, on Feb. 26, 2024, for the assignment co-filed with the application; however, there is no Consent of Assignee signed by an individual/person who possesses apparent authority at least since no Consent of Assignee has been filed as associated with the assignment filed Sept. 18, 2024 or, at least there is no evidence that the individual who provided consent as co-filed with this application has authority to provide consent for current assignment. For this reason, the consent is defective.
Generally, we (the Office) accept anyone of the following as having apparent authority to sign on behalf of the organization/assignee:
(i) Anyone who signs the unmodified standard form for POA (Form AIA /826) below the statement “The undersigned (whose title is supplied below) is authorized to act on behalf of the applicant”;
(ii) Anyone who signs the unmodified standard 3.73c statement (Form AIA /96) below the statement “The undersigned (whose title is supplied below) is authorized to act on behalf of the assignee”;
(iii) Anyone having POA in the current application; and
(iv) Anyone having one of the titles listed in MPEP 325(V).
Since none of these apply in this case, the consent is defective. A proper consent of the assignee in compliance with 37 CFR 1.172 and 3.73 is required in response to this Office action.
Reissue Application Declaration
The following is a quotation of the appropriate paragraphs of pre-AIA 35 U.S.C. §251 that form the basis for the rejections under this section made in this Office action:
(a) IN GENERAL.—Whenever any patent is, through error, deemed wholly or partly inoperative or invalid, by reason of a defective specification or drawing, or by reason of the patentee claiming more or less than he had a right to claim in the patent, the Director shall, on the surrender of such patent and the payment of the fee required by law, reissue the patent for the invention disclosed in the original patent, and in accordance with a new and amended application, for the unexpired part of the term of the original patent. No new matter shall be introduced into the application for reissue.
Claims 34-41 are rejected as being based upon a defective reissue application declaration by the inventor under 35 U.S.C. 251 as set forth above. See 37 CFR 1.175. The nature of the defect(s) in the declaration is set forth in the discussion above in this Office action. The co-filed reissue declaration is defective because it fails to specifically identify/declare at least one error for at least one pending claim which is relied upon to support the reissue application. See 37 CFR 1.175 and MPEP § 1414. MPEP 1414 II.(B) states in part: “In identifying the error, it is sufficient that the reissue oath/declaration identify a single word, phrase, or expression in the specification or in an original patent claim, and how it renders the original patent wholly or partly inoperative or invalid.” The statement in the co-filed declaration fails to specifically identify an error. "Any error in the claims must be identified by reference to the specific claim(s) and the specific claim language wherein lies the error." See MPEP 1414 II. (C). Essentially, the declaration states “An error being corrected by the present reissue application is that the patentee claimed less than they had a right to claim in US. Pat. No. 10,488,996, The new independent claims are broader than the original Issued independent claims. This is a broadening reissue.” This statement merely indicates this application broadens 10499996 but it fails to identify a specific claim and how an identified limitation in a specified pending claim renders the patent partly inoperative or invalid as an error that is relied as basis for this reissue application since it fails to specify a how a particular limitation of a claim renders the patent inoperative or invalid. See 37 CFR 1.175 and MPEP § 1414. See In re Tanaka, 640 F.3d 1246, 1251, 98 USPQ2d 1331, 1334 (Fed. Cir. 2011).
Claims 34-39 are rejected under 35 U.S.C. 251 as being an improper recapture of broadened claimed subject matter surrendered in the application for the patent upon which the present reissue is based. See Greenliant Systems, Inc. et al v. Xicor LLC, 692 F.3d 1261, 103 USPQ2d 1951 (Fed. Cir. 2012); In re Shahram Mostafazadeh and Joseph O. Smith, 643 F.3d 1353, 98 USPQ2d 1639 (Fed. Cir. 2011); North American Container, Inc. v. Plastipak Packaging, Inc., 415 F.3d 1335, 75 USPQ2d 1545 (Fed. Cir. 2005); Pannu v. Storz Instruments Inc., 258 F.3d 1366, 59 USPQ2d 1597 (Fed. Cir. 2001); Hester Industries, Inc. v. Stein, Inc., 142 F.3d 1472, 46 USPQ2d 1641 (Fed. Cir. 1998); In re Clement, 131 F.3d 1464, 45 USPQ2d 1161 (Fed. Cir. 1997); Ball Corp. v. United States, 729 F.2d 1429, 1436, 221 USPQ 289, 295 (Fed. Cir. 1984). A broadening aspect is present in the reissue which was not present in the application for patent. The record of the application for the patent shows that the broadening aspect (in the reissue) relates to claimed subject matter that applicant previously surrendered during the prosecution of the application (e.g. surrender-generating limitation). Accordingly, the narrow scope of the claims in the patent was not an error within the meaning of 35 U.S.C. 251, and the broader scope of claim subject matter surrendered in the application for the patent cannot be recaptured by the filing of the present reissue application.
Below are the pertinent findings of fact relevant to this rejection:
09/26/2017
The 15/561666 application (hereafter the ‘666 application) was filed with 32 claims.
03/27/2019
A first official action on the merits was mailed rejecting the claims 1, 6, 15-20, 24-26 and 32 as indefinite under 112(2nd para), rejecting claims 1, 15, 20, 27-29 and 32 over Sarvestani (PG Pub. 2010/0039506) in view of Moreau-Gaudry et al. (PG Pub. 2010/0256504), rejecting claims 5, 6 and 30 as unpatentable over Sarvestani in view of Moreau-Gaudry as applied to claim 1 and further in view of Revie et al. (PG Pub. 2008/0269596), rejecting claims 7 and 22 over Sarvestani in view of Moreau-Gaudry as applied to claim 1 and further in view of Liu (PG Pub 2013/0281821) and rejecting claims 31 over Sarvestani in view of Moreau-Gaudry as applied to claim 1 and further in view of Tardieu (PG Pub 2010/0009314). Claims 2-4, 8-14, 16-19, 21, and 23-26 were indicated as containing allowable subject matter but were objected to for being based on a rejected base claim.
06/27/2019
Applicant filed an amendment stating claims 1, 3-7, 9-16, 20, 22-25, and 32 are currently amended. Claim 2 is cancelled. On pages 18-19 under the heading “Amendment to the Claims”, applicant states “Specifically, independent claims 1 and 32 have been amended to include the allowable subject matter of former dependent claim 2, new independent claim 33 includes the subject matter of original claim 1 and the allowable subject matter of dependent claim 11 (both as amended for clarity), and new independent claim 34 includes the subject matter of original claim 1 and the allowable subject matter of dependent claim 16 (again, both as amended for clarity).” Thus, claim 2 was cancelled, while new claims 33 and 34 were added.
07/25/2019
Examiner issued a Notice of Allowance with claims 1, 3-34 (renumbered as 1-32) allowed citing Applicant’s reply filed June 27, 2019.
A reissue will not be granted to "recapture" claimed subject matter which was surrendered in an application to obtain the original patent. See MPEP §1412.20. A three step process is used to apply the recapture rule:
(1) first, we determine whether, and in what respect, the reissue claims are broader in scope than the original patent claims;
(2) next, we determine whether the broader aspects of the reissue claims relate to subject matter surrendered in the original prosecution; and
(3) finally, we determine whether the reissue claims were materially narrowed in other respects, so that the claims may not have been enlarged, and hence avoid the recapture rule.” See MPEP §1412.02(I).
The first step of the three-step process we determine whether, and in what respect, the reissue claims are broader in scope than the original patent claims. Upon review of the new independent claims 34 of the present reissue application in comparison to claims 1-33 of ‘996 Patent, the Examiner finds that the Applicant through the Dec 17, 2024 Amendment herein has broadened the claims by omitting features as shown in table.
Claim 34 herein
Claim 1 of ‘996
A system for video based computer-aided surgery and diagnosis in anatomical regions of a patient comprising rigid, non-deformable anatomical parts or tissues comprising:
An image processing system comprising: one or more storage devices storing processor-executable instructions, and one or more processors configured to execute the instructions to cause the image processing system to perform operations including:
(i). a free-moving camera that is meant to acquire intra-operative images and video of an operating field, articular joint or anatomical cavity;
(ii). a visual marker, referred to as a World Marker or WM, which is an object that can be rigidly attached to a surface, and that comprises at least one planar facet with a known pattern having a system of coordinates that defines a global or world system of coordinates;
(iii). a touch-probe that is a tool or instrument comprising a handgrip and a tip, and that has at least one visual marker, referred to as a Tool Marker or TM, comprising at least one planar facet with a known pattern that defines a local system of coordinates where a position of the tip PTT is known;
(iv). one or more surgical instruments for surgical execution, where each surgical instrument has at least one visual marker, also referred to as a Tool Marker or TM, comprising at least one planar facet with a known pattern that defines a local system of coordinates in which a position of relevant points and parts in the instrument are known; and
(v). an image processing system, that may correspond to or may be part of a computer and/or any other computing device, that receives as input images and video acquired by the camera (i), as well as commands that may be entered by a button panel, keyboard, camera buttons, foot switch and/or any other input interface, and that outputs processing results and guidance information to a display;
wherein the system is configured to execute steps comprising:
(vi). attachment of the WM of (ii) in an arbitrary location in a surface of a rigid anatomical part of interest, such that the WM of (ii) and the rigid anatomical part of interest do not move one with respect to the other;
(vii). 3D reconstruction of points in the surface of the rigid anatomical part of interest by using the touch-probe of (iii) to pin-point the points in the surface while keeping both the WM and the TM of the probe in a Field-of-View or FOV of the camera of (i) for determining the location of the touch-probe in the world system of coordinates to create 3D reconstruction results;
(viii). use the 3D reconstruction results to perform measurements, make inferences, or overlay a surgical plan with an anatomical region of the patient, in which case a 3D registration method may be used for determining a transformation that maps local coordinates in a surgical plan into world coordinates in the WM attached to the anatomical part of interest; and
(ix). assisted execution of a clinical procedure by using the surgical instruments of (iv) while keeping both WM and TMs of the surgical instruments in the FOV of the camera of (i) such that the surgical instruments can be located in world coordinates and real-time guidance can be provided by relating surgical instrument locations in world coordinates with the 3D reconstruction results and/or overlaid surgical plan;
receiving one or more images and/or frames of video, wherein the one or more images and/or frames of video depict (i) a first marker that does not move with respect to an anatomical part of a patient, and (ii) a second marker coupled to a contactless probe;
receiving a 3D reconstruction of a plurality of points of the anatomical part of the patient in a first coordinate system defined using the first marker, the plurality of points having been illuminated by light emitted by the contactless probe; and outputting information derived from the 3D reconstruction for display.
For instance, the claims in the amendment omits the limitation of “where the camera in (i) is calibrated such that image points u expressed in pixel coordinates are mapped into image points x expressed in metric units according to
PNG
media_image1.png
151
732
media_image1.png
Greyscale
” as recited in issued claims 1 and 31; omitting the limitation of “wherein an arbitrary point P is reconstructed in global or world coordinates by acquiring an image as described in (vii) that undergoes the following processing steps: detect, identify, and estimate a 3D pose C of the WM in camera coordinates; detect, identify, and estimate a 3D pose ¥ of the TM in camera coordinates; reconstruct a point P that is in contact with the tip of the touch-probe by making
PNG
media_image2.png
80
219
media_image2.png
Greyscale
and store 3D coordinates of point P in a memory” as recited in issued claim 32; and omitting the limitation of “wherein the assisted execution of the clinical procedure is accomplished by acquiring continuous video according to (ix), the video comprising a plurality of images, with each consecutive image undergoing the following processing steps in real-time: a. detect, identify, and estimate a 3D pose C of the WM in camera coordinates; b. detect, identify, and estimate a 3D pose ¥ of the TM of each of the one or more surgical instruments in camera coordinates; c. compute the 3D pose T of TM in WM coordinates using the following equation:
PNG
media_image3.png
42
134
media_image3.png
Greyscale
d. map the information of surgical instrument calibration, that can be points, axes, or CAD models, into world coordinates using the rigid transformation T; e. relate the surgical instrument calibration information with 3D data stored in memory to make measurements and inferences for the purpose of real-time guidance; and f. display aiding features either by using Augmented Reality (AR), in which case guidance information is overlaid in image or by animating a Virtual Reality (VR) 3D model” as recited in issued claim 33.
Regarding step 2, Examiner finds that some of the broadening aspects relate to subject matter surrendered during prosecution of the 15/561666 Application leading to the original claims of ‘996 Patent. The Examiner finds that the Applicant through their Dec. 17, 2024 Amendment in this reissue has broadened the claims by omitting the features relating to the above identified limitation as originally filed claim 2 of ‘666 application as now recited in issued claims 1 and 31, omitting the above identified limitation as originally filed claim 11 of ‘666 as now recited in issued claim 32 and above identified limitation as originally filed claim 16 of ‘666 application as now recited in issued claim 33 that regards language added and concurrently argued by Applicant to original filed claims in the June 27, 2019 amendment in 15/561666. As noted above in the findings of fact, Examiner allowed certain claims because the prior art of record do not disclose the limitations of originally filed claims 2, 11 and 16 of ‘666 application as present in issued claims 1, 31, 32 and 33 in ‘996 patent as argued by Applicant in their June 27, 2019 amendment. It is emphasized that Applicant added these limitations and concurrently argued in their response that these added limitations relating to the now omitted feature (See original claims 1-32 of 15/561666; See Jun 27, 2019 Amendment pp 2, 3 and 14-19 therein) with consideration of the Comparison of claims 34, 54 and 73 herein as compared to issued claims 1, 31, 32 and 33 of ‘996 Patent) were a distinguishing limitation over the applied art (e.g., a surrender-generating limitation). As noted above, Applicant argued these features in their response/amendment during examination. Thus, the noted limitations in issued claims 1, 31, 32 and 33 that contained limitations from original claims 2, 11 and 16 are subject matter surrendered during prosecution of the '666 Application leading to the original ‘996 Patent. From prosecution of ‘666 Application, at least one of the noted limitations in originally filed claims 2, 11 and 16 must be recited in present claims to avoid improper recapture. The other features omitted, as shown in table above, do not regard a surrender-generating limitation as each of those features were shown by the applied references.
Regarding step 3, Examiner further determines that to the extent there is/are narrowing limitation(s) herein as compared to ‘996 claims, the added limitation(s) herein not present in ‘996 claims such as limitations in at least claims 40, 43, 45-53, 55-62, 64 and 70, do not materially narrow the surrendered subject matter such that they are not directed to the surrendered subject matter. Examiner specifically finds that the claims 34, 54 and 73 (and their associated dependent claims) have eliminated features/limitations through amendments in this application for which the claims of the original Patent were allowed. The Dec 2021 amendment herein attempts to improperly recapture surrendered subject matter explicitly surrendered during prosecution of Application leading to the ‘996 Patent since limitations of claims 2, 11 and 16 (as originally filed in ‘666 application) is eliminated in this reissue application that coincidently regards subject matter relied on during prosecution of parent application to obtain the original patent. Claim scope that was canceled or amended is deemed surrendered and therefore barred from reissue. Clement, 131 F.3d at 1470, 45 USPQ2d at 1165. In re Mostafazadeh, 98 USPQ2d 1639 (Fed Cir 2011), In re Youman, 102 USPQ2d 1862 (Fed Cir 2012). Next, the features added by the preliminary amendment herein are not materially narrowed relative to the surrender-generating limitations. Specifically, independent claims 34, 54 and 73 (and their associated dependent claims) herein have omitted the limitation “where the camera in (i) is calibrated such that image points u expressed in pixel coordinates are mapped into image points x expressed in metric units according to
PNG
media_image1.png
151
732
media_image1.png
Greyscale
” as recited in issued claims 1 and 31; omitted the limitation “wherein an arbitrary point P is reconstructed in global or world coordinates by acquiring an image as described in (vii) that undergoes the following processing steps: detect, identify, and estimate a 3D pose C of the WM in camera coordinates; detect, identify, and estimate a 3D pose ¥ of the TM in camera coordinates; reconstruct a point P that is in contact with the tip of the touch-probe by making
PNG
media_image2.png
80
219
media_image2.png
Greyscale
and store 3D coordinates of point P in a memory” as recited in issued claim 32; and omitted the limitation “wherein the assisted execution of the clinical procedure is accomplished by acquiring continuous video according to (ix), the video comprising a plurality of images, with each consecutive image undergoing the following processing steps in real-time: a. detect, identify, and estimate a 3D pose C of the WM in camera coordinates; b. detect, identify, and estimate a 3D pose ¥ of the TM of each of the one or more surgical instruments in camera coordinates; c. compute the 3D pose T of TM in WM coordinates using the following equation:
PNG
media_image3.png
42
134
media_image3.png
Greyscale
d. map the information of surgical instrument calibration, that can be points, axes, or CAD models, into world coordinates using the rigid transformation T; e. relate the surgical instrument calibration information with 3D data stored in memory to make measurements and inferences for the purpose of real-time guidance; and f. display aiding features either by using Augmented Reality (AR), in which case guidance information is overlaid in image or by animating a Virtual Reality (VR) 3D model” as recited in issued claim 33 (see December 10, 2021 Amendment herein). While to the extent there is/are added element(s) herein that are not present in ‘996 claims, those element(s) is/are not directed to the noted surrendered subject matter. If surrendered subject matter has been entirely eliminated from a claim present in the reissue application, then a recapture rejection under 35 U.S.C. 251 is proper and must be made for that claim. See MPEP 1412.02 (I)(C). Stated another way, if a claim limitation present in the original patent that was added to overcome a rejection or that was argued by applicant to distinguish over the prior art is entirely eliminated from a claim in the reissue application, then a recapture rejection under 35 U.S.C. 251 is proper and must be made for that claim. See Id.
In view of the forgoing, the Applicant has attempted in the December 2024 Amendment herein to improperly recapture subject matter explicitly surrendered during prosecution of the ‘666 Application leading to the ‘996 Patent for reasons stated above.
Nonstatutory Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 34-41 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-102 of U.S. Patent No. RE49930 in view of US Patent Application Publication No. 2010/0039506 A1, hereafter Sarvestani or US Patent Application Publication No. 2010/0256504 A1, hereafter Moreau-Gaudry. The table below provides the limitations of claim 1 of RE49930 as compared to limitations of new claim 34 herein.
Claim 34 herein
Claim 1 of RE49930
An image processing system comprising:
one or more storage devices storing processor-executable instructions, and
one or more processors configured to execute the instructions to cause the image processing system to perform operations including:
A system for video based computer-aided surgery and diagnosis in anatomical regions of a patient comprising rigid, non-deformable anatomical parts or tissues comprising:
receiving one or more images and/or frames of video, wherein the one or more images and/or frames of video depict (i) a first marker that does not move with respect to an anatomical part of a patient, and (ii) a second marker coupled to a contactless probe;
receiving a 3D reconstruction of a plurality of points of the anatomical part of the patient in a first coordinate system defined using the first marker, the plurality of points having been illuminated by light emitted by the contactless probe; and
outputting information derived from the 3D reconstruction for display.
(i). a free-moving camera that is meant to acquire intra-operative images and video of an operating field, articular joint or anatomical cavity;
(ii). a visual marker, referred to as a World Marker or WM, which is an object that can be rigidly attached to a surface, and that comprises at least one planar facet with a known pattern having a system of coordinates that defines a global or world system of coordinates;
(iii). a touch-probe that is a tool or instrument comprising a handgrip and a tip, and that has at least one visual marker, referred to as a Tool Marker or TM, comprising at least one planar facet with a known pattern that defines a local system of coordinates where a position of the tip PTT is known;
(iv). one or more surgical instruments for surgical execution, where each surgical instrument has at least one visual marker, also referred to as a Tool Marker or TM, comprising at least one planar facet with a known pattern that defines a local system of coordinates in which a position of relevant points and parts in the instrument are known; and
(v). an image processing system, that may correspond to or may be part of a computer and/or any other computing device, that receives as input images and video acquired by the camera (i), as well as commands that may be entered by a button panel, keyboard, camera buttons, foot switch and/or any other input interface, and that outputs processing results and guidance information to a display;
wherein the system is configured to execute steps comprising:
(vi). attachment of the WM of (ii) in an arbitrary location in a surface of a rigid anatomical part of interest, such that the WM of (ii) and the rigid anatomical part of interest do not move one with respect to the other;
(vii). 3D reconstruction of points in the surface of the rigid anatomical part of interest by using the touch-probe of (iii) to pin-point the points in the surface while keeping both the WM and the TM of the probe in a Field-of-View or FOV of the camera of (i) for determining the location of the touch-probe in the world system of coordinates to create 3D reconstruction results;
(viii). use the 3D reconstruction results to perform measurements, make inferences, or overlay a surgical plan with an anatomical region of the patient, in which case a 3D registration method may be used for determining a transformation that maps local coordinates in a surgical plan into world coordinates in the WM attached to the anatomical part of interest; and
(ix). assisted execution of a clinical procedure by using the surgical instruments of (iv) while keeping both WM and TMs of the surgical instruments in the FOV of the camera of (i) such that the surgical instruments can be located in world coordinates and real-time guidance can be provided by relating surgical instrument locations in world coordinates with the 3D reconstruction results and/or overlaid surgical plan;
Lacking evidence to the contrary, although the claims at issue are not identical, they are not patentably distinct from each other because the limitations of claim 1 of RE49930 involve/include the process recited by new claim 34 herein where the “system for computer-aided system and diagnosis” of claims of RE49920 would have been understood by a POSITA to involve “one or more storage devices storing processor-executable instructions, and one or more processors configured to execute the instructions to cause the image processing system to perform operations” as similarly shown by Sarvestani (Abstract, paragraphs (para.) [0007]-[0014] and [0033]-[0038]), or Moreau-Gaudry ([0016]-[0017], [0031]-[0035]) and where claims of RE49930 further recites the limitation of “where the camera in (i) is calibrated such that image points u expressed in pixel coordinates are mapped into image points x expressed in metric units according to
PNG
media_image1.png
151
732
media_image1.png
Greyscale
” as recited in issued claims 1. Essentially, the system for computer aided surgery and diagnosis in claims of RE49930 would have been understood to include the storage devices storing processor-executable instructions, and processors configured to execute the instructions as within knowledge of a POSITA or it would have been obvious in view of teachings of Sarvestani or Moreau-Gaudry as within skill of a POSITA as shown by cumulative art in record where the addition of the storage devices storing processor-executable instructions, and processors configured to execute the instructions as taught by Sarvestani or Moreau-Gaudry to claims of RE49930 is simply applying using known structures performing known processes to perform the process of claims of RE49930. Sarvestani and Moreau-Gaudry regard art known to applicant as both were applied during examination of ‘996.
Regarding claim 35-37 herein, claim 24 of RE49930 further recite “a laser pointer” that involves/overlaps “a light source configured to emit a beam of light” and “wherein the beam of light is collimated.”
Regarding claim 38-39, claim 1 of RE49930 further recites “that receives as input images and video acquired by the camera” and “a free-moving camera.”
Regarding claims 40-41, claim 1 of RE49930 recites of “where the camera in (i) is calibrated such that image points u expressed in pixel coordinates are mapped into image points x expressed in metric units according to
PNG
media_image1.png
151
732
media_image1.png
Greyscale
” that overlaps limitations recited in claims 40 and 41 herein.
Claims 2-102 of RE49920 recite further limitations overlapping limitations of new claims 34-41 herein.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 40 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The undefined variables “u” and “x” in the limitation “wherein the camera is calibrated such that one or more image points u expressed in pixel coordinates are mapped into one or more image points x expressed in units of distance according to an inverse projection function, wherein the inverse projection function depends on one or more camera intrinsic and/or distortion parameters” are indefinite in so far as it is unclear whether the variables represent whole numbers, integers or real numbers.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 34-39 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application Publication No. 2010/0039506 A1, hereafter Sarvestani, in view of US Patent Application Publication No. 2010/0256504 A1, hereafter Moreau-Gaudry. Regarding claim 34, lacking evidence to the contrary, although Sarvestani uses different words/phrases, a POSITA or an artisan (i.e., person of ordinary skill in the art) would have understood Sarvestani substantially describes an image processing system comprising one or more storage devices storing processor-executable instructions, and one or more processors configured to execute the instructions to cause the image processing system to perform operations (Sarvestani: Abstract, [0007]-[0014] and [0033]-[0038]),
including:
receiving one or more images and/or frames of video, wherein the one or more images and/or frames of video depict (i) a first marker that does not move with respect to an anatomical part of a patient (Id., Figs. 1-5, and para. [0007]-[0014], [0033]-[0040], [0050]-[0052], reference 40; markers on patient), and (ii) a second marker coupled to a contactless probe (Fig. 5, [0041]-[0054]; Fig. 11, reference 40, 60, 64, 66, 100 and 102; marker on probe; Figures 5 and 11 shows a surgical probe/instrument with one or more reference units as markers, in which the known geometry of the surgical instrument in combination with the known position of the surgical instrument by means of the reference unit essentially means that the position of the tip of said instrument is also known; Sarvestani also describes using a pointing device reference 40, 100);
receiving a 3D reconstruction of a plurality of points of the anatomical part of the patient in a first coordinate system defined using the first marker (Figs. 1-5, 8, 10, 11 and 15, elements 40, 60, 64, 66, 94, 100, and 102 and [0007]-[0014], [0018], [0033]-[0056]; Sarvestani describes establishing a first system of coordinates of a coordinate system using markers 40, 60, 64, 66 where the marker may be any shape; Sarvestani also discloses a coordinate system of the pattern into points in the coordinate system of the camera [0054]; Sarvestani describes defining information with respect to the image data, such as a planned trajectory for a surgical instrument(s), an operation plan, and/or other similar virtual surgical plan information (representations of which can be superimposed on the image data set); additionally, as the position of the tip of the surgical instrument is known as discussed in Sarvestani, the points of tip touches during use are also known and registered; Sarvestani would have been understood to involve 3D reconstruction of a plurality of points of the patient in coordinate system defined using marker),
the plurality of points having been illuminated by light emitted by the contactless probe (Id.); and
outputting information derived from the 3D reconstruction for display (Id., Sarvestani describes defining information with respect to the image data, such as a planned trajectory for a surgical instrument(s), an operation plan, and/or other similar virtual surgical plan information (representations of which can be superimposed on the image data set), and visually disclosing the reference unit of the surgical instrument and the reference units attached to the anatomical part of interest are both in the field of view of the camera; additionally, as the position of the tip of the surgical instrument is known, the points of said tip touches during use are also known and registered; Sarvestani would have been understood to involve 3D reconstruction of a plurality of points of the patient in coordinate system defined using marker).
In the alternative, to the extent, a POSITA or artisan would have interpreted that although Sarvestani describes a contactless probe 100 with marker 40 and the plurality of points having been illuminated by light where Sarvestani does not discuss the light source coupled/attached to probe, Sarvestani lacks emitted by the contactless probe. However, a light source attached/coupled to probe so light is emitted by the contactless probe is well known as described by related reference Moreau-Gaudry (Moreau-Gaudry: [0025], [0032], [0040]-[0043], [0047], [0063]-[0082], [0101], [0108], Figs. 4, 5 and 7).
The examiner deems Moreau-Gaudry is relevant prior art due to either being in the same field of endeavor or being reasonably pertinent to the particular problem with which the Applicant was faced. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992). The level of ordinary skill in the art is shown by the applied art herein. Since Sarvestani and Moreau-Gaudry each regard method and system for assisting in therapeutic/surgical procedures and diagnosis, in consideration consistent with US Supreme Court decision in KSR that 'known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art', in this case, it would have been obvious to an artisan at a time prior to the invention to add emitted by the contactless probe as described by Moreau-Gaudry to improve the image processing system of Sarvestani to produce the predictable result of enabling the acquisition, interpretation, and utilization of 3D multimodality data on articular cartilage in an integrated, flexible, and updatable manner (see Moreau-Gaudry, Fig. 1, and para. [0015]-[0016]). In the alternative, in consideration consistent with US Supreme Court decision in KSR that 'known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art', in this case, it would have been obvious to an artisan at a time prior to the invention to substitute one known probe to receive points of the anatomical part for the other to yield the expected result of enabling the acquisition, interpretation, and utilization of 3D multimodality data on articular cartilage in an integrated, flexible, and updatable manner (see Moreau-Gaudry, Fig. 1, and para. [0015]-[0016]).
Regarding claim 35-37 herein, facts noted above show Moreau-Gaudry describes “a laser pointer” that involves/overlaps “a light source configured to emit a beam of light” and “wherein the beam of light is collimated.” Id. Thus, for same reasons as stated above relied on herein, Sarvestani combined with Moreau-Gaudry renders these limitations obvious.
Regarding claim 38-39, Sarvestani (Figs. 1-2, reference 24, and [0039] and [0048]) also describes “that receives as input images and video acquired by the camera” and “a free-moving camera.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Mark Sager whose telephone number is (571) 272-4454. The examiner can normally be reached M-Th, 6:30 AM - 3 PM, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexander Kosowski can be reached at (571) 272-3744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA)
or 571-272-1000.
/MARK SAGER/Primary Examiner, Art Unit 3992
Conferees:
/JEFFREY D CARLSON/Primary Examiner, Art Unit 3992
/ALEXANDER J KOSOWSKI/Supervisory Patent Examiner, Art Unit 3992