DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Claim Interpretation
The interpretation of the term “determinable” is withdrawn considering the term has been deleted.
Applicant argues “processor” is sufficient structure because it is definite structure to a person of ordinary skill in the art. In response, there is no evidence before the Examiner that supports Applicant’s contention. Nothing is found in the specification that the term is limited to structure. The broadest reasonable interpretation is “one that processes” (https://www.merriam-webster.com/dictionary/processor) which is not definite structure. See WSOU Investments LLC v. Google LLC, Nos. 2022-1063, 2022-1065 (Fed. Cir. 2023)) which did not find that the term “processor” is definite structure or sufficient structure.
As to the “surgical imaging device,” Applicant argues the specification provides examples of imaging devices and thus easily identifiable as a surgical imaging device. Applicant’s argument is not clear to the Examiner how disclosure of corresponding structure avoids interpretation under 35 U.S.C. § 112(f).
35 U.S.C. § 101
Applicant argues claim 1 has been amended to recite “output a control signal configured to…” cannot be done in the human mind.” In response, this may not be done in the human mind, but it may be done by human activity, e.g., voice command.
35 U.S.C. § 102/103
Applicant’s arguments with respect to claim(s) 1-18 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Double Patenting
Considering the amendment to the claims, the rejection under double patenting has been withdrawn.
Claim Interpretation
Claim interpretation - 35 U.S.C. 112(f)
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
"processor…" in claims 1-9. See WSOU Investments LLC v. Google LLC, Nos. 2022-1063, 2022-1065 (Fed. Cir. 2023)), where "processor" was found to invoke 35 U.S.C. § 112(f) and it was held "As such, there is no categorical rule regarding whether the term 'processor' connotes sufficient structure to avoid interpretation in means-plus-function format" (page 8).
"first surgical imaging device" in claims 10-18.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, abstract idea, without significantly more.
Step 1: Claim 1 is directed to product and claim 10 is directed to a process and are thus one of the statutory categories of invention.
Step 2A:
A claim is eligible at revised Step 2A unless it recites a judicial exception and the exception is not integrated into a practical application of the application.
Prong 1: Prong One of Step 2A evaluates whether the claim recites a judicial exception.
Examiner notes that independent claims 1 and 10 recite the step of –identify, based on the registration information, a first location, within the first imaging data stream, associated with the visualized surgical structure all fall under the grouping of
The determining of the first location is done by mathematics. See paragraphs [0150]-[0158] of the specification as published. Alternatively, the location can be performed in the human mind by looking at two images of different perspectives of the same scene, compare the common landmarks, shapes, and their orientation and determine the location. The determination of the location does not require calculating numerical coordinates in space, but only requires a generic location.
Hence under Prong One of Step 2A, claim 1 recites a judicial exception.
Prong 2: Prong Two of Step 2A evaluates whether the claim recites additional elements that integrate the judicial exception into a practical application of the exception.
The only additional elements recited by the claims, beyond the judicial exception, are recitations of a processor in claim 1. The focus of the claim is not on improvement in processors, but on using a processor as a tool.
As to the limitation “output a control signal configured to…” it may be done by human activity, e.g., voice command. Furthermore, steps that do nothing more than spell out what is means to “apply it on a computer” cannot confer patent eligibility. Thus, the claimed limitations are not indicative of integration into a practical application.
Hence, under Prong Two of PEG 2019, the independent claims do not integrate the abstract idea into a practical application.
For the above reasons, claims are ineligible under Step 2A.
Step 2B:
In Step 2B, the evaluation consists of whether the claim recites additional elements that amount to an inventive concept (aka “significantly more”) than the recited judicial exception.
Claim 1 recites the steps of "capture a first imaging data stream…" and "communicate registration information…." These steps are directed to gathering information in order to perform the abstract idea of identifying the location. These do not amount to significantly more as data gathering is insignificant extra-solution activity.
Hence, the claim is ineligible under Step 2B.
Therefore, the claim(s) are rejected under 35 U.S.C. 101 as being directed to a judicial exception without significantly more.
Claims 2-9 are not found to add additional elements/significantly more.
Dependent claims 2 and 7 are directed to the mathematical calculation of the abstract idea.
Dependent claims 3-8 recite the imaging devices are intra-operative, endoscopes, laparoscope, etc. which do not add significantly more because these elements are not elements of the claimed device but rather the claims identify the source of the insignificant data gathering. In addition, these instruments are well known/conventional for providing image data.
Dependent claims 9 and 18 recite that the data is gathered at the same time. This does not add significantly more because the recitation is directed to how the processor intended to be used for claim 9, and do not add to the claimed processor/algorithm. Furthermore, the intended manner of use is also directed to the insignificant data gathering and capturing images at the same time is well known and conventional for the insignificant gathering of data.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-18 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 has been amended to recite that the processor is configured to output a control signal to align the first and second local reference geometries. Support has been found for the alignment of the first and second local reference geometries and an analysis engine that outputs location information that has already been adjusted; however, nothing was found for outputting a control signal to perform the alignment, nor that the processor outputs such a control signal.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 5 and 9 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends.
Claims 5 depends on claim 1 which is drawn to the structure of an imaging device that comprises a processor and algorithm for performing the claimed functions. No other structure is recited to be an element of the device save the processor and algorithm. The recited data stream is an object intended to be worked upon by the device that comprises the processor/algorithm. Dependent claim 5 describes the object that is imaged and what the reference landmark is. These are in digital form within the recited data stream and therefore are also not an element of the claimed device. As such, the object and the landmark are not found to further limit the structure of the processor or algorithm, because no additional structure is recited as being an element of the device, nor narrow the scope of the processor and algorithm. Rather, claim 5 limits the data stream. See MPEP 2115. Because the structure covered by claim 5 is the same as the structure covered by claim 1, claim 5 does not further limit claim 1 and thus, is rejected under 35 U.S.C. § 112(d).
Claim 9 depends on claim 1 which is drawn to the structure of an imaging device that comprises a processor and algorithm for performing the claimed functions. No other structure is recited to be an element of the device save the processor and algorithm. Dependent claim 9 is not found to further limit the structure of the processor or algorithm, because no additional structure is recited as being an element of the device, nor narrow the scope of the processor and algorithm. Rather, claim 5 recites how the imaging device is intended to be used., i.e., at the same time as the second imaging device. Because the structure covered by claim 9 is the same as the structure covered by claim 1, claim 9 does not further limit claim 1 and thus is rejected under 35 U.S.C. § 112(d).
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hiroi et al. (US 2020/0267365) in view of Shelton et al. (US 2023/0116781).
Regarding claims 1 and 10 (the method flowing from the function of the claimed device), Hiroi shows:
A first
a processor (processor 172) configured to:
capture a first imaging data stream of a visualized (Para. [0018]: "each camera 111 repeatedly performs imaging at each predetermined timing and sequentially outputs image data"; See also paras [0047]-[0051);
communicate registration information with a second (Para. [0018]: "each camera 111 repeatedly performs imaging at each predetermined timing and sequentially outputs image data"); and
identify, based on the registration information, a first location, within the first imaging data stream, associated with the visualized (Para. [0019]: "each of the cameras 111a, 111b…output distance information in respective ξηζ orthogonal coordinate systems (hereinafter referred to as local coordinate systems of the cameras 111)…"), wherein the first location comprises the processor being configured to determine the first location based on a first local reference geometry associated with a field of view of the first surgical imaging device (Para. [0019]: “the cameras 111a, 111b, . . . output distance information in respective ξηζ orthogonal coordinate systems (hereinafter referred to as local coordinate systems of the cameras 111) where lateral directions in the fields of view are…”), a second local reference geometry associated with a field of view of the second surgical imaging device (Para. [0019]), a transformation between the first local reference geometry and the second local reference geometry, and a reference landmark common to the first imaging data stream and the second imaging data stream (Para. [0049]: “Next, the relative position calculating section 34 rotationally transforms the local coordinate system of the second camera 111b such that information regarding the direction of the second camera 111b”); and
output a control signal configured to align the first and second local reference geometries based on the first location (Paras. [0049]-[0051]: “Next, the relative position calculating section 34 rotationally transforms..such that information regarding the direction of the second camera 111b matches the information regarding the direction of the first camera 111a.”).
Hiroi does not show that the control system is used in a surgical application such that the camera is a surgical imaging device and the visualized object is a surgical structure.
Shelton shows the desire to perform multi-source imaging during surgery. Before the effective filing date of the claimed invention, it would have been obvious modify and apply the multi-source imaging of Hiroi in a surgical environment thereby expanding the usability of Hiroi’s imaging system.
Alternatively, claim(s) 1-18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shelton et al. in view Hiroi.
Sheldon shows:
1. A first surgical imaging device (control system 133) comprising:
a processor (processor 172) configured to:
capture a first imaging data stream of a visualized surgical structure (Para. [0124]: "The first and second video output signals 640, 642 include data representative of the position of the critical structure on a three-dimensional surface model, which is provided to an integration module 643.");
communicate registration information with a second surgical imaging device capturing a second imaging data stream of the visualized surgical structure (Para. [0124]); and
identify, based on the registration information, a first location, within the first imaging data stream, associated with the visualized surgical structure (Para. [0168]: "In particular, characterizing, identifying, and/or visualizing surgical instruments (including their positions, orientations, and actions), tissues, structures, users, and/or other things located within the surgical field "; [Para. 0198]: "In general, identifying, determining a location of, and determining an orientation of a first and second imaging systems relative to one another can allow for controlling cooperative surgical imaging interactions.")
wherein processor being configured to identify the first location comprises the processor being configured to determine the first location
Shelton identifies a first location as cited above, but does not show the determination of the first location is based on the first and second local reference geometries and the transformation between the first and second local reference geometries, and the reference landmark common to the two imaging data streams.
Hiroi shows the determination of each of the locations based on the first and second local reference geometries and the transformation between the first and second local reference geometries, and the reference landmark common to the two imaging data streams as discussed above.
Before the effective filing date of the claimed invention, it would have been obvious to modify Shelton with the transformation taught by Hiroi in order to predictably identify the first location.
2. The first surgical imaging device of claim 1, wherein the processor is further configured to calculate the transformation between the first reference geometry and the second reference geometry based on a first physical orientation of the first surgical imaging device, a second physical orientation of the second surgical imaging device, and the reference landmark (See citation of the transformation described by Hiroi. Also, Shelton shows at para. [0198]: "determining a location of, and determining an orientation of a first and second imaging systems relative to one another" The location and orientation being equivalent to the claimed “local reference geometry.”).
3. The first surgical imaging device of claim 1, wherein the first surgical imaging device is an intraoperative imaging device (Para. [0086]: "the surgical visualization system 100 can be used intraoperatively") and the second surgical imaging device is a pre-operative imaging device (this is not an element of the claimed imaging device, but rather an object intended to be used in conjunction with the claimed imaging device and thus it does not impart distinguishing structure from Shelton) and wherein the processor is further configured to calculate the transformation between the first reference geometry and the second reference geometry based on a difference between a first patient position during capture of the first imaging data stream and a second patient position during capture of the second imaging data stream (see citation given for claim 2).
4. The first surgical imaging device of claim 1, wherein the first surgical imaging device is an intraoperative imaging device (Para. [0086]: "the surgical visualization system 100 can be used intraoperatively") and the second surgical imaging device is a pre-operative imaging device (this is not an element of the claimed imaging device, but rather an objected intended to be used in conjunction with the claimed imaging device and thus it does not impart distinguishing structure from Shelton. In addition see para. [0081]:"A surgical visualization system can allow for intraoperative identification"), wherein the first imaging data stream is captured with a patient in a first patient position and the second imaging data stream is captured with the patient in a second patient position, and wherein the reference landmark comprises an anatomical structure that has relatively little movement between capture of the first and second imaging data streams (see citation given for claim 2).
5. The first surgical imaging device of claim 1, wherein the visualized surgical structure comprises an object to be removed from a patient (this claim does not serve to distinguish for reasons discussed above in the rejection of claim 5 under 35 U.S.C. § 112(d). In addition, see para. [0130]: "the tumor 324 being visualized on a display screen associated with or coupled to the imaging system,…to ensure complete removal of the tumor 324"), and wherein the reference landmark comprises an anatomical structure of the patient (This limitation is drawn to the intended operation of the device and thus does not serve to structurally distinguish from Shelton. See also para. [0200]:"a first imaging system located on a first side of a tissue wall can be configured to use common anatomic landmarks").
6. The first surgical imaging device of claim 1, wherein the first surgical imaging device is an endoscopic device, the second surgical imaging device is a laparoscopic device (this is not an element of the claimed imaging device, but rather an objected intended to be used in conjunction with the claimed imaging device and thus it does not impart distinguishing structure from Shelton), the first surgical imaging device and the second surgical imaging device are separated by a tissue barrier (This limitation is drawn to the intended operation of the device and thus does not serve to structurally distinguish from Shelton. Also, See Paras. [0007]-[0008]).
7. The first surgical imaging device of claim 6, wherein the processor is further configured to calculate the transformation between the first reference geometry and the second reference geometry based on one or more lights projected, by the laparoscopic device, through the tissue barrier, and detected by the endoscopic device (The recitation to the light projected, the laparoscopic device, tissue barrier, and endoscopic device are not elements of the claimed device nor found to impart any particular logic to the algorithm within the processor. As to the transformation between the reference geometries, see the discussion for claim 1. See also para. [0016] regarding the tissue barrier: "the second imaging device can have a point of view from a first side of a tissue wall, the first imaging device can have a point of view from a second side of the tissue wall that is opposite to the first side of the tissue wall").
8. The first surgical imaging device of claim 1, wherein the processor is further configured to:
generate an augmented reality (AR) overlay of the first imaging data stream, the second imaging data stream, based on the transformation between the first reference geometry and the second reference geometry, wherein the AR overlay depicts the first imaging data stream and the second imaging data stream overlapped onto one another based on the transformation(Para. [0116]: "For example, the control system 600 can be employed to create of three-dimensional data set for surgical use in a system with augmentation image overlays."); and
output the AR overlay to a display device for displaying (Para. [0126]: "The video monitors 652 are configured to output the integrated/augmented views from the image overlay controller 610").
9. The first surgical imaging device of claim 1, wherein the first surgical imaging device and the second surgical imaging device are capturing video data at the same time (implicit; Para. [0019]: "a surgical method includes gathering, with a first imaging device, first images of a surgical site during performance of a surgical procedure, gathering, with a second imaging device, second images of the surgical site during the performance of the surgical procedure").
10. A method performed by a first surgical imaging device, the method comprising (See citations given for claim 1 above):
capturing a first imaging data stream of a visualized surgical structure;
communicating registration information with a second surgical imaging device capturing a second imaging data stream of the visualized surgical structure; and
identifying, based on the registration information, a first location, within the first imaging data stream, associated with the visualized surgical structure, wherein the first location is determinable based on: a first reference geometry associated with a field of view of the first surgical imaging device, a second reference geometry associated with a field of view of the second surgical imaging device, a transformation between the first reference geometry and the second reference geometry, and a reference landmark common to the first imaging data stream and the second imaging data stream.
11. The method of claim 10, wherein the method further comprises calculating the transformation between the first reference geometry and the second reference geometry based on a first physical orientation of the first surgical imaging device, a second physical orientation of the second surgical imaging device, and the reference landmark (See citations given for claim 2 above).
12. The method of claim 10, wherein the first surgical imaging device is an intraoperative imaging device and the second surgical imaging device is a pre-operative imaging device, and wherein the method further comprises calculating the transformation between the first reference geometry and the second reference geometry based on a difference between a first patient position during capture of the first imaging data stream and a second patient position during capture of the second imaging data stream (See citations given for claim 3 above).
13. The method of claim 10, wherein the first surgical imaging device is an intraoperative imaging device and the second surgical imaging device is a pre-operative imaging device (Para. [0116]: "techniques can be employed both intraoperatively and preoperatively using additional visual information"), wherein the first imaging data stream is captured with a patient in a first patient position and the second imaging data stream is captured with the patient in a second patient position (implicit; furthermore the first and second patient position covers both the same position and different position), and wherein the reference landmark comprises an anatomical structure that has relatively little movement between capture of the first and second imaging data streams (Para. [0200]: "For another example, a first imaging system located on a first side of a tissue wall can be configured to use common anatomic landmarks to detect a location and orientation of a second imaging system located on a second, opposite side of the tissue wall.").
14. The method of claim 10, wherein the visualized surgical structure comprises an object to be removed from a patient (Para. [0130]:"The margin 330 can indicate the area or amount of tissue that should be excised to ensure complete removal of the tumor 324."), and wherein the reference landmark comprises an anatomical structure of the patient (Para. [0200]:"For another example, a first imaging system located on a first side of a tissue wall can be configured to use common anatomic landmarks to detect a location and orientation of a second imaging system located on a second, opposite side of the tissue wall.").
15. The method of claim 10, wherein the first surgical imaging device is an endoscopic device, the second surgical imaging device is a laparoscopic device, the first surgical imaging device and the second surgical imaging device are separated by a tissue barrier (Para. [0007]: "The first imaging system is…a laparoscopic imaging system. The method also includes visualizing, with an endoscopic imaging system, a second, opposite side of the tissue wall during the performance of the surgical procedure.").
16. The method of claim 15, wherein the method further comprises calculating the transformation between the first reference geometry and the second reference geometry based on one or more lights projected, by the laparoscopic device, through the tissue barrier, and detected by the endoscopic device (This limitation is not required to be performed because it is directed to the "determinable" clause of claim 10 as discussed above in the Claim Interpretation section above).
17. The method of claim 10, wherein the method further comprises: generating an augmented reality (AR) overlay of the first imaging data stream, the second imaging data stream, based on the transformation between the first reference geometry and the second reference geometry, wherein the AR overlay depicts the first imaging data stream and the second imaging data stream overlapped onto one another based on the transformation; and outputting the AR overlay to a display device for displaying (see citations for claim 8).
18. The method of claim 10, wherein the first surgical imaging device and the second surgical imaging device are capturing video data at the same time (see citations for claim 9).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hwa Andrew S Lee whose telephone number is (571)272-2419. The examiner can normally be reached Mon-Fri 9am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Iacoletti can be reached at (571) 270-5789. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Hwa Andrew Lee/Primary Examiner, Art Unit 2877
Prosecution Insights