DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 , 4, 5, 12, 14, 16-19, and 21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Philipp et al., (WO2022161930A1) (however for examination purposes, the US application “US20240407861A1” shall be used for citation of Philipp) .
Regarding claim 1, Philipp teaches a computer implemented method for controlling a surgical microscopy system having a stand and a microscope carried by the stand (fig. 1 the microscope 801 is shown to be on a stand that is controlled by the operating device 808 that is connected to the eye piece that has cross beam 850 [0104]),
wherein the method comprises:
driving a camera of the surgical microscopy system so as to obtain a sequence of images ([0014] at least two microscopy images are taken),
determining movement information for each of two or more objects depicted in the sequence of images, based on the sequence of images ([0100] from at least two microscopy images, the movement of instruments between the images are compared; [0108] the instruments 51 and 52 are arranged in the intervention area 50, and [0113] the two microscopy images are of the intervention area 50),
determining prioritization information for the two or more objects based on the movement information ([0042] instruments which have high dynamic characteristic may be comparatively ), and
based on the prioritization information, carrying out an assistance functionality in connection with the two or more objects ([0042]-[0044] the assistance functionality can be performed based on the order of the areas taking account of the dynamic characteristic of the surgical instruments).
Regarding claim 4, Philipp teaches the method of claim 1, wherein Philipp further teaches wherein the prioritization information is obtained without any classification ([0013] the map is tool-type agnostic, so it doesn’t differentiate between he types of surgical instruments) and, in particular, without any instance segmentation of the objects ([0013] instance segmentation is not used).
Regarding claim 5, Philipp teaches the method of claim 1, wherein Philipp further teaches wherein the prioritization information is determined based on the movement information using at least one relative criterion that is determined based on the movement information ([0040]-[0041] the areas of the objects are compared based on the activities of the instruments, such as the movement).
Regarding claim 12, Philipp teaches the method of claim 1, wherein Philipp further teaches wherein the movement information comprises an optical flow between successive images of the sequence of images ([0041] the dynamic characteristic can be quantified y the intensity of the optical flow between the images; [0075] the system captures at least two images, the two images would be successive images).
Regarding claim 14, Philipp teaches the method of claim 1, wherein Philipp further teaches wherein the prioritization information comprises a segmentation map with a prioritization label (fig. 6 the mask 151 effects binary segmentation into regions within and outside the activity area, and the system has probability values, and uses them to define the Prescence of the instrument [0129]-[0130]).
Regarding claim 16, Philipp teaches the method of claim 1, wherein Philipp further teaches carrying out the assistance functionality comprises:
determining a target configuration of the surgical microscopy system based on the prioritization information and a positioning of the two or more objects ([0073]-[0074] the distance between the activity area and the center field of view of the microscope, and a recognized activity area can be arranged in the center of the field of view of the microscope), and
driving at least one component of the surgical microscopy based on the target configuration ([0073] auto centering is performed).
Regarding claim 17, Philipp teaches the method of claim 16, wherein Philipp further teaches wherein the target configuration of the surgical microscopy system comprises an alignment of a field of view of the microscope with respect to at least one of the two or more objects that is selected on the basis of the prioritization information ([0073] and [0150] the assistance functionality centers the activity area in the center of the field of view of the microscope, and would be done on the surgical instruments in the activity areas).
Regarding claim 18, Philipp teaches the method of claim 16, wherein Philipp further teaches wherein the target configuration of the surgical microscopy system comprises auto-focusing of the microscope with respect to one of the two or more objects that is selected on the basis of the prioritization information ([0072] the assistance functionality can be selected to adapt a focus of the surgical microscope, and this would be on the surgical instruments in the activity areas.).
Regarding claim 19, Philipp teaches a data processing device or controlling a surgical microscopy system, wherein the data processing device comprises a processor that is configured to be able to load program code from a memory and to execute it, wherein the execution of the program code causes the processor to carry out the following steps: (fig. 3 the microscopy system ([0011]) comprises a PC with a process unit and a memory that can execute code [0110]),
wherein the method comprises:
driving a camera of the surgical microscopy system so as to obtain a sequence of images ([0014] at least two microscopy images are taken),
determining movement information for each of two or more objects depicted in the sequence of images, based on the sequence of images ([0100] from at least two microscopy images, the movement of instruments between the images are compared; [0108] the instruments 51 and 52 are arranged in the intervention area 50, and [0113] the two microscopy images are of the intervention area 50),
determining prioritization information for the two or more objects based on the movement information ([0042] instruments which have high dynamic characteristic may be comparatively ), and
based on the prioritization information, carrying out an assistance functionality in connection with the two or more objects ([0042]-[0044] the assistance functionality can be performed based on the order of the areas taking account of the dynamic characteristic of the surgical instruments).
Regarding claim 21, Philip teaches a surgical microscopy system comprising the data processing device according claim 19 (fig. 1 is of a surgical microscope, and is used by the processor device of fig. 3 to evaluate microscopy images [0081]-[0083]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 3, 6, 8-11, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Philipp as applied to claim 1 above, and further in view of Basafa et al., (US20250114157A1).
Regarding claim 2, Philipp teaches the method of claim 1, wherein Philipp fails to explicitly teach wherein the prioritization information comprises a class assignment of the two or more objects at least into a first class and a second class, wherein the assistance functionality is carried out in connection with at least one object assigned to the first class, wherein the assistance functionality is not carried out in connection with at least one further object assigned to the second class.
However in the same surgery field of endeavor, Basafa teaches wherein the prioritization information comprises a class assignment of the two or more objects at least into a first class and a second class ([0319] there are instruments that go through “minor motion”, such those that exceeds a speed or acceleration threshold and/or if the detected motion trajectory of the tracked instrument matches a predefined motion trajectory that is known to not be followed, and an instrument that goes through “major motion”, such as if the detected speed or acceleration of the tracked instrument as it leaves the predefined boundary region is below a predetermined speed or acceleration threshold, and/or if the detected motion trajectory of the tracked instrument matches a predefined motion trajectory that is known to be followed), wherein the assistance functionality is carried out in connection with at least one object assigned to the first class, wherein the assistance functionality is not carried out in connection with at least one further object assigned to the second class ([0319] the system follows an instrument going through “major motion” but not one’s going through “minor motion”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the classifications of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 3, Philipp teaches the method of claim 1, wherein Philipp further teaches wherein the prioritization information is determined based on the movement information using at least one predefined criterion ([0065] the control algorithm could take into account a predefined set of rules), but is silent regarding wherein the at least one predefined criterion comprises a fixed movement threshold value.
In the same surgical field of endeavor, Basafa teaches wherein the at least one predefined criterion comprises a fixed movement threshold value ([0319] the threshold for determining major and minor motion is a predetermined speed or acceleration value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the classifications of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 6, Philipp teaches the method of claim 5, wherein but is silent regarding wherein the at least one or more relative criterion comprises a ranking of the movement information in relation to the two or more objects.
However in the same surgical field of endeavor, Basafa teaches wherein the at least one or more relative criterion comprises a ranking of the movement information in relation to the two or more objects ([0319] there are instruments that go through “minor motion”, such those that exceeds a speed or acceleration threshold and/or if the detected motion trajectory of the tracked instrument matches a predefined motion trajectory that is known to not be followed, and an instrument that goes through “major motion”, such as if the detected speed or acceleration of the tracked instrument as it leaves the predefined boundary region is below a predetermined speed or acceleration threshold, and/or if the detected motion trajectory of the tracked instrument matches a predefined motion trajectory that is known to be followed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the classifications of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 8, Philipp teaches the method of claim 1, but fails to explicitly disclose wherein the prioritization information is determined by way of a machine-learned model that receives the movement information as input.
However, in the same surgery field of endeavor Basafa further teaches wherein the prioritization information is determined by way of a machine-learned model that receives the movement information as input ([0285] the system uses machine learning algorithms on the image data to identify the instruments; [0311] the neural network is trained using a data set of motion trajectories to identify the instrument).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the machine learning of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 9, Philipp teaches the method of claim 1, but is silent regarding wherein the prioritization information is also determined based on a positioning of the two or more objects.
In the same surgery field of endeavor, Basafa teaches wherein the prioritization information is also determined based on a positioning of the two or more objects ([0321] the instrument to be followed can be based on the direction of the instrument)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the prioritization information of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 10, Philipp teaches the method of claim 1, but is silent regarding wherein the prioritization information is also determined based on semantic context information for a scene associated with the two or more objects.
In the same surgery field of endeavor, Basafa teaches wherein the prioritization information is also determined based on semantic context information for a scene associated with the two or more objects ([0311] and [0319]the instrument should be followed not only from movement information but also from semantic context information including the identified type of surgical instrument and the surgical task being performed, which are used to determine if the instrument should be followed.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the prioritization information of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 11, Philipp teaches the method of claim 1, but is silent regarding wherein the method furthermore comprises: - receiving a user command requesting the assistance functionality in relation to a non-specific one of the two or more objects.
In the same ultrasound field of endeavor, Basafa teaches wherein the method furthermore comprises: receiving a user command requesting the assistance functionality in relation to a non-specific one of the two or more objects ([0310]-[0311] the operator inputs to reposition the laparoscope, and the system then decides which object is followed/tracked and is automatically repositioned).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the prioritization information of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 13, Philipp teaches the method of claim 1, but is silent regarding wherein the prioritization information is linked to a positioning of the two or more objects.
In the same surgery field of endeavor, Basafa teaches wherein the prioritization information is linked to a positioning of the two or more objects ([0321] the instrument to be followed can be based on the direction of the instrument)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the prioritization information of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Regarding claim 15, Philipp teaches the method of claim 1, but is silent regarding wherein the prioritization information comprises a point localization with a prioritization label.
In the same surgery field of endeavor, Basafa teaches wherein the prioritization information comprises a point localization ([0311] the system determines the 2D trajectory/location of the surgical instrument in image space, which would be a point) with a prioritization label ([0319] the instrument is labeled as major or minor motion).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philip with the prioritization information of Basafa, as this would help ensure that the tip of the surgical instrument remains within a predefined optimum position (see Basafa [0311]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Philipp as applied to claim 5 above, and further in view of Roessler (US20170333137A1).
Regarding claim 7, Philipp teaches the method of claim 5, but is silent regarding wherein the at least one relative criterion comprises a relative movement of the two or more objects in relation to one another.
In the same surgical field of endeavor, Roessler teaches wherein the at least one relative criterion comprises a relative movement of the two or more objects in relation to one another ([0079] the common coordinate system can be used to track relative moment of the surgical instrument, target site, and physical objects).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to modify the method of Philipp with the teaching of using relative motion between objects to provide information for distinguishing objects, as this would improve tracking results (see Roessler [0009]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL Y FANG whose telephone number is (571)272-0952. The examiner can normally be reached Mon - Friday 9:30 am - 6:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at 5712722714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL YIMING FANG/Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights