Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
Receipt is acknowledged of a request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e) and a submission, filed on 10/28/2025.
Response to Amendment
Examiner acknowledges the amendments made to the claims 1, 10, 11 and 20, with claims 2-3 and 7-9 canceled in the present application. Claims 1, 4-6, 8-20 are currently pending in the present applicant. The amendments made to the claim 1 and 20 overcome the prior 112 rejection set forth in the previous office action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 10-20 is/are rejected under 35 U.S.C. 103 as being anticipated by Sato (US Patent No 20140371527) in view of Venugopal (US Patent No 20200245967).
Regarding claim 1, Sato teaches a method for controlling a surgical high frequency (HF) generator during a HF surgical procedure performed with a handheld HF surgical instrument supplied with HF energy by the HF generator (treatment tool 210 which is a handheld treatment tool that is connected by a generator to generate high frequency energy output for treatment, [0049]), the method comprising: evaluating a succession of images of an operating area that are captured in an image sequence during the HF surgical procedure, including subjecting the images to automatic real-time image recognition to detect a predetermined structure and/or a predetermined operating situation (the system contains imaging section 200 which captures images of the target object and outputs the captured images to the processor section 300 for indicating distance and displacement information in regards to a blood vessel, [0041]-[0042], which also equates to detecting a predetermined structure), in response to detection of the predetermined structure and/or the predetermined operating situation, suggesting or performing a change of at least one of an operating parameter and an operating mode of the HF generator (see [0051] in which the user is notified of the state of the HF treatment tool 210 via the control section 160 and the processor 300 wherein the treatment tool 210 may be controlled to change the mode setting in response to the image processing, or detection) detecting a bleeding as the predetermined structure and detecting at least one of a size and blood volume of the bleeding (see [0079], in which Sato describes how the treatment tool 210 is able to determine the extent of the bleeding based on the size of the blood vessel and then incision or coagulation mode is determined) with the image recognition (treatment tool control 160 is able to detect and determine if bleeding has occurred, [0076], which comes as a result of the image acquisition 110, see also fig 2), and selecting an operating mode of the HF generator suitable for coagulation of the bleeding based on at least one of the detected size and blood volume of the bleeding (wherein the treatment tool 210 has several modes, one being for incision, but one also including a bleeding arrest mode which coagulates the tissue, [0049]), wherein the suggesting or performing a change includes suggesting or applying the selected operating mode of the HF generator (see [0076] wherein the treatment tool controls the setting and the electrical output of the treatment based on the diameter and size of the blood vessel and the closeness value which changes the generator output applied).
Sato does not teach wherein the bleeding is detected by means of an algorithm based on machine learning, wherein the machine learning is a neural network or a support vector machine, which has been trained with images or videos of organic structures with bleedings, and the algorithm is further trained with current captured images based on feedback from the operator whether the detected bleeding is a bleeding or not.
However, the analogous bleed detection system which is disclosed by Venugopal does teach wherein the bleeding is detected by means of an algorithm based on machine learning, wherein the machine learning is a neural network or a support vector machine (see from Venugopal, [0045], in which it uses a machine learning neural network algorithm 150 to compile a library of bleed waveforms 126 which can represent bleed sizes in the body in order to detect or estimate the size of the bleed event), which has been trained with images or videos of organic structures with bleedings, and the algorithm is further trained with current captured images based on feedback from the operator whether the detected bleeding is a bleeding or not (see form Venugopal, [0045] – [0046] in which the algorithm 150 is trained and learning using the forward and backwards components as well as raw imagery added to the algorithm by the operator in real time).
Therefore, it would have been obvious for one skilled in the art prior to the effective filing date to combine the high frequency imaging and control capabilities of Sato, with that of the neural network bleed size detection and estimation system which is taught by Venugopal, as it is a known method in the art to incorporate artificial intelligence for bleed detections as to create a quicker and more automated detection system, as taught by Venugopal, [0045].
Furthermore, Venugopal does not explicitly teach wherein a coagulation mode is suggested to an operator based on the detected bleeding.
However, as the prior art of Sato does teach a treatment tool 210 which has several modes, one being for incision, but one also including a bleeding arrest mode which coagulates the tissue, [0049], and wherein the treatment tool controls the setting and the electrical output of the treatment to achieve coagulation based on the diameter and size of the blood vessel and the closeness value which changes the generator output applied, see [0076]. It would have been obvious for one skilled in the art to take treatment tool control and coagulation modes based on the bleed size which are disclosed by Sato, and apply the artificial intelligence bleed size detection system of Venugopal, in order to achieve a control system which suggests a coagulation treatment based on the detected size of the bleeding received from an AI network.
Regarding claim 10, Sato teaches the method according to claim 1, wherein the feedback from the operator whether the detected bleeding is a bleeding or not is determined based on whether a site of the detected bleeding is treated by the suggested coagulation mode or whether a different operating mode or operating parameter that is different from the suggested coagulation mode is used (see specifically [0076] as well as fig 6, which describes how the specific treatment mode is chosen, including feedback sensing capabilities which switch between the incision and coagulating mode based on if there is blood present or not within the treatment site).
Regarding claim 11, the combination teaches the method according to claim 1, wherein the further training is carried out individually for various operators (see from Venugopal, [0029] in which the imagery bleed size training is carried out with different operators and a processing component 44 and memory 46 may contain that operator).
Regarding claim 12, Sato teaches the method according to Claim 1, wherein the captured images are analyzed for an operating situation in which the handheld HF surgical instrument is visible in a captured image (wherein the treatment tool is inserted into the end of the imaging section, [0037]), and in which the handheld HF surgical instrument is approaching a blood vessel that can be sealed by the handheld HF surgical instrument, and the handheld HF surgical instrument is identified from external data sources or from an image analysis designed for this purpose (the treatment tool is seen a distance to the reference point, which is the blood vessel, and the shape and size of the tool is known and determined in advance prior to insertion so that correct image analysis may be performed, [0037]) and a probability is calculated that the blood vessel is to be sealed and an acoustic warning signal indicating an incomplete seal is suppressed if the probability lies below a predetermined or predeterminable threshold (see [0048] in which it explains how the bleeding arrest capability is predicted using a degree of closeness value, in which the blood vessel is more likely to be arrest or sealed as the degree of closeness increases which in turn increases the incision capabilities and decreases the coagulation mode. Therefore, suppressing the incomplete seal as the degree of closeness becomes smaller than a specific threshold).
Regarding claim 13, Sato teaches the method according to Claim 12, wherein the probability of whether the blood vessel is to be sealed is determined by taking account of a progress of the approach, and/or taking account of the conditions of the blood vessel (see [0048] in which it explains how the bleeding arrest capability is predicted using a degree of closeness value or progress of approach, in which the blood vessel is more likely to be arrest or sealed as the degree of closeness increases).
Regarding claim 14, Sato teaches the method according to Claim 13, wherein taking account of the progress of the approach includes taking account of a decreasing approach speed or a pause at the blood vessel (distance acquisition system 120 measures the position of the tool from the blood vessel in order to calculate information such as the speed or position from the blood vessel, [0046]), and taking account of the conditions of the blood vessel includes taking account of skeletization of the blood vessel, if applicable (treatment tool 210 also takes into account the tissue denaturation from high frequency output, [0049]).
Regarding claim 15, the combination teaches the method according to Claim 12, wherein the captured images are analyzed for an operating situation on the basis of a machine learning algorithm including a trained neural network (see Venugopal, [0045], in which the detection and size of the bleed event waveforms and imagery are trained in the system through a neural network).
Regarding claim 16, Sato teaches the method according to Claim 15, wherein the algorithm is further trained with current captured images based on a decision by an operator whether or not to seal the blood vessel (see specifically [0076] as well as fig 6, which describes how the specific treatment mode is chosen, including feedback sensing capabilities which switch between the incision and coagulating mode based on if there is blood present or not within the treatment site).
Regarding claim 17, the combination teaches the method according to Claim 16, wherein the further training is carried out individually for various operators (see from Venugopal, [0029] in which the imagery bleed size training is carried out with different operators and a processing component 44 and memory 46 may contain that operator).
Regarding claim 18, Sato teaches the method according to Claim 1, wherein the images of the operating area are captured by a video endoscope monitoring the operating area (endoscope apparatus monitoring the blood vessel in the treatment site, [0029]).
Regarding claim 19, Sato teaches a system for controlling a surgical HF generator during a HF surgical procedure, the system comprising: the HF generator, at least one handheld HF surgical instrument that is configured to be supplied with HF energy by the HF generator (treatment tool 210 which is a handheld treatment tool that is connected by a generator to generate high frequency energy output for treatment, [0049]), a display device (a display section 400, [0040]), a video endoscope (endoscope apparatus monitoring the blood vessel in the treatment site, [0029]), and a processor that is signal-connected to the video endoscope, and is configured to: receive and to evaluate image signals from the video endoscope, and perform the method according to Claim 1 (the imaging section 200 found within the endoscope captures and outputs the image date to the processor section 300 for analysis, [0042]).
Regarding claim 20, Sato teaches a non-transitory computer readable storage medium having stored therein a program to be executable by a processor (the processor may have a central CPU and GPU for processing the imaging data and also can have a memory unit with programs to help operate the processor 300, [0116]), the program causing the processor to execute: evaluating a succession of images of an operating area that are captured in an image sequence during a HF surgical procedure, including subjecting the images to automatic real-time image recognition to detect a predetermined structure and/or a predetermined operating situation (the system contains imaging section 200 which captures images of the target object and outputs the captured images to the processor section 300 for indicating distance and displacement information in regards to a blood vessel, [0041]-[0042], which also equates to detecting a predetermined structure), and in response to detection of the predetermined structure and/or the predetermined operating situation, suggesting or performing a change of at least one of an operating parameter and an operating s mode of a HF generator (see [0051] in which the user is notified of the state of the HF treatment tool 210 via the control section 160 and the processor 300 wherein the treatment tool 210 may be controlled to change the mode setting in response to the image processing, or detection) detecting a bleeding as the predetermined structure and detecting at least one of a size and blood volume of the bleeding (see [0079], in which Sato describes how the treatment tool 210 is able to determine the extent of the bleeding based on the size of the blood vessel and then incision or coagulation mode is determined) with the image recognition (treatment tool control 160 is able to detect and determine if bleeding has occurred, [0076], which comes as a result of the image acquisition 110, see also fig 2), and selecting an operating mode of the HF generator suitable for coagulation of the bleeding based on one of the detected size and detected blood volume of the bleeding (wherein the treatment tool 210 has several modes, one being for incision, but one also including a bleeding arrest mode which coagulates the tissue, [0049]), wherein the suggesting or performing a change includes suggesting or applying the selected operating mode of the HF generator (see [0076] wherein the treatment tool controls the setting and the electrical output of the treatment based on the diameter and size of the blood vessel and the closeness value which changes the generator output applied).
Sato does not teach wherein the bleeding is detected by means of an algorithm based on machine learning, wherein the machine learning is a neural network or a support vector machine, which has been trained with images or videos of organic structures with bleedings, and the algorithm is further trained with current captured images based on feedback from the operator whether the detected bleeding is a bleeding or not.
However, the analogous bleed detection system which is disclosed by Venugopal does teach wherein the bleeding is detected by means of an algorithm based on machine learning, wherein the machine learning is a neural network or a support vector machine (see from Venugopal, [0045], in which it uses a machine learning neural network algorithm 150 to compile a library of bleed waveforms 126 which can represent bleed sizes in the body in order to detect or estimate the size of the bleed event), which has been trained with images or videos of organic structures with bleedings, and the algorithm is further trained with current captured images based on feedback from the operator whether the detected bleeding is a bleeding or not (see form Venugopal, [0045] – [0046] in which the algorithm 150 is trained and learning using the forward and backwards components as well as raw imagery added to the algorithm by the operator in real time).
Therefore, it would have been obvious for one skilled in the art prior to the effective filing date to combine the high frequency imaging and control capabilities of Sato, with that of the neural network bleed size detection and estimation system which is taught by Venugopal, as it is a known method in the art to incorporate artificial intelligence for bleed detections as to create a quicker and more automated detection system, as taught by Venugopal, [0045].
Furthermore, Venugopal does not explicitly teach wherein a coagulation mode is suggested to an operator based on the detected bleeding.
However, as the prior art of Sato does teach a treatment tool 210 which has several modes, one being for incision, but one also including a bleeding arrest mode which coagulates the tissue, [0049], and wherein the treatment tool controls the setting and the electrical output of the treatment to achieve coagulation based on the diameter and size of the blood vessel and the closeness value which changes the generator output applied, see [0076]. It would have been obvious for one skilled in the art to take treatment tool control and coagulation modes based on the bleed size which are disclosed by Sato, and apply the artificial intelligence bleed size detection system of Venugopal, in order to achieve a control system which suggests a coagulation treatment based on the detected size of the bleeding received from an AI network.
Claim(s) 4-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sato (US Patent No 20140371527) in view of Venugopal (US Patent No 20200245967) further in view of Buckler (US Patent No 20190159737).
Regarding claim 4, Sato and Venugopal teach the method according to Claim 1.
The prior combination does not teach further comprising effecting a semantic segmentation of the captured images according to anatomical structures.
However, using semantic segmentation is a commonly known way to capture and analyze images known in the art. For example, see the analogous medical imaging device of Buckler which uses semantic segmentation to view and analyze anatomical structures such vein and artery cross sections to find the bleeds as taught in paragraphs [0304]- [0307].
Therefore, it would have been obvious for one skilled in the art prior to the effective filing date to combine the method of controlling and imaging a HF device taught by Sato and Venugopal to include the semantic segmentation taught by Buckler in order to get a more in-depth analysis at the exact location of bleeding in the blood vessel as taught by Buckler, [0304]- [0307].
Regarding claim 5, Sato teaches the method according to Claim 4, wherein the anatomical structures include at least one of tissue types, organs, and blood vessels (treatment tool 210 is able to determine the extent of the bleeding based on the size of the blood vessel after image acquisition 110, see fig 2).
Regarding claim 6, the combination teaches the method according to Claim 1, further comprising effecting a semantic segmentation of the captured images according to anatomical structures, wherein: the detected bleeding is attributed to a prevailing anatomical structure in a segment based on a position of the detected bleeding in the segment of the image (see Buckler, semantic segmentation to view and analyze anatomical structures such vein and artery cross sections to find the bleeds as taught in paragraphs [0304]-[0307]), and the operating mode of the HF generator is selected based on a quality of the anatomical structure in addition to the size and/or blood volume of the bleeding (see Sato [0079], in which describes how the treatment tool 210 is able to determine the extent of the bleeding based on the size of the blood vessel and then incision or coagulation mode is determined).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
In regards to the arguments presented with respect to the independent claims 1 and 20 and how the previous prior art of record of Sato in view of Schroer does not teach the limitations of the claim as amended has been considered but ultimately falls moot. As further search and consideration were applied, necessitated by the amendments, it has been found that the new prior art of record of Venugopal does teach wherein the bleeding is detected by means of an algorithm based on machine learning, wherein the machine learning is a neural network or a support vector machine (see from Venugopal, [0045], in which it uses a machine learning neural network algorithm 150 to compile a library of bleed waveforms 126 which can represent bleed sizes in the body in order to detect or estimate the size of the bleed event), which has been trained with images or videos of organic structures with bleedings, and the algorithm is further trained with current captured images based on feedback from the operator whether the detected bleeding is a bleeding or not (see form Venugopal, [0045] – [0046] in which the algorithm 150 is trained and learning using the forward and backwards components as well as raw imagery added to the algorithm by the operator in real time), as stated in the amended limitations. Therefore, as the new prior art of record of Venugopal teaches the artificial intelligence bleed size detection system as claimed in the independent claims 1 and 20, they remain rejected under the new prior art of record rejection of Sato in view of Venugopal set forth in the present office action.
As no further arguments or remarks have been made regarding any dependent claims, they too remain rejected under the new prior art of record rejection set forth in the present office action.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE M BROWN whose telephone number is (703)756-4534. The examiner can normally be reached 8:00-5:00pm EST, Mon-Fri, alternating Fridays off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Linda Dvorak can be reached on 571-272-4764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LINDA C DVORAK/Primary Examiner, Art Unit 3794
/KYLE M. BROWN/Examiner, Art Unit 3794