DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims: claims 21, 23-24, 26, 28-43. Claims 1-20, 22, 25 and 27 are cancelled.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/16/2026 has been entered.
Response to Arguments
Applicant's arguments filed 9/16/2026 have been fully considered but they are not persuasive.
Applicant remark – (page 10-12) Applicant argued lack of teaching by the prior art regarding “connective tissue”. Please see the claims and Remarks for further detail.
Examiner response – Examiner respectfully disagree. Wolf et al teaches many surgical operations on the anatomy with different surgical tools to cut or bind various part of the body together. Wolf et al teaches operation of connection tissues such as skin graft (a connective tissue in paragraph 0474) and fibrous tissue (another connective tissue in paragraph 0475) that area operated on where such operations excision, altered or amended the tissue. Furthermore all parts of the body are connected thus all are connective tissue. The claim language regarding “connective tissue” is view as too broad and thus any part of the body is view as connective tissue since the body as a whole is connected to teach other to function. Please see Office Action below for further detail.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21, 23-24, 26, 28-43 are rejected under 35 U.S.C. 103 as being unpatentable over Wolf et al (US 2021/0012868) in view of MOTTRAM et al (US 2023/0149101).
Claim 21, similarly claim 29 and 40:
Wolf et al (US 2021/0012868) teach the following subject matter:
An inference device connected between a surgical robot and a console controlling the surgical robot, the inference device comprising:
one or more processors; and a storage storing instructions causing any of the one or more processors to execute processing of (figure 14 and 0290 teaches of structure of processor, storage/database/memory, non-transitory computer readable medium to contain and execute instruction):
acquiring an operative field image shot by an imaging unit of the surgical robot (0285 teaches surgical event with image frames, where 0537,0538 and 0595 detail the use of surgical robot with camera for capturing and transmitting images, video footage over network directly wired and/or wirelessly; 0090 teaches camera for moving to object with field of view and function such as adjust zoom, direction, speed);
performing an inference process on the acquired operative field image (0285 detail inferred base on set of images from surgical area (operative field image)); [[and]]
inferring at least one of a position of an object to be recognized in the operative field image and an event occurring in the operative field image (0080 detail inferred output with classification of item depicted in the image (operative field image) with output including inferred value, size, volume, age, cost, product);
wherein the object to be recognized is connective tissue, where the connective tissue is an anatomical structure that serves as a landmark for a layer that can be excised, and an area of the connective tissue increases or decreases as a result of a surgeon's excision operation (0151 detail operation procedure (surgeon’s excision operation) on a patient body including cutting, abrading, suturing, or other techniques that involve physically changing body tissues and organs with list of organ and tissues can all be operated on and are all connected together; 0474 detail surgical operations such as skin graft is connective tissue; 0475 and 0484 detail specific surgical procedure such as fibrous tissue which is a connective tissue. Where all these connective tissues during surgery can be excision, altered or amended);
computing an area of the recognized object (0080 detail inferred value such as volume (area or 3D object); paragraph 0172 detail calculating characteristic with numerical value: a dimension, a length, an area, a volume);
generating the control information in order to move the imaging unit following a portion in which the computed area increases or decreases (0086 detail control application positioning camera to capture video/image ROI (region of interest) with identify structure, surgical tool, hand of surgeon, motion, particular location, tracking location with camera 115; 0087 teaches camera to image ROI (area) to larger using camera 115 to zoom in or out (increase or decrease)); and
transmitting at least one of the acquired operative field image and information based on an inference result to the console according to transmission settings by the console (0285-0286, specifically 0286 teaches additional surgeon assistance and/or guidance is requested base on set of frames (field image) that are inferred, where figure 14 and 0290 teaches transmission to 1410 (paragraph 0309-0311)); and
transmitting the control information generated by the one or more processors to the console (figure 4 and 0110 detail control through user interface 700 (console) which includes processors; 0086 teaches control of camera as well as infrared laser by operator; 0553 detail provide support for surgical procedures to user with recommendation and control commands to surgical robots).
Wolf et al teaches all the subject matter above but do not teach the following subject matter: generating control information for controlling an operation of the surgical robot according to the inference result.
MOTTRAM et al (US 2023/0149101) teaches the following subject matter: generating control information for controlling an operation of the surgical robot according to the inference result (paragraph 0049 detail control configuration of robot based on inferred means, where 0036 detail such function for surgical robot).
Wolf et al and MOTTRAM et al are both in the field of image analysis, especially generating control information for surgical robot based on ROI imaging such that the combine outcome is predictable.
Therefore it would have been obvious to one having ordinary skill before the effective filing date to modify Wolf et al by MOTTRAM et al such using that information, the control system can model the effect of gravity on the components of the robot arm for the current configuration of the robot arm and estimate a force or torque due to gravity on each joint of the robot arm as disclosed by MOTTRAM et al in 0049.
Regarding claim 39, Wolf et al addresses method (figure 5).
Regarding claim 40, Wolf et al addresses non-transitory computer readable recording medium (figure 14 and 0290 teaches of structure of processor, storage/database/memory, non-transitory computer readable medium)
Claim 23:
Wolf et al teach:
The inference device according claim 21, wherein instructions causing the any of the one or more processors to execute processing of:
generating image data indicating the position of the object; and transmitting the image data using one-way communication with the console (0086 detail identify position/location of object (surgical tool, hand of surgeon, bleeding, motion, anatomical structure), where human operator viewing the ROI (region of interest) which is one way).
Claim 24:
Wolf et al teach:
The inference device according claim 23, wherein instructions causing the any of the one or more processors to execute processing of:
generating positional information indicating the position of the object; and transmitting the positional information using two-way communication with the console (0086 teaches position data, tracking of different object and ROI to human operator (one way); figure 14 teaches network 1418 with communication going both ways).
Claim 26:
Wolf et al teach:
The inference device according claim 21, wherein instructions causing the any of the one or more processors to execute processing of:
generating the control information in order to move the imaging unit following a tip portion of the recognized surgical device (0086 detail identify position/location of object such as surgical tool).
Claim 28:
Wolf et al teach:
The inference device according to claim 21, wherein the instructions causing the any of the one or more processors to execute processing of:
generating the control information in order to move the imaging unit to a designated position on the object (0086 teaches control information to move camera to zoom in and out of object (surgical tool, hand of surgeon, bleeding, motion, anatomical structure)).
Claim 29:
Wolf et al teach:
The inference device according to claim 21, wherein the instructions causing the any of the one or more processors to execute processing of:
generating the control information in order to move the imaging unit according to a distance between the imaging unit and the object (0086 teaches control information to move camera to zoom in and out of object (surgical tool, hand of surgeon, bleeding, motion, anatomical structure), where 0091 detail distance consideration between object and camera).
Claim 30:
Wolf et al teach:
The inference device according to claim 21, wherein the instructions causing the any of the one or more processors to execute processing of:
inferring a motion or gesture of an operator operating the console; and generating control information in order to control an operation of the imaging unit or a surgical device according to an inference result (0089 detail control motions such as orientation, zoom of camera in given surgical procedure to track motion of surgeon hands).
Claim 31:
Wolf et al teach:
The inference device according to claim 21, wherein the instructions causing the any of the one or more processors to execute processing of:
computing an area or shape of the recognized object; and generating control information for selecting or controlling a surgical device to be used according to the area or shape of the object (0182 teaches consideration such as color, shape, structure and condition cause changes to surgical procedure (control of surgery, which above is the camera as well as surgical robot)).
Claim 32:
Wolf et al teach:
The inference device according to claim 21, wherein the instructions causing the any of the one or more processors to execute processing of:
computing a confidence of the inference result (0629 and 0630 teaches calculating confidence probability or score); and changing a resolution of the operative field image according to the computed confidence (0089 teaches camera setting such as resolution is controlled; 0124 teaches playing of resolution based on decision marker, where 0560 and 0579 detail decision marker is based on confident level).
Claim 33:
Wolf et al teach:
The inference device according to claim 21, wherein the instructions causing the any of the one or more processors to execute processing of:
computing a confidence of the inference result (0629 and 0630 teaches calculating confidence probability or score); acquiring information of the surgical robot from the console; and computing a score of a surgery performed by the surgical robot on the basis of the computed confidence and the acquired information (0579 teaches surgical robot with confidence level).
Claim 34:
Wolf et al teach:
The inference device according to claim 21, wherein the inference device according to wherein the imaging unit of the surgical robot is configured to output an operative field image for a left eye and an operative field image for a right eye, and wherein the instructions causing the any of the one or more processors to execute processing of:
acquiring the operative field image for the left eye and the operative field image for the right eye output from the imaging unit, and inferring each of the acquired operative field image for the left eye and the acquired operative field image for the right eye (0091 teaches stereo camera for ROI, where stereo image provide field of view each for left eye and right eye individually).
Claim 35:
Wolf et al teach:
The inference device according to claim 34, wherein the instructions causing the any of the one or more processors to execute processing of:
computing a confidence of the inference result; and outputting an alert on the basis of a difference between the confidence of the inference result computed for the operative field image for the left eye and the confidence of the inference result computed for the operative field image for the right eye (0637-0638 teaches image-related data structure with correlated from other data that change the probability of confidence would generated alertness to undertake other surgical actions, where above teaches use of stereo images (left and right eye images)).
Claim 36:
Wolf et al teach:
The inference device according to claim 34, wherein the instructions causing the any of the one or more processors to execute processing of:
computing a confidence of the inference result; generating control information for moving the imaging unit according to a difference between the confidence of the inference result computed for the operative field image for the left eye and the confidence of the inference result computed for the operative field image for the right eye; and transmitting the generated control information to the console (0637-0638 teaches image-related data structure with correlated from other data that change (difference) the probability of confidence would generated alertness to undertake other surgical actions (generated controls information), where above teaches use of stereo images (left and right eye images)).
Claim 37:
Wolf et al teach:
The inference device according to claim 34, wherein the instructions causing the any of the one or more processors to execute processing of:
computing depth information on the basis of the operative field image for the left eye and the operative field image for the right eye; and transmitting the computed depth information to the console (0691 teaches frames consideration to depth of incision during surgery, where neural network configure to identify more specific intraoperative in 0692, above teaches use of image/frame from stereo camera for left and right eye image).
Claim 38:
Wolf et al teach:
The inference device according to claim 34, wherein the instructions causing the any of the one or more processors to execute processing of:
computing depth information on the basis of the operative field image for the left eye and the operative field image for the right eye; generating control information for controlling an operation of the surgical robot according to the computed depth information, and transmitting the generated control information to the console (teaches frames consideration to depth of incision during surgery, where neural network configure to identify more specific intraoperative (generated control information) in 0692, above teaches use of image/frame from stereo camera for left and right eye image).
Claim 41, similarly claim 42-43:
Wolf et al (US 2021/0012868) teach the following subject matter:
An inference device connected between a surgical robot and a console controlling the surgical robot, the inference device comprising:
one or more processors; and a storage storing instructions causing any of the one or more processors to execute processing of (figure 14 and 0290 teaches of structure of processor, storage/database/memory, non-transitory computer readable medium to contain and execute instruction):
acquiring an operative field image shot by an imaging unit of the surgical robot (0285 teaches surgical event with image frames, where 0537,0538 and 0595 detail the use of surgical robot with camera for capturing and transmitting images, video footage over network directly wired and/or wirelessly; 0090 teaches camera for moving to object with field of view and function such as adjust zoom, direction, speed);
performing an inference process on the acquired operative field image (0285 detail inferred base on set of images from surgical area (operative field image));
inferring at least one of a position of an object to be recognized in the operative field image and an event occurring in the operative field image (80 detail inferred output with classification of item depicted in the image (operative field image) with output including inferred value, size, volume, age, cost, product);
wherein the object to be recognized is connective tissue, where the connective tissue is an anatomical structure that serves as a landmark for a layer that can be excised, and an area of the connective tissue increases or decreases as a result of a surgeon's excision operation (0151 detail operation procedure (surgeon’s excision operation) on a patient body including cutting, abrading, suturing, or other techniques that involve physically changing body tissues and organs with list of organ and tissues can all be operated on and are all connected together; 0474 detail surgical operations such as skin graft is connective tissue; 0475 and 0484 detail specific surgical procedure such as fibrous tissue which is a connective tissue. Where all these connective tissues during surgery can be excision, altered or amended);
generating the control information in order to move the imaging unit according to a distance between the imaging unit and the object (0086 detail control application positioning camera to capture video/image ROI (region of interest) with identify structure, surgical tool, hand of surgeon, motion, particular location, tracking location with camera 115; 0087 teaches camera to image ROI (area) to larger using camera 115 to zoom in or out (increase or decrease));
transmitting at least one of the acquired operative field image and information based on an inference result to the console according to transmission settings by the console (0285-0286, specifically 0286 teaches additional surgeon assistance and/or guidance is requested base on set of frames (field image) that are inferred, where figure 14 and 0290 teaches transmission to 1410 (paragraph 0309-0311)); and
transmitting the control information generated by the control unit to the console (figure 4 and 0110 detail control through user interface 700 (console); 0086 teaches control of camera as well as infrared laser by operator; 0553 detail provide support for surgical procedures to user with recommendation and control commands to surgical robots).
Wolf et al teaches all the subject matter above but do not teach the following subject matter:
generating control information for controlling an operation of the surgical robot according to the inference result
MOTTRAM et al (US 2023/0149101) teaches the following subject matter:
generating control information for controlling an operation of the surgical robot according to the inference result (paragraph 0049 detail control configuration of robot based on inferred means, where 0036 detail such function for surgical robot).
Wolf et al and MOTTRAM et al are both in the field of image analysis, especially generating control information for surgical robot based on ROI imaging such that the combine outcome is predictable.
Therefore it would have been obvious to one having ordinary skill before the effective filing date to modify Wolf et al by MOTTRAM et al such using that information, the control system can model the effect of gravity on the components of the robot arm for the current configuration of the robot arm and estimate a force or torque due to gravity on each joint of the robot arm as disclosed by MOTTRAM et al in 0049.
Regarding claim 42, Wolf et al addresses method (figure 5).
Regarding claim 43, Wolf et al addresses non-transitory computer readable recording medium (figure 14 and 0290 teaches of structure of processor, storage/database/memory, non-transitory computer readable medium)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Shelton et al (2022/0233241) teaches SURGICAL PROCEDURE MONITORING - surgical computing system may receive usage data associated with movement of a surgical instrument and user inputs to the surgical instrument. The surgical computing system may receive motion and biomarker sensor data from sensing systems applied to the operator of the surgical instrument. The surgical computing system may determine, based on at least one of the usage data and/or the sensor data, an evaluation of the actions of the operator of the surgical instrument. The surgical computing system may determine, based on the evaluation, to provide feedback. The feedback may comprise instructions for the surgical instrument to provide haptic feedback and/or to modify its configuration. The feedback may comprise instructions for a display unit to present notifications instructing the healthcare professional.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSUNG-YIN TSAI whose telephone number is (571)270-1671. The examiner can normally be reached 7am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at (571) 272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSUNG YIN TSAI/Primary Examiner, Art Unit 2656
Prosecution Insights