Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, filed 10/27/2025 with respect to rejection of claims 25 and 26 under 35 USC 112(b) have been fully considered and persuasive. Therefore, the rejection has been withdrawn.
Applicant’s arguments, with respect to 102 rejection of claim 11 have been fully considered and are persuasive. Rod does not teach “wherein the edge device further includes processing circuitry configured as a haptic optical converter that converts the haptic information output from the haptic sensing devices into light intensity signals, and a light intensity-controlled light emitter attached to a distal end portion of the edge device that emits light of an intensity corresponding to the light intensity signals obtained by the haptic optical converter”.
No art was found to teach the above limitations. Therefore, claim 11 is deemed novel, hence the 102 rejection of claim 11 has been withdrawn.
Applicant's arguments with respect to claim 12 have been fully considered but they are not persuasive. Applicant stated:
the cited references also fail to teach or suggest "an imaging device to image the target and the edge device; and the processing circuitry further configured as a vision-based sensory transmission device, wherein the edge device further includes a passive marker, and the vision-based sensory transmission device identifies a position of the passive marker, generates visual haptic image information indicating an image corresponding to haptic information indicating the haptic sensation detected by the haptic sensing device, and superimposes marker point identification information obtained by a marker point identification function, the visual haptic image information obtained by a visual haptic image generation function, and imaging information obtained by the imaging device," as recited by amended Claim 12
Examiner respectfully disagrees. Amendment of claim 12 necessitated new grounds of rejection. Hence, the thrust of 103 rejection has been changed i.e., Panescu is primary art, and is modified thorough Rod.
Panescu teaches: an imaging device to image the target and the edge device ([0074], disclosing operator views surgical site including instruments and target. An imaging device is necessary to capture surgical site including instruments and target); and
the processing circuitry further configured as a vision-based sensory transmission device ([0128], disclosing artificial haptic sensors on the patient-side device to acquire haptic information, and haptic displays on the surgeon side to convey the sensed information to the surgeon),
wherein the edge device further includes a passive marker ([0065], disclosing one or more instruments may include a marker 189 to assist in acquisition and tracking of the instruments), and the
vision-based sensory transmission device identifies a position of the passive marker, generates visual haptic image information indicating an image corresponding to haptic information indicating the haptic sensation detected by the haptic sensing device, and superimposes marker point identification information obtained by a marker point identification function, the visual haptic image information obtained by a visual haptic image generation function, and imaging information obtained by the imaging device ((Abstract, disclosing measuring tissue structure deformation. [0156], disclosing using visual markers to provide tactile feedback. And visual indicia include a color-coded marker overlaid onto an image of the instrument in a surgical scene. The marker is colored green in response to less than an ideal amount of force applied. The marker is colored red in response to more than an ideal amount of force applied)
Panescu does not explicitly disclose marker to be a passive marker. However, it would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to modify Panescu to utilize passive markers o assist in acquisition and tracking of the instruments.
Although artificial haptic sensors on patient side are interpreted as present on edge device. Examiner acknowledges that Panescu does not explicitly disclose position of the sensors.
Rod teaches wherein the edge device includes a haptic sensing device to detect a haptic sensation to the target (column 3, disclosing tactile sensor 3b is disposed at a position corresponding to a fingertip of the slave robot R).
Rod and Panescu are analogous arts as they are in same field of endeavor i.e., remote operation of robots. It would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to further modify art of Panescu to wherein the edge device includes a haptic sensing device to detect a haptic sensation to the target as taught by Rod as additional means for detecting force/pressure/contact at surgical instrument.
Applicant’s arguments were primarily applicable to Rod because tactile sensation for Rod was provided to operator’s hands and not as a visual feedback (as claimed in claim 12). However Panescu teaches of tactile sensation as visual feedback through color coded markers. Hence Panescu modified through Rod teaches the above limitations. Therefore, rejection of claim 12 is maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 12-15 and 19-22 are rejected under 35 U.S.C. 103 as being unpatentable over Panescu (US 20170181808) in view of Rod (US 12017351).
For Claim 12, Rod teaches: A remote operation system comprising:
an edge device to contact a target ([0124], disclosing instrument tip 2206. Abstract, disclosing surgical instrument deforms a tissue i.e., a target);
processing circuitry configured as a motion operation transmission device to receive an operator operation for operating the edge device and output motion transmission information corresponding to the operator operation to the edge device ([0059-0060], disclosing a teleoperation surgery system); and
a visual display device to display an image of the target and the edge device ([0074], disclosing operator views surgical site including instruments and target),
an imaging device to image the target and the edge device ([0074], disclosing operator views surgical site including instruments and target. An imaging device is necessary to capture surgical site including instruments and target); and
the processing circuitry further configured as a vision-based sensory transmission device ([0128], disclosing artificial haptic sensors on the patient-side device to acquire haptic information, and haptic displays on the surgeon side to convey the sensed information to the surgeon),
wherein the edge device further includes a passive marker ([0065], disclosing one or more instruments may include a marker 189 to assist in acquisition and tracking of the instruments), and
the vision-based sensory transmission device identifies a position of the passive marker,
generates visual haptic image information indicating an image corresponding to haptic information indicating the haptic sensation detected by the haptic sensing device, and
superimposes marker point identification information obtained by a marker point identification function, the visual haptic image information obtained by a visual haptic image generation function, and imaging information obtained by the imaging device (Abstract, disclosing measuring tissue structure deformation. [0156], disclosing using visual markers to provide tactile feedback. And visual indicia include a color-coded marker overlaid onto an image of the instrument in a surgical scene. The marker is colored green in response to less than an ideal amount of force applied. The marker is colored red in response to more than an ideal amount of force applied)
Panescu does not explicitly disclose marker to be a passive marker. However, it would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to modify Panescu to utilize passive markers o assist in acquisition and tracking of the instruments.
NOTE: Examiner relied on Row (US 20230133825) as evidentiary art to conclude that active and passive markers are used in surgical robots. See Row [0036].
Although artificial haptic sensors on patient side are interpreted as present on edge device. Examiner acknowledges that Panescu does not explicitly disclose position of the sensors.
Rod teaches wherein the edge device includes a haptic sensing device to detect a haptic sensation to the target (column 3, disclosing tactile sensor 3b is disposed at a position corresponding to a fingertip of the slave robot R).
Rod and Panescu are analogous arts as they are in same field of endeavor i.e., remote operation of robots. It would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to further modify art of Panescu to wherein the edge device includes a haptic sensing device to detect a haptic sensation to the target as taught by Rod as additional means for detecting force/pressure/contact at surgical instrument.
For Claim 13, modified Panescu teaches: The remote operation system according to claim 12, wherein the haptic sensing device and the passive marker provided to the edge device are attached to a distal end portion to contact the target ([0065] and figure 6, disclosing marker 189 attached to distal end. [0156], disclosing using visual markers to provide tactile feedback. And visual indicia include a color-coded marker overlaid onto an image of the instrument in a surgical scene. The marker is colored green in response to less than an ideal amount of force applied. Abstract, disclosing measuring tissue structure deformation. [0013], disclosing A surgical instrument disposed within the field of view is operable to deform a tissue structure within the field of view. One or more processors are configured to produce a Q3D model that includes information indicative of a measure of tissue structure deformation. [0128], disclosing Kinesthetic or force feedback systems typically measure or estimate the forces applied to the patient by the surgical instrument).
For claim 14, modified Panescu teaches: The remote operation system according to claim 12,
wherein the vision-based sensory transmission device generates marker filter information for filtering the passive marker detected ([0156], disclosing coloring visual markers. Furthermore, marker used in [0065] is for tracking instruments and not for displaying to user. And as color is overlayed for feedback to used, original feature(s) of marker is/are filtered).
For claim 15, modified Panescu teaches: The remote operation system according to claim 13, wherein the vision-based sensory transmission device generates marker filter information for filtering the passive marker detected ([0156], disclosing coloring visual markers. Furthermore, marker used in [0065] is for tracking instruments and not for displaying to user. And as color is overlayed for feedback to used, original feature(s) of marker is/are filtered).
For Claim 19, modified Panescu teaches: The remote operation system according to claim 12, wherein the vision-based sensory transmission device generates, based on the imaging information, haptic image pattern selection information for selecting a haptic image pattern that is easy for a person to recognize, and marker filter image selection information for selecting a naturally harmonious image ([0156], disclosing overlaying color based on force exerted. Color of marker is generating haptic image pattern is selecting a haptic image pattern that is easy for a person to recognize. Furthermore, changing marker based on force is selecting a harmonious image).
For claim 20, modified Panescu teaches: The remote operation system according to claim 13, wherein the vision-based sensory transmission device generates, based on the imaging information, haptic image pattern selection information for selecting a haptic image pattern that is easy for a person to recognize, and marker filter image selection information for selecting a naturally harmonious image ([0156], disclosing overlaying color based on force exerted. Color of marker is generating haptic image pattern is selecting a haptic image pattern that is easy for a person to recognize. Furthermore, changing marker based on force is selecting a harmonious image).
For Claim 21, modified Panescu teaches: The remote operation system according to claim 14, wherein the vision-based sensory transmission device generates, based on the imaging information, haptic image pattern selection information for selecting a haptic image pattern that is easy for a person to recognize, and marker filter image selection information for selecting a naturally harmonious image ([0156], disclosing overlaying color based on force exerted. Color of marker is generating haptic image pattern is selecting a haptic image pattern that is easy for a person to recognize. Furthermore, changing marker based on force is selecting a harmonious image).
For Claim 22, modified Panescu teaches: The remote operation system according to claim 15, wherein the vision-based sensory transmission device generates, based on the imaging information, haptic image pattern selection information for selecting a haptic image pattern that is easy for a person to recognize, and marker filter image selection information for selecting a naturally harmonious image ([0156], disclosing overlaying color based on force exerted. Color of marker is generating haptic image pattern is selecting a haptic image pattern that is easy for a person to recognize. Furthermore, changing marker based on force is selecting a harmonious image).
Claims 25 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Panescu in view of Yasuke (WO 2019059364, disclosed in IDS submitted on 11/18/2023).
For claim 25, modified Panescu teaches: The remote operation system according to claim 12,
Panescu does not teach: wherein the vision-based sensory transmission device includes an edge device model memory to store geometric information on a point at which the edge device contacts the target.
Yasuke teaches: wherein the vision-based sensory transmission device includes an edge device model memory to store geometric information on a point at which the edge device contacts the target (abstract, disclosing remote control system and a model image generating unit. Page 2, disclosing a humanoid robot. Page 8, disclosing presentation image generation unit superimposes tactile image on model photograph to generate tactile display photographed image. And page 11, model image may simply be superimposed on the captured image. thus can accurately grasp the state of the humanoid robot 1 and can remotely control the humanoid robot 1 more easily).
Yasuke and Panescu are analogous arts as they are in same field of endeavor i.e., remote control of robots. It would have been obvious to one having ordinary skill in the art before effective filing date of claimed invention to further modify art of Panescu to wherein the vision-based sensory transmission device includes an edge device model memory to store geometric information on a point at which the edge device contacts the target as taught by Yasuke to improve operator perception of state of robot and control it easily.
Claim 26 recites limitations similarly in scope to claim 25, hence is similarly rejected.
Allowable Subject Matter
Claims 16, 17 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 11, 23, 24, 27 and 28 are allowed
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARSLAN AZHAR whose telephone number is (571)270-1703. The examiner can normally be reached Mon-Fri 7:30 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached at (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARSLAN AZHAR/Examiner, Art Unit 3656