DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over JP362 (JP7722362 B2) in view of Shelton (US 20250049293 A1).
Regarding claim 1, JP362 teaches A system for generating an endoscopic image, comprising:
an endoscope assembly (fig. 7, element 1102, p. 13, para. 12, imaging device 1102 e.g. a medical endoscope) for being used with a surgical instrument, wherein the endoscope assembly includes a plurality of image capturing elements;
a robot device (fig. 1, element 103, p. 2, para. 6, computerized surgical device 103) connecting with the endoscope assembly, wherein the robot device is configured to move the endoscope assembly to vicinity of a surgical site (p. 13, para. 12, autonomous arm holds an imaging device),
wherein the first image part is an image of a part of the surgical instrument (fig. 4b, element 401, p. 11, para. 3, tool 401 obstructs the camera’s view), the second image part is an image of a part of the surgical site that is not blocked by the part of the surgical instrument (fig. 4b, element 300, p. 11, para. 3, patient’s liver 300), at least one of the current images further includes a third image part (fig. 4b, element 403, p. 11, para. 3, additional image 403), and the third image part and the first image part correspond to a same area or a same position of the surgical site; and
a processing device (fig. 2, element 201, p. 4, para. 3, processor 201) electrically connected to the endoscope assembly, wherein the processing device is configured to:
identify positions and contours of the surgical instrument in the plurality of current images (fig. 4b, element 401, p. 11, para. 3, tool 401 obstructs the camera’s view);
divide the plurality of current images to separate the first image parts, the second image parts, and the third image parts of the plurality of current images; and
replace one of the first image parts with one of the third image parts, so as to obtain a processed image (p. 11, para. 3, camera perspective rotated 180 degrees to view the surgical site in an unobstructed field of view).
JP362 does not explicitly teach such that the plurality of image capturing elements capture a plurality of current images from different viewing angles, respectively, and one of the current images includes a first image part and a second image part;
However, Shelton teaches such that the plurality of image capturing elements capture a plurality of current images from different viewing angles, respectively, and one of the current images includes a first image part and a second image part (fig. 81, element 6520, [0656], a view in combination with any corresponding views from any additional camera(s) 6520 being utilized);
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the image assembly of JP362 to include multiple camera devices as taught in Shelton in order to create a more optimal FOV without repositioning cameras (Shelton [0656]).
Regarding claim 2, JP362 in view of Shelton teaches The system for generating the endoscopic image according to claim 1
Further, Shelton teaches wherein the process of replacing the first image part with the third image part to obtain the processed image further includes:
comparing the second image parts of the plurality of current images,
and selecting one of the current images as a main image; and
overlapping the current images ([0657, expansion of the visual field by merging images]), wherein:
if the first image part of the main image overlaps with the third image part of another current image, the first image part is replaced with the third image part of the another current image; or
if the third image part of the main image overlaps with the third image part of another current image, the third image part of the main image is retained for display ([0658], incorporation of multiple FOVs merged and overlapped).
Regarding claim 3, JP362 in view of Shelton teaches The system for generating the endoscopic image according to claim 1
Further, Shelton teaches wherein the endoscope assembly further includes an outer sleeve (fig. 14, element 1110, [0244], trocar 1110) and an inner sleeve (fig. 14, element 1102, [0244], surgical device 1102) that are detachable from each other, the outer sleeve has a first opening and a second opening that communicate with each other (Trocar is a hollow tube shape), the inner sleeve is disposed at the first opening, the plurality of image capturing elements are disposed on the inner sleeve and distributed around the inner sleeve (fig. 14, element 1106/1108, [0244], emitter 1106 and receiver 1108 on the same surgical device are both required to capture an image), and the plurality of image capturing elements faces the second opening of the outer sleeve.
Regarding claim 4, JP362 in view of Shelton teaches The system for generating the endoscopic image according to claim 1
Further, Shelton teaches further comprising a plurality of navigation marker frames that are deformable, wherein the plurality of navigation marker frames are disposed on the robot device, the endoscope assembly, and the surgical site for establishing a spatial coordinate system;
wherein the robot device is configured to guide the surgical instrument to move to vicinity of the surgical site based on the spatial coordinate system, and through the spatial coordinate system and the endoscope assembly, the processing device obtains position information of the surgical instrument in the surgical site and generates the processed image corresponding to the position information (fig. 25a, element 4642, [0368], synchronization is improved by markings on the end effector 4642 to develop the virtual representation of the internal component).
Regarding claim 5, JP362 teaches A method of generating an endoscopic image, comprising:
providing a robot device (fig. 1, element 103, p. 2, para. 6, computerized surgical device 103) to connect an endoscope assembly (fig. 7, element 1102, p. 13, para. 12, imaging device 1102 e.g. a medical endoscope) and move the endoscope assembly to vicinity of a surgical site (p. 13, para. 12, autonomous arm holds an imaging device), wherein the endoscope assembly is suitable for being used with a surgical instrument;
wherein the first image part is an image of a part of the surgical instrument (fig. 4b, element 401, p. 11, para. 3, tool 401 obstructs the camera’s view), the second image part is an image of a part of the surgical site that is not blocked by the part of the surgical instrument (fig. 4b, element 300, p. 11, para. 3, patient’s liver 300), at least one of the current images further includes a third image part (fig. 4b, element 403, p. 11, para. 3, additional image 403), and the third image part and the first image part correspond to a same area or a same position of the surgical site; and
providing a processing device (fig. 2, element 201, p. 4, para. 3, processor 201) that is electrically connected to the endoscope assembly to:
identify positions and contours of the surgical instrument in the plurality of current images (fig. 4b, element 401, p. 11, para. 3, tool 401 obstructs the camera’s view);
divide the plurality of current images to separate the first image parts, the second image parts, and the third image parts of the plurality of current images; and
replace one of the first image parts with one of the third image parts, so as to obtain a processed image (p. 11, para. 3, camera perspective rotated 180 degrees to view the surgical site in an unobstructed field of view).
JP362 does not explicitly teach providing a plurality of image capturing elements that are disposed on the endoscope assembly to respectively capture a plurality of current images of the surgical site from different viewing angles, and one of the current images includes a first image part and a second image part,
However, Shelton teaches providing a plurality of image capturing elements that are disposed on the endoscope assembly to respectively capture a plurality of current images of the surgical site from different viewing angles, and one of the current images includes a first image part and a second image part (fig. 81, element 6520, [0656], a view in combination with any corresponding views from any additional camera(s) 6520 being utilized);
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the image assembly of JP362 to include multiple camera devices as taught in Shelton in order to create a more optimal FOV without repositioning cameras (Shelton [0656]).
Regarding claim 6, JP362 in view of Shelton teaches The method of generating the endoscopic image according to claim 5
Further, Shelton teaches wherein the process of replacing the first image part with the third image part to obtain the processed image further includes:
comparing the second image parts of the plurality of current images,
and selecting one of the current images as the main image; and
overlapping the current images ([0657, expansion of the visual field by merging images]), wherein:
if the first image part of the main image overlaps with the third image part of another current image, the first image part is replaced with the third image part of the another current image; or if the second image part of the main image overlaps with the third image part of another current image, the second image part of the main image is retained for display ([0658], incorporation of multiple FOVs merged and overlapped).
Regarding claim 7, JP362 in view of Shelton teaches The method of generating the endoscopic image according to claim 5
Further, Shelton teaches wherein the endoscope assembly further includes an outer sleeve (fig. 14, element 1110, [0244], trocar 1110) and an inner sleeve (fig. 14, element 1102, [0244], surgical device 1102) that are detachable from each other, the outer sleeve has a first opening and a second opening that communicate with each other (Trocar is a hollow tube shape), the inner sleeve is disposed at the first opening and is located inside the outer sleeve, the plurality of image capturing elements are disposed on the inner sleeve (fig. 14, element 1106/1108, [0244], emitter 1106 and receiver 1108 on the same surgical device are both required to capture an image) and distributed around the inner sleeve, and the plurality of image capturing elements faces the second opening of the outer sleeve.
Regarding claim 8, JP362 in view of Shelton teaches The method of generating the endoscopic image according to claim 7
Further, Shelton teaches wherein the process of moving the endoscope assembly to the surgical site further includes:
calibrating a plurality of lenses of the plurality of image capturing elements ([0219], follower arm and camera can be programmed to track the other camera and maintain a particular distance or lens angle, hence calibration to follow the other arm); and
assembling the inner sleeve onto the outer sleeve (fig. 14, insertion of endoscope into trocar).
Regarding claim 9, JP362 in view of Shelton teaches The method of generating the endoscopic image according to claim 8
Further, Shelton teaches wherein, before the process of assembling the inner sleeve onto the outer sleeve, the method further includes:
assembling a plurality of deformable navigation marker frames (fig. 25a, element 4642, [0368], synchronization is improved by markings on the end effector 4642 to develop the virtual representation of the internal component) onto the robot device, the outer sleeve, and the surgical site to establish a spatial coordinate system;
utilizing the processing device to plan a surgical path (fig. 30, element 4135, recommended treatment path) based on the spatial coordinate system, so as to determine a position and an angle at which the outer sleeve moves to vicinity of the surgical site; and
utilizing the robot device to move the outer sleeve vicinity of the surgical site based on the surgical path.
Regarding claim 10, JP362 in view of Shelton teaches The method of generating the endoscopic image according to claim 9
Further, Shelton teaches wherein, after the process of utilizing the robot device to move the outer sleeve to the surgical site based on the surgical path, the method further includes:
utilizing the robot device to obtain position information of the surgical instrument in the surgical site and the processed image corresponding to the position information through the spatial coordinate system and the endoscope assembly ([0213],robotic arms can be positioned and registered to a particular coordinate system).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIMOTHY TUAN LUU whose telephone number is (703)756-4592. The examiner can normally be reached Monday-Tuesday, Thursday-Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Carey can be reached at 5712707235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIMOTHY TUAN LUU/ Examiner, Art Unit 3795
/MICHAEL J CAREY/ Supervisory Patent Examiner, Art Unit 3795
Prosecution Insights