DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed December 18th, 2025 has been entered. Claims 1, 15, 17, and 19 have been amended. Claims 1-20 are now pending in the application, with claims 12-13, 20 withdrawn. The previous 35 U.S.C. 112(b) rejections of claims 15 and 17 are withdrawn in light of Applicant's amendment.
Response to Arguments
Applicant's arguments filed December 18th, 2025 have been fully considered but they are not persuasive.
The Applicant makes the argument that Teranuma (US 20190183321 A1) does not disclose or suggest the following features: "acquire a first image captured by the image sensor, detect a first angle of a surgical instrument, the first angle being an angle formed by the surgical instrument in the first image with respect to a predetermined reference line that is set with respect to a plane of the first image, calculate a difference between the first angle and a predetermined target angle, and generate a second image rotated by the difference with respect to the first image, the surgical instrument forming a second angle in the second image with respect to the predetermined reference line such that the second angle is equal to the predetermined target angle." The examiner respectfully disagrees. As detailed below, the processor (CPU 110) of Teranuma has the functionality and structure to acquire a first angle (angle v) formed by a surgical instrument (400) in a first image and then calculate a difference (Y) between the first angle and a predetermined target angle (z), as is disclosed by the equation “Y=v-z,” and then rotate a second image based on that difference [FIGS. 13-14, 0084-0085]. Moreover, the applicant argues that claim 1 clearly defines a configuration in which a plane is to be in the vertical direction and needs to remain constant, however, the invention as claimed is not limited to exclusively read on the limitations with such an interpretation and is reasonably interpreted differently due to the language in the claims, as detailed in the rejection below.
The examiner asserts that the amendments to the invention as claimed do not overcome the prior art, as detailed below.
Examiner’s Comments
The present rejection(s) reference specific passages from cited prior art. However, Applicant is advised that the rejections are based on the entirety of each cited prior art. That is, each cited prior art reference “must be considered in its entirety”. (See MPEP 2141.02(VI)) Therefore, Applicant is advised to review all portions of the cited prior art if traversing a rejection based on the cited prior art.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3-11, 14, 19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Teranuma (US 20190183321 A1).
Regarding Claim 1, Teranuma discloses
A controller (control device 100) configured to control an image that is captured by an image sensor (first sensor unit 501) of an endoscope (camera 200A) and is displayed on a display screen (par. 64 discloses image screen) of a display device (display 300; par. 42 discloses control device causes the photographed image to be rotated and displayed in a preferred orientation, i.e. controls the image),
the controller comprising a processor (central processing unit (CPU) 110) comprising hardware (FIG. 3, par. 52 disclose hardware configuration of control device, i.e. CPU has hardware),
the processor being configured to:
acquire a first image (par. 84 discloses preprocessed image) captured by the image sensor (par. 63 discloses CPU acquires image data from camera),
detect a first angle (angle of orientation (v), FIGS. 13-14) of a surgical instrument (treatment instruments 400, FIG. 13, par. 83-84 disclose CPU acquires angle and orientation info of treatment instrument from preprocessed image),
the first angle being an angle formed by the surgical instrument in the first image with respect to a predetermined reference line (FIG. 14 depicts a dashed line, i.e. reference line) that is set with respect to a plane of the first image (FIGS. 13-14, par. 84 disclose angle of the orientation of the treatment instrument corresponds to the camera on a plane), and
calculate a difference (display correction angle Y, FIG. 14) between the first angle and a predetermined target angle (angle of orientation (z), FIGS. 13-14, par. 84-85 disclose display correction angle is calculated based on the following equation: Y=v-z), and
generate a second image rotated by the difference with respect to the first image (par. 85 discloses rotated image acquired based on first image being rotated by the display correction angle),
the surgical instrument forming a second angle in the second image with respect to the predetermined reference line such that the second angle is equal to the predetermined target angle (par. 85 discloses CPU rotates image by a certain angle based on the display correction angle, i.e. forms a new angle which lines up with the preferred correction angle).
Regarding Claim 3, Teranuma discloses
The controller according to claim 1, wherein the processor generates the second image by rotating the first image through image processing (par. 85 discloses CPU rotates photographed image, i.e. image processing).
Regarding Claim 4, Teranuma discloses
The controller according to claim 1, wherein the target angle is 0° (par. 69-71 disclose orientation of camera is taken at 0 degrees and the correction angle is set to match the first/ second angle, wherein the image is rotated by that same amount i.e. the rotation/ correction is meant to counter the offset and return to a target of 0 degrees).
Regarding Claim 5, Teranuma discloses
The controller according to claim 1, wherein the processor rotates the first image about a rotation axis parallel to an optical axis of the endoscope (par. 84 discloses image is rotated and corrected based on vector data from an orientation which is the image photographing direction, i.e. optical axis of camera, as depicted in FIG. 13).
Regarding Claim 6, Teranuma discloses
The controller according to claim 5, wherein the rotation axis passes through a center point of the first image (depicted in FIG. 14).
Regarding Claim 7, Teranuma discloses
The controller according to claim 1, wherein the first angle is an angle formed by a longitudinal axis of a shaft of the surgical instrument in the first image with respect to the reference line, and
the second angle is an angle formed by the longitudinal axis of the shaft of the surgical instrument in the second image with respect to the reference line (par. 84 discloses angles formed from the orientation v of the treatment instrument, i.e. longitudinal axis of surgical instrument, as depicted in FIG. 13-14).
Regarding Claim 8, Teranuma discloses
The controller according to claim 1, wherein the reference line is a straight line that forms a predetermined angle with respect to a horizontal line in the first image and is fixed relative to the first image (FIG. 14 depicts orientation z being perpendicular to a horizontal dashed line, i.e. forming a 90-degree angle).
Regarding Claim 9, Teranuma discloses
The controller according to claim 1, wherein the reference line is a horizontal line that is set in the first image and corresponds to a horizontal line on the display screen (depicted in FIG. 14).
Regarding Claim 10, Teranuma discloses
The controller according to claim 1, wherein the reference line is a vertical line that is set in the first image and corresponds to a vertical line on the display screen (FIG. 14 depicts orientation z as a vertical line).
Regarding Claim 11, Teranuma discloses
The controller according to claim 1, wherein the processor generates the second image when an absolute value of a difference between the first angle and the target angle is larger than a threshold value (par. 69-71 discloses that when the correction angle and another angle reach a specific value, the image is rotated).
Regarding Claim 14, Teranuma discloses
The controller according to claim 1, wherein the processor sets an angle of the surgical instrument at the target angle (par. 69-71, FIG. 9 disclose angle of treatment instrument is set to match correction angle by CPU), the surgical instrument having a tip disposed on a predetermined horizontal line on the display screen (FIG. 13 depicts tip of treatment instrument on horizontal x axis).
Regarding Claim 19, Teranuma discloses
A control method for controlling an image (par. 160-164 disclose an image output method for causing an image to be rotated) that is captured by an image sensor (first sensor unit 501) of an endoscope (camera 200A) and is displayed on a display screen (par. 64 discloses image screen) of a display device (display 300), the control method comprising:
acquiring a first image (par. 84 discloses preprocessed image) that is an image captured by the image sensor (par. 63 discloses CPU acquires image data from camera);
detecting a first angle (angle of orientation (v), FIGS. 13-14) of a surgical instrument (treatment instruments 400, FIG. 13, par. 83-84 disclose CPU acquires angle and orientation info of treatment instrument from preprocessed image),
the first angle being an angle formed by the surgical instrument in the first image with respect to a predetermined reference line (FIG. 14 depicts a dashed line, i.e. reference line) that is set with respect to a plane of the first image (FIGS. 13-14, par. 84 disclose angle of the orientation of the treatment instrument corresponds to the camera on a plane);
calculating a difference (display correction angle Y, FIG. 14) between the first angle and a predetermined target angle (angle of orientation (z), FIGS. 13-14, par. 84-85 disclose display correction angle is calculated based on the following equation: Y=v-z), and
generating a second image rotated by the difference with respect to the first image (par. 85 discloses rotated image acquired based on first image being rotated by the display correction angle),
the surgical instrument forming a second angle in the second image with respect to the predetermined reference line such that the second angle is equal to the predetermined target angle (par. 85 discloses CPU rotates image by a certain angle based on the display correction angle, i.e. forms a new angle which lines up with the preferred correction angle).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Teranuma (US 20190183321 A1).
Regarding Claim 2, Teranuma in a first embodiment discloses all of the elements of the current invention disclosed in claim 1, however, Teranuma in a first embodiment does not disclose wherein the processor generates the second image by rotating the image sensor.
Teranuma in a second embodiment, having the same control device (100) and camera (200A), discloses the control device rotates the image sensor of the camera [0107].
It would have been obvious to one of ordinary skill in the art at the effective filing date of
the invention to provide the first embodiment of Teranuma with the rotatable image sensor of the second embodiment of Teranuma in order to view the actual vertical upper side of the of the photographed target based on the data indicating the slant, the posture, and the like of the camera [0107].
Claim(s) 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Teranuma (US 20190183321 A1) as applied to claim 1 above, in view of Haraguchi (US 20180140168 A1).
Regarding Claim 15, Teranuma discloses all of the elements of the current invention disclosed in claim 1, however, Teranuma does not disclose wherein the processor is operable in a rotation mode, and in the rotation mode, the processor acquires the first image, detects the first angle, and generates the second image.
Haraguchi teaches an analogous controller (control device 50), having a processor (image-processing device 35A) which acquires image data output by an endoscope (imaging unit 8), detects rotation angle information, rotates the image, and outputs the rotated image signal to a display device (40) [0078]. Moreover, the control device (50) is connected to a rotation-driving instruction generation unit (53) which is operable to generate rotation-driving instructions, whether it be start or stop of the operation [0059].
It would have been obvious to one of ordinary skill in the art at the effective filing date of the invention to provide the controller of Teranuma with the rotation-driving instruction generation unit of Haraguchi in order to have more organization and control over the specific operations having to do with rotation, i.e. forward rotation, reverse rotation, and stopping [Haraguchi - 0064].
Regarding Claim 16, Teranuma, as previously modified by Haraguchi, discloses all of the elements of the current invention disclosed in claim 15, Haraguchi further teaches further comprising a user interface (switch unit 28) that receives an input of turn-on/turn-off of the rotation mode (par. 59 discloses rotation operation dependent on ON/OFF signals sent by switches of switch unit).
It would have been obvious to one of ordinary skill in the art at the effective filing date of the invention to provide the controller of Teranuma with the switch unit of Haraguchi in order to provide users the ability to selectively control which operations are running [0047].
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Teranuma (US 20190183321 A1) & Haraguchi (US 20180140168 A1), as applied to claim 16 above, and further in view of Wolf et al. (US 10729502 B1, hereinafter Wolf).
Regarding Claim 17, the combination of Teranuma & Haraguchi discloses all of the elements of the current invention disclosed in claim 16, however, the combination of Teranuma & Haraguchi does not disclose further comprising a storage unit that stores the predetermined target angle corresponding to each of a plurality of anatomical characteristics, wherein when the user interface receives the input of the turn-off of the rotation mode, the processor recognizes an anatomical characteristic in the first image from the plurality of anatomical characteristics, and the processor generates a fourth image rotated with respect to the first image on a basis of the predetermined target angle corresponding to the anatomical characteristic recognized by the processor, the predetermined target angle being stored in the storage unit.
Wolf teaches an analogous controller (computer-based camera control application) which positions, rotates, and tracks one or more moving cameras (115-125, i.e. endoscope) which are utilized for viewing surgical instruments, anatomical structures, or even their interactions [Col.13, ln.55-65]. Moreover, the anatomical structure information, associated complexity information, and particular anatomical structure information (i.e., type of structure) may be stored in storage locations that are accessible by the controller/ processor [Col. 79, Ln. 40-67 & Col. 141, Ln.15-35].
It would have been obvious to one of ordinary skill in the art at the effective filing date of the invention to provide the endoscope system of the combination of Teranuma & Haraguchi with the storage unit of Wolf in order to provide a database storing necessary instructions, program information, recommended sequence of events, anatomical structure information, and other useful information [Wolf - Col. 141, Ln.15-35].
Additionally, with this obvious modification implemented to the combined invention of Teranuma & Haraguchi, the storage unit of Wolf will have the capability to store the target angle information of Teranuma, as the storage unit is capable of storing various kinds of information [Wolf - Col. 141, Ln.15-35], and the processor of Teranuma will be able to carry out its regular operation, as the control device is capable of operating in accordance with several parameters/ instructions [Teranuma - 0120].
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Teranuma (US 20190183321 A1) as applied to claim 1 above, and further in view of Birkenbach (US 20220408002 A1).
Regarding Claim 18, Teranuma discloses all of the elements of the current invention disclosed in claim 1, and Teranuma further discloses,
An endoscope system (image output system 1, FIG. 13) comprising:
an endoscope (camera 200A, FIG. 13; par. 41 discloses camera is an endoscope);
However, Teranuma does not disclose a moving device that comprises a robot arm and that moves the endoscope in a subject.
Birkenbach teaches an analogous controller (control device 6), having a processor (par. 65 discloses digital processor) which performs an invention control adjustment method (depicted in FIG. 1), wherein first and second image data are acquired via an endoscope camera (5), moreover, the endoscope is supported by a motorized arm moving device (semi-robotic support arm 4) [0063-65].
It would have been obvious to one of ordinary skill in the art at the effective filing date of the invention to provide the endoscope system of Teranuma with the moving device of Birkenbach in order to remotely control the movement of the endoscope, via the control unit/ processor, on the basis of image data, and to better direct the endoscopes line of sight towards the target of interest [Birkenbach - 0038 & 0064].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDUL HADI ABBASI whose telephone number is (571)272-4076. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDUL HADI ABBASI/Examiner, Art Unit 3795
/RYAN N HENDERSON/Primary Examiner, Art Unit 3795
Prosecution Insights