The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claim 10 is objected to because of the following informalities:
Claim 10 is objected to because “an” in line 10 should state “and” – “mounted in an operating room including the imaging device and the display device;”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Mizoguchi et al. (JP 2006-346106, machine translation provided) alone.
Mizoguchi discloses a system (i.e., “an apparatus for observing a three-dimensional image for an operation”, see Abstract) comprising:
A display device (see numeral 22 in Figures 1 and 3);
An actuated structure mounted to the display device and configured to move the display device in three-dimensional space (see stepping motor 24b1, and 24c11 in Figure 3; see page 6, two paragraphs starting at “The left-right direction rotation shaft portion24b…”);
An imaging device (see numeral 11 in Figure 1, which is “surgical stereoscopic microscope image pickup apparatus 1”) configured to capture images of a patient’s eye during an ophthalmic surgery (see Background-Art section on page 2 and also see Industrial Applicability section spanning pages 15-16 where it teaches that this system is used in ophthalmology surgeries, which inherently, or at the very least it would be obvious that this will be acquiring images of the patient’s eye(s)) and display the images on the display device (see Figure 2, which illustrates acquisition of images via imaging device 11 on the right-side, passing of the image data to camera control units (CCU) 45, and passing of that data to left and right projectors 21L and 21R which are also shown in Figure 1 projecting PL and PR onto display 22; see paragraph spanning pages 4-5 and first two or three paragraphs on page 5); and
A controller (the Abstract teaches “a display position control means for control rotation of the display surface 22 by the visual field movement means 24 on the basis of the detection result of the detection means 25) configured to activate the actuated structure to maintain the display device in a field of view of a surgeon performing the ophthalmic surgery (see same quote, also see page 8, paragraph beginning “Based on the detection result of the observation position …”).
Regarding claims 2-3, Mizoguchi teaches that “The surgical stereoscopic image microscope system includes a surgical stereoscopic microscope image capturing apparatus 1” (see page 4, first two paragraphs under Best-Mode header). Also note that as discussed above in the rejection of claim 1, the Background-Art section on page 2 and also the Industrial Applicability section spanning pages 15-16 teaches that this system is used in ophthalmology surgeries, which inherently, or at the very least it would be obvious that this will be acquiring images of the patient’s eye(s). Therefore, this teaches that the imaging device is an ophthalmic microscope, as in claim 2. The first quote in this rejection specifically states that it is a stereoscopic image microscope system, which therefore teaches that the imaging device is a stereoscopic ophthalmic microscope, as in claim 3.
Regarding claim 4, Mizoguchi teaches, on page 7 in the paragraph beginning “FIG. 8 is a diagram…”, that “A center position Y of a straight line connecting these center points PL0 and PR0 is a reference position for position control. Then, the center position Y between the center points PL0 and PR0 of the left and right eyes ER and EL of the observer D detected at the time of the initial position setting operation described later is set as the initial setting center position Y0.” From this paragraph through page 8 in the paragraph beginning “Based on the detection result…”, it teaches that the system compares any change from this initial setting center position Y0 in the surgeon’s pupil position and “Based on the detection result of the observation position detection device 25, the display position control means (not shown) moves the image display panel 22 in the vertical and horizontal directions…”. By effect, this teaches that the orientation and distance of the display device is maintained relative to the surgeon, which will inherently or obvious enable the same three-dimensional perception by the surgeon as was obtained in the initial setting center position Y0.
Regarding claim 10, Mizoguchi illustrates the receiving one of more inputs including “one or more second camera images received from one or more second cameras mounted to the display device” (see numeral 25 in Figures 1 and/or 3).
Claims 5-7 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Mizoguchi as applied to claim 1 above, and further in view of Hallen (US Patent Pub. No. 2019/009926).
Regarding claim 5, Mizoguchi teaches that the controller is configured to activate the actuated structure to maintain the display device in the field of view of the surgeon performing the ophthalmic surgery (see above rejections of claim 1, also see the rejection of claim 4). However, there is no discussion of using a surgeon profile for determining a position and orientation of the display.
Hallen teaches that “surgeons can become fatigued after being bent over a microscope eyepiece. A high dynamic range digital camera system can be used to capture a multidimensional visualization of a patient's eye that can be transmitted to a high-definition display. While such displays allow a surgeon to conduct surgery in a heads-up posture, the potential for using stereoscopic visualization on a heads-up display capable of performing advanced operations for improving other areas of a surgical practice is not currently realized” (see paragraph 3). Particularly, Hallen teaches receiving a surgeon profile (see paragraph 81, where it teaches that “the method 650 involves accessing surgeon preferences from profile database”), selecting a position and orientation of the display device according to the surgeon profile (see paragraph 81, “adjusting display orientation settings based on surgeon preferences … 666”; see Figure 6C); and activating the actuated structure to achieve the position and orientation (see step 674 in Figure 6C, also see last sentence of paragraph 76). Also see all of paragraphs 70-81.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize a surgeon profile with preferred settings by the particular surgeon, as taught by Hallen, within the system and methods of Mizoguchi because by providing these settings, as well as those discussed in paragraph 75 of Hallen, the system can adjust automatically based on the surgeon or based on the step or phase of the surgery, thereby increasing efficiency of the system of Mizoguchi.
Regarding claim 6, Hallen taches in paragraph 79 that surgical procedure data, which can include a pre-operative surgical plan, can be provided to the surgical suite optimization engine. It is from this that steps of Figure 6C are described as moving the display based on the step or phase of the surgical procedure (see, for instance, step 670; also see paragraphs 70-81).
Regarding claim 7, it is noted that Figures 6A and 6B of Hallen illustrate a surgical suite in which a stool is located in a superior position. Therefore, it would be obvious to one of ordinary skill in the art that any treatment plan provided to this system would place the surgeon in the superior position, and Figures 6A and 6B clearly illustrate the display 604 in a position according to the surgeon in the superior position, such that its distance D1 or D2 is optimized based on the surgeon’s profile and/or the step or phase of the procedure, as described in paragraphs 70-81.
Regarding claim 9, Figure 3 of Mizoguchi illustrates an articulating/robotic arm 23, including “a plurality of arms 23a, a multi-joint arm having a joint 23b that rotatably connects the arms 23a, and a stereoscopic image display means support arm 23c” (see page 5, paragraph beginning “The stereoscopic image display means support unit 23 includes…”). The arm includes multiple degrees of freedom via the plurality of joints and rotatable axes. For instance, rotation about vertical arm 23a in Figure 3 provides for at least two degrees of freedom (i.e., an x-direction and a z-direction), rotation about axes O1 and O2 provides at least one degree of freedom each, and the incorporation of Hallen providing height adjustment adds at least a fifth degree of freedom in the z-direction.
Claims 8 are rejected under 35 U.S.C. 103 as being unpatentable over Mizoguchi in view of Hallen as applied to claim 7 above, and further in view of Riederer et al. (US Patent Pub. No. 2007/0188603).
Regarding claim 8, Mizoguchi teaches that the controller is configured to activate the actuated structure to maintain the display device in the field of view of the surgeon performing the ophthalmic surgery (see above rejections of claim 1, also see the rejection of claim 4). Meanwhile, Hallen teaches that “the support member 632 can be adjusted in the x- and y-directions. For example, the support member 632 can include a vertical telescopic member that can be adjusted in the y-direction...” (see paragraph 73 of Hallen), while also teaching receiving a surgeon profile for adjustment of a display (see rejection of claim 5 above, which cites paragraph 81, where it teaches that “the method 650 involves accessing surgeon preferences from profile database”). However, while Hallen teaches height adjustment capabilities, it is not taught as being related to the surgeon.
Riederer teaches a stereoscopic display cart and system (see Title). Figure 16 illustrates the cart, with vertical column 52 as the first connection of the cart to the display 36. In paragraph 237, Riederer teaches that “the vertical column is adjustable such that the projectors, (if configured), can be moved up or down simultaneously with the screen such that the height of the screen can be set as desired by the user”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application that utilizing adjustment of the height of the projectors and screen as desired by the user, as taught by Riederer, within the system and methods of Mizoguchi as combined with Hallen is a capability already provided with the adjustment of height taught by Hallen and to change the height according to the desires of the user (as taught in Riederer) within the system of Hallen would obviously be improved by having this desired set up in the accessing of the surgeron preferences from the profile database, thereby automating the process of display set up and improving the overall ease of use of the system.
Claims 11-14 are rejected under 35 U.S.C. 103 as being unpatentable over Mizoguchi in view of Wada et al. (US Patent Pub. No. 2019/0328479).
Mizoguchi discloses a system (i.e., “an apparatus for observing a three-dimensional image for an operation”, see Abstract) comprising:
A display device (see numeral 22 in Figures 1 and 3);
An actuated structure mounted to the display device and configured to move the display device in three-dimensional space (see stepping motor 24b1, and 24c11 in Figure 3; see page 6, two paragraphs starting at “The left-right direction rotation shaft portion24b…”);
An imaging device (see numeral 11 in Figure 1, which is “surgical stereoscopic microscope image pickup apparatus 1”) configured to capture surgical images of a patient’s eye during an ophthalmic surgery (see Background-Art section on page 2 and also see Industrial Applicability section spanning pages 15-16 where it teaches that this system is used in ophthalmology surgeries, which inherently, or at the very least it would be obvious that this will be acquiring images of the patient’s eye(s)) and display the images on the display device (see Figure 2, which illustrates acquisition of images via imaging device 11 on the right-side, passing of the image data to camera control units (CCU) 45, and passing of that data to left and right projectors 21L and 21R which are also shown in Figure 1 projecting PL and PR onto display 22; see paragraph spanning pages 4-5 and first two or three paragraphs on page 5); and
A controller (the Abstract teaches “a display position control means for control rotation of the display surface 22 by the visual field movement means 24 on the basis of the detection result of the detection means 25) configured to… activate the actuated structure to maintain the display device… in a field of view of a surgeon performing the ophthalmic surgery (see same quote, also see page 8, paragraph beginning “Based on the detection result of the observation position …”).
However, Mizoguchi teaches a camera mounted to the display unit which governs actuation of the display device, as opposed to a camera on the imaging device, and that the controller receives camera images from the camera, evaluates representations of the display device in the camera images, and that actuation of the structure is based on the representations of the display device.
Wada teaches control devices and methods for more safely controlling support arms (see Technical Problem section). In Figure 1, numeral 111 represents a surgical microscope, numeral 130 is a peripheral image acquisition camera, and numeral 120 is a display device (see paragraph 37). The peripheral image acquisition camera 130 is described in paragraph 47, while Figure 3 illustrates an example image obtained by this camera, which includes display device 120 (see paragraph 73). Figure 4 is a diagram illustrating an example of a peripheral image with distance information (see paragraph 75), which includes bold outlining of the display device 120. In paragraph 65, Wada teaches that “although the peripheral image acquisition camera 130 is mounted in the microscope unit 111 in the above-described configuration example, the present embodiment is not limited thereto. The peripheral image acquisition camera 130 may be installed at an arbitrary position”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to provide the camera of Mizoguchi on the microscope and image the surrounding environment of the microscope, such as the surgeons and other professionals, support arm(s), display devices, etc., as taught by Wada, and to utilize such a camera and acquired images to determine distance information and light of sight information (see paragraph 60 of Wada) for moving the display to reduce fatigue (which is the stated purpose of Mizoguchi).
Regarding claims 12-13, Mizoguchi teaches that “The surgical stereoscopic image microscope system includes a surgical stereoscopic microscope image capturing apparatus 1” (see page 4, first two paragraphs under Best-Mode header). Also note that as discussed above in the rejection of claim 1, the Background-Art section on page 2 and also the Industrial Applicability section spanning pages 15-16 teaches that this system is used in ophthalmology surgeries, which inherently, or at the very least it would be obvious that this will be acquiring images of the patient’s eye(s). Therefore, this teaches that the imaging device is an ophthalmic microscope, as in claim 12. The first quote in this rejection specifically states that it is a stereoscopic image microscope system, which therefore teaches that the imaging device is a stereoscopic ophthalmic microscope, as in claim 13.
Regarding claim 14, Mizoguchi teaches, on page 7 in the paragraph beginning “FIG. 8 is a diagram…”, that “A center position Y of a straight line connecting these center points PL0 and PR0 is a reference position for position control. Then, the center position Y between the center points PL0 and PR0 of the left and right eyes ER and EL of the observer D detected at the time of the initial position setting operation described later is set as the initial setting center position Y0.” From this paragraph through page 8 in the paragraph beginning “Based on the detection result…”, it teaches that the system compares any change from this initial setting center position Y0 in the surgeon’s pupil position and “Based on the detection result of the observation position detection device 25, the display position control means (not shown) moves the image display panel 22 in the vertical and horizontal directions…”. By effect, this teaches that the orientation and distance of the display device is maintained relative to the surgeon, which will inherently or obvious enable the same three-dimensional perception by the surgeon as was obtained in the initial setting center position Y0.
Claims 15-17 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Mizoguchi in view of Wada as applied to claim 11 above, and further in view of Hallen.
Regarding claim 15, Mizoguchi in combination with Wada teaches that the controller is configured to activate the actuated structure to maintain the display device in the field of view of the surgeon performing the ophthalmic surgery (see above rejections of claim 1, also see the rejection of claim 4). However, there is no discussion of using a surgeon profile for determining a position and orientation of the display.
Hallen teaches that “surgeons can become fatigued after being bent over a microscope eyepiece. A high dynamic range digital camera system can be used to capture a multidimensional visualization of a patient's eye that can be transmitted to a high-definition display. While such displays allow a surgeon to conduct surgery in a heads-up posture, the potential for using stereoscopic visualization on a heads-up display capable of performing advanced operations for improving other areas of a surgical practice is not currently realized” (see paragraph 3). Particularly, Hallen teaches receiving a surgeon profile (see paragraph 81, where it teaches that “the method 650 involves accessing surgeon preferences from profile database”), selecting a position and orientation of the display device according to the surgeon profile (see paragraph 81, “adjusting display orientation settings based on surgeon preferences … 666”; see Figure 6C); and activating the actuated structure to achieve the position and orientation (see step 674 in Figure 6C, also see last sentence of paragraph 76). Also see all of paragraphs 70-81.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to utilize a surgeon profile with preferred settings by the particular surgeon, as taught by Hallen, within the system and methods of Mizoguchi as combined with Wada because by providing these settings, as well as those discussed in paragraph 75 of Hallen, the system can adjust automatically based on the surgeon or based on the step or phase of the surgery, thereby increasing efficiency of the system of Mizoguchi.
Regarding claim 16, Hallen taches in paragraph 79 that surgical procedure data, which can include a pre-operative surgical plan, can be provided to the surgical suite optimization engine. It is from this that steps of Figure 6C are described as moving the display based on the step or phase of the surgical procedure (see, for instance, step 670; also see paragraphs 70-81).
Regarding claim 17, it is noted that Figures 6A and 6B of Hallen illustrate a surgical suite in which a stool is located in a superior position. Therefore, it would be obvious to one of ordinary skill in the art that any treatment plan provided to this system would place the surgeon in the superior position, and Figures 6A and 6B clearly illustrate the display 604 in a position according to the surgeon in the superior position, such that its distance D1 or D2 is optimized based on the surgeon’s profile and/or the step or phase of the procedure, as described in paragraphs 70-81.
Regarding claims 19-20, Figure 3 of Mizoguchi illustrates an articulating/robotic arm 23, including “a plurality of arms 23a, a multi-joint arm having a joint 23b that rotatably connects the arms 23a, and a stereoscopic image display means support arm 23c” (see page 5, paragraph beginning “The stereoscopic image display means support unit 23 includes…”). The arm includes multiple degrees of freedom via the plurality of joints and rotatable axes. For instance, rotation about vertical arm 23a in Figure 3 provides for at least two degrees of freedom (i.e., an x-direction and a z-direction), rotation about axes O1 and O2 provides at least one degree of freedom each, and the incorporation of Hallen providing height adjustment adds at least a fifth degree of freedom in the z-direction.
Claims 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mizoguchi in view of Wada and Hallen as applied to claim 17 above, and further in view of Riederer et al. (US Patent Pub. No. 2007/0188603).
Regarding claim 18, Mizoguchi teaches that the controller is configured to activate the actuated structure to maintain the display device in the field of view of the surgeon performing the ophthalmic surgery (see above rejections of claim 1, also see the rejection of claim 4). Meanwhile, Hallen teaches that “the support member 632 can be adjusted in the x- and y-directions. For example, the support member 632 can include a vertical telescopic member that can be adjusted in the y-direction...” (see paragraph 73 of Hallen), while also teaching receiving a surgeon profile for adjustment of a display (see rejection of claim 5 above, which cites paragraph 81, where it teaches that “the method 650 involves accessing surgeon preferences from profile database”). However, while Hallen teaches height adjustment capabilities, it is not taught as being related to the surgeon.
Riederer teaches a stereoscopic display cart and system (see Title). Figure 16 illustrates the cart, with vertical column 52 as the first connection of the cart to the display 36. In paragraph 237, Riederer teaches that “the vertical column is adjustable such that the projectors, (if configured), can be moved up or down simultaneously with the screen such that the height of the screen can be set as desired by the user”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application that utilizing adjustment of the height of the projectors and screen as desired by the user, as taught by Riederer, within the system and methods of Mizoguchi as combined with Wada and Hallen is a capability already provided with the adjustment of height taught by Hallen and to change the height according to the desires of the user (as taught in Riederer) within the system of Hallen would obviously be improved by having this desired set up in the accessing of the surgeron preferences from the profile database, thereby automating the process of display set up and improving the overall ease of use of the system.
Conclusion
Additional prior art is made of record in the attached PTO-892, and is considered pertinent to applicant's disclosure, but not relied upon in the rejections above.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES KISH whose telephone number is (571)272-5554. The examiner can normally be reached M-F 10:00a - 6p EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Unsu Jung can be reached at (571) 272-8506. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES KISH/ Primary Examiner, Art Unit 3792
Prosecution Insights