CTNF 18/729,113 CTNF 84603 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claim 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a Judicial Exception in the form of an Abstract Idea, without significantly more: Beginning with independent claim 1 , a device claim, which recites: An image processing device comprising: at least one memory configured to store instructions; and at least one processor configured to execute the instructions to: generate, based on endoscopic images of an examination target captured by an endoscope, reconstruction data obtained by three-dimensionally reconstructing the examination target; perform matching between a three-dimensional model of the examination target and the reconstruction data; generate, based on a result of the matching, complemented reconstruction data which is the reconstruction data complemented with the three-dimensional model; and display the complemented reconstruction data on a display device. The claim recites abstract ideas: “An image processing device, comprising at least one memory configured to store instructions; and at least one processor, and the instructions that is stored in the memory and capable of being executed the processor to perform the “generating” steps, the “performing” step, the “displaying” step is considered being performed by a generic computer. Therefore, If the apparatus, processor and memory are removed from the claim, the method can be easily performed by a human being without the need of any of a computer component. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. “generating” is mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity. A process that encompass a human performing the steps mentally with or without a physical aid in the form of the “displaying” step, with the “generating” steps, the “performing” step being pre-solution acts of processing information which could be performed visually and/or mentally; and A method of organizing human behavior in the form of a social activity of following rules or instructions informing a person to perform “generating” steps, the “performing” step, the “displaying”. These two abstract ideas will be considered together for analysis as a single abstract idea per MPEP 2106: PNG media_image1.png 468 1527 media_image1.png Greyscale This judicial exception is not integrated into a practical application because there are no recited additional elements that amount to a practical application, such as but no limited to the following as noted in MPEP 2106: PNG media_image2.png 453 1451 media_image2.png Greyscale The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception for the same reason: There are not additional elements other than the abstract idea. Independent claim 11 , a processing claim, which recites: An image processing method executed by a computer, the image processing method comprising: generating, based on endoscopic images of an examination target captured by an endoscope, reconstruction data obtained by three-dimensionally reconstructing the examination target; performing matching between a three-dimensional model of the examination target and the reconstruction data; generating, based on a result of the matching, complemented reconstruction data which is the reconstruction data complemented with the three-dimensional model; and displaying the complemented reconstruction data on a display device. The claim recites abstract ideas: “generating” is mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity. A process that encompass a human performing the steps mentally with or without a physical aid in the form of the “displaying” step, with the “generating” steps, the “performing” step being pre-solution acts of processing information which could be performed visually and/or mentally; and A method of organizing human behavior in the form of a social activity of following rules or instructions informing a person to perform “generating” steps, the “performing” step, the “displaying”. These two abstract ideas will be considered together for analysis as a single abstract idea per MPEP 2106: PNG media_image1.png 468 1527 media_image1.png Greyscale This judicial exception is not integrated into a practical application because there are no recited additional elements that amount to a practical application, such as but no limited to the following as noted in MPEP 2106: PNG media_image2.png 453 1451 media_image2.png Greyscale The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception for the same reason: There are not additional elements other than the abstract idea. Independent claim 12 , a device claim, which recites: A non-transitory computer readable storage medium storing a program executed by a computer, the program causing the computer to: generate, based on endoscopic images of an examination target captured by an endoscope, reconstruction data obtained by three-dimensionally reconstructing the examination target; perform matching between a three-dimensional model of the examination target and the reconstruction data; generate, based on a result of the matching, complemented reconstruction data which is the reconstruction data complemented with the three-dimensional model; and display the complemented reconstruction data on a display device. The claim recites abstract ideas: “generating” is mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity. A process that encompass a human performing the steps mentally with or without a physical aid in the form of the “displaying” step, with the “generating” steps, the “performing” step being pre-solution acts of processing information which could be performed visually and/or mentally; and A method of organizing human behavior in the form of a social activity of following rules or instructions informing a person to perform “generating” steps, the “performing” step, the “displaying”. These two abstract ideas will be considered together for analysis as a single abstract idea per MPEP 2106: PNG media_image1.png 468 1527 media_image1.png Greyscale This judicial exception is not integrated into a practical application because there are no recited additional elements that amount to a practical application, such as but no limited to the following as noted in MPEP 2106: PNG media_image2.png 453 1451 media_image2.png Greyscale The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception for the same reason: There are not additional elements other than the abstract idea. Independent claims 1. 11 and 12 are merely a generic computer implementation of the abstract ideas and likewise do not amount to significantly more. See MPEP 2106: PNG media_image3.png 249 1434 media_image3.png Greyscale Likewise, the following dependent claims have been analyzed and do not recite elements that recite a practical application or significantly more and remain rejected under 35 USC 101: Claims 2-9 . Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-3 and 8-12 are rejected under 35 U.S.C. 103 as being unpatentable over Kobayashi’127 (US 12,266,127), and further in view of Popovic’225 (WO 2013061225) . With respect to claim 1 , Kobayashi’127 teaches an image processing device [regarding the system shown in Fig.1] comprising: at least one memory (Fig.10, items 903, 904 and 905) configured to store instructions; and at least one processor (Fig.10, item 902) configured to execute the instructions to: generate, based on endoscopic images of an examination target captured by an endoscope, reconstruction data obtained by three-dimensionally reconstructing the examination target (Fig. 3); Kobayashi’127 does not teach perform matching between a three-dimensional model of the examination target and the reconstruction data; generate, based on a result of the matching, complemented reconstruction data which is the reconstruction data complemented with the three-dimensional model; and display the complemented reconstruction data on a display device. Popovic’225 teaches performs matching between a three-dimensional model of the examination target and the reconstruction data (Fig.12, step S173); generate, based on a result of the matching, complemented reconstruction data which is the reconstruction data complemented with the three-dimensional model [A stage S173 of flowchart 170 encompasses an update of the pre-operative volumetric image. In practice, the updated image 133 may full illustrate the complete arterial tree or may eliminate the bypassed portion of the complete arterial tree. For example, as shown in stage S173 of FIG. 12, an updated image 133a of the pre-operative volumetric image 133 illustrates the complete arterial tree including the bypass 181 or an updated image 134b of the pre-operative volumetric image 133 illustrates the arterial tree exclusive of the bypassed portion of the arterial tree. Flowchart 170 returns to stage S32 (FIG. 2) where the updated image 133 may be used to re-register the intra-operative endoscopic image 14 to the pre-operative volumetric image 44 and guide robot 11 during stages S32-S34 (page 13). Therefore, the update of the pre-operative volumetric image is considered generated according to the matching operation (the complemented reconstruction data which is the reconstruction data complemented with the three-dimensional model)]; and display the complemented reconstruction data on a display device [In practice, if the graph matching of stage S32 (FIG. 2) yields multiple results, then all possible overlays may be displayed to the surgeon whereby the surgeon may select the matching result the surgeon believes is the most likely match via a graphical user interface (page 10)]. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kobayashi’127 according to the teaching of Popovic’225 to update of the pre-operative volumetric image because this will enhance the safety of the surgery operation. With respect to claim 2 , which further limits claim 1, Kobayashi’127 does not teach wherein the least one processor is configured to execute the instructions to display the complemented reconstruction data on the display device in a state where a display mode of a region of the examination target based on the three-dimensional model is different from a display mode of a region of the examination target based on the reconstruction data. Popovic’225 teaches wherein the least one processor is configured to execute the instructions to display the complemented reconstruction data on the display device in a state where a display mode of a region of the examination target based on the three-dimensional model is different from a display mode of a region of the examination target based on the reconstruction data [In practice, if the graph matching of stage S32 (FIG. 2) yields multiple results, then all possible overlays may be displayed to the surgeon whereby the surgeon may select the matching result the surgeon believes is the most likely match via a graphical user interface (page 10)]. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kobayashi’127 according to the teaching of Popovic’225 to update of the pre-operative volumetric image and then to display it because this will enhance the safety of the surgery operation. With respect to claim 3 , which further limits claim 1, Kobayashi’127 does not teach wherein the least one processor is configured to execute the instructions complement means is configured to generate the complemented reconstruction data obtained by adding the three-dimensional model corresponding to a region of the examination target which is not shown in the endoscopic images to the reconstruction data. Popovic’225 teaches wherein the least one processor is configured to execute the instructions complement means is configured to generate the complemented reconstruction data obtained by adding the three-dimensional model corresponding to a region of the examination target which is not shown in the endoscopic images to the reconstruction data [In practice, if the graph matching of stage S32 (FIG. 2) yields multiple results, then all possible overlays may be displayed to the surgeon whereby the surgeon may select the matching result the surgeon believes is the most likely match via a graphical user interface (page 10)]. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kobayashi’127 according to the teaching of Popovic’225 to update of the pre-operative volumetric image and then to display it because this will enhance the safety of the surgery operation. With respect to claim 8 , which further limits claim 1, Kobayashi’127 teaches wherein the three-dimensional model is data generated based on scan data of the examination target obtained in a preliminary examination conducted prior to an examination using the endoscope (Fig.3). With respect to claim 9 , which further limits claim 1, Kobayashi’127 teaches wherein the least one processor is configured to execute the instructions to display a position and a photographed range of the endoscope, on an image representing the complemented reconstruction data (Fig.5, step S15). With respect to claim 10 , which further limits claim 1, Kobayashi’127 teaches wherein the least one processor is configured to execute the instructions to display information indicating a part of interest to be noticed in the examination target, on an image representing the complemented reconstruction data (Fig.5, step S15). With respect to claim 11 , it is a method claim that claims how an image processing device of claim 1 to process an image acquired in endoscopic examination. Claim 1 is obvious in view of Kobayashi’127 and Popovic’225 because the claimed combination operates at the same manner as described in the rejected claim 1. In addition, the reference has disclosed an image processing device to process an image acquired in endoscopic examination, the process (method) to process an image acquired in endoscopic examination is inherent disclosed to be performed by a processor in the image processing device when the image processing device performs the operation to process an image acquired in endoscopic examination. With respect to claim 12 , it is a claim regarding to a non-transitory computer-readable storage medium storing thereon a computer program. Claim 12 claims how the image processing device of claim 1 to execute to process an image acquired in endoscopic examination. Claim 12 is obvious in view of Kobayashi’127 and Popovic’225 because the claimed combination operates at the same manner as described in the rejected claim 1. In addition, the reference discloses a process, the process would be implemented by a processor that requires a non-transitory computer readable medium, e.g., a RAM, to function, thus, the medium is inherently present 07-21-aia AIA Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Kobayashi’127 (US 12,266,127), Popovic’225 (WO 2013061225) and further in view of Tewfik’521 (US 2011/0044521). With respect to claim 4 , which further limits claim 1, the combination of Kobayashi’127 and Popovic’225 does not teach wherein the least one processor is configured to execute the instructions to generate, based on the three-dimensional model, data for patching a hole in the reconstruction data of the examination target, and generate the complemented reconstruction data by adding the data to the reconstruction data. Tewfik’521 teaches wherein the least one processor is configured to execute the instructions to generate, based on the three-dimensional model, data for patching a hole in the reconstruction data of the examination target [the holes on the sample organ surface are being patched (Fig.2C)], and generate the complemented reconstruction data by adding the data to the reconstruction data [as shown in Fig.2C, when the entire surface is being reconstructed with patching the holes, data associated with patching the holes is considered being added to the reconstructed data]. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Kobayashi’127 and Popovic’225 according to the teaching of Tewfik’521 to patch a hole in the reconstruction data of the examination target because this will restore critical anatomical details that might otherwise be lost, leading to better diagnosis and treatment planning . 07-21-aia AIA Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Kobayashi’127 (US 12,266,127), Popovic’225 (WO 2013061225), Tewfik’521 (US 2011/0044521) and further in view of Liu’226 (US 2015/0082226) . With respect to claim 5 , which further limits claim 4, the combination of Kobayashi’127, Popovic’225 and Tewfik’521 does not teach wherein the least one processor is configured to execute the instructions to display information regarding a non-examined region of the examination target based on the region of the examination target corresponding to the hole. Liu’226 teaches wherein the least one processor is configured to execute the instructions to display information regarding a non-examined region of the examination target based on the region of the examination target corresponding to the hole [as shown in Fig.2, the graphical user interface displays a virtual device model 202, an internal organ view 204, and an external organ view 206. In certain example embodiments, the internal organ view 204 and the external organ view 206 are generated from a virtual organ model. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to recognize to display the information regarding a non-examined region of the examination target based on the region of the examination target corresponding to the hole on the graphical user interface because this will enhance the safety of the surgery operation.]. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Kobayashi’127, Popovic’225 and Tewfik’521 according to the teaching of Liu’226 to include a graphical user interface to display both of the external organ and the internal organ because this will enhance the safety of the surgery operation . 07-21-aia AIA Claim s 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Kobayashi’127 (US 12,266,127), Popovic’225 (WO 2013061225) and further in view of Yoshida’917 (US 2016/0004917). With respect to claim 6 , which further limits claim 1, the combination of Kobayashi’127 and Popovic’225 does not teach wherein the least one processor is configured to execute the instructions to display, together with the complemented reconstruction data, information regarding another organ adjacent to the examination target on the display device. Yoshida’917 teaches wherein the least one processor is configured to execute the instructions to display, together with the complemented reconstruction data, information regarding another organ adjacent to the examination target on the display device (Fig.20). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Kobayashi’127 and Popovic’225 according to the teaching of Liu’226 to include a graphical user interface to display both of the organ and its nearby organs because this will enhance the safety of the surgery operation. With respect to claim 7 , which further limits claim 6, the combination of Kobayashi’127 and Popovic’225 does not teach wherein the least one processor is configured to execute the instructions to highlight a region of the examination target adjacent to the other organ, on an image representing the complemented reconstruction data. Yoshida’917 teaches wherein the least one processor is configured to execute the instructions to highlight a region of the examination target adjacent to the other organ, on an image representing the complemented reconstruction data [As shown in Fig.20, the focus N 1 a is considered as the highlight a region of the examination target adjacent to the other organs]. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Kobayashi’127 and Popovic’225 according to the teaching of Liu’226 to include a graphical user interface to display both of the organ and its nearby organs because this will enhance the safety of the surgery operation. Contact Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUO LONG CHEN whose telephone number is (571)270-3759. The examiner can normally be reached on M-F 9am - 5pm. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tieu, Benny can be reached on (571) 272-7490. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /HUO LONG CHEN/Primary Examiner, Art Unit 2682 Application/Control Number: 18/729,113 Page 2 Art Unit: 2682 Application/Control Number: 18/729,113 Page 3 Art Unit: 2682 Application/Control Number: 18/729,113 Page 4 Art Unit: 2682 Application/Control Number: 18/729,113 Page 5 Art Unit: 2682 Application/Control Number: 18/729,113 Page 6 Art Unit: 2682 Application/Control Number: 18/729,113 Page 7 Art Unit: 2682 Application/Control Number: 18/729,113 Page 8 Art Unit: 2682 Application/Control Number: 18/729,113 Page 9 Art Unit: 2682 Application/Control Number: 18/729,113 Page 10 Art Unit: 2682 Application/Control Number: 18/729,113 Page 11 Art Unit: 2682 Application/Control Number: 18/729,113 Page 12 Art Unit: 2682 Application/Control Number: 18/729,113 Page 13 Art Unit: 2682 Application/Control Number: 18/729,113 Page 14 Art Unit: 2682 Application/Control Number: 18/729,113 Page 15 Art Unit: 2682
Prosecution Insights