DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 10/23/2025 have been fully considered but they are not persuasive. Applicant’s arguments will now be addressed:
First, Applicant argues (Remarks; p. 13, 1st paragraph) that “Menini et al. (Menini et al., US 2014/0355844 A1) and Lang (Lang, US 2021/0137634 A1), either alone or in combination, also do not teach or disclose generating a temporary image based on result data, the provision of which includes application of a first processing function to further data, and providing a result image including applying a second processing function to the further data”. The Examiner respectfully disagrees. Menini teaches generating a temporary data based on result data (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]), the provision of which includes application of a first processing function to further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]), and providing a result including applying a second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]). Menini teaches generating data ([0105-0111]) and wherein an image can be generated for view (Fig. 2; [0119-0121]). However, Menini does not explicitly teach generating a temporary “image” or a result “image”. Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and wherein visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]) (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]) (wherein reconstructions can be applied) ([0257]); and visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the result image (displaying scans and/or images from any time intervals/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]); and wherein a combination of (a) and (b), e.g. automatic registration with manual adjustment option, e.g. by moving the virtual image data in relation to the live image data after image processing software and/or pattern recognition software and/or matching software have identified a potential match or performed an initial matching, which can then be followed by manual/operator based adjustments. Alternatively, manual/operator based matching and registration can be performed first, followed then by fine-tuning via software or algorithm (image processing, pattern recognition, etc.) based matching and registration ([0331]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]). Thus, the combination of prior arts teach “generating a temporary image based on result data, the provision of which includes application of a first processing function to further data, and providing a result image including applying a second processing function to the further data”. The Examiner would also like to point out that the broadest reasonable interpretation of a first and second “processing function” can relate to any processing done to the one or more images. Also, the claim language states “comprising applying the second processing function to the further data, the result data, or the further data and the result data”, and thus, means that the second processing function can be applied to the further data or the result data as claimed.
Applicant also argues (Remarks; p. 13, 2nd paragraph) that “Menini et al. do not, however, teach or disclose acquiring further data that maps the examination object at a second instant after the first instant”. The Examiner respectfully disagrees. Menini teaches acquiring further data (acquiring movement measurements 2) (Fig. 1; [0097-0100]) that maps the examination object (that maps the movement of the anatomical object; such as displacement maps) ([0062-0064], [0067], and [0106]) at a second instant after the first instant (wherein the image reconstruction process uses a model that adapts to the real image acquisition chain (images in time), taking into account the information on the movements) ([0106]). The Examiner would also like to point out that in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]). The Examiner would also like to point out that “map/mapping” based on the broadest reasonable interpretation can just mean having data that corresponds to the “examination object”.
Applicant also argues (Remarks; p. 13, 2nd paragraph to the top of page 14) that “It follows that Menini et al. do not teach or disclose applying a first processing function and a second processing function to such further data to provide result data and a result image, respectively”. The Examiner respectfully disagrees. The Examiner would like to point out that in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Menini teaches generating a temporary data based on result data (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]), the provision of which includes application of a first processing function to further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]), and providing a result including applying a second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]). Menini teaches generating data ([0105-0111]) and wherein an image can be generated for view (Fig. 2; [0119-0121]). However, Menini does not explicitly teach generating a temporary “image” or a result “image”. Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and wherein visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]) (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]) (wherein reconstructions can be applied) ([0257]); and visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the result image (displaying scans and/or images from any time intervals/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]); and wherein a combination of (a) and (b), e.g. automatic registration with manual adjustment option, e.g. by moving the virtual image data in relation to the live image data after image processing software and/or pattern recognition software and/or matching software have identified a potential match or performed an initial matching, which can then be followed by manual/operator based adjustments. Alternatively, manual/operator based matching and registration can be performed first, followed then by fine-tuning via software or algorithm (image processing, pattern recognition, etc.) based matching and registration ([0331]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]). Thus, the combination of prior arts teach “applying a first processing function and a second processing function to such further data to provide result data and a result image, respectively”. The Examiner would also like to point out that the broadest reasonable interpretation of a first and second “processing function” can relate to any processing done to the one or more images. Also, the claim language states “comprising applying the second processing function to the further data, the result data, or the further data and the result data”, and thus, means that the second processing function can be applied to the further data or the result data as claimed.
Applicant also argues (Remarks; top of page 14) that “Accordingly, it follows that Menini et al. do not teach or disclose generating a temporary image based on result data, the provision of which includes application of a first processing function to further data, and providing a result image including applying a second processing function to the further data, as recited by independent claim 1”. The Examiner respectfully disagrees. The Examiner would like to point out that in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Menini teaches generating a temporary data based on result data (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]), the provision of which includes application of a first processing function to further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]), and providing a result including applying a second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]). Menini teaches generating data ([0105-0111]) and wherein an image can be generated for view (Fig. 2; [0119-0121]). However, Menini does not explicitly teach generating a temporary “image” or a result “image”. Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and wherein visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]) (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]) (wherein reconstructions can be applied) ([0257]); and visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the result image (displaying scans and/or images from any time intervals/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]); and wherein a combination of (a) and (b), e.g. automatic registration with manual adjustment option, e.g. by moving the virtual image data in relation to the live image data after image processing software and/or pattern recognition software and/or matching software have identified a potential match or performed an initial matching, which can then be followed by manual/operator based adjustments. Alternatively, manual/operator based matching and registration can be performed first, followed then by fine-tuning via software or algorithm (image processing, pattern recognition, etc.) based matching and registration ([0331]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]). Thus, the combination of prior arts teach “generating a temporary image based on result data, the provision of which includes application of a first processing function to further data, and providing a result image including applying a second processing function to the further data”. The Examiner would also like to point out that the broadest reasonable interpretation of a first and second “processing function” can relate to any processing done to the one or more images. Also, the claim language states “comprising applying the second processing function to the further data, the result data, or the further data and the result data”, and thus, means that the second processing function can be applied to the further data or the result data as claimed.
Applicant also argues (Remarks; p. 14, 1st paragraph) that “Menini et al. still do not teach or disclose application of a first processing function to such further data to generate a temporary image and application of a second processing function to such further data to provide a result image, as required by independent claim 1”. Specifically, that “The Office Action cites the movement measurements 2 as being the "further data" required by independent claim 1. The movement measurements 2 are not, however, data mapping an examination object at a timepoint, as required by independent claim 1”. The Examiner respectfully disagrees. The Examiner would like to point out that in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Menini teaches generating a temporary data based on result data (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]), the provision of which includes application of a first processing function to further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]), and providing a result including applying a second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]). Menini teaches generating data ([0105-0111]) and wherein an image can be generated for view (Fig. 2; [0119-0121]). However, Menini does not explicitly teach generating a temporary “image” or a result “image”. Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and wherein visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]) (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]) (wherein reconstructions can be applied) ([0257]); and visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the result image (displaying scans and/or images from any time intervals/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]); and wherein a combination of (a) and (b), e.g. automatic registration with manual adjustment option, e.g. by moving the virtual image data in relation to the live image data after image processing software and/or pattern recognition software and/or matching software have identified a potential match or performed an initial matching, which can then be followed by manual/operator based adjustments. Alternatively, manual/operator based matching and registration can be performed first, followed then by fine-tuning via software or algorithm (image processing, pattern recognition, etc.) based matching and registration ([0331]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]). The argument that the movement measurements of Menini “are not, however, data mapping an examination object at a timepoint” is also not persuasive. First, the broadest reasonable interpretation of “data mapping” can just mean having data that corresponds to the “examination object”. Menini teaches data mapping the examination object (that maps the movement of the anatomical object; such as displacement maps) ([0062-0064], [0067], and [0106]) at a timepoint (wherein the image reconstruction process uses a model that adapts to the real image acquisition chain (images in time), taking into account the information on the movements) ([0106]). The Examiner would also like to point out that in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]).
Applicant also argues (Remarks; p. 14, 1st paragraph) that “Further, Menini et al. do not teach or disclose applying different processing functions to the movement measurements 2”. As discussed above, Menini teaches application of a first processing function to further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]), and providing a result including applying a second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]). The Examiner would also like to point out that the broadest reasonable interpretation of a first and second “processing function” can relate to any processing done to the one or more images. Also, the claim language states “comprising applying the second processing function to the further data, the result data, or the further data and the result data”, and thus, means that the second processing function can be applied to the further data or the result data as claimed.
Applicant also argues (Remarks; top of page 15) that “The simulated experimental data is not, however, an image. Accordingly, Menini et al. do not teach or disclose generating a temporary image based on result data, the provision of which includes application of a first processing function to further data, and providing a result image including applying a second processing function to the further data, as recited by independent claim 1”. The Examiner respectfully disagrees and points out (as above) that in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Menini teaches generating a temporary data based on result data (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]), the provision of which includes application of a first processing function to further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]), and providing a result including applying a second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]). Menini teaches generating data ([0105-0111]) and wherein an image can be generated for view (Fig. 2; [0119-0121]). However, Menini does not explicitly teach generating a temporary “image” or a result “image”. Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and wherein visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]) (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]) (wherein reconstructions can be applied) ([0257]); and visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the result image (displaying scans and/or images from any time intervals/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]); and wherein a combination of (a) and (b), e.g. automatic registration with manual adjustment option, e.g. by moving the virtual image data in relation to the live image data after image processing software and/or pattern recognition software and/or matching software have identified a potential match or performed an initial matching, which can then be followed by manual/operator based adjustments. Alternatively, manual/operator based matching and registration can be performed first, followed then by fine-tuning via software or algorithm (image processing, pattern recognition, etc.) based matching and registration ([0331]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]). Thus, the combination of prior arts teach “generating a temporary image based on result data, the provision of which includes application of a first processing function to further data, and providing a result image including applying a second processing function to the further data”. The Examiner would also like to point out that the broadest reasonable interpretation of a first and second “processing function” can relate to any processing done to the one or more images. Also, the claim language states “comprising applying the second processing function to the further data, the result data, or the further data and the result data”, and thus, means that the second processing function can be applied to the further data or the result data as claimed.
Lastly, Applicant argues (Remarks; p. 15, 1st paragraph) that “Lang does not fill the gaps”. The Examiner respectfully disagrees and points to the arguments above of how the combination of Menini and Lang teach all the limitations within the independent claim 1.
Claims 1-18 are pending; claims 1, 15, and 16 have currently been amended.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 15, and 16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the first data mapping…" and “the further data mapping…”. There is insufficient antecedent basis for this limitation in the claim. The Examiner believes a potential way to overcome this rejection would be to have the claim state “a
Claim 15 recites the limitation "the first data mapping…" and “the further data mapping…”. There is insufficient antecedent basis for this limitation in the claim. The Examiner believes a potential way to overcome this rejection would be to have the claim state “a
Claim 16 recites the limitation "the first data mapping…" and “the further data mapping…”. There is insufficient antecedent basis for this limitation in the claim. The Examiner believes a potential way to overcome this rejection would be to have the claim state “a
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 3-11, and 13-18 are rejected under 35 U.S.C. 103 as being unpatentable over Menini et al., US 2014/0355844 A1 (Menini), and further in view of Lang, US 2021/0137634 A1 (Lang).
Regarding claim 1, Menini teaches a method for generating a temporary (generating temporary data, represented by the reconstruction error that is approximated by the residual 9) (Fig. 1; [0111]), the method comprising:
acquiring first data (acquiring experimental measurements 1) (Fig. 1; [0079]) of an examination object (of a subject) ([0081]) using a medical imaging device (using a Magnetic Resonance Imaging (MRI) system) ([0079] and [0081]), the first data mapping the examination object (that maps the movement of the anatomical object; such as displacement maps) ([0062-0064], [0067], and [0106]) at a first timepoint (wherein the image reconstruction process uses a model that adapts to the real image acquisition chain (images in time), taking into account the information on the movements) ([0106]);
providing at least one initialization image (wherein an image 6 can be created) (Fig. 1; [0083-0084]), the providing of the at least one initialization image comprising applying a first processing function, a second processing function, or the first processing function and the second processing function to the first data (wherein applying an adaptive reconstruction to the MRI measurements 1) (Fig. 1, step 5; [0105-0109]), wherein the first processing function and the second processing function are at least partially different (wherein the first processing function at step 5 is different than the second processing function in step 7) (Fig. 1);
visualizing, by a presentation apparatus, a graphical representation of the at least one initialization image (a visual representation of the reconstructed initialization image 6) (Fig. 1; [0083]);
acquiring further data (acquiring movement measurements 2) (Fig. 1; [0097-0098]) of the examination object using the medical imaging device (wherein the movement measurements can be obtained by the MRI) ([0097-0099]), the further data mapping the examination object (that maps the movement of the anatomical object; such as displacement maps) ([0062-0064], [0067], and [0106]) at a second timepoint, the second timepoint being after the first timepoint (wherein the image reconstruction process uses a model that adapts to the real image acquisition chain (images in time), taking into account the information on the movements; thus, being in real-time the second timepoint would be after the first timepoint) ([0106]);
providing result data, the providing of the result data comprising applying the first processing function to the further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]);
providing a result (providing result info, such as an estimation of the reconstruction error) (Fig. 1; [0111]), the providing of the result comprising applying the second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]), the result data (applying the simulation processing step 7 also to the output of the reconstruction step 5) (Fig. 1), or the further data and the result data, wherein the result data is provided before the result image (wherein the result data is provided from step 5 before the output of step 7) (Fig. 1);
generating the temporary based on the result data and the at least one initialization image (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]);
wherein the acquiring of the further data (acquiring movement measurements 2) (Fig. 1; [0097-0098]), the providing of the result data (applying the reconstruction step 5 to the movement measurements 2; providing result data) (Fig. 1; [0105-0109]), the providing of the result (providing result info, such as an estimation of the reconstruction error) (Fig. 1; [0111]), are executed repeatedly until occurrence of a termination condition (wherein the process is repeated iteratively until optimization occurs from a termination condition) (Fig. 3; [0157] and [0167]), and
wherein the result data (applying the reconstruction step 5 to the movement measurements 2; providing result data) (Fig. 1; [0105-0109]), the result (providing result info, such as an estimation of the reconstruction error) (Fig. 1; [0111]), or the result data (applying the reconstruction step 5 to the movement measurements 2; providing result data) (Fig. 1; [0105-0109]) and the result (providing result info, such as an estimation of the reconstruction error) (Fig. 1; [0111]) are provided as the at least one initialization image during the repeated execution (wherein through the iterative process the result can be used as the initialization) ([0157-0158]).
Menini teaches imaging acquisition techniques such as nuclear magnetic resonance imaging (MRI), computed X-ray tomography scanning and positron emission tomography (PET) ([0057]); and wherein an image can be generated for view (Fig. 2; [0119-0121]).
However, Menini does not explicitly teach generating a temporary “image”, a result “image”, or “generating the temporary image based on the result data and the at least one initialization image; visualizing, by the presentation apparatus, a graphical representation of the temporary image; visualizing, by the presentation apparatus, a graphical representation of the result image; wherein the visualizing of the graphical representation of the at least one initialization image, the visualizing of the graphical representation of the temporary image, and the visualizing of the graphical representation of the result image are successively executed”.
Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and wherein visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]) (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]) (wherein reconstructions can be applied) ([0257]); visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the result image (displaying scans and/or images from any time intervals/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]); wherein the visualizing of the graphical representation of the at least one initialization image (first image from a first time) ([1356]) (wherein reconstructions can be applied) ([0257]), the visualizing of the graphical representation of the temporary image, and the visualizing of the graphical representation of the result image are successively executed (wherein the visualization of all the images/scans can including displaying the images sequentially) ([0099]). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]); and wherein a combination of (a) and (b), e.g. automatic registration with manual adjustment option, e.g. by moving the virtual image data in relation to the live image data after image processing software and/or pattern recognition software and/or matching software have identified a potential match or performed an initial matching, which can then be followed by manual/operator based adjustments. Alternatively, manual/operator based matching and registration can be performed first, followed then by fine-tuning via software or algorithm (image processing, pattern recognition, etc.) based matching and registration ([0331]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]).
Regarding claim 3, Menini teaches an initialization image (wherein an image 6 can be created) (Fig. 1; [0083-0084]); result data (applying the reconstruction step 5 to the movement measurements 2; providing result data) (Fig. 1; [0105-0109]); temporary data (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]); wherein the images are taken at different times that are sufficiently close ([0088-0089]); wherein the physiological movements 2 for the movement model of index i at time t (Fig. 1; [0102-0104]); and wherein the object is an anatomy of the subject ([0067]).
However, Menini does not explicitly teach “wherein the at least one initialization image maps the examination object at an initial instant, wherein the result data maps the examination object at a further instant after the initial instant, and wherein the temporary image is generated such that the temporary image maps the examination object at the further instant or at an instant between the initial instant and the further instant”.
Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); wherein the at least one initialization image maps the examination object (anatomic structure of a patient) ([0005]) at an initial instant (taking an image at a first time segment or time interval) ([1356]), wherein the result data maps the examination object (anatomic structure of a patient) ([0005]) at a further instant after the initial instant (taking an image at a second time segment of time interval) ([1356]), and wherein the temporary image is generated such that the temporary image maps the examination object (anatomic structure of a patient) ([0005]) at the further instant or at an instant between the initial instant and the further instant (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include wherein the temporary image is between two images in time since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]).
Regarding claim 4, Menini teaches wherein the result data maps a change in the examination object with respect to the at least one initialization image (wherein the result data is based on a change in the anatomic object from the initialization image based on physiological movement) ([0016]), wherein generating the temporary comprises determining a movement model characterizing the change using the at least one initialization image or using the at least one initialization image and the result data (wherein generating data that consists of displacements for physiological movements from the initialization image and the result data) (Fig. 1; [0016-0020] and [0110-0113]), and wherein the temporary is generated also based on the movement model (wherein the temporary data is generated based on the movement perturbation model) (Fig. 1; [0110-0113]).
However, Menini doesn’t explicitly teach a temporary “image” or “that the temporary image maps the change at the further instant or at the instant between the initial instant and the further instant”.
Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); wherein generating a temporary image (wherein the temporary image is generated based on interpolation between the first and second images; which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]); and such that the temporary image maps the change at the further instant or at the instant between the initial instant and the further instant (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include wherein the temporary image is between two images in time since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]).
Regarding claim 5, Menini teaches wherein the acquiring of the further data (acquiring movement measurements 2) (Fig. 1; [0097-0098]), the providing of the result data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]), the providing of the result (providing result info, such as an estimation of the reconstruction error) (Fig. 1; [0111]), the generating of the temporary (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]), are repeated at least once (wherein the process is repeated iteratively until optimization occurs from a termination condition) (Fig. 3; [0157] and [0167]), and wherein determining the movement model comprises determining the movement model using previous initialization images and the respectively current result data (wherein generating data that consists of displacements for physiological movements from the initialization image and the result data for modeling the movements in a movement model) (Fig. 1; [0026] and [0096-0112]).
However, Menini does not explicitly teach a result “image”, temporary “image”, or “the visualizing of the graphical representation of the temporary image, and the visualizing of the graphical representation of the result image”.
Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and wherein visualizing a graphical representation of the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]); visualizing a graphical representation of the result image (displaying scans and/or images from any time intervales/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]); and wherein optimization can include being iteratively done ([1032]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]).
Regarding claim 6, Menini teaches further comprising receiving a movement signal that describes a physiological movement of the examination object, a movement of a medical object in the examination object, or the physiological movement of the examination object and the movement of a medical object in the examination object (receiving measurements that describe a plurality of physiological movements of the anatomy) ([0047] and [0067]), and wherein the movement model is also determined using the movement signal (wherein the plurality of movement models are associated with said physiological movements) ([0047]). Lang teaches wherein the movement is based on the examination of anatomy of the subject ([0076]) and/or of a medical instrument ([0096] and [0103]).
Regarding claim 7, Lang teaches wherein the generating of the temporary image comprises generating at least one further temporary image based on the result data and the at least one initialization image (wherein the temporary image is generated based on interpolation between the first and second images; which allows for computation of an intermediate image data set between a first image data set and a second image data set; wherein the interpolation can create one or more additional images/scans) ([1356]), wherein the temporary image and the at least one further temporary image maps the examination object (anatomic structure of a patient) ([0005]) at different instants (wherein the temporary/interpolated images are at different times) ([1356]), the different instants comprising the further instant, the instant between the initial instant and the further instant, or the further instant and the instant between the initial instant and the further instant (comprising the initial time, the further time, and the times in-between the initial and further times) ([1356]), and wherein visualizing the graphical representation of the temporary image also comprises visualizing, by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]), a graphical representation of the at least one further temporary image (wherein displaying scans and/or images can come from any time interval/segment) ([0099]).
Regarding claim 8, Lang teaches wherein the instant at which the temporary image maps the examination object (anatomic structure of a patient) ([0005]) lies between the initial instant and the further instant (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]), and wherein the graphical representations are successively displayed by the presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]) and at an interval corresponding to the respective instant of the mapping (wherein the visualization of all the images/scans can including displaying the images sequentially; wherein the displaying can be based on time intervals) ([0099]).
Regarding claim 9, Menini teaches wherein the first processing function comprises a first image reconstruction, a first artifact reduction, a first movement correction, a first filtering of the first data, the further data, or the first data and the further data, or any combination thereof (wherein the first processing is applying an adaptive reconstruction to the MRI measurements 1) (Fig. 1, step 5; [0105-0109]).
Regarding claim 10, Menini teaches wherein the second processing function comprises: a second image reconstruction; a second artifact reduction; a second movement correction; a second filtering of the first data, the further data, the result data, or any combination thereof; or any combination thereof (wherein the second processing includes reconstruction as well as movement correction based on a reconstruction error) ([0110-0112]). Lang teaches image reconstructions ([0257]), coordinate correction ([0527]), and filtering ([0260]).
Regarding claim 11, Menini teaches wherein generating the temporary comprises weighted, averaging, addition, subtraction, multiplication, interpolation, or any combination thereof of the result data and of the at least one initialization image (generating temporary data based on the difference between the simulated data 7 and the experimental data 1) ([0111]). Lang teaches wherein the temporary image is generated based on interpolation between the first and second images in time ([1356]).
Regarding claim 13, Menini teaches wherein generating the temporary image comprises identifying a portion of the result data (wherein sub-components can be used from the simulated data) (Fig. 1; [0088-0090]), the portion having a deviation with respect to a corresponding portion in the at least one initialization image (the sub-component having a difference with the initialized image) (Fig. 1; [0088-0090] and [0111]), wherein generating the temporary image comprises generating the temporary image based on the at least one initialization image and the portion of the result data (wherein the temporary data is generated based on the difference from the initial image and the simulated data using sub-components of the data) (Fig. 1; [0088-0090] and [0111]). Lang also teaches wherein generating the temporary image comprises identifying a portion of the result data (wherein a portion of the result data can be identified based on segmentation of the data) ([0260-0262]), the portion having a deviation with respect to a corresponding portion in the at least one initialization image (the portion having a difference, based on interpolation, with the initial image) ([0260-0262] and [1356]), wherein generating the temporary image comprises generating the temporary image based on the at least one initialization image and the portion of the result data (wherein the temporary image is generated based on interpolation between the first and second images; which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]).
Regarding claim 14, Menini teaches wherein providing the at least one initialization image (wherein an image 6 can be created) (Fig. 1; [0083-0084]) comprises: providing a first initialization image (wherein an image 6 can be created) (Fig. 1; [0083-0084]), the providing of the first initialization image comprising applying the first processing function to the first data (wherein applying an adaptive reconstruction to the MRI measurements 1) (Fig. 1, step 5; [0105-0109]); and providing a second initialization image, the providing of the second initialization image comprising applying the second processing function to the first data (D1) (providing second image data based on the adaptive reconstruction that adapts to the real image acquisition chain, taking into account the information of the movements) (Fig. 1; [0106]), wherein the temporary image is generated based on the result data, the first initialization image (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]), and the second initialization image (the second initialization image can be used in a subsequent iteration for generating the difference) (Figs. 1 and 3; [0111-0112], [0157], and [0166-0167]), and wherein during the repeated execution, the result data is provided as the first initialization image and the result image is provided as the second initialization image (wherein the method is repeated in an iterative optimization using the estimates from the previous iteration are used to calculate new estimations until termination condition is reached) (Fig. 3; [0157] and [0166-0167]). Lang teaches wherein the images can come from any time or time interval and can be reconstructed ([0099]), [0256-0257], [0293], and [0638]).
Regarding claim 15, Menini teaches a system comprising:
a medical imaging device (medical imaging and the imaging of biological media using a device) ([0001-0002]) (such as using a Magnetic Resonance Imaging (MRI) system) ([0079] and [0081]) configured to acquire first data (acquiring experimental measurements 1) (Fig. 1; [0079]) and further data (acquiring movement measurements 2) (Fig. 1; [0097-0098]), the first data mapping an examination object (that maps the movement of the anatomical object; such as displacement maps) ([0062-0064], [0067], and [0106]) at a first timepoint (wherein the image reconstruction process uses a model that adapts to the real image acquisition chain (images in time), taking into account the information on the movements) ([0106]) and the further data mapping the examination object (that maps the movement of the anatomical object; such as displacement maps) ([0062-0064], [0067], and [0106]) at a second timepoint, the second timepoint being after the first timepoint (wherein the image reconstruction process uses a model that adapts to the real image acquisition chain (images in time), taking into account the information on the movements; thus, being in real-time the second timepoint would be after the first timepoint) ([0106]);
a provisioning unit (MRI or external sensors) ([0098-0100]) configured to:
provide at least one initialization image (wherein an image 6 can be created) (Fig. 1; [0083-0084]), the provision of the at least one initialization image comprising application of a first processing function, a second processing function, or the first processing function and the second processing function to the first data (wherein applying an adaptive reconstruction to the MRI measurements 1) (Fig. 1, step 5; [0105-0109]);
provide result data, the provision of the result data comprising application of the first processing function to the further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]);
provide a result (providing result info, such as an estimation of the reconstruction error) (Fig. 1; [0111]), the provision of the result comprising application of the second processing function to the further data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]), the result data, (applying the simulation processing step 7 also to the output of the reconstruction step 5) (Fig. 1) or the further data and the result data (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]) (applying the simulation processing step 7 also to the output of the reconstruction step 5) (Fig. 1); and
generate a temporary based on the result data and the at least one initialization image (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]).
Menini teaches imaging acquisition techniques such as nuclear magnetic resonance imaging (MRI), computed X-ray tomography scanning and positron emission tomography (PET) ([0057]); and wherein an image can be generated for view (Fig. 2; [0119-0121]).
However, Menini does not explicitly teach a result “image”, a temporary “image” or “a presentation apparatus configured to successively visualize graphical representations of the at least one initialization image, the temporary image, and the result image, respectively”.
Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and a presentation apparatus (displayed by the OHMD and/or a computer monitor) ([0098]) configured to successively visualize graphical representations (wherein the visualization of all the images/scans can including displaying the images sequentially) ([0099]) of the at least one initialization image (first image from a first time) ([1356]) (wherein reconstructions can be applied) ([0257]), the temporary image (displaying scans and/or images from any time intervales/segments; as well as being able to toggle between unsubtracted and/or subtracted images) ([0099] and [1097]) (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set) ([1356]) (wherein reconstructions can be applied) ([0257]), and the result image (displaying scans and/or images from any time intervals/segments) ([0099]) (wherein the image can be reconstructed) ([0256-0257], [0293], and [0638]), respectively (wherein the visualization of all the images/scans can including displaying the images sequentially) ([0099]). Lang also teaches the OHMD can move virtual data or virtual images, e.g. from a pre-operative scan, that can match the movement of the tissues and/or organs during the cardiac and/or respiratory cycle of the patient, e.g. during an intervention ([0098-0100]); in this manner, the computer system can maintain the display of the virtual data or virtual images superimposed onto and/or aligned with the corresponding anatomic structures, tissues and/or organs both inside the physical patient and/or in virtual data acquired, for example, (e.g. in real-time) from the physical patient, e.g. an intra-operative angiogram, run-off or bolus chase study, e.g. displayed by the OHMD and/or a computer monitor ([0098-0100]); and wherein a combination of (a) and (b), e.g. automatic registration with manual adjustment option, e.g. by moving the virtual image data in relation to the live image data after image processing software and/or pattern recognition software and/or matching software have identified a potential match or performed an initial matching, which can then be followed by manual/operator based adjustments. Alternatively, manual/operator based matching and registration can be performed first, followed then by fine-tuning via software or algorithm (image processing, pattern recognition, etc.) based matching and registration ([0331]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Menini to include displaying the different images since it is advantageous for any type of surgery in which the surgeon utilizes pre-operative ultrasound, CT, MRI, SPECT, and/or PET scan data (Lang; [1000]), while being beneficial for more accurate measurement of changes in position or orientation of the surgeon or the patient (Lang; [0291]).
Regarding claim 16, see the rejection made to claim 1, as well as prior art Lang for a non-transitory computer-readable storage medium (storage media) ([0635]) that stores instructions (software) ([0635]) executable by one or more processors (computer including a processor) ([0005] and [0635]), for they teach all the limitations within this claim.
Regarding claim 17, see the rejection made to claim 2, as well as prior art Lang for a non-transitory computer-readable storage medium (storage media) ([0635]) that stores instructions (software) ([0635]) executable by one or more processors (computer including a processor) ([0005] and [0635]), for they teach all the limitations within this claim.
Regarding claim 18, see the rejection made to claim 3, as well as prior art Lang for a non-transitory computer-readable storage medium (storage media) ([0635]) that stores instructions (software) ([0635]) executable by one or more processors (computer including a processor) ([0005] and [0635]), for they teach all the limitations within this claim.
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over Menini et al., US 2014/0355844 A1 (Menini), Lang, US 2021/0137634 A1 (Lang), and further in view of Wang et al., US 2020/0051210 A1 (Wang).
Regarding claim 2, Menini teaches wherein providing the result data comprises applying the first processing function to the further data (applying the reconstruction step 5 to the movement measurements 2) (Fig. 1; [0105-0109]) the second processing function for providing the result image (applying the simulation processing step 7 to the movement measurements 2) (Fig. 1; [0085-0086] and [0111]). Lang teaches taking a vascular 3D image that can include one or more of an ultrasound image, an echocardiogram image, a CT scan image, an MRI scan image, a CT angiogram image, and/or an MR angiogram image (Abstract and [0034]); and (wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals)) ([1356]).
However, neither explicitly teaches that that a first processing has a “first provisioning duration, the first provisioning duration being shorter than a second provisioning duration” of the second processing function.
Wang teaches image processing using a first and second circuit (such as for demosaicing) (Abstract); and wherein a first processing function has a first provisioning duration, the first provisioning duration being shorter than a second provisioning duration of the second processing function (wherein the first processing has a latency lower than a latency of the second processing) ([0057]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of prior arts to include a first processing with a lower latency since it allows the first processed image to be provided to help with the second processing for better image understanding such as image segmentation as well as allow for adjustments for better imaging (Wang; [0057]).
Claim(s) 12 is rejected under 35 U.S.C. 103 as being unpatentable over Menini et al., US 2014/0355844 A1 (Menini), Lang, US 2021/0137634 A1 (Lang), and further in view of Munkberg et al., US 2020/0126191 A1 (Munkberg).
Regarding claim 12, Menini teaches wherein generating the temporary (generating temporary data, represented by the reconstruction error that is approximated by the residual 9) (Fig. 1; [0111]), wherein the input data is based on the at least one initialization image and the result data (generating temporary data, represented by the reconstruction error that is approximated by the residual 9, which corresponds to the difference between the simulated data and the experimental data) (Fig. 1; [0111]). Lang teaches wherein the temporary image is generated based on interpolation between the first and second images (first and second time segments or intervals); which allows for computation of an intermediate image data set between a first image data set and a second image data set ([1356]).
However, neither explicitly teaches “applying a trained function to input data” or “wherein at least one parameter of the trained function is adjusted by a comparison of a training temporary image with a comparison temporary image”.
Munkberg teaches a neural network structure that can be applied to reconstruct image data such as data acquired by medical imaging ([0023]); applying a trained function to input data (training of the temporal adaptive sampling and denoising system 200; which includes sample map estimator neural network 210 and the denoiser neural network model and combiner 220 applied to the input data) (Fig. 2D; [0069]); and wherein at least one parameter of the trained function is adjusted by a comparison of a training temporary image with a comparison temporary image (wherein at least one parameter is adjusted based on the parameter adjustment unit 245; which is based on a comparison between the reconstructed image frames and the target image frames) (Fig. 2G; [0084-0086] and [0093-0094]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of prior arts to include using a trained function for parameter adjustment since it allows for the system to achieve a threshold accuracy for detecting a match (Munkberg; [0093]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J VANCHY JR whose telephone number is (571)270-1193. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J VANCHY JR/Primary Examiner, Art Unit 2666 Michael.Vanchy@uspto.gov