DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 3-14 and 21 are objected to because of the following informalities:
In claims 3-13, “Claim 1” should read “Claim 2”.
In claims 14 and 21, “at least one frame with high image quality”, in the end of the claim, should read “the at least one frame with high image quality”.
Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 2- 21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3-10, 12, 14, 16-19, 20, 41-43, and 48 of U.S. Patent No. 12061257.
Although the claims at issue are not identical, they are not patentably distinct from each other because Claim 2 of the instant application correspond to the parent claim 1. In particular, claim 1 of the ‘257 patent recites the “medical ultrasound imaging method comprising:
a) forming a frame of an ultrasound image sequence with high image quality;
b) forming a frame of the ultrasound image sequence with reduced image quality;
c) improving the quality of the frame of the ultrasound image sequence with reduced image quality by applying a machine learning model designed to restore a frame of the ultrasound image sequence with reduced image quality by predicting a restored image based on a frame of the ultrasound image sequence with reduced image quality and two anchor frames that are closest in time to the frame of the ultrasound image sequence with reduced image quality, the two anchor frames being of high image quality; and
d) repeating steps a) through c) until the medical ultrasound imaging is completed;
wherein the frames are formed using a transducer.”
Other clams correspond to each other as follows.
Claims 6 and 11 of the instant application correspond to claim 1 of U.S. Patent No. 12061257.
Claim 3 of the instant application correspond to claim 3 of U.S. Patent No. 12061257.
Claim 4 of the instant application correspond to claim 5 of U.S. Patent No. 12061257.
Claim 5 of the instant application correspond to claim 6 of U.S. Patent No. 12061257.
Claims 7-10 and 12 of the instant application correspond to claims 7-10 and 12 of U.S. Patent No. 12061257, respectively.
Claims 13-14 and 20 of the instant application correspond to claim 20 of U.S. Patent No. 12061257.
Claim 15 of the instant application corresponds to claim 14 of U.S. Patent No. 12061257.
Claim 16 of the instant application corresponds to claim 16 of U.S. Patent No. 12061257.
Claim 17 of the instant application corresponds to claim 41 of U.S. Patent No. 12061257.
Claim 18 of the instant application corresponds to claim 42 of U.S. Patent No. 12061257.
Claim 19 of the instant application corresponds to claim 43 of U.S. Patent No. 12061257.
Claim 21 of the instant application corresponds to claim 48 of U.S. Patent No. 12061257.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-13 and 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 2 recites the "enhancing image quality of a comprising”. For examination purposes, Examiner of record takes this to be the “enhancing image quality of ultrasound images comprising”.
Claim 6 recites the "wherein the plurality of frames are closest in time to the lower-quality frame”. This recitation makes the claim indefinite because there is not frame of reference for being the “closest in time”. Because no other frames have been recited, the plurality of frames are automatically closest in time even if the respective points in time are distant. For examination purposes, Examiner of record takes this to be “wherein the plurality of frames are comprised in a sequence of frames, wherein the plurality of frames are closest in time to the lower-quality frame as compared to other frames in the sequence”.
Claim 21 recites the "ultrasound imaging device” in line 5. The relationship between this limitation and the “ultrasound transducer” in line 2 is unclear. For examination purposes, Examiner of record takes this to be the “ultrasound imaging device comprising the ultrasound transducer”.
Claims dependent upon the rejected claims above, but not directly addressed, are also rejected because they inherit the indefiniteness of the claim(s) they respectively depend upon.
Claim Rejections - 35 USC § 103
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-4 and 7-13 are rejected under 35 U.S.C. 103 as being unpatentable over Ogino et al (US 20200175675), hereinafter Ogino, in view of Miller et al (US 6669638), hereinafter Miller, and Jancsary et al (US 20150030237), hereinafter Jancsary.
Regarding claim 2, Ogino teaches a medical ultrasound imaging method for enhancing image quality of ultrasound images (“FIG. 16 is a diagram showing the overall configuration of the medical imaging device (an ultrasonic imaging device)” [0028]; “the image processing using the learned CNN" [0091]) comprising:
forming a plurality of images (51, [0042], Fig. 5) of an ultrasound image sequence each having a high image quality (a sequence of “correct images" [0043] seen in Fig. 5 containing the “correct image 51” [0042], “high-quality image" [0059]; “the image obtained … without undersampling or … where the device is free from the restrictions." [0100]) with high image quality (“Here, as a pair of learning images, as shown in FIG. 5, a learning dataset 50 including a pair of a correct image 51 and a test image 52 with a degraded image quality generated based on the correct image is used.” [0042]; “correct images" [0043]. “Thus, the image data in which only the ROI is converted to a high-quality image is obtained." [0059]; “the ultrasonic image captured under conditions that generate the highest image quality without restrictions such as the imaging time, or the ultrasonic image captured using a high-specification model,” [0087]. “The reconstruction unit 700 includes an ultrasonic image generator 730 that generates an ultrasonic image such as a B-mode image and an M-mode image, and an image processing unit 750 having a function corresponding to the image processing unit 200 of the first embodiment. The reconstruction unit 700 may further include a Doppler processing unit” [0084]; “the image obtained under conditions without undersampling or under conditions where the device is free from the restrictions." [0100]);
forming a lower-quality image (52, [0042], Fig. 5) of the ultrasound image sequence with reduced image quality (“a test image 52 with a degraded image quality generated based on the correct image is used. .. The image quality of the test image is adjusted to be degraded depending on the imaging method (undersampling method) in the imaging unit 100." [0042]; “the ultrasonic
image that has undergone a predetermined degradation process” [0087]; “the case of improving the image quality degraded due to imaging conditions such as frequency and device restrictions such as the element pitch has been described. The present embodiment can also be applied to the data that has been undersampled by controlling a driving method of a number of elements, for example, by thinning the elements at random, the data that has been captured at a low frequency at the expense of resolution in order to increase the depth, or the like.” [0090]);
forming an improved-quality image of the ultrasound image sequence using the lower-quality image and the plurality of images (“The image processing unit 750 improves the image quality by applying the restorer 240 to the ultrasonic image captured under such restrictions. For example, the ultrasonic image captured under conditions that generate the highest image quality without restrictions such as the imaging time, or the ultrasonic image captured using a high-specification model, and the ultrasonic image that has undergone a predetermined degradation process are used as the learning data, and are clustered, to prepare the restorer 240 trained for each cluster.” [0087]; “the image processing unit 750 first receives the ultrasonic image created by the ultrasound image generator 730, performs patch processing and clustering, and then selects the restorer (CNN) applied to each cluster. Each CNN receives the data of the patch and outputs the data with improved image quality. Finally, the data from each CNN is integrated into the ultrasonic image.” [0088], Fig. 16).
While teaching using the lower-quality image and a high-quality image (“the ultrasonic image captured under conditions that generate the highest image quality without restrictions” [0087]), Ogino does not explicitly teach that images are frames and using the lower-quality image and the plurality of images.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches forming frames (“In 2D mode, one sweep from left to right is a frame, and the number of sweeps in a second is the frame rate (or fps--frames per second). Conventional frame rates ranges from about 12 fps to about 30 fps." Col. 2, l. 31-35), the ultrasound image sequence (“The 2D Mode is two-dimensional (imaging) mode, where B Mode is spatially applied by sweeping the beam (as described above) so that structures are seen as a function of depth and width.” Col. 3, l. 18-22) comprising a plurality of frames (“Other system parameters to display include... the frame rate" Col. 11, l. 4-6).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to form images that are frames, as claimed, as taught by Miller, in order to facilitate ultrasonic imaging of the regions of interest.
Ogino modified by Miller does not teach using the lower-quality image and the plurality of images.
However, in the image restoration field of endeavor, Jancsary discloses image restoration cascade, which is analogous art. Jancsary teaches using the lower-quality image (200) and the plurality of images (“natural images” [0029]; “empirical images and/or synthesized images” [0051]) (“A poor quality image 200 such as a digital photograph, medical image, depth image, or any other two or higher dimensional image is provided as input to a trained machine learning predictor 202. The image 200 is of poor quality because it has been subject to one or more sources of noise as described above. The trained machine learning predictor 202 comprises statistical models of natural images and imaging systems for removing the effects of noise sources and it outputs a restored image 204 and optionally also, certainty information associated with the restored image.” [0029]; “the machine learning predictors are trained using training data 800 comprising pairs of poor quality images and corresponding high quality restored images. The training data may comprise empirical images and/or synthesized images and the particular training data used depends on the particular task for which the image restoration system is to be used (e.g. de-blurring, de-noising or other). The training data may be partitioned by a partitioning system 802 into training data sets 804, 806, 808 so that each machine learning predictor in a given cascade may be trained on a different training data set.” [0051]; Figs. 1-8).
Therefore, based on Jancsary’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the combined invention of Ogino and Miller to use the lower-quality image and the plurality of images, as taught by Jancsary, in order to correct existing distortions in the medical image data. In the combined the invention of Ogino, Miller, and Jancsary, the images are frames.
Regarding claim 3, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino does not explicitly teach that the image quality comprises a spatial resolution or a contrast resolution.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches the image quality comprises a spatial resolution or a contrast resolution (“Harmonic imaging has a number of advantages. The beam formed at the harmonic frequency is narrower and has lower side-lobes, thereby significantly improving grayscale contrast resolution.” Col. 4, l. 16-21.).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to have the image quality that comprises a spatial resolution or a contrast resolution, as claimed, as taught by Miller, in order to facilitate ultrasonic imaging of the regions of interest.
Regarding claim 4, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino does not explicitly teach that the image quality comprises a signal- to- noise ratio.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches the image quality comprises a signal- to- noise ratio (“Maximizing the acoustic intensity… maximizing the signal to noise ratio” Col. 4, l. 48-53) (“It is desirable for the ultrasonic system to operate at the highest frequency (for the reasons discussed above) and at the maximum acoustic intensity. Maximizing the acoustic intensity increases imaging performance by increasing the depth penetration and maximizing the signal to noise ratio (SNR).” Col. 4, l. 48-53).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to have the image quality that comprises a signal- to- noise ratio, as claimed, as taught by Miller, in order to facilitate ultrasonic imaging of the regions of interest.
Regarding claim 7, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino does not explicitly teach that each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame is a 2-dimensional image.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches that each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame is a 2-dimensional image (“In 2D mode, one sweep from left to right is a frame, and the number of sweeps in a second is the frame rate (or fps--frames per second). Conventional frame rates ranges from about 12 fps to about 30 fps." Col. 2, l. 31-35) (“The 2D Mode is two-dimensional (imaging) mode, where B Mode is spatially applied by sweeping the beam (as described above) so that structures are seen as a function of depth and width.” Col. 3, l. 18-22).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to have each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame is a 2-dimensional image, as taught by Miller, in order to image structures are seen as a function of depth and width (Miller: Col. 3, l. 18-22).
Regarding claim 8, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino does not explicitly teach that each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame is a 3-dimensional image.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches that each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame is a 3-dimensional image (“although 3D imaging is not explicitly discussed, the present invention could be easily applied to 3D imaging.”; Col. 12, l. 13-16).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to have each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame that is a 3-dimensional image, as taught by Miller, in order to facilitate imaging of sample volumes (Miller: Col. 4, l. 40-42).
Regarding claim 9, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino teaches that each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame is a B-Mode image (“The reconstruction unit 700 includes an ultrasonic image generator 730 that generates an ultrasonic image such as a B-mode image …, and an image processing unit 750 having a function corresponding to the image processing unit 200 of the first embodiment.” [0084], Fig. 16).
Regarding claim 10, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino does not explicitly teach that each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame is a color Doppler image or a spectral Doppler strip.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches that each frame is a color Doppler image (“Pulsed wave Doppler effect techniques have proven to be very accurate in blood flow studies. However, if the velocity of the blood flow being measured exceeds the Nyquist Limit (half the PRF), the ultrasonic readings become inaccurate. Most Doppler techniques try to achieve a high a PRF as possible in order to avoid this effect. One type of imaging, Color Flow Imaging or CFI, uses this effect (called "aliasing") to detect flow disturbances, e.g., transitions from laminar to turbulent flow. In CFI, multiple sample volumes are detected and displayed utilizing color mapping for direction and velocity flow data.” Col. 4, l. 32-47).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to have each frame of the plurality of frames of the ultrasound image sequence and the lower-quality frame that is a color Doppler image, as taught by Miller, in order to image direction and velocity flow data to facilitate blood flow studies (Miller: Col. 4, l. 32-47).
Regarding claim 11, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino teaches using an ultrasound transducer to form the plurality of frames and to form the lower-quality frame ("the ultrasonic probe 711 includes a large number of transducers arranged in a one-dimensional direction or a two-dimensional direction, and repeats transmission and reception of the ultrasonic wave while electronically switching the transducers at high speed. Resolution and artifacts in ultrasonic imaging are affected by probe frequency, device transmission/reception conditions, transducer element pitch, and the like." [0086], Fig. 16).
Regarding claim 12, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino teaches that the forming the lower-quality frame of the ultrasound image sequence with the reduced image quality comprises forming the lower-quality frame of the ultrasound image sequence with one or more of:
low spatial sampling incurred by using a reduced number of transducer elements;
low temporal sampling incurred by using a low temporal sampling rate; low spatial frequency sampling; using temporal delay quantization during a beamforming process; omitting phase aberration correction; omitting aperture coherence function-based imaging techniques; sending a reduced number of transmissions; increasing line spacing; or reducing ensemble length (“the case of improving the image quality degraded due to imaging conditions such as frequency and device restrictions such as the element pitch has been described. The present embodiment can also be applied to the data that has been undersampled by controlling a driving method of a number of elements, for example, by thinning the elements at random” [0090]).
Regarding claim 13, Ogino modified by Miller and Jancsary teaches the method of claim 2.
Ogino modified by Miller does not teach predicting a restored image based on the plurality of frames and the lower-quality frame.
However, in the image restoration field of endeavor, Jancsary discloses image restoration cascade, which is analogous art. Jancsary teaches predicting a restored image based on the plurality of frames (“natural images” [0029]; “empirical images and/or synthesized images” [0051]) and the lower-quality frame (200) (“A poor quality image 200 such as a digital photograph, medical image, depth image, or any other two or higher dimensional image is provided as input to a trained machine learning predictor 202. The image 200 is of poor quality because it has been subject to one or more sources of noise as described above. The trained machine learning predictor 202 comprises statistical models of natural images and imaging systems for removing the effects of noise sources and it outputs a restored image 204 and optionally also, certainty information associated with the restored image.” [0029]; “the machine learning predictors are trained using training data 800 comprising pairs of poor quality images and corresponding high quality restored images. The training data may comprise empirical images and/or synthesized images and the particular training data used depends on the particular task for which the image restoration system is to be used (e.g. de-blurring, de-noising or other). The training data may be partitioned by a partitioning system 802 into training data sets 804, 806, 808 so that each machine learning predictor in a given cascade may be trained on a different training data set.” [0051]; Figs. 1-8).
Therefore, based on Jancsary’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the combined invention of Ogino and Miller to have the step of predicting a restored image based on the plurality of frames and the lower-quality frame, as taught by Jancsary, in order to correct existing distortions in the medical image data. In the combined the invention of Ogino, Miller, and Jancsary, the images are frames.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Ogino, Miller, and Jancsary as applied to claims 2, and further in view of Wegner (US 20150080725), hereinafter Wegner.
Regarding claim 5, Ogino modified by Miller teaches the method of claim 2.
Ogino modified by Miller and Jancsary does not teach that the image quality comprises a signal dynamic range.
However, in the image formation field of endeavor, Wegner discloses coherent spread-spectrum coded waveforms in synthetic aperture image formation, which is analogous art. Wegner teaches that the image quality comprises a signal dynamic range ("The A/D Converter Module 213 can include A/D converters that have … spurious-free dynamic range (SFDR)" [0052]).
Therefore, based on Wegner’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the combined invention of Ogino, Miller, and Jancsary to have the image quality that comprises a signal dynamic range, as taught by Wegner, in order to correct existing distortions in the medical image data.
Claims 14, 16-17, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Ogino et al (US 20200175675), hereinafter Ogino, in view of Miller et al (US 6669638), hereinafter Miller.
Regarding claim 14, Ogino teaches a medical ultrasound imaging system (70, “FIG. 16 is a diagram showing the overall configuration of the medical imaging device (an ultrasonic imaging device)” [0028]; “the image processing using the learned CNN" [0091]) comprising:
a medical ultrasound imaging device (710, Fig. 16; “the ultrasonic imaging unit 710 includes an ultrasonic probe 711 that transmits an ultrasonic wave, a transmitting unit 712 that transmits an ultrasonic drive signal to the probe 711, an ultrasonic receiving unit 713 that receives an ultrasonic signal (RF signal) from the probe 711, a phasing addition unit 715 that performs phasing addition (beam forming) on the signal received by the ultrasonic receiving unit 713, and an ultrasonic transmission/reception control unit 714 that controlls the ultrasonic transmitting unit 712 and the ultrasonic receiving unit 713” [0083]) comprising an ultrasound transducer ("the ultrasonic probe 711 includes a large number of transducers arranged in a
one-dimensional direction or a two-dimensional direction, and repeats transmission and reception of the ultrasonic wave while electronically switching the transducers at high speed. Resolution and artifacts in ultrasonic imaging are affected by probe frequency, device
transmission/reception conditions, transducer element pitch, and the like." [0086], Fig. 16); and
at least one processor (“As schematically shown in FIG. 4, the CNN is a calculation unit constructed on a computer configured to repeat a large number of convolution operations 42 and pooling 43 on a multilayer network between an input layer 41 and an output layer 44.” [0041]; “the ultrasonic transmission/reception control unit 714 and the reconstruction unit 700 are built in one CPU…The ultrasonic transmission/reception control unit 714 may be built in a CPU different from the reconstruction unit 700, and may be a combination of hardware such as a transmission/reception circuit and control software." [0084]. "The image processing unit 750 improves the image quality by applying the restorer 240 to the ultrasonic image captured under such restrictions." [0087], Fig. 16);
wherein the system configured to:
generate, using the ultrasound transducer, a plurality of images (51, [0042], Fig. 5) of an ultrasound image sequence (a sequence of “correct images" [0043] seen in Fig. 5 containing the “correct image 51” [0042], “high-quality image" [0059]; “the image obtained … without undersampling or … where the device is free from the restrictions." [0100]), the plurality of images including at least one image with high image quality (51) (“a correct image 51” [0042]; “correct images" [0043]. “the ultrasonic image captured under conditions that generate the highest image quality without restrictions such as the imaging time, or the ultrasonic image captured using a high-specification model,” [0087]; “the image obtained under conditions without undersampling or under conditions where the device is free from the restrictions." [0100]), wherein the reduced image quality results from one or more techniques configured to reduce power consumption, size, or cost of the medical ultrasound imaging device (“The image quality of the test image is adjusted to be degraded depending on the imaging method (undersampling method) in the imaging unit 100." [0042]; “the ultrasonic image that has undergone a predetermined degradation process” [0087]; “the case of improving the image quality degraded due to imaging conditions such as frequency and device restrictions such as the element pitch has been described. The present embodiment can also be applied to the data that has been undersampled by controlling a driving method of a number of elements, for example, by thinning the elements at random, the data that has been captured at a low frequency at the expense of resolution in order to increase the depth, or the like.” [0090]); and
improve, using the at least one processor, the quality of the at least one image with reduced image quality by applying a machine learning model (240) that predicts a restored image based on the at least one image with reduced image quality (52) (“a test image 52 with a degraded image quality generated based on the correct image is used... The image quality of the test image is adjusted to be degraded depending on the imaging method (undersampling method) in the imaging unit 100." [0042]; “the ultrasonic image that has undergone a predetermined degradation process” [0087]) and at least one image with high image quality (“correct image 51” [0042], “high-quality image" [0059]) (“The image processing unit 750 improves the image quality by applying the restorer 240 to the ultrasonic image captured under such restrictions. For example, the ultrasonic image captured under conditions that generate the highest image quality without restrictions such as the imaging time, or the ultrasonic image captured using a high-specification model, and the ultrasonic image that has undergone a predetermined degradation process are used as the learning data, and are clustered, to prepare the restorer 240 trained for each cluster.” [0087]; “the image processing unit 750 first receives the ultrasonic image created by the ultrasound image generator 730, performs patch processing and clustering, and then selects the restorer (CNN) applied to each cluster. Each CNN receives the data of the patch and outputs the data with improved image quality. Finally, the data from each CNN is integrated into the ultrasonic image.” [0088], Fig. 16).
Ogino does not explicitly teach that images are frames, the ultrasound image sequence comprising a plurality of frames.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches that images are frames (“In 2D mode, one sweep from left to right is a frame, and the number of sweeps in a second is the frame rate (or fps--frames per second). Conventional frame rates ranges from about 12 fps to about 30 fps." Col. 2, l. 31-35), the ultrasound image sequence comprising a plurality of frames (“The 2D Mode is two-dimensional (imaging) mode, where B Mode is spatially applied by sweeping the beam (as described above) so that structures are seen as a function of depth and width.” Col. 3, l. 18-22) comprising a plurality of frames (“Other system parameters to display include... the frame rate" Col. 11, l. 4-6), wherein each frame is generated at a distinct time point (a time point of the “one sweep from left to right” Col. 2, l. 31-35; “switching is performed based on how much time has elapsed” Col. 6, l. 53-56; “frame rate (the system decreases the frame rate, i.e., number of sweeps per second)” Col. 10, l. 10-12; “alternating frames,” Col. 11, l. 55-59).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to have images that are frames as claimed, as taught by Miller, in order to facilitate ultrasonic imaging of the regions of interest.
Regarding claim 16, Ogino modified by Miller teaches the medical ultrasound imaging system of claim 14.
Ogino teaches that the frames are formed using a transducer, and wherein the transducer comprises a 1.25D, 1.5D, 1.75D, or 2D array of elements ("There are various types of the ultrasonic probe 711. Generally, the ultrasonic probe 711 includes a large number of transducers arranged in … a two-dimensional direction" [0086], Fig. 16. Note that “arranged in …a two dimensional direction” is seen as 2D array of elements).
Regarding claim 17, Ogino modified by Miller teaches the medical ultrasound imaging system of claim 14.
Ogino teaches that the at least one processor comprises an application-specific integrated circuit (ASIC) (“Some functions of the image processing unit 200 can also be implemented by hardware such as an ASIC (Application Specific Integrated Circuit)" [0033], Fig. 2).
Regarding claim 21, Ogino teaches a medical ultrasound imaging method (“the image processing using the learned CNN" [0091]) comprising:
forming, using an ultrasound transducer ("the ultrasonic probe 711 includes a large number of transducers arranged in a one-dimensional direction or a two-dimensional direction, and repeats transmission and reception of the ultrasonic wave while electronically switching the transducers at high speed. Resolution and artifacts in ultrasonic imaging are affected by probe frequency, device transmission/reception conditions, transducer element pitch, and the like." [0086], Fig. 16), a plurality of images of an ultrasound image sequence (a sequence of “correct images" [0043] seen in Fig. 5 containing the “correct image 51” [0042], “high-quality image" [0059]; “the image obtained … without undersampling or … where the device is free from the restrictions." [0100]), the plurality of images including at least one frame with high image quality (51) (“a correct image 51” [0042]; “correct images" [0043]) and at least one image with reduced image quality (52) (“a degraded image quality” [0042], Fig. 5), wherein the reduced image quality results from one or more techniques configured to reduce power consumption, size, or cost of an ultrasound imaging device (100) (711) (“(undersampling method) in the imaging unit 100." [0042]; “an ultrasonic probe 711 that transmits an ultrasonic wave” [0083]. “The present embodiment can also be applied to the data that has been undersampled by controlling a driving method of a number of elements, for example, by thinning the elements at random, the data that has been captured at a low frequency at the expense of resolution in order to increase the depth, or the like.” [0090]); and
improving, using at least one processor (“As schematically shown in FIG. 4, the CNN is a calculation unit constructed on a computer configured to repeat a large number of convolution operations 42 and pooling 43 on a multilayer network between an input layer 41 and an output layer 44.” [0041]; “the ultrasonic transmission/reception control unit 714 and the reconstruction unit 700 are built in one CPU " [0084]; Fig. 16), the quality of the at least one image with reduced image quality by applying a machine learning model (240) that predicts a restored image based on the at least one image with reduced image quality (52) (“a test image 52 with a degraded image quality generated based on the correct image is used... The image quality of the test image is adjusted to be degraded depending on the imaging method (undersampling method) in the imaging unit 100." [0042]; “the ultrasonic image that has undergone a predetermined degradation process” [0087]) and at least one image with high image quality (“correct image 51” [0042], “high-quality image" [0059]) (“The image processing unit 750 improves the image quality by applying the restorer 240 to the ultrasonic image captured under such restrictions. For example, the ultrasonic image captured under conditions that generate the highest image quality without restrictions such as the imaging time, or the ultrasonic image captured using a high-specification model, and the ultrasonic image that has undergone a predetermined degradation process are used as the learning data, and are clustered, to prepare the restorer 240 trained for each cluster.” [0087]; “the image processing unit 750 first receives the ultrasonic image created by the ultrasound image generator 730, performs patch processing and clustering, and then selects the restorer (CNN) applied to each cluster. Each CNN receives the data of the patch and outputs the data with improved image quality. Finally, the data from each CNN is integrated into the ultrasonic image.” [0088], Fig. 16).
Ogino does not explicitly teach that images are frames, the ultrasound image sequence comprising a plurality of frames.
However, in the ultrasound imaging field of endeavor, Miller discloses imaging ultrasound transducer temperature control system and method, which is analogous art. Miller teaches that images are frames (“In 2D mode, one sweep from left to right is a frame, and the number of sweeps in a second is the frame rate (or fps--frames per second). Conventional frame rates ranges from about 12 fps to about 30 fps." Col. 2, l. 31-35), the ultrasound image sequence comprising a plurality of frames (“The 2D Mode is two-dimensional (imaging) mode, where B Mode is spatially applied by sweeping the beam (as described above) so that structures are seen as a function of depth and width.” Col. 3, l. 18-22) comprising a plurality of frames (“Other system parameters to display include... the frame rate" Col. 11, l. 4-6), wherein each frame is generated at a distinct time point (a time point of the “one sweep from left to right” Col. 2, l. 31-35; “switching is performed based on how much time has elapsed” Col. 6, l. 53-56; “frame rate (the system decreases the frame rate, i.e., number of sweeps per second)” Col. 10, l. 10-12; “alternating frames,” Col. 11, l. 55-59).
Therefore, based on Miller’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Ogino to have images that are frames as claimed, as taught by Miller, in order to facilitate ultrasonic imaging of the regions of interest.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Ogino and Miller as applied to claims 14, and further in view of von Ramm et al (US 10605903), hereinafter Von Ramm.
Regarding claim 15, Ogino modified by Miller teaches the medical ultrasound imaging system of claim 14.
Ogino modified by Miller does not teach that the transducer comprises a pMUT device.
However, in the ultrasound imaging field of endeavor, Von Ramm discloses a pMUT array for ultrasonic imaging, and related apparatuses, systems, and methods, which is analogous art. Von Ramm teaches that the transducer comprises a pMUT device (“Piezoelectric Micromachined Ultrasound Transducer ( pMUT) arrays for ultrasonic imaging, and related apparatuses, systems, and methods are disclosed... The plurality of array elements are configured to transmit and receive at least one ultrasound beam based on the predetermined fundamental mode vibration. By sizing the pMUTs to correspond to a desired fundamental mode vibration, the pMUT array has improved sensitivity, and can be produced relatively cheaply compared to conventional dicing methods.” Abstract).
Therefore, based on Von Ramm’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the combined invention of Ogino and Miller to have the transducer that comprises a pMUT device, as taught by Von Ramm, in order to improve sensitivity and produce transducers relatively cheaply (Von Ramm: Abstract).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Ogino and Miller as applied to claims 14, and further in view of Wegner (US 20150080725), hereinafter Wegner.
Regarding claim 18, Ogino modified by Miller teaches the medical ultrasound imaging system of claim 14.
Ogino does not explicitly teach that the at least one processor comprises a mobile computing device in communication with the medical ultrasound imaging device.
However, in the ultrasound imaging field of endeavor, Wegner discloses coherent spread-spectrum coded waveforms in synthetic aperture image formation, which is analogous art. Wegner teaches that the at least one processor comprises a mobile computing device in communication with the medical ultrasound imaging device (“The System Controller 202 can be implemented as one of various data processing architectures, such as a personal computer (PC), laptop, tablet, and mobile communication device architectures.” [0041]; Fig. 2A).
Therefore, based on Wegner’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the combined invention of Ogino and Miller to have the at least one processor that comprises a mobile computing device in communication with the medical ultrasound imaging device, as taught by Wegner, in order to facilitate using various data processing architectures thereby simplifying user interaction with the system (Wegner: [0041]).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Ogino and Miller as applied to claims 14, and further in view of Yamamoto (US 20150201909), hereinafter Yamamoto.
Regarding claim 19, Ogino modified by Miller teaches the medical ultrasound imaging system of claim 14.
Ogino modified by Miller does not teach that the frames with high image quality are formed independent of a motion of the medical ultrasound imaging device.
However, in the ultrasound imaging field of endeavor, Yamamoto discloses ultrasound inspection apparatus, signal processing method for ultrasound inspection apparatus, and recording medium, which is analogous art. Yamamoto teaches that the frames with high image quality (“a high-quality image” [0112]) are formed independent of a motion of the medical ultrasound imaging device (“The ultrasound probe 12 has a transducer array 36" [0046]; Fig. 1; “when the ultrasound probe is moved by the operator, the determination for the image quality based on the preliminary sound velocity will have a negative result, and when the determination for the image quality based on the preliminary sound velocity obtained from a nearby region has a negative result, then the optimal sound velocity is found; thus, a high-quality image can ultimately be obtained.” [0112]).
Therefore, based on Yamamoto’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the combined invention of Ogino and Miller to have the frames with high image quality that are formed independent of a motion of the medical ultrasound imaging device, as taught by Yamamoto, in order to facilitate obtaining high-quality images regardless of whether or not the ultrasound probe is moved by the operator (Yamamoto: [0112]).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Ogino and Miller as applied to claim 14, and further in view of Jancsary et al (US 20150030237), hereinafter Jancsary.
Regarding claim 20, Ogino modified by Miller teaches the medical ultrasound imaging system of claim 14.
Ogino modified by Miller does not teach that the at least one processor is configured to improve the image quality of the lower-quality frame using the plurality of frames by predicting a restored image based on the plurality of frames and the lower- quality frame.
However, in the image restoration field of endeavor, Jancsary discloses image restoration cascade, which is analogous art. Jancsary teaches that the at least one processor (202) is configured to improve the image quality of the lower-quality image (200) using the plurality of images (“natural images” [0029]; “empirical images and/or synthesized images” [0051]) by predicting a restored image based on the plurality of frames and the lower- quality image (“A poor quality image 200 such as a digital photograph, medical image, depth image, or any other two or higher dimensional image is provided as input to a trained machine learning predictor 202. The image 200 is of poor quality because it has been subject to one or more sources of noise as described above. The trained machine learning predictor 202 comprises statistical models of natural images and imaging systems for removing the effects of noise sources and it outputs a restored image 204 and optionally also, certainty information associated with the restored image.” [0029]; “the machine learning predictors are trained using training data 800 comprising pairs of poor quality images and corresponding high quality restored images. The training data may comprise empirical images and/or synthesized images and the particular training data used depends on the particular task for which the image restoration system is to be used (e.g. de-blurring, de-noising or other). The training data may be partitioned by a partitioning system 802 into training data sets 804, 806, 808 so that each machine learning predictor in a given cascade may be trained on a different training data set.” [0051]; Figs. 1-8).
Therefore, based on Jancsary’s teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the combined invention of Ogino and Miller to employ the at least one processor that is configured to improve the image quality of the lower-quality image using the plurality of images by predicting a restored image based on the plurality of frames and the lower- quality image, as taught by Jancsary, in order to correct existing distortions in the medical image data. In the combined the invention of Ogino, Miller, and Jancsary, the images are frames.
Conclusion
Any inquiry concern