DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant's election with traverse of species A in the reply filed on 07/22/2026 is acknowledged. The traversal is on the ground(s) that species A and species B are not mutually exclusive, but rather different aspects of a single inventive concept directed to the use of annotations associated with a first ultrasound image to control subsequent operation of an ultrasound system. This is not found persuasive because species A claims acquiring and analyzing another ultrasonic image different from the first ultrasonic image, but Species B claims performing measurement based on the first ultrasonic image, so the two species are mutually exclusive.
The requirement is still deemed proper and is therefore made FINAL.
Claim Objections
Claims 3 and 5 are objected to because of the following informalities:
Claim 3, Line 6, recites “the ultrasonic image”, which should be changed to “the first ultrasonic image”.
Claim 5, Line 6, recites “the image recognition algorithm”, which misses antecedent basis and should be changed to “an image recognition algorithm”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 5, Line 3, recites “another ultrasonic image”, and Line 6 recites “the other ultrasonic image”. It is unclear whether the recited “another” ultrasonic image and “the other” ultrasonic image refer to a same image that is different from the recited first ultrasonic image (Line 2) or second ultrasonic image (Lines 11 and 12). For present purposes of examination, Examiner interprets the recited “another” ultrasonic image and “the other” ultrasonic image to be a same image, and the image to be different from either the first or the second ultrasonic image.
Claim 5, Line 7, recites “the region represented by the annotation”, Lines 9-10 recite “the site represented by the annotation”, and further, Claim 2, Line 3, recites “a region of interest”. It is unclear whether the recited “the region” in Claim 5 is the same as the recited “the site” of Claim 5, or as the recited “a region” of Claim 2. For present purposes of examination, the recited “the region represented by the annotation” in Claim 5, Line 7 is interpreted to refer to “a site represented by the annotation”.
Claims 6-7 are also rejected under 35 U.S.C. 112(b) because they inherit the indefiniteness of the claim(s) they respectively depend upon.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6, 11 and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tanigawa (US 20220211353 A1; hereafter Tanigawa).
With regard to Claim 1, Tanigawa discloses an ultrasonic diagnostic device comprising one or more processors (Tanigawa, Para 0028; “The ultrasonic image display system 1 also comprises the processor 7 for controlling the transmit beamformer 3, transmitter 4, receiver 5, and receive beamformer 6.”) for controlling operation of the ultrasonic diagnostic device (Tanigawa, Para 0029; “The processor 7 may control the ultrasonic probe 2 to acquire ultrasound data.”) for acquiring a second ultrasonic image (color Doppler image) of the same examination site as a first ultrasonic image (B-mode image) based on an annotation added to (Tanigawa, Para 0047; “The memory 9 stores therein a learned model trained for a correlation between a B-mode image and an evaluation value for the B-mode image … the evaluation value is the probability that the B-mode image matches an image of a body part of the patient … The body parts of the patient may include the aforementioned mammary gland, neck, lower limbs, thyroid gland, carotid artery, lower-limb veins, and lower-limb arteries …”. The identified body part of interest that matches or correlates with the B-mode image corresponds to the claimed annotation added to the image) the first ultrasonic image (Tanigawa, Para 0007; “The processor also defines a region of interest in the B-mode image, controls the ultrasonic probe to perform a first color Doppler scan for a first color Doppler image on the patient …”. The disclosed B-mode image and color Doppler image correspond to the claimed first and second ultrasonic image, respectively, and both cover the disclosed region of interest).
With regard to Claim 2, Tanigawa discloses the ultrasonic diagnostic device according to Claim 1, and further discloses wherein controlling the operation of the ultrasonic diagnostic device includes at least one of adjusting image quality of the second ultrasonic image and determining a position and a size of a region of interest of the second ultrasonic image (Tanigawa, Para 0051; “at Step S3, the processor 7 determines parameters regarding acquisition of data for a color Doppler image according to the body part of interest identified at Step S2. … The parameters include the number of acoustic lines and transmission frequency in the color Doppler scan …”. The disclosed determining parameters corresponds to the claimed adjusting image quality).
With regard to Claim 3, Tanigawa discloses the ultrasonic diagnostic device according to Claim 2, and further discloses wherein the one or more processors execute determining whether the first ultrasonic image has had an annotation added thereto (Tanigawa, Para 0061; “At Step S12, the processor 7 decides whether or not a body part of interest can be identified based on the evaluation value obtained at Step S11.” If a body part of interest can be identified in an image, such identification is added to the image) and when the first ultrasonic image has had an annotation added thereto, controlling operation of the ultrasonic diagnostic device to acquire the second ultrasonic image of the same examination site as the first ultrasonic image based on the annotation (Tanigawa, Para 0073; “The processor 7 identifies the body part of the patient in the first region R1 based on the first evaluation value. … in the case that there are obtained the first evaluation values such that the probability of an image of the mammary gland is 10%, the probability of an image of the neck is 80%, and the probability of an image of the lower limbs is 10%, the processor 7 identifies the body part of the patient in the first region R1 to be the neck.” The disclosed first evaluations correspond to an identified body part (the neck in this example), so correspond to the claimed annotation) and an image recognition algorithm that analyzes the ultrasonic image (Tanigawa, Para 0078; “The processor 7 identifies the body part of interest using one of the second learned models SLM1 and SLM2 that is regarding the body part identified at Step S31”; Para 0081; “in the case that there are obtained the second evaluation values such that the probability of an image of the thyroid gland is 80% and the probability of an image of the carotid artery is 20%, the processor 7 identifies the body part of interest to be the thyroid gland.” With the body part of patient in an image identified as the neck, the second learned model SLM1 is further used to identify the body part of interest to be the thyroid gland or the carotid artery).
With regard to Claim 4, Tanigawa discloses the ultrasonic diagnostic device according to Claim 3, and further discloses wherein, when the first ultrasonic image does not have an annotation added thereto, the one or more processors control operation of the ultrasonic diagnostic device to acquire the second ultrasonic image of the same examination site as the first ultrasonic image based on the image recognition algorithm (Tanigawa, Para 0061; “At Step S12, the processor 7 decides whether or not a body part of interest can be identified based on the evaluation value obtained at Step S11. … In the case that the body part of interest is not identified, the processing goes to Step S13.”; Para 0062; “At Step S13 … is processing of obtaining an evaluation value using the learned model described earlier …”. These disclosures indicate that when a body part cannot be identified in a larger 1st region, then a learned model is applied to a smaller 2nd region).
With regard to Claim 5, Tanigawa discloses the ultrasonic diagnostic device according to Claim 2, and further discloses wherein:
the ultrasonic diagnostic device acquires the first ultrasonic image by scanning an examination site of a subject (Tanigawa, Para 0085; “at Step S40 … a B-mode image BI is produced and displayed.”), and then acquires another ultrasonic image different from the first ultrasonic image by scanning the same examination site as the examination site again (Tanigawa, Para 0086; “at Step S41, the processor 7 controls the ultrasonic probe 2 to perform a first color Doppler scan for a first color Doppler image on the patient.”. Fig. 10 shows that the disclosed B-mode image and first color Doppler image are on a same examination site); and
the one or more processors execute:
analyzing the other ultrasonic image using the image recognition algorithm (Tanigawa, Para 0090; “at Step S43, the processor 7 identifies a body part of interest serving as a target in the region of interest RO in the patient based on the data for the first color Doppler image CI1 in the second region R2”);
determining whether the region represented by the annotation is detected from the other ultrasonic image by analyzing the other ultrasonic image (Tanigawa, Para 0061; “At Step S12, the processor 7 decides whether or not a body part of interest can be identified based on the evaluation value obtained at Step S11 …”);
determining a position and a size of the region of interest when the site represented by the annotation is detected (Tanigawa, Para 0090; “The body part of interest … is a body part of the patient that is to be a target for producing a second color Doppler image CI2”. The disclosed body part of interest, or region of interest RO, corresponds to the claimed region of interest, and has a position and a size. Fig. 11 shows an example of the second color Doppler image);
determining a value of a parameter of the second ultrasonic image (Tanigawa, Para 0097; “at Step S44, parameters regarding acquisition of data for a second color Doppler image are determined”); and
generating the second ultrasonic image in the region of interest based on the value of the parameter (Tanigawa, Para 0098; “at Step S45, the parameters determined at Step S44 are used to perform a second color Doppler scan, and data for a second color Doppler image is created.”).
With regard to Claim 6, Tanigawa discloses the ultrasonic diagnostic device according to Claim 5, and further discloses wherein the second ultrasonic image is a color Doppler image (Tanigawa, Para 0098; “at Step S45 … data for a second color Doppler image is created.”), and the parameter is a flow velocity parameter (Tanigawa, Para 0051; “The parameters include the number of acoustic lines and transmission frequency in the color Doppler scan …”. The disclosed transmission frequency is a key parameter in calculating flow velocity using the measured Doppler shift, and can be regarded as a flow velocity parameter).
With regard to Claim 11, Tanigawa discloses the ultrasonic diagnostic device according to Claim 1, and further discloses wherein the annotation includes information representing an examination site (Tanigawa, Para 0045; “The processor 7 may also identify the body part of interest …”; Para 0046; “The body part of interest includes a mammary gland, a neck, lower limbs, etc.”).
With regard to Claim 19, Tanigawa discloses the ultrasonic diagnostic device according to Claim 1, and further discloses wherein the first ultrasonic image is a B-mode image, and the second ultrasonic image is a color Doppler image (Tanigawa, Para 0007; “The processor also defines a region of interest in the B-mode image, controls the ultrasonic probe to perform a first color Doppler scan for a first color Doppler image on the patient …”).
With regard to Claim 20, Tanigawa discloses a non-transitory computer readable storage medium in which a command is stored, wherein the command, when executed by one or more processors (Tanigawa, Para 0039; “In the non-transitory storage medium, programs executed by the processor 7 are stored.”), causes the one or more processors to execute controlling an operation of the ultrasonic diagnostic device for acquiring a second ultrasonic image of the same examination site as that of a first ultrasonic image based on an annotation added to the first ultrasonic image (Tanigawa, Para 0044; “… at Step S2, the processor 7 identifies, based on the data for the B-mode image BI, a body part of interest serving as a target for producing a color Doppler image of the patient.” The disclosed B-mode image and color Doppler image correspond to the claimed first and second ultrasonic images, and the identified body part of interest corresponds to the claimed annotation labeled to the first image).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 7 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Tanigawa, in view of Brandl et al (US 20210315538 A1; hereafter Brandl).
With regard to Claim 7, Tanigawa discloses the ultrasonic diagnostic device according to Claim 5, but does not explicitly and clearly disclose wherein, when the image recognition algorithm detects a site other than the examination site represented by the annotation, the image recognition algorithm notifies a user that the annotation may be incorrect.
Brandl in the same field of endeavor discloses wherein, when the image recognition algorithm detects a site other than the examination site represented by the annotation, the image recognition algorithm notifies a user that the annotation may be incorrect (Brandl, Para 0047; “… if the vessel actually being imaged is different than the vessel indicated in the indication of the vessel being imaged received at 315, method 300 may identify an error based on a comparison of the indication of the vessel output by the machine learning model and the indication of the vessel received at 315. In such examples, method 300 may optionally display a notification requesting confirmation of the vessel being imaged.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanigawa, as suggested by Brandl, in order to notify a user of potentially incorrectly annotated region of interest. One of ordinary skill in the art would have been motivated to make the modification for the benefit of improving diagnostic accuracy by verifying the region to be imaged by a human operator.
With regard to Claim 10, Tanigawa discloses the ultrasonic diagnostic device according to Claim 3, but does not explicitly and clearly disclose wherein the image recognition algorithm uses a deep learning model.
Brandl in the same field of endeavor discloses wherein the image recognition algorithm uses a deep learning model (Brandl, Para 0044; “… method 300 may perform the image classification with a trained convolutional neural network, as an illustrative example, or another suitable image classification technique, such as segmenting the vessel in the ultrasound image and comparing the segmented vessel to a dictionary of vessel segments to identify the vessel.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanigawa, as suggested by Brandl, in order to use a deep learning model for image recognition. One of ordinary skill in the art would have been motivated to make the modification for the benefit of detecting or classifying objects in images with high reliability.
Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Tanigawa, in view of Maack (US 20220301690 A1; hereafter Maack).
With regard to Claim 8, Tanigawa discloses the ultrasonic diagnostic device according to Claim 3, but does not explicitly and clearly disclose wherein the one or more processors execute identification of a meaning of the annotation.
Maack in the same field of endeavor discloses wherein the one or more processors execute identification of a meaning of the annotation (Maack, Para 0055-0057; “… the image data associated with the descriptive label comprises a digital overlay … the digital overlay is provided from DICOM projection information … The processing unit is configured to implement an image processing algorithm to identify the descriptive label.” The disclosed descriptive label is some text provided by DICOM information, so once the digital overlay is recognized, its meaning is identified). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanigawa, as suggested by Maack, in order to display descriptive label in an image and identify the label’s meaning using a processor or algorithm. One of ordinary skill in the art would have been motivated to make the modification for the benefit of showing a descriptive label for assisting image examining and diagnosis by doctors, and identifying the label’s meaning by processor for automatically checking if the labeling is correct (Maack, Para 0016; “… immediate and fool proof feedback mechanism is provided to aid the work of radiographers, through a check of the consistency between an X-ray image and the label.”).
With regard to Claim 9, Tanigawa and Maack disclose the ultrasonic diagnostic device according to Claim 8, but as discussed above do not explicitly and clearly disclose wherein the one or more processors identify the meaning of the annotation added to the first ultrasonic image using an annotation list including symbols permitted to be used as annotations and meanings represented by the symbols.
Maack further discloses wherein the one or more processors identify the meaning of the annotation added to the first ultrasonic image using an annotation list including symbols permitted to be used as annotations and meanings represented by the symbols (Maack, Para 0056; “… the digital overlay is provided from DICOM projection information.” It is known to one of ordinary skill in the art that DICOM of an image provides different codes, including the disclosed DICOM projection information, which contain information related to the image. For DICOM projection information as disclosed in Maack, at least symbols AP and PA are included, representing “from anterior to posterior” and “from posterior to anterior” respectively). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanigawa and Maack, as further suggested by Maack, in order to use a list of symbols with defined meanings. One of ordinary skill in the art would have been motivated to make the modification for the benefit of allowing rapid recognition of image labels so as to efficiently and correctly finish an imaging exam.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Van der Veen et al (US 20210330284 A1) discloses guiding acquisition of color Doppler data based on user-selected clinical object and a B-mode image. Moon et al (US 20190053788 A1) discloses creating an text-based annotation and displaying it with an ultrasound image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LEI ZHANG whose telephone number is (571)272-7172. The examiner can normally be reached Monday-Friday 8am-5pm E.T..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pascal Bui-Pho can be reached at (571) 272-2714. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/L.Z./Examiner, Art Unit 3798
/PASCAL M BUI PHO/Supervisory Patent Examiner, Art Unit 3798
Prosecution Insights