Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Responses to formal issues
The objection to the title has been withdrawn.
The objection the specification has also been withdrawn.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 7 and 8 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. However applicant argues that the cited prior art fails to teach the limitation “determining an examination action to be performed next on the basis of workflow information indicating an order in which a plurality of examination actions are to be performed in an ultrasonic examination and an accuracy of specifying the imaging scene for said each of the specifying processing, from among the plurality of examination actions defined in the workflow information.”
Dhatt the newly added prior art teaches determines one or more operations/commands to control the ultrasound machine and can present selectable next operations. (Section 0023 ,lines 2-10 “image information of an ultrasound image being displayed by the ultrasound machine, a list of operations (e.g a workflow) being performed by the user. The next action to be performed by the user of the ultrasound machine”.
Also in Section 0027, lines 1-3 thus the other data such as worklist shows an operation that the user wants to perform and lines 8-10 also says the lip reading recognition routine determines an operation that the user wants to perform.
All the paragraphs shows that Dharts teaches determining the next examination actions.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6, 8, 9, 12-14, 17 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bhownick et al., (US Pub No. 2023003864) in view of Dhatt (US20200170622).
NOTE: The Examiner has used US Pub No. 20230038364 to Bhownick as an equivalent publication of European Patent Application 20152186.1 published 7/21/2021 as EP 3852054 A1.
Claim 1: Bhownick discloses an ultrasonic imaging apparatus comprising one or more processors and a program storage device tangibly embodying a program of instructions executable by the one or more processors [Abstract], the ultrasound imaging apparatus including the one or more processors performing a method comprising:
receiving an ultrasound image generated by transmitting and receiving an ultrasonic wave [ultrasound probe, p0170-0171 – NOTE: although transmitting and receiving an ultrasonic wave is not explicitly stated, these features are inherent in an ultrasound probe], specifying an imaging scene by a plurality of types of specifying processing, and calculating an accuracy of specifying the imaging scene for each of the specifying processing [receiving a video stream of medical images acquired during an ultrasound scan of the subject … a computational unit configured for applying an object detector function to the medical image, wherein the object detector function is trained to detect a plurality of classes of anatomical structures, thereby generating as output the coordinates of a plurality of bounding boxes and a confidence score for each bounding box, the confidence score giving the probability of the bounding box containing an anatomical structure belonging to one of the plurality of classes, p0060, p0086-0087 & p0104-0106],
the plurality of types of specifying processing including (a) identification processing of a cross-section of the ultrasound image [medical images capturing the largest cross-section of a certain anatomical structure are used for further evaluation … detect a first plurality of classes of larger-sized anatomical structures, thereby generating as output the coordinates of at least one first bounding box and the confidence score of it containing a larger-sized anatomical structure, p0039, p0088 & p0104-0106], (b) processing of detecting an abnormality shown in the ultrasound image [hierarchical inferencing scheme of the bounding boxes provides automatic identification of normal IUP and non-IUP cases, like ectopic. Thereby, the diagnostic approach can be made more systematic, and a simple scheme may be used to detect abnormal pregnancy, p0062 & p0105], and (c) processing of specifying a site of a subject shown in the ultrasound image [detect at least one second class of smaller-sized anatomical structures, thereby generating as output the coordinates of at least one second bounding box and the confidence score of it containing a smaller-sized anatomical structure, p0090 & p0104-0106], and
an accuracy of specifying the imaging scene for said each of the specifying processing, from among the plurality of examination actions defined in the workflow information [bounding boxes and the confidence score of each bounding box containing a certain class of anatomical structure. The output may be visualized by outputting the medical image and/or the cropped image, and displaying the bounding boxes having the highest confidence score thereon, e.g. in a contrasting color scheme, or overlaid on the image, p0027 & p0091].
Bhownick fails to disclose determining an examination action to be performed next on the basis of workflow information indicating an order in which a plurality of examination actions are to be performed in an ultrasonic examination
Dhatt discloses determining an examination action to be performed next on the basis of workflow information indicating an order in which a plurality of examination actions are to be performed in an ultrasonic examination. (Section 0023 determines one or more operations/commands to control the ultrasounds machine and can present selectable next operations)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to modify the teaching of Bhownick with the teachings of determining the next action within the workflow. The motivation is that the system works faster within delay for human decision making.
Claim 2: Bhownick in view of Dhatt discloses the ultrasonic imaging apparatus according to claim 1, wherein the method performed by the ultrasound imaging apparatus including the one or more processors further comprises: comparing an image of a predetermined standard cross-section with the ultrasound image to identify the cross-section shown in the ultrasound image; and calculating an accuracy of identifying the cross-section as the accuracy of specifying the imaging scene [Bhownick: a weighted average of the bounding boxes and the confidence scores have been used in formulating a confidence interval which has been subsequently used for best plane computations. For example, those medical images capturing the largest cross-section of a certain anatomical structure are used for further evaluation. These planes/images along with the corresponding bounding boxes may then be used for automated computation of key parameters [interpreted as comparison between images], p0039].
Claim 3: Bhownick in view of Dhatt discloses the ultrasonic imaging apparatus according to claim 2, wherein the method performed by the ultrasound imaging apparatus including the one or more processors further comprises: calculating an accuracy of detection in the processing of detecting the abnormality as the accuracy of specifying the imaging scene [Bhownick: rendered probability may be 1 or 0, but may also take a value in-between, and may depend on the confidence score of the bounding boxes used for its determination. The inferencing scheme may for example comprise a number of IF ELSE commands, which are implemented in an algorithm in any computational language. In the case of first trimester pregnancy, the inferencing scheme may for example be as follows: Let the bounding box of uterus be denoted as B.sub.u and of GS as B.sub.GS. If B.sub.GS is a subset of B.sub.u, then it is inferred as normal IUP, else it is termed as abnormal pregnancy. In the second level automation of using the second neural network, yolk sac (YS) and embryo are detected and localised. Let the bounding boxes for yolk sac and embryo be termed B.sub.YS and B.sub.E. If B.sub.YS and/or B.sub.E are subsets of B.sub.GS, then it is termed as a normal pregnancy. If YS and embryo are not detected within GS, then the chances of abnormal pregnancy are increased. An “increased” probability of a medical condition may for example mean that its probability is above a pre-defined threshold, for example above a value in the range of 50-95%, preferably 60-90%, p0035].
Claim 4: Bhownick in view of Dhatt discloses the ultrasonic imaging apparatus according to claim 3, wherein the method performed by the ultrasound imaging apparatus including the one or more processors further comprises: (aa) calculating an accuracy of specifying in the processing of specifying the site of the subject as the accuracy of specifying the imaging scene [Bhownick: rendered probability may be 1 or 0, but may also take a value in-between, and may depend on the confidence score of the bounding boxes used for its determination. The inferencing scheme may for example comprise a number of IF ELSE commands, which are implemented in an algorithm in any computational language. In the case of first trimester pregnancy, the inferencing scheme may for example be as follows: Let the bounding box of uterus be denoted as B.sub.u and of GS as B.sub.GS. If B.sub.GS is a subset of B.sub.u, then it is inferred as normal IUP, else it is termed as abnormal pregnancy. In the second level automation of using the second neural network, yolk sac (YS) and embryo are detected and localized [interpreted as the site of the subject]. Let the bounding boxes for yolk sac and embryo be termed B.sub.YS and B.sub.E. If B.sub.YS and/or B.sub.E are subsets of B.sub.GS, then it is termed as a normal pregnancy. If YS and embryo are not detected within GS, then the chances of abnormal pregnancy are increased. An “increased” probability of a medical condition may for example mean that its probability is above a pre-defined threshold, for example above a value in the range of 50-95%, preferably 60-90%, p0035 & p0106-0107].
Claim 6: Bhownick in view of Dhatt discloses the ultrasonic imaging apparatus according to claim 1, wherein the method performed by the ultrasound imaging apparatus including the one or more processors further comprises: determining a setting of a body mark to be set in the ultrasound image as the examination action to be performed next [Bhownick: displaying the medical image together with the at least one first bounding box and the at least one second bounding box displayed in a contrasting color, so that a user may check the accuracy of the prediction, p0084].
Claim 8: the Non Transitory computer readable recording medium program herein has been executed or performed by the apparatus of claim 1 and is therefore likewise rejected.
Claim 9: Bhownick in view of Dhatt discloses the non-transitory computer-readable recording medium according to claim 8, wherein the method performed by the computer executing the program further comprises: displaying, on a display, information for performing the examination action or information prompting an examiner to perform the examination action [Bhownick: displaying the medical image together with the at least one first bounding box and the at least one second bounding box displayed in a contrasting color, so that a user may check the accuracy of the prediction, p0084].
Claim 12: Bhownick in view of Dhatt discloses the non-transitory computer-readable recording medium according to claim 9, wherein the method performed by the computer executing the program further comprises: calculating a matching degree between a cross-section of the ultrasound image and a standard cross-sectional image; calculating an accuracy of an abnormal object detected from the ultrasound image [Bhownick: a weighted average of the bounding boxes and the confidence scores have been used in formulating a confidence interval which has been subsequently used for best plane computations. For example, those medical images capturing the largest cross-section of a certain anatomical structure are used for further evaluation. These planes/images along with the corresponding bounding boxes may then be used for automated computation of key parameters [interpreted as comparison between images], p0039]; and determining the examination action on the basis of the matching degree and the accuracy of the abnormal object [Bhownick bounding boxes and the confidence score of each bounding box containing a certain class of anatomical structure. The output may be visualized by outputting the medical image and/or the cropped image, and displaying the bounding boxes having the highest confidence score thereon, e.g. in a contrasting colour scheme, or overlaid on the image … n the case of the field-of-view covering the uterus of a woman in early pregnancy, the invention may serve to make the diagnosis of normal or abnormal early pregnancy more systematic, in particular the diagnosis of ectopic pregnancies, molar pregnancies or adnexal features, p0027-0028 & p0091].
Claim 13: Bhownick in view of Dhatt discloses the non-transitory computer-readable recording medium according to claim 12, wherein the ultrasound image is an image generated by transmission and reception of an ultrasonic wave to and from a fetus [Bhownick: receiving a video stream of medical images acquired during an ultrasound scan of the subject … e.g., gestational Yolk sac, embryo sac, embryo Fetus NT, fetal heart as shown in Table 1 … provided by an ultrasound probe, p0025-0026, p0060 & p0170-0171 – NOTE: although transmitting and receiving an ultrasonic wave is not explicitly stated, these features are inherent in an ultrasound probe].
Claim 14: the apparatus herein has been executed or performed by the program of claim 9 and is likewise rejected.
Claim 17: the apparatus herein has been executed or performed by the program of claim 12 and is likewise rejected.
Claim 18: the apparatus herein has been executed or performed by the program of claim 13 and is likewise rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bhownick et al., (US Pub No. 2023003864) in view of Dhatt (US20200170622) and further in view of Li (US Pub No. 20190065897).
Claim 5: Bhownick in view of Dhart discloses the ultrasonic imaging apparatus according to claim 4.
Bhownick in view of Dhart does not appear to explicitly disclose wherein at least one of the result of the calculation in (aa) and a result determined on the basis of the result of the calculation is used to determine the examination action to be performed next.
Li discloses in a related system from the same field of endeavor [Abstract] wherein at least one of the result of the calculation in (aa) [in a case that the classifier 101 is the Softmax classifier, the plurality of scores is a plurality of prediction probabilities [i.e., accuracy calculation], and the classification result corresponds to a maximum prediction probability among the plurality of prediction probabilities, p0095] and a result determined on the basis of the result of the calculation is used to determine the examination action to be performed next [each computer aided diagnosis device in the computer aided diagnosis device group 300 corresponds to a classification result, and is configured to analyze features of the medical image corresponding to the classification result. Each computer aided diagnosis device is further configured to output an analysis result for subsequent operations, p0097 & p0110].
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the invention to have included in the teachings of Bhownick in view of Dhart the support wherein at least one of the result of the calculation in (aa) and a result determined on the basis of the result of the calculation is used to determine the examination action to be performed next as disclosed by Li because it allows the doctor to combine the analysis results and the medical images to perform disease diagnosis, treatment and other operations on a patient as discussed by Li in at least paragraph 0110.
Claim 7: Bhownick in view of Dhart discloses the ultrasonic imaging apparatus according to claim 1.
Bhownick in view of Dhart appears to fail to explicitly disclose wherein the method performed by the ultrasound imaging apparatus including the one or more processors further comprises: determining an attachment of the ultrasound image on a report as the examination action to be performed next.
Li discloses determining an attachment of the ultrasound image on a report as the examination action to be performed next [display the analysis results and/or medical images. A doctor can view the analysis results and/or the medical images through the monitor, a projection screen, or the like [interpreted as attachment in view of the description in the instant application], and combine the analysis results and the medical images to perform disease diagnosis, treatment and other operations [i.e., next action to be performed] on a patient, p0110].
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the invention to have included in Bhownick the support for determining an attachment of the ultrasound image on a report as the examination action to be performed next as disclosed by Li because it allows the doctor to combine the analysis results and the medical images to perform disease diagnosis, treatment and other operations on a patient as discussed by Li in at least paragraph 0110.
Claim(s) 10, 11, 15 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bhownick et al., (US Pub No. 2023003864) in view of Dhatt (US20200170622) and further in view of Owen et al., (US Pub No. 20220280139).
Claim 10: Bhownick in view of Dharts discloses the non-transitory computer-readable recording medium according to claim 9, wherein the examination action is an action related to image adjustment of the ultrasound image [displaying the medical image together with the at least one first bounding box and the at least one second bounding box displayed in a contrasting color, so that a user may check the accuracy of the prediction, p0084].
Bhownick does not appear to disclose where the action is related to image adjustment.
Owen discloses in a related system from the same field of endeavor [Abstract] wherein the examination action is an action related to image adjustment of the ultrasound image [the system can alert the clinician that a change in operating levels or conditions would be prudent for the present exam, Abstract].
It would have been obvious to persons of ordinary skill in the art before the effective filing date of the invention to have included in Bhownick the support wherein the examination action is an action related to image adjustment of the ultrasound image as taught by Owen because it allows the clinician to make operating adjustments to obtain clearer images while maintaining patient safety as discussed in the Abstract and at least paragraphs 0013-0014.
Claim 11: Bhownick in view of Dharts discloses the non-transitory computer-readable recording medium according to claim 10, wherein the ultrasound image is an image generated by transmission and reception of an ultrasonic wave to and from a fetus [receiving a video stream of medical images acquired during an ultrasound scan of the subject … e.g., gestational Yolk sac, embryo sac, embryo Fetus NT, fetal heart as shown in Table 1 … provided by an ultrasound probe, p0025-0026, p0060 & p0170-0171 – NOTE: although transmitting and receiving an ultrasonic wave is not explicitly stated, these features are inherent in an ultrasound probe].
Owen also discloses wherein the ultrasound image is an image generated by transmission and reception of an ultrasonic wave to and from a fetus [p0015 & p0020].
Claim 15: the apparatus herein has been executed or performed by the program of claim 10 and is likewise rejected.
Claim 16: the apparatus herein has been executed or performed by the program of claim 11 and is likewise rejected.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Akwasi M Sarpong whose telephone number is (571)270-3438. The examiner can normally be reached Mon-Fri. 8:00am-4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AKWASI M SARPONG/SPE, Art Unit 2681 7/21/2026
Prosecution Insights