DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments and amendments in the Amendment filed July 6, 2026 (herein “Amendment”), with respect to the objection to claim 1, and therefore all claims depending therefrom, have been fully considered and are persuasive. The objection to claim 1, and therefore all claims depending therefrom has been withdrawn.
Applicant’s amendments to claim 1 with respect to the rejection of claim 1 under 35 U.S.C. 102 have been fully considered and are persuasive in part. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Rothberg, US Patent Application Publication No. US 2019/0059851 A1. Applicants arguments consisted of conclusory statements that the primary reference Chen does not teach or suggest the limitations of claim 1, without specifically pointing out limitations and discussing the cited portions of Chen in the rejection. However, in regards to the newly recited “the sample ultrasound image comprising an ensemble of consecutive image frames,” Chen teaches receiving a sequence of medical images (ultrasound image frames) in fig. 14, reference number 1402 and the associated disclosure thereof. Further Chen teaches the newly claimed “calculate a level of uncertainty associated with a prediction of a predefined abnormal condition of the one or more identified types of tissue and/or anatomical structures,” in disclosing a calculated score for its disease diagnosis in col. 13, l. 64 – col. 14, l. 16. Chen also teaches the newly claimed “if the calculated level of uncertainty falls within a predefined acceptable value indicating more likely than not that the patient has the abnormal condition,” in col. 14, 11. 7–20, and fig. 5B, as set forth in more detail below in the substance of the updated rejection rationale for claim 1. However, for the newly recited “the level of uncertainty being based, at least in part, on aggregated prediction associated with natural variations occurring over a succession of the ensemble of consecutive image frames,” this limitation is not explicitly taught by Chen, and so for this new limitation, the newly cited Rothberg is applied.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–14 and 16–17 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al., US Patent No. US 11,830,189 B2 (herein “Chen”) in view of Rothberg, United States Patent Application Publication No. US 2019/0059851 A1 (herein “Rothberg”) .
Regarding claim 1, with deficiencies of Chen noted in square brackets [], Chen teaches a system for providing automated medical imaging analysis, the system comprising (Chen, Figs 1 and 2, col. 2, ll. 55–60, network environment (system) for processing electronic medical images):
a computing system comprising a hardware processor coupled to non-transitory, computer-readable memory containing instructions executable by the processor to cause the computing system to (Chen fig. 1, col. 5, ll. 23–44, col. 7, ll. 41–51, col. 8, ll. 24–35, and col. 41, l. 55–col. 42, l. 26, components shown in fig. 1 integrated into one system, including processing devices (processor) and a RAM memory including instructions executed by the processor):
run a neural network (Chen col. 6, ll. 36–41, deployment of a random deep forest machine learning technique and a deep neural network such as a convolutional neural network/CNN), wherein the neural network has been trained using a plurality of training data sets, each training data set comprises reference ultrasound image data associated with known tissue and/or anatomical structures and known condition data associated with the known tissue and/or anatomical structures (Chen col. 6, ll. 41–47, col. 10, ll. 22–53, supervised learning trains the neural network using training data comprised of ultrasound medical images annotated by human technicians to label one or more musculoskeletal disorders (known condition data) and anatomical features such as bones, tendons, ligaments, muscles, nerves, and also disruptive features such as lesions, scars, malignant soft tissue tumors, the annotations may indicate an area or volume of any of those features);
receive and analyze a sample ultrasound image of a target site of a patient by using the neural network and based on an association of the known condition data with the reference ultrasound image data (Chen col. 12, l. 44 – col. 13, l. 6, a medical ultrasound image of an anatomical structure of a patient being image is received and input to the trained machine learning model Chen col. 10, 11. 22–53, where by virtue of the supervised learning, the trained neural network will analyze the input image based on the trained association of the training data labels including the musculoskeletal disorder condition (the condition data) and the training ultrasound images (reference ultrasound image data)), the sample ultrasound image comprising an ensemble of consecutive image frames (Chen fig. 14, reference number 1402, col. 29, ll. 12–34, receiving a sequence of medical images from the ultrasound imaging system during a same session (thus consecutive));
identify, based on the analysis, one or more types of tissue and/or anatomical structures and an associated condition of the one or more types of tissue and/or anatomical structures in the sample ultrasound image (Chen col. 13, ll. 15–29, identifications of an object relative to a given anatomical structure, and a predicted diagnosis including an indication of a presence or absence of a musculoskeletal disorder);
calculate a level of uncertainty associated with a prediction of a predefined abnormal condition of the one or more identified types of tissue and/or anatomical structures, (Chen col. 13, l. 64–col. 14, l. 16, a score associated with the predicted diagnosis is displayed which indicates a likelihood (level of uncertainty associated with a prediction) that the predicted diagnosis for the musculoskeletal (type of tissue and/or anatomical structure) disorder is present within the image) [the level of uncertainty being based, at least in part, on aggregated predictions associated with natural variations occurring over a succession of the ensemble of consecutive image frames]; and
output, via a display, an augmented ultrasound image comprising a visual representation of the one or more identified types of tissue and/or anatomical structures at the target site of the patient and an associated abnormal condition of the one or more identified types of tissue and/or anatomical structures, if present (Chen col. 13, ll. 36–65, fig. 5B, medical image and predicted diagnosis is provided to the computing device for display, for example as shown in fig. 5B showing tendons of the rotator cuff structure and specifically the suprapinatus muscle/tendon complex, and the predicted diagnosis of a full thickness tear) if the calculated level of uncertainty falls within a predefined acceptable value indicating more likely than not that the patient has the abnormal condition (Chen col. 14, ll. 2–20, fig. 5B, the physician feels confident in using the predicted diagnosis when there is a high value (level of uncertainty falls within the physician’s acceptable value indicating more likely than not that the diagnosis is correct) for the score for the predicted diagnosis that corresponds to what the physician is able to visualize with their own eyes).
Chen does not explicitly teach where Rothberg teaches the level of uncertainty being based, at least in part, on aggregated predictions associated with natural variations occurring over a succession of the ensemble of consecutive image frames (Rothberg ¶198, landmark localization used in identifying structures of the human heart (tissue/anatomical structure) in ultrasound video sequences (succession of the ensemble of consecutive images) is made more accurate (level of uncertainty based on) by regression processing within a neural network that considers standard deviations of the predictions (natural variations occurring)).
Therefore, taking the teachings of Chen and Rothberg together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the score calculation of Chen to include the standard deviation calculations and considerations disclosed above in Rothberg at least because doing so would show accuracy on par with inter-user variability (Rothberg ¶198) and provide greater ability for users of an ultrasound system to be able to capture a medically relevant ultrasound image (Rothberg ¶51).
Regarding claim 2, Chen teaches wherein the visual representation comprises at least a first layer of content overlaid upon the sample ultrasound image (Chen col. 22, ll. 20–41, fig. 10, machine learning model trained to identify objects such as a tendon and calcium deposit in an ultrasound image and output to the display a visualization including the medical image with the tendon and the calcium deposit labeled in the image).
Chen does not anticipate the limitations of claim 2 because the teachings of Chen that apply to claim 2 are disclosed with respect to a different embodiment than that relied upon above for claim 1 which claim 2 depends. However, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified and combined the teachings from the fig. 5B and fig. 10 embodiments of Chen at least because Chen in col. 43, ll. 9–14 teach that the given embodiments are considered exemplary only and not to be restrictive of the disclosure. Additionally, such a combination of the trained machine model of fig. 5B of Chen to include the anatomical structure labeling and outlining shown in fig. 10 of Chen would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention as doing so would be simple substitution of one known element for another to obtain predictable results. See MPEP §2143(I)(B).
Regarding claim 3, Chen teaches wherein the first layer of content comprises semantic segmentation (Chen col. 5, l. 64–col. 6, l. 6, training images used to train the neural network annotated using semantic segmentation, therefore conditioning the machine learning model during inferencing to segment images according to semantic segmentation) of different types of tissue within the sample ultrasound image and/or different anatomical structures within the sample ultrasound image (Chen col. 22, ll. 20–41, fig. 10, displaying the visualization including the medical image with the tendon and the calcium deposit (different anatomical structures) labeled in the image).
Regarding claim 4, Chen teaches wherein the first layer of content comprises one or more shaded zones overlaid upon one or more respective portions of the sample ultrasound image (Chen col. 34, ll. 13–27, fig. 18, user interface displaying image along with a shading, patterning and /or highlighting including color portion of a detected anatomical features such as a nerve, bone and tendon respectively). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified and combined the teachings from the fig. 5B and fig. 18 embodiments of Chen at least because Chen in col. 43, ll. 9–14 teach that the given embodiments are considered exemplary only and not to be restrictive of the disclosure. Additionally, such a combination of the trained machine model of fig. 5B of Chen to include the shading shown in fig. 18 of Chen would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention as doing so would be simple substitution of one known element for another to obtain predictable results. See MPEP §2143(I)(B).
Regarding claim 5, Chen teaches wherein each of the one or more shaded zones corresponds to a respective one of the identified types of tissue and/or identified anatomical structures (Chen col. 34, ll. 13–27, fig. 18, user interface displaying image along with a shading, patterning and /or highlighting including color portion of a detected anatomical features separately such as a nerve in a second color, bone in a first color, and tendon in a third color).
Regarding claim 6, Chen teaches wherein each of the identified types of tissue and/or identified anatomical structures comprises a shaded zone with a distinct color and/or pattern to distinguish from one another (Chen col. 34, ll. 13–27, fig. 18, user interface displaying image along with a shading, patterning and /or highlighting including color portion of a detected anatomical features separately such as a nerve in a second color, bone in a first color, and tendon in a third color).
Regarding claim 7, Chen teaches wherein the first layer of content further comprises text associated with each of the one or more shaded zones, wherein the text identifies the respective one of the identified types of tissue and/or identified anatomical structures (Chen col. 34, ll. 13–27, fig. 18, user interface displaying image along with a shading, patterning and /or highlighting including color portion of a detected anatomical features separately such as a nerve in a second color, bone in a first color, and tendon in a third color, and the anatomical features are labeled in all axes).
Regarding claim 8, Chen teaches wherein the identified types of tissue and/or anatomical structures are selected from the group consisting of muscles, bones, organs, blood vessels, and nerves (Chen col. 34, ll. 13–27, fig. 18, user interface displaying image along with separate identifications for a nerve, bone, and tendon, where col. 32, ll. 15–21 teaches the real-time recognition of muscle, veins (tubular organs), arteries (blood vessels)).
Regarding claim 9, Chen teaches wherein the visual representation comprises a second layer of content overlaid upon the sample ultrasound image (Chen col. 22, ll. 30–51, and fig. 10, area = X and volume = y (second layer of content) shown on top of the image).
Regarding claim 10, Chen teaches wherein the second layer of content comprises a visual indication of an abnormal condition associated with the one or more identified types of tissue and/or identified anatomical structures (Chen col. 13, ll. 36–65, fig. 5B, fig. 10, col. 22, ll. 30-51, considering the table annotation as a second layer of content, the predicted diagnosis of a full thickness tear and probability, and fig. 10 showing the additional content as an overlay on the image). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified and combined the teachings from the fig. 5B and fig. 10 embodiments of Chen including the text information shown in fig. 5B to be overlayed onto the image as shown in fig. 10, at least because Chen in col. 43, ll. 9–14 teach that the given embodiments are considered exemplary only and not to be restrictive of the disclosure. Additionally, such a combination would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention as doing so would be simple substitution of one known element for another to obtain predictable results. See MPEP §2143(I)(B).
Regarding claim 11, Chen teaches wherein the visual indication comprises text identifying the specific abnormal condition (Chen col. 13, ll. 36–65, fig. 5B , the table annotation as text stating an abnormal condition of a full thickness tear).
Regarding claim 12, Chen teaches wherein the visual indication comprises a marking and/or shaded zone corresponding to at least a portion of one of the identified types of tissue and/or identified anatomical structures to which the abnormal condition is associated (Chen col. 34, ll. 13–27, fig. 18, user interface displaying image along with a shading, patterning and /or highlighting including color portion of a detected anatomical features separately such as a nerve in a second color, bone in a first color, and tendon in a third color).
Regarding claim 13, Chen teaches wherein the abnormal condition comprises a structural abnormality (Chen col. 13, ll. 36–65, fig. 5B showing tendons of the rotator cuff structure and specifically the suprapinatus muscle/tendon complex, and the predicted diagnosis of a full thickness tear, thus the tear being the abnormality in the suprapinatus muscle/tendon complex structure).
Regarding claim 14, Chen teaches wherein the abnormal condition comprises a disease (Chen col. 22, ll. 20–41, fig. 10, displaying the visualization including the medical image with a calcium deposit labeled in the image where col. 15, ll. 19–23 teach the calcium deposit visualizing disease). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified and combined the teachings from the fig. 5B and fig. 10 embodiments of Chen by combining the visualization shown in fig. 5B to include a disease type abnormal condition as disclosed in fig. 10, at least because Chen in col. 43, ll. 9–14 teach that the given embodiments are considered exemplary only and not to be restrictive of the disclosure. Additionally, such a combination would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention as doing so would be simple substitution of one known element for another to obtain predictable results. See MPEP §2143(I)(B).
Regarding claim 16, Chen teaches wherein the computing system comprises a machine learning system selected from the group consisting of a neural network, a random forest, a support vector machine, a Bayesian classifier, a Hidden Markov model, an independent component analysis method, and a clustering method (Chen col. 6, ll. 36–41, deployment of a random deep forest machine learning technique and a deep neural network such as a convolutional neural network/CNN).
Regarding claim 17, Chen teaches wherein the computing system comprises an autonomous machine learning system that associates the condition data with the reference ultrasound image data (Chen col. 16, ll. 45–64, machine learning model in a validation or test phase analyzes a first training image (input image) by comparing it to the known object types (reference ultrasound data) identified by the corresponding label, and col. 11, ll. 51–54 teaching that the image also has a corresponding label of diagnosis (condition data associated)). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified and combined the teachings from the col. 16 and col. 11 disclosed embodiments of Chen, at least because Chen in col. 43, ll. 9–14 teach that the given embodiments are considered exemplary only and not to be restrictive of the disclosure. Additionally, such a combination would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention as doing so would be simple substitution of one known element for another to obtain predictable results. See MPEP §2143(I)(B).
Claims 15 and 18–20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Rothberg, as set forth above, and further in view of Rajagopal et al., US Patent Application Publication No. US 2025/0069290 A1 (herein “Rajagopal”).
Regarding claim 15, with deficiencies of Chen noted in square brackets [], Chen teaches wherein the analysis of the sample ultrasound image comprises [correlating] sample image data with the known tissue and/or anatomical structures of the reference ultrasound image data and known condition data associated therewith (Chen col. 16, ll. 45–64, machine learning model in a validation or test phase analyzes a first training image (input image) by comparing it to the known object types (known tissues/anatomical structures) identified by the corresponding label, and col. 11, ll. 51–54 teaching that the image also has a corresponding label of diagnosis (known condition data associated therewith)).
While Chen teaches comparing image data to the known values, Chen does not explicitly teach correlating the data. However, Rajagopal teaches correlating data (Rajagopal ¶71, during test/validation, images input the machine learning model are evaluated using correlation techniques versus ground truth (known) images).
Therefore, taking the teachings of Chen and Rajagopal together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the comparison taught by Chen to be specifically a correlation as disclosed by Rajagopal at least because doing so would allow for fine-tuning existing pre-trained models, thus optimizing the quality of the machine learning output. See Rajagopal ¶¶ 71 and 5–6.
Regarding claim 18, with deficiencies of Chen noted in square brackets [], Chen teaches wherein the machine learning system comprises a deep learning neural network that includes an input layer, [a plurality of hidden layers], and an output layer (Chen col. 6, ll. 36–41, machine learning system including a deep neural network such as a CNN, where deep neural network has a plain meaning of at least two layers, and given Chen’s teachings of receiving an input, and producing an output, Chen’s CNN at least has an input layer and an output layer).
Chen does not explicitly teach, where Rajagopal teaches a plurality of hidden layers (Rajagopal ¶¶62, and 76, machine learning with multiple layers including lower layers and higher layers between the input and output (hidden layers)).
Therefore, taking the teachings of Chen and Rajagopal together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the neural network of Chen to include hidden layers as disclosed by Rajagopal at least because the number of hidden layers is a hyperparameter (see Rajagopal ¶76) and as such would be a mere matter of design choice with predictable results. See MPEP §2144.04(VI)(C).
Regarding claim 19, with deficiencies of Chen noted in square brackets [], Chen teaches wherein the autonomous machine learning system represents the training data set using (Chen col. 11, ll. 47–62, training data incorporated into the machine learning model by modifying altering weights and biases) [a plurality of features, wherein each feature comprises a feature vector].
Chen does not explicitly teach, where Rajagopal teaches a plurality of features, wherein each feature comprises a feature vector (Rajagopal ¶¶62,67, 92, 96 3D UNet trained using input vectors, the input being features from the input images).
Therefore, taking the teachings of Chen and Rajagopal together as a whole, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning representation taught by Chen to include input features and their vectors as disclosed by Rajagopal at least because doing so would allow for fine-tuning existing pre-trained models, thus optimizing the quality of the machine learning output. See Rajagopal ¶¶ 71 and 5–6.
Regarding claim 20, Chen teaches wherein the autonomous machine learning system comprises a convolutional neural network (CNN) (Chen col. 6, ll. 36–41, the deep neural network being a convolutional neural network/CNN).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights