DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The specification of the claimed invention is objected to for the following informalities:
“showed” reads as typographical error for “shown” throughout the specification.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Accordingly, the claim language that meets the prongs of 35 U.S.C. 112(f) is: “implemented by technical means”; “means arranged to and/or programmed to and/or configured to receive the input”; “means arranged to and/or programmed to and/or configured to construct a training set”; “means arranged to and/or programmed to and/or configured to construct a first training set”; “means arranged to and/or programmed to and/or configured to construct a second training set”; “means arranged to and/or programmed to and/or configured to construct a third training set”; “means arranged to and/or programmed to and/or configured to train the bone score artificial intelligence”; “means arranged to and/or programmed to and/or configured to train the first artificial intelligence”; “means arranged to and/or programmed to and/or configured to train the second artificial intelligence”; “means arranged to and/or programmed to and/or configured to train the third artificial intelligence”; “wherein the means arranged to and/or programmed to and/or configured to train the first artificial intelligence and the means arranged to and/or programmed to and/or configured to train the second artificial intelligence are arranged together to and/or programmed to and/or configured together to check”.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Objections
Claims 41, 43-45, 50-51, 53, 60, 62-64, 69, 70, and 72 are objected to for the following informalities:
“showed” reads as a typographical error for “shown”
Claim 69 is additionally objected to for the following informalities:
Reference numbers (9) and (19) read as errors.
Claim 77 is objected to for the following informalities:
“a computerized image” reads as a typographical error for “a computerized tomography image” as recited in parallel claim 58. As computerized images represent the overwhelming majority of feasible methods implicated by the specification and claims, the examiner assumes this recitation to be in error, and will analyze the claim in line with parallel claim 58’s language.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 41-80 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
With respect to claims 41 and 60, MPEP § 2173.02(I) reads “For example, if the language of a claim, given its broadest reasonable interpretation, is such that a person of ordinary skill in the relevant art would read it with more than one reasonable interpretation, then a rejection under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph is appropriate. Examiners, however, are cautioned against confusing claim breadth with claim indefiniteness. A broad claim is not indefinite merely because it encompasses a wide scope of subject matter provided the scope is clearly defined. Instead, a claim is indefinite when the boundaries of the protected subject matter are not clearly delineated and the scope is unclear.”
As relevant to claims 41/60, the claims first recite a bone score analysis giving two resultant scores “without any calculation or determination of an experimental variogram of the gray levels of the received input x-ray image”. However, to determine such scores, the analysis depend[s] “at least on a trabecular bone score which quantifies the local variations in gray levels from the experimental variogram of the gray levels of the trabecular part of the input bone showed on the received input x-ray image, and/or a trabecular bone score which quantifies the local variations in gray levels from the experimental variogram of the gray levels of the trabecular part of the input bone showed on the received input x-ray image”
As the “the experimental variogram” is referred to with antecedent basis, the claims seemingly require the artificial intelligence to compute a result using an input that it is supposed to neither calculate or even determine. Accordingly, it is unclear whether the claims require the experimental variogram to be utilized or not in the bone score analysis, and if so to what degree.
For the purposes of compact prosecution, the examiner will interpret the claims as simulating an experimental variogram-derived trabecular bone score, but otherwise not requiring an experimental variogram-derived trabecular bone score. The word “determine” given the broadest reasonable interpretation, necessarily involves a computer even so much as recognizing that a certain value is within its data – regardless of whether said computer calculated the value itself or received it from an external source.
Recognizing that other interpretations may hold reasonable weight, the claims are accordingly rejected under MPEP § 2173.02(I). All remaining claims, incorporating either the language of claim 41 or claim 60, are rejected as well.
Claims 41 and 60 are further rejected under similar grounds of antecedent basis. As “the experimental variogram” was introduced first only in the negative, it is further unclear whether it can be relied upon again (later in the claim) as a positive limitation, since the variogram only existed in its absence. Accordingly, claims 41 and 60 are further rejected, as well as all remaining claims incorporating either the language of claim 41 or claim 60.
Claims 41 and 60 are further rejected under additional grounds. MPEP § 2173.05(q) reads “Attempts to claim a process without setting forth any steps involved in the process generally raise an issue of indefiniteness under 35 U.S.C. 112(b)…”. Notwithstanding “receiving a digitized input image x-ray image showing an input bone”, the remaining limitations of the method recite nouns rather than process steps. “a bone score analysis…giving as a result of this bone score analysis, a global score...and/or a trabecular bone score…”. The rationale of MPEP §2173.05(q) and its cited case law hinges on the ambiguity of the “process” to a potential infringer, wherein it is unclear if infringement occurs when a product possesses the recited functional structure, or only when that functional steps are actually performed. Similar issues are present in the language of claims 45/64.
For the purposes of compact prosecution, the examiner will read as prior art, references which both possess the means to perform the analyses as well as the performance of the analyses themselves.
Accordingly, the claims are rejected. All remaining claims, incorporating either the language of claim 41 or claim 60, are rejected as well.
Claims 41 and 60 are further rejected under additional grounds. “…the trabecular part of the input bone” lacks antecedent basis, as no trabecular part of the input bone was recited prior, and x-rays of bones do not necessarily include trabecular portion. Accordingly, the claims are rejected. All remaining claims, incorporating either the language of claim 41 or claim 60, are rejected as well.
Claims 53 and 72 are further rejected under additional grounds. “…the scores obtained from the first type of training image and the scores obtained from the second type of training image” is indefinite, as a plurality of “scores” are recited within the language of the claims and their chains of dependency, including scores which are only obtained optionally under and/or clauses. As such it is unclear if some are used, or all are used. Accordingly, the claims are rejected.
Claim 74 is further rejected under additional grounds, as “first artificial intelligence” and “second artificial intelligence” lack antecedent basis under its chain of dependency.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 79-80 are rejected under 35 U.S.C. 101 because the claims do not appear directed to any of the four statutory categories, and instead appear directed to non-statutory subject matter, specifically software and signals per se. See MPEP § 2106.03(I), “Non-limiting examples of claims that are not directed to any of the statutory categories include…a computer program per se (often referred to as “software per se”) …Transitory forms of signal transmission (often referred to as “signals per se”), such as a propagating electrical or electromagnetic signal or carrier wave”.
In the context of the flowchart illustrated in MPEP § 2106(III), claims 79-80 fail at Step 1 of the Subject Matter Eligibility test (Step 1: No). As relevant, pages 12 and 52 of Applicant’s Specification does not explicitly disclaim transitory signals or a computer program without any tangible form. Exemplary/non-limiting embodiments, even if generally understood to be ‘non-transitory’ in nature, do not serve to preclude an interpretation covering a transitory signal/medium embodiment or software per se, which does not fall within the definition of a process, machine, manufacture or composition of matter (In re Nujiten, 500 F.3d 1346, 1354, 84 USPQ2d 1495, 1500 (Fed. Cir. 2007)).
Accordingly, the claims are rejected.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 41-42, 44, 58, 60-61, 63, 77, and 79-80 are rejected under 35 U.S.C. 103 as being unpatentable over Hsieh et. al Automated bone mineral density prediction and fracture risk assessment using plain radiographs via deep learning (Hereinafter, “Hsieh”) in view of Kopperdahl et. al (US 20180116584 A1) (Hereinafter, “Kopperdahl”)
With respect to claim 41, Hsieh teaches:
A method for analyzing a texture of a bone, comprising:
receiving a digitized input x-ray image showing an input bone (Fig. 1)
a bone score analysis of the received input x-ray image by a bone score artificial intelligence implemented by technical means, the bone score artificial intelligence giving as a result of this bone score analysis, but without any calculation or determination of an experimental variogram of the gray levels of the received input x-ray image ([Abstract] “We present an automated tool to identify fractures, predict BMD, and evaluate fracture risk using plain radiographs”; Fig. 1: Table 2; Fig. 2)
a global score [Results] “Table 3 illustrates the discriminatory performance of the model to classify hip or spine osteoporosis, and identify patients with greater 10-year risks of major osteoporotic fractures (≥20%) and hip fractures (≥3%)”)
Hsieh does not explicitly teach:
depending at least on a trabecular bone score
However, Kopperdahl, in the same field of endeavor of bone assessment, teaches:
A method for analyzing a texture of a bone, comprising:
receiving a digitized input x-ray image showing an input bone (Fig. 2; [0069] “One specific embodiment, as depicted in FIG. 2, is a system 100 in which a medical image or scan 110 of a patient's bone or portion thereof is received and used to provide results for the classification”)
a bone score analysis of the received input x-ray image [0010] “In development in academia for over 20 years, the BCT method combines image processing of clinical CT scans, bone biomechanics, and the engineering computational mechanics technique of finite element analysis to provide a “virtual stress test” of a bone. The primary outcome of the test is an estimate of the strength (in units of Newtons) of the whole bone or a portion thereof, for example, a femur, a vertebral body, or a proximal femur”; [0071] “and, a strength-type measure could be calculated as the product of said area and a density squared of the bone contained within the area”; [0072]; “The finite element model is virtually subjected to certain loading conditions in order to obtain a measurement of strength or another structural parameter for that loading condition”; [0078] “One such correspondence is a linear regression 610 between femoral strength for a simulated sideways fall, obtained by a virtual stress test using finite element analysis of CT scans, and the femoral neck BMD T-score, obtained from quantitative analysis of the same CT scans…”) of the received input x-ray image (Fig. 5; [0018] “The method can also be applied to define and use interventional threshold values for any outcomes of a patient-specific finite element analysis of a patient's CT scan, for example, but not limited to, a strength, a stiffness, or a load-to-strength ratio, or for any other outcomes of any type of patient-specific bone structural or biomechanical analysis performed on any type of medical image, not necessarily a finite element analysis, and not necessarily a CT scan”; [0064]; [0075]; [0081] “Step 440 in system 400 determines an overall fracture risk classification based on classifications of both osteoporosis and fragile-bone-strength”)
a global score depending at least on a trabecular bone score ([0071] “or a trabecular bone score (often abbreviated as TBS)”) which quantifies the local variations in gray levels from the experimental variogram of the gray levels of the trabecular part of the input bone showed on the received x-ray image (Fig. 5; Fig. 8; Fig. 9; [0011] “A comparison of DXA and CT based methods for estimating the strength of the femoral neck in post-menopausal women. Osteoporos Int, 24:1379-88, 2013) and Pothuaud (Pothuaud L, Carceller P, Hans D: Correlations between grey-level variations in 2D projection images (TBS) and 3D microarchitecture: applications in the study of human trabecular bone microarchitecture. Bone 42:775-87, 2008)—all of which are expressly included herein in their entireties by reference”; [0071]; [0078] “The statistical correspondence used to derive strength-based interventional threshold values for fragile bone from the BMD-based interventional threshold values for osteoporosis can take various forms, as shown in FIGS. 7 and 8. One such correspondence is a linear regression 610 between femoral strength for a simulated sideways fall, obtained by a virtual stress test using finite element analysis of CT scans, and the femoral neck BMD T-score, obtained from quantitative analysis of the same CT scans…While femoral neck BMD T-score is preferable for defining osteoporosis in a hip fracture application, other BMD measures may be more appropriate depending on the anatomic site and the type of fracture. For example, for a spine fracture application, volumetric trabecular BMD for the spine is known to be a better predictor than the DXA BMD T-score. One could also round down the threshold values, which would decrease sensitivity but increase specificity. Such an approach might be preferable at the spine, for which BMD tends to produce many false positives and thus an improved specificity would be clinically beneficial in reducing the number of patients who are treated but who may not need or benefit from treatment”; [0081]; [0092] discussing global score)
PNG
media_image1.png
779
1033
media_image1.png
Greyscale
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Hsieh to include the limitations of trabecular score inclusion, as taught by Kopperdahl. Doing so would provide an explicit means to derive the ultimate global score, which is the object of both references. Hsieh further expects similar values to be utilized, as shown by its citations to other research (Hsieh [Introduction] “In addition, the spatial resolution of radiographs is excellent, allowing the visualization of fine bone texture, which is correlated with bone density25 and can distinguish patients with osteoporotic fractures from controls25–27” (citing to Benhamou, C. L. et al. Fractal analysis of radiographic trabecular bone texture and bone mineral density: two complementary parameters related to osteoporotic fractures. J. Bone Miner. Res. 16, 697–704 (2001).) The systems readily integrate, as trabecular scores can be reasonably used as a more explicit means to accomplish the underlying goal of Hsieh.
With respect to claim 42, Hsieh and Kopperdahl teach:
The method according to claim 41, wherein the bone score artificial intelligence is a neural network ([Introduction] “Here, we proposed and validated a fully automated deep learning-based tool to 1. extract the hip and spine region of interests (ROIs), 2. identify hip fracture, VCF, or morphological abnormalities, 3. check the radiograph quality to ensure that implants and foreign bodies were absent from the ROIs, 4. predict BMD and estimate the probability of a fracture within the next 10 years based on the FRAX (Fig. 1).”; [Methods]; [Introduction] “Therefore, this retrospective cohort study was performed to test the hypothesis that an automated deep neural network-based tool could effectively predict BMD and risk of fragility fractures using plain radiographs of the pelvis and lumbar spine”)
With respect to claim 44, Hsieh and Kopperdahl teach:
The method according to claim 41, comprising:
constructing a training set by implementing several times (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] discussing training loss) the following steps:
obtaining a first type of training image showing a trabecular part of a training bone (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh Fig. 1 (noting that trabecular part is visible under BRI); Kopperdahl [0071] “a DXA scan could be analyzed to measure a hip femoral neck axis length, a buckling ratio, or a trabecular bone score (often abbreviated as TBS)”)
obtaining an associated second type of training image that is an x-ray based image, showing the same training bone (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “During training, ROIs were augmented by random affine transformation and subsequently resized to 512 × 512 pixels”; Hsieh, Fig. 1; Kopperdahl [0071])
determining, by technical means, from the first type of training image
a density score depending on a bone mineral density of the training bone showed on the first type of training image (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh, Fig. 1), and
a trabecular bone score depending on a texture of the trabecular part of the training bone showed on the first type of training image (Kopperdahl [0071] “a DXA scan could be analyzed to measure a hip femoral neck axis length, a buckling ratio, or a trabecular bone score (often abbreviated as TBS)”)
training the bone score artificial intelligence by providing to the bone score artificial intelligence the second type of training image with its associated ground truth comprising:
the density score determined for the training image of the first type associated with this training image of the second type Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “The L1 distance between the predicted BMD and the ground truth BMD obtained from DXA was regarded as the training loss”; Hsieh, Fig. 1)
With respect to claim 58, Hsieh and Kopperdahl teach:
The method according to claim 41, wherein the received input x-ray image is not a dual x-ray absorptiometry image, a peripheral quantitative computed tomography image and/or High Resolution peripheral quantitative computed tomography image, a computerized tomography image, or a quantitative ultrasound image. (Kopperdahl [0014] “Instead of using a patient-specific finite element analysis of a CT scan, various prior art methods for estimating bone strength or a bone structural parameter from any medical image (such as CT, MRI, ultrasound, or DXA), referenced above and incorporated herein, can also be used instead for this step”)
With respect to claim 60, it is functionally parallel to the method of claim 1, with the exception that it includes “means arranged to and/or programmed to and/or configured to receive a digitized input x-ray image showing an input bone”. Hsieh and Kopperdahl teach the broad high-level hardware necessary for the same (Kopperdahl [0093]-[0094]). Accordingly, the claim is rejected in line with the analysis above.
With respect to claims 61, 63, and 77, they are rejected in line with claims 60, 42, 44 and 58 respectively.
With respect to claim 79, Hsieh and Kopperdahl teach:
A computer program comprising instructions which, when executed in a computer, implement the steps of the method according to claim 41 (Kopperdahl [0093]-[0094])
With respect to claim 80, Hsieh and Kopperdahl teach:
A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the method according to claim 41 (Kopperdahl [0093]-[0094])
Claims 43, 55-57, 62 and 74-76 are rejected under 35 U.S.C. 103 as being unpatentable over Hsieh and Kopperdahl in view of Lee et. al Diagnosis of Osteoporosis by Quantification of Trabecular Microarchitectures from Hip Radiographs Using Artificial Neural Networks (Hereinafter, “Lee”)
With respect to claim 43, Hsieh and Kopperdahl teach:
The method according to claim 41, comprising:
constructing a training set by implementing several times (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]) the following steps:
obtaining a first type of training image showing a trabecular part of a training bone (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh Fig. 1 (noting that trabecular part is visible under BRI); Kopperdahl [0071] “a DXA scan could be analyzed to measure a hip femoral neck axis length, a buckling ratio, or a trabecular bone score (often abbreviated as TBS)”)
obtaining an associated second type of training image that is an x-ray based image, showing the same training bone (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “During training, ROIs were augmented by random affine transformation and subsequently resized to 512 × 512 pixels”; Kopperdahl [0071])
determining, by technical means, from the first type of training image
a density score depending on a bone mineral density of the training bone showed on the first type of training image (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh, Fig. 1; Kopperdahl [0071]), and
a trabecular bone score depending on a texture of the trabecular part of the training bone showed on the first type of training image (Kopperdahl [0071])
determining, by technical means, from:
the density score depending on a bone mineral density of the training bone showed on the first type of training image (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh, Fig. 1), and the trabecular score depending on a texture of the trabecular part of the training bone showed on the first type of training image (Kopperdahl [0071])
a global score depending on these density score and trabecular score (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “Ten-year probabilities of major fracture and hip fracture with total hip BMD were calculated for each patient using the FRAX tool with risk estimators specific to the Taiwanese population..”; Kopperdahl [0071]; Kopperdahl Fig. 5, noting that “Measure Strength” and “Measure BMD” are combined to create “Overall Risk Classification” 440; Kopperdahl [0011] where it is known that measures of trabecular microarchitecture constitute “structural measures associated with the strength and structure of the bone…”)
training the bone score artificial intelligence by providing to the bone score artificial intelligence the second type of training image with its associated ground truth comprising the Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “The L1 distance between the predicted BMD and the ground truth BMD obtained from DXA was regarded as the training loss”; Hsieh, Fig. 1)
Hsieh and Kopperdahl do not explicitly teach:
comprising the global score determined for the training image of the first type associated with this training image of the second type
As they teach the bone mineral density prediction’s optimization rather than explicitly the fracture risk value
However, Lee, in the same field of endeavor of bone diagnosis, teaches:
constructing a training set by implementing several times ([2]) the following steps:
obtaining a first type of training image showing a trabecular part of a training bone ([2])
obtaining an associated second type of training image that is an x-ray based image, showing the same training bone ([2], image after augmentation)
determining, by technical means, from the first type of training image
a trabecular bone score depending on a texture of the trabecular part of the training bone showed on the first type of training image ([2])
PNG
media_image2.png
772
706
media_image2.png
Greyscale
determining, by technical means, from:
a global score depending on the[3] “The overall classification accuracies of the SVM classifiers for osteoporosis were high when compared with the ANN classifiers (Table 1)”)
training the bone score artificial intelligence by providing to the bone score artificial intelligence the second type of training image [Abstract] “For the classification, a two-layered feed forward ANNs was designed using the Levenberg-Marquardt training algorithm”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Hsieh and Kopperdahl to include the limitations of global score optimization, as taught by Lee. Doing so would provide a more direct means to optimize the final value, as compared to Hsieh/Kopperdahl. The systems readily integrate, as Lee represents merely the optimization of a downstream value Hsieh is otherwise configured to compute. A person of ordinary skill in the art would understand that optimizing the output of a function (rather than a key input within the function itself) could reasonably lend itself to a predictable improvement.
With respect to claim 55, Hsieh, Kopperdahl, and Lee teach:
The method according to claim 43, wherein the first type of training image and the second type of training image are acquired on the same training bone are acquired less than 6 months apart (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Lee [2]; noting that under claim construction, the image modifications of Hsieh or Lee result in a “second type of training image” in the exact instant the images are modified, which in digital processing are near instantaneous and thus “less than 6 months apart”)
With respect to claim 56, Hsieh, Kopperdahl and Lee teach:
The method according to claim 43, wherein the first type of training image is a dual x-ray absorptiometry image (Hsieh [Results, Data source]; Kopperdahl [0011]; Lee [2])
With respect to claim 57, Hsieh, Kopperdahl and Lee teach:
wherein the second type of training image is not a dual x-ray absorptiometry image, a peripheral quantitative computed tomography image and/or High Resolution peripheral quantitative computed tomography image, a computerized tomography image, or a quantitative ultrasound image. (Kopperdahl [0014] “Instead of using a patient-specific finite element analysis of a CT scan, various prior art methods for estimating bone strength or a bone structural parameter from any medical image (such as CT, MRI, ultrasound, or DXA), referenced above and incorporated herein, can also be used instead for this step”)
With respect to claims 62 and 74-76, they are rejected in line with claims 60, 43, and 55-57 respectively.
Claims 45-50, 52, 64-69 and 71 are rejected under 35 U.S.C. 103 as being unpatentable over Hsieh and Kopperdahl in view of Nissinen et. al Detecting pathological features and predicting fracture risk from dual-energy X-ray absorptiometry images using deep learning (Hereinafter, “Nissinen”) and Sobti et. al (US 20220130535 A1) (Hereinafter, “Sobti”)
With respect to claim 45, Hsieh and Kopperdahl teach:
The method according to claim 41, comprising:
receiving the input x-ray image showing an input bone (Hsieh, Fig. 1)
a first analysis of the received input x-ray image by a first artificial intelligence implemented by technical means, the first artificial intelligence giving as a result of the first analysis
a global score depending both:
on a density score depending on a bone mineral density of the input bone showed on the received input x-ray image (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Kopperdahl [0011]; Kopperdahl, Fig. 5; Kopperdahl, [0071])
on a trabecular bone score depending on a texture of the trabecular part of the input bone showed on the received input x-ray image (Kopperdahl [0011]; Kopperdahl, Fig. 5; Kopperdahl, [0071])
Hsieh and Kopperdahl do not explicitly teach:
a second analysis of the received input x-ray image by a second artificial intelligence implemented by technical means, the second artificial intelligence giving as a result of the second analysis
the density score depending on a bone mineral density of the input bone showed on the received input x-ray image, and/or
the trabecular bone score depending on a texture of the trabecular part of the input bone showed on the received input x-ray image
a third analysis by a third artificial intelligence implemented by technical means, the third artificial intelligence having as input the results of the first and second analysis and having as output a result depending on the consistency between the result of the first analysis and the result of the second analysis
However, Nissinen, in the same field of endeavor of bone estimation, teaches:
receiving the input x-ray image showing an input bone (Fig. 3)
a first analysis of the received input x-ray image by a first artificial intelligence implemented by technical means, the first artificial intelligence giving as a result of the first analysis (Fig. 1)
a global score depending both:
on a density score depending on a bone mineral density of the input bone showed on the received input x-ray image (Fig. 3; [3.3] “The fracture prediction produced an average AUC of 0.63.”; Table 1)
on a trabecular bone score depending on a texture of the trabecular part of the input bone showed on the received input x-ray image (Fig. 3; [2.3] “For benchmarking the deep learning approach in fracture prediction, we built logistic regression models based on one or more predictor variables. The selected variables were the lumbar spine BMD T-score (the minimum of vertebrae L1-L4), the hip BMD T-score (the minimum of the femoral neck or femoral total), the minimum BMD T-score (from spine or hip), TBS (the average of vertebrae L1-L4), and the age of the patient. In addition, the fracture prediction outputs from the deep learning model were included as one variable. This enabled us to analyze the deep learning output’s significance when used together with the other predictors”; [3.3] “The logistic regression model with the minimum BMD T-score together with TBS and age did not improve the performance (AUC 0.63) compared to the minimum T-score alone. The minimum T-score combined with the prediction probabilities from the deep learning model improved the AUC to 0.64. The coefficient analysis showed that in this model the minimum T-score (p =0.006) was statistically significant (p < 0.05) whereas the deep learning output (p =0.158) was not. However, when the deep learning output (p =0.041) was combined with the spine T-score (p =0.028), they both remained significant. TBS did not improve the prediction and was not statistically significant when combined with any of the BMD T-score predictors or deep learning output”; Table 1)
a second analysis (the examiner choosing the earlier analysis, noting that “second” as a label is without effect as to sequence) of the received input x-ray image [2.1] “In clinical use, the DXA device produces a report including the segmented images and measured BMD values”; Fig. 1), noting that while the DXA device does not explicitly utilize “artificial intelligence” in generating its report, it is known that a plurality of candidate “second” artificial intelligences are capable of deriving BMD from an image and could be used for the same, see Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “We evaluated multiple backbone networks (i.e., VGG-11, VGG-16, ResNet-18, ResNet-34) in earlier experiments and empirically found that VGG-16 and ResNet-34 produce the best BMD prediction results for spine and hip BMD prediction, respectively”)
the density score depending on a bone mineral density of the input bone showed on the received input x-ray image (Fig. 1; “In DXA measurement, the bone mineral density is calculated for each semi-automatically segmented vertebra (L1-L4)”),
a third analysis [3.3] “The coefficient analysis showed that in this model the minimum T-score (p =0.006) was statistically significant (p < 0.05) whereas the deep learning output (p =0.158) was not. However, when the deep learning output (p =0.041) was combined with the spine T-score (p =0.028), they both remained significant. TBS did not improve the prediction and was not statistically significant when combined with any of the BMD T-score predictors or deep learning output”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Hsieh and Kopperdahl to include the limitation of output comparisons as taught by Nissinen. Doing so would provide additional information as to the effect of each calculation on the overall score. The systems readily integrate, as Hsieh/Kopperdahl already encourages the consideration of multiple parameters. A person of ordinary skill in the art would understand that, to optimize efficiency, a parameter without statistical significance should be discard and methods should be incorporated to achieve such ends.
Hsieh, Kopperdahl, and Nissninen do not explicitly teach:
a third artificial intelligence [for performing statistical analysis]
As Nissinen’s coefficient analysis is not explicitly performed by an artificial intelligence
However, Sobti, in the same field of endeavor of diagnostic testing, teaches:
a third artificial intelligence [for performing statistical analysis] ([0068])
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Hsieh, Kopperdahl and Nissinen to include the limitations of a third artificial intelligence, as taught by Sobti. Doing so would have the advantage of performing the coefficient analysis of Hsieh/Kopperdahl/Nissinen with increased computational efficiency. The systems readily integrate, as the underlying methods understand the usage of AI for a wide variety of statistical tasks.
With respect to claim 46, Hsieh, Kopperdahl, Nissinen, and Sobti further teach:
The method according to claim 45, wherein the third artificial intelligence uses as further input at least one parameter among: age of the patient on whom the received input x-ray image was acquired (Nissinen, Table 1 “Age”; [3.3] discussing coefficient analysis. Statistically significant inputs were labeled with an asterisk in Table 1 but all coefficients determined not to be significant were otherwise analyzed as well, necessarily or at least predictably including age as listed in [3.3] and Table 1)
With respect to claim 47, Hsieh, Kopperdahl, Nissinen, and Sobti further teach:
The method according to claim 45, wherein the first artificial intelligence is a neural network and the second artificial intelligence is a neural network (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction])
With respect to claim 48, Hsieh, Kopperdahl, Nissinen, and Sobti further teach:
The method according to claim 45, wherein the first artificial intelligence and the second artificial intelligence and the third artificial intelligence are three distinct artificial intelligences (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Sobti [0068])
With respect to claim 49, Hsieh, Kopperdahl, Nissinen, and Sobti further teach:
The method according to claim 45, wherein the technical means for implementing the first and second and third artificial intelligences are the same technical means (technical means is read in line with Page 25 of the claimed invention’s specification. Hsieh [Methods, Setting] “Nvidia Triton architecture”; Sobti [0111]-[0112] discussing computer implementation)
With respect to claim 50, Hsieh, Kopperdahl, Nissinen, and Sobti further teach:
The method according to claim 45, comprising:
constructing a training set by implementing several times (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Nissinen [2.3] “over training epochs (iterations over the training set)”) the following steps:
obtaining a first type of training image showing a trabecular part of a training bone (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh Fig. 1 (noting that trabecular part is visible under BRI); Kopperdahl [0071] “a DXA scan could be analyzed to measure a hip femoral neck axis length, a buckling ratio, or a trabecular bone score (often abbreviated as TBS)”)
obtaining an associated second type of training image that is an x-ray based image, showing the same training bone (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “During training, ROIs were augmented by random affine transformation and subsequently resized to 512 × 512 pixels”; Kopperdahl [0071])
determining, by technical means, from the first type of training image
a density score depending on a bone mineral density of the training bone showed on the first type of training image (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh, Fig. 1; Kopperdahl [0071]), and
a trabecular bone score depending on a texture of the trabecular part of the training bone showed on the first type of training image (Kopperdahl [0071])
determining, by technical means, from:
the density score depending on a bone mineral density of the training bone showed on the first type of training image (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction]; Hsieh, Fig. 1), and the trabecular score depending on a texture of the trabecular part of the training bone showed on the first type of training image (Kopperdahl [0071])
a global score depending on these density score and trabecular score (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “Ten-year probabilities of major fracture and hip fracture with total hip BMD were calculated for each patient using the FRAX tool with risk estimators specific to the Taiwanese population..”; Kopperdahl [0071]; Kopperdahl Fig. 5, noting that “Measure Strength” and “Measure BMD” are combined to create “Overall Risk Classification” 440; Kopperdahl [0011] where it is known that measures of trabecular microarchitecture constitute “structural measures associated with the strength and structure of the bone…”)
training the bone score artificial intelligence by providing to the bone score artificial intelligence the second type of training image with its associated ground truth comprising the global score (similar to rejection of claim 43 above, while Hsieh does not optimize based on the global score, optimization off the “global score” can be found in Nissinen; Nissinen, Fig. 7; Nissinen, [3.2] “The learning curves (Fig. 7(c)) show that the model slowly learns the optimal weights before heavy overfitting starts after 60 epochs”. The rationale to combine remains the same as in claim 43.) determined for the training image of the first type associated with this training image of the second type (Hsieh, [Methods, Algorithm development and training procedure for BMD prediction] “The L1 distance between the predicted BMD and the ground truth BMD obtained from DXA was regarded as the training loss”; Hsieh, Fig. 1)
With respect to claim 52, Hsieh, Kopperdahl, Nissinen, and Sobti further teach:
The method according to claim 50, wherein the first artificial intelligence and the second artificial intelligence are trained using a same database of first type of training images and second type of training images (Nissinen, [2.1]; Nissinen, [2.3] “The OSTPRE dataset was split into 10 random subsets using stratified sampling to retain the same class distribution across all subsets. To obtain predictions for the whole dataset, the model was trained 10 times holding out different subset for validation each time. This process was again repeated 100 times using different random seeds. Mean performance measures and confidence intervals were calculated from the resulting 1000 iterations. The method ensures that, during these iterations, the validation patients are never present in the respective training set. Using repeated 10-fold cross-validation aims at a better estimation of the model skill independent of the split to training and validation samples.”; Table 1, “Repeated 10-fold performance results in scoliosis detection, unreliability detection, and fracture prediction using the OSTPRE dataset”; Nissinen, [3.2]; Nissinen, Fig. 2)
With respect to claims 64-69 and 71, they are rejected in line with claims 60, 45-50 and 52 respectively.
Claims 54 and 73 are rejected under 35 U.S.C. 103 as being unpatentable over Hsieh, Kopperdahl, Nissinen, and Sobti in view of Nichols et. al Machine learning: applications of artificial intelligence to imaging and diagnosis (Hereinafter, “Nichols”)
With respect to claim 54, Hsieh, Kopperdahl, Nissinen, and Sobti further teach:
The method according to claim 50, wherein the first, second and third artificial intelligences are trained separately (across all references, distinct embodiments are disclosed that could serve as “first”, “second” or third” artificial intelligences respectively, and each have their requisite training. Wherein the first and second may correspond to Hsieh for reasons outlined above, the second to Nissinen, and the third to Sobti)
As Sobti’s disclosure does not explicitly rule out the remote possibility that it was trained with a pipeline identical to Hsieh or Nissinen, the examiner cites Nichols to show that a wide variety of training methods exist for artificial intelligence applications, which can vary in both datasets chosen, dataset size, bias, and supervision. These represent obvious differences a person of ordinary skill in the art would understand, and accordingly implement depending on the precise nature of the tasks involved.
With respect to claim 73, it is rejected in line with claims 60 and 54 respectively.
Claims 59 and 78 are rejected under 35 U.S.C. 103 as being unpatentable over Hsieh and Kopperdahl in view of Park et. al (US 20090148022 A1) (Hereinafter, “Park”)
With respect to claim 59, Hsieh and Kopperdahl teach:
The method according to claim 41, wherein the received input x-ray image is a digital x-ray image (Hsieh, [Results, Data source]; Kopperdahl [0067] “As used herein, unless qualified specifically, the terms “medical image” or “scan” or “exam” are used to mean substantially the same thing, namely, some type of digital image of a body part, typically taken of a live patient for some medical purpose”)
Hsieh and Kopperdahl do not explicitly teach:
having a spatial resolution of less than 1mm per pixel
However, Park, in the same field of endeavor of x-ray imaging, teaches:
wherein the received input x-ray image is a digital x-ray image, having a spatial resolution of less than 1mm per pixel ([0036] “In the example illustrated in FIG. 2, a spatial resolution of the X-ray image 21 is set to 200 pixels per inch (PPI), and 256 (8-bit) gray levels are used”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Hsieh and Kopperdahl to include the limitations of spatial resolution, as taught by Park. Doing so would have the advantage of utilizing information-rich images. The systems readily integrate, as Hsieh and Kopperdahl are already configured to receive digital x-ray images.
With respect to claim 78, it is rejected in line with claims 60 and 58 respectively.
Allowable Subject Matter
Claims 51, 53, 70, 72 would be allowable if rewritten to overcome the rejections under 35 U.S.C.112(b) set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
With respect to claims 51 and 70, it would not have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify the cited references to arrive at the recited limitations. Namely as the “second artificial intelligence” within intervening claim 45 was employed towards a fundamentally different purpose. To the extent Nissinen was modified to include a second artificial intelligence already required a significant degree of modification, since it the same was not explicitly disclosed within Nissinen. To further modify the second artificial intelligence to perform the iterative steps of claim 51 would constitute impermissible hindsight.
With respect to claims 53 and 72, it would not have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify the cited references to arrive at the recited limitations. The third artificial intelligence, which is a product of Nissinen and Sobti in particular, performs under claim 45 a basic one-time statistical analysis. Iteratively training the third network under the specific pipeline of claim 53 would result in further modifications that would rise to the level of impermissible hindsight.
Additional References
Additionally cited references (see attached PTO-892) otherwise not relied upon above have been made of record in view of the manner in which they evidence the general state of the art.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH WILLIAM BOYAR whose telephone number is 571-272-8392. The examiner can normally be reached 10:00 – 6:00 EST, Monday – Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH W BOYAR/Examiner, Art Unit 2669
/IAN L LEMIEUX/Primary Examiner, Art Unit 2669
Prosecution Insights