DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Responsive to the communication dated 01/12/2026
Claims 1-20 are presented for examination
Drawings
The drawings dated 02/08/2022 have been reviewed. They are accepted.
Abstract
The abstract dated 02/08/2022 has been reviewed. It has 121 words, and contains no legal phraseology. It is accepted.
Finality
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Response to Arguments - 112
Applicant's arguments filed 01/12/2026 have been fully considered but they are not fully persuasive.
Applicant argues that the use of the term ‘dynamic’ is not indefinite.
Upon further consideration, the examiner is inclined to agree, and the rejection for this specific purpose is withdrawn. However, see the below for maintained 112 rejections.
Applicant argues that the term “low flip angle” should not be indefinite in view of the disclosure.
Examiner responds by explaining that this term is still considered indefinite. The specification gives an example of 5 degrees as an example of a flip angle, but the wording of the specification makes it clear that this is not the same as the “low flip angle.” See [Par 49] of the specification that recites “In a preferred embodiment, the black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet. Also, the MRI can utilize an echo sequence with a low flip angle gradient providing high contrast between bone and tissue.” If the use of a low flip angle is an alternative to the 5 degrees, evidenced by the word “also,” then the 5 degrees is not the only embodiment of the low flip angle. There still are not sufficiently described metes and bounds for the term. What is the limit of what can be considered a “low” flip angle? Is 5.000001 degrees to high? 10 degrees? 20? Without defined boundaries on what is considered “low,” this term renders the claims indefinite.
Applicant argues that the term “high contrast” is not indefinite in view of the disclosure.
Examiner responds by explaining that, in view of the disclosure, this term is still considered indefinite. No evidence is present or has been provided that one of ordinary skill in the art would have recognized definite metes and bounds for this term. What is the cutoff for contrast between bone and other tissue to be considered “high?” For example, if there is a 50% intensity difference between bone and other tissue, is this enough contrast to be considered “high?” Is a 10% difference enough? 1%? Without defined boundaries on what level of contrast is considered “high,” this term renders the claims indefinite.
Further, see the new objections relating to the issues caused by the removal of the term “similar” from the claims without appropriately adjusting the claim language.
Response to Arguments - 101
Applicant's arguments filed 01/12/2026 have been fully considered but they are not persuasive.
Applicant argues that the generation of virtual models cannot be performed in the human mind.
Examiner responds by firstly explaining that very little is specified about the structure of the model. With such a broad limitation, generating such a tissue model from medical imagery or a bone model from the black bone data set amounts to no more than observing the data and drawing a representation of it with a pencil and paper.
Doing this for a “dynamic virtual” model amounts to no more than instructions to apply this mental process on a general purpose computer.
Applicant argues that generating a user interface and displaying it cannot be considered abstract.
Examiner responds by explaining that such a user interface could be created mentally by drawing a representation of the interface with a pen and paper; these kinds of interface mockups are frequently used by graphic designers. Further, performing this mental process of generating the interface and displaying it on a computer amounts to no more than mere instructions to apply.
Additionally, should it be found that the displaying is not mere instructions to apply, it is also an example of insignificant post-solution activity, as it merely presents the results of the abstract idea.
Response to Arguments - 103
Applicant's arguments filed 01/12/2026 have been fully considered but they are not persuasive.
Applicant argues that no prior art teaches preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; … generating a dynamic virtual bone model of the bone structures of the particular patient from the processed black bone dataset, wherein said virtual bone model is configured to highlight bone structures of the patient; providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Examiner responds by explaining that these features are taught by the previously cited prior art references in combination with new reference Farley (US 12144551 B1). In particular:
Elay_2020 makes obvious
generating a dynamic virtual bone model of the bone structures of the particular patient from the processed black bone dataset, wherein said virtual bone model is configured to highlight bone structures of the patient; ([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
PNG
media_image1.png
386
917
media_image1.png
Greyscale
([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
Farley makes obvious preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; ([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only … only soft tissue structures…”)
generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to a bone model to create a combined dynamic virtual model showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only bony anatomy, only soft tissue structures, or it may combine both into a single 3D model.” [Col 39 line 13-20] “Various embodiments are described herein that relate to using an augmented reality navigation system to aid in viewing patient anatomy (e.g., viewing soft tissue, bony landmarks, and/or the combination of both.). The system generally includes a navigation system, one or more displays, including but not limited to, HMDs, with onboard navigation capabilities and instrumentation tracking to target tool trajectories relative to the plating system.” [Col 9 line 22-40] “The Display 125 provides graphical user interfaces (GUIs) that display images collected by the Tissue Navigation System 120 as well other information relevant to the surgery. For example, in one embodiment, the Display 125 overlays image information collected from various modalities (e.g., CT, MRI, X-ray, fluorescent, ultrasound, etc.) collected pre-operatively or intra-operatively to give the surgeon various views of the patient's anatomy as well as real-time conditions. The Display 125 may include, for example, one or more computer monitors. As an alternative or supplement to the Display 125, one or more members of the surgical staff may wear an Augmented Reality (AR) Head Mounted Device (HMD). For example, in FIG. 1 the Surgeon 111 is wearing an AR HMD 155 that may, for example, overlay pre-operative image data on the patient or provide surgical planning suggestions. Various example uses of the AR HMD 155 in surgical procedures are detailed in the sections that follow.”)
Farley is analogous art because it is within the field of medical imaging. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017, Hayashi, and Elay_2020 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to make more accurate measurements. As noted by Farley, a known issue is medical imaging is the effect of electromagnetic interference from other devices causing inaccuracy in measurements and instrument readings ([Col 7 line 59 – Col 8 line 2] “In another embodiment, the above systems may allow for a multi-sensor navigation system that can detect and correct for field distortions that plague electromagnetic tracking systems. It should be understood that field distortions may result from movement of any ferromagnetic materials within the reference field. Thus, as one of ordinary skill in the art would know, a typical OR has a large number of devices (e.g., an operating table, LCD displays, lighting equipment, imaging systems, surgical instruments, etc.) that may cause interference. Furthermore, field distortions are notoriously difficult to detect.”) To this end, Farley presents a method for detecting and warning of such distortions, allowing for inaccurate instrument readings to be avoided ([Col 8 line 2-12] “The use of multiple EM sensors enables the system to detect field distortions accurately, and/or to warn a user that the current position measurements may not be accurate. Because the sensors are rigidly fixed to the bony anatomy (e.g., via the pin/needle), relative measurement of sensor positions (X, Y, Z) may be used to detect field distortions. By way of non-limiting example, in some embodiments, after the EM sensors are fixed to the bone, the relative distance between the two sensors is known and should remain constant. Thus, any change in this distance could indicate the presence of a field distortion.”) Overall, one of ordinary skill in the art would have recognized that combining Farley with Elay_2017, Hayashi, and Elay_2020 would result in a system that allowed for the detection of conditions that would cause the imaging process to be inaccurate, ultimately enabling changes to be made and such inaccuracies to be avoided, resulting in an overall more accurate system.
Claim Objections
Claims 1-20 objected to because of the following informalities:
The amendments to claims 3, 8, 14, and 19 remove the word “similar” without properly adjusting the claim language resulting in phrases like “detects the pixels of intensity range to that of bone.” This does not make sense; it is recommended to amend the claims to read something like “detects the pixels of intensity range related to that of bone.” or “detects the pixels of intensity range the same as that of bone.”
Claims 1, 10, 14, and 19 recite “the patient.” It is clear that this is meant to refer to the particular patient introduced prior, and should be amended to reflect this to avoid any potential issues with antecedent basis.
Claim 19 recites the “bone dtrucctures…” It is clear that this meant to read “bone structures” and should be amended as such.
Claims 10, 11, 17 and 19-20 recite “said virtual model.” Within the context of the amendments it is clear that this is meant to refer to the “virtual bone model,” and should be amended to reflect such.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4, 6, and 14-18 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “low flip angle” in claims 4, 6, and 14 is a relative term which renders the claim indefinite. The term “low flip angle” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. What is the cutoff for an angle being too high to be considered a “low” flip angle? Later claims state that the angle used by the system is 5 degrees, but is 10 degrees still “low?” 20 degrees? 45 degrees? Without defined boundaries on what is considered “low,” this term renders the claims indefinite.
The term “high contrast” in claims 4 and 6 is a relative term which renders the claim indefinite. The term “high contrast” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. What is the cutoff for contrast between bone and other tissue to be considered “high?” For example, if there is a 50% intensity difference between bone and other tissue, is this enough contrast to be considered “high?” Is a 10% difference enough? 1%? Without defined boundaries on what level of contrast is considered “high,” this term renders the claims indefinite.
For the purposes of this examination, “high contrast” is considered any amount of contrast between the bone and other materials, including the complete visual removal/suppression of non-bone materials.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because they are directed to an abstract idea without significantly more.
Claim 1 (Statutory Category – Process)
Step 2A – Prong 1: Judicial Exception Recited?
Yes, the claim recites a mental process, specifically:
MPEP 2106.04(a)(2)(Ill): “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, Judgments, and opinions.”
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation.”
A method for processing a black bone MRI dataset into a virtual model, comprising the steps of: preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; … generating a dynamic virtual bone model of the bone and tissue of the patient from the processed black bone dataset.
Generating a model of bone and/or tissue is a mental process equivalent to drawing such a model. For example, a person could look at medical imagery and make a drawn model based on those observations using a pencil and paper. Specifying that this is a “dynamic virtual” model amounts to no more than mere instructions to apply the exception.
processing the black bone dataset;
It is clear from the specification and based on the other claims that this processing consists of inverting the data set. (See [Par 36] of the specification “The post processing software first inverts the dataset”) Such simple flipping of data is simple enough to perform in the human mind. For example, given the data set {1,1,1,1} a person could invert it to create the data set {-1,-1,-1,-1}.
generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and
Generating such a combined model that includes both bone and soft tissue imagery is a mental process equivalent to observing the features of the tissue model and bone model, and drawing a new model that combines the features of both, for example maybe drawing an image that shows the skull but also the eyeballs. Specifying that this is a “dynamic virtual” model and that it is displayed on a user display amounts to no more than mere instructions to apply the exception.
Should it be found that the displaying is not mere instructions to apply, it is also an example of insignificant post-solution activity, as it merely presents the results of the abstract idea.
providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Such a user interface could be created mentally by drawing a representation of the interface with a pen and paper; these kinds of interface mockups are frequently used by graphic designers. Further, performing this mental process of generating the interface and displaying it on a computer amounts to no more than mere instructions to apply.
Step 2A – Prong 2: Integrated into a Practical Solution?
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
performing a black bone MRI on a bone and tissue of a particular patient; obtaining a black bone dataset of the patient from the black bone MRI;
Performing an MRI on a patient is merely the act of gathering data on that patient, and is therefore an example of mere insignificant data gathering.
Insignificant post-solution activity:
…showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user
Displaying imagery on generated through abstract means amounts to no more that presenting the results of the abstract idea, and is therefore merely insignificant post-solution activity.
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “a virtual model, a dynamic virtual mode, a dynamic virtual tissue model, a dynamic virtual bone model, a combined dynamic virtual model, displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitations are mere data gathering and do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
performing a black bone MRI on a bone and tissue of a particular patient; obtaining a black bone dataset of the patient from the black bone MRI;
Performing an MRI on a patient is merely the act of gathering data on that patient, and is therefore an example of mere insignificant data gathering.
A claim element that amounts to merely gathering data is not indicative of integration into a
practical solution nor evidence that the claim provides an inventive concept, as exemplified by ((MPEP
2106.05)(g)(Mere Data Gathering) i. Performing clinical tests on individuals to obtain input for an
equation, In re Grams, 888 F.2d 835, 839-40; 12 USPQ2d 1824, 1827-28 (Fed. Cir. 1989); iv. Obtaining
information about transactions using the Internet to verify credit card transactions, CyberSource v.
Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011);
Insignificant post-solution activity:
…showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user … providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.”
Displaying imagery generated through abstract means amounts to no more that presenting the results of the abstract idea, and is therefore merely insignificant post-solution activity.
This element merely acts on the results of the previous abstract steps. A claim element that merely acts on a series of previous abstract steps is not indicative of integration into a practical solution nor evidence that the claim provides an inventive concept, as exemplified by ((MPEP 2106.05)(g)(Insignificant application) i. Cutting hair after first determining the hair style, In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016) and ii. Printing or downloading generated menus, Ameranth, 842 F.3d at 1241-42, 120 USPQ2d at 1854-55.)
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “a virtual model, a dynamic virtual mode, a dynamic virtual tissue model, a dynamic virtual bone model, a combined dynamic virtual model, displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
The additional elements have been considered both individually and as an ordered combination in the consideration of whether they constitute significantly more, and have been determined not to constitute such.
The claim is ineligible.
Claim 14 (Statutory Category – Process)
Step 2A – Prong 1: Judicial Exception Recited?
Yes, the claim recites a mental process, specifically:
MPEP 2106.04(a)(2)(Ill): “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, Judgments, and opinions.”
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation.”
A method for processing a black bone MRI dataset into a virtual model, comprising the steps of: preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; … generating a dynamic virtual bone model of the bone structures of the particular patient from the processed black bone dataset, wherein said virtual bone model is configured to highlight bone structures of the patient.
Generating a model of bone and/or tissue is a mental process equivalent to drawing such a model. For example, a person could look at medical imagery and make a drawn model based on those observations using a pencil and paper. Specifying that this is a “virtual” model amounts to no more than mere instructions to apply the exception.
processing the black bone dataset utilizing an auto detection algorithm that detects the pixels of intensity range to that of bone from the black bone dataset;
A person could perform this kind of detection mentally by observing at imagery that has been magnified so individual pixels are visible and, by comparing the color of observed pixels to a known color of bone, judge which pixels are depicting part of a bone.
Using an auto detection algorithm to perform this operation amounts to no more than mere instructions to apply the judicial exception.
generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and
Generating such a combined model that includes both bone and soft tissue imagery is a mental process equivalent to observing the features of the tissue model and bone model, and drawing a new model that combines the features of both, for example maybe drawing an image that shows the skull but also the eyeballs. Specifying that this is a “dynamic virtual” model and that it is displayed on a user display amounts to no more than mere instructions to apply the exception.
Should it be found that the displaying is not mere instructions to apply, it is also an example of insignificant post-solution activity, as it merely presents the results of the abstract idea.
providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Such a user interface could be created mentally by drawing a representation of the interface with a pen and paper; these kinds of interface mockups are frequently used by graphic designers. Further, performing this mental process of generating the interface and displaying it on a computer amounts to no more than mere instructions to apply.
Step 2A – Prong 2: Integrated into a Practical Solution?
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
performing a black bone MRI on a bone and tissue of a particular patient, wherein said black bone MRI utilizes an echo sequence with a low flip angle gradient between bone and tissue; obtaining a black bone dataset of the patient from the black bone MRI;
Performing an MRI on a patient is merely the act of gathering data on that patient, and is therefore an example of mere insignificant data gathering.
Insignificant post-solution activity:
showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user;
Displaying imagery on generated through abstract means amounts to no more that presenting the results of the abstract idea, and is therefore merely insignificant post-solution activity.
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “an auto detection algorithm, a virtual model, a dynamic virtual model; a dynamic virtual tissue model, a dynamic virtual bone model, a combined dynamic virtual model, displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitations are mere data gathering and do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
performing a black bone MRI on a bone and tissue of a particular patient, wherein said black bone MRI utilizes an echo sequence with a low flip angle gradient providing high contrast between bone and tissue; obtaining a black bone dataset of the patient from the black bone MRI;
Performing an MRI on a patient is merely the act of gathering data on that patient, and is therefore an example of mere insignificant data gathering.
A claim element that amounts to merely gathering data is not indicative of integration into a
practical solution nor evidence that the claim provides an inventive concept, as exemplified by ((MPEP
2106.05)(g)(Mere Data Gathering) i. Performing clinical tests on individuals to obtain input for an
equation, In re Grams, 888 F.2d 835, 839-40; 12 USPQ2d 1824, 1827-28 (Fed. Cir. 1989); iv. Obtaining
information about transactions using the Internet to verify credit card transactions, CyberSource v.
Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011);
Insignificant post-solution activity:
showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user;
Displaying imagery on generated through abstract means amounts to no more that presenting the results of the abstract idea, and is therefore merely insignificant post-solution activity.
This element merely acts on the results of the previous abstract steps. A claim element that merely acts on a series of previous abstract steps is not indicative of integration into a practical solution nor evidence that the claim provides an inventive concept, as exemplified by ((MPEP 2106.05)(g)(Insignificant application) i. Cutting hair after first determining the hair style, In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016) and ii. Printing or downloading generated menus, Ameranth, 842 F.3d at 1241-42, 120 USPQ2d at 1854-55.)
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “an auto detection algorithm, a virtual model, a dynamic virtual model; a dynamic virtual tissue model, a dynamic virtual bone model, a combined dynamic virtual model, displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
The additional elements have been considered both individually and as an ordered combination in the consideration of whether they constitute significantly more, and have been determined not to constitute such.
The claim is ineligible.
Claim 19 (Statutory Category – Process)
Step 2A – Prong 1: Judicial Exception Recited?
Yes, the claim recites a mental process, specifically:
MPEP 2106.04(a)(2)(Ill): “Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, Judgments, and opinions.”
Further, the MPEP recites “The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation.”
A method for processing a black bone MRI dataset into a virtual model, comprising the steps of: preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient;… generating a dynamic virtual bone model of the bone structures of the particular patient from the processed black bone dataset, wherein said virtual bone model is a 3D 360VR model configured to highlight bone structures of the patient.
Generating a model of bone and/or tissue is a mental process equivalent to drawing such a model. For example, a person could look at medical imagery and make a drawn model based on those observations using a pencil and paper. Specifying that this is a 3D 360VR virtual model or a “dynamic virtual” model amounts to no more than mere instructions to apply the exception.
processing the black bone dataset by inverting the black bone dataset and utilizing an auto detection algorithm that detects the pixels of intensity range to that of bone from the black bone dataset; and
A person could mentally invert a data set by flipping the values of each piece of data. For example, given the data set {1,1,1,1} a person could invert it to create the data set {-1,-1,-1,-1}.
A person could perform the detection mentally by observing at imagery that has been magnified so individual pixels are visible and, by comparing the color of observed pixels to a known color of bone, judge which pixels are depicting part of a bone.
Using an auto detection algorithm to perform this operation amounts to no more than mere instructions to apply the judicial exception.
generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and
Generating such a combined model that includes both bone and soft tissue imagery is a mental process equivalent to observing the features of the tissue model and bone model, and drawing a new model that combines the features of both, for example maybe drawing an image that shows the skull but also the eyeballs. Specifying that this is a “dynamic virtual” model and that it is displayed on a user display amounts to no more than mere instructions to apply the exception.
Should it be found that the displaying is not mere instructions to apply, it is also an example of insignificant post-solution activity, as it merely presents the results of the abstract idea.
providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Such a user interface could be created mentally by drawing a representation of the interface with a pen and paper; these kinds of interface mockups are frequently used by graphic designers. Further, performing this mental process of generating the interface and displaying it on a computer amounts to no more than mere instructions to apply.
Step 2A – Prong 2: Integrated into a Practical Solution?
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
performing a black bone MRI on a bone and tissue of the particular patient, wherein said black bone MRI utilizes an echo sequence and wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle;
Performing an MRI on a patient is merely the act of gathering data on that patient, and is therefore an example of mere insignificant data gathering.
Insignificant post-solution activity:
showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user;
Displaying imagery on generated through abstract means amounts to no more that presenting the results of the abstract idea, and is therefore merely insignificant post-solution activity.
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “an auto detection algorithm, a virtual model, a 3D 360VR model, a dynamic virtual model, a dynamic virtual tissue model, a dynamic virtual bone model, a combined dynamic virtual model, displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitations are mere data gathering and do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
Insignificant Extra-Solution Activity (MPEP 2106.05(g)) has found mere data gathering and
post solution activity to be insignificant extra-solution activity.
Data gathering:
performing a black bone MRI on a bone and tissue of a particular patient, wherein said black bone MRI utilizes an echo sequence wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle;
Performing an MRI on a patient is merely the act of gathering data on that patient, and is therefore an example of mere insignificant data gathering.
A claim element that amounts to merely gathering data is not indicative of integration into a
practical solution nor evidence that the claim provides an inventive concept, as exemplified by ((MPEP
2106.05)(g)(Mere Data Gathering) i. Performing clinical tests on individuals to obtain input for an
equation, In re Grams, 888 F.2d 835, 839-40; 12 USPQ2d 1824, 1827-28 (Fed. Cir. 1989); iv. Obtaining
information about transactions using the Internet to verify credit card transactions, CyberSource v.
Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011);
Insignificant post-solution activity:
showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user;
Displaying imagery on generated through abstract means amounts to no more that presenting the results of the abstract idea, and is therefore merely insignificant post-solution activity.
This element merely acts on the results of the previous abstract steps. A claim element that merely acts on a series of previous abstract steps is not indicative of integration into a practical solution nor evidence that the claim provides an inventive concept, as exemplified by ((MPEP 2106.05)(g)(Insignificant application) i. Cutting hair after first determining the hair style, In re Brown, 645 Fed. App'x 1014, 1016-1017 (Fed. Cir. 2016) and ii. Printing or downloading generated menus, Ameranth, 842 F.3d at 1241-42, 120 USPQ2d at 1854-55.)
Moreover, Mere Instructions To Apply An Exception (MPEP 2106.05(f)) has found that simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. In light of this, the additional generic computer component elements of “an auto detection algorithm, a virtual model, a 3D 360VR model, a dynamic virtual model, a dynamic virtual tissue model, a dynamic virtual bone model, a combined dynamic virtual model, displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model” are not sufficient to integrate a judicial exception into a practical application nor provide evidence of an inventive concept.
The additional elements have been considered both individually and as an ordered combination in the consideration of whether they constitute significantly more, and have been determined not to constitute such.
The claim is ineligible.
Claim 2 recites “wherein said processing includes inverting the black bone dataset.”
A person could mentally invert a data set by flipping the values of each piece of data. For example, given the data set {1,1,1,1} a person could invert it to create the data set {-1,-1,-1,-1}.
Claim 3 recites “wherein said processing includes utilizing an auto detection algorithm that detects the pixels of intensity range to that of bone from the inverted black bone dataset.”
A person could perform the detection mentally by observing at imagery that has been magnified so individual pixels are visible and, by comparing the color of observed pixels to a known color of bone, judge which pixels are depicting part of a bone.
Using an auto detection algorithm to perform this operation amounts to no more than mere instructions to apply the judicial exception.
Claim 4 recites “wherein said black bone MRI utilizes an echo sequence with a low flip angle gradient providing high contrast between bone and tissue.”
This merely clarifies the specifics of the MRI scan, and is therefore merely an extension of the mere data gathering steps.
Claim 5 recites “wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet.”
This merely clarifies the specifics of the MRI scan, and is therefore merely an extension of the mere data gathering steps.
Claim 6 recites “wherein said black bone MRI utilizes an echo sequence with a low flip angle gradient providing high contrast between bone and tissue.”
This merely clarifies the specifics of the MRI scan, and is therefore merely an extension of the mere data gathering steps.
Claim 7 recites “wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet.”
This merely clarifies the specifics of the MRI scan, and is therefore merely an extension of the mere data gathering steps.
Claim 8 recites “wherein said processing includes utilizing an auto detection algorithm that detects the pixels of intensity range to that of bone from the black bone dataset.”
A person could perform the detection mentally by observing at imagery that has been magnified so individual pixels are visible and, by comparing the color of observed pixels to a known color of bone, judge which pixels are depicting part of a bone.
Using an auto detection algorithm to perform this operation amounts to no more than mere instructions to apply the judicial exception.
Claim 9 recites “wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet.”
This merely clarifies the specifics of the MRI scan, and is therefore merely an extension of the mere data gathering steps.
Claim 10 recites “wherein said virtual model is a 3D 360VR model configured to highlight bone structures of the patient.”
This merely clarifies the specific form of the virtual model, and is therefore merely an extension of the mere instructions to apply.
Claim 11 recites “further comprising the step of using said virtual model for visualizing and diagnosing bony pathologies.”
Diagnosing issues with bones from imagery is a mental process frequently done by medical professionals. For example, a doctor could observer an X-ray of bone that is shattered into two pieces and conclude that the bone is broken.
Claim 12 recites “further comprising the step of removing pixels out of intensity range within designated bounds of an area using an erase tool.”
This step is a mental process equivalent to erasing elements of an image, as with a pencil and paper.
Claim 13 recites “A system for implementing the method of claim 1.”
This claim amounts to no more than a generic hardware system on which the method is performed. In other words, it the mere instruction to apply the judicial exception on a general-purpose computer.
Claim 15 recites “wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet.”
This merely clarifies the specifics of the MRI scan, and is therefore merely an extension of the mere data gathering steps.
Claim 16 recites “further comprising the step of removing pixels out of intensity range within designated bounds of an area using an erase tool.”
This step is a mental process equivalent to erasing elements of an image, as with a pencil and paper.
Claim 17 recites “further comprising the step of using said virtual model for visualizing and diagnosing bony pathologies.”
Diagnosing issues with bones from imagery is a mental process frequently done by medical professionals. For example, a doctor could observer an X-ray of bone that is shattered into two pieces and conclude that the bone is broken.
Claim 18 recites “A system for implementing the method of claim 14.”
This claim amounts to no more than a generic hardware system on which the method is performed. In other words, it the mere instruction to apply the judicial exception on a general-purpose computer.
Claim 20 recites “further comprising the step of using said virtual model for visualizing and diagnosing bony pathologies.”
Diagnosing issues with bones from imagery is a mental process frequently done by medical professionals. For example, a doctor could observer an X-ray of bone that is shattered into two pieces and conclude that the bone is broken.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
(1) Claims 1-9, 13-16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over “Black Bone” MRI: a novel imaging technique for 3D printing (Hereinafter Elay_2017) in view of Non-invasive three-dimensional bone–vessel image fusion using black bone MRI based on FIESTA-C (Hereinafter Hayashi) in further view of Automated 3D MRI rendering of the craniofacial skeleton: using ZTE to drive the segmentation of black bone and FIESTA-C images (hereinafter Elay_2020) as well as Farley (US 12144551 B1).
Claim 1. Eley_2017 teaches A method for processing a black bone MRI dataset into a virtual model, comprising the steps of: ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet”) a bone and tissue of the particular patient; obtaining a black bone dataset of the patient from the black bone MRI; ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 3-4] “All digital imaging and communications in medicine data were imported into both Mimics v. 14.01 (Materialise, Leuven, Belgium) and Osirix v. 4.1.2 (Open Source), and adapted thresholding techniques were used to segment the 3D data sets. In Osirix, this involved creating a region of interest (ROI) around the soft tissue/gelatine edge and setting the pixel values outside this ROI to a value that no longer coincided with bone. An upper and lower threshold value was set to segment the bone. To standardize the results, a smooth factor of 30 was used in Osirix for all of the cube data sets. In Mimics, a “mask” was created by thresholding the bone. The mask was cropped to remove as much external air as possible and was edited using the “multiple slice edit” function and “edit mask in 3D.” A stereolithographic (STL) file was created…”)
Elay_2017 fails to explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; processing the black bone dataset; and generating a dynamic virtual bone model of the bone and tissue of structures of the particular patient from the processed black bone dataset; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Hayashi makes obvious processing the black bone dataset; and ([Page 326.e17 Col 1 Par 1] “Next, the whole volume of the VR image was inverted from black to white” [Examiner’s note: it is clear from the specification (see Par 36) that this processing step consists of inverting the dataset and this limitation is interpreted as such])
Hayashi is analogous art because it is within the field of black bone MRI processing. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to allow for faster scanning. As noted by Hayashi, the MRI scanning procedure used therein takes less time than the scanning procedure of Elay_2017 ([Page 326.e19 Col 2 Par 3] “Black-bone MRI acquisition, first described by Eley et al., used the parameter settings of a 3D gradient echo sequence with a proton-density weighted imaging-like contrast.6 Their scanning time was described as around 4 minutes. In comparison, the present study used the FIESTA-C sequence of the b-SSFP technique to obtain both high SNR and a shorter scanning time. Recent MRI systems have been developed to allow the parameter settings with much shorter TRs and TEs; such developments could lead to more stable image quality of the b-SSFP sequence. In the current study, acquisition of black-bone MRI was performed successfully with around 3-minute scanning with sufficient VR image visibility. The present FIESTA-C sequence design can be used to achieve almost the same image quality with a short scanning time compared to the previously reported 3D gradient echo sequence technique.”) Overall, one of ordinary skill in the art would have recognized that combining Hayashi with Elay_2017 would result in a significantly faster scanning process.
The combination of Elay_2017 and Hayashi does not explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; generating a dynamic virtual bone model of the bone and tissue of structures of the particular patient from the processed black bone dataset; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Elay_2020 makes obvious generating a dynamic virtual model of the bone and tissue of the patient from the processed black bone dataset; ([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
PNG
media_image1.png
386
917
media_image1.png
Greyscale
([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
Elay_2020 is analogous art because it is within the field of black bone MRI processing. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017 and Hayashi before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination to provide better segmentation. As noted by Elay_2020, previous usages of black bone MRIs to attempt to develop 3D models have been held back by the requirement for lengthy segmentation processes ([Page 92 Col 1 Par 1] “Previous attempts with GRE-BB to produce 3D rendered imaging of the craniofacial skeleton have shown considerable promise, but remain limited by time-intensive segmentation techniques [2–8].”) This includes previous works like Elay_2017 in which the segmentation cannot be fully automated and still requires certain manual assistance([Elay_2017 Page 3 Col 1 Par 2] “The initial technical feasibility of producing anatomical models from “Black Bone” MRI was explored using an adult volunteer “Black Bone” data set. The mandible was selected in view of the relative simplicity in 3D rendering this region and its comparatively small size. The mandible was segmented in Mimics by applying a threshold and manually removing regions outside of the ROI using the 3D edit functions. An STL file was created from the surface of the 3D rendered image and was subsequently 3D printed.”) To this end, Elay_2020 presents a system for more accurate, automatic segmentation of black bone MRI images ([Page 92 Col 1 Par 4] “The primary objective of this study was to ascertain whether the 3D imaging output of BB could be enhanced by using the high contrast of ZTE to drive segmentation of the high-resolution imaging of GRE-BB data” [Page 92 Col 2 Par 3] “Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets”) Overall, one of ordinary skill in the art would have recognized that combining Elay_2020 with Elay_2017 and Hayashi would result in easier, more accurate 3D image segmentation, allowing better separation of bone from other material.
The combination of Elay_2017, Hayashi, and Elay_2020 does not explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to a bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Farley makes obvious preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; ([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only … only soft tissue structures…”) generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to a bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model. ([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only bony anatomy, only soft tissue structures, or it may combine both into a single 3D model.” [Col 39 line 13-20] “Various embodiments are described herein that relate to using an augmented reality navigation system to aid in viewing patient anatomy (e.g., viewing soft tissue, bony landmarks, and/or the combination of both.). The system generally includes a navigation system, one or more displays, including but not limited to, HMDs, with onboard navigation capabilities and instrumentation tracking to target tool trajectories relative to the plating system.” [Col 9 line 22-40] “The Display 125 provides graphical user interfaces (GUIs) that display images collected by the Tissue Navigation System 120 as well other information relevant to the surgery. For example, in one embodiment, the Display 125 overlays image information collected from various modalities (e.g., CT, MRI, X-ray, fluorescent, ultrasound, etc.) collected pre-operatively or intra-operatively to give the surgeon various views of the patient's anatomy as well as real-time conditions. The Display 125 may include, for example, one or more computer monitors. As an alternative or supplement to the Display 125, one or more members of the surgical staff may wear an Augmented Reality (AR) Head Mounted Device (HMD). For example, in FIG. 1 the Surgeon 111 is wearing an AR HMD 155 that may, for example, overlay pre-operative image data on the patient or provide surgical planning suggestions. Various example uses of the AR HMD 155 in surgical procedures are detailed in the sections that follow.”)
Farley is analogous art because it is within the field of medical imaging. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017, Hayashi, and Elay_2020 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to make more accurate measurements. As noted by Farley, a known issue is medical imaging is the effect of electromagnetic interference from other devices causing inaccuracy in measurements and instrument readings ([Col 7 line 59 – Col 8 line 2] “In another embodiment, the above systems may allow for a multi-sensor navigation system that can detect and correct for field distortions that plague electromagnetic tracking systems. It should be understood that field distortions may result from movement of any ferromagnetic materials within the reference field. Thus, as one of ordinary skill in the art would know, a typical OR has a large number of devices (e.g., an operating table, LCD displays, lighting equipment, imaging systems, surgical instruments, etc.) that may cause interference. Furthermore, field distortions are notoriously difficult to detect.”) To this end, Farley presents a method for detecting and warning of such distortions, allowing for inaccurate instrument readings to be avoided ([Col 8 line 2-12] “The use of multiple EM sensors enables the system to detect field distortions accurately, and/or to warn a user that the current position measurements may not be accurate. Because the sensors are rigidly fixed to the bony anatomy (e.g., via the pin/needle), relative measurement of sensor positions (X, Y, Z) may be used to detect field distortions. By way of non-limiting example, in some embodiments, after the EM sensors are fixed to the bone, the relative distance between the two sensors is known and should remain constant. Thus, any change in this distance could indicate the presence of a field distortion.”) Overall, one of ordinary skill in the art would have recognized that combining Farley with Elay_2017, Hayashi, and Elay_2020 would result in a system that allowed for the detection of conditions that would cause the imaging process to be inaccurate, ultimately enabling changes to be made and such inaccuracies to be avoided, resulting in an overall more accurate system.
Claim 2. Hayashi teaches wherein said processing includes inverting the black bone dataset. ([Page 326.e17 Col 1 Par 1] “Next, the whole volume of the VR image was inverted from black to white”)
Claim 3. Elay_2020 teaches wherein said processing includes utilizing an auto detection algorithm that detects the pixels of intensity range to that of bone from ([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
PNG
media_image1.png
386
917
media_image1.png
Greyscale
Hayashi teaches the inverted black bone dataset. ([Page 326.e17 Col 1 Par 1] “Next, the whole volume of the VR image was inverted from black to white”)
Claim 4. Elay_2017 teaches wherein said black bone MRI utilizes an echo sequence with a low flip angle gradient providing high contrast between bone and tissue. ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”)
Claim 5. Elay_2017 teaches wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet. ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”)
Claim 6. Elay_2017 teaches wherein said black bone MRI utilizes an echo sequence with a low flip angle gradient providing high contrast between bone and tissue. ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”)
Claim 7. Elay_2017 teaches wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”)
Claim 8. Elay_2020 teaches wherein said processing includes utilizing an auto detection algorithm that detects the pixels of intensity range to that of bone from the inverted black bone dataset. ([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
PNG
media_image1.png
386
917
media_image1.png
Greyscale
Claim 9. Elay_2017 teaches wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet. ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”)
Claim 13. Elay_2020 teaches A system for implementing the method of claim 1.([Page 93 Col 2 Par 1] “The image processing time to produce the automated 3D outputs was on average 9 min 30 s per dataset on a standard Intel i7-9750H 2.60GHz CPU laptop, without any code optimisations (average CPU usage approximately 10%).”)
Claim 14. Elay_2017 teaches A method for processing a black bone MRI dataset into a virtual model, comprising the steps of: ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet”) ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”) ([Page 2 Col 1 Par 1] “The most simple segmentation technique utilizes thresholding which separates the components of the image according to intensity, aided by the predefined CT numbers (Hounsfield unit) of structures from bone (1000 HU) at one end of the spectrum to air at the other (21000 HU).” [Page 2 Col 2 Par 3] “All digital imaging and communications in medicine data were imported into both Mimics v. 14.01 (Materialise, Leuven, Belgium) and Osirix v. 4.1.2 (Open Source), and adapted thresholding techniques were used to segment the 3D data sets. In Osirix, this involved creating a region of interest (ROI) around the soft tissue/gelatine edge and setting the pixel values outside this ROI to a value that no longer coincided with bone. An upper and lower threshold value was set to segment the bone.”)
Elay_2017 does not explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; processing the black bone dataset utilizing an auto detection algorithm; generating a dynamic virtual bone model of the bone structures of the particular patient from the processed black bone dataset, wherein said virtual bone model is configured to highlight bone structures of the patient; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Hayashi makes obvious ([Page 326.e17 Col 1 Par 1] “Next, the whole volume of the VR image was inverted from black to white” [Examiner’s note: it is clear from the specification (see Par 36) that this processing step consists of inverting the dataset and this limitation is interpreted as such])
Hayashi is analogous art because it is within the field of black bone MRI processing. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to allow for faster scanning. As noted by Hayashi, the MRI scanning procedure used therein takes less time than the scanning procedure of Elay_2017 ([Page 326.e19 Col 2 Par 3] “Black-bone MRI acquisition, first described by Eley et al., used the parameter settings of a 3D gradient echo sequence with a proton-density weighted imaging-like contrast.6 Their scanning time was described as around 4 minutes. In comparison, the present study used the FIESTA-C sequence of the b-SSFP technique to obtain both high SNR and a shorter scanning time. Recent MRI systems have been developed to allow the parameter settings with much shorter TRs and TEs; such developments could lead to more stable image quality of the b-SSFP sequence. In the current study, acquisition of black-bone MRI was performed successfully with around 3-minute scanning with sufficient VR image visibility. The present FIESTA-C sequence design can be used to achieve almost the same image quality with a short scanning time compared to the previously reported 3D gradient echo sequence technique.”) Overall, one of ordinary skill in the art would have recognized that combining Hayashi with Elay_2017 would result in a significantly faster scanning process.
The combination of Elay_2017 and Hayashi does not explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; utilizing an auto detection algorithm; generating a dynamic virtual bone model of the bone structures of the particular patient from the processed black bone dataset, wherein said virtual bone model is configured to highlight bone structures of the patient; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Elay_2020 makes obvious utilizing an auto detection algorithm and generating a dynamic virtual bone model of the bone structures of the particular patient from the processed black bone dataset, wherein said virtual bone model is configured to highlight bone structures of the patient; ([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
PNG
media_image1.png
386
917
media_image1.png
Greyscale
([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
Elay_2020 is analogous art because it is within the field of black bone MRI processing. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017 and Hayashi before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination to provide better segmentation. As noted by Elay_2020, previous usages of black bone MRIs to attempt to develop 3D models have been held back by the requirement for lengthy segmentation processes ([Page 92 Col 1 Par 1] “Previous attempts with GRE-BB to produce 3D rendered imaging of the craniofacial skeleton have shown considerable promise, but remain limited by time-intensive segmentation techniques [2–8].”) This includes previous works like Elay_2017 in which the segmentation cannot be fully automated and still requires certain manual assistance([Elay_2017 Page 3 Col 1 Par 2] “The initial technical feasibility of producing anatomical models from “Black Bone” MRI was explored using an adult volunteer “Black Bone” data set. The mandible was selected in view of the relative simplicity in 3D rendering this region and its comparatively small size. The mandible was segmented in Mimics by applying a threshold and manually removing regions outside of the ROI using the 3D edit functions. An STL file was created from the surface of the 3D rendered image and was subsequently 3D printed.”) To this end, Elay_2020 presents a system for more accurate, automatic segmentation of black bone MRI images ([Page 92 Col 1 Par 4] “The primary objective of this study was to ascertain whether the 3D imaging output of BB could be enhanced by using the high contrast of ZTE to drive segmentation of the high-resolution imaging of GRE-BB data” [Page 92 Col 2 Par 3] “Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets”) Overall, one of ordinary skill in the art would have recognized that combining Elay_2020 with Elay_2017 and Hayashi would result in easier, more accurate 3D image segmentation, allowing better separation of bone from other material.
The combination of Elay_2017, Hayashi, and Elay_2020 does not explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to a bone model to create a combined dynamic virtual model showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Farley makes obvious preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; ([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only … only soft tissue structures…”) generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to a bone model to create a combined dynamic virtual model showing a combination view of the bone structure with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model. ([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only bony anatomy, only soft tissue structures, or it may combine both into a single 3D model.” [Col 39 line 13-20] “Various embodiments are described herein that relate to using an augmented reality navigation system to aid in viewing patient anatomy (e.g., viewing soft tissue, bony landmarks, and/or the combination of both.). The system generally includes a navigation system, one or more displays, including but not limited to, HMDs, with onboard navigation capabilities and instrumentation tracking to target tool trajectories relative to the plating system.” [Col 9 line 22-40] “The Display 125 provides graphical user interfaces (GUIs) that display images collected by the Tissue Navigation System 120 as well other information relevant to the surgery. For example, in one embodiment, the Display 125 overlays image information collected from various modalities (e.g., CT, MRI, X-ray, fluorescent, ultrasound, etc.) collected pre-operatively or intra-operatively to give the surgeon various views of the patient's anatomy as well as real-time conditions. The Display 125 may include, for example, one or more computer monitors. As an alternative or supplement to the Display 125, one or more members of the surgical staff may wear an Augmented Reality (AR) Head Mounted Device (HMD). For example, in FIG. 1 the Surgeon 111 is wearing an AR HMD 155 that may, for example, overlay pre-operative image data on the patient or provide surgical planning suggestions. Various example uses of the AR HMD 155 in surgical procedures are detailed in the sections that follow.”)
Farley is analogous art because it is within the field of medical imaging. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017, Hayashi, and Elay_2020 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to make more accurate measurements. As noted by Farley, a known issue is medical imaging is the effect of electromagnetic interference from other devices causing inaccuracy in measurements and instrument readings ([Col 7 line 59 – Col 8 line 2] “In another embodiment, the above systems may allow for a multi-sensor navigation system that can detect and correct for field distortions that plague electromagnetic tracking systems. It should be understood that field distortions may result from movement of any ferromagnetic materials within the reference field. Thus, as one of ordinary skill in the art would know, a typical OR has a large number of devices (e.g., an operating table, LCD displays, lighting equipment, imaging systems, surgical instruments, etc.) that may cause interference. Furthermore, field distortions are notoriously difficult to detect.”) To this end, Farley presents a method for detecting and warning of such distortions, allowing for inaccurate instrument readings to be avoided ([Col 8 line 2-12] “The use of multiple EM sensors enables the system to detect field distortions accurately, and/or to warn a user that the current position measurements may not be accurate. Because the sensors are rigidly fixed to the bony anatomy (e.g., via the pin/needle), relative measurement of sensor positions (X, Y, Z) may be used to detect field distortions. By way of non-limiting example, in some embodiments, after the EM sensors are fixed to the bone, the relative distance between the two sensors is known and should remain constant. Thus, any change in this distance could indicate the presence of a field distortion.”) Overall, one of ordinary skill in the art would have recognized that combining Farley with Elay_2017, Hayashi, and Elay_2020 would result in a system that allowed for the detection of conditions that would cause the imaging process to be inaccurate, ultimately enabling changes to be made and such inaccuracies to be avoided, resulting in an overall more accurate system.
Claim 15. Elay_2017 teaches wherein said black bone MRI utilizes an echo time (TE) sequence of 4.2 ms. and a repetition time of 8.6 ms. at a 5 degree flip angle using a 1.5T or 3.OT magnet. ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”)
Claim 16. Elay_2017 teaches further comprising the step of removing pixels out of intensity range within designated bounds of an area using an erase tool. ([Page 2 Col 2 Par 3] “In Osirix, this involved creating a region of interest (ROI) around the soft tissue/gelatine edge and setting the pixel values outside this ROI to a value that no longer coincided with bone… In Mimics, a “mask” was created by thresholding the bone. The mask was cropped to remove as much external air as possible and was edited using the “multiple slice edit” function” [Page 3 Col 1 Par 2] “The mandible was segmented in Mimics by applying a threshold and manually removing regions outside of the ROI using the 3D edit functions”)
Claim 18. Elay_2020 teaches A system for implementing the method of claim 14. ([Page 93 Col 2 Par 1] “The image processing time to produce the automated 3D outputs was on average 9 min 30 s per dataset on a standard Intel i7-9750H 2.60GHz CPU laptop, without any code optimisations (average CPU usage approximately 10%).”)
(2) Claims 10-12, 17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over “Black Bone” MRI: a novel imaging technique for 3D printing (Hereinafter Elay_2017) in view of Non-invasive three-dimensional bone–vessel image fusion using black bone MRI based on FIESTA-C (Hereinafter Hayashi) in further view of Automated 3D MRI rendering of the craniofacial skeleton: using ZTE to drive the segmentation of black bone and FIESTA-C images (hereinafter Elay_2020) as well as Farley (US 12144551 B1) and Poltaretskyi (US 20190380792 A1)
Claim 10. Elay_2020 teaches wherein said virtual model is a ([Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
The combination of Elay_2017, Hayashi, Elay_2020, and Farley fails to explicitly teach a 3D 360VR model
Poltaretskyi makes obvious a 3D 360VR model ([Par 914] “For example, during preoperative phase 302 (FIG. 3) and intraoperative phase 306 (FIG. 3), orthopedic surgical system 100 (FIG. 1) may provide XR visualizations (e.g., MR visualizations or VR visualizations) that include patient-specific virtual 3D models of a patient's ankle anatomy.”)
Poltaretskyi is analogous art because it is within the field of medical imagery visualization. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017, Hayashi, Elay_2020, and Farley before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to help doctors more easily perform procedures on patients based on the created imagery. As noted by Poltaretskyi, surgery is an incredibly difficult, delicate art requiring precise planning that can be significantly challenging ([Par 2] “ Surgical joint repair procedures involve repair and/or replacement of a damaged or diseased joint. Many times, a surgical joint repair procedure, such as joint arthroplasty as an example, involves replacing the damaged joint with a prosthetic that is implanted into the patient's bone. Proper selection of a prosthetic that is appropriately sized and shaped and proper positioning of that prosthetic to ensure an optimal surgical outcome can be challenging. To assist with positioning, the surgical procedure often involves the use of surgical instruments to control the shaping of the surface of the damaged bone and cutting or drilling of bone to accept the prosthetic.”) To this end, Poltaretskyi, presents an intuitive system that allows a medical professional to interact with available information, such as medical imagery and models, to design procedures in a three-dimensional environment ([Par 162-163] “ Oftentimes, once in the actual operating environment, the surgeon may desire to verify the preoperative surgical plan intraoperatively relative to the patient's actual bone. This verification may result in a determination that an adjustment to the preoperative surgical plan is needed, such as a different implant, a different positioning or orientation of the implant, and/or a different surgical guide for carrying out the surgical plan. In addition, a surgeon may want to view details of the preoperative surgical plan relative to the patient's real bone during the actual procedure in order to more efficiently and accurately position and orient the implant components. For example, the surgeon may want to obtain intra-operative visualization that provides guidance for positioning and orientation of implant components, guidance for preparation of bone or tissue to receive the implant components, guidance for reviewing the details of a procedure or procedural step, and/or guidance for selection of tools or implants and tracking of surgical procedure workflow. Accordingly, this disclosure describes systems and methods for using a mixed reality (MR) visualization system to assist with creation, implementation, verification, and/or modification of a surgical plan before and during a surgical procedure. Because MR, or in some instances VR, may be used to interact with the surgical plan, this disclosure may also refer to the surgical plan as a “virtual” surgical plan… A surgical plan, e.g., as generated by the BLUEPRINT™ system or another surgical planning platform, may include information defining a variety of features of a surgical procedure, such as features of particular surgical procedure steps to be performed on a patient by a surgeon according to the surgical plan including, for example, bone or tissue preparation steps and/or steps for selection, modification and/or placement of implant components. Such information may include, in various examples, dimensions, shapes, angles, surface contours, and/or orientations of implant components to be selected or modified by surgeons, dimensions, shapes, angles, surface contours and/or orientations to be defined in bone or tissue by the surgeon in bone or tissue preparation steps, and/or positions, axes, planes, angle and/or entry points defining placement of implant components by the surgeon relative to patient bone or tissue. Information such as dimensions, shapes, angles, surface contours, and/or orientations of anatomical features of the patient may be derived from imaging (e.g., x-ray, CT, MM, ultrasound or other images), direct observation, or other techniques.”) Overall one of ordinary skill in the art would have recognized that combining Poltaretskyi with Elay_2017, Hayashi, Elay_2020, and Farley would result in a system that would allow medical professionals to integrate the generated 3D black body models into practical surgical plans in an intuitive, three-dimensional manner.
Claim 11. Poltaretskyi teaches further comprising the step of using said virtual model for visualizing and diagnosing bony pathologies. ([Par 914] “For example, during preoperative phase 302 (FIG. 3) and intraoperative phase 306 (FIG. 3), orthopedic surgical system 100 (FIG. 1) may provide XR visualizations (e.g., MR visualizations or VR visualizations) that include patient-specific virtual 3D models of a patient's ankle anatomy.” [Par 223] “In the example of FIG. 8, a model of the area of interest is generated (800). For example, a scan (e.g., a CT scan, MRI scan, or other type of scan) of the area of interest may be performed. For example, if the area of interest is the patient's shoulder, a scan of the patient's shoulder may be performed. Furthermore, a pathology in the area of interest may be classified (802). In some examples, the pathology of the area of interest may be classified based on the scan of the area of interest. For example, if the area of interest is the user's shoulder, a surgeon may determine what is wrong with the patient's shoulder based on the scan of the patient's shoulder and provide a shoulder classification indicating the classification or diagnosis, e.g., such as primary glenoid humeral osteoarthritis (PGHOA), rotator cuff tear arthropathy (RCTA) instability, massive rotator cuff tear (MRCT), rheumatoid arthritis, post-traumatic arthritis, and osteoarthritis.”)
Claim 12. Elay_2017 teaches further comprising the step of removing pixels out of intensity range within designated bounds of an area using an erase tool. ([Page 2 Col 2 Par 3] “In Osirix, this involved creating a region of interest (ROI) around the soft tissue/gelatine edge and setting the pixel values outside this ROI to a value that no longer coincided with bone… In Mimics, a “mask” was created by thresholding the bone. The mask was cropped to remove as much external air as possible and was edited using the “multiple slice edit” function” [Page 3 Col 1 Par 2] “The mandible was segmented in Mimics by applying a threshold and manually removing regions outside of the ROI using the 3D edit functions”)
Claim 17. Poltaretskyi teaches further comprising the step of using said virtual model for visualizing and diagnosing bony pathologies. ([Par 914] “For example, during preoperative phase 302 (FIG. 3) and intraoperative phase 306 (FIG. 3), orthopedic surgical system 100 (FIG. 1) may provide XR visualizations (e.g., MR visualizations or VR visualizations) that include patient-specific virtual 3D models of a patient's ankle anatomy.” [Par 223] “In the example of FIG. 8, a model of the area of interest is generated (800). For example, a scan (e.g., a CT scan, MRI scan, or other type of scan) of the area of interest may be performed. For example, if the area of interest is the patient's shoulder, a scan of the patient's shoulder may be performed. Furthermore, a pathology in the area of interest may be classified (802). In some examples, the pathology of the area of interest may be classified based on the scan of the area of interest. For example, if the area of interest is the user's shoulder, a surgeon may determine what is wrong with the patient's shoulder based on the scan of the patient's shoulder and provide a shoulder classification indicating the classification or diagnosis, e.g., such as primary glenoid humeral osteoarthritis (PGHOA), rotator cuff tear arthropathy (RCTA) instability, massive rotator cuff tear (MRCT), rheumatoid arthritis, post-traumatic arthritis, and osteoarthritis.”)
Poltaretskyi is analogous art because it is within the field of medical imagery visualization. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017, Hayashi, Elay_2020 and Farley before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to help doctors more easily perform procedures on patients based on the created imagery. As noted by Poltaretskyi, surgery is an incredibly difficult, delicate art requiring precise planning that can be significantly challenging ([Par 2] “ Surgical joint repair procedures involve repair and/or replacement of a damaged or diseased joint. Many times, a surgical joint repair procedure, such as joint arthroplasty as an example, involves replacing the damaged joint with a prosthetic that is implanted into the patient's bone. Proper selection of a prosthetic that is appropriately sized and shaped and proper positioning of that prosthetic to ensure an optimal surgical outcome can be challenging. To assist with positioning, the surgical procedure often involves the use of surgical instruments to control the shaping of the surface of the damaged bone and cutting or drilling of bone to accept the prosthetic.”) To this end, Poltaretskyi, presents an intuitive system that allows a medical professional to interact with available information, such as medical imagery and models, to design procedures in a three-dimensional environment ([Par 162-163] “ Oftentimes, once in the actual operating environment, the surgeon may desire to verify the preoperative surgical plan intraoperatively relative to the patient's actual bone. This verification may result in a determination that an adjustment to the preoperative surgical plan is needed, such as a different implant, a different positioning or orientation of the implant, and/or a different surgical guide for carrying out the surgical plan. In addition, a surgeon may want to view details of the preoperative surgical plan relative to the patient's real bone during the actual procedure in order to more efficiently and accurately position and orient the implant components. For example, the surgeon may want to obtain intra-operative visualization that provides guidance for positioning and orientation of implant components, guidance for preparation of bone or tissue to receive the implant components, guidance for reviewing the details of a procedure or procedural step, and/or guidance for selection of tools or implants and tracking of surgical procedure workflow. Accordingly, this disclosure describes systems and methods for using a mixed reality (MR) visualization system to assist with creation, implementation, verification, and/or modification of a surgical plan before and during a surgical procedure. Because MR, or in some instances VR, may be used to interact with the surgical plan, this disclosure may also refer to the surgical plan as a “virtual” surgical plan… A surgical plan, e.g., as generated by the BLUEPRINT™ system or another surgical planning platform, may include information defining a variety of features of a surgical procedure, such as features of particular surgical procedure steps to be performed on a patient by a surgeon according to the surgical plan including, for example, bone or tissue preparation steps and/or steps for selection, modification and/or placement of implant components. Such information may include, in various examples, dimensions, shapes, angles, surface contours, and/or orientations of implant components to be selected or modified by surgeons, dimensions, shapes, angles, surface contours and/or orientations to be defined in bone or tissue by the surgeon in bone or tissue preparation steps, and/or positions, axes, planes, angle and/or entry points defining placement of implant components by the surgeon relative to patient bone or tissue. Information such as dimensions, shapes, angles, surface contours, and/or orientations of anatomical features of the patient may be derived from imaging (e.g., x-ray, CT, MM, ultrasound or other images), direct observation, or other techniques.”) Overall one of ordinary skill in the art would have recognized that combining Poltaretskyi with Elay_2017, Hayashi, Elay_2020 and Farley.
Claim 19. Elay_2017 makes obvious A method for processing a black bone MRI dataset into a virtual model, comprising the steps of: ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet”) obtaining a black bone dataset of the patient from the black bone MRI; ([Page 2 Col 1 Par 2-3, 5] “We previously reported the potential of an MRI technique that minimizes soft tissue contrast to enhance the bone–soft tissue boundary; hence, the term “Black Bone”. 7–11 The novel concept of minimizing soft tissue contrast and any signal returned from bone, rather than using traditional processes to increase signal intensity from bone [e.g. ultrashort echo time (TE)], makes it possible to segment bone from the surrounding soft tissues to produce 3D reconstructed images. Early applications of this technique have been particularly promising for the craniofacial skeleton, with results approaching those expected of 3D CT imaging, and in distinguishing normal cranial sutures from those that are prematurely fused (craniosynostosis).7 The “Black Bone” MRI technique utilizes a gradient echo (GRE) sequence such as 3D fast GRE (GE Medical Systems Ltd, Chalfont St Giles, UK), volumetric interpolated breath-hold examination (Siemens Healthcare Ltd, Camberley, UK) or an equivalent T1 weighted GRE 3D volume sequence (Phillips Healthcare, Guildford, UK), with a short TE (4.2 ms)/ repetition time (8.6 ms) and a flip angle of 5° with a 1.5- or 3.0-T magnet. … “Black Bone” MRI data sets utilized for this study were previously acquired from infants and adult volunteers, for which ethical approval had been granted from the Oxfordshire Research Ethics Committee (09/H0606/2).7 All patient and volunteer imaging used in this study was acquired on a 1.5-T magnet” [Page 2 Col 2 Par 4] “A stereolithographic (STL) file was created…”) ([Page 2 Col 1 Par 1] “The most simple segmentation technique utilizes thresholding which separates the components of the image according to intensity, aided by the predefined CT numbers (Hounsfield unit) of structures from bone (1000 HU) at one end of the spectrum to air at the other (21000 HU).” [Page 2 Col 2 Par 3] “All digital imaging and communications in medicine data were imported into both Mimics v. 14.01 (Materialise, Leuven, Belgium) and Osirix v. 4.1.2 (Open Source), and adapted thresholding techniques were used to segment the 3D data sets. In Osirix, this involved creating a region of interest (ROI) around the soft tissue/gelatine edge and setting the pixel values outside this ROI to a value that no longer coincided with bone. An upper and lower threshold value was set to segment the bone.”)
Elay_2017 fails to explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; processing the black bone dataset by inverting the black bone dataset and utilizing an auto detection algorithm and generating a dynamic virtual bone model of the bone dtrucctures of the particular patient from the processed black bone dataset, wherein said virtual model is a 3D 360VR model configured to highlight bone structures of the patient; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Hayashi makes obvious processing the black bone dataset by inverting the black bone dataset and ([Page 326.e17 Col 1 Par 1] “Next, the whole volume of the VR image was inverted from black to white”)
Hayashi is analogous art because it is within the field of black bone MRI processing. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to allow for faster scanning. As noted by Hayashi, the MRI scanning procedure used therein takes less time than the scanning procedure of Elay_2017 ([Page 326.e19 Col 2 Par 3] “Black-bone MRI acquisition, first described by Eley et al., used the parameter settings of a 3D gradient echo sequence with a proton-density weighted imaging-like contrast.6 Their scanning time was described as around 4 minutes. In comparison, the present study used the FIESTA-C sequence of the b-SSFP technique to obtain both high SNR and a shorter scanning time. Recent MRI systems have been developed to allow the parameter settings with much shorter TRs and TEs; such developments could lead to more stable image quality of the b-SSFP sequence. In the current study, acquisition of black-bone MRI was performed successfully with around 3-minute scanning with sufficient VR image visibility. The present FIESTA-C sequence design can be used to achieve almost the same image quality with a short scanning time compared to the previously reported 3D gradient echo sequence technique.”) Overall, one of ordinary skill in the art would have recognized that combining Hayashi with Elay_2017 would result in a significantly faster scanning process.
The combination of Elay_2017 and Hayashi fails to explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; utilizing an auto detection algorithm and generating a dynamic virtual bone model of the bone dtrucctures of the particular patient from the processed black bone dataset, wherein said virtual model is a 3D 360VR model configured to highlight bone structures of the patient; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to said dynamic virtual bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Elay_2020 makes obvious utilizing an auto detection algorithm and generating a dynamic virtual bone model of the bone dtrucctures of the particular patient from the processed black bone dataset, wherein said virtual model is ([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
PNG
media_image1.png
386
917
media_image1.png
Greyscale
([Page 92 Col 2 Par 1-4] “Following ethical approval (Cambridge Human Biology Research Ethics Committee, HBREC.2016.13), 11 adult volunteers underwent MRI examination on a 3T system (SIGNA PET/ MR, GE Healthcare, MP26 software version). The age range of volunteers was between 28 and 39 years, with a male predominance (female n = 4; male n = 7). The protocol included the sequential acquisition of ZTE (3D radial), GRE-BB and FIESTA-C imaging. The imaging parameters are shown in Table 1. All imaging was completed using a 21-element head and neck receive coil (GE GEM HNU). Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets. An automated 3D rendered image with rotating movie clip was produced for each of the datasets” [Page 93 Col 1 Par 2] “The 3D renderings were enhanced with the modified algorithm utilising ZTE to drive the BB segmentation (Fig. 4) with the modified algorithm successful in removing non-bony tissues such as ligaments and muscles” [Fig. 4] Shows images generated from a model with a combination of bone and tissues and another model in which soft tissue was removed. Note that ZTE was the technique used by the researchers to remove non-bony tissue from the initial top model)
Elay_2020 is analogous art because it is within the field of black bone MRI processing. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017 and Hayashi before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination to provide better segmentation. As noted by Elay_2020, previous usages of black bone MRIs to attempt to develop 3D models have been held back by the requirement for lengthy segmentation processes ([Page 92 Col 1 Par 1] “Previous attempts with GRE-BB to produce 3D rendered imaging of the craniofacial skeleton have shown considerable promise, but remain limited by time-intensive segmentation techniques [2–8].”) This includes previous works like Elay_2017 in which the segmentation cannot be fully automated and still requires certain manual assistance([Elay_2017 Page 3 Col 1 Par 2] “The initial technical feasibility of producing anatomical models from “Black Bone” MRI was explored using an adult volunteer “Black Bone” data set. The mandible was selected in view of the relative simplicity in 3D rendering this region and its comparatively small size. The mandible was segmented in Mimics by applying a threshold and manually removing regions outside of the ROI using the 3D edit functions. An STL file was created from the surface of the 3D rendered image and was subsequently 3D printed.”) To this end, Elay_2020 presents a system for more accurate, automatic segmentation of black bone MRI images ([Page 92 Col 1 Par 4] “The primary objective of this study was to ascertain whether the 3D imaging output of BB could be enhanced by using the high contrast of ZTE to drive segmentation of the high-resolution imaging of GRE-BB data” [Page 92 Col 2 Par 3] “Craniofacial bone segmentation was performed using a fully automated segmentation algorithm, implemented in C++ using the Insight Segmentation and Registration Toolkit (ITK), including image denoising, intensity normalisation, head mask generation, N4 bias correction, skin removal, intensity rescaling and masking (Fig. 1). Segmentation was firstly completed for BB. A modified version of the algorithm was then implemented (Fig. 2), wherein the bone mask yielded by ZTE segmentation was used to initialise the segmentation of BB (ZTE+BB). The two segmentation algorithms were subsequently applied to the FIESTA-C datasets”) Overall, one of ordinary skill in the art would have recognized that combining Elay_2020 with Elay_2017 and Hayashi would result in easier, more accurate 3D image segmentation, allowing better separation of bone from other material.
The combination of Elay_2017, Hayashi, and Elay_2020 does not explicitly teach preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; a 3D 360VR model; generating a combined dynamic virtual model of the particular patient showing said bone structure adding features of said soft tissues and organs of the particular patient from the dynamic virtual tissue model to a bone model to create a combined dynamic virtual model showing a combination view of the bone with soft tissue and organ anatomy of the particular patient for displaying on a user display to a user; and providing a user interface for accepting user inputs from the user for dynamic interaction of the user with the combined dynamic virtual model.
Farley makes obvious preparing a dynamic virtual tissue model from medical images of a particular patient, said model incorporating soft tissues and organs of the particular patient; ([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only … only soft tissue structures…”) ([Col 35 line 57-63] “Once the medical image data has been received from the medical imager (e.g., an MRI machine 1002), the images may be segmented to create a 3D model of a patient's bony anatomy and soft tissue structures. It should be understood that the creation of the 3D model may comprise only bony anatomy, only soft tissue structures, or it may combine both into a single 3D model.” [Col 39 line 13-20] “Various embodiments are described herein that relate to using an augmented reality navigation system to aid in viewing patient anatomy (e.g., viewing soft tissue, bony landmarks, and/or the combination of both.). The system generally includes a navigation system, one or more displays, including but not limited to, HMDs, with onboard navigation capabilities and instrumentation tracking to target tool trajectories relative to the plating system.” [Col 9 line 22-40] “The Display 125 provides graphical user interfaces (GUIs) that display images collected by the Tissue Navigation System 120 as well other information relevant to the surgery. For example, in one embodiment, the Display 125 overlays image information collected from various modalities (e.g., CT, MRI, X-ray, fluorescent, ultrasound, etc.) collected pre-operatively or intra-operatively to give the surgeon various views of the patient's anatomy as well as real-time conditions. The Display 125 may include, for example, one or more computer monitors. As an alternative or supplement to the Display 125, one or more members of the surgical staff may wear an Augmented Reality (AR) Head Mounted Device (HMD). For example, in FIG. 1 the Surgeon 111 is wearing an AR HMD 155 that may, for example, overlay pre-operative image data on the patient or provide surgical planning suggestions. Various example uses of the AR HMD 155 in surgical procedures are detailed in the sections that follow.”)
Farley is analogous art because it is within the field of medical imaging. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017, Hayashi, and Elay_2020 before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to make more accurate measurements. As noted by Farley, a known issue is medical imaging is the effect of electromagnetic interference from other devices causing inaccuracy in measurements and instrument readings ([Col 7 line 59 – Col 8 line 2] “In another embodiment, the above systems may allow for a multi-sensor navigation system that can detect and correct for field distortions that plague electromagnetic tracking systems. It should be understood that field distortions may result from movement of any ferromagnetic materials within the reference field. Thus, as one of ordinary skill in the art would know, a typical OR has a large number of devices (e.g., an operating table, LCD displays, lighting equipment, imaging systems, surgical instruments, etc.) that may cause interference. Furthermore, field distortions are notoriously difficult to detect.”) To this end, Farley presents a method for detecting and warning of such distortions, allowing for inaccurate instrument readings to be avoided ([Col 8 line 2-12] “The use of multiple EM sensors enables the system to detect field distortions accurately, and/or to warn a user that the current position measurements may not be accurate. Because the sensors are rigidly fixed to the bony anatomy (e.g., via the pin/needle), relative measurement of sensor positions (X, Y, Z) may be used to detect field distortions. By way of non-limiting example, in some embodiments, after the EM sensors are fixed to the bone, the relative distance between the two sensors is known and should remain constant. Thus, any change in this distance could indicate the presence of a field distortion.”) Overall, one of ordinary skill in the art would have recognized that combining Farley with Elay_2017, Hayashi, and Elay_2020 would result in a system that allowed for the detection of conditions that would cause the imaging process to be inaccurate, ultimately enabling changes to be made and such inaccuracies to be avoided, resulting in an overall more accurate system.
The combination of Elay_2017, Hayashi, Elay_2020, and Farley fails to explicitly teach a 3D 360VR model
Poltaretskyi makes obvious a 3D 360VR model ([Par 914] “For example, during preoperative phase 302 (FIG. 3) and intraoperative phase 306 (FIG. 3), orthopedic surgical system 100 (FIG. 1) may provide XR visualizations (e.g., MR visualizations or VR visualizations) that include patient-specific virtual 3D models of a patient's ankle anatomy.”)
Poltaretskyi is analogous art because it is within the field of medical imagery visualization. It would have been obvious to one of ordinary skill in the art to combine it with Elay_2017, Hayashi, and Elay_2020, and Farley before the effective filing date. One of ordinary skill in the art would have been motivated to make this combination in order to help doctors more easily perform procedures on patients based on the created imagery. As noted by Poltaretskyi, surgery is an incredibly difficult, delicate art requiring precise planning that can be significantly challenging ([Par 2] “ Surgical joint repair procedures involve repair and/or replacement of a damaged or diseased joint. Many times, a surgical joint repair procedure, such as joint arthroplasty as an example, involves replacing the damaged joint with a prosthetic that is implanted into the patient's bone. Proper selection of a prosthetic that is appropriately sized and shaped and proper positioning of that prosthetic to ensure an optimal surgical outcome can be challenging. To assist with positioning, the surgical procedure often involves the use of surgical instruments to control the shaping of the surface of the damaged bone and cutting or drilling of bone to accept the prosthetic.”) To this end, Poltaretskyi, presents an intuitive system that allows a medical professional to interact with available information, such as medical imagery and models, to design procedures in a three-dimensional environment ([Par 162-163] “ Oftentimes, once in the actual operating environment, the surgeon may desire to verify the preoperative surgical plan intraoperatively relative to the patient's actual bone. This verification may result in a determination that an adjustment to the preoperative surgical plan is needed, such as a different implant, a different positioning or orientation of the implant, and/or a different surgical guide for carrying out the surgical plan. In addition, a surgeon may want to view details of the preoperative surgical plan relative to the patient's real bone during the actual procedure in order to more efficiently and accurately position and orient the implant components. For example, the surgeon may want to obtain intra-operative visualization that provides guidance for positioning and orientation of implant components, guidance for preparation of bone or tissue to receive the implant components, guidance for reviewing the details of a procedure or procedural step, and/or guidance for selection of tools or implants and tracking of surgical procedure workflow. Accordingly, this disclosure describes systems and methods for using a mixed reality (MR) visualization system to assist with creation, implementation, verification, and/or modification of a surgical plan before and during a surgical procedure. Because MR, or in some instances VR, may be used to interact with the surgical plan, this disclosure may also refer to the surgical plan as a “virtual” surgical plan… A surgical plan, e.g., as generated by the BLUEPRINT™ system or another surgical planning platform, may include information defining a variety of features of a surgical procedure, such as features of particular surgical procedure steps to be performed on a patient by a surgeon according to the surgical plan including, for example, bone or tissue preparation steps and/or steps for selection, modification and/or placement of implant components. Such information may include, in various examples, dimensions, shapes, angles, surface contours, and/or orientations of implant components to be selected or modified by surgeons, dimensions, shapes, angles, surface contours and/or orientations to be defined in bone or tissue by the surgeon in bone or tissue preparation steps, and/or positions, axes, planes, angle and/or entry points defining placement of implant components by the surgeon relative to patient bone or tissue. Information such as dimensions, shapes, angles, surface contours, and/or orientations of anatomical features of the patient may be derived from imaging (e.g., x-ray, CT, MM, ultrasound or other images), direct observation, or other techniques.”) Overall one of ordinary skill in the art would have recognized that combining Poltaretskyi with Elay_2017, Hayashi, and Elay_2020, and Farley would result in a system that would allow medical professionals to integrate the generated 3D black body models into practical surgical plans in an intuitive, three-dimensional manner.
Claim 20. Poltaretskyi teaches further comprising the step of using said virtual model for visualizing and diagnosing bony pathologies. ([Par 914] “For example, during preoperative phase 302 (FIG. 3) and intraoperative phase 306 (FIG. 3), orthopedic surgical system 100 (FIG. 1) may provide XR visualizations (e.g., MR visualizations or VR visualizations) that include patient-specific virtual 3D models of a patient's ankle anatomy.” [Par 223] “In the example of FIG. 8, a model of the area of interest is generated (800). For example, a scan (e.g., a CT scan, MRI scan, or other type of scan) of the area of interest may be performed. For example, if the area of interest is the patient's shoulder, a scan of the patient's shoulder may be performed. Furthermore, a pathology in the area of interest may be classified (802). In some examples, the pathology of the area of interest may be classified based on the scan of the area of interest. For example, if the area of interest is the user's shoulder, a surgeon may determine what is wrong with the patient's shoulder based on the scan of the patient's shoulder and provide a shoulder classification indicating the classification or diagnosis, e.g., such as primary glenoid humeral osteoarthritis (PGHOA), rotator cuff tear arthropathy (RCTA) instability, massive rotator cuff tear (MRCT), rheumatoid arthritis, post-traumatic arthritis, and osteoarthritis.”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael P Mirabito whose telephone number is (703)756-1494. The examiner can normally be reached M-F 10:30 am - 6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emerson Puente can be reached at (571) 272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.P.M./Examiner, Art Unit 2187
/EMERSON C PUENTE/Supervisory Patent Examiner, Art Unit 2187
Prosecution Insights