DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, or 365(c) is acknowledged.
Oath/Declaration
Oath/Declaration as filed on December 5, 2024 is noted by the Examiner.
Claim Objections
Claim 2 is objected to because of the following informalities:
The claim recites limitation term “one or more morbid bones of the patient” in first thru second lines of the claim, but it not exactly clear whether the limitation term is referring to a same one or more bones of a patient recited in third thru fourth lines of claim 1, or to different bones. Therefore, Examiner suggests the limitation term should be amended, without adding new matter, in a manner that clarifies exactly what the limitation term is referring to. In addition, any claim(s) dependent on claim 2 are objected to based on same above reasoning.
Claim 13 is objected to because of the following informalities:
The claim recites limitation term “one or more morbid bones of the patient” in second of the claim, but it not exactly clear whether the limitation term is referring to a same one or more bones of a patient recited in third thru fourth lines of claim 12, or to different bones. Therefore, Examiner suggests the limitation term should be amended, without adding new matter, in a manner that clarifies exactly what the limitation term is referring to. In addition, any claim(s) dependent on claim 13 are objected to based on same above reasoning.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1, 8, and 11 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 11 and 16 of copending Application No. 18/872,550. Although the claims at issue are not identical, they are not patentably distinct from each other because the scope of the independent claims, mentioned above, are substantially the same.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. The following is an example for comparing claim 1 of this application and respective claim 11 of copending Application No. 18/872,550:
Instant Application
Co-pending Application No.
18/872,550
Claim 1
Claim 11
A method for estimating landmarks on a morbid bone, the method comprising: obtaining, by a computing system, a first point cloud representing one or more bones of a patient;
A method for surgical planning, the method comprising: obtaining, by a computing system, a first point cloud representing at least a portion of a bone;
processing, by the computing system, the first point cloud using one or more point cloud neural networks to generate an output point cloud, the output point cloud including labels indicating locations of one or more landmarks on the one or more bones of the patient; and outputting, by the computing system, the output point cloud.
applying, by the computing system, a point cloud neural network to generate a second point cloud based on the first point cloud, the second point cloud comprising points representing an axis along the bone; and generating, by the computing system, surgical planning information based on the second point cloud.
Independent claim 1 of the instant application teaches “A method for estimating landmarks on a morbid bone, the method comprising: obtaining, by a computing system, a first point cloud representing one or more bones of a patient; processing, by the computing system, the first point cloud using one or more point cloud neural networks to generate an output point cloud, the output point cloud including labels indicating locations of one or more landmarks on the one or more bones of the patient; and outputting, by the computing system, the output point cloud”. The co-pending patent application 18/872,550 does not expressly teach: outputting, by the computing system, the output point cloud. However, Zimmermann teaches outputting, by the computing system, the output point cloud (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Furthermore, co-pending patent application 18/872,550 and Zimmermann are considered to be analogous art because they are from the same field of endeavor with respect to a neural network, and involve the same problem of suitably training the neural network. Therefore, before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to modify the method and system of co-pending patent application 18/872,550 based on Zimmermann for outputting, by the computing system, the output point cloud. One reason for the modification as taught by Zimmermann is to have a suitable a surgical system for surgical registration of patient bones to a surgical plan (ABSTRACT of Zimmermann). In addition, in regard to co-pending patent application 18/872,550, it would have been obvious to one of ordinary skill in the art to remove the further limitations “generating, by the computing system, surgical planning information based on the second point cloud”, since at least omitting the further limitation does not prevent the method from performing properly, and the claim is in “comprising” format indicating other elements could even be added. In addition, dependent claims 8 and 11 of the instant application are rejected at least based on same above reasoning.
Claims 12, 19, and 22 is rejected on the ground of nonstatutory double patenting as being unpatentable over claims 28 and 33 of copending Application No. 18/872,550. Although the claims at issue are not identical, they are not patentably distinct from each other because the scope of the independent claims, mentioned above, are substantially the same.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. The following is an example for comparing claim 12 of this application and respective claim 28 of copending Application No. 18/872,550:
Instant Application
Co-pending Application No.
18/872,550
Claim 12
Claim 28
A computing system configured to estimate landmarks on a morbid bone, the computing system comprising: a memory configured to store a first point cloud representing one or more bones of a patient; and one or more processors in communication with the memory, the one or more processors configured to: obtain the first point cloud representing the one or more bones of the patient;
A method for surgical planning, the method comprising: obtaining, by a computing system, a first point cloud representing at least a portion of a bone;
process the first point cloud using one or more point cloud neural networks to generate an output point cloud, the output point cloud including labels indicating locations of one or more landmarks on the one or more bones of the patient; and output the output point cloud.
applying, by the computing system, a point cloud neural network to generate a second point cloud based on the first point cloud, the second point cloud comprising points representing an axis along the bone; and generating, by the computing system, surgical planning information based on the second point cloud.
Independent claim 12 of the instant application teaches “A computing system configured to estimate landmarks on a morbid bone, the computing system comprising: a memory configured to store a first point cloud representing one or more bones of a patient; and one or more processors in communication with the memory, the one or more processors configured to: obtain the first point cloud representing the one or more bones of the patient; process the first point cloud using one or more point cloud neural networks to generate an output point cloud, the output point cloud including labels indicating locations of one or more landmarks on the one or more bones of the patient; andoutput the output point cloud”. The co-pending patent application 18/872,550 does not expressly teach: output the output point cloud. However, Zimmermann teaches output the output point cloud (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Furthermore, co-pending patent application 18/872,550 and Zimmermann are considered to be analogous art because they are from the same field of endeavor with respect to a neural network, and involve the same problem of suitably training the neural network. Therefore, before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to modify the method and system of co-pending patent application 18/872,550 based on Zimmermann to output the output point cloud. One reason for the modification as taught by Zimmermann is to have a suitable a surgical system for surgical registration of patient bones to a surgical plan (ABSTRACT of Zimmermann). In addition, in regard to co-pending patent application 18/872,550, it would have been obvious to one of ordinary skill in the art to remove the further limitations “generating, by the computing system, surgical planning information based on the second point cloud”, since at least omitting the further limitation does not prevent the system from functioning properly, and the claim is in “comprising” format indicating other elements could even be added. In addition, dependent claims 19 and 22 of the instant application are rejected at least based on same above reasoning.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 7, 12-13, and 18 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Zimmermann et al., U.S. Patent Application Publication 2022/0125517 A1 (hereinafter Zimmermann).
Regarding claim 1, Zimmermann teaches a method for estimating landmarks on a morbid bone, the method comprising: obtaining, by a computing system, a first point cloud representing one or more bones of a patient (FIGS. 1-2, and 4A-5A, paragraph[0145] of Zimmermann teaches the classified or segregated 3D bone surface point cloud (Step 600) of the ultrasound based multiple bone registration process 503 starts off with the intraoperative ultrasound images being taken of most, if not all, of the patient surface area surrounding the patient joint and each bone thereof in the vicinity of the patient joint; for example, for a knee arthroplasty, the ultrasound sweeps are over most if not all of the knee, up and down one or more times to get ultrasound image data of the bone surface of each bone (femur, tibia and patella) of the patient knee; the intraoperative ultrasound images are then algorithmically analyzed via machine learning to determine which individual points of millions of individual points of the acquired ultrasound image points belong to each bone of the patient joint, resulting in a classified or segregated point cloud pertaining to each bone; in other words, in the context of a knee arthroplasty, the algorithm appropriately assigns each point or pixel of the intraoperative ultrasound images to its respective bone of the knee joint such that each point or pixel can be said to be classified or segregated to correspond to its respective bone, thereby resulting in the classified or segregated 3D bone surface point cloud; and stated another way, each point or pixel of the intraoperative ultrasound images are transformed into the classified or segregated 3D bone surface point cloud such that the ultrasound image pixels or points of the classified or segregated 3D bone surface point cloud are each correlated to a corresponding bone surface of the patient bones, and See also at least ABSTRACT, paragraphs[0087], [0144], and [0146]-[0149] of Zimmermann (i.e., Zimmermann teaches a system for performing an ultrasound-based registration process that allows for multiple bones of a patient joint to be imaged via ultrasound at one time by performing ultrasound sweeps, wherein the images are analyzed via machine learning to determine which individual points of an images belong to each bone of the patient joint that results in a classified or segregated point cloud associated to each bone));
processing, by the computing system, the first point cloud using one or more point cloud neural networks to generate an output point cloud, the output point cloud including labels indicating locations of one or more landmarks on the one or more bones of the patient; and outputting, by the computing system, the output point cloud (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Regarding claim 2, Zimmermann teaches the method of claim 1, wherein the first point cloud represents one or more morbid bones of the patient, and wherein processing, by the computing system, the first point cloud using the one or more point cloud neural networks to generate the output point cloud comprises: processing, by the computing system, the first point cloud using a first point cloud neural network to generate the output point cloud (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Regarding claim 7, Zimmermann teaches the method of claim 1, wherein the output point cloud includes points representing a target bone of the patient and further includes labels indicating the locations of one or more landmarks on the target bone (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Regarding claim 12, Zimmermann teaches a computing system configured to estimate landmarks on a morbid bone, the computing system comprising: a memory configured to store a first point cloud representing one or more bones of a patient; and one or more processors in communication with the memory, the one or more processors configured to: obtain the first point cloud representing the one or more bones of the patient (100 FIGS. 1-2, and 4A-5A, paragraph[0145] of Zimmermann teaches the classified or segregated 3D bone surface point cloud (Step 600) of the ultrasound based multiple bone registration process 503 starts off with the intraoperative ultrasound images being taken of most, if not all, of the patient surface area surrounding the patient joint and each bone thereof in the vicinity of the patient joint; for example, for a knee arthroplasty, the ultrasound sweeps are over most if not all of the knee, up and down one or more times to get ultrasound image data of the bone surface of each bone (femur, tibia and patella) of the patient knee; the intraoperative ultrasound images are then algorithmically analyzed via machine learning to determine which individual points of millions of individual points of the acquired ultrasound image points belong to each bone of the patient joint, resulting in a classified or segregated point cloud pertaining to each bone; in other words, in the context of a knee arthroplasty, the algorithm appropriately assigns each point or pixel of the intraoperative ultrasound images to its respective bone of the knee joint such that each point or pixel can be said to be classified or segregated to correspond to its respective bone, thereby resulting in the classified or segregated 3D bone surface point cloud; and stated another way, each point or pixel of the intraoperative ultrasound images are transformed into the classified or segregated 3D bone surface point cloud such that the ultrasound image pixels or points of the classified or segregated 3D bone surface point cloud are each correlated to a corresponding bone surface of the patient bones, and See also at least ABSTRACT, paragraphs[0087], [0144], and [0146]-[0149] of Zimmermann (i.e., Zimmermann teaches a system, which has a memory device capable of storing data and/or computer, for performing an ultrasound-based registration process that allows for multiple bones of a patient joint to be imaged via ultrasound at one time by performing ultrasound sweeps, wherein the images are analyzed via machine learning to determine which individual points of an images belong to each bone of the patient joint that results in a classified or segregated point cloud associated to each bone));
process the first point cloud using one or more point cloud neural networks to generate an output point cloud, the output point cloud including labels indicating locations of one or more landmarks on the one or more bones of the patient; andoutput the output point cloud (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Regarding claim 13, Zimmermann teaches the computing system of claim 12, wherein the first point cloud represents one or more morbid bones of the patient, and wherein to process the first point cloud using the one or more point cloud neural networks to generate the output point cloud, the one or more processors are further configured to: process the first point cloud using a first point cloud neural network to generate the output point cloud (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Regarding claim 18, Zimmermann teaches the computing system of claim 12, wherein the output point cloud includes points representing a target bone of the patient and further includes labels indicating the locations of one or more landmarks on the target bone (FIGS. 1-2, 4A-5A, and 12, paragraph[0153] of Zimmermann teaches as indicated in FIG. 5A, in one embodiment, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ either of two alternative classification processes, namely by ultrasound image bone surface via pixel classification neural network (Step 528) or by ultrasound image bone surface detection via likelihood classification neural network (Step 530); in other embodiments, generating classified bone surface pixels 526 from the set of 2D ultrasound images 523, the classification module (Step 527) may employ other processes, such as, for example, deducing the none classification from the distance of the two navigation markers 46, 47 fixed on the bones 10, 11, and/or running a classification algorithm on the resulting point cloud itself, without looking at the image but just the 3D arrangement of the different points; in yet other embodiments, the classification of the point cloud is solved via other non-machine learning processes or other networks besides classification and convolution, such as, for example, random forest; in still further embodiments, the classification of the point cloud may be solved via non-machine learning processes; in one embodiment, the classification of the point cloud may be solved via geometric analysis of the point cloud; for example, such geometric analysis of the point cloud may include splitting main axes like principal component analysis (e.g. in the context of knee arthroplasty), clustering methods like connected component analysis (e.g. in the context of spine/vertebrae procedures), and shape properties like convex/concave/tubular/etc; and ultimately, for purposes of not unduly limiting this disclosure, there is a classification module 527 that receives an image as input and a bone classified point cloud is output from the classification module, and there are many different processes that can be part of the classification module to achieve these ends, and See also at least ABSTRACT, paragraphs[0087], and [0144]-[0152] of Zimmermann (i.e., Zimmermann teaches a classification neural network and a likelihood classification neural network for generating classified bone surface pixels, wherein a classification module runs employs processes that including running a classification algorithm on a resulting point cloud from the ultrasound sweeps, wherein the classification module outputs a classified point cloud, and wherein the classified point cloud is algorithmically categorized as a match to a point cloud of an initial registration and its respective surfaces of a 3D CASD bone model having anatomical landmarks)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Zimmermann, in view of Claessen et al., U.S. Patent Application Publication 2021/0174543 A1 (hereinafter Claessen).
Regarding claim 3 Zimmermann teaches the method of claim 2, further comprising but does not expressly teach training the first point cloud neural network, wherein training the first point cloud neural network comprises: generating a training dataset based on point clouds of a plurality of morbid bones; and training the first point cloud neural network using the training dataset.
However, Claessen teaches training the first point cloud neural network, wherein training the first point cloud neural network comprises: generating a training dataset based on point clouds of a plurality of morbid bones; and training the first point cloud neural network using the training dataset (FIGS. 3A-4C, paragraph[0103] of Claessen teaches FIGS. 4A-C show illustrations of training targets and results as may be used by a method as described with reference to FIGS. 3A-3D; FIG. 4A depicts three slices 4001-3 of a 3D data set, in this example a CBCT scan of a 3D dental structure and associated slices of the 3D voxel maps for the x′, y′ and z′ coordinate as may be used to train a 3D deep neural network; these 3D voxel maps comprise the desired predictions of the canonical x′ coordinate 4021, the canonical y′ coordinate 4022 and the canonical z′ coordinate 4023; the grayscale values visualize the gradients of (encoded) values for coordinates according to the canonical coordinate system; the coordinates (x, y, z) indicate the position of a voxel of the 3D dental structure based on a coordinate system associated with the CBCT scan; the axes as visualized including their directions are denoted top-left per picture; also noteworthy is that the grayscale values of the gradients displayed have been appropriately scaled to have the same grayscale value for the same value across all of FIGS. 4A-C; this allows for better visual comparison of what are effectively translations towards the canonical coordinate system as encoded (for training) or predicted; and finally note that all visualizations are 2D representations of a single middle ‘slice’ (effectively pixels of 2D image data), as sliced from the actually employed 3D data set and the associated voxel maps, as denoted by the slice number visible top-left per illustration (i.e., Claessen teaches a 3D data set, which is of a 3D dental structure, offered for coordinates used to train a 3D deep neural network)).
Furthermore, Zimmermann and Claessen are considered to be analogous art because they are from the same field of endeavor with respect to a neural network, and involve the same problem of suitably training the neural network. Therefore, before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to modify the method and system of Zimmermann based on Claessen for training the first point cloud neural network, wherein training the first point cloud neural network comprises: generating a training dataset based on point clouds of a plurality of morbid bones; and training the first point cloud neural network using the training dataset. One reason for the modification as taught by Claessen is to have a suitable system for automated determination of a canonical pose of a 3D object and for automated superimposition of 3D objects (paragraph[0002] of Claessen). The same motivation and rationale to combine for claim 3 mentioned above, in light of corresponding statement of grounds of rejection, applies to each claim mentioned in the corresponding statement of grounds of rejection.
Regarding claim 14 Zimmermann teaches the computing system of claim 13, but does not expressly teach wherein the one or more processors are further configured to train the first point cloud neural network, wherein to train the first point cloud neural network, the one or more processors are configured to: generate a training dataset based on point clouds of a plurality of morbid bones; and train the first point cloud neural network using the training dataset.
However, Claessen teaches wherein the one or more processors are further configured to train the first point cloud neural network, wherein to train the first point cloud neural network, the one or more processors are configured to: generate a training dataset based on point clouds of a plurality of morbid bones; and train the first point cloud neural network using the training dataset (FIGS. 3A-4C, paragraph[0103] of Claessen teaches FIGS. 4A-C show illustrations of training targets and results as may be used by a method as described with reference to FIGS. 3A-3D; FIG. 4A depicts three slices 4001-3 of a 3D data set, in this example a CBCT scan of a 3D dental structure and associated slices of the 3D voxel maps for the x′, y′ and z′ coordinate as may be used to train a 3D deep neural network; these 3D voxel maps comprise the desired predictions of the canonical x′ coordinate 4021, the canonical y′ coordinate 4022 and the canonical z′ coordinate 4023; the grayscale values visualize the gradients of (encoded) values for coordinates according to the canonical coordinate system; the coordinates (x, y, z) indicate the position of a voxel of the 3D dental structure based on a coordinate system associated with the CBCT scan; the axes as visualized including their directions are denoted top-left per picture; also noteworthy is that the grayscale values of the gradients displayed have been appropriately scaled to have the same grayscale value for the same value across all of FIGS. 4A-C; this allows for better visual comparison of what are effectively translations towards the canonical coordinate system as encoded (for training) or predicted; and finally note that all visualizations are 2D representations of a single middle ‘slice’ (effectively pixels of 2D image data), as sliced from the actually employed 3D data set and the associated voxel maps, as denoted by the slice number visible top-left per illustration, and See also at least ABSTRACT, paragraphs[0022], and Claim 1 of Claessen (i.e., Claessen teaches a 3D data set, which is of a 3D dental structure, offered for coordinates used to train a 3D deep neural network)).
Claims 9 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zimmermann, in view of Ryan et al., U.S. Patent 10,398,514 B2 (hereinafter Ryan).
Regarding claim 9 Zimmermann teaches the method of claim 1, further comprising:; but does not expressly teach generating, by the computing system, based on the output point cloud, a Mixed Reality visualization indicating the locations of one or more landmarks on the one or more bones of the patient.
However, Ryan teaches generating, by the computing system, based on the output point cloud, a Mixed Reality visualization indicating the locations of one or more landmarks on the one or more bones of the patient (FIGS. 6-7, Cols. 11-12, Lines 53-67 and Lines 1-18 respectively of Ryan teach in one exemplary embodiment of the present invention and referring to FIG. 6, the system 10 is used for hip replacement surgery wherein a first marker 600 is attached via a fixture 602 to a pelvis 604 and a second marker 606 is attached to an impactor 608; the user 106 can see the mixed reality user interface image (“MXUI”) shown in FIG. 6 via the display device 104; the MXUI provides stereoscopic virtual images of the pelvis 604 and the impactor 604 in the user's field of view during the hip replacement procedure; the combination of markers (600, 606) on these physical objects, combined with the prior processing and specific algorithms allows calculation of measures of interest to the user 106, including real time version and inclination angles of the impactor 608 with respect to the pelvis 604 for accurate placement of acetabular shell 612; further, measurements of physical parameters from pre- to post-operative states can be presented, including but not limited to change in overall leg length; presentation of data can be in readable form 610 or in the form of imagery including, but not limited, to 3D representations of tools or other guidance forms; FIG. 7 depicts an alternate view of the MXUI previously shown in FIG. 6, wherein a virtual target 700 and a virtual tool 702 are presented to the user 106 for easy use in achieving the desired version and inclination; in this embodiment, further combinations of virtual reality are used to optimize the natural feeling experience for the user by having a virtual target 700 with actual tool 702 fully visible or a virtual tool (not shown) with virtual target fully visible; other combinations of real and virtual imagery can optionally be provided; and presentation of data can be in readable form 704 or in the form of imagery including but not limited to 3D representations of tools or other guidance forms, and See also at least Claims 1-2 of Ryan (i.e., Ryan teaches a mixed reality user interface that provides virtual images of features of a bone structure)).
Furthermore, Zimmermann and Ryan are considered to be analogous art because they are from the same field of endeavor with respect to a surgical system, and involve the same problem of forming the surgical system to allow suitable access to bony structures. Therefore, before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to modify the method and system of Zimmermann based on Ryan for generating, by the computing system, based on the output point cloud, a Mixed Reality visualization indicating the locations of one or more landmarks on the one or more bones of the patient. One reason for the modification as taught by Ryan is to provide suitable positioning, localization, and situational awareness during medical procedures including but not limited to surgical, diagnostic, therapeutic and anesthetic procedures (Col. 1, Lines 18-22 of Ryan). The same motivation and rationale to combine for claim 9 mentioned above, in light of corresponding statement of grounds of rejection, applies to each claim mentioned in the corresponding statement of grounds of rejection.
Regarding claim 20 Zimmermann teaches the computing system of claim 12,; but does not expressly teach wherein the one or more processors are further configured to: generate based on the output point cloud, a Mixed Reality visualization indicating the locations of one or more landmarks on the one or more bones of the patient.
However, Ryan teaches wherein the one or more processors are further configured to: generate based on the output point cloud, a Mixed Reality visualization indicating the locations of one or more landmarks on the one or more bones of the patient (FIGS. 6-7, Cols. 11-12, Lines 53-67 and Lines 1-18 respectively of Ryan teach in one exemplary embodiment of the present invention and referring to FIG. 6, the system 10 is used for hip replacement surgery wherein a first marker 600 is attached via a fixture 602 to a pelvis 604 and a second marker 606 is attached to an impactor 608; the user 106 can see the mixed reality user interface image (“MXUI”) shown in FIG. 6 via the display device 104; the MXUI provides stereoscopic virtual images of the pelvis 604 and the impactor 604 in the user's field of view during the hip replacement procedure; the combination of markers (600, 606) on these physical objects, combined with the prior processing and specific algorithms allows calculation of measures of interest to the user 106, including real time version and inclination angles of the impactor 608 with respect to the pelvis 604 for accurate placement of acetabular shell 612; further, measurements of physical parameters from pre- to post-operative states can be presented, including but not limited to change in overall leg length; presentation of data can be in readable form 610 or in the form of imagery including, but not limited, to 3D representations of tools or other guidance forms; FIG. 7 depicts an alternate view of the MXUI previously shown in FIG. 6, wherein a virtual target 700 and a virtual tool 702 are presented to the user 106 for easy use in achieving the desired version and inclination; in this embodiment, further combinations of virtual reality are used to optimize the natural feeling experience for the user by having a virtual target 700 with actual tool 702 fully visible or a virtual tool (not shown) with virtual target fully visible; other combinations of real and virtual imagery can optionally be provided; and presentation of data can be in readable form 704 or in the form of imagery including but not limited to 3D representations of tools or other guidance forms, and See also at least Claims 1-2 of Ryan (i.e., Ryan teaches a processor unit for communication with a head worn display to provide a mixed reality user interface that provides virtual images of features of a bone structure)).
Claims 11 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Zimmermann, in view of Dhruwdas, U.S. Patent Application Publication 2017/0323443 A1 (hereinafter Dhruwdas).
Regarding claim 11 Zimmermann teaches the method of claim 1, further comprising:; but does not expressly teach determining one or more of a tibia mechanical axis, a tibia anatomical axis, a tibia axial plane, a tibia medial gutter line, a fibula lateral gutter line, a tibial mortise AP (anteroposterior) axis, or a tibial mortise ML (mediolateral) axis based on the locations of the one or more landmarks.
However, Dhruwdas teaches determining one or more of a tibia mechanical axis, a tibia anatomical axis, a tibia axial plane, a tibia medial gutter line, a fibula lateral gutter line, a tibial mortise AP (anteroposterior) axis, or a tibial mortise ML (mediolateral) axis based on the locations of the one or more landmarks (FIG. 3B, paragraph[0049] of Dhruwdas teaches for a tibia, the following landmarks were identified, on an AP view X-ray image: Tibial Proximal-Lateral condylar landmark—a position of the Extreme lateral point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour; Tibial Proximal-Medial condylar landmark—a position of the Extreme medial point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour; Tibial Distal-Lateral condylar landmark—position of the Extreme lateral point perpendicular to the direction of tibial anatomical axis, of the tibial distal condyle region of the contour; and Tibial Distal-Medial condylar landmark position of the Extreme medial point perpendicular to the direction of tibial anatomical axis, of the tibial distal condyle region of the contour, and See also at least paragraphs[0033] and [0089] of Dhruwdas (i.e., Dhruwdas teaches at least identifying a landmark on an X-ray image, wherein the landmark includes a position of a point perpendicular to a direction of tibial anatomical axis)).
Furthermore, Zimmermann and Dhruwdas are considered to be analogous art because they are from the same field of endeavor with respect to a surgical system, and involve the same problem of forming the surgical system to allow suitably provide images of a bone. Therefore, before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to modify the method and system of Zimmermann based on Dhruwdas for determining one or more of a tibia mechanical axis, a tibia anatomical axis, a tibia axial plane, a tibia medial gutter line, a fibula lateral gutter line, a tibial mortise AP (anteroposterior) axis, or a tibial mortise ML (mediolateral) axis based on the locations of the one or more landmarks. One reason for the modification as taught by Dhruwdas is to transform 2D anatomical X-ray images into 3D renderings for surgical preparation (ABSTACT and paragraph[0004] of Dhruwdas). The same motivation and rationale to combine for claim 11 mentioned above, in light of corresponding statement of grounds of rejection, applies to each claim mentioned in the corresponding statement of grounds of rejection.
Regarding claim 22 Zimmermann teaches the computing system of claim 12, wherein the one or more processors are further configured to:; but does not expressly teach determine one or more of a tibia mechanical axis, a tibia anatomical axis, a tibia axial plane, a tibia medial gutter line, a fibula lateral gutter line, a tibial mortise AP (anteroposterior) axis, or a tibial mortise ML (mediolateral) axis based on the locations of the one or more landmarks.
However, Dhruwdas teaches determining one or more of a tibia mechanical axis, a tibia anatomical axis, a tibia axial plane, a tibia medial gutter line, a fibula lateral gutter line, a tibial mortise AP (anteroposterior) axis, or a tibial mortise ML (mediolateral) axis based on the locations of the one or more landmarks (FIGS. 1 and 3B, paragraph[0049] of Dhruwdas teaches for a tibia, the following landmarks were identified, on an AP view X-ray image: Tibial Proximal-Lateral condylar landmark—a position of the Extreme lateral point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour; Tibial Proximal-Medial condylar landmark—a position of the Extreme medial point perpendicular to the direction of tibial anatomical axis, of the tibial proximal condyle region of the contour; Tibial Distal-Lateral condylar landmark—position of the Extreme lateral point perpendicular to the direction of tibial anatomical axis, of the tibial distal condyle region of the contour; and Tibial Distal-Medial condylar landmark position of the Extreme medial point perpendicular to the direction of tibial anatomical axis, of the tibial distal condyle region of the contour, and See also at least paragraphs[0033] and [0089] of Dhruwdas (i.e., Dhruwdas teaches processors for at least identifying a landmark on an X-ray image, wherein the landmark includes a position of a point perpendicular to a direction of tibial anatomical axis)).
Potentially Allowable Subject Matter
Claims 4-6, 8, 15-17, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten to overcome applicable double patenting rejection(s) and objection(s), if any, indicated above, and if rewritten in independent form including all of the limitations of the base claim and any intervening claims, because for each of claims 4-6, 8, 15-17, and 19 the prior art references of record do not teach the combination of all element limitations as presently claimed.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDUL-SAMAD A ADEDIRAN whose telephone number is (571)272-3128. The examiner can normally be reached on Monday through Thursday, 8:00 am to 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amr Awad can be reached on 571-272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDUL-SAMAD A ADEDIRAN/Primary Examiner, Art Unit 2621
Prosecution Insights