DETAILED ACTION
.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6,8-18, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chow ( US patent 11446092, cited from IDS) in view of Mwikirize (US 20210330289).
Regarding claim 1, Chow teaches a method for providing a medical visualization, the method comprising:
providing, by one or more processors of a computing entity, a medical image in a particular image format(315 in Fig. 3; 410 in Fig. 4) to an input of a trained object detection model( 320 in Fig. 3; 140 in Fig. 4).
causing, by the one or more processors, the trained object detection model to be executed to process the medical image to generate processed image information, the processed image information configured to at least identify one or more features within the medical image( 325 in Fig. 3; 430 in fig. 4).
receiving, by the one or more processors, the processed image information from an output of the trained object detection model(330 in Fig. 3; 180 in Fig. 4) ; and
causing, by the one or more processors, display of the medical image with a representation of at least a portion of the processed image information overlaid on the medical image(635 in Fig. 6), wherein the trained object detection model was trained using a training process comprising:
performing an instance segmentation on a plurality of training images( Col 8, line 39-50, image segmentation), each of the plurality of training images being in the particular image format;
assigning semantic labels to each object identified via the instance segmentation( Col 15, line 15-25, images in the data set of anatomical structure training data 210 may be labeled ); and
using at least a portion of the plurality of training images, the instance segmentation of the at least a portion of the plurality of training images, and the semantic labels assigned to objects identified in the at least a portion of the plurality of training images to train an object detection model(Col 15, line 15-25, supervised … learning techniques).
Chow does not expressly teach bony features.
However, Mwikirize teaches bony features ([0069], experiments were performed on freshly excised bovine tissue, porcine tissue and chicken breast overlaid on a lumbosacral spine phantom ).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Chow and Mwikirize, by substituting images and neural network in Chow with images and neural network in Mwikirize, with motivation “for real-time localization of a needle in an ultrasound video”( Mwikirize, [0007]);
Regarding claim 2, Chow in view of Mwikirize teaches the method of claim 1, wherein the processed image information is further configured to identify at least one medical device within the medical image(Mwikirize, [0007], a needle).
Regarding claim 3, Chow in view of Mwikirize teaches the method of claim 2, wherein the at least one medical device is an epidural needle( Mwikirize, [0035], data was collected … with insertion of a 17 gauge (1.5 mm diameter, 90 mm length) Tuohy epidural needle) .
Regarding claim 4, Chow in view of Mwikirize teaches the method of claim 2, wherein the medical image is one of a time series of medical images and the processed image information includes trajectory information indicating a trajectory of the at least one medical device, the trajectory being at least one of a past trajectory of the at least one medical device, a future trajectory of the at least one medical device(Mwikirize, Fig. 5A-5C; Fig. 10), or a goal trajectory of the at least one medical device, and causing display of the representation of the at least a portion of the processed image information overlaid on the medical image comprises causing display of a representation of the trajectory on the medical image.
Regarding claim 5, Chow in view of Mwikirize teaches the method of claim 4, wherein the trajectory information further indicates whether the trajectory of the at least one medical device is likely to result in the at least one medical device reaching a goal destination(Mwikirize ,[0038], tracking small perturbations of the needle tip close to the target ).
Regarding claim 6, Chow in view of Mwikirize teaches the method of claim 1, wherein the plurality of training images comprises one or more time series of training images that are associated with respective trajectory labels( Mwikirize, [0052], The ground-truth needle tip location was determined from intensity changes and tissue deformation … by retrospectively inspecting the frame sequences… the radiologist performed manual labeling of the dataset … and compared … with manual data).
Regarding claim 8, Chow in view of Mwikirize teaches the method of claim 1, wherein the trained object detection model is a single step object detection model(Mwikirize, FIG. 8).
Regarding claim 9, Chow in view of Mwikirize teaches the method of claim 1, wherein the display of the medical image with the representation of the at least a portion of the processed image information overlaid on the medical image occurs in real-time with respect to the providing of the medical image to the trained object detection model( Mwikirize, FIG. 4).
Regarding claim 10, Chow in view of Mwikirize teaches the method of claim 1.
Chow in view of Mwikirize does not expressly teach wherein the particular image format is .png.
However, there are only finite number of image formats known to art. Therefore, it would have been obvious to try, by one of ordinary skill before the effective filing date of claimed invention to choose a particular one of image format, png included, with reasonable expectation of success in image processing.
Regarding claim 11, Chow in view of Mwikirize teaches the method of claim 1.
Chow in view of Mwikirize does not expressly teach wherein the particular image format is a raster-graphics file format that supports lossless data compression.
However, there are only finite number of raster-graphics file format that support lossless data compression known to the art. Therefore, it would have been obvious to try, by one of ordinary skill before the effective filing date of claimed invention to choose a particular one of raster-graphics file format that supports lossless data compression, such as PNG, TIFF or BMP , with reasonable expectation of success in image processing.
Regarding claim 12, Chow in view of Mwikirize teaches the method of claim 1, wherein the trained object detection model is executed via a CPU of the computing entity( Mwikirize, [0036], CPU).
Regarding claim 13, Chow in view of Mwikirize teaches the method of claim 1, wherein the object detection model comprises a feature detector and a prediction model( Mwikirize, [0029]-[0030], detects scene changes caused by needle motion in the ultrasound image scene … each frame (k) is modeled as the previous frame (k−1)) .
Regarding claim 14, Chow in view of Mwikirize teaches the method of claim 13, wherein the feature detector is trained to identify and classify objects within the medical image and the prediction model is configured to generate a prediction based at least in part on identification and classification of objects in the medical image performed by the feature detector ( Mwikirize, [0030], it possible to identify the objects that have moved between the two frames: the needle tip) .
Regarding claim 15, Chow in view of Mwikirize teaches the method of claim 14, wherein the prediction comprises at least one of a goal location for a medical device for a medical procedure( Mwikirize, [0030], it possible to identify the objects that have moved between the two frames: the needle tip) or a most advantageous level for medical intervention.
Claim 16-18 recites the computer for the method in claims 1-6, 8-15. Since Chow teaches a computing system ( 600 in Fig. 6), claims 16-18 are also rejected.
Claim 20 recite the medium for the method in claim 1, and is also rejected.
Claim(s) 7 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chow in view of Mwikirize, further in view of Casey ( US 20230023440)
Regarding claim 7, Chow in view of Mwikirize teaches the method of claim 1.
Chow in view of Mwikirize does not expressly teach wherein the processed image information is further configured to identify a bony feature of the one or more bony features which exhibits the most irregularity of the one or more bony features.
However, Casey teaches to identify a bony feature of the one or more bony features which exhibits the most irregularity of the one or more bony features ( [0023], determining one or more intraoperative correction values … The intraoperative correction values can include at least one of a maximum distraction, lordosis correction, kyphosis correction, scoliosis correction, and spondylolisthesis correction).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Chow in view of Mwikirize with that of Casey, by executing the additional analysis taught by Casey on the images generated by Chow in view of Mwikirize, with motivation of “ generating a surgical plan for achieving the corrected anatomical configuration” ( Casey, Abstract)
Claim 19 recites the computer for the method in claim 7. Since Chow teaches a computing system ( 600 in Fig. 6), claims 19 is also rejected.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIANGENG SUN whose telephone number is (571)272-3712. The examiner can normally be reached 8am to 5pm, EST, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Randolph Vincent can be reached at 571 272 8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JIANGENG SUN
Examiner
Art Unit 2661
/Jiangeng Sun/Examiner, Art Unit 2671