DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 3/27/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-12 and 14-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Tu (U.S. Patent Pub. No 2023/0230231).
Regarding claim 1, Tu teaches an apparatus for viewing images of a vessel and for identifying side branches from an image, comprising (¶1 The present invention relates to the field of medical image processing, in particular to a training method and apparatus for angiography image processing, a method and apparatus for automatically processing a vessel image:)
a processor and a memory storage device coupled to the processor, the memory comprising instructions executable by the processor, the instructions when executed cause the apparatus to (¶39 The present invention provides a computer device, including a processor and a memory, wherein the memory stores at least one instruction:)
receive an image frame associated with a vessel of a patient (Fig. 7, ¶103 the present invention provides a method for automatically processing a vessel image, including: data containing a to-be-processed original angiography image are obtained;)
infer, using a plurality of initial machine learning (ML) models, indications of at least one side branch of the vessel from the image frame; and (¶103 the data containing the to-be-processed original angiography image are processed based on the neural network obtained through the above training method for angiography image processing, to obtain data containing a local segmentation image of side branch vessels on a determined main branch vessel; The process of Tu uses a single neural network, however according to MPEP 2144.04 V.C. it is not novel to separate into multiples for individual function.)
infer, using a post-processing model, a side branch mask from the indications of the at least one side branch of the vessel (¶87 referring to FIG. 16A-FIG. 16E, a local segmentation Mask of each side branch vessel is obtained after local segmentation of the side branch vessels, and then all side branch vessel Masks are merged to form the local segmentation result data containing all the side branch vessels to serve as the second label; ¶114 the apparatus for automatically processing the vessel image further includes a post-processing module 46 configured to fuse the second result image data and the segmentation image data of the determined main branch vessel to obtain fused image data.)
wherein the image frame is captured using a first imaging modality, and (Fig. 7, ¶103 the present invention provides a method for automatically processing a vessel image, including: data containing a to-be-processed original angiography image are obtained;)
wherein the image frame can be co-registered to one or more other image frames based on the side branch mask, wherein the one or more other modality image frames are captured using a second imaging modality different than the first imaging modality (claim 27: A method for registering an endoluminal image and an angiography image, the method comprising: obtaining a local segmentation image of a side branch vessel by adopting the method for automatically processing the vessel image of claim 13; obtaining a segmentation image of a main branch vessel to which the side branch vessel belongs; and matching the segmentation image of the main branch vessel and the segmentation image of the subordinate side branch vessel with the endoluminal image of the main branch vessel; ¶3 endoluminal images such as optical coherence tomography (OCT) and intravascular ultrasound (IVUS) can evaluate the stability of coronary plaques with a high resolution.)
Regarding claim 2, Tu teaches the apparatus of claim 1, wherein the first imaging modality is angiography and wherein the second imaging modality is intravascular ultrasound or optical coherence tomography or CT angiography (Tu, ¶121 the present invention further provides a method for registering an endoluminal image and an angiography image, which adopts the method for automatically processing the vessel image in the above embodiment to obtain a local segmentation image of side branch vessels; ¶3 endoluminal images such as optical coherence tomography (OCT) and intravascular ultrasound (IVUS) can evaluate the stability of coronary plaques with a high resolution.)
Regarding claim 3, Tu teaches the apparatus of claim 1, the instructions when executed further cause the apparatus to co-register the first image frame with the one or more other image frames based in part on the side branch mask (Tu, ¶100 an image fusing a main branch vessel image segmented in an angiography image and an original angiography image may be masked, and a contour of the main branch vessel in the original angiography image may be marked after fusing.)
Regarding claim 4, Tu teaches the apparatus of claim 1, the instructions when executed further cause the apparatus to generate a graphical information element comprising indications of the image frame and the side branch mask (Tu, ¶82 the segmented main branch vessel image in the angiography image may be fused with the original angiography image to form an image mask. FIG. 14 presents a schematic diagram of an image corresponding to data formed by fusing the segmentation result data of the main branch vessel and the original angiography image data. After fusion, a contour of the main branch vessel in the original angiography image will be marked. Or, the contour of the main branch vessel may also be directly marked in the original angiography image, so as to obtain the angiography image marked with the main branch vessel) and send the graphical information element to a display (¶109 he present embodiment may set a display step to directly display the output result of the first neural network for the user to view conveniently.)
Regarding claim 5, Tu teaches the apparatus of claim 1, the instructions when executed to infer indications of the at least one side branch of the vessel from the image frame cause the apparatus to infer indications of the at least one side branch of the vessel from the image frame and a vessel centerline (Tu, ¶79 With the positions of the cross-sections, perpendicular to the center lines of the side branch vessels, of the bifurcation crest points of the segmented side branch vessels as beginning ends, the segmentation terminals are determined between the head ends and the tail ends of the side branch vessels in the extending direction.)
Regarding claim 6, Tu teaches the apparatus of claim 5, the instructions when executed further cause the apparatus to receive the vessel centerline (Tu, ¶21 the segmentation terminal at least crosses a cross-section, perpendicular to a center line of the side branch vessel, of a bifurcation crest point of the segmented side branch vessel in the extending direction; this shows the centerline is received or determined.)
Regarding claim 7, Tu teaches the apparatus of claim 5, the instructions when executed further cause the apparatus to determine the vessel centerline (Tu, ¶21 the segmentation terminal at least crosses a cross-section, perpendicular to a center line of the side branch vessel, of a bifurcation crest point of the segmented side branch vessel in the extending direction; this shows the centerline is received or determined.)
Regarding claim 8, Tu teaches the apparatus of claim 5, wherein the side branch mask comprises an indication of locations of the one or more side branches with respect to the centerline of the vessel (Tu, ¶87 referring to FIG. 16A-FIG. 16E, a local segmentation Mask of each side branch vessel is obtained after local segmentation of the side branch vessels, and then all side branch vessel Masks are merged to form the local segmentation result data containing all the side branch vessels.)
Regarding claim 9, Tu teaches the apparatus of claim 1, the instructions when executed to infer indications of the at least one side branch of the vessel from the image frame cause the apparatus to infer indications of the at least one side branch of the vessel from the image frame and a segmentation of the image frame (Tu, ¶79 With the positions of the cross-sections, perpendicular to the center lines of the side branch vessels, of the bifurcation crest points of the segmented side branch vessels as beginning ends, the segmentation terminals are determined between the head ends and the tail ends of the side branch vessels in the extending direction.)
Regarding claim 10, Tu teaches the apparatus of claim 9, the instructions when executed further cause the apparatus to receive the segmentation of the image frame or determine the segmentation of the image frame (Tu, ¶87 As for making of the second sample, for example, referring to FIG. 15, FIG. 15 is the side branch vessels locally segmented according to the original angiography image in FIG. 12, the image data in FIG. 15 are correspondingly made into second label data, the correspondingly obtained second label is the local segmentation result data of the side branch vessels, and the second label made through the method may be configured to implement semantic segmentation of the side branch vessels. Or, referring to FIG. 16A-FIG. 16E, a local segmentation Mask of each side branch vessel is obtained after local segmentation of the side branch vessels, and then all side branch vessel Masks are merged to form the local segmentation result data containing all the side branch vessels to serve as the second label.)
Regarding claim 11, Tu teaches the apparatus of claim 1, the instructions when executed further cause the apparatus to:
receive a plurality of intravascular images of the vessel; and infer indications of the at least one side branch of the vessel from the image frame and the plurality of intravascular image frames (Tu, ¶73 local segmentation avoids the problems of segmentation deviation caused by overlapping and exudation effects, segmentation results are accurate, the segmentation difficulty is greatly lowered, and the segmentation time of the angiography images is shortened; This shows that the process taught by Tu can be used for multiple angiographic images.)
Regarding claim 12, Tu teaches the apparatus of claim 11, the plurality of initial ML models comprising a first group ML models and a second group of ML models, wherein the first group of ML models is different than the second group of ML models, and the instructions when executed to infer indications of the at least one side branch of the vessel from the image frame and the plurality of intravascular image frames cause the apparatus to:
infer, using the first group of ML models, ones of the indications of the at least one side branch of the vessel from the image frame; and infer, using the second group of ML models, other ones of the indications of the at least one side branch of the vessel from the plurality of intravascular images (Tu, ¶74 When the angiography image is matched with the OCT or IVUS images, positions, sizes and other information of intersections on the OCT or IVUS images can be compared with positions, sizes and other information of the intersections on the segmented angiography image of the vessels, the angiography image can be matched with intravascular images in position, and the matching between the intravascular images and coronary angiography is realized without changing a current operation process; The process of Tu uses a single neural network, however according to MPEP 2144.04 V.C. it is not novel to separate into multiples for individual function.)
Regarding claim 14, claim 14 has been analyzed with regard to claim 1 and is rejected for the same reasons of anticipation as used above as well as in accordance with Tu further teaching on: A computer-readable storage device, comprising instructions executable by a processor of a computing device coupled to an intravascular imaging device and/or a fluoroscope device (Fig. 10; ¶40 The present invention provides a computer readable storage medium, storing at least one instruction , wherein the at least one instruction, when executed, implements the above training method for angiography image processing, or the above method for automatically processing the vessel image.)
Claim 15 recites limitations similar to claim 2 and is rejected under the same reasoning.
Claim 16 recites limitations similar to claim 3 and is rejected under the same reasoning.
Claim 17 recites limitations similar to claim 4 and is rejected under the same reasoning.
Claim 18 recites limitations similar to claim 5 and is rejected under the same reasoning.
Regarding claim 19, claim 19 has been analyzed with regard to claim 1 and is rejected for the same reasons of anticipation as used above as well as in accordance with Tu further teaching on: A method for a vascular co-registration system (¶41 The present invention further provides a method for registering an endoluminal image and an angiography image.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Tu (U.S. Patent Pub. No. 2023/0230231) in view of Wang (U.S. Patent Pub. No. 2025/0000588) in view of Halmann (U.S. Patent Pub. No. 2023/0131115).
Regarding Claim 13, Tu teaches the apparatus of claim 1 (see rejection of claim 1)
Tu does not explicitly disclose wherein the side branch mask is a one-dimensional (1D) mask defined with respect to the image frame,
wherein the image frame is one of a plurality of image frames in a cine loop, and/or wherein the image frame is a 256 pixel by 256 pixel angiography image (Tu teaches an angiography image and it could be a simple design choice by a person skilled in the art to preset the size of the image to 256x256 pixels for computational efficiency, but Halmann is referenced below to explicitly teach using a frame from a cine loop.)
Wang is in the same field of art of image analysis. Further, Wang teaches wherein the side branch mask is a one-dimensional (1D) mask defined with respect to the image frame (143 the positioning information (one dimensional) of the target structure mask may include position information of a bounding rectangle of the target structure mask, such as coordinate information of a border line of the bounding rectangle.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tu by using a one-dimensional mask that is taught by Wang; thus, one of ordinary skilled in the art would be motivated to combine the references such that the segmentation accuracy and efficiency can be improved while reducing the amount of segmentation calculation, thereby saving memory resources (Wang ¶150).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Halmann is in the same field of art of image analysis. Further, Halmann teaches wherein the image frame is one of a plurality of image frames in a cine loop, and/or wherein the image frame is a 256 pixel by 256 pixel angiography image (¶26 FIGS. 2 and 6-8, when analyzing a particular ultrasound image 202 and/or the image data utilized to form the image 202 or a particular frame of a video or cine loop corresponding to the image 202, in block 300 the detection and recognition system 200 determines for each video or cine frame or ultrasound image 202)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Tu by using a frame from a cine loop that is taught by Halmann; thus, one of ordinary skilled in the art would be motivated to combine the references for accurate indications (Halmann ¶6).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. THEMELIS (U.S. Patent Pub. No. 2023/0248464) ¶102 teaches the usage of a machine-learning algorithm may imply the usage of an underlying machine-learning model (or of a plurality of underlying machine-learning models).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
Prosecution Insights