Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to the applicant's communication filed on 11/15/2024. In virtue of this communication, claims 1-20 filed on 11/15/2024 are currently pending in the instant application.
Information Disclosure Statement
The information Disclosure statement (IDS) form PTO-1449, filed on 11/11/2024 and 01/01/2026 are in compliance with the provisions of CFR 1.97. Accordingly, the information disclosed therein was considered by the examiner.
Drawings
The drawings received on 11/15/2024 have been reviewed by Examiner and they are acceptable.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. The independent claim(s) 1 and 11 recite(s) obtaining serialized medical images; receiving a first manual annotation on a first slice image of the serialized medical images and a second manual annotation on a second slice image of the serialized medical images; and execute a bidirectional inference mechanism to generate final annotation labels on intermediate slice images of the serialized medical images based on the first manual annotation and the second manual annotation respectively.
Step 1:
With regard to Step 1, the instant claims are directed to an apparatus and a method, , all among the statutory categories of invention.
Step 2A — Prong 1:
With regard to Step 2A — Prong 1, for example in method Claim 11, the limitations “obtaining serialized medical images;” “receiving a first manual annotation on a first slice image of the serialized medical images and a second manual annotation on a second slice image of the serialized medical images;” and “execute a bidirectional inference mechanism to generate final annotation labels on intermediate slice images of the serialized medical images based on the first manual annotation and the second manual annotation respectively”, as recited, is a method that, under its broadest reasonable interpretation, covers performance of the limitation in the mind/observation of a person inspecting a medical image/picture of a body, and annotate or label the images based on requested inquiry and one may render an opinion about how to do the annotation from first to end or …. That is, other than reciting “by a computer" nothing in the claim steps preclude the limitations from practically being performed in the mind or through observation of a person inspecting an image. The recited computer is simply a generic device. If a claim limitation, under its broadest reasonably interpretation covers performance of the limitation in the mind but for the recitation of a generic components, then it falls within the "Mental processes" grouping of the abstract idea, which include concepts performed in the human mind, including an observation, evaluation, judgement, opinion. Accordingly, the claim recites an abstract idea. In addition, the additional components recited in independent Claims 1 and 11, i.e., a memory, a processor, and a non-transitory computer-readable medium are simply generic computing components, accordingly, these independent claims include the above- described abstract idea.
Step 2A — Prong 2:
The 2019 PEG defines the phrase “integration into a practical application’ to require an additional element or a combination of additional elements in the claim
to apply, rely on, or use the judicial exception. In the instant case, the additional elements in the claims do not apply, rely on, or use the judicial exception.
This judicial exception is not integrated into a practical application because the claims only recite additional elements using a computer, a memory, or a processor, for instance, that includes to perform the recited elements/functions/steps. These computing components in all are recited at high-level of generality and there are no other recited additional limitations in the claims. Accordingly, these additional steps/elements do not integrate the abstract idea into a practical application because it is a field-of-use limitation that does not impose any meaningful limits on practicing the abstract idea. Therefore, independent Claims 1 and 11 recite an abstract idea.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B. The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception because as discussed above with respect to integration of the abstract idea into practical application, the additional element of using a computer, a memory or a processor medium to execute programming instructions to perform the step amounts to no more than mere instructions to apply the exception using a generic apparatus component. Mere instructions to apply an exception using generic apparatus component cannot provide an inventive concept. The claim is not patent eligible.
The remaining dependent claims have been given the full two-part analysis including analyzing the additional limitations both individually and in combination. The dependent claims, when analyzed individually, and in combination, are also held to be patent ineligible under 35 U.S.C. 101. The additional recited limitations of the dependent claims fail to establish that the claims do not recite an abstract idea because the additional recited limitations of the dependent claims merely further narrow the abstract idea. The limitations of the dependent claims fail to integrate an abstract idea into a practical application because the dependent claims do not introduce additional elements; and performing the further narrowed abstract ideas of the dependent claims on the additional elements of independent claims, individually or in combination, does not impose any meaningful limits on practicing the abstract ideas and does not provide improvements to the functioning of computing systems or to another technology or technical field; therefore, the claims amount to merely using a computer, in its ordinary capacity, as a tool to perform the abstract idea. Similarly, the additional recited limitations of the dependent claims fail to establish that the claims provide an inventive concept because claims that merely use a computer, in its ordinary capacity, as a tool to perform the abstract idea cannot provide an inventive concept. The claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 10, 11, and 20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Syeda-Mahmood et al.(US 2018/0182103), hereinafter Syeda.
As per claim 1, A medical imaging annotation device, comprising: an interface;, a memory, storing serialized medical images; and a processor, coupled to the interface and the memory, and the processor being configured to:”(Syeda ,¶[0012],¶[0015-0016], ¶[0041],¶[0044-0045] discloses rapid annotation of medical imaging data, user interaction through Input/Output (I/O) interfaces, series of medical image slices and computer memory and storage devices).
“receive, via the interface, a first manual annotation on a first slice image of the serialized medical images and a second manual annotation on a second slice image of the serialized medical images;” (Syeda, ¶[0021] discloses First slice 401 and second slice 402 are manually labeled.¶[0025] discloses annotations are read for a subset of the 2D slices. ¶[0046] discloses user interaction through Input/Output (I/O) interfaces.)
“and execute a bidirectional inference mechanism to generate final annotation labels on intermediate slice images of the serialized medical images based on the first manual annotation and the second manual annotation respectively.”(Syeda, ¶[0020] discloses For slices that have labels propagated from two manually labeled slices, a patch-based label fusion approach is applied to fuse the labels,¶[0021] discloses two way registration is applied, and then the labels are propagated to intermediate slice 403, followed by label fusion.)
Claim 11 has been analyzed and is rejected for the reasons indicated in claim 1 above.
As per claim 10,The medical imaging annotation device of claim 1, “wherein the serialized medical images comprises Magnetic Resonance Imaging (MRI) scan images or Computed Tomography (CT) scan images.”(Syeda, ¶[0015] discloses 3D cardiac CT anatomy annotation task. ¶[0027] discloses Cardiac CT studies were axially acquired by a Siemens CT Scanner.)
Claim 20 has been analyzed and is rejected for the reasons indicated in claim 10 above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2, 5, 12, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Syeda-Mahmood et al.( US 2018/0182103), hereinafter Syeda, in view of Yan et al. (US 2026/0188008),refer to CN priority date.
As per claim 2, The medical imaging annotation device of claim 1, however Syeda does not explicitly disclose the following which would have been obvious in view of Yan from similar field of endeavor “wherein the bidirectional inference mechanism executed by the processor comprises: “executing a tracking model based on the first manual annotation on the first slice image, for generating forward prediction annotations on the intermediate slice images of the serialized medical images sequentially in a forward order;”(Yan, ¶[0067] discloses a bidirectional propagation manner may be used to annotate the intermediate frame of the target sub-segment. according to the difference between the positions of the intermediate frame and the first frame and the end frame respectively. ¶[0070] discloses the forward propagation algorithm may include an algorithm such as a machine learning algorithm or a neural network algorithm to perform feature propagation on the first frame annotation result of the first frame to the intermediate frame located after the first frame, to obtain the forward propagation feature of the intermediate frame.¶[0072] discloses propagation of on a label of a first frame starting from a first frame to another image located after first frame(frame by frame) until the target image number. The first frame annotation result is used as a feature propagation mask to participate in the feature calculation. Further see ¶[0081] and ¶[0087] discloses the semi-supervised segmentation algorithm is used to perform, frame by frame, backward feature propagation processing on the target sub-segment from the end frame until the backward propagation feature at the image sequence number is obtained.)
“ executing the tracking model based on the second manual annotation on the second slice image, for generating backward prediction annotations on the intermediate slice images of the serialized medical images sequentially in a backward order;”(Yan, ¶[0074] discloses the backward propagation algorithm may include an algorithm such as a machine learning algorithm or a neural network algorithm, and may be obtained by training. The backward propagation algorithm may propagate the end frame annotation result to the intermediate frame before the end frame to obtain the backward propagation feature of the intermediate frame. ¶[0075] discloses the backward propagation feature may refer to an image feature obtained by performing feature propagation, frame by frame, on a label of an end frame to another image located before the end frame, and propagating to an image corresponding to an image sequence number, then stop the propagation. Similarly, the end frame annotation result may also be used as a feature propagation mask to participate in the feature calculation. ¶[0087] discloses the semi-supervised segmentation algorithm is used to perform, frame by frame, backward feature propagation processing on the target sub-segment from the end frame until the backward propagation feature at the image sequence number is obtained. )
“and merging the forward prediction annotations and the backward prediction annotations to produce the final annotation labels on the intermediate slice images.” (Yan, ¶[0076] discloses perform feature fusion processing on the forward propagation feature and the backward propagation feature, to obtain a target image feature of the intermediate frame. ¶[0077] discloses determine the annotation result of the intermediate frame based on the target image feature. ¶[0079] discloses The fusion of the forward propagation feature and the backward propagation feature may enable the target image feature to fuse the first frame annotation result and the end frame annotation result, the annotation feature of the intermediate frame may be better represented through the target image feature, and the annotation precision and accuracy of the intermediate frame are improved.)
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine Yan technique of video annotation into Syeda technique to provide the known and expected uses and benefits of Yan technique over medical image slices registration and annotation technique of Syeda. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Yan to Syeda in order to improve the accuracy, cost, and efficiency of image frame annotations. (Refer to Yan paragraph [0004].)
Claim 12 has been analyzed and is rejected for the reasons indicated in claim 2 above.
As per claim 5, The medical imaging annotation device of claim 2, “wherein regarding to a Kth slice image of the serialized medical images, a final annotation label on the Kth slice image is produced by:”(Yan, ¶[0091] discloses determining an image sequence number of the intermediate frame in the target sub-segment.¶[0095] discloses the image sequence number of the intermediate frame may refer to an occurrence sequence of the intermediate frame in the target sub-segment in the target sub-segment.)
“calculating a first distance between the first slice image and the Kth slice image with a second distance between the Kth slice image and the second slice image;”(Yan, ¶[0095] discloses the position of the intermediate frame in the target sub-segment may be determined through the image sequence number. The corresponding image sequence number of each image frame may be determined according to its annotation sequence. ¶[0098] discloses calculating a sequence number ratio between the image sequence number of the intermediate frame and the end frame sequence number corresponding to the end frame of the target sub-segment. ¶[0101] discloses The image sequence number is K, the sequence number of the end frame is N, and the sequence number ratio is K/N. Determining the forward propagation weight and the backward propagation weight based on the sequence number ratio may include: determining that the sequence number ratio K/N is the backward propagation weight, and determining a difference between the integer 1 and the sequence number ratio, that is, 1−K/N.(K/N represents the normalized distance between first and second endpoints, and 1-K/N represent the normalized remaining distance between Kth image and second endpoint.))
“and generating the final annotation label on the Kth slice image based on a weighted sum between a forward prediction annotation and a backward prediction annotation on the Kth slice image according to the first distance and the second distance.” (¶[0100] discloses performing feature fusion processing weighted summation on the forward propagation feature and the backward propagation feature according to the forward propagation weight and the backward propagation weight to obtain the target image feature of the intermediate frame.¶[0101] discloses determining that the sequence number ratio K/N is the backward propagation weight, and determining a difference between the integer 1 and the sequence number ratio, that is, 1−K/N, which is the forward propagation weight. ¶[0102] discloses Calculating the product of the forward propagation weight: 1−K/N and the forward propagation feature F forward to obtain a first feature; calculating a product of the backward propagation weight: K/N and the backward propagation feature F backward to obtain a second feature; and adding the first feature and the second feature to obtain the target image feature F current. ¶[0110] discloses the target area is used as an annotation result of the intermediate frame. The annotation result obtained from the fused target feature corresponds to the final annotation label.)
Claim 15 has been analyzed and is rejected for the reasons indicated in claim 5 above.
Claim(s) 3 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Syeda-Mahmood et al.(US 2018/0182103), hereinafter Syeda, in view of Yan et al. (US 2026/0188008), further in view of Yang et al. "Track anything: Segment anything meets videos." arXiv preprint arXiv, April(2023).
As per claim 3, The medical imaging annotation device of claim 2, Syeda as modified by Yan further discloses “reveres execution” “ Model is configured to detect, associate, and follow the second manual annotation on the second slice image across the intermediate slice images in the backward order.”(Yan, ¶[0074-0075] disclose the backward propagation algorithm may include an algorithm such as a machine learning algorithm or a neural network algorithm, and may be obtained by training. The backward propagation algorithm may propagate the end frame annotation result to the intermediate frame before the end frame to obtain the backward propagation feature of the intermediate frame and performing feature propagation, frame by frame, on a label of an end frame to another image located before the end frame, and propagating to an image corresponding to an image sequence number, then stop the propagation. Similarly, the end frame annotation result may also be used as a feature propagation mask to participate in the feature calculation. ¶[0088] discloses the semi-supervised segmentation algorithm may be specifically a semi-supervised object segmentation algorithm. The image feature of the current frame may be calculated, for the target sub-segment, by using the image feature of the previous frame starting from the first frame or the end frame through a semi-supervised segmentation algorithm. Until the forward propagation feature or backward propagation feature corresponding to the image sequence number are obtained.)
However Syeda as modified by Yan does not explicitly disclose the following which would have been obvious in view of Yang from similar filed of endeavor “wherein the tracking model comprises a Track-Anything-Model (TAM), the Track-Anything-Model is configured to detect, associate, and follow the first manual annotation on the first slice image across the intermediate slice images in the forward order, and the Track-Anything-Model is configured to detect, associate, and follow the second manual annotation on the second slice image across the intermediate slice images.” (Yang, page 2, paragraph 1, discloses Track Anything Model (TAM) can track and segment any objects in a given video with only one-pass inference. users can interactively initialize the SAM, i.e., clicking on the object, to define a target object. then, XMem is used to give a mask prediction of the object in the next frame according to both temporal and spatial correspondence. Further page 3, section 3.2, step2, discloses Given the initialized mask, XMem performs semi-supervised VOS on the following frames and output predicted masks.)
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine Yang technique of using track anything model (TAM) for object segmentation and tracking into Syeda as modified by Yan technique to provide the known and expected uses and benefits of Yang technique over medical image slices registration and annotation technique of Syeda as modified by Yan. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Yang into Syeda as modified by Yan in order to provide consistent and precise predication for accurately distinguishing objects in challenging scenarios. (Refer to Yang page3, Step 3, and page 4, Step 4.)
Claim 13 has been analyzed and is rejected for the reasons indicated in claim 3 above.
Claim(s) 4 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Syeda-Mahmood et al.( US 2018/0182103), hereinafter Syeda, in view of Yan et al. (US 2026/0188008) , further in view of Yan, Kun, et al. "Two-shot video object segmentation." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Jun 2023, hereinafter Kun.
As per claim 4, The medical imaging annotation device of claim 2, although Syeda as modified by Yan discloses “serialized medical images” (Syeda ,¶[0012]), However Syeda as modified by Yan does not explicitly disclose the following which would have been obvious in view of Kun from similar filed of endeavor “wherein regarding to a Kth slice image of the serialized images, a final annotation label on the Kth slice image is produced by:” (Kun, page 2261, Col. 1, section 3.4, Figure 3, discloses each of two labeled frames is used as a reference to infer predications for unlabeled frames, resulting in two predications for each unlabeled frame, after which one predication is selected.)
“calculating a first distance between the first slice image and the Kth slice image with a second distance between the Kth slice image and the second slice image;” (Kun, page 2261, Col. 1, section 3.4, figure 3 discloses selecting prediction inferred by the labeled frame that is closest to the unlabeled frame.)
“ in response to the first distance is shorter than the second distance, selecting a forward prediction annotation on the Kth slice image as the final annotation label on the Kth slice image;” (Kun, page 2261, Col. 1, section 3.4, figure 3 discloses inferring prediction from a labeled reference frame toward the end frame and selecting the prediction associated with the labeled frame closest to the unlabeled frame.)
“and in response to the second distance is shorter than the first distance, selecting a backward prediction annotation on the Kth slice image as the final annotation label on the Kth slice image.” (Kun, page 2261, Col. 1, section 3.4, figure 3 discloses inference in a reverse manner toward the beginning frame and selection of the prediction associated with the labeled frame closest to the unlabeled frame.)
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine Kun technique of video annotation and object tracking into Syeda as modified by Yan technique to provide the known and expected uses and benefits of Kun technique over medical image slices registration and annotation technique of Syeda as modified by Yan. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Kun into Syeda as modified by Yan in order to reduce error propagation across intermediate images. (Refer to Kun page 2259, Col. 2, paragraph 2.)
Claim 14 has been analyzed and is rejected for the reasons indicated in claim 4 above.
Claim(s) 6 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Syeda-Mahmood et al.( US 2018/0182103), hereinafter Syeda, further in view of Cheng et al. "Tracking anything with decoupled video segmentation." arXiv, Sep 2023.
As per claim 6, The medical imaging annotation device of claim 1, However Syeda does not explicitly disclose the following which would have been obvious in view of Cheng from similar filed of endeavor “wherein the bidirectional inference mechanism executed by the processor comprises: executing a bidirectional XMem model for generating the final annotation labels on the intermediate slice images based on the first manual annotation and the second manual annotation.” (Cheng, page 4, Col. 1, overview section, discloses modifying the video object segmentation model XMem for use as the temporal propagation model. Further Page 8, Col. 2, section 4.3 discloses forward- and backward-propagate from the key frame. Page 18, section C.3, Col. 2, discloses initialize forward and backward propagation from frame tk without incorporating additional image segmentations. The propagation is implemented as standard semi-supervised video object seg mentation inference with the keyframe as initial guidance. During propagation, the internal memory H is updated using model’s own prediction.)
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine Cheng technique of using XMem model in image processing into Syeda technique to provide the known and expected uses and benefits of Cheng technique over medical image slices registration and annotation technique of Syeda. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Cheng into Syeda in order to provide temporal consistency between past and future models. (Refer to Cheng page 4, Col. 1, overview section.)
Claim 16 has been analyzed and is rejected for the reasons indicated in claim 6 above.
Allowable Subject Matter
Claims 7-9, 17-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and on the pending conditions of the rejected and objected matter set forth in this action.
The following is a statement of reasons for the indication of allowable subject matter: the prior art of record, alone or in combination, fails to teach or suggest the limitations set forth by each of claims 7-9 and 17-19.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAGHAYEGH AZIMA whose telephone number is (571)272-1459. The examiner can normally be reached Monday-Friday, 9:30-6:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAGHAYEGH AZIMA/Examiner, Art Unit 2671
Prosecution Insights