DETAILED ACTION
Response to Arguments
Applicant’s amendments to the claims submitted 10/27/2025 have been recorded. As such Examiner’s previous objections to claims 4 and 5 are withdrawn.
Applicant has amended claims 1, 4, 5, 7, and 20; claims 1-20 are currently pending.
Applicant’s arguments, filed 10/27/2025, with respect to the rejection(s) of claim(s) 20 under 35 U.S.C. 102 and claim(s) 1-19 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Tian (Tian Z, Shen C, Chen H, He T. Fcos: Fully convolutional one-stage object detection. InProceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 9627-9636).), as detailed below. Therefore this action is made FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee (CenterMask: Real-Time Anchor-Free Instance Segmentation from applicant’s admitted prior art) in view of Tian (Tian Z, Shen C, Chen H, He T. Fcos: Fully convolutional one-stage object detection. InProceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 9627-9636).).
Regarding claim 20, Lee discloses, One or more non-transitory computer-readable media storing data descriptive of a machine-learned segmentation model, the machine-learned segmentation model comprising: (Lee Abstract; computer code implementing the disclosed method can be found at the github site listed, and is therefore stored in memory.) a feature extractor model configured to receive input tensor data and, in response to receipt of the input tensor data, produce as output a feature map representative of one or more features of the input tensor data; (Lee Fig. 2; it can be seen that an image (tensor) is input to the Backbone + FPN which create a feature map in a feature pyramid.) an anchor-free detector model configured to detect one or more objects of the input data, the anchor-free detector model comprising one or more tensor heads configured to receive the feature map and, in response to receipt of the feature map, produce as output one or more output object tensors descriptive of objects within the feature map; and (Lee Fig. 2 and Section 2.1 FCOS; feature maps are passed to FCOS Box Head which produces bounding boxes around objects of interest. It is an anchor free method.) an instance segmentation branch configured to provide a segmentation of the input tensor data, the instance segmentation branch comprising: (Lee Fig. 2 and Section 1. Introduction ¶3 found on p. 2; the SAG-Mask block outputs instance segmentation for the detected objects.) a per-instance crop model configured to crop a cropped region from (Lee Fig. 2 and Section 2.1 FCOS; feature maps are passed to FCOS Box Head which produces bounding boxes around objects of interest. It is an anchor free method.) a deep mask head network configured to receive at least the cropped region and, in response to receipt of the at least cropped region, produce as output the segmentation of the input tensor data. (Lee Fig. 2 and Section 1. Introduction ¶3 found on p. 2; the SAG-Mask block receives bounding boxes of detected objects (cropped region) outputs instance segmentation for the detected objects.)
Lee utilizes FCOS (Lee Section 2.1 FCOS), however specific algorithmic details are not discussed and Lee does not explicitly disclose a pixel embedding model configured to receive the feature map and, in response to receipt of the feature map, produce as output an embedding map of the feature map; a per-instance crop model configured to crop a cropped region from the embedding map of the feature map.
However, Tian discloses a pixel embedding model configured to receive the feature map and, in response to receipt of the feature map, produce as output an embedding map of the feature map; (Tian Section 3.3 Center-ness for FCOS – found on p. 9631 and Fig. 3; “center-ness” is calculated which is the normalized distance of a point in the image to the center of an object, as can be visualized by the heat map in Fig. 3. Fig. 2 shows that the center-ness is calculated from feature maps (see box on right).) a per-instance crop model configured to crop a cropped region from the embedding map of the feature map. (Tian Section 3.3 Center-ness for FCOS - ¶2 – found on p. 9631; bounding boxes for detected objects (cropped regions) are weighted by the “center-ness” embedding to filter out low-quality bounding boxes. Therefore the bounded regions are found using the center-ness (i.e. the embedding).)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the computer readable media of Lee with the teaches of Tian by including the algorithmic details specified in Tian in order to remove low-quality bounding boxes from the possible crops that are further segmented and improve detection performance (Tian Section 3.3. Center-ness for FCOS ¶2- found on p. 9631).
Claim(s) 1 and 4-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee (CenterMask: Real-Time Anchor-Free Instance Segmentation from applicant’s admitted prior art) in view of Tian (Tian Z, Shen C, Chen H, He T. Fcos: Fully convolutional one-stage object detection. InProceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 9627-9636).) and Abedalla (Abedalla A, Abdullah M, Al-Ayyoub M, Benkhelifa E. The 2ST-UNet for pneumothorax segmentation in chest X-Rays using ResNet34 as a backbone for U-Net. arXiv preprint arXiv:2009.02805. 2020 Sep 6.).
Regarding claim 1, Lee discloses A computer-implemented method (Lee Abstract; computer code implementing the disclosed method can be found at the github site listed.) for partially supervised image segmentation having improved strong mask generalization, the method comprising: obtaining, by a computing system comprising one or more computing devices, (Cheng Section 2.3 Object Detection Using Segmentation Results and Contour Tracking – ¶3 – found on p. 5; the algorithm improving function of memory of a computer is discussed, and is therefore implemented on a computer with memory.) a machine-learned segmentation model, the machine-learned segmentation model comprising a feature extractor model, an anchor-free detector model, a pixel embedding model, a per-instance crop model, (Lee Fig. 2 and Section 2.1 FCOS; feature maps (see Backbone +FPN block) are passed to FCOS Box Head which detects objects of interest and produces bounding boxes around objects of interest. It is an anchor free method. “Center-ness” is disclosed as an embedding and instance crops can be seen coming out of the FCOS Box Head in Fig. 2) and a deep mask head network, the deep mask head network comprising an encoder-decoder structure having a plurality of layers; (Lee Fig. 2 and Section 1. Introduction ¶3 found on p. 2; the SAG-Mask block outputs instance segmentation for the detected objects.) obtaining, by the computing system, input data comprising tensor data; providing, by the computing system, the input data as input to the machine-learned segmentation model; (Lee Fig. 2; it can be seen that an image (tensor) is input, and Section 2.2 Architecture ¶1 describes how the segmentation is created in a per-pixel manner inside cropped region of the image.) producing, by the computing system and using the feature extractor model, a feature map representative of one or more features of the input tensor data from the input tensor data; and (Lee Fig. 2 and Section 2.1 FCOS; It can be seen in the Backbone+FPN box of figure 2 that feature maps are passed to FCOS Box Head.) cropping, by the computing system and using the per-instance crop model, a cropped region from (Lee Fig. 2 and Section 2.1 FCOS; feature maps are passed to FCOS Box Head which produces bounding boxes around objects of interest. It is an anchor free method.) producing, by the computing system and using the deep mask head network, a segmentation of the tensor data from the cropped region, the segmentation comprising one or more instance masks; (Lee Fig. 2 and Section 1. Introduction ¶3 found on p. 2; the SAG-Mask block receives bounding boxes of detected objects (cropped region) outputs instance segmentation for the detected objects.) and receiving, by the computing system, output data from the machine-learned segmentation model, the output data comprising a segmentation of the tensor data, the segmentation comprising the one or more instance masks. (Lee Fig. 2 and Section 2.2 Architecture ¶1; per-pixel segmentations (instance) are output from the SAG-Mask block.)
Lee utilizes FCOS (Lee Section 2.1 FCOS), however specific algorithmic details are not discussed and Lee does not explicitly disclose producing, by the computing system and using the pixel embedding model, an embedding map of the feature map; cropping, by the computing system and using the per-instance crop model, a cropped region from the embedding map of the feature map;
However, Tian producing, by the computing system and using the pixel embedding model, an embedding map of the feature map; (Tian Section 3.3 Center-ness for FCOS – found on p. 9631 and Fig. 3; “center-ness” is calculated which is the normalized distance of a point in the image to the center of an object, as can be visualized by the heat map in Fig. 3. Fig. 2 shows that the center-ness is calculated from feature maps (see box on right).) cropping, by the computing system and using the per-instance crop model, a cropped region from the embedding map of the feature map; (Tian Section 3.3 Center-ness for FCOS - ¶2 – found on p. 9631; bounding boxes for detected objects (cropped regions) are weighted by the “center-ness” embedding to filter out low-quality bounding boxes. Therefore the bounded regions are found using the center-ness (i.e. the embedding).)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the computer readable media of Lee with the teaches of Tian by including the algorithmic details specified in Tian in order to remove low-quality bounding boxes from the possible crops that are further segmented and improve detection performance (Tian Section 3.3. Center-ness for FCOS ¶2- found on p. 9631).
The combination of Lee and Tian does not explicitly disclose the deep mask head network comprising an encoder-decoder structure.
Abedalla, however, discloses the deep mask head network comprising an encoder-decoder structure (Lee Fig. and Section 4.2 Model Architecture; a U- Net for segmentation is disclosed with ResNet-34 backbone (having multiple layers). It is disclosed that it has an encoder-decoder structure.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the method of the combination of Lee and Tian with the teachings of Abedalla by including the U-Net encoder/decoder structure, including skip connections from Abedalla in order to reduce information loss as feature maps are down-sampled during segmentation.
Regarding claim 4, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above. They further disclose wherein the encoder-decoder structure comprises an encoder comprising one or more encoder layers of the plurality of layers, the one or more encoder layers configured to reduce dimensionality, (Abedalla Fig. 2 and Section 4.2 Model Architecture; it can be seen on the left (the encoder side) has multiple layers where the dimensionality is reduced down to the bottom layer.) and a decoder comprising one or more decoder layers of the plurality of layers, the one or more decoder layers configured to increase dimensionality. (Abedalla Fig. 2 and Section 4.2 Model Architecture; it can be seen on the right side (the decoder side) has multiple layers where the dimensionality is increased back to the output.)
Regarding claim 5, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 4, as described above. They further disclose wherein the deep mask head network comprises one or more skip connections configured to connect a corresponding encoder layer to a corresponding decoder layer having a same feature map size as the encoder layer. (Abedalla Fig. 2; skip connections can be seen connecting encoder and decoder layers.)
Regarding claim 6, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above. They further disclose, wherein the plurality of layers comprises greater than 10 layers. (Abedalla Fig. 2 and Section 4.2 Model Architecture; the backbone is ResNet-34, which has 34 layers.)
Regarding claim 7, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above. They further disclose, wherein the feature extractor model configured to receive input tensor data and, (Lee Fig. 2; it can be seen that an image (tensor) is input to the Backbone + FPN) in response to receipt of the input tensor data, produce as output a feature map representative of one or more features of the input tensor data; (Lee Fig. 2; it can be seen that an image (tensor) is input to the Backbone + FPN which create a feature map in a feature pyramid.) and wherein providing, by the computing system, the input data as input to the machine- learned segmentation model comprises: providing, by the computing system, the input data as input to the feature extractor model; and receiving, by the computing system, a feature map representative of one or more features of the input data. (Lee Fig. 2; it can be seen that an image (tensor) is input to the Backbone + FPN which create a feature map in a feature pyramid of the input image.)
Regarding claim 8, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 7, as described above. They further disclose, wherein the anchor-free detector model comprises one or more tensor heads configured to receive an input feature map and, in response to receipt of the input feature map, produce as output one or more output object tensors descriptive of objects within the input feature map; (Lee Fig. 2 and Section 2.1 FCOS-2.2 Architecture; it can be seen that an image (tensor) is input to the Backbone + FPN which create a feature map in a feature pyramid. Objects of interest are then cropped to be sent to the segmentation block. This includes a 4D vector (a tensor) with a class label and spatial location.) and wherein providing, by the computing system, the input data as input to the machine- learned segmentation model comprises: providing, by the computing system, the feature map representative of one or more features of the input data to the one or more tensor heads; and receiving, by the computing system, one or more object tensors descriptive of objects within the feature map. (Lee Fig. 2 and Section 2.1 FCOS-2.2 Architecture; it can be seen that an image (tensor) is input to the Backbone + FPN which create a feature map in a feature pyramid. Objects of interest are then cropped to be sent to the segmentation block. This includes a 4D vector (a tensor) with a class label and spatial location.)
Regarding claim 9, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 8, as described above. They further disclose, wherein the one or more object tensors comprise a center heatmap tensor denoting a heatmap of a plurality of object centers, (Lee Fig. 2 and Section 2.1 FCOS; a centerness branch is disclosed that predicts the deviation of a pixel to center of its corresponding bounding box.) a scale tensor trained to regress to the width and height of each object center, (Lee Fig. 2 and Section 2.3 Adaptive RoI Assignment Function; the height and width of each RoI is disclosed.)and an offset tensor comprising a correction term for each of the plurality of object centers to counteract a resolution error. (Lee Fig. 2 and Section 2.3 Adaptive RoI Assignment Function; RoI are predicted from different levels of feature maps at different scales.)
Regarding claim 10, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 7, as described above. They further disclose, wherein the machine-learned segmentation model comprises an instance segmentation branch, the instance segmentation branch comprising (Lee Fig. 2 instance segmentation is output ) a pixel embedding model configured to receive an input feature map and, in response to receipt of the input feature map, produce as output an output embedding map of the input feature map; (Tian Section 3.3 Center-ness for FCOS – found on p. 9631 and Fig. 3; “center-ness” is calculated which is the normalized distance of a point in the image to the center of an object, as can be visualized by the heat map in Fig. 3. Fig. 2 shows that the center-ness is calculated from feature maps (see box on right). Wherein it would have been obvious to utilize the FCOS algorithmic details in Tian in order to remove low-quality bounding boxes from the possible crops.) and wherein providing, by the computing system, the input data as input to the machine- learned segmentation model comprises: providing, by the computing system, the feature map representative of one or more features of the input data to the one or more tensor heads; and receiving, by the computing system, an embedding map of the feature map. (Tian Fig. 2; it can be seen that features maps are feed into heads and “center-ness” is output (see dotted box on the right middle output).)
Regarding claim 11, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 10, as described above. They further disclose, wherein the instance segmentation branch further comprises a per-instance crop model configured to crop a cropped region from the feature map (Lee Fig. 2 and Section 2.1 FCOS-2.2 Architecture; Objects of interest are cropped to be sent to the SAG-Mask block which produces instance segmentation, as can be seen in the output.)
Regarding claim 12, the combination of Lee and Abedalla discloses the claim limitations of claim 11, as described above. They further disclose, wherein the per-instance crop model comprises a ROIAlign model. (Lee Section 2.4 Spatial Attention-Guided Mask; utilizing ROIAlign in the crop model is disclosed.)
Regarding claim 13, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 10, as described above. They further disclose, wherein the instance segmentation branch further comprises a plurality of coordinate embeddings relative to a plurality of object centers. (Lee Section 2.1 FCOS and Section 2.6 Implementation details ¶2-3; a centerness branch is described for the objects describing the center of the bounding box of the RoI. Further centerness is shared with the box branch, and utilized in training loss.)
Regarding claim 14, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 10, as described above. They further disclose, wherein the instance segmentation branch further comprises an instance embedding model configured to extract an embedding vector at each of a plurality of object centers. (Lee Section 2.1 FCOS and Section 2.6 Implementation details ¶2-3; a centerness branch is described for the objects describing the center of the bounding box of the RoI. Further centerness is shared with the box branch, and utilized in training loss.)
Regarding claim 15, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 11, as described above. They further disclose, wherein the deep mask head network is configured to receive at least the cropped region and, in response to receipt of the at least cropped region, produce as output the segmentation of the tensor data. (Lee Fig. 2; the cropped RoI are passed to the SAG-Mask and instance segmentation is output (can be seen as an overlay on the input image, which can be considered a tensor).)
Regarding claim 16, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above. They further disclose, wherein the deep mask head network comprises an hourglass network. (Abedalla Fig. 2 and Section 4.2 Model Architecture; The U-net has an hourglass shape.)
Regarding claim 17, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above. They further disclose, wherein the deep mask head network comprises a ResNet network. (Abedalla Fig. 2 and Section 4.2 Model Architecture; use of ResNet backbone is disclosed.)
Regarding claim 18, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above. They further disclose, wherein a number of channels increases gradually throughout the deep mask head network. (Abedalla Fig. 2 and Section 4.2 Model Architecture; it can be seen that the number of channels increases on the encoder side of the u-net.)
Regarding claim 19, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above. They further disclose, wherein the deep mask head network comprises a bottleneck layer. (Abedalla Fig. 2 and Section 4.2 Model Architecture; a bottle neck layer can be seen at the bottom of the U-net between the encoder and decoder sides.)
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee (CenterMask: Real-Time Anchor-Free Instance Segmentation from applicant’s admitted prior art) in view of Tian (Tian Z, Shen C, Chen H, He T. Fcos: Fully convolutional one-stage object detection. InProceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 9627-9636).), Abedalla (Abedalla A, Abdullah M, Al-Ayyoub M, Benkhelifa E. The 2ST-UNet for pneumothorax segmentation in chest X-Rays using ResNet34 as a backbone for U-Net. arXiv preprint arXiv:2009.02805. 2020 Sep 6.) and Biertimpel (Biertimpel D, Shkodrani S, Baslamisli AS, Baka N. Prior to Segment: Foreground Cues for Weakly Annotated Classes in Partially Supervised Instance Segmentation. arXiv preprint arXiv:2011.11787. 2020 Nov 23.).
Regarding claim 2, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above.
They do not explicitly disclose, wherein the machine-learned segmentation model is trained using a partially supervised segmentation training dataset comprising one or more training data entries comprising ground truth data descriptive of ground truth instance masks for one or more seen classes and ground truth bounding boxes for one or more unseen classes, and wherein the input data comprises data included in one of the unseen classes.
Biertimpel, however, discloses wherein the machine-learned segmentation model is trained using a partially supervised segmentation training dataset (Biertimpel Section 1 Introduction ¶2-3; partially supervised segmentation is disclosed.) comprising one or more training data entries comprising ground truth data descriptive of ground truth instance masks for one or more seen classes and ground truth bounding boxes for one or more unseen classes, and wherein the input data comprises data included in one of the unseen classes. (Biertimpel Section 1 Introduction ¶2-3; seen and unseen classes are disclosed with bounding boxes and full segmentation provided as ground truth for the different classes.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the method of the combination of Lee, Tian, and Abedalla with the teachings of Biertimpel by including the partially supervised segmentation with seen and unseen classes in training data from Biertimpel in order to provide lower cost training data sets on more classes because full segmentations are not needed.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee (CenterMask: Real-Time Anchor-Free Instance Segmentation from applicant’s admitted prior art) in view of Tian (Tian Z, Shen C, Chen H, He T. Fcos: Fully convolutional one-stage object detection. InProceedings of the IEEE/CVF international conference on computer vision 2019 (pp. 9627-9636).), Abedalla (Abedalla A, Abdullah M, Al-Ayyoub M, Benkhelifa E. The 2ST-UNet for pneumothorax segmentation in chest X-Rays using ResNet34 as a backbone for U-Net. arXiv preprint arXiv:2009.02805. 2020 Sep 6.) and Duan (CenterNet: Keypoint Triplets for Object Detection from applicant’s admitted prior art.).
Regarding claim 3, the combination of Lee, Tian, and Abedalla discloses the claim limitations of claim 1, as described above.
They do not explicitly disclose, wherein the anchor-free detector model comprises a CenterNet detector model.
Duan, however, discloses wherein the anchor-free detector model comprises a CenterNet detector model. (Duan Abstract and Fig. 1; using CenterNet to detect bounding boxes of object instances is disclosed.)
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to modify the method of the combination of Lee and Abedalla with the teachings of Duan by including the CenterNet detector model from Duan in order to capture richer and more recognizable visual patterns (Duan Section 3.3 Enriching Center and Corner Information – Center Pooling).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEREDITH TAYLOR whose telephone number is (571)270-5805. The examiner can normally be reached M-Th 7:30-5. Examiner’s email is Meredith.taylor@uspto.gov.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEREDITH TAYLOR/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
