DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Invention III (claims 8-11) in the reply filed on Feb 25th 2026 is acknowledged. The application has pending claims 1-20 (withdrawn claims 2-5, 12-15, 17 and 19-20 are withdrawn from further consideration).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 6–10, 16 and 18 are rejected under 35 U.S.C. §102(a)(1) as being anticipated by Cao (Cao et al, HASSOD: Hierarchical Adaptive Self-Supervised Object Detection, 2023).
Regarding claim 7, Cao teaches a system comprising:
one or more memory devices; and
one or more processors coupled to the one or more memory devices that cause the system to perform operations comprising:
( [Appendix J, “Hyper-Parameters and Implementation Details”]: Cao discloses implementing HASSOD on computing hardware using PyTorch and Detectron2, with training performed on 4× NVIDIA A100 GPUs. )
extracting, utilizing an encoder neural network, features representing a digital image;
( [Sec. 3.1, “Hierarchical Adaptive Clustering”]: Cao discloses using a frozen ViT-B/8 model pre-trained by DINO to extract visual features for each image, where each spatial element in the feature map corresponds to an 8×8 patch in the original image. )
generating, for the digital image, pseudo-labels indicating segmentations of object entities of the digital image and a hierarchical segmentation indicating hierarchical relations among the object entities based on the features; and
( [Sec. 3, “Approach”], [Sec. 3.1, “Hierarchical Adaptive Clustering”], [Fig. 2], [Fig. 3]: Cao discloses generating initial pseudo-labels from unlabeled images using self-supervised visual representations; progressively merging adjacent regions with the highest feature similarities into object masks; recording masks at multiple merging thresholds; and identifying hierarchical levels by analyzing coverage relations between masks and constructing tree structures. )
generating, utilizing a segmentation model comprising parameters generated according to the pseudo-labels, a segmentation map comprising a predicted hierarchical segmentation of the object entities of the digital image
( [Sec. 3.2, “Hierarchical Level Prediction”], [Sec. 3.3, “Mean Teacher Training with Adaptive Targets”], [Fig. 2], [Fig. 4-5]; [Paragraph “Improvement over initial pseudo-labels”]: Cao discloses learning an object detection and instance segmentation model using the initial pseudo-labels generated in the first stage, and further discloses a hierarchical level prediction head that classifies each predicted object as a whole object, part object, or subpart object, such that the learned model outputs predicted hierarchical segmentation results based on parameters learned according to the pseudo-labels. )
Regarding claim 8, Cao teaches the system of claim 7, wherein extracting the features representing the digital image comprises:
determining a plurality of patches of pixels within the digital image; and
extracting, for each patch of the plurality of patches, a feature vector representing visual features of the pixels within the patch.
( [Sec. 3.1, “Hierarchical Adaptive Clustering”], [Fig. 3]: Cao discloses extracting visual features with a frozen DINO ViT-B/8, where each spatial element of the resulting feature map corresponds to an 8×8 patch in the original image, and each patch is initialized as an individual region, thereby teaching patch-wise feature-vector extraction for a plurality of patches in the digital image. )
Regarding claim 9, Cao teaches the system of claim 8, wherein generating the pseudo-labels comprises:
merging at least some of the plurality of patches into a first group of clusters based on similarities of the features at a first merging threshold; and
( [Sec. 3.1, “Hierarchical Adaptive Clustering”] with [Fig. 3], [Sec. 4.5, “Quality of initial pseudo-labels”]: Cao discloses computing the pairwise cosine similarity between adjacent patch regions, iteratively identifying the pair of adjacent regions with the highest feature similarity, merging those regions, and stopping when the similarity becomes smaller than a pre-set threshold θmerge; further recording the derived object masks when one of the pre-set thresholds is reached. )
determining mask hierarchies of the object entities within the digital image from the first group of clusters.
( [Sec. 3.2, “Hierarchical Level Prediction”, Fig. 3]: Cao discloses leveraging coverage relations between masks to construct a forest of trees, where roots are whole objects and descendants are part or subpart objects [object entities], thereby determining mask hierarchies from the clustered masks. )
Regarding claim 10, Cao teaches the system of claim 9, wherein generating the pseudo-labels further comprises:
merging at least some of the plurality of patches into a second group of clusters based on similarities of the features at a second merging threshold;
( [Sec. 3.1, “Hierarchical Adaptive Clustering”, Fig. 3], [Sec. D, “Choosing Thresholds for Hierarchical Adaptive Clustering”]: Cao discloses that it is beneficial to use multiple pre-set thresholds {
θ
i
m
e
r
g
e
}, e.g., 0.1, 0.2, and 0.4, and to record the derived object masks when the currently highest feature similarity reaches one of those thresholds. )
combining the first group of clusters and the second group of clusters into a pool of regions;
( [Sec. 3.1, “Hierarchical Adaptive Clustering”, Fig. 3]: Cao discloses ensembling results from multiple pre-set thresholds {
θ
i
m
e
r
g
e
} and combining the resulting object masks to ensure better coverage of potential objects. )
determining a modified pool of regions by removing duplicate regions from the pool of regions; and
( [Sec. 3.1, “Hierarchical Adaptive Clustering”, Fig. 3], [Sec. 4.5, “Quality of initial pseudo-labels”]: Cao discloses performing post-processing to refine and select the object masks after combining the results from multiple thresholds, and further discloses that post-processing removes a substantial portion of the labels while improving quality. )
determining the mask hierarchies of the object entities within the digital image based on the modified pool of regions.
( [Sec. 3.2, “Hierarchical Level Prediction”], [Appendix J, “Hyper-Parameters and Implementation Details”]: Cao discloses that, after ensembling results from the multiple thresholds and after post-processing, HASSOD identifies the hierarchical levels of each object mask based on coverage relation analysis. )
Regarding claims 1, 6, 16 and 18. The rationale provided for claim 7-10 is incorporated herein. In addition, the method for hierarchical entity segmentation of claim 1 and 6 correspond to the system of claims 7-10, as well as the non-transitory computer-readable medium of claims 16 and 18, and performs the steps disclosed herein. Therefore, the claims are all rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 11 is rejected under 35 U.S.C. §103 as being unpatentable over Cao in view of SOHES (Cao et al, “SOHES: Self-Supervised Open-World Hierarchical Entity Segmentation.” Openreview.net, Jan 16 2024).
Regarding claim 11, Cao teaches the system of claim 10, wherein the one or more processors further cause the system to perform operations comprising:
However, Cao does not clearly teach where SOHES teaches:
selecting a subset of regions from the pool of regions, wherein each region of the subset of regions is smaller than a predetermined threshold percentage of the digital image;
cropping, for each region of the subset of regions, a local image from the digital image;
merging at least some of the plurality of patches within the local image into a reclustered pool of regions based on similarities of the features of the patches within the local image; and
determining the mask hierarchies of the object entities within the digital image based on the reclustered pool of regions.
( [Sec. 3.1, Step 1 “Global Clustering”, Step 2 “Local re-clustering”, Step 4 “Hierarchy Analysis”]; [Fig. 3]; [Appendix A, “Self-exploration”]. SOHES discloses selecting from the pool those candidate regions that are smaller than θsmall% of the total image area, cropping a local image around each such small candidate region, re-clustering the local image with the same patch-feature-similarity procedure used in Step 1 so as to obtain local subregions, and then using the remaining subregions together with the larger regions as the pseudo-label set for hierarchy analysis, thereby determining the mask hierarchies based on the reclustered pool of regions. )
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Cao to further select smaller regions from Cao’s pooled candidate regions, crop local images for those smaller regions, and re-cluster the local images as taught by SOHES, because SOHES expressly teaches that global clustering alone leaves many small regions noisy and that local re-clustering of regions smaller than a threshold percentage of the image improves the quality of the remaining pseudo-labels and the resulting hierarchy analysis. Such a modification would have been no more than applying SOHES’s local-refinement technique to Cao’s known hierarchical adaptive clustering pipeline to obtain the predictable result of improved small-object discovery and more accurate whole/ part/ subpart hierarchical relationships.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEN KUDO whose telephone number is (571)272-4498. The examiner can normally be reached M-F 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KEN KUDO
Examiner
Art Unit 2671
/KEN KUDO/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights