DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 9/21/2026 has been entered. Claims 1-21 remain pending in the application. Amendments to the claims overcome objections set forth in the Non-Final Office Action mailed 6/22/2026 (Hereinafter, the “NFOA”)
Response to Arguments/Remarks re. 35 U.S.C. § 112(b) Rejections
Applicant’s arguments have been considered and are fully persuasive with regards to the claims as amended. Accordingly, the rejections are withdrawn.
Response to Arguments/Remarks re. 35 U.S.C. § 102/103 Rejections
Applicant’s arguments have been fully considered but are not persuasive.
Applicant first states that “Hirota…does not remove images”. This is not within the plain language of the claims, which instead requires that Hirota “remov[es] at least a portion of a plurality of first visual content samples with respect to an object in order to create a plurality of removed portion visual content items…”. Under [0046] of Hirota cited, the masked region meets this requirement. The pixel mask generated has the effect of nullifying pixel values in the image which under the broadest reasonable interpretation, constitutes removing a portion of the visual content samples: Hirota, [0046]: “…where x∈
PNG
media_image1.png
29
29
media_image1.png
Greyscale
d is an image, ω∈[0, 1]d is a person mask, a is a labeled image attribute…” The value of zero directly nullifies the original pixel values of x in that region, thus removing a “portion” of an image, which was merely the sum of its original data. To the extent the mask still defines underlying object boundaries, it is without effect, as the claim merely requires that a “portion” be removed with no limiting recitation as to what this portion specifically corresponded to. The mask results in a scrubbed portion of the image that is then fed to the model for inpainting, which meets the requirements of the claim under the broadest reasonable interpretation.
Applicant further argues that Hirota does not then teach “providing the plurality of removed portion visual content items to a generative machine learning model”, but the paragraph cited in the NFOA specifically recites: [0012] “using a text-guided inpainting model to inpaint a person mask in an original image”. [0046] also cited, further clarifies “This model, guided by t(g), inpaints ω in x with a synthetic person from protected group g described in t(g)”. Therefore, as the masked region is provided to the model, the limitation is taught.
Applicant next argues that the provisional application to which Hirota claims benefit, does not otherwise provide a basis for the portions of the non-provisional application cited. In the screenshot supplied by the Applicant, the examiner notes the text under the section “Notation”, which explicitly defines a binary segmentation mask as the same element-wise matrix multiplication of the original image, in line with paragraph 46 of the non-provisional. As the mask is generated based on the original image, it too has pixel data removed within the broadest reasonable interpretation of the claim. [4.1] of the provisional specification also explicitly states that the mask ‘m’ is provided to the inpainting model, which then “inpaints the masked area”.
Hirota was otherwise not denied its provisional benefit in its requisite case of record, and the examiner also relies on this prior determination as persuasive. See MPEP § 2154.01(b), “As a result of this distinction, the question of whether a patent or published application is actually entitled to priority or benefit with respect to any of its claims is not at issue in determining the date the patent or published application was “effectively filed” for prior art purposes. Thus, there is no need to evaluate whether any claim of a U.S patent document is actually entitled to priority or benefit under 35 U.S.C. 119, 120, 121, 365, or 386, when applying such a document as prior art.”
Applicant further recites disclosure from an Adobe article, which states that nondestructive editing does not “overwrit[e] original image data” and that masking is one such nondestructive technique. The Adobe article was referring only to layer and vector masks in its context – neither of which are comparable to Hirota’s mask, which is a data array mask. The examiner cites in turn, Liu et. al Image Inpainting for Irregular Holes Using Partial Convolutions, stating “Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value)”; and describing a similar process to [0046] in detail:
PNG
media_image2.png
741
690
media_image2.png
Greyscale
The substitution of pixel values with 0 is tantamount to a removal of visual content in the context of Hirota’s [0046] operation, and for meeting the broad language of the claims. Accordingly, the rejection of claims 1, 11, and 12 under 35 U.S.C. § 102 is maintained.
Applicant further challenges rejection of dependent claims under 35 U.S.C. § 102/103, as deficient for Hirota’s alleged failure to teach the elements of base claims 1 and 12. For the reasons outlined above, Hirota sufficiently teaches these elements, and thus all other rejections under 35 U.S.C. § 102/103 are also maintained.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 9-15, and 20-21 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Hirota et. al (US 20250148657 A1) (Hereinafter, “Hirota”).
With respect to claim 1, Hirota discloses:
A method for augmented machine learning ([Abstract]), comprising:
synthesizing a plurality of second visual content samples ([0007] “Using masked person images and text prompts, aspects of the present invention can generate counterfactual images by inpainting only the masked regions, addressing ethical concerns of altering nonconsensual persons and ensuring equal representation of protected groups across attributes”)
wherein synthesizing the plurality of second visual content samples further comprises removing at least a portion of a plurality of first visual content samples with respect to an object in order to create a plurality of removed portion visual content items ([0007] “Using masked person images and text prompts, aspects of the present invention can generate counterfactual images by inpainting only the masked regions, addressing ethical concerns of altering nonconsensual persons and ensuring equal representation of protected groups across attributes”; [0012] “Embodiments of the present invention provide a computer-implemented method for generating a synthetic training dataset with group-independent image attribute distributions, comprising using a text-guided inpainting model to inpaint a person mask in an original image of an original dataset with a synthetic person from a protected group to generate a synthetic image… maintaining consistent context of the original image…”; wherein the “removed portion visual content items” corresponds to the resulting masked images; [0046] “ω ∈ [0, 1]d is a person mask…inpaints ω in x with a synthetic person from protected group g described in t(g)”; [0054]) and providing the plurality of removed portion visual content items to a generative machine learning model ([0012] “using a text-guided inpainting model to inpaint a person mask in an original image”; [0046]), wherein the generative machine learning model is trained to generate at least a portion of visual content with respect to the plurality of removed portion visual content items ([0005] “While equal group distributions in real-world datasets are challenging to achieve, generative text-to-image models now enable targeted image modification”; [0012] “text-guided inpainting model”; [0046])
creating a training set including the synthesized second visual content samples ([0007]; [0012] “generating a synthetic training dataset with group-independent image attribute distributions”; [0046]-[0047])
training a machine learning model using the training set, wherein the machine learning model is trained to classify visual content with respect to the object (Figs. 4-7; [0008] “Unlike conventional approaches, training on the counterfactual data decorrelates both labeled and unlabeled attributes from protected groups without impacting model performance. Comprehensive evaluations show approaches according to aspects of the present invention significantly reduces prediction rules based on spurious correlations in multi-label classification and image captioning across various architectures (e.g., ResNet-50, Swin Transformer), datasets (COCO, Open-Images), and protected groups (gender, skin tone).”; [0053]; [0055])
With respect to claim 2, Hirota discloses:
The method of claim 1, wherein removing at least a portion of the plurality of first visual content samples further comprises:
segmenting the plurality of first visual content samples with respect to the object, wherein the at least a portion of the plurality of first visual content samples is removed based on the segmenting of the plurality of the first visual content samples with respect to the object ([0007]; [0012] mask generation; [0067])
With respect to claim 3, Hirota discloses:
The method of claim 2, wherein segmenting each of the plurality of first visual content samples results in at least one set of pixels corresponding to the object for each first visual content sample, wherein the at least one set of pixels corresponding to the object are removed from the at least one of the plurality of first visual content samples (Figs. 2A-2D; [0007]; [0012] mask generation; [0046] “where x ∈
PNG
media_image1.png
29
29
media_image1.png
Greyscale
d is an image, ω ∈ [0, 1]d is a person mask”)
With respect to claim 4, Hirota discloses:
The method of claim 1, further comprising:
querying the generative machine learning model with respect to the plurality of first visual content samples, wherein the query indicates a variation of the object for which the at least a portion of visual content is to be generated by the generative machine learning model ([0005] “While equal group distributions in real-world datasets are challenging to achieve, generative text-to-image models now enable targeted image modifications”; [0012] “text-guided inpainting model”; [0044] “Training datasets were created with group-independent image attribute distributions by using masked person images and text prompts with a diffusion model, as outlined in FIGS. 2A through 2E”; [0046])
With respect to claim 9, Hirota discloses:
The method of claim 1, wherein the generative machine learning model is a diffusion model ([0044] “Training datasets were created with group-independent image attribute distributions by using masked person images and text prompts with a diffusion model, as outlined in FIGS. 2A through 2E.”)
With respect to claim 10, Hirota discloses:
applying the trained machine learning model to a plurality of third visual content samples ([0055] “This results in 28,487/13,487 train/test samples”; wherein the 13,487 test samples correspond to the “third visual content samples”)
With respect to claim 11, Hirota discloses:
A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process ([0011]), the processing comprising: [the method of claim 1, rejected in line with the analysis above]
With respect to claim 12, Hirota discloses:
A system for augmented machine learning ([Abstract]), comprising:
a processing circuitry ([0032])
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to ([0034]): [perform the method of claim 1, rejected in line with the analysis above]
With respect to claims 13-15 and 20-21, they are functionally parallel to claims 2-4 and 9-10 respectively, with the exception that the steps are performed by the processing circuitry of claim 12. Accordingly, the claims are rejected in line with claims 12, 2-4, and 9-10.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Hirota in view of Kirichenko et. al Last Layer Re-Training Is Sufficient For Robustness to Spurious Correlations (Hereinafter, “Kirichenko”)
With respect to claim 5, Hirota teaches the method of claim 1.
Hirota does not explicitly teach the further limitations of claim 5.
However, Kirichenko, in the same field of endeavor of model debiasing, teaches:
wherein the trained machine learning model has a plurality of weights ([1] “While both the relevant and spurious features are learned, the spurious features can be highly weighted in the final classification layer of the model, leading to poor predictions on the minority groups…we simply retrain the last layer of a classification model trained with standard Empirical Risk Minimization (ERM)…”), further comprising
applying the trained machine learning model to at least one evenly distributed data set ([2] “In this paper, we study the effect of spurious correlations on the features learned by standard neural networks, and based on our findings propose a simple way of reducing the reliance on spurious features assuming access to a small set of data where the groups are equally represented”) in order to produce a set of outputs with respect to the at least one evenly distributed data set (Appendix, B “For the logistic regression model, we first extract the embeddings from the penultimate layer of the network, then use the logistic regression implementation (sklearn.linear_model.LogisticRegression) from the scikit-learn package (Pedregosa et al., 2011). We use ℓ1 regularization and tune the inverse regularization strength parameter C in the range {0.1,10,100,1000}. For more details on Deep Feature Reweighting implementation and tuning, see section C.”), each evenly distributed data set including an evenly distributed set of visual content samples (Appendix A “In DFR ,we subsample the reweighting dataset to be group balanced instead of using class- or group-balanced sampling. Specifically, we produce a dataset where the number of examples in each group is the same, and only use these datapoints. This detail is hugely important, as group-balanced sampling does not produce classifiers robust to spurious correlations (e.g. see RWG and SUBG methods in Idrissi et al., 2021).”; Appendix C.1, Table 10 “We train the logistic regression model on the validation data several times with random subsets of the data (we take all of the data from the smallest group, and subsample the other groups randomly to have the same number of datapoints) and average the weights of the learned models. Averaging more than 1 linear model leads to improved performance.”)
wherein the trained machine learning model outputs a classification for each visual sample among each evenly distributed data set (Fig. 1; [2] “For example, in the Waterbirds dataset (Sagawa et al., 2019), the task is to classify whether an image shows a landbird or a waterbird. The groups correspond to images of waterbirds on water background (G1), waterbirds on land background (G2), landbirds on water background (G3) and landbirds on land background (G4). See Figure 6 for a visual description of the Waterbirds data.”)
adjusting at least one weight of the plurality of plurality of weights based on the set of outputs with respect to the at least one evenly distributed data set (Fig. 1; [1] “With DFR, we simply reweight these features by retraining the last linear layer on a small dataset where the backgrounds are not spuriously correlated with the foreground. The resulting DFR model primarily relies on the foreground and performs much better on images with confusing backgrounds”; Appendix C.1, Table 10)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Hirota to include the limitations of weight adjustment amongst evenly distributed data sets, as taught by Kirichenko. Doing so would have the advantage of reducing bias, which is the ultimate goal of both methods. The systems readily integrate, as Hirota also features weighting ([0047]; [0054]; [0057]).
With respect to claim 16, it is functionally parallel to claim 5, with the exception that the steps are performed by the processing circuitry of claim 12. Accordingly, the claim is rejected in line with claims 12 and 5.
Claims 6, 8, 17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Hirota in view of Liusie et. al Mitigating Word Bias in Zero-shot Prompt-based Classifiers (Hereinafter, “Liusie”)
With respect to claim 6, Hirota teaches the method of claim 1.
Hirota does not explicitly teach the further limitations of claim 6.
However, Liusie, in the same field of endeavor of bias reduction, teaches:
wherein the generative machine learning model is a first generative machine learning model, wherein the trained machine learning model has a plurality of thresholds ([2]; [3.1] “The oracle upper-bound performance, found by optimising the optimal accuracy threshold via equation 4 (optimal)”; [3.2] “Figure 4 shows a scatter plot of the weights found by the optimal threshold search α∗ (equation 4)”), further comprising:
PNG
media_image3.png
377
490
media_image3.png
Greyscale
querying a second generative machine learning model for distribution data of a population set ([2] “Prompt-based classifiers”; [2] “Null-Input Approximation”, “Inspired by Zhao et al. (2021), we consider a resource-free approximation of the word prior (equation 8) by considering the output word probabilities of the null input ∅ (i.e. an empty string).”; equations 8 and 9)
adjusting at least one threshold of the plurality of thresholds based on the distribution data of the population set ([3.2] equations 8 and 9 output used in equation 3 and 4 for finding optimal threshold).
With respect to claim 8, Hirota and Liusie teach:
The method of claim 6, wherein the second generative machine learning model is a large language model ([2]; [2] “Null-Input Approximation”)
PNG
media_image4.png
181
525
media_image4.png
Greyscale
With respect to claims 17 and 19, they are functionally parallel to claims 6 and 8 respectively, with the exception that the steps are performed by the processing circuitry of claim 12. Accordingly, the claims are rejected in line with claims 12, 6, and 8.
Claims 7 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Hirota and Liusie in view of Yang et. al (US 20220207707 A1) (Hereinafter, “Yang”)
With respect to claim 7, Hirota and Liusie teach the method of claim 6.
Hirota and Liusie do not explicitly teach the further limitations of claim 7.
However, Yang, in the same field of endeavor of image model training, teaches:
wherein the at least one threshold is adjusted such that the trained machine learning model achieves a predetermined precision rate ([0035] “In one embodiment, the threshold is adjusted according to a preset recall rate or a preset accuracy rate. The preset accuracy rate is a proportion of the number of correctly detected defect sample images to the number of all detected defect sample images, and the preset recall rate is a proportion of the number of correctly detected defect sample images to the number of all real defect sample images”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Hirota and Liusie to include the limitations of threshold adjustment, as taught by Yang. Doing so would offer increased accuracy. The systems readily integrate, as Hirota and Liusie are already configured to contain adjustable thresholds.
With respect to claim 18, it is functionally parallel to claim 7, with the exception that the steps are performed by the processing circuitry of claim 12. Accordingly, the claim is rejected in line with claims 12 and 7.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH WILLIAM BOYAR whose telephone number is 571-272- 8392. The examiner can normally be reached 10:00 AM – 6:00 PM EST, Monday – Friday. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH W BOYAR/
Examiner, Art Unit 2669
/CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights