DETAILED ACTION
Contents
Notice of Pre-AIA or AIA Status 2
Claim Rejections - 35 USC § 103 2
Conclusion 9
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to applicant’s claim set received on 1/24/25. Claims 1-4 are currently pending.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimedinvention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al (IET: “Patch strategy for deep face recognition”) in view of Oh et al (NeurIPS: “Differentially Private CutMix for Split Learning with Vision Transformer”).
Regarding claim 1, Zhang teaches an information processing method comprising: a dividing step of dividing an image to be used for training of a learning model into a plurality of patches (see section 3.1; divide an input training image into different patches online), the learning model being composed of a plurality of first models (see section 3.2; In our network architectures, a complex structure is proposed
for the entire images that have rich information. For the cropped patches, it is appropriate to use a network with relatively fewer parameters to learn efficient patch features. This is due to the difference in size and the semantic meaning between a holistic image and its patches. Consequently, a structure similar to but simpler than CNN-1 is designed for the local patches with less information. To evaluate the contribution of each sampled patch to the face representation, we adopt the same network architectures for different patches, avoiding the influence from CNN structures. More specifically, the parameters of CNN-2 for various patches are not shared so that the features learned by each CNN-2 are adaptive to multimodal information included in different face regions) and a second model different from the first models (see section 3.2; outputs from the CNN are concatenated and processed by a downstream fusion layer); a first input step of inputting the patches to the first models without overlapping (see section 3.1; After cropping, the layer sends the patches to different network branches.). Zhang does not teach expressly an adding step of adding noise to each of a plurality of calculation results output from the first models; and a second input step of inputting, to the second model, the calculation results to each of which the noise has been added.
Oh, in the same field of endeavor, teaches an adding step of adding noise to each of a plurality of calculation results output from the first models (see section 1; To address the aforementioned issues, inspired from the patchfied smashed data in ViT [6] and the CutMix technique [16], we propose DP-CutMixSL 2, a differentially private (DP) SL framework with ViT via patch-level randomized CutMix. As Fig. 1 demonstrates, following the Gaussian DP mechanism [17, 18, 19], each device in DP-CutMixSL first injects random Gaussian noise into smashed data, followed by punching randomly selected patches, yielding Cutout smashed data as analogous to those of Cutout [20]. These Cutout smashed data are uploaded to and put together by the server, resulting in DP-CutMix smashed data that continue feed-forward propagation. Compared to SL with the Gaussian DP mechanism (DP-SL), we theoretically prove
that the proposed randomized CutMix operation in DP-CutMixSL amplifies the DP guarantee of
smashed data, by up to its upper-bound baseline DP-MixSL obtained by replacing CutMix with
Mixup [21] that simply superimposes the entire patches from each of different smashed data. By
experiment, we show that DP-CutMixSL achieves the highest accuracy, followed by DP-MixSL and
DP-SL. It is worth noting that while most of the existing works apply Cutout and CutMix at pixel
levels for intra-dataset interpolations [22, 23], we utilize them at patch levels for privacy-preserving
inter-dataset interpolations across different devices, i.e., privacy-preserving distributed ML); and a second input step of inputting, to the second model, the calculation results to each of which the noise has been added (see section 2; Algorithm 1 DP-CutMixSL requirements: w = [wc;i;ws]T (wc;i: lower model segment, ws: upper model segment) _: learning rate while w not converged do
/*Runs on mixer*/ samples {a1; ::; an} _ Dir(__) generates pseudo random sequences Mi for all i . Pseudorandom binary mask generation unicasts Mi to i-th client for all I /*Runs on client i 2 C*/
generates smashed data si by passing input data xi through wc;I produces _si by masking si via Mi . Cutout smashed data produces _s0 i by applying Gaussian mechanism . DP-Cutout smashed data
uploads _s0 i to the server /*Runs on server*/ produces ~s0 i via _s0
i aggregation for all i . DP-CutMix smashed data generates loss P
i Li by passing ~s0 i through ws in parallel updates ws via ws ws _ _ rws (
P i Li) . Upper model segment update unicasts i-th cut-layer gradient to i-th client for all i
/*Runs on client i 2 C*/ updates wc;i via wc;i wc;i _ _ rwc;i (P i Li) . Lower model segment update
end while (a) Raw images. (b) Smashed data. Figure 2: Examples of data obtained by performing various interpolation schemes on (a) raw image and (b) smashed data. we assume that a gaussian mechanism is applied to them, generating the following DP-Cutout smashed data and label containing white gaussian noise of Ns and Ny, respectively: _s0 i = _si + Ns = Mi _ si + Ns; (1) _y0i
= _yi + Ny: (2) The server aggregates DP-Cutout smashe data from all clients and generates DP-CutMix smashed data in the following way:
~s0
i;j = _s0
i + _s0
j ; ~y0
i;j = _i _ _y0i
+ _j _ _y0j
; for j 6= i: (3)
Next, the rest of DP-CutMixSL’s operation, equal to that of Vanilla SL, performing FP & BP on the
server-side model follows. The said operation of DP-CutMixSL is detailed by the pseudo code of
Algorithm 1. Fig. 2 also provides image samples of smashed data as well as input data to which the
proposed patch CutMix is applied compared to those of Mixup and Vanilla CutMix.
As a result, DP-CutMixSL can benefit both in terms of privacy leakage and communication cost, in a
way that only fraction of the smashed data is shared to the server, even ejected with gaussian noise.
Note that random sequences used for smashed data masking are mutually exclusive and collectively
exhaustive at the patch-level, so that there are no blank patches in DP-CutMix smashed data).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Zhang to utilize the cited limitations as suggested by Oh. The suggestion/motivation for doing so would have been to achieve higher accuracy in interpolation (see abstract). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Zhang, while the teaching of Oh continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Regarding claim 4, Zhang with Oh teaches all elements as mentioned above in claim 1. Zhang with Oh does not teach expressly Gaussian noise.
Oh, in the same field of endeavor, teaches Gaussian noise (see section 2-3).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Zhang with Oh to utilize the cited limitations as suggested by Oh. The suggestion/motivation for doing so would have been to achieve higher accuracy in interpolation (see abstract). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Zhang with Oh, while the teaching of Oh continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 2 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al (IET: “Patch strategy for deep face recognition”) with Oh et al (NeurIPS: “Differentially Private CutMix for Split Learning with Vision Transformer”), and further Noroozi et al (CV: “Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles”).
Regarding claim 2, Zhang with Oh teaches all elements as mentioned above in claim 1. Zhang with Oh does not teach expressly patches are input to the first models randomly without overlapping.
Noroozi, in the same field of endeavor, teaches patches are input to the first models randomly without overlapping (see section 3.2, 3.1).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Zhang with Oh to utilize the cited limitations as suggested by Noroozi. The suggestion/motivation for doing so would have been to outperform other learning visual representation systems (see abstract). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Zhang with Oh, while the teaching of Noroozi continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Claim 3 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al (IET: “Patch strategy for deep face recognition”), Oh et al (NeurIPS: “Differentially Private CutMix for Split Learning with Vision Transformer”) with Noroozi et al (CV: “Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles”), and further Ye et al (US 2024/0104392 A1).
Regarding claim 3, Zhang with Oh with Noroozi teaches all elements as mentioned above in claim 2. Zhang with Oh does not teach expressly each of which the noise has been added are integrated based on information indicating a correspondence between positions of the patches in the image and the first models to which the patches have been input, and then an integration result of the calculation results to each of which the noise has been added is input to the second model.
Ye, in the same field of endeavor, teaches each of which the noise has been added are integrated based on information indicating a correspondence between positions of the patches in the image and the first models to which the patches have been input, and then an integration result of the calculation results to each of which the noise has been added is input to the second model (see 0070-0071, 0081-0083).
It would have been obvious (before the effective filing date of the claimed invention) or (at the time the invention was made) to one of ordinary skill in the art to modify Zhang with Oh with Noroozi to utilize the cited limitations as suggested by Ye. The suggestion/motivation for doing so would have been to improve multi-task learning performance (see 0011). Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and/or programming techniques, without changing a “fundamental” operating principle of Zhang with Oh with Noroozi, while the teaching of Ye continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD PARK. The examiner’s contact information is as follows:
Telephone: (571)270-1576 | Fax: 571.270.2576 | Edward.Park@uspto.gov
For email communications, please notate MPEP 502.03, which outlines procedures pertaining to communications via the internet and authorization. A sample authorization form is cited within MPEP 502.03, section II.
The examiner can normally be reached on M-F 9-6 CST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M. Villecco, can be reached on (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EDWARD PARK/Primary Examiner, Art Unit 2661
Prosecution Insights