DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
This final office action is in response to the amendment filed May 5, 2026. Claims 1-3, 5-9, and 11-12 are pending in this application and have been considered below. Claims 4 and 10 are canceled by the applicant.
Applicant’s arguments with respect to claims 1-3, 5-9, and 11-12 have been considered but are moot in view of new ground(s) of rejection, Van Deventer et al. (US 8,995,774 B) in view of McMahan et al. (US 11,475,350 B2) further in view of Karame et al. (US 2021/0051169 A1), because of the amendments. Karame, which describes a federated learning server that holds a test set, applies that test set to the updated model, compares the model's predictions to the expected labels, and rejects the update when the resulting distances exceed a threshold, updating the model only with accepted updates (Karame ¶¶ 36, 87-88), and then sends the new state of the model to the clients (Karame ¶ 34). Karame is available as prior art under 35 U.S.C. 102(a)(2) as of the August 15, 2019 filing date of its Provisional Application No. 62/886,991, which predates Applicant's August 30, 2019 foreign priority date, the relied-upon subject matter being supported in the provisional, which incorporates in its entirety the paper,
"Thwarting Model Poisoning in Federated Learning" (provisional ¶ 20) disclosing at Section 5.1 that the server holds a test set to analyze the updated model and rejects the update when its value falls outside the accepted range.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-3, 5-9, and 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Van Deventer et al. (US 8,995,774 B1 – hereinafter “Van Deventer”) in view of McMahan et al. (US 11,475,350 B2 – hereinafter “McMahan”) further in view of Karame et al. (US 2021/0051169 A1, hereinafter "Karame").
Claims 1 and 7.
Van Deventer discloses a user terminal for distributed learning of an ID card recognition model, the user terminal (Van Deventer C4:L25-30 discloses “When the system 200 is a server-based distributed application, it may include a central component residing on a server and one or more client applications residing on client devices and communicating with the central component via a network 110 (emphasis added).”) comprising:
a receiving unit configured to receive an ID card image (Van Deventer C4:L21-25 discloses “A system 200 for automated document recognition, identification, and data extraction may provide document recognition capabilities via an online portal, special-purpose application, cloud-based application, server-based distributed application, and so forth.”; C5:L10-12 discloses “The image 145 of the ID document 120 may be taken by a camera associated with the user 140.”; See also C7:L20-30 for the various types of documents received by the camera on the smart phone);
an outputting unit configured to output an ID card recognition result using the ID card recognition (Van Deventer teaches processing the image to extract data, identifying the document type, and structuring the data; See C6:L60-45, C7:L45-60, C8:L50-67) that (C5:L10-12 discloses “The image 145 of the ID document 120 may be taken by a camera associated with the user 140.”);
wherein the ID card recognition model is released to the user terminal so that the user terminal uses the ID card recognition model (Van Deventer teaches providing the recognition application to the user/client device to perform recognition; see C4:L20-30),
Van Deventer discloses all of the subject matter as described above except for specifically teaching the portions of the claim that have been stricken through. However, McMahan in the same field of endeavor teaches model … that performs at least one convolution operation (McMahan discloses CNNs; see C7:L55-60) … a training unit configured to train the ID card recognition model using the ID card recognition result and generate update information regarding the at least one convolution operation (McMahan teaches federated learning which includes training a neural network model on a client device to update a global model; see C3:L10-20 "client computing devices can be used as nodes for performing computation on their local data in order to update a global model"; see also C4:L40-50); and a transmitting unit configured to transmit the update information to a server updating the ID card recognition model (McMahan teaches sending the model update; see C3:L20-35 "only the less sensitive model update can be transmitted"; C5:L15-25 "provide the local update to the one or more server computing devices"), wherein the training unit is further configured to generate weight change information of a layer performing the convolution operation as the update information by comparing an output value of the ID card recognition model with a user response (McMahan teaches calculating a gradient/difference based on "actual user inputs" [user response] compared to the model parameters; see C1:L15-20; see also C4:L40-60 "determining the local update can include determining a difference between the locally-trained model and the machine-learned model ... difference between the global set of parameters ... and the updated local values"; C1:L15-20 "trained by datasets including actual user inputs"; C5:L5-15 "The local update can be determined by clipping each layer j") … wherein the ID card recognition model is re-released to the user terminal and updated using the transmitted update information periodically according to a regular release cycle of the release or re-released ID card recognition model or irregularly in response to an occurrence of a set event (McMahan teaches the server updates the global model and provides it back to clients [re-release] in an iterative process driven by device availability [events] or probability [periodic/regular]; see C21:L10-15 "providing ... the updated machine-learned model to one or more client computing devices"; McMahan teaches selecting devices for these updates based on probability parameters [regular/periodic cycle] or availability status such as being idle/plugged-in [set event]; see C4:L5-15 "available client computing devices can be client computing devices which are powered on, plugged into a power source, and/or in an idle mode" [set event]; C4:L20-25 "random-sized batch ... independently selected with a probability q" [regular/periodic cycle].).
Van Deventer and McMahan discloses all of the subject matter as described above except for specifically teaching the portions of the claim that have been stricken through. However, Karame in the same field of endeavor teaches wherein the receiving unit is further configured to receive, from the server, the ID card recognition model re-released after the server verifies validity of the update information by comparing the update information with a predetermined reference and updates the ID card recognition model using the verified update information (Karame ¶87, the server holds a test set used to analyze a new updated model; ¶36, applying test data and comparing the model's predictions to expected labels, flagging the model as poisoned where the distances exceed a threshold [comparing the update information with a predetermined reference]; ¶88, accepting an update whose value lies within [µ-zσ, µ+zσ] and otherwise rejecting it [updates the model using the verified update information]; ¶34, the server sends the new state of the model to the clients [re-released
to the terminal]).
It would have been obvious to one of ordinary skill in the art to modify the ID card recognition system of Van Deventer to incorporate the distributed learning, weight update generation, and re-release cycle taught by McMahan before the effective filing date of the claimed invention. The motivation for this combination of references would have improve the accuracy of the recognition model over time by learning from actual user corrections and inputs while simultaneously preserving user privacy and reducing network bandwidth. McMahan teaches that transmitting "model updates" (weight changes) instead of raw data (which Van Deventer handles) ensures that sensitive user information remains local on the device (see McMahan C3:L20-35 "user data that is privacy sensitive can remain at the user's computing device and need not be uploaded"). This is particularly advantageous for Van Deventer's ID card system, which handles personal identification documents. Furthermore, implementing the periodic or event-driven re-release cycle of McMahan would ensure the Van Deventer system remains up-to-date with the latest recognition capabilities without continuously draining device resources.
It would have been further obvious to incorporate the server-side update verification of Karame into the combined system. Van Deventer's model operates on personal identification documents, and the federated arrangement of McMahan has the user terminals submit model updates to the server without the server testing those updates. Karame teaches a federated learning server that screens a received update for poisoning by applying a predetermined test set to the updated model and rejecting the update when its metrics fall outside an accepted range (Karame ¶¶ 36, 87-88). One of ordinary skill would have applied this known server side verification technique to the combined ID card recognition system to prevent a compromised or malicious user terminal from corrupting the shared recognition model, producing the predictable result of a model updated only with verified update information.
Claims 2 and 8.
The combination of Van Deventer, McMahan, and Karame discloses the user terminal of claim 7, wherein the update information is generated using the user response for the ID card recognition result provided through the user terminal (McMahan teaches the model is trained on “datasets including actual user input on users’ mobile devices” rather than proxy data; See C1:Background, C7:§2.1. Van Deventer teaches presenting structured data and recognition results to the user on the mobile device for verification or filling forms; see C6:L5-20, C8:L15-25).
Claims 3 and 9.
The combination of Van Deventer, McMahan, and Karame discloses the user terminal of claim 7, wherein the ID card recognition model is updated by verifying the transmitted update information based on the layer or a version of the ID card recognition model (McMahan teaches “per-layer clipping” where the update “delta” for each specific layer j is evaluated and clipped independently to ensure the norm is bounded before updating the model; see C5:L5-15, C11:L5-15. McMahan further teaches an iterative process of updating relative to the “global set of parameters” [current version] and applied to generate an “updated machine-learning model” [new version]; see C4:L25-35 & 50-60).
Claims 5 and 11.
The combination of Van Deventer, McMahan, and Karame discloses the user terminal of claim 7, wherein the ID card recognition model generates one-time update information according to an input of a captured single ID card image (Van Deventer teaches “A still image of the improved representation of the ID document may be automatically extracted from the video stream”; see C3:L40-45, C5:L35-55. It would have been obvious to generate this as a “one-time” update for that specific interaction. McMahan teaches generating a “local update” which is then discarded or summarized into the global update; see C3:L10-20).
Claims 6 and 12.
The combination of Van Deventer, McMahan, and Karame discloses the user terminal of claim 8, wherein the training unit is further configured to perform a training process of training the ID card recognition model in a background environment according to an input of the ID card recognition result reflecting the user response, and wherein the transmitting unit is further configured to transmitting the update information generated by performing the training process to the server (McMahan teaches the system selects client devices that are “available,” which are defined as "powered on, plugged into a power source, and/or in an idle mode" thereby ensuring the training occurs in the background without disrupting the user; see C3:L10-20, C19:L55-65. See also C4:L35-60 "provide the local update to the one or more server computing devices … determining the local update can include determining a difference between the locally-trained model and the machine-learned model ... difference between the global set of parameters ... and the updated local values").
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Sheller et al. (US 2019/0042937 A1) discloses a federated neural-network training system in which a server-implemented trusted aggregator screens received model updates and excludes extreme or malicious updates, using Byzantine Gradient Descent, before updating the central model; and
Yu et al. (US 2020/0320349 A1) discloses transaction management of machine learning model updates in which an update is validated against a defined performance level before acceptance.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights