Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the amendments and remarks filed on 12/23/2025.
Claims 1-20 are pending.
Claims 1-3, 9, 12, 11-13, and 19 have been amended.
Response to Arguments
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 101 abstract idea, have been fully considered and are persuasive. Therefore, the rejections regarding the abstract idea set forth in the previous office action have been withdrawn.
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. 103, have been considered but they are not persuasive. Applicant argues that no reference teaches the amended claim limitation now reciting “automatically selecting one of the plurality of ML models to deploy at the new near-edge node by excluding ML models having a bootstrap error exceeding a threshold value and selecting, from the remaining ML models, the ML model having a smallest convergence value”, since “Kohavi does not address distributed or near-edge systems, does not distinguish datasets by node origin, and does not disclose using bootstrap-based metrics to select or deploy one model among multiple trained models…Kohavi evaluates a given model but does not compare or select among multiple trained models. Kohavi's bootstrap analysis assumes a fixed model, not a plurality of candidate models subject to deployment selection”. The examiner respectfully disagrees.
Kohavi, section 4-5.2 and Fig. 4 teach determining “bootstrap estimates” for multiple machine learning models ordering the models according to bootstrapping variance results (discloses using bootstrap-based metrics to select or deploy one model among multiple trained models/compare as argued) with the lowest being the best model excluding ML models having a bootstrap error exceeding a threshold value), and determining the best performing model (selecting, from the remaining ML models, the ML model having a smallest convergence value). Here, the applicant argues are narrower scope than what is claimed regarding “does not distinguish datasets by node origin” since it is not recited in the limitation. Further, Kohavi teachings of model bootstrap comparison is taught in combination with Zhang that teaches the central and edge server model deployment.
See 35 U.S.C 103 section for full mapping of claim limitations necessitated by applicant amendments.
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. 103, have been considered but they are not persuasive. Applicant argues that no reference teaches the amended claim limitation now reciting “determining a first test error for each of a plurality of machine-learning (ML) models maintained at a central node when the ML models are trained using a first dataset, the first dataset formed by joining of a plurality of datasets collected from a plurality of near-edge nodes associated with different operational environments”, since “Zhang aggregates model parameters, not datasets. Ho does not aggregate datasets at all. Kohavi resamples a single dataset, it does not join datasets from different operational environments or nodes. Kohavi's bootstrap samples are statistical resamplings of one dataset, not datasets collected from multiple near-edge nodes and joined together”. The examiner respectfully disagrees.
Zhang, sections 3-4 teach global models are trained on a central server, wherein “the central server-side agent data can be effectively updated to match the data of the edge equipment side” (first dataset formed by joining of a plurality of datasets) of selected IIoT equipment (edge nodes) through different “wireless network access point[s]” (near-edge nodes associated with different operational environments) and optimizes a “global loss function” of the trainings based on the uploaded, trained models (first test error for each of a plurality of ML models). Here, the applicant is arguing a narrower scope than what is claimed, since parameters are still data that is transferred and the applicant is encouraged to amend to distinguish the argued language from the prior art.
See 35 U.S.C 103 section for full mapping of claim limitations necessitated by applicant amendments.
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. 103, have been considered but they are not persuasive. Applicant argues that no reference teaches the amended claim limitation now reciting “determining a second test error for each of the plurality of ML models when the plurality of ML models are trained using a second dataset, the second dataset collected exclusively from a new near-edge node that is not part of the plurality of near-edge nodes that the first dataset was collected from”, since “Zhang does not distinguish nodes as ‘new’ versus existing. Ho assumes learning within an environment, not onboarding a new one. Kohavi does not consider dataset provenance across nodes at all. Kohavi's bootstrap evaluation does not involve a new data source, much less a new near-edge node distinct from those contributing to a joined dataset”. The examiner respectfully disagrees due to the broadness of the claim language.
Zhang, sections 3-4 teach selecting new IIoT equipment data through a “wireless network access point” (second dataset collected exclusively from a new near-edge node that is not part of the plurality of near-edge nodes that the first dataset was collected from) to train the global models over multiple iterations “until the loss function converges or reaches the maximum number of iterations”. Further, section 4 teach this process is repeated for “newly selected IIoT equipment selection”; therefore Zhang is maintained as reading on the claimed language.
See 35 U.S.C 103 section for full mapping of claim limitations necessitated by applicant amendments.
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. 103, have been considered but they are not persuasive. Applicant argues that no reference teaches the amended claim limitation now reciting “determining a bootstrap error for each of the plurality of ML models by calculating a difference between the second test error and the first test error”, since “Zhang and Ho teach performance feedback, but do not teach a bootstrap error are now recited in the amended claims. Kohavi does teach bootstrap error estimation, but this bootstrap error estimation estimates variance or confidence intervals for a single dataset/model…[and] does not compare errors across different datasets, evaluate generalization from aggregated environments to a new environment, or define bootstrap error as a cross-dataset discrepancy”. The examiner respectfully disagrees due to the broadness of the claim language.
Zhang, sections 3-4 and Fig. 2 teach determining the loss values from training iterations on different IIoT equipment and “Repeat the above process until the loss function converges or reaches the maximum number of iterations” to find a minimum loss value of the models (difference). Section 4 further teaches randomly sampling the equipment subset data for testing the global model and generating the standard loss function which is a form of bootstrapping.
See 35 U.S.C 103 section for full mapping of claim limitations necessitated by applicant amendments.
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1 and 11 under 35 U.S.C. 103, have been considered but they are not persuasive. Applicant argues that no reference teaches the amended claim limitation now reciting “determining a convergence value for each of the plurality of ML models as an epoch value derived from a training loss curve when the ML models are trained using the first dataset”, since “Zhang teaches convergence used for training stability. Ho teaches convergence of RL policy during optimization. Kohavi does not address convergence or training loss curves at all”. The examiner respectfully disagrees due to the broadness of the claim language.
Zhang, sections 3-4 and Algorithm 1 teach determining the loss values from training iterations on IIoT equipment for each local “epoch” as taught in preceding limitations and “Repeat the above process until the loss function converges or reaches the maximum number of iterations”, thus maintained as reading on the broadness of the claimed language.
See 35 U.S.C 103 section for full mapping of claim limitations necessitated by applicant amendments.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-8, 11, and 13-18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al (“Deep Reinforcement Learning Assisted Federated Learning Algorithm for Data Management of IIoT”, 2020) hereinafter Zhang, in view of Kohavi (“A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection”, 1995), in view of Ho et al (“Federated Deep Reinforcement Learning for Task Scheduling in Heterogeneous Autonomous Robotic System”, 2022) hereinafter Ho.
Regarding claims 1 and 11, Zhang teaches a method; non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising (section 4 teaches using a “CPU” to perform the embodiments of the disclosure, which is well known to be included in a computer system communicatively coupled to one or more memories):
determining a first test error for each of a plurality of machine-learning (ML) models maintained at a central node when the ML models are trained using a first dataset, the first dataset formed by joining of a plurality of datasets collected from a plurality of near-edge nodes associated with different operational environments, the plurality of ML models being configured to control the operation of one or more edge-nodes that are associated with each of the plurality of near-edge nodes (sections 3-4 teach global models are trained on a central server, wherein “the central server-side agent data can be effectively updated to match the data of the edge equipment side” (first dataset formed by joining of a plurality of datasets) of selected IIoT equipment (edge nodes) through different “wireless network access point[s]” (near-edge nodes associated with different operational environments) and optimizes a “global loss function” of the trainings based on the uploaded, trained models (first test error for each of a plurality of ML models));
determining a second test error for each of the plurality of ML models when the plurality of ML models are trained using a second dataset, the second dataset collected exclusively from a new near-edge node that is not part of the plurality of near-edge nodes that the first dataset was collected from (sections 3-4 teach selecting new IIoT equipment data through a “wireless network access point” (second dataset collected exclusively from a new near-edge node that is not part of the plurality of near-edge nodes that the first dataset was collected from) to train the global models over multiple iterations “until the loss function converges or reaches the maximum number of iterations”);
determining a bootstrap error for each of the plurality of ML models by calculating a difference between the second test error and the first test error (sections 3-4 and Fig. 2 teach determining the loss values from training iterations on different IIoT equipment and “Repeat the above process until the loss function converges or reaches the maximum number of iterations” to find a minimum loss value of the models (difference));
determining a convergence value for each of the plurality of ML models as an epoch value derived from a training loss curve when the ML models are trained using the first dataset (sections 3-4 and Algorithm 1 teach determining the loss values from training iterations on IIoT equipment for each local “epoch” as taught above and “Repeat the above process until the loss function converges or reaches the maximum number of iterations”);
.
However, Zhang does not explicitly teach automatically selecting one of the plurality of ML models to deploy at the new near-edge node by excluding ML models having a bootstrap error exceeding a threshold value and selecting, from the remaining ML models, the ML model having a smallest convergence value.
Kohavi teaches automatically selecting one of the plurality of ML models to deploy at the new near-edge node by excluding ML models having a bootstrap error exceeding a threshold value and selecting, from the remaining ML models, the ML model having a smallest convergence value (section 4-5.2 and Fig. 4 teach determining “bootstrap estimates” for multiple machine learning models ordering the models according to bootstrapping variance results with the lowest being the best model excluding ML models having a bootstrap error exceeding a threshold value), and determining the best performing model (selecting, from the remaining ML models, the ML model having a smallest convergence value)).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to implement machine learning model bootstrapping estimate evaluations for determining the best performing model for selection as taught by Kohavi into Zhang‘s teaching of central server global models training for deployment in IIoT equipment in order to accurately select the best model based on evaluation metrics (Kohavi, sections 4-6).
Further, Zhang at least implies determining a first test error for each of a plurality of machine-learning (ML) models maintained at a central node when the ML models are trained using a first dataset, the first dataset formed by joining of a plurality of datasets collected from a plurality of near-edge nodes associated with different operational environments, the plurality of ML models being configured to control the operation of one or more edge-nodes that are associated with each of the plurality of near-edge nodes (see mappings above); however, Ho teaches determining a first test error for each of a plurality of machine-learning (ML) models maintained at a central node when the ML models are trained using a first dataset, the first dataset formed by joining of a plurality of datasets collected from a plurality of near-edge nodes associated with different operational environments, the plurality of ML models being configured to control the operation of one or more edge-nodes that are associated with each of the plurality of near-edge nodes (section 3, Algorithm 2, and Fig. 1 teach a “Central server selects a subset of M workstations (near-edge nodes) at random” to send model parameters for global updating (maintained at a central server), wherein the workstations further collect information from the warehouse “robot[s]” (edge nodes), and locally train the received global policy network parameters on the obtained local information (first dataset formed by joining of a plurality of datasets collected from a plurality of near-edge nodes associated with different operational environments), and computing an “error” of the policy networks when training and a “convergence” measure. Then continuing the process to other workstations.).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Zhang‘s teaching of central server global models training for deployment in IIoT equipment as modified by machine learning model bootstrapping estimate evaluations for determining the best performing model for selection as taught by Kohavi, to include federated learning hierarchical learning structure for warehouse applications as taught by Ho in order to “obtain an optimal policy for the formulated MDP” for a specific warehouse application (Ho, section 3).
Regarding claims 2 and 12, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 1 and 11 above; and further teaches wherein the threshold value is configurable based on a desired generalization tolerance for the new near-edge node (Ho, section 3 teaches each workstation model update being maximized by being compared to a determined value (threshold value is configurable)).
Zhang, Kohavi, and Ho are combinable for the same rationale as set forth above with respect to claims 1 and 11.
Regarding claims 3 and 13, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 2 and 12 above; and further teach wherein the training loss curve corresponds to a loss computed during iterative training of each of the plurality of ML model using the first dataset (Zhang, sections 3-4, Algorithm 1, and Fig. 2 teach determining the loss values from training iterations on different IIoT equipment and “Repeat the above process until the loss function converges or reaches the maximum number of iterations” to find a minimum loss value of the models (difference))).
Regarding claims 4 and 14, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 1 and 11 above; and further teach wherein the plurality of near-edge nodes are a warehouse (Ho, sections 2-3 and Fig. 1 teach “Central server selects a subset of M workstations (near-edge nodes) at random”, wherein the workstations collect information from the warehouse “robot[s]” (edge nodes); and “We assume that each warehouse is operated by a WMS implemented in a workstation”).
Zhang, Kohavi, and Ho are combinable for the same rationale as set forth above with respect to claims 1 and 11.
Regarding claims 5 and 15, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 4 and 14 above; and further teach wherein the plurality of near-edge nodes receive the plurality of datasets comprising the first dataset from the one or more edge-nodes that operate in the warehouse (Ho, section 3 and Fig. 1 teach “Central server selects a subset of M workstations (near-edge nodes) at random”, wherein the workstations collect information from the warehouse “robot[s]” (edge nodes)”).
Zhang, Kohavi, and Ho are combinable for the same rationale as set forth above with respect to claims 1 and 11.
Regarding claims 6 and 16, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 5 and 15 above; and further teach wherein the plurality of edge-nodes comprise one of a forklift or an Autonomous Mobile Robot (AMR) that operate in the warehouse (Ho, section 2 and Fig. 1 teach “Each warehouse is with a set Km of Km heterogeneous autonomous robots (HAR)…robots are heterogeneous with different mobility capabilities, i.e.,…autonomous mobile robots (AMRs)”).
Zhang, Kohavi, and Ho are combinable for the same rationale as set forth above with respect to claims 1 and 11.
Regarding claims 7 and 17, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 6 and 16 above; and further teach wherein the plurality of datasets comprising the first dataset comprise sensor data or event data of the forklifts or AMR (Zhang, sections 3-4 teach global models are trained on a central server, wherein “the central server-side agent data can be effectively updated to match the data of the edge equipment side” (first dataset comprising a joining of a plurality of datasets) of selected IIoT equipment (edge nodes) through “wireless network access point[s]” (near-edge nodes) and optimizes a “global loss function” of the trainings based on the uploaded, trained models (first test error for each of a plurality of ML models)).
Regarding claims 8 and 18, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 1 and 11 above; and further teach wherein: the new near-edge node is a warehouse, the new near-edge node receives the second dataset from one or more edge-nodes that operate in the warehouse, and the one or more edge-nodes comprise one of a forklift or an Autonomous Mobile Robot (Ho, section 3 and Fig. 1 teach a “Central server selects a subset of M workstations (near-edge nodes) at random”, wherein the workstations collect information from the warehouse “robot[s]” (edge nodes), and trains policy network parameters on the obtained local information (first dataset comprising a joining of a plurality of datasets obtained from a plurality of near-edge nodes), and computing an “error” of the policy networks when training and a “convergence” measure. Then continuing the process to other workstations (new near-edge node is a warehouse). Section 2 and Fig. 1 teach “Each warehouse is with a set Km of Km heterogeneous autonomous robots (HAR)…robots are heterogeneous with different mobility capabilities, i.e.,…autonomous mobile robots (AMRs)”.).
Zhang, Kohavi, and Ho are combinable for the same rationale as set forth above with respect to claims 1 and 11.
Regarding claims 9 and 19, the combination of Zhang, Kohavi, and Ho teach all the claim limitations of claims 8 and 18 above; however, the combination does not explicitly teach wherein determining a convergence value for each of the plurality of ML models when the ML models are trained using the first dataset comprises: evaluating a training loss curve for each of the plurality of ML models; and determining the epoch value based on the training loss curve.
Kohavi teaches wherein determining a convergence value for each of the plurality of ML models when the ML models are trained using the first dataset comprises: evaluating a training loss curve for each of the plurality of ML models; and determining the epoch value based on the training loss curve (section 4-5.2 and Fig. 4 teach “We chose six datasets from a wide variety of domains, such that the learning curve for both algorithms did not flatten out too early, that is, before one hundred instances (convergence)”; and the training and testing “was repeated 50 times at points where the learning curve was sloping up (determining the epoch value based on the training loss curve)”).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Zhang‘s teaching of central server global models training for deployment in IIoT equipment as modified by Ho’s teachings of federated learning hierarchical learning structure for warehouse applications, to include machine learning model bootstrapping estimate evaluations for determining the best performing model for selection as taught by Kohavi in order to accurately select the best model based on evaluation metrics (Kohavi, sections 4-6).
Regarding claims 10 and 20, the combination of Zhang, Ho, and Kohavi teach all the claim limitations of claims 9 and 19 above; and further teach wherein the selected ML model that is deployed at the new near-edge node is used to control an operation of one or more edge-nodes associated with the new near-edge node (Ho, section 3 and Fig. 1 teach creating policies for implementing on a robot’s policy network for performing warehouse tasks).
Zhang, Kohavi, and Ho are combinable for the same rationale as set forth above with respect to claims 1 and 11.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CLINT MULLINAX whose telephone number is 571-272-3241. The examiner can normally be reached on Mon - Fri 8:00-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.M./Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123