DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is in response to the instant application filled on 3/20/2024. Claims 1-20 are pending and have been examined.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 3/20/2024 was filed after the mailing is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1,
Step 1: The claim recites a method which falls into the statutory category of process.
Step 2A Prong 1: The claim recites multiple abstract ideas: selecting, by the processing hardware, batch samples from among the replay samples, the selecting based on selection frequencies of the respective replay samples, a mental process given a human being can mentally select batches, determining, by the processing hardware, a freeze layer group of the neural network model based on forward propagation of the neural network model using the batch samples, a mental process given a human being can mentally determine a freeze layer group, and training, by the processing hardware, the neural network model based on backward propagation of layers of the neural network model that are not in the freeze layer group, a mathematical calculation.
Step 2A Prong 2: Claim 1 does not integrate the abstract idea into a practical application since the additional elements of a computing device comprising storage hardware, is mere instructions to apply the abstract idea, and storing replay samples selected from online stream samples in a replay buffer comprised in the storage hardware, is insignificant extra-solution activity: data gathering.
Step 2B: Claim 1 does not integrate the abstract idea into a practical application since the additional elements of storing samples is considered well-understood, conventional, routine activity (MPEP 2106.05(d)(II)(IV)) and using a generic computing device is considered mere instructions to apply the abstract idea (MPEP 2106.05(f)). Claim 1 is not patent eligible.
Regarding claim 2, the rejection of claim 1 is incorporated, further the claim recites: wherein the selection frequencies correspond to how many times the respective replay samples were previously selected as batch samples, and wherein the higher the selection frequency of a replay sample the less likely the replay is to be selected by the selecting for inclusion in the batch. This limitation merely describes the selection process, which can still be done in the mind, and amounts to more specifics of the abstract idea of selecting samples. Claim 2 is not patent eligible.
Regarding claim 3, the rejection of claim 1 is incorporated, further the claim recites: further comprising: determining the selection frequencies of the replay samples based further on similarity scores of the respective replay samples. This limitation merely describes the selection process, which can still be done in the mind, and amounts to more specifics of the abstract idea of selecting samples. Claim 3 is not patent eligible.
Regarding claim 4, the rejection of claim 3 is incorporated, further the claim recites: wherein the selection frequency of a first replay sample among the replay samples comprises a direct component, which increases each time the first replay sample is selected to be used as one of the batch samples, and an indirect component, which increases each time another replay sample among the replay samples is selected to be used as one of the batch samples. This limitation merely describes the selection process, which can still be done in the mind, and amounts to more specifics of the abstract idea of selecting samples. Claim 4 is not patent eligible.
Regarding claim 5, the rejection of claim 4 is incorporated, further the claim recites: wherein the direct component increases in proportion to a number of times the first replay sample is selected as a batch sample. This limitation merely describes the selection process, which can still be done in the mind, and amounts to more specifics of the abstract idea of selecting samples. Claim 5 is not patent eligible.
Regarding claim 6, the rejection of claim 4 is incorporated, further the claim recites: wherein the indirect component increases in proportion to the number of times another replay sample is selected as a batch sample and a similarity score corresponding to similarity between the first replay sample and the other replay sample. This limitation merely describes the selection process, which can still be done in the mind, and amounts to more specifics of the abstract idea of selecting samples. Claim 6 is not patent eligible.
Regarding claim 7, the rejection of claim 3 is incorporated, further the claim recites: wherein each similarity score is determined based on corresponding output data of the neural network model. This limitation merely describes the selection process, which can still be done in the mind, and amounts to more specifics of the abstract idea of selecting samples. Claim 7 is not patent eligible.
Regarding claim 8, the rejection of claim 1 is incorporated, further the claim recites: wherein the determining of the freeze layer group comprises: estimating an operation amount and an information amount of layers of the neural network model; and determining the freeze layer group based on the operation amount and the information amount. This limitation merely describes the process of determining a freeze layer group, which can still be done in the mind, and amounts to more specifics of the abstract idea of determining a freeze layer group. Claim 8 is not patent eligible.
Regarding claim 9, the rejection of claim 8 is incorporated, further the claim recites: wherein the estimating of the operation amount and the information amount comprises: estimating the operation amount based on a partial operation amount for backward propagation of a first layer to an n-th layer of the neural network model; and estimating the information amount based on a partial information amount of an n+1-th layer to an L-th layer of the neural network model, wherein the "L" is a total number of the layers of the neural network model. This limitation merely describes the process estimating the operation and information amount when determining a freeze layer group, which can still be done in the mind, and amounts to more specifics of the abstract idea of determining a freeze layer group. Claim 9 is not patent eligible.
Regarding claim 10, the rejection of claim 9 is incorporated, further the claim recites: wherein the determining of the freeze layer group comprises: determining a value of "n" that maximizes the information amount relative to the operation amount. This limitation merely describes the process of determining a freeze layer group, which can still be done in the mind, and amounts to more specifics of the abstract idea of determining a freeze layer group. Claim 10 is not patent eligible.
Regarding claim 11, the rejection of claim 1 is incorporated, further the claim recites: wherein the online stream samples are used for online training of the neural network model. This limitation amounts to merely linking the abstract idea to a field of use: online training. Claim 11 is not patent eligible.
Regarding claims 12-18, the inventive concept is essentially the same as claims 1-11, as such Claims 12-18 are not patent eligible for the reasons listed above.
Regarding claim 19:
Step 1: The claim recites a method which falls into the statutory category of process.
Step 2A Prong 1: The claim recites multiple abstract ideas: selecting replay samples, from among the online training samples, to be reused for training of the neural network model and based on the usage statistics, selecting, from among the replay samples, batch samples to be used for training the neural network and updating the usage statistics of the selected replay samples based on the selection thereof as batch samples, a mental process given a human being can mentally select samples and update statistics in their mind.
Step 2A Prong 2: The claim does not integrate the abstract idea into a practical application since the additional elements of:
maintaining usage statistics of the respective replay samples, including updating the usage statistic of each respective replay sample each time the replay sample is selected for reuse in training the neural network model, is considered insignificant extra-solution activity: data gathering.
performing online training of a neural network with a stream of online training samples, merely links the abstract idea to a field of use: online training.
Step 2B: Claim 19 does not integrate the abstract idea into a practical application since the additional elements of performing online training generally links the abstract idea to a field of use (MPEP 2106.05(h)) and storing and updating data is considered well-understood, conventional, routine activity (MPEP 2106.05(d)(II)(IV)). Claim 19 is not patent eligible.
Regarding claim 20, the rejection of claim 19 is incorporated, further the claim recites: wherein the updating the usage statistics comprises updating counts of how many times the respective replay samples have been selected as batch samples, and wherein the higher a replay sample's count the less likely the replay sample is to be selected as a batch sample. This limitation amounts to more specifics of the abstract idea of updating usage statistics. Claim 20 is not patent eligible
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 2, 8-13, 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Pellegrini (Latent Replay for Real-Time Continual Learning) in view of Kauvar (Curious Replay for Model-based Adaptation).
Regarding claim 1, Pellegrini teaches a training method of training a neural network model performed by a computing device comprising storage hardware storing the neural network model and processing hardware (Abs, In our experiments we show that Latent Replay, combined with existing continual learning techniques, achieves state-of-the-art performance on complex video benchmarks such as CORe50 NICv2 (with nearly 400 small and highly non-i.i.d. batches) and OpenLORIS. Finally, we demonstrate the feasibility of nearly real-time continual learning on the edge through the deployment of the proposed technique on a smartphone device), the training method comprising: storing replay samples selected from online stream samples in a replay buffer comprised in the storage hardware (Section 3, In [10] it was shown that a very simple rehearsal implementation (hereafter denoted as native rehearsal), where for every training batch a random subset of the batch patterns is added to the external storage); determining, by the processing hardware, a freeze layer group of the neural network model based on forward propagation of the neural network model using the batch samples; and training, by the processing hardware, the neural network model based on backward propagation of layers of the neural network model that are not in the freeze layer group (Section 4, When latent replay is implemented with mini-batch SGD training: (i) in the forward step, a concatenation is performed at the replay layer (on the mini-batch dimension) to join patterns coming from the input layer with activations coming from the external storage; (ii) the backward step is stopped just before the replay layer for the replay patterns, Fig 3, in the limit case the layers are completely frozen; when training the model the backwards step of backward propagation does not involve the frozen group, it only propagates through layers up until the freeze layer group).
Pellegrini fails to teaches selecting, by the processing hardware, batch samples from among the replay samples, the selecting based on selection frequencies of the respective replay samples and merely selects all the samples in the buffer. Kauvar teaches selecting, by the processing hardware, batch samples from among the replay samples, the selecting based on selection frequencies of the respective replay samples (Section 3.3, Prioritization relies on tracking the visit count vi, the number of times an experience has been revisited…..The counter v ∈ R, for buffer capacity |R|, is used with hyperparameter β ∈ [0,1] to prioritize sampling as p_{i} = β_{v_i}, Alg 1, batches are sampled from the replay buffer with probability p_i which is based on the visit count v_i).
Pellegrini and Kauvar are analogous to the claimed invention because they are in the field of online learning using a replay buffer. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have used the method in Kauvar to sample batches from the replay buffer to “encourage an accurate, adaptive world model by prioritizing optimization on experiences that have been trained on the fewest times” (Kauvar Section 3.2).
Regarding claim 2, Pellegrini and Kauvar teach the method of claim 1, further Kauvar teaches wherein the selection frequencies correspond to how many times the respective replay samples were previously selected as batch samples, and wherein the higher the selection frequency of a replay sample the less likely the replay is to be selected by the selecting for inclusion in the batch (Section 3.3, Prioritization relies on tracking the visit count vi, the number of times an experience has been revisited…..The counter v ∈ R, for buffer capacity |R|, is used with hyperparameter β ∈ [0,1] to prioritize sampling as p_{i} = β_{v_i}, Alg 1, batches are sampled from the replay buffer with probability p_i and the higher the selection frequency (visit count) the lower p_i is and the less likely a sample is to be chosen).
Regarding claim 8, Pellegrini and Kauvar teaches the method of claim 1, further Pellegrini teaches wherein the determining of the freeze layer group comprises: estimating an operation amount and an information amount of layers of the neural network model; and determining the freeze layer group based on the operation amount and the information amount (Table 1, Computation, storage, and accuracy trade-off with Latent Replay at different layers, Section 5, (i) computation refers to the percentage cost in terms of ops of a partial forward (from the latent replay layer on) relative to a full forward step from the input layer (iii) accuracy and ∆ accuracy quantify the absolute accuracy at the end of the training, computation is the operation amount and accuracy the information amount).
Regarding claim 9, Pellegrini and Kauvar teaches the method of claim 8, further Pellegrini teaches wherein the estimating of the operation amount and the information amount comprises: estimating the operation amount based on a partial operation amount for backward propagation of a first layer to an n-th layer of the neural network model (Section 5, (i) computation refers to the percentage cost in terms of ops of a partial forward (from the latent replay layer on) relative to a full forward step from the input layer, cost in terms of ops of partial backward propagation from n+1 to L relative to the cost from 1 to L which effectively estimates the cost from 1 to n ); and estimating the information amount based on a partial information amount of an n+1-th layer to an L-th layer of the neural network model, wherein the "L" is a total number of the layers of the neural network model (Section 5, (iii) accuracy and ∆ accuracy quantify the absolute accuracy at the end of the training, accuracy at the end of the training including training through the n+1 to L layer).
Regarding claim 10, Pellegrini and Kauvar teaches the method of claim 9, further Pellegrini teaches wherein the determining of the freeze layer group comprises: determining a value of "n" that maximizes the information amount relative to the operation amount (Table 1, Computation, storage, and accuracy trade-off with Latent Replay at different layers, tradeoff between computation (operation amount) and accuracy (information amount)).
Regarding claim 11, Pellegrini and Kauvar teaches the method of claim 1, further Pellegrini teaches wherein the online stream samples are used for online training of the neural network model (Title, Latent Replay for Real-Time Continual Learning, real-time continual learning is training a model using an online/real time stream of samples).
Regarding claims 12, 13, and 18, the inventive concept is essentially the same as claims 1, 2, and 8 with the addition of an electronic device comprising one or more processors and a memory which taught by Pellegrini (Abs, Finally, we demonstrate the feasibility of nearly real-time continual learning on the edge through the deployment of the proposed technique on a smartphone device).
Regarding claims 19 and 20, the inventive concept is essentially the same as claims 1 and 2 but with usage statistics replacing selection frequencies which is still taught by Kauvar (Section 3.3, Prioritization relies on tracking the visit count vi, the number of times an experience has been revisited…..The counter v ∈ R, for buffer capacity |R|, is used with hyperparameter β ∈ [0,1] to prioritize sampling as p_{i} = β_{v_i}, Alg 1, batches are sampled from the replay buffer with probability p_i and the higher the selection frequency (visit count) the lower p_i is and the less likely a sample is to be chosen).
Claim(s) 3-7, 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Pellegrini and Kauvar as applied to claim 1 above, and further in view of Aljundi (Gradient based sample selection for online continual learning).
Regarding claim 3, Pellegrini and Kauvar teaches the method of claim 1 but fails to teach determining the selection frequencies of the replay samples based further on similarity scores of the respective replay samples. Aljundi teaches determining the selection frequencies of the replay samples based further on similarity scores of the respective replay samples (Section 3.4.2, The score is computed by the maximal cosine similarity of the current sample with a fixed number of other random samples in the buffer…. More formally, denote the score as Ci for sample i in the buffer. Sample i is selected as a candidate to be replaced with probability P(i) = Ci/ ∑Cj, the more similar a sample is to the other samples, the more likely it is to be removed from the buffer which affects whether it is selected).
Aljundi is analogous to the claimed invention because it is also in the field of online continual learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to have added the sample selection method from Aljundi along with the combination of Pellegrini and Kauvar because it helps with “maximizing the diversity of samples in the repay buffer” (Aljundi Abs).
Regarding claim 4, Pellegrini, Kauvar, and Aljundi teaches the method of claim 3, further Kauvar teaches wherein the selection frequency of a first replay sample among the replay samples comprises a direct component, which increases each time the first replay sample is selected to be used as one of the batch samples (Alg 1, the direct component (v_i) increases each time the sample is selected), and an indirect component, which increases each time another replay sample among the replay samples is selected to be used as one of the batch samples (Alg 1, the indirect component (p_i/∑p_j) increases each time another sample is chosen given the sum on the denominator decreases because p_j decreases for the sample chosen).
Regarding claim 5, Pellegrini, Kauvar, and Aljundi teaches the method of claim 3, further Kauvar teaches wherein the direct component increases in proportion to a number of times the first replay sample is selected as a batch sample (Alg 1, the direct component (v_i/visit count) increases each time the sample is selected).
Regarding claim 6, Pellegrini, Kauvar, and Aljundi teaches the method of claim 4, further Kauvar teaches wherein the indirect component increases in proportion to the number of times another replay sample is selected as a batch sample (Alg 1, the indirect component (p_i/∑p_j) increases each time another sample is chosen given the sum on the denominator decreases because p_j decreases for the sample chosen) and Aljundi teaches a similarity score corresponding to similarity between the first replay sample and the other replay sample (Section 3.4.2, The score is computed by the maximal cosine similarity of the current sample with a fixed number of other random samples in the buffer…. More formally, denote the score as Ci for sample i in the buffer. Sample i is selected as a candidate to be replaced with probability P(i) = Ci/ ∑Cj, the more similar a sample is to the other samples, the more likely it is to be removed from the buffer which affects whether it is selected). It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to combine the initial indirect component from Kauvar with the similarity sampling method in Aljundi to create a new indirect component to “maximize the diversity of samples in the repay buffer” (Aljundi Abs).
Regarding claim 7, Pellegrini, Kauvar, and Aljundi teaches the method of claim 3 further Aljundi teaches wherein each similarity score is determined based on corresponding output data of the neural network model (Alg 2, Algorithm 2 describes the main steps of our gradient based greedy sample selection procedure, line 6 uses gradients to determine similarity which implies output data of the neural network being used).
Regarding claims 14-17 the inventive concept is essentially the same as claims 3-7 with the addition of an electronic device comprising one or more processors and a memory which taught by Pellegrini (Abs, Finally, we demonstrate the feasibility of nearly real-time continual learning on the edge through the deployment of the proposed technique on a smartphone device).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NATNAEL A ASEGDEW whose telephone number is (571)270-0407. The examiner can normally be reached 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATNAEL A ASEGDEW/Examiner, Art Unit 2122
/MICHAEL H HOANG/ PRIMARY EXAMINER, Art Unit 2122
Prosecution Insights