DETAILED ACTION
Claims 1, 3-13, and 15-18 are presented for examination.
This office action is in response to submission of application on 02-JANUARY-2026.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 28-FEBRUARY-2022 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Amendment
The amendment filed 02-JANUARY-2026 in response to the previous office action mailed 17-OCTOBER-2025 has been entered. Claims 1, 3-13, and 15-18 remain pending in the application.
With regards to the non-final office action’s rejections under 103, the amendments to the claims necessitated a new consideration of the art. After this consideration, the examiner respectfully disagrees with the applicant’s arguments that the art referenced in the previous office action does not teach the amendment claim limitations. A new 103 rejection over the prior art has been provided.
Rolfe in view of Maragakis discloses the amended limitation concatenating virtual tokens with a projected input set within the set neural network
Rolfe teaches an approximating posterior distribution which is considered analogous to virtual tokens as it approximates inputs through encoding (Rolfe, Paragraph 14). However, Rolfe is combined with Maragakis to explicitly address the use of tokens (Maragakis, Paragraph 119). This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Rolfe teaches finding a KL-divergence between the true prior distribution and the approximating posterior distribution, wherein determining the KL-divergence would include determining a gradient of each distribution (Rolfe, Paragraph 21). That both the approximating posterior distribution and the true prior distribution are over the discrete space indicates that they are concatenated together for the purposes of finding the KL-divergence.
Furthermore, the neural network of Rolfe (Rolfe, Paragraph 15) is a set neural network as it takes in a set of variables (Rolfe, Paragraph 14) which can be further delineated into subsets.
Rolfe discloses the limitation thereby improving a convergence speed and a generalization capability of the set neural network:
Rolfe teaches that its methodology improves generalization accuracy and reduces the training time i.e. through improving a convergence speed (Rolfe, Paragraph 65).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 8, 10-13, and 15-18 rejected under 35 U.S.C. 103 as being unpatentable over Rolfe (Pub. No. US 20180247200 A1. Filed August 18th 2016, hereinafter Rolfe) in view of Maragakis et al. (Pub. No. US 20220230713 A1. Filed May 29th 2020, hereinafter Maragakis).
Regarding claim 1:
Claim 1 recites:
A method for minimizing information loss in a set neural network, the method comprising: concatenating virtual tokens with a projected input set within the set neural network; determining an information loss term for the set neural network that internally uses the virtual tokens such that the information loss term minimizes a divergence between a distribution containing the virtual tokens and a distribution containing input tokens; and training the set neural network with training data from a data source that is expressed as sets using the information loss term, thereby improving a convergence speed and a generalization capability of the set neural network.
Rolfe discloses concatenating virtual tokens with a projected input set within the set neural network; determining an information loss term for a set neural network that internally uses virtual tokens such that the information loss term minimizes a divergence between a distribution containing the virtual tokens and a distribution containing input tokens
Rolfe teaches a minimization of a KL-divergence from a data distribution to that of the model, wherein the minimization is in order to maximize a log-likelihood of a dataset under a model (Paragraph 82). This would be minimization of a divergence between two distributions. Furthermore, the log-likelihood would be analogous to the information loss term for the model as it describes the fit of the model to the information, which would describe how much of the information to model has lost.
Furthermore, Rolfe teaches that these distributions may be a distribution containing the virtual tokens and a distribution containing input tokens, as it teaches that a KL-divergence with respect to an approximating posterior distribution and a true prior distribution may be found (Paragraph 21). Therefore the minimization of a divergence between distribution may apply to these two distributions as well, wherein the approximating posterior distribution would be analogous to virtual tokens as from the input an approximation is created for further processing in a neural network that is not solely reliant on input data, wherein the true prior distribution would be the input tokens.
Furthermore, the neural network of Rolfe is a set neural network as it takes in a set of variables (Paragraph 14) which can be further delineated into subsets.
Finally, Rolfe teaches finding a KL-divergence between the true prior distribution and the approximating posterior distribution, wherein determining the KL-divergence would include determining a gradient of each distribution (Paragraph 21). That both the approximating posterior distribution and the true prior distribution are over the discrete space indicates that they are concatenated together for the purposes of finding the KL-divergence.
However, Rolfe does not solely teach the use of virtual tokens. This aspect of the limitation is taught by Maragakis further below.
Rolfe discloses training the set neural network with training data from a data source that is expressed as sets using the information loss term, thereby improving a convergence speed and a generalization capability of the set neural network:
Rolfe teaches machine learning methods which may be a neural network (Paragraph 15) that are trained on training sets (Paragraph 2) which would be a data source expressed as a set.
Furthermore, Rolfe teaches that its methodology improves generalization accuracy and reduces the training time i.e. through improving a convergence speed (Paragraph 65).
Furthermore, Maragakis discloses the use of virtual tokens:
Maragakis in the same field of endeavor of reinforcement learning teaches tokenization of a string for use in a neural network (Paragraph 119). This teaches the use of tokens as the tokenization of Maragakis is used to encode inputs, as the tokenization of further inputs can be seen as encoding by breaking apart a larger whole into smaller discrete parts.
Furthermore, Rolfe teaches forming an encoding distribution comprising an approximating distribution conditioned on the input space (Paragraph 14) – or essentially, a space that is meant to approximate through encoding the input space. This describes virtual tokens as from the input an approximation is created for further processing in a neural network that is not solely reliant on input data.
Rolfe, Maragakis, and the present application are analogous art because they are all in the same field of endeavor of reinforcement learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Regarding claim 3, which depends upon claim 1:
Claim 3 recites:
The method of claim 2, wherein the divergence is computed using a Kullback-Leibler divergence, Wasserstein distance, or Jensen-Shannon divergence.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 3 depends. Furthermore, Rolfe discloses the limitation of claim 3:
Rolfe teaches that the divergence is a KL-divergence, which is used to refer to a Kullback-Leibler divergence and hence the divergence would be computed through its computation (Paragraph 28).
Regarding claim 4, which depends upon claim 1:
Claim 4 recites:
The method of claim 1, further comprising testing the trained set neural network.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 4 depends. Furthermore, Maragakis discloses the limitation of claim 4:
Maragakis teaches the use of a testing set to evaluate the machine learning model (Paragraph 112) which may be a neural network (Paragraph 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Regarding claim 5, which depends upon claim 1:
Claim 5 recites:
The method of claim 1, further comprising using the trained set neural network to produce a compressed representation of input data in a machine learning task.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 5 depends. Furthermore, Maragakis discloses the limitation of claim 5:
Maragakis teaches that a representation of data is transformed into a compressed representation as part of its machine learning process (Paragraph 61). This would be using the trained set neural network to produce a compressed representation of input data in a machine learning task.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Regarding claim 8, which depends upon claim 1:
Claim 8 recites:
The method of claim 1, wherein the method further comprises: depicting a protein molecule as a set of 3D points as the data source that can be expressed as sets; and using the trained set neural network to predict a protein binding candidate from a protein representation dataset.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 8 depends. Furthermore, Maragakis discloses the limitation of claim 8:
Maragakis teaches using a machine learning model is order to discover molecules that will bind to a drug target through using a 3D-model of a drug target, where possible pairs are included in the training set (Paragraph 117). The drug target would here be analogous to a protein molecule, and is depicted as a set of 3D points as the data source that can be expressed in sets through the 3D model, wherein the model predicts a binding candidate from a training set that would act as the database.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Regarding claim 10, which depends upon claim 1:
Claim 10 recites:
The method of claim 1, wherein the set neural network internally uses mean and variance of the virtual tokens during the training.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 10 depends. Furthermore, Rolfe discloses the limitation of claim 10:
Rolfe teaches the use of mean and variance of a distribution, which would include the virtual tokens, for each layer (Paragraph 187).
Regarding claim 11, which depends upon claim 1:
Claim 11 recites:
The method of claim 1, further comprising encoding a compressed representation of data of the data source that is expressed as sets using the trained set neural network
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 11 depends. Furthermore, Maragakis discloses the limitation of claim 11:
Maragakis teaches that the compressed representation is implemented through byte-pair encoding (Paragraph 62). This would be analogous to encoding a compressed representation of the data.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Regarding claim 12, which depends upon claim 1:
Claim 12 recites:
The method of claim 1, wherein the divergence approximates an input token space.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 12 depends. Furthermore, Rolfe discloses the limitation of claim 12:
Rolfe teaches a divergence (Paragraph 28). Furthermore, Maragakis teaches the use of tokens in machine learning (Maragakis, Paragraph 119), which could be used as the distributions.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Claim 13 recites a system that parallels the method of claim 1. Therefore, the analysis discussed above with respect to claim 1 also applies to claim 13. Accordingly, claim 13 is rejected based on substantially the same rationale as set forth above with respect to claim 1.
Claim 15 recites a non-transitory computer readable storage medium that parallels the method of claim 1. Therefore, the analysis discussed above with respect to claim 1 also applies to claim 15. Accordingly, claim 15 is rejected based on substantially the same rationale as set forth above with respect to claim 1.
Regarding claim 16, which depends upon claim 1:
Claim 16 recites:
The method of claim 1, wherein the input tokens represent a true distribution of an input set for the set neural network and the virtual tokens represent an approximation distribution of the input set for the set neural network based on concatenating the input set
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 16 depends. Furthermore, Rolfe discloses the limitation of claim 16:
Rolfe teaches an input space that would represent a true distribution of the input set (Paragraph 14) as it comprises a subset of the training dataset of samples of its respective variables.
Furthermore, Rolfe teaches an encoding distribution that is analogous to virtual tokens for the manner in which they assist in the processing of data through encoding without requiring further input data that comprises an approximating posterior distribution based on the input space (Paragraph 14) wherein a cumulative distribution function comprising functions of a full distribution of at least one or more discrete random variables of the first latent space, which would include the encoding distribution (Paragraph 14). Therefore, the cumulative distribution function would include the virtual tokens, wherein the accumulation is analogous to concatenation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Regarding claim 17, which depends upon claim 1:
Claim 17 recites:
The method of claim 1, wherein the set neural network comprises a transformer model.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 17 depends. Furthermore, Maragakis discloses the limitation of claim 17:
Maragakis teaches that the neural network may be a transformer network (Paragraph 60).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Regarding claim 18, which depends upon claim 1:
Claim 18 recites:
The method of claim 1, wherein the input tokens are generated based on a received input set, and wherein the virtual tokens are generated by the set neural network based on a projection of the input set.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 18 depends. Furthermore, Rolfe discloses the limitation of claim 18:
Rolfe teaches an input space that would represent a true distribution of the input set (Paragraph 14) as it comprises a subset of the training dataset of samples of its respective variables, wherein this would be based on the received input set.
Furthermore, Rolfe teaches an encoding distribution that is analogous to virtual tokens for the manner in which they assist in the processing of data through encoding without requiring further input data that comprises an approximating posterior distribution based on the input space (Paragraph 14), wherein a posterior distribution is based on a projection of the input set.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe and the teachings of Maragakis. This would have provided the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38).
Claims 6-7 rejected under 35 U.S.C. 103 as being unpatentable over Rolfe in view of Maragakis further in view of Rhee et al. (Pub. No. US 20220027668 A1, filed June 7th 2021, hereinafter Rhee).
Regarding claim 6, which depends upon claim 1:
Claim 6 recites:
The method of claim 1, wherein the method further comprises: obtaining minutiae of fingerprints as the training data from the data source that is expressed as sets; and encoding in a compressed representation the minutiae of fingerprints.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 6 depends. Furthermore, Rhee discloses the limitation of claim 6:
Rhee in the same field of endeavor of reinforcement learning teaches a computing apparatus that may perform a target task, wherein the target task may be an inference task that is fingerprint recognition of a user of a smartphone (Paragraph 117), which would be obtaining minutiae of the fingerprints.
Furthermore, Rolfe in view of Maragakis have previous taught training data from the data source that is expressed as sets.
Rolfe, Maragakis, Rhee, and the present application are analogous art because they are all in the same field of endeavor of reinforcement learning.
Furthermore, Rhee discloses encoding in a compressed representation the minutiae of fingerprints:
Rhee teaches that a compressed representation can be generated to lead to an inference result (Paragraph 5), which would make the compressed representation’s generation part of the inference task. As the inference task may be fingerprint recognition, this would be encoding in a compressed representation the minutiae of fingerprints.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe in view of Maragakis and the teachings of Rhee. This would have provided the advantage of improving the performance of a neural network through the use of a compressed representation (Rhee, Paragraph 57).
Regarding claim 7, which depends upon claim 6:
Claim 7 recites:
The method of claim 6, wherein the method comprises: using the compressed representation to match a fingerprint.
Rolfe in view of Maragakis further in view of Rhee disclose the method of claim 6 upon which claim 7 depends. Furthermore, Rhee discloses the limitation of claim 7:
Rhee teaches an inference task that may be fingerprint recognition for the user of a smartphone (Paragraph 117). This would consist of a task wherein data obtained by the smartphone on a fingerprint is required to match a particular fingerprint. Furthermore, Rhee also teaches that a processor may obtain a result for a task of recognizing an object, such as a fingerprint, using the compressed representation (Paragraph 123).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe in view of Maragakis and the teachings of Rhee. This would have provided the advantage of improving the performance of a neural network through the use of a compressed representation (Rhee, Paragraph 57).
Claim 9 rejected under 35 U.S.C. 103 as being unpatentable over Rolfe in view of Maragakis further in view of Shahzad et al. (Pub. No. US 20210374664 A1, filed May 25th 2021, hereinafter Shahzad).
Regarding claim 9, which depends upon claim 1:
Claim 9 recites:
The method of claim 1, further comprising: defining an object by a set of 3D points as the training data from the data source that is expressed as sets; and using the trained set neural network to classify the object into an object class.
Rolfe in view of Maragakis disclose the method of claim 1 upon which claim 9 depends. Furthermore, Rolfe discloses using the trained set neural network to classify the object into an object class:
Rolfe teaches object classification (Paragraph 156), which is the act of using a neural network to classify an object into an object class.
Shahzad discloses defining an object by a set of 3D points as the training data from the data source that is expressed as sets:
Shahzad in the same field of endeavor of reinforcement learning teaches a 3D point cloud representation of an object (Paragraph 156), which is a set of 3D points defining an object.
Rolfe has previously taught data from a data source expressed as sets (Rolfe, Paragraph 2).
Rolfe, Maragakis, Shahzad, and the present application are analogous art because they are all in the same field of endeavor of reinforcement learning.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to implement a method that utilized the teachings of Rolfe in view of Maragakis and the teachings of Shahzad. This would have provided the advantage of easily identifying an object and its state (Shahzad, Paragraphs 3-4).
Response to Arguments
Applicant’s arguments filed 02-JANUARY-2026 have been fully considered, but the examiner believes that not all are fully persuasive.
Regarding the applicant’s remarks on the non-final office action’s 103 rejection of the claims, the applicant argues that Rolfe in view of Maragakis do not teach the amended limitations of these claims. As such, the applicant argues that all claims dependent on the above would additionally not be obvious under 103. However, the examiner believes that Rolfe in view of Maragakis does teach the amended limitations and respectfully requests applicant’s consideration of the following:
The applicant argues that the examiner does not explain how Rolfe’s approximating posterior distribution would be modified in view of Maragakis’s tokenization, nor how that modification would achieve the related features of the virtual tokens in claims 1, 13, and 15 (Applicant Arguments, page 12). Regarding the argument, the examiner respectfully requests consideration of the following:
Rolfe teaches forming an encoding distribution comprising an approximating distribution conditioned on the input space (Paragraph 14) – or essentially, a space that is meant to approximate through encoding the input space. This describes virtual tokens as from the input an approximation is created for further processing in a neural network that is not solely reliant on input data. However, since Rolfe does not teach tokenization exactly, the examiner has used Maragakis to teach tokenization. Maragakis teaches tokenization of a string for use in a neural network (Paragraph 119). This teaches the use of tokens as the tokenization of Maragakis is used to encode inputs, as the tokenization of further inputs can be seen as encoding by breaking apart a larger whole into smaller discrete parts. In this manner, the tokenization of Maragakis can be seen performing a similar role as the approximation of encoding taught by Rolfe, wherein the principles taught by Maragakis of the use of tokens may be applied to Rolfe in order to provide the advantage of increased accuracy and quality of neural networks (Maragakis, Paragraph 38). From there, the combination of Rolfe and Maragakis may perform the other functions as required by claim 1 as taught by Rolfe in the full rejection above.
Furthermore, the applicant argues that Rolfe is silent to any concatenation of the true prior distribution, which is asserted to disclose the claimed distribution containing the input tokens, and the approximating posterior distribution, which is asserted to be analogous to the claimed distribution containing the virtual tokens, as the cumulative distribution function merely includes random variable and specific inputs (Applicant Arguments, page 13). The examiner respectfully requests consideration of the following:
Rolfe teaches finding a KL-divergence between the true posterior distribution and the approximating posterior distribution, wherein determining the KL-divergence would include determining a gradient of each distribution (Paragraph 21). That both the approximating posterior distribution and the true prior distribution are over the discrete space indicates that they are concatenated together for the purposes of finding the KL-divergence.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDRIA JOSEPHINE MILLER whose telephone number is (703)756-5684. The examiner can normally be reached Monday-Thursday: 7:30 - 5:00 pm, every other Friday 7:30 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.J.M./Examiner, Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights