DETAILED ACTION
This Office Action is sent in response to Applicant’s Communication received 9/7/2023 for application number 18/462,914.
Claims 1-30 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. The Examiner notes that the “means” language in claim 30 is being interpreted under 112(f).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claim 1 (representative of independent claims 11, 21, and 30) recites:
A processing system comprising: a memory comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions and cause the processing system to: access a set of training data; determine a transformation group comprising a plurality of group elements; generate, based on the set of training data, a first set of unconstrained weights for a first layer of a machine learning model; generate, based on the set of training data, a first set of parameter values for a first likelihood function for the first layer; and generate a first set of constrained weights, based at least in part on the first likelihood function and the first set of unconstrained weights, such that the first set of constrained weights is equivariant with respect to at least a first subset of the plurality of group elements.
(2A, prong 1) The underlined portions of the claim recite an abstract idea, specifically a mental process and mathematical calculations and equations. A human can determine a transformation group (see spec. para. 0077 as published). The broadest reasonable interpretation of generating unconstrained weights, parameter values, and constrained weights are mathematical calculations and equations: Generating the unconstrained weights uses backpropagation and SGD which are mathematical calculations (spec. para. 0031), and generating the parameter values and constrained weights uses mathematical equations (see equations at spec. para. 0033-51).
(2A, prong 2) This judicial exception is not integrated into a practical application. The claims recite the additional elements of [a] generic computer components like processor and memory, and [b] accessing training data. Additional element [a] is a mere instruction to apply the exception because it merely adds generic computer components after-the-fact to the abstract idea. Additional element [b] is insignificant extra-solution activity because it is mere data gathering for the abstract idea. Even when all of the additional elements are considered in ordered combination with the recited abstract idea, the claim as a whole does not integrate the abstract idea into a practical application because the additional elements only add mere instructions to apply the exception and insignificant extra-solution activity to the abstract idea.
(2B) The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Additional element [a] is a mere instruction to apply the exception as explained above. Additional element [b] is well-understood, routine, and conventional activity, analogous to storing and retrieving information in memory, see MPEP 2106.05(d) citing Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). Even when all of the additional elements are considered in ordered combination with the recited abstract idea, the claim as a whole does not amount to significantly more than the abstract idea itself because the additional elements only add mere instructions to apply the exception and insignificant extra-solution activity that is well-understood, routine, and conventional to the abstract idea.
With respect to dependent claims 2-10, 12-20, and 22-29, these claims add further mathematical calculations to the abstract idea (see calculations and equations at spec. para. 0033-51).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 6, 10-11, 16, 20-21, 26, 29-30 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wang et al., Approximately Equivariant Networks for Imperfectly Symmetric Dynamics (see NPL [U]).
In reference to claim 1, Wang discloses a processing system comprising: a memory comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions (the machine learning of Wang would be implemented on a generic computer) and cause the processing system to: access a set of training data (training data like simulated smoke and jet flow data, page 2); determine a transformation group comprising a plurality of group elements (group of elements is determined, 3. Approximately Equivariant Networks, pages 3-4); generate, based on the set of training data, a first set of unconstrained weights for a first layer of a machine learning model (symmetry-independent weights are determined through training, 3.2. Relaxed Steerable Convolution and 3.3. Soft Equivariance Regularization, pages 3-4); generate, based on the set of training data, a first set of parameter values for a first likelihood function for the first layer (parameters for loss function are determined based on training data, 3.3. Soft Equivariance Regularization, page 4); and generate a first set of constrained weights, based at least in part on the first likelihood function and the first set of unconstrained weights, such that the first set of constrained weights is equivariant with respect to at least a first subset of the plurality of group elements (set of constrained weights that are equivariant are determined, 3.2. Relaxed Steerable Convolution and 3.3. Soft Equivariance Regularization, pages 3-4).
In reference to claim 6, Wang discloses the processing system of claim 1, wherein, to generate the first set of parameter values for the first likelihood function, the one or more processors are configured to execute the processor-executable instructions to cause the processing system to compute a loss based on divergence between the first likelihood function and a uniform distribution (loss is computed, 3.3. Soft Equivariance Regularization, page 4).
In reference to claim 10, Wang discloses the processing system of claim 1, wherein the first likelihood function defines, for each respective group element of the plurality of group elements, a respective non-binary degree of equivariance for the first layer (the loss function controls the degree of equivariance, 3.3. Soft Equivariance Regularization, page 4).
In reference to claim 11, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 1 and is therefore rejected under a similar rationale.
In reference to claim 16, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 6 and is therefore rejected under a similar rationale.
In reference to claim 20, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 10 and is therefore rejected under a similar rationale.
In reference to claim 21, this claim is directed to a method associated with the system claimed in claim 1 and is therefore rejected under a similar rationale.
In reference to claim 26, this claim is directed to a method associated with the system claimed in claim 6 and is therefore rejected under a similar rationale.
In reference to claim 29, this claim is directed to a method associated with the system claimed in claim 10 and is therefore rejected under a similar rationale.
In reference to claim 30, this claim is directed to a system similar in scope to the system claimed in claim 6 and is therefore rejected under a similar rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 2-4, 7-8, 12-14, 17-18, 22-24, and 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al., Approximately Equivariant Networks for Imperfectly Symmetric Dynamics (see NPL [U]) in view of van der Ouderaa, Relaxing Equivariance Constraints with Non-stationary Continuous Filters (cited in IDS of 9/7/2023).
In reference to claim 2, Wang does not explicitly teach the processing system of claim 1, wherein the one or more processors are configured to further execute the processor-executable instructions to cause the processing system to generate a respective set of constrained weights for each respective layer of the machine learning model based on the first likelihood function.
Van der Ouderaa teaches the processing system of claim 1, wherein the one or more processors are configured to further execute the processor-executable instructions to cause the processing system to generate a respective set of constrained weights for each respective layer of the machine learning model based on the first likelihood function (equivariance learned “in each layer,” Automatic symmetry discovery on page 3, also see fig. 2 on page 6: each layer can have a set of constrained weights generated).
It would have been obvious to one of ordinary skill in art, having the teachings of Wang and van der Ouderaa before the earliest effective filing date, to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa.
One of ordinary skill in the art would have been motivated to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa because van der Ouderaa explicitly states it is an improvement over Wang (van der Ouderaa, 2. Related Work, page 2).
In reference to claim 3, Wang does not explicitly teach the processing system of claim 1, wherein the one or more processors are configured to further execute the processor-executable instructions to cause the processing system to: generate, based on the set of training data, a second set of unconstrained weights for a second layer of the machine learning model; generate, based on the set of training data, a second set of parameter values for a second likelihood function for the second layer; and generate a second set of constrained weights, based at least in part on the second likelihood function and the second set of unconstrained weights, such that the second set of constrained weights is equivariant with respect to at least a second subset of the plurality of group elements.
Van der Ouderaa teaches the processing system of claim 1, wherein the one or more processors are configured to further execute the processor-executable instructions to cause the processing system to: generate, based on the set of training data, a second set of unconstrained weights for a second layer of the machine learning model; generate, based on the set of training data, a second set of parameter values for a second likelihood function for the second layer; and generate a second set of constrained weights, based at least in part on the second likelihood function and the second set of unconstrained weights, such that the second set of constrained weights is equivariant with respect to at least a second subset of the plurality of group elements (see 5.3 Gradient-based learning of relaxed equivariance, pages 8-9 and Intuition behind frequency parameters, page 6: each layer can has different levels of equivariance constraints for different subgroups, so a second layer could have a different set of constrained weights that are equivariant for a different subset of group elements).
It would have been obvious to one of ordinary skill in art, having the teachings of Wang and van der Ouderaa before the earliest effective filing date, to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa.
One of ordinary skill in the art would have been motivated to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa because van der Ouderaa explicitly states it is an improvement over Wang (van der Ouderaa, 2. Related Work, page 2).
In reference to claim 4, van der Ouderaa teaches the processing system of claim 3, wherein the first and second likelihood functions differ with respect to at least one group element of the plurality of group elements (see 5.3 Gradient-based learning of relaxed equivariance, pages 8-9 and Intuition behind frequency parameters, page 6: each layer can has different levels of equivariance constraints for different subgroups, so a second layer could have a different set of constrained weights that are for different elements).
In reference to claim 7, Wang does not explicitly teach the processing system of claim 1, wherein, to generate the first set of constrained weights, the one or more processors are configured to execute the processor-executable instructions to cause the processing system to project the first set of unconstrained weights to the first set of constrained weights based on the first likelihood function.
Van der Ouderaa teaches the processing system of claim 1, wherein, to generate the first set of constrained weights, the one or more processors are configured to execute the processor-executable instructions to cause the processing system to project the first set of unconstrained weights to the first set of constrained weights based on the first likelihood function (unconstrained weights are projected to get constrained weights, see 3.3 The group convolution to 4.2 Parameterising the kernel, pages 3-6).
It would have been obvious to one of ordinary skill in art, having the teachings of Wang and van der Ouderaa before the earliest effective filing date, to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa.
One of ordinary skill in the art would have been motivated to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa because van der Ouderaa explicitly states it is an improvement over Wang (van der Ouderaa, 2. Related Work, page 2).
In reference to claim 8, Wang does not explicitly teach the processing system of claim 1, wherein the first set of parameter values comprises Fourier series coefficients.
Van der Ouderaa teaches the processing system of claim 1, wherein the first set of parameter values comprises Fourier series coefficients (see Intuition behind frequency parameters, page 6).
It would have been obvious to one of ordinary skill in art, having the teachings of Wang and van der Ouderaa before the earliest effective filing date, to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa.
One of ordinary skill in the art would have been motivated to modify the layers of Wang to include the set of constrained weights for each layer of van der Ouderaa because van der Ouderaa explicitly states it is an improvement over Wang (van der Ouderaa, 2. Related Work, page 2).
In reference to claim 12, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 2 and is therefore rejected under a similar rationale.
In reference to claim 13, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 3 and is therefore rejected under a similar rationale.
In reference to claim 14, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 4 and is therefore rejected under a similar rationale.
In reference to claim 17, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 7 and is therefore rejected under a similar rationale.
In reference to claim 18, this claim is directed to a non-transitory computer-readable medium associated with the system claimed in claim 8 and is therefore rejected under a similar rationale.
In reference to claim 22, this claim is directed to a method associated with the system claimed in claim 2 and is therefore rejected under a similar rationale.
In reference to claim 23, this claim is directed to a method associated with the system claimed in claim 3 and is therefore rejected under a similar rationale.
In reference to claim 24, this claim is directed to a method associated with the system claimed in claim 4 and is therefore rejected under a similar rationale.
In reference to claim 27, this claim is directed to a method associated with the system claimed in claim 7 and is therefore rejected under a similar rationale.
Allowable Subject Matter
Claims 5, 9, 15, 19, 25, and 28 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claims 5, 15, and 25 recite, “wherein, to generate the second set of parameter values for the second likelihood function, the one or more processors are configured to execute the processor-executable instructions to cause the processing system to compute a loss based on divergence between the second likelihood function and the first likelihood function.” While Wang in view of van der Ouderaa teaches calculating loss and first and second likelihood functions, they do not teach generating the second set of parameters for the second loss function based on a divergence between the first and second likelihood functions.
Claims 9, 19, and 28 recite, “initialize the first set of unconstrained weights using randomly generated values; and initialize the first set of parameter values such that the first likelihood function is a uniform distribution.” The prior art does not teach initializing the first set of parameter values so that the first likelihood function is a uniform distribution.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The references in the Notice of References Cited and not used above are generally relevant to equivariance in machine learning.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Andrew T. Chiusano whose telephone number is (571)272-5231. The examiner can normally be reached M-F, 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571-272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW T CHIUSANO/Primary Examiner, Art Unit 2144
Prosecution Insights