DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that does not use the word “means,” and are interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are, where the generic place holder has been underlined and the functional language italicize:
“means for accessing a first feature tensor as input to a portion of a machine learning model;
means for generating a second feature tensor based on processing the first feature tensor using the portion of the machine learning model;
means for generating a frequency tensor based on processing the first feature tensor using a Fourier transform operation;
means for generating a transformed frequency tensor based on processing the frequency tensor using a trained adapter corresponding to the portion of the machine learning model;
means for generating a third feature tensor based on processing the transformed frequency tensor using an inverse Fourier transform operation;
and means for generating a fourth feature tensor as output from the portion of the machine learning model based on aggregating the second and third feature tensors” in claim 20.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112(a)
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 20 is further rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
Claim 20 recites claim limitations interpreted under 35 U.S.C. 112(f) wherein the means lack written description of the invention in full, clear, concise and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
“means for accessing a first feature tensor as input to a portion of a machine learning model;
means for generating a second feature tensor based on processing the first feature tensor using the portion of the machine learning model;
means for generating a frequency tensor based on processing the first feature tensor using a Fourier transform operation;
means for generating a transformed frequency tensor based on processing the frequency tensor using a trained adapter corresponding to the portion of the machine learning model;
means for generating a third feature tensor based on processing the transformed frequency tensor using an inverse Fourier transform operation;
and means for generating a fourth feature tensor as output from the portion of the machine learning model based on aggregating the second and third feature tensors.”
In light of the specification of the instant application in para. [0139], the means may be performed purely by software components, thus sufficient corresponding structure to the means are not clearly linked. (“[0139] The methods disclosed herein comprise one or more steps or actions for achieving the methods. The method steps and/or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and/or use of specific steps and/or actions may be modified without departing from the scope of the claims. Further, the various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and/or software component(s) and/or module(s), including, but not limited to a circuit, an application specific integrated circuit (ASIC), or processor.”)
Further, there is insufficient written description for the specific algorithms for achieving the means of the limitations below:
“means for generating a frequency tensor based on processing the first feature tensor using a Fourier transform operation;
means for generating a third feature tensor based on processing the transformed frequency tensor using an inverse Fourier transform operation
and means for generating a fourth feature tensor as output from the portion of the machine learning model based on aggregating the second and third feature tensors”
In light of the specification of the instant application, para. [0083] recites “a first frequency tensor is generated based on processing the first feature tensor using a Fourier transform operation.” wherein para. [0036] recites “In the illustrated example, the feature tensor 205 is first processed using a Fourier transform 210 (e.g., a fast Fourier transform (FFT)), sometimes referred to as a Fourier transform operation, to generate a frequency tensor 215.” However, the recited fast Fourier transform is merely provided as an example and does not explicitly link the Fourier transform operation as a fast Fourier transform.
Similarly, para. [0042] merely recites an inverse fast Fourier transform as an example of an inverse Fourier transform operation and does not explicitly link the inverse Fourier transform operation as an inverse fast Fourier transform. “In the illustrated example, the transformed frequency tensor 255 can then be processed using an inverse Fourier transform 260 (e.g., an inverse FFT (IFFT)), sometimes referred to as an inverse Fourier transform operation, to generate the feature tensor 265.”
Further, para. [0028] recites aggregation may be elementwise summation, concatenation, and the like and fails to explicitly link the specific algorithm to the means for generating a fourth feature tensor “As illustrated, the input data 110 is processed by a first layer 115A (or other component or portion of the base model), as well as a first adapter 120A. The resulting output from the layer 115A (e.g., a first feature tensor generated by the layer 115A) is then aggregated with the output of the adapter 120A (e.g., a second feature tensor) using operation 125A. For example, the operation 125A may include elementwise summation, concatenation, and the like.” See MPEP § 2163.03, subsection VI.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 20 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim limitation “means for accessing a first feature tensor as input to a portion of a machine learning model;
means for generating a second feature tensor based on processing the first feature tensor using the portion of the machine learning model;
means for generating a frequency tensor based on processing the first feature tensor using a Fourier transform operation;
means for generating a transformed frequency tensor based on processing the frequency tensor using a trained adapter corresponding to the portion of the machine learning model;
means for generating a third feature tensor based on processing the transformed frequency tensor using an inverse Fourier transform operation;
and means for generating a fourth feature tensor as output from the portion of the machine learning model based on aggregating the second and third feature tensors” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. See 112(a) rejection above. Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In regards to claim 1,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A – Prong 1: Judicial Exception Recited?
MPEP 2106.04(a)(2)(II) “The mathematical concepts grouping is defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations.”
Further, the MPEP recites “It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula." In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). See, e.g., SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 127 USPQ2d 1597, 1599 (Fed. Cir. 2018) (holding that claims to a ‘‘series of mathematical calculations based on selected information’’ are directed to abstract ideas); Digitech Image Techs., LLC v. Elecs. for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (holding that claims to a ‘‘process of organizing information through mathematical correlations’’ are directed to an abstract idea); and Bancorp Servs., LLC v. Sun Life Assurance Co. of Can. (U.S.), 687 F.3d 1266, 1280, 103 USPQ2d 1425, 1434 (Fed. Cir. 2012) (identifying the concept of ‘‘managing a stable value protected life insurance policy by performing calculations and manipulating the results’’ as an abstract idea).”
Yes, the claim recites a mathematical concept, specifically:
generate a first frequency tensor based on processing the first feature tensor using a Fourier transform operation;
This limitation encompasses performing a mathematical calculation (Fourier transform) on a given tensor to generate another tensor.
generate a third feature tensor based on processing the first transformed frequency tensor using an inverse Fourier transform operation;
This limitation encompasses performing a mathematical calculation (inverse Fourier transform) on a given tensor to generate another tensor.
and generate a fourth feature tensor as output from the first portion of the machine learning model based on aggregating the second and third feature tensors
This limitation encompasses performing a mathematical calculation (aggregation) on two given tensors to generate another tensor.
Therefore, the claim recites a mathematical concept.
Step 2A – Prong 2: Integrated into a Practical Solution?
MPEP 2106.05(f) Mere Instructions To Apply An Exception has found simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. The following steps are mere instructions to apply:
A processing system comprising: one or more memories comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions and cause the processing system to: (generic computer components to apply the abstract idea)
first portion of a machine learning model
generate a second feature tensor based on processing the first feature tensor using the first portion of the machine learning model (mere instructions to apply the first portion of a ML model to the first feature tensor to obtain the second feature tensor without specifying how the first portion of the ML model processes the first feature tensor)
generate a first transformed frequency tensor based on processing the first frequency tensor using a first trained adapter corresponding to the first portion of the machine learning model (mere instructions to apply a first trained adapter to the first frequency tensor to obtain the first transformed frequency tensor without specifying how the first trained adapter processes the first frequency tensor)
MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering to be insignificant extra-solution activity. The following steps are insignificant extra-solution activities:
Mere data gathering:
access a first feature tensor as input to a first portion of a machine learning model (receiving the first feature tensor to transmit to the first portion of a machine learning model)
The additional elements have been considered both individually and as an ordered combination in to determine whether they integrate the exception into a practical application. Therefore, no meaningful limits are imposed on practicing the abstract idea.
The claim is directed to the abstract idea.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitation is mere data gathering (Insignificant Extra-Solution Activity) and Mere Instructions To Apply An Exception and a generic device do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
The claim recites using generic computer components to apply the abstract idea wherein use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more.
Further, the recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
The claim further recites receiving and transmitting data by a generic device.
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to
gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary
computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607,
610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP
Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)
(sending messages over a network); buy SAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112
USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);
but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106
(Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how
interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides
the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
(emphasis added)).
The additional elements have been considered both individually and as an ordered
combination in the significantly more consideration.
The claim is ineligible.
In regards to claim 2,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
generate a second frequency tensor based on processing the first feature tensor using the Fourier transform operation; generate a fifth feature tensor based on processing the second transformed frequency tensor using the inverse Fourier transform operation
These limitations direct to mathematical calculations. See MPEP 2106.04(a)(2)(I)(C.)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the first portion of the machine learning model; and generate the fourth feature tensor based further on the fifth feature tensor
These limitations direct to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely reciting only the idea of a solution or outcome. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the first portion of the machine learning model; and generate the fourth feature tensor based further on the fifth feature tensor
These limitations direct to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely reciting only the idea of a solution or outcome. See MPEP 2106.05(f)
In regards to claim 3,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
generate a second frequency tensor based on processing the fifth feature tensor using the Fourier transform operation; generate a sixth feature tensor based on processing the second transformed frequency tensor using the inverse Fourier transform operation; and generate a seventh feature tensor as output from the second portion of the machine learning model based on aggregating the fifth and sixth feature tensors
These limitations direct to mathematical calculations. See MPEP 2106.04(a)(2)(I)(C.)
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
generate a fifth feature tensor based on processing the fourth feature tensor using a second portion of the machine learning model; generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the second portion of the machine learning model
These limitations directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
generate a fifth feature tensor based on processing the fourth feature tensor using a second portion of the machine learning model; generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the second portion of the machine learning model
These limitations directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
In regards to claim 4,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the trained adapter comprises a frequency mask generator
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the trained adapter comprises a frequency mask generator
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
In regards to claim 5,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
generate a frequency mask, using the frequency mask generator, based on the first frequency tensor; and generate the first transformed frequency tensor based on the frequency mask
These limitations directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
generate a frequency mask, using the frequency mask generator, based on the first frequency tensor; and generate the first transformed frequency tensor based on the frequency mask
These limitations directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
In regards to claim 6,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
generate an intermediate tensor based on processing the first frequency tensor using a first portion of the first trained adapter; generate the frequency mask based on processing the intermediate tensor using the frequency mask generator; apply the frequency mask to the intermediate tensor to generate a masked intermediate tensor; and generate the first transformed frequency tensor based on processing the masked intermediate tensor using a second portion of the first trained adapter
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
generate an intermediate tensor based on processing the first frequency tensor using a first portion of the first trained adapter; generate the frequency mask based on processing the intermediate tensor using the frequency mask generator; apply the frequency mask to the intermediate tensor to generate a masked intermediate tensor; and generate the first transformed frequency tensor based on processing the masked intermediate tensor using a second portion of the first trained adapter
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
In regards to claim 7,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
apply an adapter weight to the intermediate tensor
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
apply an adapter weight to the intermediate tensor
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
In regards to claim 8,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the frequency mask generator was trained to mask frequencies correlated with mode collapse in model output
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Examiner’s note: Training is recited at the highest level of generality and appears to be recited in order to claim the solution of masking frequencies correlated with mode collapse in the model output without specifying how the frequency mask generator was trained.
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the frequency mask generator was trained to mask frequencies correlated with mode collapse in model output
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Examiner’s note: Training is recited at the highest level of generality and appears to be recited in order to claim the solution of masking frequencies correlated with mode collapse in the model output without specifying how the frequency mask generator was trained.
In regards to claim 9,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
wherein the first trained adapter comprises a frequency-domain low-rank adapter
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
wherein the first trained adapter comprises a frequency-domain low-rank adapter
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
In regards to claim 10,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A Prong 1: The claim recites the following abstract ideas:
The abstract idea(s) in the parent claim(s).
Step 2A Prong 2: The claim recites the following additional elements which, considered individually and as an ordered combination, do not integrate the abstract idea into a practical application:
The additional element(s) in the parent claim(s).
generate a model output based on the fourth feature tensor
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
wherein the model output comprises image data
This limitation merely indicates a field of use or technological environment in which the judicial exception is performed. This type of limitation merely confines the use of the abstract idea to a particular technological environment (image analysis) and thus fails to add an inventive concept to the claims. See MPEP § 2106.05(h)
Step 2B: The claim recites the following additional elements which, considered individually and as an ordered combination, do not amount to significantly more than the abstract idea:
The additional element(s) in the parent claim(s).
generate a model output based on the fourth feature tensor
This limitation directs to merely applying (or equivalent) an abstract idea, or implementing an abstract idea on a computer, or using a computer as a tool to perform an abstract idea or merely recites only the idea of a solution or outcome. See MPEP 2106. See MPEP 2106.05(f)
wherein the model output comprises image data
This limitation merely indicates a field of use or technological environment in which the judicial exception is performed. This type of limitation merely confines the use of the abstract idea to a particular technological environment (image analysis) and thus fails to add an inventive concept to the claims. See MPEP § 2106.05(h)
Claim 11 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 1 as they are substantially similar, respectively, Mutatis mutandis.
Claim 12 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 2 as they are substantially similar, respectively, Mutatis mutandis.
Claim 13 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 3 as they are substantially similar, respectively, Mutatis mutandis.
Claim 14 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 4 as they are substantially similar, respectively, Mutatis mutandis.
Claim 15 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 5 as they are substantially similar, respectively, Mutatis mutandis.
Claim 16 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 6 as they are substantially similar, respectively, Mutatis mutandis.
Claim 17 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 7 as they are substantially similar, respectively, Mutatis mutandis.
Claim 18 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 8 as they are substantially similar, respectively, Mutatis mutandis.
Claim 19 (method) is rejected on the same grounds under 35 U.S.C. 101 as claim 10 as they are substantially similar, respectively, Mutatis mutandis.
In regards to claim 20,
Step 1: This part of the eligibility analysis evaluates whether the claim(s) falls within any statutory category. See MPEP 2106.03:
The claim directs to a statutory category – machine.
Step 2A – Prong 1: Judicial Exception Recited?
MPEP 2106.04(a)(2)(II) “The mathematical concepts grouping is defined as mathematical relationships, mathematical formulas or equations, and mathematical calculations.”
Further, the MPEP recites “It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula." In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). See, e.g., SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 127 USPQ2d 1597, 1599 (Fed. Cir. 2018) (holding that claims to a ‘‘series of mathematical calculations based on selected information’’ are directed to abstract ideas); Digitech Image Techs., LLC v. Elecs. for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (holding that claims to a ‘‘process of organizing information through mathematical correlations’’ are directed to an abstract idea); and Bancorp Servs., LLC v. Sun Life Assurance Co. of Can. (U.S.), 687 F.3d 1266, 1280, 103 USPQ2d 1425, 1434 (Fed. Cir. 2012) (identifying the concept of ‘‘managing a stable value protected life insurance policy by performing calculations and manipulating the results’’ as an abstract idea).”
Yes, the claim recites a mathematical concept, specifically:
means for generating a frequency tensor based on processing the first feature tensor using a Fourier transform operation
This limitation encompasses performing a mathematical calculation (Fourier transform) on a given tensor to generate another tensor.
means for generating a third feature tensor based on processing the transformed frequency tensor using an inverse Fourier transform operation;
This limitation encompasses performing a mathematical calculation (inverse Fourier transform) on a given tensor to generate another tensor.
and means for generating a fourth feature tensor as output from the portion of the machine learning model based on aggregating the second and third feature tensors
This limitation encompasses performing a mathematical calculation (aggregation) on two given tensors to generate another tensor.
Therefore, the claim recites a mathematical concept.
Step 2A – Prong 2: Integrated into a Practical Solution?
MPEP 2106.05(f) Mere Instructions To Apply An Exception has found simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. The following steps are mere instructions to apply:
A processing system comprising: (generic computer components to apply the abstract idea)
first portion of a machine learning model
means for generating a second feature tensor based on processing the first feature tensor using the portion of the machine learning model (mere instructions to apply the first portion of a ML model to the first feature tensor to obtain the second feature tensor without specifying how the portion of the ML model processes the first feature tensor)
means for generating a transformed frequency tensor based on processing the frequency tensor using a trained adapter corresponding to the portion of the machine learning model (mere instructions to apply a trained adapter to the frequency tensor to obtain the transformed frequency tensor without specifying how the trained adapter processes the first frequency tensor)
MPEP 2106.05(g) Insignificant Extra-Solution Activity has found mere data gathering to be insignificant extra-solution activity. The following steps are insignificant extra-solution activities:
Mere data gathering:
means for accessing a first feature tensor as input to a portion of a machine learning model (receiving the first feature tensor to transmit to the first portion of a machine learning model)
The additional elements have been considered both individually and as an ordered combination in to determine whether they integrate the exception into a practical application. Therefore, no meaningful limits are imposed on practicing the abstract idea.
The claim is directed to the abstract idea.
Step 2B: Claim provides an Inventive Concept?
No, as discussed with respect to Step 2A, the additional limitation is mere data gathering (Insignificant Extra-Solution Activity) and Mere Instructions To Apply An Exception and a generic device do not impose any meaningful limits on practicing the abstract idea and therefore the claim does not provide an inventive concept in Step 2B.
The claim recites using generic computer components to apply the abstract idea wherein use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more.
Further, the recitation of claim limitations that attempt to cover any solution to an identified problem with no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result, does not integrate a judicial exception into a practical application or provide significantly more because this type of recitation is equivalent to the words "apply it".
The claim further recites receiving and transmitting data by a generic device.
This has been determined to be insignificant extra-solution activity as found in MPEP § 2106.05(d)(II)(i): Receiving or transmitting data over a network, e.g., using the Internet to
gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary
computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607,
610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP
Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)
(sending messages over a network); buy SAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112
USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);
but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106
(Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how
interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides
the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink."
(emphasis added)).
The additional elements have been considered both individually and as an ordered
combination in the significantly more consideration.
The claim is ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 9-13 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over US Pub no. US20220101494A1 Korani et al. (“Korani”) in view of US Pub No. US20250156684A1 Liu et al. (“Liu”)
In regards to claim 1 and analogous claims 11 and 20,
Korani teaches A processing system comprising: one or more memories comprising processor-executable instructions; and one or more processors configured to execute the processor-executable instructions and cause the processing system to:
(Korani, “[0570] Operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. In at least one embodiment, a process such as those processes described herein (or variations and/or combinations thereof) is performed under control of one or more computer systems configured with executable instructions and is implemented as code (e.g., executable instructions, one or more computer programs or one or more applications) executing collectively on one or more processors, by hardware or combinations thereof.”)
Korani teaches access a first feature tensor as input to a first portion of a machine learning model; generate a second feature tensor based on processing the first feature tensor using the first portion of the machine learning model;
PNG
media_image1.png
610
417
media_image1.png
Greyscale
(Korani, “[0078] FIG. 4 is a block diagram illustrating Fast Fourier Transform-based up-sampling of an input feature map 402 [access a first feature tensor ie input feature map 402 as input to a first portion of a machine learning model] to an output expanded feature map 424 [generate a second feature tensor ie an expanded feature map 424 based on processing the first feature tensor using the first portion of the machine learning model], according to at least one embodiment.”)
Korani teaches generate a first frequency tensor based on processing the first feature tensor using a Fourier transform operation;
(Korani, [0078] “In at least one embodiment, Fast Fourier Transform (FFT) up-sampling 404 is data values and/or software instructions that, when executed, generate an expanded feature map 424 from an input feature map 402 based, at least in part, on up-sampling data converted using a FFT 406, as described above in conjunction with FIG. 3…
[0080] In at least one embodiment, FFT up-sampling 404 comprises a Fast Fourier Transform (FFT) 406. In at least one embodiment, a FFT 406 is data values and software instructions that, when executed, apply a Fast Fourier Transform to an input feature map h i 402. In at least one embodiment, FFT up-sampling 404 applies a FFT 406 to an input feature map h i 402 such that {tilde over (h)}i:=FFT (hi) [generate a first frequency tensor ie output of applying FFT to the input feature map ({tilde over (h)}i) based on processing the first feature tensor using a Fourier transform operation].”)
Korani teaches generate a third feature tensor based on processing the first transformed frequency tensor using an inverse Fourier transform operation;
(Korani, [0075], “FFT up-sampling, in an embodiment, converts an enlarged output FFT feature map {tilde over (H)}i back to a data format associated with input 304 textured image x, such as RGB, by applying an Inverse Fast Fourier Transform (IFFT) as Hi=IFFT ({tilde over (H)}i) [generate a third feature tensor ie output after applying IFFT to the feature map (another expanded feature map) based on processing the first transformed frequency tensor ie the modified data provided by the adapter of Liu using an inverse Fourier transform operation]. In at least one embodiment, an IFFT is an implementation of an inverse Fourier transform.”)
Korani teaches and generate a fourth feature tensor as output from the first portion of the machine learning model based on aggregating the second and third feature tensors.
(Korani, “[0077] In at least one embodiment, a decoder 310 generates a large or expanded (up-sampled) output 312 [and generate a fourth feature tensor as output from the first portion of the machine learning model] textured image by applying a decoder function fθ dec , on expanded (up-sampled) feature maps {Hi}i=1 I as fθ dec ({Hi}i=1 I)…
In at least one embodiment, a decoder 310 is a convolutional decoder where said decoder 310 concatenates features from feature maps {Hi}i=1 I at multiple scales for improved decoding. In at least one embodiment, a decoder 310 performs concatenations [based on aggregating the second and third feature tensors] for output from upsampling 8×8, 16×16, and 32×32 feature maps {hi}i=1 M generated by an encoder 306 in a neural network 302.”)
However, Korani does not explicitly teach generate a first transformed frequency tensor based on processing the first frequency tensor using a first trained adapter corresponding to the first portion of the machine learning model;
Liu teaches generate a first transformed frequency tensor based on processing the first frequency tensor using a first trained adapter corresponding to the first portion of the machine learning model;
(Liu, [0037], “The method comprises connecting, using a connector of the adapter, the adapter to the base model such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model. The method further comprises modifying, using a non-linear modifier of the adapter, the data received from the base model non-linearly before returning the modified portion [generate a first transformed frequency tensor ie modified data based on processing the first frequency tensor (wherein Korani provides the first frequency tensor to the adapter of Liu) using a first trained adapter corresponding to the first portion of the machine learning model (wherein the base model of Liu is interpreted to be the model of Korani)] of the data back to the base model.”)
Korani is considered to be analogous to the claimed invention because they are in the same field of generative neural networks, particularly image synthesis. Liu is considered to be analogous to the claimed invention because they are in the same field of applying adapters to base models and parameter-efficient fine-tuning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Korani to incorporate the teachings of Liu in order to provide a PEFT technique in the form of adapters as doing so enables fine-tuning of the model that greatly reduces computational and memory costs and quickly adapt to new tasks (Liu, “[0005] In certain cases, parameter-efficient fine-tuning (PEFT) techniques may be used in scenarios where computational resources or labeled task-specific data are limited. The PEFT techniques may enable to strike a balance between the knowledge encoded in pre-trained models and adapting to the specifics of a target task/domain with a reduced number of trainable parameters. For example, PEFT techniques only fine-tune a small set of parameters, which may be a subset of the existing parameters of the pre-trained models or a set of newly added parameters, thereby greatly reducing the computational and memory costs. PEFT techniques also allow to store only a small number of model parameters for domain adaptation in addition to the pre-trained model. To this end, for multiple downstream tasks, PEFT techniques greatly save storage, while the full fine-tuning is needed to generate a new large model for each downstream task. Besides parameter savings, PEFT makes it possible to quickly adapt to new tasks without catastrophic forgetting or overfitting, which has been often observed during the full fine-tuning of AI models.”)
In regards to claim 2 and analogous claim 12,
Korani and Liu teach The processing system of claim 1,
Korani teaches wherein the one or more processors are configured to further execute the processor-executable instructions and cause the processing system to: generate a second frequency tensor based on processing the first feature tensor using the Fourier transform operation;
(Korani, [0073], “FFT up-sampling 306 is, in an embodiment, data values and/or software instructions that, when executed, up-sample one or more input feature maps {hi}i=1 M into one or more expanded feature maps {hi}i=1 M by at least converting said input feature maps {hi}i=1 M to a frequency domain representation [generate a second frequency tensor ie another expanded feature map after FFT based on processing the first feature tensor using the Fourier transform operation] before up-sampling and then restoring up-sampled frequency domain data back to input feature maps {hi}i=1 M original data representation.”)
Korani teaches generate a fifth feature tensor based on processing the second transformed frequency tensor using the inverse Fourier transform operation;
(Korani, [0075], “FFT up-sampling, in an embodiment, converts an enlarged output FFT feature map {tilde over (H)}i back to a data format associated with input 304 textured image x, such as RGB, by applying an Inverse Fast Fourier Transform (IFFT) as Hi=IFFT ({tilde over (H)}i) [generate a third feature tensor ie output after applying IFFT to the feature map (another expanded feature map) based on processing the first transformed frequency tensor ie the modified data provided by the adapter of Liu using an inverse Fourier transform operation]. In at least one embodiment, an IFFT is an implementation of an inverse Fourier transform.”)
Korani teaches and generate the fourth feature tensor based further on the fifth feature tensor.
(Korani, “[0077] In at least one embodiment, a decoder 310 generates a large or expanded (up-sampled) output 312 [and generate the fourth feature tensor] textured image by applying a decoder function fθ dec , on expanded (up-sampled) feature maps {Hi}i=1 I as fθ dec ({Hi}i=1 I)…
In at least one embodiment, a decoder 310 is a convolutional decoder where said decoder 310 concatenates features from feature maps {Hi}i=1 I at multiple scales for improved decoding. In at least one embodiment, a decoder 310 performs concatenations [based further on the fifth feature tensor] for output from upsampling 8×8, 16×16, and 32×32 feature maps {hi}i=1 M generated by an encoder 306 in a neural network 302.”)
However, Korani does not explicitly teach generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the first portion of the machine learning model;
Liu teaches generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the first portion of the machine learning model;
(Liu, “[0068] In an example, an architecture of the adapter is designed to be modular and easily pluggable, allowing the incorporation of multiple adapters [using a second trained adapter corresponding to the first portion of the machine learning model (wherein the base model of Liu is interpreted to be the model of Korani)] for different tasks without significantly modifying the original pre-trained base model. In certain cases, an adapter may be plugged to another adapter for further fine-tuning of the base model for a particular domain or task.”)
(Liu, [0037], “The method comprises connecting, using a connector of the adapter, the adapter to the base model such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model. The method further comprises modifying, using a non-linear modifier of the adapter, the data received from the base model non-linearly before returning the modified portion [generate a second transformed frequency tensor ie modified data based on processing the second frequency tensor (wherein Korani provides the second frequency tensor to the adapter of Liu)] of the data back to the base model.”)
In regards to claim 3 and analogous claim 13,
Korani and Liu teach The processing system of claim 1,
Korani teaches wherein the one or more processors are configured to further execute the processor-executable instructions and cause the processing system to: generate a fifth feature tensor based on processing the fourth feature tensor using a second portion of the machine learning model;
Korani discloses deployment pipelines may be sequentially applied to imaging data
(Korani, “[0513] FIG. 37 is a system diagram for an example system 3700 for generating and deploying an imaging deployment pipeline, in accordance with at least one embodiment…
[0520] In at least one embodiment, deployment system 3606 may execute deployment pipelines 3710. In at least one embodiment, deployment pipelines 3710 may include any number of applications that may be sequentially, non-sequentially, or otherwise applied to imaging data (and/or other data types) generated by imaging devices, sequencing devices, genomics devices, etc.—including AI-assisted annotation, as described above.”)
Korani further discloses preparing an output for a next application
(Korani, [0505], “In at least one embodiment, post-processing may be performed on an output of one or more inferencing tasks or other processing tasks of a pipeline to prepare an output data for a next application and/or to prepare output data for transmission and/or use by a user (e.g., as a response to an inference request). In at least one embodiment, inferencing tasks may be performed by one or more machine learning models [using a second portion of the machine learning model], such as trained or deployed neural networks, which may include output models 3616 of training system 3604.”)
(Korani, “[0078] FIG. 4 is a block diagram illustrating Fast Fourier Transform-based up-sampling of an input feature map 402 [based on processing the fourth feature tensor ie the output from the first portion of the ML model wherein the second portion of the ML model is interpreted as a sequential imaging deployment pipeline] to an output expanded feature map 424 [generate a fifth feature tensor ie an expanded feature map 424], according to at least one embodiment.”)
Korani teaches generate a second frequency tensor based on processing the fifth feature tensor using the Fourier transform operation;
(Korani, [0078] “In at least one embodiment, Fast Fourier Transform (FFT) up-sampling 404 is data values and/or software instructions that, when executed, generate an expanded feature map 424 from an input feature map 402 based, at least in part, on up-sampling data converted using a FFT 406, as described above in conjunction with FIG. 3…
[0080] In at least one embodiment, FFT up-sampling 404 comprises a Fast Fourier Transform (FFT) 406. In at least one embodiment, a FFT 406 is data values and software instructions that, when executed, apply a Fast Fourier Transform to an input feature map h i 402. In at least one embodiment, FFT up-sampling 404 applies a FFT 406 to an input feature map h i 402 such that {tilde over (h)}i:=FFT (hi) [generate a second frequency tensor ie output of applying FFT to the input feature map ({tilde over (h)}i) based on processing the fifth feature tensor using the Fourier transform operation].”)
Korani teaches generate a sixth feature tensor based on processing the second transformed frequency tensor using the inverse Fourier transform operation;
(Korani, [0075], “FFT up-sampling, in an embodiment, converts an enlarged output FFT feature map {tilde over (H)}i back to a data format associated with input 304 textured image x, such as RGB, by applying an Inverse Fast Fourier Transform (IFFT) as Hi=IFFT ({tilde over (H)}i) [generate a sixth feature tensor ie output after applying IFFT to the feature map (another expanded feature map) based on processing the second transformed frequency tensor ie the modified data provided by the adapter of Liu using the inverse Fourier transform operation]. In at least one embodiment, an IFFT is an implementation of an inverse Fourier transform.”)
Korani teaches and generate a seventh feature tensor as output from the second portion of the machine learning model based on aggregating the fifth and sixth feature tensors.
(Korani, “[0077] In at least one embodiment, a decoder 310 generates a large or expanded (up-sampled) output 312 [and generate a seventh feature tensor as output from the second portion of the machine learning model] textured image by applying a decoder function fθ dec , on expanded (up-sampled) feature maps {Hi}i=1 I as fθ dec ({Hi}i=1 I)…
In at least one embodiment, a decoder 310 is a convolutional decoder where said decoder 310 concatenates features from feature maps {Hi}i=1 I at multiple scales for improved decoding. In at least one embodiment, a decoder 310 performs concatenations [based on aggregating the fifth and sixth feature tensors] for output from upsampling 8×8, 16×16, and 32×32 feature maps {hi}i=1 M generated by an encoder 306 in a neural network 302.”)
However, Korani does not explicitly teach generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the second portion of the machine learning model;
Liu teaches generate a second transformed frequency tensor based on processing the second frequency tensor using a second trained adapter corresponding to the second portion of the machine learning model;
(Liu, “[0068] In an example, an architecture of the adapter is designed to be modular and easily pluggable, allowing the incorporation of multiple adapters [using a second trained adapter corresponding to the second portion of the machine learning model (wherein the base model of Liu is interpreted to be the model of Korani)] for different tasks without significantly modifying the original pre-trained base model. In certain cases, an adapter may be plugged to another adapter for further fine-tuning of the base model for a particular domain or task.”)
(Liu, [0037], “The method comprises connecting, using a connector of the adapter, the adapter to the base model such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model. The method further comprises modifying, using a non-linear modifier of the adapter, the data received from the base model non-linearly before returning the modified portion [generate a second transformed frequency tensor ie modified data based on processing the second frequency tensor (wherein Korani provides the second frequency tensor to the adapter of Liu)] of the data back to the base model.”)
In regards to claim 9,
Korani and Liu teach The processing system of claim 1,
Korani in view of Liu teach wherein the first trained adapter comprises a frequency-domain low-rank adapter.
Examiner’s note: Examiner interprets “a frequency-domain low-rank adapter” as a low-rank adapter applied in the frequency domain in light of the specification of the instant application. (“[0018] In some aspects of the present disclosure, low-rank adapter operations can be applied in the frequency domain, rather than in the spatial domain, to improve model performance. For example, in some aspects, low-rank adaptation (LoRA) adapters may be trained to operate in the frequency domain, which may improve generation diversity and prevent (or at least reduce) generation bias. In some aspects, these adapters may be referred to as frequency-domain low-rank adapters.”)
Korani teaches frequency-domain
(Korani, [0081], “In at least one embodiment, {tilde over (h)}i output from a FFT 406 is frequency domain data.”)
Liu teaches low-rank adapter
(Liu, “[0014] Some embodiments are based on a recognition that an adapter tuning technique based on Low-Rank Adaptation (LoRA) demonstrates performance comparable to full fine-tuning, despite having significantly fewer trainable parameters.”)
In regards to claim 10 and analogous claim 19,
Korani and Liu teach The processing system of claim 1,
Korani teaches wherein the one or more processors are configured to further execute the processor-executable instructions and cause the processing system to generate a model output based on the fourth feature tensor, wherein the model output comprises image data.
(Korani, “[0090] FIG. 5 is a block diagram illustrating an end-to-end neural network architecture 500 to generate an expanded output textured image 530 [generate a model output based on the fourth feature tensor, wherein the model output comprises image data] output from an input textured image 502 using Fast Fourier Transform up- sampling 514, 516, 518, according to at least one embodiment.”)
Claim(s) 4-7 and 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over US Pub no. US20220101494A1 Korani et al. (“Korani”) in view of US Pub No. US20250156684A1 Liu et al. (“Liu”) in further view of Xie, Jiahao, et al. "Masked frequency modeling for self-supervised visual pre-training." arXiv preprint arXiv:2206.07706 (2022). (“Xie”)
In regards to claim 4 and analogous claim 14,
Korani and Liu teach The processing system of claim 1,
Korani and Liu does not explicitly teach wherein the trained adapter comprises a frequency mask generator.
Xie teaches wherein the trained adapter comprises a frequency mask generator.
(Xie, “Figure 2: Overview of our MFM pre-training pipeline. We convert each input image into frequency domain via FFT and mask a portion of frequencies [wherein the trained adapter comprises a frequency mask generator; wherein the operations of Xie occurs in the adapter of Liu in the context of the applied references] on the frequency spectrum via a low-pass (top) or high-pass (bottom) filter.”
PNG
media_image2.png
229
924
media_image2.png
Greyscale
)
Xie is considered to be analogous to the claimed invention because they are in the same field of masked modeling in frequency-domain neural networks, particularly for visual models. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Korani and Liu to incorporate the teachings of Xie in order to provide masked frequency modeling as doing so allows the model to learn good representations (Xie, Abstract, “We present Masked Frequency Modeling (MFM), a unified frequency-domain based approach for self-supervised pre-training of visual models. Instead of randomly inserting mask tokens to the input embeddings in the spatial domain, in this paper, we shift the perspective to the frequency domain. Specifically, MFM first masks out a portion of frequency components of the input image and then predicts the missing frequencies on the frequency spectrum. Our key insight is that predicting masked components in the frequency domain is more ideal to reveal underlying image patterns rather than predicting masked patches in the spatial domain, due to the heavy spatial redundancy. Our findings suggest that with the right configuration of mask-and-predict strategy, both the structural information within high-frequency components and the low-level statistics among low-frequency counterparts are useful in learning good representations.”)
In regards to claim 5 and analogous claim 15,
Korani and Liu and Xie teach The processing system of claim 4,
Xie teaches wherein the one or more processors are configured to further execute the processor-executable instructions and cause the processing system to: generate a frequency mask, using the frequency mask generator, based on the first frequency tensor;
(Xie, “Figure 2: Overview of our MFM pre-training pipeline. We convert each input image into frequency domain via FFT and mask a portion of frequencies [generate a frequency mask, using the frequency mask generator, based on the first frequency tensor; wherein recall the first frequency tensor was processed from applying FFT on the first feature tensor in Korani] on the frequency spectrum via a low-pass (top) or high-pass (bottom) filter.”
PNG
media_image2.png
229
924
media_image2.png
Greyscale
)
Xie teaches and generate the first transformed frequency tensor based on the frequency mask.
(Xie, pg. 5 para. 2, “MFM decoder. The decoder accomplishes the frequency reconstruction task. It can be of arbitrary form as long as its input is compatible with the encoder’s output. Here, we simply adopt a lightweight linear layer as our decoder for efficiency, after which we perform FFT to convert each output image into the frequency domain for frequency reconstruction [generate the first transformed frequency tensor ie reconstructed frequency tensor based on the frequency mask].”)
In regards to claim 6 and analogous claim 16,
Korani and Liu and Xie teach The processing system of claim 5,
Liu teaches wherein, to generate the first transformed frequency tensor, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to: generate an intermediate tensor based on processing the first frequency tensor using a first portion of the first trained adapter;
(Liu, [0037], “The method comprises connecting, using a connector of the adapter, the adapter to the base model such that during an operation of the AI system at least some portion of data transformed by the base model is propagated from the base model to the adapter and back from the adapter to the base model [generate an intermediate tensor ie modified data based on processing the first frequency tensor (wherein Korani provides the first frequency tensor to the adapter of Liu) using a first portion of the first trained adapter].”)
Xie teaches generate the frequency mask based on processing the intermediate tensor using the frequency mask generator; apply the frequency mask to the intermediate tensor to generate a masked intermediate tensor;
(Xie, “Figure 2: Overview of our MFM pre-training pipeline. We convert each input image into frequency domain via FFT and mask a portion of frequencies [generate the frequency mask based on processing the intermediate tensor using the frequency mask generator; apply the frequency mask to the intermediate tensor to generate a masked intermediate tensor] on the frequency spectrum via a low-pass (top) or high-pass (bottom) filter.”
PNG
media_image2.png
229
924
media_image2.png
Greyscale
)
Xie teaches and generate the first transformed frequency tensor based on processing the masked intermediate tensor using a second portion of the first trained adapter.
(Xie, pg. 5 para. 2, “MFM decoder. The decoder accomplishes the frequency reconstruction task. It can be of arbitrary form as long as its input is compatible with the encoder’s output. Here, we simply adopt a lightweight linear layer as our decoder for efficiency, after which we perform FFT to convert each output image into the frequency domain for frequency reconstruction [and generate the first transformed frequency tensor based on processing the masked intermediate tensor using a second portion of the first trained adapter; wherein the operations of Xie occurs in the adapter of Liu in the context of the applied references].”)
In regards to claim 7 and analogous claim 17,
Korani and Liu and Xie teach The processing system of claim 6,
Liu teaches wherein the one or more processors are configured to further execute the processor-executable instructions and cause the processing system to apply an adapter weight to the intermediate tensor.
(Liu, “In particular, an AI architecture of the linear path(s) modifies the data linearly using one or multiple weight matrices [apply an adapter weight to the intermediate tensor] associated with the corresponding set of layers.”)
Claim(s) 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over US Pub no. US20220101494A1 Korani et al. (“Korani”) in view of US Pub No. US20250156684A1 Liu et al. (“Liu”) in further view of Xie, Jiahao, et al. "Masked frequency modeling for self-supervised visual pre-training." arXiv preprint arXiv:2206.07706 (2022). (“Xie”) in further view of J. Wang, F. Gao, J. Dong and Q. Du, "Adaptive DropBlock-Enhanced Generative Adversarial Networks for Hyperspectral Image Classification," in IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 6, pp. 5040-5053, June 2021, doi: 10.1109/TGRS.2020.3015843. (“Wang”)
In regards to claim 8 and analogous claim 18,
Korani and Liu and Xie teach The processing system of claim 4,
However, Korani and Liu and Xie do not explicitly teach wherein the frequency mask generator was trained to mask frequencies correlated with mode collapse in model output.
Wang teaches wherein the frequency mask generator was trained to mask frequencies correlated with mode collapse in model output
(Wang, pg. 2 col. 2 para. 4, “Another critical issue is mode collapse. The generator fools the discriminator by only producing data from the same data mode [44]. It leads to a weak generator that can generate samples within a narrow scope of the data space. Therefore, the generated samples are too similar for the model to learn the true data distribution, and the model can hardly learn the full data distribution. The model collapse can be considered as a consequence of overfitting to the feedback of the discriminator. In a disparate line of work, DropBlock [45] was designed in CNNs to alleviate overfitting. In DropBlock, features in a square mask from one feature map are dropped together during training. It is demonstrated that DropBlock can learn more spatially distributed representations. However, when dealing with objects with various shapes, the fixed square masks are inflexible. We argue that if irregularly shaped masks are taken into account [wherein the frequency mask generator was trained to mask frequencies correlated with mode collapse in model output; wherein irregularly shaped masks are considered for the frequency mask generator], the mode collapse problem can be alleviated.”)
Wang is considered to be analogous to the claimed invention because they are reasonably pertinent to the problem the inventors faced of mode collapse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Korani and Liu and Xie to incorporate the teachings of Wang in order to update the frequency mask modeling of Xie to consider adaptive shapes as doing so alleviates the mode collapse problem while boosting classification performance. (Wang, pg. 3 para. 2, “For the purpose of alleviating the mode collapse problem, we propose the AdapDrop for regularization. The AdapDrop generates masks with adaptive shapes, which can boost the classification performance.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
NPL: Zhang, Yifan, and Bryan Hooi. "Hipa: Enabling one-step text-to-image diffusion models via high-frequency-promoting adaptation." arXiv preprint arXiv:2311.18158 (2023).
NPL: He, Xuanhua, et al. "Frequency-adaptive pan-sharpening with mixture of experts." Proceedings of the AAAI conference on artificial intelligence. Vol. 38. No. 3. 2024.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASMINE THAI whose telephone number is (703)756-5904. The examiner can normally be reached M-F 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.T./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights