DETAILED ACTION
Claims 1-17 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
The invention, as taught in Claims 1-17, is directed to “mental steps” and “mathematical steps” without significantly more.
The claims recite:
• generating,…, a set of class probabilities for an unlabelled dataset based on a labelling function
• unlabelled dataset
• associated with the primary ML model
• unlabelled dataset and the associated set of class probabilities
• secondary ML model
Claim 1
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “1. A method for managing knowledge of a primary Machine Learning (ML) model, the method comprising:…” Therefore, it is a “method” (or “process”), which is a statutory category of invention. Therefore, the answer to the inquiry is: “YES.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 1 that recite abstract ideas?
YES. The following limitations in Claim 1 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• generating,…, a set of class probabilities for an unlabelled dataset based on a labelling function
• unlabelled dataset
• associated with the primary ML model
• unlabelled dataset and the associated set of class probabilities
• secondary ML model
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) A “computing system”
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model”
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique”
(1) A “computing system” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2106.04(d)(I) recites:
The courts have also identified limitations that did not integrate a judicial exception into a practical application:
• Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f);
• Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g); and
• Generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h).
This “computing system” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” is a broad term which is described at a high level.
Applicant’s Claim 1 merely teaches the embodiment where the claimed “ML model,” which is an “additional element”. The ML model is not used to calculate anything at all. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
This “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” is a broad term for merely applying a computer, which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); …
M.P.E.P. § 2106.05 (f)(2) recites in part:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. The court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner. 823 F.3d at 612, 118 USPQ2d at 1747. The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information). 823 F.3d at 612-13, 118 USPQ2d at 1747-48. In other words, the claims invoked the telephone unit and server merely as tools to execute the abstract idea. Thus, the court found that the additional elements did not add significantly more to the abstract idea because they were simply applying the abstract idea on a telephone network without any recitation of details of how to carry out the abstract idea.
This “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) A “computing system”
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model”
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique”
(1) A “computing system” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2106.05 (I)(A)(i-ii) recites:
Limitations that the courts have found not to be enough to qualify as “significantly more” when recited in a claim with a judicial exception include:
i. Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP § 2106.05(f));
ii. Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known to the industry, as discussed in Alice Corp., 573 U.S. at 225, 110 USPQ2d at 1984 (see MPEP § 2106.05(d));
Further, M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process.
Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
Applicant's Specification, [073] recites:
[073] The disclosed methods and systems may be implemented on a conventional or a general-purpose computer system, such as a personal computer (PC) or server computer. FIG. 6 is a block diagram that illustrates a system architecture 600 of a computer system 601 for managing knowledge of a primary ML model, in accordance with an exemplary embodiment of the present disclosure. Variations of computer system 601 may be used for implementing server 101 for determination of personality traits of agents in a contact center. Computer system 601 may include a central processing unit ("CPU" or "processor") 602. Processor 602 may include at least one data processor for executing program components for executing user-generated or system-generated requests. A user may include a person, a person using a device such as such as those included in this disclosure, or such a device itself. The processor may include specialized processing units such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc. The processor may include a microprocessor, such as AMD® ATHLON®, DURON® OR OPTERON®, ARM's application, embedded or secure processors, IBM® POWERPC®,INTEL® CORE® processor, ITANIUM® processor, XEON® processor, CELERON® processor or other line of processors, etc. The processor 602 may be implemented using mainframe, distributed processor, multi-core, parallel, grid, or other architectures. Some embodiments may utilize embedded technologies like application-specific integrated circuits (ASICs), digital signal processors (DSPs), Field Programmable Gate Arrays (FPGAs), etc.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” is a broad term which is described at a high level. Applicant's Specification, paragraph [002] recites:
[002] This disclosure relates generally to Machine Learning (ML) Operations, and more particularly to a method and system for knowledge transfer between different ML model architectures.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); …
Further, M.P.E.P. § 2106.05(d)(I)(2) recites in part:
2. A factual determination is required to support a conclusion that an additional element (or combination of additional elements) is well-understood, routine, conventional activity. Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018). However, this does not mean that a prior art search is necessary to resolve this inquiry. Instead, examiners should rely on what the courts have recognized, or those in the art would recognize, as elements that are well-understood, routine, conventional activity in the relevant field when making the required determination. For example, in many instances, the specification of the application may indicate that additional elements are well-known or conventional. See, e.g., Intellectual Ventures v. Symantec, 838 F.3d at 1317; 120 USPQ2d at 1359 ("The written description is particularly useful in determining what is well-known or conventional"); Internet Patents Corp. v. Active Network, Inc., 790 F.3d 1343, 1348, 115 USPQ2d 1414, 1418 (Fed. Cir. 2015) (relying on specification’s description of additional elements as "well-known", "common" and "conventional"); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 614, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (Specification described additional elements as "either performing basic computer functions such as sending and receiving data, or performing functions ‘known’ in the art.").
Further, Applicant's Specification, [021] recites:
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
Merely using the conventional computer to receive data is well known, understood, and conventional. Thus, it adds nothing significantly more to the judicial exception.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 1 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 2
Claim 2 recites:
2. The method of claim 1, further comprising:
exporting the trained secondary ML model from a first environment to a second environment based on a privacy protection technique.
Applicant’s Claim 2 merely teaches the routine use of a computer network.
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 2 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 3
Claim 3 recites:
3. The method of claim 2, wherein the first environment is a production environment.
Applicant’s Claim 3 merely teaches the routine use of a computer network.
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 3 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 4
Claim 4 recites:
4. The method of claim 2, wherein the second environment is one of a pre-production environment, or a test environment.
Applicant’s Claim 4 merely teaches the routine use of a computer network.
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 4 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 5
Claim 5 recites:
5. The method of claim 2, wherein the privacy protection technique is one of a Fully Homomorphic Encryption (FHE) technique, a Multi-Party Computation (MPC) technique, a Trusted Execution Environments (TEEs) technique, a secure enclave technique, a secure communication channel technique, and an obfuscation technique.
Applicant’s Claim 5 merely teaches a generic, well-understood, routine and conventional “privacy protection technique” (e.g., “a secure communication channel technique”.) It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 5 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 6
Claim 6 recites:
6. The method of claim 1, further comprising removing a pre-assigned label from an associated labelled dataset present in the first environment to generate the unlabelled dataset.
Applicant’s Claim 6 merely teaches the mental step of removing a label. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 6 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 7
Claim 7 recites:
7. The method of claim 1, wherein the knowledge transfer technique corresponds to a knowledge distillation technique.
Applicant’s Claim 7 merely teaches a well-understood, routine and conventional process. Applicant's Specification, paragraph [024] recites:
[024] The knowledge distillation technique has been traditionally used to shrink large ML models into smaller efficient ML models so that it may be run on mobile or other edge devices. However, the knowledge distillation technique discussed herein in one or more embodiments of the present disclosure may be used in Machine Learning Operations (MLOps) to transfer the knowledge of the primary ML model 102A from existing architecture (e.g., the first ML model architecture) to a new architecture (e.g., the second ML model architecture) of the secondary ML model 103A. The term MLOps or is the practice of applying DevOps (Development operations) principles to machine learning workflows, enabling the efficient development, deployment, and maintenance of machine learning models at scale. In the process of knowledge distillation, the ML model is made more resilient to data drift issues and make algorithmic changes in the knowledge distillation technique in such as way so that it may cater to a more complex and/or bigger and new ML model architecture.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 7 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 8
Claim 8 recites:
8. The method of claim 1, wherein the labelling function corresponds to a soft max function.
Applicant’s Claim 8 merely teaches a mathematical operation used to transform a vector of raw numerical scores, called logits, into a probability distribution. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 8 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 9
Step 1 inquiry: Does this claim fall within a statutory category?
.
The preamble of the claim recites “9. A system for managing knowledge of a primary Machine Learning (ML) model, the system comprising:…” Therefore, it is a “system” (or “apparatus”), which is a statutory category of invention. Therefore, the answer to the inquiry is: “YES.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 9 that recite abstract ideas?
YES. The following limitations in Claim 9 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• generate,…, a set of class probabilities for an unlabelled dataset based on a labelling function
• unlabelled dataset
• associated with the primary ML model
• unlabelled dataset and the associated set of class probabilities
• secondary ML model
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) A “processing circuitry”
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model”
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique”
(4) A “memory”
(1) A “processing circuitry” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2106.04(d)(I) recites:
The courts have also identified limitations that did not integrate a judicial exception into a practical application:
• Merely reciting the words “apply it” (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f);
• Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g); and
• Generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h).
This “processing circuitry” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” is a broad term which is described at a high level.
Applicant’s Claim 9 merely teaches the embodiment where the claimed “ML model,” which is an “additional element”. The ML model is not used to calculate anything at all. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
This “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” is a broad term for merely applying a computer, which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); …
M.P.E.P. § 2106.05 (f)(2) recites in part:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. The court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner. 823 F.3d at 612, 118 USPQ2d at 1747. The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information). 823 F.3d at 612-13, 118 USPQ2d at 1747-48. In other words, the claims invoked the telephone unit and server merely as tools to execute the abstract idea. Thus, the court found that the additional elements did not add significantly more to the abstract idea because they were simply applying the abstract idea on a telephone network without any recitation of details of how to carry out the abstract idea.
This “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
(4) A “memory” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
This “memory” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) A “processing circuitry”
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model”
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique”
(4) A “memory”
(1) A “processing circuitry” is a broad term which is described at a high level and includes general purpose computers. M.P.E.P. § 2106.05 (I)(A)(i-ii) recites:
Limitations that the courts have found not to be enough to qualify as “significantly more” when recited in a claim with a judicial exception include:
i. Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP § 2106.05(f));
ii. Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, e.g., a claim to an abstract idea requiring no more than a generic computer to perform generic computer functions that are well-understood, routine and conventional activities previously known to the industry, as discussed in Alice Corp., 573 U.S. at 225, 110 USPQ2d at 1984 (see MPEP § 2106.05(d));
Further, M.P.E.P. § 2016.05(f) recites:
2106.05(f) Mere Instructions To Apply An Exception [R-10.2019]
Another consideration when determining whether a claim integrates a judicial exception into a practical application in Step 2A Prong Two or recites significantly more than a judicial exception in Step 2B is whether the additional elements amount to more than a recitation of the words “apply it” (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer. As explained by the Supreme Court, in order to make a claim directed to a judicial exception patent-eligible, the additional element or combination of elements must do “‘more than simply stat[e] the [judicial exception] while adding the words ‘apply it’”. Alice Corp. v. CLS Bank, 573 U.S. 208, 221, 110 USPQ2d 1976, 1982-83 (2014) (quoting Mayo Collaborative Servs. V. Prometheus Labs., Inc., 566 U.S. 66, 72, 101 USPQ2d 1961, 1965). Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984 (warning against a § 101 analysis that turns on “the draftsman’s art”).
Further, M.P.E.P. § 2106.05(f)(2) recites:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process.
Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
Applicant's Specification, [073] recites:
[073] The disclosed methods and systems may be implemented on a conventional or a general-purpose computer system, such as a personal computer (PC) or server computer. FIG. 6 is a block diagram that illustrates a system architecture 600 of a computer system 601 for managing knowledge of a primary ML model, in accordance with an exemplary embodiment of the present disclosure. Variations of computer system 601 may be used for implementing server 101 for determination of personality traits of agents in a contact center. Computer system 601 may include a central processing unit ("CPU" or "processor") 602. Processor 602 may include at least one data processor for executing program components for executing user-generated or system-generated requests. A user may include a person, a person using a device such as such as those included in this disclosure, or such a device itself. The processor may include specialized processing units such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc. The processor may include a microprocessor, such as AMD® ATHLON®, DURON® OR OPTERON®, ARM's application, embedded or secure processors, IBM® POWERPC®,INTEL® CORE® processor, ITANIUM® processor, XEON® processor, CELERON® processor or other line of processors, etc. The processor 602 may be implemented using mainframe, distributed processor, multi-core, parallel, grid, or other architectures. Some embodiments may utilize embedded technologies like application-specific integrated circuits (ASICs), digital signal processors (DSPs), Field Programmable Gate Arrays (FPGAs), etc.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
(2) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” is a broad term which is described at a high level. Applicant's Specification, paragraph [002] recites:
[002] This disclosure relates generally to Machine Learning (ML) Operations, and more particularly to a method and system for knowledge transfer between different ML model architectures.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
(3) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); …
Further, M.P.E.P. § 2106.05(d)(I)(2) recites in part:
2. A factual determination is required to support a conclusion that an additional element (or combination of additional elements) is well-understood, routine, conventional activity. Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018). However, this does not mean that a prior art search is necessary to resolve this inquiry. Instead, examiners should rely on what the courts have recognized, or those in the art would recognize, as elements that are well-understood, routine, conventional activity in the relevant field when making the required determination. For example, in many instances, the specification of the application may indicate that additional elements are well-known or conventional. See, e.g., Intellectual Ventures v. Symantec, 838 F.3d at 1317; 120 USPQ2d at 1359 ("The written description is particularly useful in determining what is well-known or conventional"); Internet Patents Corp. v. Active Network, Inc., 790 F.3d 1343, 1348, 115 USPQ2d 1414, 1418 (Fed. Cir. 2015) (relying on specification’s description of additional elements as "well-known", "common" and "conventional"); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 614, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (Specification described additional elements as "either performing basic computer functions such as sending and receiving data, or performing functions ‘known’ in the art.").
Applicant's Specification, [021] recites:
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
Merely using the conventional computer to receive data is well known, understood, and conventional. Thus, it adds nothing significantly more to the judicial exception.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
(4) A “memory” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
***
iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
Further, Applicant's Specification, paragraph [032] recites:
[032] FIG. 2 is a diagram that illustrates a process for managing knowledge of a primary ML model, in accordance with an exemplary embodiment of the present disclosure. FIG. 2 is explained in conjunction with elements from FIG. 1. The computing system 101 may include a processing circuitry 201 and a memory 202 communicatively coupled to the processing circuitry 201 via a communication bus 203. The memory 202 may be a non-volatile memory or a volatile memory. Examples of non-volatile memory may include, but are not limited to, a flash memory, a Read Only Memory (ROM), a Programmable ROM (PROM), Erasable PROM (EPROM), and Electrically EPROM (EEPROM) memory. Examples of volatile memory may include, but are not limited to, Dynamic Random Access Memory (DRAM), and Static Random-Access Memory (SRAM).
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 9 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 10
Claim 10 recites:
10. The system of claim 9, wherein the processor instructions, on execution, further cause the processing circuitry to export the trained secondary ML model from a first environment to a second environment based on a privacy protection technique.
Applicant’s Claim 10 merely teaches the routine use of a computer network.
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 10 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 11
Claim 1 recites:
11. The system of claim 10, wherein the first environment is a production environment.
Applicant’s Claim 11 merely teaches the routine use of a computer network.
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 11 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 12
Claim 12 recites:
12. The system of claim 10, wherein the second environment is one of a pre-production environment, or a test environment.
Applicant’s Claim 12 merely teaches the routine use of a computer network.
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 12 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 13
Claim 13 recites:
13. The system of claim 10, wherein the privacy protection technique is one of a Fully Homomorphic Encryption (FHE) technique, a Multi-Party Computation (MPC) technique, a Trusted Execution Environments (TEEs) technique, a secure enclave technique, a secure communication channel technique, and an obfuscation technique.
Applicant’s Claim 13 merely teaches a generic, well-understood, routine and conventional “privacy protection technique” (e.g., “a secure communication channel technique”.) It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 13 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 14
Claim 14 recites:
14. The system of claim 1, wherein the processor instructions, on execution, further cause the processing circuitry to remove a pre-assigned label from an associated labelled dataset present in the first environment to generate the unlabelled dataset.
Applicant’s Claim 14 merely teaches the mental step of removing a label. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 14 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 15
Claim 15 recites:
15. The system of claim 9, wherein the knowledge transfer technique corresponds to a knowledge distillation technique.
Applicant’s Claim 15 merely teaches a well-understood, routine and conventional process. Applicant's Specification, paragraph [024] recites:
[024] The knowledge distillation technique has been traditionally used to shrink large ML models into smaller efficient ML models so that it may be run on mobile or other edge devices. However, the knowledge distillation technique discussed herein in one or more embodiments of the present disclosure may be used in Machine Learning Operations (MLOps) to transfer the knowledge of the primary ML model 102A from existing architecture (e.g., the first ML model architecture) to a new architecture (e.g., the second ML model architecture) of the secondary ML model 103A. The term MLOps or is the practice of applying DevOps (Development operations) principles to machine learning workflows, enabling the efficient development, deployment, and maintenance of machine learning models at scale. In the process of knowledge distillation, the ML model is made more resilient to data drift issues and make algorithmic changes in the knowledge distillation technique in such as way so that it may cater to a more complex and/or bigger and new ML model architecture.
Claim 15 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 16
Claim 16 recites:
16. The system of claim 9, wherein the labelling function corresponds to a soft max function.
Applicant’s Claim 16 merely teaches a mathematical operation used to transform a vector of raw numerical scores, called logits, into a probability distribution. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
Claim 16 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Claim 17
Step 1 inquiry: Does this claim fall within a statutory category?
The preamble of the claim recites “17. A non-transitory computer-readable medium storing computer-executable instructions for managing knowledge of a primary Machine Learning (ML) model, the computer-executable instructions configured for…” Therefore, it is a “non-transitory computer-readable medium” (or “product of manufacture”), which is a statutory category of invention. Therefore, the answer to the inquiry is: “YES.”
Step 2A (Prong One) inquiry:
Are there limitations in Claim 17 that recite abstract ideas?
YES. The following limitations in Claim 17 recite abstract ideas that fall within at least one of the groupings of abstract ideas enumerated in the 2019 PEG. Specifically, they are “mental steps” and “mathematical steps”:
• generating,…, a set of class probabilities for an unlabelled dataset based on a labelling function
• unlabelled dataset
• associated with the primary ML model
• unlabelled dataset and the associated set of class probabilities
• secondary ML model
Step 2A (Prong Two) inquiry:
Are there additional elements or a combination of elements in the claim that apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception?
Applicant’s claims contain the following “additional elements”:
(1) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model”
(2) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique”
(1) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” is a broad term which is described at a high level.
Applicant’s Claim 17 merely teaches the embodiment where the claimed “ML model,” which is an “additional element”. The ML model is not used to calculate anything at all. It does not integrate the abstract idea to a practical application, nor is it anything significantly more than the abstract idea. (See, 2106.05(a)(II).)
This “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
(2) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” is a broad term for merely applying a computer, which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); …
M.P.E.P. § 2106.05 (f)(2) recites in part:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process. Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. The court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner. 823 F.3d at 612, 118 USPQ2d at 1747. The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information). 823 F.3d at 612-13, 118 USPQ2d at 1747-48. In other words, the claims invoked the telephone unit and server merely as tools to execute the abstract idea. Thus, the court found that the additional elements did not add significantly more to the abstract idea because they were simply applying the abstract idea on a telephone network without any recitation of details of how to carry out the abstract idea.
This “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” limitation does not integrate the additional element into a practical application and represents “insignificant extra-solution activity”. (See, M.P.E.P. § 2106.05(I)(A)).
The answer to the inquiry is “NO”, no additional elements integrate the claimed abstract idea into a practical application.
Step 2B inquiry:
Does the claim provide an inventive concept, i.e., does the claim recite additional element(s) or a combination of elements that amount to significantly more than the judicial exception in the claim?
Applicant’s claims contain the following “additional elements”:
(1) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model”
(2) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique”
(1) A “primary ML model employs a first ML model architecture”/ “secondary ML model”/ “first ML model” is a broad term which is described at a high level. Applicant's Specification, paragraph [002] recites:
[002] This disclosure relates generally to Machine Learning (ML) Operations, and more particularly to a method and system for knowledge transfer between different ML model architectures.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
(2) A “transferring, by the computing system, the unlabelled (sic.) dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique” is a broad term which is described at a high level. M.P.E.P. § 2106.05(d)(II) recites:
The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity.
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); …
Further, M.P.E.P. § 2106.05(d)(I)(2) recites in part:
2. A factual determination is required to support a conclusion that an additional element (or combination of additional elements) is well-understood, routine, conventional activity. Berkheimer v. HP, Inc., 881 F.3d 1360, 1368, 125 USPQ2d 1649, 1654 (Fed. Cir. 2018). However, this does not mean that a prior art search is necessary to resolve this inquiry. Instead, examiners should rely on what the courts have recognized, or those in the art would recognize, as elements that are well-understood, routine, conventional activity in the relevant field when making the required determination. For example, in many instances, the specification of the application may indicate that additional elements are well-known or conventional. See, e.g., Intellectual Ventures v. Symantec, 838 F.3d at 1317; 120 USPQ2d at 1359 ("The written description is particularly useful in determining what is well-known or conventional"); Internet Patents Corp. v. Active Network, Inc., 790 F.3d 1343, 1348, 115 USPQ2d 1414, 1418 (Fed. Cir. 2015) (relying on specification’s description of additional elements as "well-known", "common" and "conventional"); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 614, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (Specification described additional elements as "either performing basic computer functions such as sending and receiving data, or performing functions ‘known’ in the art.").
Applicant's Specification, [021] recites:
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
[022] The communication network 104 may facilitate the computing system 101 to perform one or more operations in order to manage knowledge of the primary ML model 102A. In particular, the computing system 101 may communicate with the first computing device 102, and the second computing device 103 to transfer knowledge of the primary ML model 102A having a first ML model architecture 102B to the secondary ML model 103A having a second ML model architecture 103B based on a knowledge transfer technique (for example, a knowledge distillation technique). It should be noted that the first ML model architecture is different from the second ML model architecture. As will be appreciated, in some embodiments the primary ML model 102A, and the secondary ML model 103A may be part of the computing system 101.
Merely using the conventional computer to receive data is well known, understood, and conventional. Thus, it adds nothing significantly more to the judicial exception.
Therefore, the claim as a whole does not amount to significantly more than the exception itself (i.e., there is no inventive concept in the claim). (See, M.P.E.P. § 2106.05(II)).
Therefore, the answer to the inquiry is “NO”, no additional elements provide an inventive concept that is significantly more than the claimed abstract ideas the claimed abstract idea into a practical application.
Claim 17 is, therefore, NOT ELIGIBLE subject matter under 35 U.S.C. § 101.
Reasons Claims not Rejected Under Art
Claims 1-17 are not rejected under art since when reading the claims in light of the specification, as per MPEP § 2111.01, none of the references of record, whether taken alone or in combination, discloses or suggests the combination of limitations specified in independent Claim 1. Specifically:
Claim 1’s "...transferring, by the computing system, the unlabelled dataset and the associated set of class probabilities for training a secondary ML model based on a knowledge transfer technique..."
Further, none of the references of record, whether taken alone or in combination, discloses or suggests the combination of limitations specified in independent Claim 9. Specifically:
Claim 9’s "...transfer, by the computing system, the unlabelled dataset and the associated set of class probabilities for training a secondary ML model based on a knowledge transfer technique..."
Further, none of the references of record, whether taken alone or in combination, discloses or suggests the combination of limitations specified in independent Claim 17. Specifically:
Claim 17’s "...transferring, by the computing system, the unlabelled dataset and the associated set of class probabilities for training a secondary ML model based on a knowledge transfer technique..."
Response to Arguments
Applicant's arguments filed 24 APR 2026 have been fully considered but they are not persuasive. Specifically, Applicant argues:
Argument 1
Likewise, the present invention is necessarily rooted in computer technology to overcome a problem specifically arising in the realm of machine learning model management (See Specification at 1 [0026]). Specifically, the claimed knowledge transfer technique is specific to and rooted in ML technology and reflects a specific implementation of a technological solution, not an abstract idea. The claimed process involves generating class probabilities for an unlabelled dataset using a labelling function applied by a trained ML model, and then using those probabilities together with the unlabelled dataset to train an entirely different ML model having a different architecture via a knowledge transfer technique. Applicant respectfully submits that these operations involving training models and performing cross-architecture knowledge transfer fundamentally cannot be practically performed in the human mind, even with the aid of pen and paper.
Accordingly, the features recited in amended claim 1 cannot be considered as mental steps. For example, at least the amended features of claim 1 reciting "generating, by a computing system, a set of class probabilities for an unlabelled dataset based on a labelling function, wherein the unlabelled dataset is associated with the primary ML model, and wherein the primary ML model employs a first ML model architecture; and transferring, by the computing system, the unlabelled dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique, for training the secondary ML model, wherein the secondary ML model employs a second ML model architecture, and wherein the first ML model architecture is different from the second ML model architecture," cannot be practically performed in the human mind.
There actually are mental steps in Applicant's argument. Two examples of this are:
• “generating class probabilities” for an unlabelled dataset (i.e., mental estimation or mathematical calculation)
• using an unspecified “labelling function” (i.e., mentally labelling or mathematically using a “labelling function”)
Applicant's argument is unpersuasive.
The rejections stand.
Argument 2
Applicant humbly submits that the Examiner's alternative characterization of the claims as "mathematical steps" is incorrect. Office Action at 2. While certain claim elements involve mathematical operations (e.g., the labelling function recited in the independent claims, further specified as a softmax function in dependent Claims 8 and 16), the claims do not recite a bare mathematical formula or equation. Rather, any mathematical operations are applied in a specific technological context within a multi-step ML knowledge transfer pipeline to achieve a concrete, technological result.
The Federal Circuit has consistently held that claims involving mathematical operations applied within specific technical processes are not directed to abstract "mathematical concepts." In McRO, Inc. V. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314 (Fed. Cir. 2016), the court found that claims directed to a specific process using specific rules to achieve a technological improvement - automated lip synchronization - were not abstract, even though the process involved the application of mathematical rules. Similarly, in Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36 (Fed. Cir. 2016), the court held that claims directed to a specific data structure improvement (a self-referential table) were not abstract because they were "directed to a specific improvement to the way computers operate."
Here, the mathematical operations recited in the claims (generating class probabilities based on a labelling function) are not claimed in the abstract. They are embedded within a specific, technology-rooted process of managing ML model knowledge by: (a) applying a labelling function within a trained primary ML model to generate class probabilities; and (b) transferring those probabilities together with the unlabelled dataset from the primary ML model to a secondary ML model to train the secondary ML model of a different architecture via a knowledge transfer technique. The result is a concrete, technological improvement i.e., a trained secondary ML model in a new architecture that has captured the knowledge of the primary production model and not merely an abstract mathematical relationship.
The 2019 PEG and the USPTO's Subject Matter Eligibility Examples further confirm this analysis. For instance, in Example 39 (Method for Training a Neural Network for Facial Detection), the USPTO found claims directed to a specific method of training a neural network to be patent-eligible because they involved specific technical operations that produced concrete results, even though mathematical operations were involved in the training process. The present claims are analogous: they recite a specific multi-step process for cross-architecture knowledge transfer for training a secondary ML model that achieves a concrete technical result, and any mathematical operations are embedded within and integral to that process.
Therefore, amended claim 1 does not recite a judicial exception. Amended independent claims 9 and 17 recite subject matter similar to that of claim 1 and is also patent eligible at Prong One of revised Step 2A, see MPEP § 2106.04(a), for this additional reason, as are dependent claims 2-8 and 10-16.
There actually are mental steps in Applicant's argument. Two examples of this are:
• “generating class probabilities” for an unlabelled dataset (i.e., mental estimation or mathematical calculation)
• using an unspecified “labelling function” (i.e., mentally labelling or mathematically using a “labelling function”)
Regarding amended independent claims 9 and 17, similar arguments for similar claims are similarly unpersuasive.
Regarding dependent claims 2-8 and 10-16, there is no eligible matter in the independent claims to incorporate by reference to the dependent claims. Therefore, the defects in those claims are not cured by the amendments.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 3
Applicant submits that these features, taken alone or as an ordered combination, integrate the alleged judicial exception into a practical application of knowledge transfer between different ML model architectures (See Specification at 1 [0002]). The claims include specific technical embodiments that address identified technical deficiencies in existing ML operations. The disclosure identifies the technical problem: "it is difficult to replicate the performance of production ML models due to the inability to move sensitive production data or production ML model to the pre-production environment. This poses challenges for testing, as production data or production ML model often contains sensitive information that cannot be shared or moved outside of the production environment due to regulatory, privacy, or security reasons." Specification at II [0005]-[0006]. The disclosure further explains that "the production ML models constantly learn and adapt to new data, which means that even when the ML model architecture is the same in both pre-production environments and production environments, the performance of these models is fundamentally different." Specification at I [0004].
The present application addresses these deficiencies by providing a specific technical solution: using the primary production ML model as a "teacher model" to generate class probabilities (soft labels) for an unlabelled production dataset, thereby capturing "data drift and dark knowledge hidden in the primary production ML model", and then transferring the unlabelled dataset and associated class probabilities to train a secondary ML model ("student model") in a new, different architecture via a knowledge transfer technique. Specification at 11 [0024], [0059]-[0060]. As the Specification explains, the "knowledge distillation technique discussed herein may be used in Machine Learning Operations (MLOps) to transfer the knowledge of the primary ML model 102A from existing architecture (e.g., the first ML model architecture) to a new architecture (e.g., the second ML model architecture) of the secondary ML model 103A. In the process of knowledge distillation, the ML model is made more resilient to data drift issues." Specification at 1 [0024].
This is a specific, technology-rooted solution to a technology-specific problem, not a generic application of mathematical concepts on a computer. The claims are analogous to the claims held patent-eligible in McRO, Inc. V. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314- 15 (Fed. Cir. 2016), where the Federal Circuit found that a specific process using specific rules to automate a previously manual animation process was not abstract because the claims were "limited to a specific process for automatically animating lip synchronization using specific, limited mathematical rules." Similarly, claim 1 is limited to a specific process for managing ML model knowledge across different architectures using specific technical operations such as a labelling function to generate class probabilities, an unlabelled dataset associated with a specific ML model, and a knowledge transfer technique applied across architectures that are expressly required to be different.
As shown before, there are abstract ideas in the independent claims. Two examples of this are:
• “generating class probabilities” for an unlabelled dataset (i.e., mental estimation or mathematical calculation)
• using an unspecified “labelling function” (i.e., mentally labelling or mathematically using a “labelling function”)
To see more, see rejections, above.
Further, Applicant has not identified the specific technology being improved. There are mentions of machine learning systems, but those are mentioned generically and abstractly. Applicant's Specification, paragraph [004], where it recites:
[004] These architectural differences may include, but are not limited to, changes to the number and type of layers, the number of neurons, their interconnection, the use of different activation functions, or the learning rate, as well as the introduction of new techniques such as attention mechanisms. Furthermore, there is a growing trend of designing machine learning systems to continuously learn with human feedback and new data in production. Over time, the production ML models constantly learn and adapt to new data, which means that even when the ML model architecture is the same in both pre-production environments (e.g., a Quality Assurance (QA) environment, DEV environment, or a test environment), and production environments, the performance of these models is fundamentally different.
As one can see, Applicant claims the general idea of some machine learning system rather than the improvement of a particular architecture.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 4
The claims are also analogous to the claims held patent-eligible in Enfish, LLC V. Microsoft Corp., 822 F.3d 1327, 1335-36 (Fed. Cir. 2016), where the court held that claims directed to a specific improvement in how data is stored in a computer (a self-referential table) were "directed to a specific improvement to the way computers operate." Here, the claims are directed to a specific improvement in how ML model knowledge is managed, preserved, and transferred across architectural changes which is an improvement to the functioning of the ML technology itself.
The Examiner's reliance on TLI Communications LLC V. AV Auto, LLC, 823 F.3d 607 (Fed. Cir. 2016), and Affinity Labs v. DirecTV, 838 F.3d 1253 (Fed. Cir. 2016), is misplaced. Those cases involved claims that merely used computers as tools to perform pre-existing, non-technical processes. In TLI Communications, the claims described steps of "recording, administration and archiving of digital images" using a telephone unit and server in their ordinary capacity. In contrast, the present claims are fundamentally rooted in ML technology. The concepts of knowledge distillation, cross-architecture teacher-student model training, class probability generation via labelling functions, and knowledge transfer have no analog outside of computer science and machine learning.
The machine learning system was being regarded as an “additional element”…like a computer. If that additional element is simply being used in its “ordinary capacity,” it adds nothing to the abstract ideas in the claim.
The machine learning systems are mentioned in the claims generically and abstractly. Applicant's Specification, paragraph [004], where it recites:
[004] These architectural differences may include, but are not limited to, changes to the number and type of layers, the number of neurons, their interconnection, the use of different activation functions, or the learning rate, as well as the introduction of new techniques such as attention mechanisms. Furthermore, there is a growing trend of designing machine learning systems to continuously learn with human feedback and new data in production. Over time, the production ML models constantly learn and adapt to new data, which means that even when the ML model architecture is the same in both pre-production environments (e.g., a Quality Assurance (QA) environment, DEV environment, or a test environment), and production environments, the performance of these models is fundamentally different.
Further, they are being used in their ordinary capacity. M.P.E.P. § 2106.05 (f)(2) recites in part:
(2) Whether the claim invokes computers or other machinery merely as a tool to perform an existing process.
Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Similarly, “claiming the improved speed or efficiency inherent with applying the abstract idea on a computer” does not integrate a judicial exception into a practical application or provide an inventive concept. Intellectual Ventures I LLC v. Capital One Bank (USA), 792 F.3d 1363, 1367, 115 USPQ2d 1636, 1639 (Fed. Cir. 2015). In contrast, a claim that purports to improve computer capabilities or to improve an existing technology may integrate a judicial exception into a practical application or provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). See MPEP §§ 2106.04(d)(1) and 2106.05(a) for a discussion of improvements to the functioning of a computer or to another technology or technical field.
TLI Communications provides an example of a claim invoking computers and other machinery merely as a tool to perform an existing process. The court stated that the claims describe steps of recording, administration and archiving of digital images, and found them to be directed to the abstract idea of classifying and storing digital images in an organized manner. 823 F.3d at 612, 118 USPQ2d at 1747. The court then turned to the additional elements of performing these functions using a telephone unit and a server and noted that these elements were being used in their ordinary capacity (i.e., the telephone unit is used to make calls and operate as a digital camera including compressing images and transmitting those images, and the server simply receives data, extracts classification information from the received data, and stores the digital images based on the extracted information). 823 F.3d at 612-13, 118 USPQ2d at 1747-48. In other words, the claims invoked the telephone unit and server merely as tools to execute the abstract idea. Thus, the court found that the additional elements did not add significantly more to the abstract idea because they were simply applying the abstract idea on a telephone network without any recitation of details of how to carry out the abstract idea.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 5
Furthermore, as the Federal Circuit noted in DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258 (Fed. Cir. 2014), "Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result - a result that overrides the routine and conventional sequence of events ordinarily triggered." Similarly, the present claims override the routine and conventional approach to ML model architecture transitions. Conventionally, when ML model architecture changes with a new release, "exporting the model weights from the production environment to the QA environment may not work, as the QA environment may not be able to use the model weights of the existing architecture with the new architecture." Specification at T [0067]. The claimed invention provides a specific technical solution that overcomes this limitation by using knowledge distillation to create a bridge between architectures that are fundamentally different, preserving the production model's accumulated knowledge in the process.
Applicant argues limitations that are not claimed.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 6
Applicant further submits that the Examiner's Prong Two analysis is deficient because it evaluates each additional element in isolation rather than considering the claim as an ordered combination. The Examiner separately dismisses the "computing system," the ML models, and the "knowledge transfer technique" as individually constituting "insignificant extra-solution activity." However, the 2019 PEG and MPEP expressly require evaluation of "all the claim limitations and how those limitations interact and impact each other." MPEP § 2106.04(d)(III). When properly evaluated as an ordered combination, the claim limitations describe a specific, multi-step technical pipeline in which: (i) a primary ML model with a first architecture generates class probabilities for an unlabelled dataset; and (ii) those probabilities and the dataset are transferred from a primary ML model to a secondary ML model for training the secondary ML model with a different architecture. This ordered combination of limitations reflects a meaningful technological solution, not a drafting effort designed to monopolize a mathematical concept.
Accordingly, the claims pass the practical application test. The claims as a whole are specifically tailored to improve ML model management technology. As opposed to merely using a computer as a generic tool to perform an abstract idea, the claims are directed to a practical application that is expressly limited to managing knowledge of a primary ML model across different architectures using a specific technical process.
Therefore, claim 1 as a whole integrates the alleged judicial exception into a practical application and is not directed to a judicial exception. Independent claims 9 and 17 recite subject matter corresponding to that of claim 1 and are also patent eligible at Prong Two of Step 2A for the same reasons, as are dependent claims 2-8 and 10-16.
Examiner, as required, looked for “additional elements” that could make the identified abstract ideas eligible, when added to them…as a whole. Applicant is correct that the claimed "computing system," the ML models, and the "knowledge transfer technique" were all insufficient, alone and in combination, to make the claims eligible. The claimed "computing system" and the claimed "knowledge transfer technique" amount to a generic computer network. The unspecified “ML models” on the computers are also generic and generally practiced on computers.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 7
As noted above, the claims do not recite and are not directed to an abstract idea, and, thus, the "significantly more" inquiry under Step 2B does not even apply. Nevertheless, even if the claims are directed to an abstract idea, to which the Applicant does not concede, the Office Action's analysis is flawed at least because the Office fails to consider the specific requirements in the claimed combinations. The claims amount to "significantly more" than any abstract idea because the claims recite "additional elements" that are not well-understood, routine, or conventional and because the claims recite technological solutions to technological problems.
The subject matter of independent claim 1 is not routine, conventional, or well understood in the art. The subject matter of claim 1 provides a method for managing knowledge of a primary ML model by generating class probabilities for an unlabelled dataset and transferring those probabilities from the primary ML model to a secondary ML model for training the secondary ML model of a different architecture via a knowledge transfer technique.
For example, paragraphs [0082] and [0083] of the as-filed specification states -
"As will be appreciated by those skilled in the art, the techniques described in the various embodiments discussed above are not routine, or conventional, or well understood in the art. The techniques discussed above provide for managing knowledge of the primary ML model. The techniques use of method of knowledge distillation to use the existing production ML model (e.g., the primary ML model) along with the unlabelled production dataset as the 'Production Teacher Model' in existing architecture to create 'Production Student Model' (e.g., the secondary ML model) in a new architecture. The techniques further export the created 'Production Student Model' to the pre-production environment in a privacy protecting manner In light of the above mentioned advantages and the technical advancements provided by the disclosed method and system, the claimed steps as discussed above are not routine, conventional, or well understood in the art, as the claimed steps enable the following solutions to the existing problems in conventional technologies. Further, the claimed steps clearly bring an improvement in the functioning of the device itself as the claimed steps provide a technical solution to a technical problem." (Emphasis added).
Because the claims recite technological solutions to technological problems, the claims are not directed to an abstract idea under Step 2A of the two-part test for subject matter eligibility, and in any case amounts to significantly more than an abstract idea under Step 2B of the two-part test.
The Examiner has not provided any prior art reference, declaration, or other evidentiary basis establishing that the specific combination of claim elements including cross-architecture knowledge transfer using soft labels derived from a labelling function applied by a primary ML model to an unlabelled dataset is well-understood, routine, and conventional. Even if individual elements of the claims were considered conventional, which Applicant does not concede, the ordered combination of elements provides significantly more than any alleged abstract idea. As the Supreme Court recognized in Alice Corp. V. CLS Bank International, 573 U.S. 208, 225 (2014), Step 2B requires consideration of the elements "both individually and 'as an ordered combination." The specific ordered combination here (i) generating class probabilities from an unlabelled dataset associated with a primary ML model using a labelling function, and (ii) transferring those probabilities with the dataset for training a secondary ML model of a different architecture via a knowledge transfer technique is not a conventional combination, as the Examiner's own prior art analysis confirms.
Applicant's “cross-architecture knowledge transfer” is generic network communication, as shown by Applicant's Specification, paragraph [021], where it recites:
[021] The environment 100 may include a computing system 101, a first computing device 102, and a secondary computing device 103. The first computing device 102 may include a primary ML model 102A. The second computing device 103 may include a secondary ML model 103A. The computing system 101, the first computing device 102, and the secondary computing device 103 are configured to communicate with each other via a communication network 104. Examples of the communication network 104 may include, but are not limited to, a wireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof.
Further, the machine learning systems, as discussed in the claims are also generic, as taught by Applicant's Specification, paragraph [004], where it recites:
[004] These architectural differences may include, but are not limited to, changes to the number and type of layers, the number of neurons, their interconnection, the use of different activation functions, or the learning rate, as well as the introduction of new techniques such as attention mechanisms. Furthermore, there is a growing trend of designing machine learning systems to continuously learn with human feedback and new data in production. Over time, the production ML models constantly learn and adapt to new data, which means that even when the ML model architecture is the same in both pre-production environments (e.g., a Quality Assurance (QA) environment, DEV environment, or a test environment), and production environments, the performance of these models is fundamentally different.
If the machine learning systems taught in the claims are generic, the transferred “knowledge” used to train those generic systems are also generic…in addition to being abstract mathematical/mental steps and data.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 8
Independent Claim 1 includes additional elements that are sufficient to amount to significantly more than the alleged judicial exception. Specifically, Claim 1 recites "generating, by a computing system, a set of class probabilities for an unlabelled dataset based on a labelling function, wherein the unlabelled dataset is associated with the primary ML model, and wherein the primary ML model employs a first ML model architecture; and transferring, by the computing system, the unlabelled dataset and the associated set of class probabilities from the primary ML model to a secondary ML model based on a knowledge transfer technique, for training the secondary ML model, wherein the secondary ML model employs a second ML model architecture, and wherein the first ML model architecture is different from the second ML model architecture." This subject matter recites a combination of connected technical elements that improve ML model management across architectural changes - the cross-architecture knowledge transfer pipeline that preserves dark knowledge and data drift awareness - which is significantly more than merely reciting mathematical calculations or mental observations.
This is a repetition of the arguments, above. Please see the responses to the arguments above.
Applicant's argument is unpersuasive.
The rejections stand.
Argument 9
Independent claims 9 and 17 have been amended to make them consistent with claim 1. Therefore, the above analysis applies to claims 9 and 17. Further, the dependent claims 1-8 and 10-16 are also allowable at least based on their dependencies on claims 1 and 9. Therefore, the Applicant requests the Examiner to withdraw all rejections of claims under 35 U.S.C § 101, against claims 1-17.
Regarding amended independent claims 9 and 17, similar arguments for similar claims are similarly unpersuasive.
Regarding dependent claims 2-8 and 10-16, there is no eligible matter in the independent claims to incorporate by reference to the dependent claims. Therefore, the defects in those claims are not cured by the amendments.
Applicant's argument is unpersuasive.
The rejections stand.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiries concerning this communication or earlier communications from the examiner should be directed to Wilbert L. Starks, Jr., who may be reached Monday through Friday, between 8:00 a.m. and 5:00 p.m. EST. or via telephone at (571) 272-3691 or email: Wilbert.Starks@uspto.gov.
If you need to send an Official facsimile transmission, please send it to (571) 273-8300.
If attempts to reach the examiner are unsuccessful the Examiner’s Supervisor (SPE), Kakali Chaki, may be reached at (571) 272-3719.
Hand-delivered responses should be delivered to the Receptionist @ (Customer Service Window Randolph Building 401 Dulany Street, Alexandria, VA 22313), located on the first floor of the south side of the Randolph Building.
Finally, information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Moreover, status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have any questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) toll-free @ 1-866-217-9197.
/WILBERT L STARKS/
Primary Examiner, Art Unit 2122
WLS
07 JUL 2026
Prosecution Insights