DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/25/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
101 Rejection Arguments
Applicant asserts:
Applicant argues, on page 8, that claims 1-20 are directed to a practical application. Specifically, the ability to adapt to new tasks without catastrophic forgetting.
Examiner response:
Examiner respectfully disagrees. The ability to adapt to new tasks without catastrophic forgetting is not a practical application but rather an abstract idea. A person could take note of previous tasks as new tasks arrive to be done.
Applicant asserts:
Applicant further argues, on page 9, that details of claim 1 set forth substantially more. Specifically, the prior art does not teach using an orthogonal vector space to detect new or unknown tasks and retraining a model. In addition, applicant states that the claims point to an improvement in ML technology in the ability to adapt to new tasks enhances system computation and statistical efficiencies and ensures that the ability to learn new tasks is not limited by constant memory setup or limited prompt pool.
Examiner response:
Examiner respectfully disagrees. The steps regarding calculating an orthogonal vector space and monitors the encoded task input to determine a match is described in a way that is abstract and can be done within the human mind. The statement of improvement is stated in a conclusory manner without details necessary to be apparent to a person of ordinary skill.
103 Rejection Arguments
Applicant asserts:
Applicant argues, on page 10, that there is no motivation to combine Zifeng and Amit because Zifeng does not have disclosure related to the orthogonal vector space. Specifically, the prior art does not disclose comparing the encoded prompt to unseen or unknown prompts representing unseen or unknown tasks.
Examiner response:
Examiner respectfully disagrees. In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the reason to combine is to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Applicant asserts:
Applicant argues, on page 11, that there is no motivation to combine Zifeng and Amit because Zifeng does not have disclosure of an orthogonal vector space that can be used to detect previously unseen tasks, trigger prompt creation, and retraining of the model. In addition, the prior art does not teach increasing the size of the prompt pool.
Examiner response:
Examiner respectfully disagrees. In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the reason to combine is to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4). Regarding the prior art not teaching increasing the size of the prompt pool, Examiner points to Pourcel for teaching increasing the size of the prompt pool as the memory space.
Applicant asserts:
Applicant argues, on page 11, that Pourcel teaches adding nodes to the model itself while noting the model and the prompt pool is distinct. Applicant further notes that Pourcel teaches a method and a model that is “completely task-free” and that the art is not compatible to combine with Zifeng and/or Amit.
Examiner response:
Examiner respectfully disagrees. In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the reason to combine is to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1). Examiner further notes that the claim language and the disclosure does not clearly define that the prompt pool cannot be a part of the model or is distinct.
Applicant asserts:
Applicant argues, on page 11, that Pourcel does not teach computing an orthogonal complement of existing prompts in a prompt pool.
Examiner response:
Examiner respectfully disagrees. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
In reference to claim 1:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a process
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“calculating an orthogonal vector space as an orthogonal complement to the prompt vector space, the orthogonal vector space including vectors that are orthogonal to the vectors comprising the prompts of the prompt pool, wherein the orthogonal vector space represents that have not been seen previously;” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could calculate an orthogonal vector space as an orthogonal complement to the prompt vector space.
“monitoring an encoded task input of the ML model to determine if the encoded task input matches a specific vector of the vectors in the orthogonal vector space;” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could monitor an input of the ML model and determine if the input matches a vector of the orthogonal vector space.
“and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool [and automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.]” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could dynamically increase a prompt pool size by asking more questions for the model and adding the question to the prompt pool/memory.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
“A method, comprising: deploying in a machine-learning (ML) model a prompt vector space, the prompt vector space comprising a prompt pool including vectors comprising T number of prompts representing T number of tasks that are performed by the ML model, each prompt encoding information about a specific one of the tasks;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, [dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool and] automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and operating the ML model in an environment” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are integrated into a practical application.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
“A method, comprising: deploying in a machine-learning (ML) model a prompt vector space, the prompt vector space comprising a prompt pool including vectors comprising T number of prompts representing T number of tasks that are performed by the ML model, each prompt encoding information about a specific one of the tasks;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, [dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool and] automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and operating the ML model in an environment” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
In reference to claim 3:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a process
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“The method of claim 1, further comprising: when it is determined that encoded task input does not match one of the vectors in the orthogonal vector space” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine that task input does not match one of the vectors in the orthogonal vector space.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
“using the prompt pool without dynamically increasing its size when the ML model performs the task associated with an encoded task input;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and attaching one of the prompts of the prompt pool that is not dynamically increased that matches the encoded task input to the embedding of the input when performing the task.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are integrated into a practical application.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
“using the prompt pool without dynamically increasing its size when the ML model performs the task associated with the encoded task input;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and attaching one of the prompts of the prompt pool that is not dynamically increased that matches the encoded task input to an embedding of the input when performing the task.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
In reference to claim 4:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a process
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“The method of claim 1, wherein determining if the encoded task input matches a given one of the vectors in the orthogonal vector space comprises: determining if a proximity value which is a function of a distance between the encoded task input and each of the vectors in the orthogonal vector space is above a predefined threshold;” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine a proximity value between task input and each of the vectors in the orthogonal vector space is above a predefined threshold.
“and determining that the encoded task input matches the given one of the vectors when the proximity value for the given one of the vectors is above the predefined threshold.” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine the task input matches the given one vector when a proximity value is above a predefined threshold.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
No
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
No
In reference to claim 5:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a process
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“The method of claim 4, wherein determining the proximity value comprises determining a distance between a key associated with each of the vectors in the orthogonal vector space and a key associated with the encoded task input.” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine a distance between a key associated with teach of the vectors in the orthogonal vector space and key associated with encoded task input.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
No
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
No
In reference to claim 11:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a manufacture
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“calculating an orthogonal vector space as an orthogonal complement to the prompt vector space, the orthogonal vector space including vectors that are orthogonal to the vectors comprising the prompts of the prompt pool, wherein the orthogonal vector space represents that have not been seen previously;” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could calculate an orthogonal vector space as an orthogonal complement to the prompt vector space.
“monitoring an encoded task input of the ML model to determine if the encoded task input matches a specific vector of the vectors in the orthogonal vector space;” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could monitor an input of the ML model and determine if the input matches a vector of the orthogonal vector space.
“and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool [and automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.]” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could dynamically increase a prompt pool size by asking more questions for the model and adding the question vector to memory/prompt pool.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
“A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising: deploying in a machine-learning (ML) model a prompt vector space, the prompt vector space comprising a prompt pool including vectors comprising T number of prompts representing T number of tasks that are performed by the ML model, each prompt encoding information about a specific one of the tasks;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, [dynamically increasing a size of the prompt pool and] automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and operating the ML model in an environment” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are integrated into a practical application.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
“A method, comprising: deploying in a machine-learning (ML) model a prompt vector space, the prompt vector space comprising a prompt pool including vectors comprising T number of prompts representing T number of tasks that are performed by the ML model, each prompt encoding information about a specific one of the tasks;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, [dynamically increasing a size of the prompt pool and] automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and operating the ML model in an environment” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
In reference to claim 13:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a manufacture
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“The non-transitory storage medium of claim 11, further comprising the following operations: when it is determined that encoded task input does not match one of the vectors in the orthogonal vector space” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine that task input does not match one of the vectors in the orthogonal vector space.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
“using the prompt pool without dynamically increasing its size when the ML model performs the task associated with the encoded task input;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and attaching one of the prompts of the prompt pool that is not dynamically increased that matches the encoded task input to an embedding of the input when performing the task.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are integrated into a practical application.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
“using the prompt pool without dynamically increasing its size when the ML model performs the task associated with the encoded task input;” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
“and attaching one of the prompts of the prompt pool that is not dynamically increased that matches the encoded task input to an embedding of the input when performing the task.” is merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
In reference to claim 14:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a manufacture
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“The non-transitory storage medium of claim 11, wherein determining if the encoded task input matches a given one of the vectors in the orthogonal vector space comprises: determining if a proximity value which is a function of a distance between the encoded task input and each of the vectors in the orthogonal vector space is above a predefined threshold;” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine a proximity value between task input and each of the vectors in the orthogonal vector space is above a predefined threshold.
“and determining that the encoded task input matches the given one of the vectors when the proximity value for the given one of the vectors is above the predefined threshold.” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine the task input matches the given one vector when a proximity value is above a predefined threshold.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
No
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
No
In reference to claim 15:
Step 1 - Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is directed to a manufacture
Step 2A Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon?
“The non-transitory storage medium of claim 14, wherein determining the proximity value comprises determining a distance between a key associated with each of the vectors in the orthogonal vector space and a key associated with the encoded task input.” which is an abstract idea because it is directed to a mental process, an observation, evaluation, judgement, or opinion. The limitation as drafted, and under a broadest reasonable interpretation, can be performed in the human mind, or by a human using a pen and paper (MPEP 2106.04(a)(2)(Ill)(c)). For example, a person could determine a distance between a key associated with teach of the vectors in the orthogonal vector space and key associated with encoded task input.
Step 2A Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application?
No
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception?
No
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zifeng Wang et al; “Learning to Prompt for Continual Learning” (hereinafter “Zifeng”) in view of Amit Singh Bhatti; “Magic of Inner product and Orthogonal compliment subspaces” in further view of Julien Pourcel et al; “Online Task-free Continual Learning with Dynamic Sparse Distributed Memory” (hereinafter “Pourcel”).
Regarding claim 1, Zifeng teaches A method, comprising: deploying in a machine-learning (ML) model a prompt vector space, the prompt vector space comprising a prompt pool including vectors comprising T number of prompts representing T number of tasks that are performed by the ML model, each prompt encoding information about a specific one of the tasks; (Zifeng Figure 1 Overview; "L2P uses a single
backbone model and learns a prompt pool to instruct the model conditionally. Task-specific knowledge is stored inside a prompt pool," Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that the prompt pool/prompt vector space is deployed into the ML model/single backbone model; the prompt vector space comprises any number of prompts representing any number of tasks that are performed by the ML model; each prompt stores encoding information/input embeddings about a specific task)
monitoring an encoded task input of the ML model [to determine if the encoded task input matches a specific vector of the vectors in the orthogonal vector space]; (Zifeng Page 2 Paragraph 1; "pre-trained language model can process parameterized inputs in order to perform prompt-specific prediction" Examiner notes that ML model is monitoring the encoded task input/parameterized input to process it)
Zifeng does not teach calculating an orthogonal vector space as an orthogonal complement to the prompt vector space, the orthogonal vector space including vectors that are orthogonal to the vectors comprising the prompts of the prompt pool; wherein the orthogonal vector space represents that have not been seen previously;
However, Amit does teach calculating an orthogonal vector space as an orthogonal complement to the prompt vector space, the orthogonal vector space including vectors that are orthogonal to the vectors comprising the prompts of the prompt pool; wherein the orthogonal vector space represents that have not been seen previously; (Amit Section "Splitting vector spaces" Paragraph 1; "Principal subspace is a subspace of vector , where each of the vector are perpendicular to the orthogonal compliment subspace." Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that cited section discusses calculating an orthogonal vector space/orthogonal compliment subspace as an orthogonal complement where orthogonal vector space includes vectors that are orthogonal to the vectors in the prompt pool/principal subspace; orthogonal complement of a vector space represents information that is perpendicular and uncorrelated meaning it represents that have not been seen previously)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Zifeng in view of Amit does not teach determine if the encoded task input matches a specific vector of the vectors in the orthogonal vector space;
and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool and automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.
And operating the ML model in an environment;
However, Pourcel does teach determine if the encoded task input matches a specific vector of the vectors in the orthogonal vector space; (Pourcel Section 2.3 Paragraph 2; “When storing a pattern, the network computes the Hamming distance between the input address vector and each hard address. Address nodes within a preset Hamming distance will be activated (i.e., set to 1).” Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added." Examiner notes that a distance metric is used to determine if the input/input address vector matches a given one of the vectors/ address vector in the orthogonal vector space)
and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool and automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size. (Pourcel Section 2.3 Paragraph 2; “When storing a pattern, the network computes the Hamming distance between the input address vector and each hard address. Address nodes within a preset Hamming distance will be activated (i.e., set to 1).” Pourcel Section 3 Paragraph 1; "In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space." Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added." Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes when the input is determined to match the given one of the vectors in the orthogonal vector space by the BMU metric, the prompt pool/memory space is dynamically increasing and evolves dynamically and continually/automatically retraining the ML model to account for changes made to the prompt pool; the prompt pool’s/memory space size is increased by adding the specific vector/input address vector to the prompt pool)
And operating the ML model in an environment; (Pourcel Section 4.1 Paragraph 1; “We ran each experiment five times on a computer with one NVIDIA RTX 2060 GPU and report the average Top-1 classification results.” Examiner notes that the ML model is operating in an environment/computer)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 2, Zifeng does not teach The method of claim 1, further comprising: recalculating the [orthogonal] vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size.
However, Pourcel does teach The method of claim 1, further comprising: recalculating the [orthogonal] vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size. (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes that evolves dynamically and continually is recalculating to model the distribution of non-stationary data streams/account for changes made to prompt pool by increasing prompt pool’s size)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Zifeng does not teach orthogonal vector space
However, Amit does teach orthogonal vector space (Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that orthogonal complement is recalculated with updated prompt pool)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Regarding claim 3, Zifeng teaches using the prompt pool without dynamically increasing its size when the ML model performs the task associated with the encoded task input; (Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that the selected prompt is selected from an unchanged prompt pool for ML model to perform the task associated with the encoded task input)
and attaching one of the prompts of the prompt pool that is not dynamically increased that matches the encoded task input to an embedding of the input when performing the task. (Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that prompt is attached to the input embeddings when performing the task)
Zifeng does not teach The method of claim 1, further comprising: when it is determined that encoded task input does not match one of the vectors in the orthogonal vector space
However, Pourcel does teach The method of claim 1, further comprising: when it is determined that encoded task input does not match one of the vectors in the orthogonal vector space (Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added." Examiner notes that the BMU distance is used to determine if the task input matches one of the vectors in the orthogonal vector space; matching to one vector in the orthogonal vector space means not matching to one of the other vectors in the orthogonal vector space)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 4, Zifeng does not teach The method of claim 1, wherein determining if the encoded task input matches a given one of the vectors in the orthogonal vector space comprises: determining if a proximity value which is a function of a distance between the encoded task input and each of the vectors in the orthogonal vector space is above a predefined threshold;
and determining that the encoded task input matches the given one of the vectors when the proximity value for the given one of the vectors is above the predefined threshold.
However, Pourcel does teach The method of claim 1, wherein determining if the encoded task input matches a given one of the vectors in the orthogonal vector space comprises: determining if a proximity value which is a function of a distance between the encoded task input and each of the vectors in the orthogonal vector space is above a predefined threshold; (Pourcel Section 3.1 Paragraph 2; "Our memory M is represented by two matrices A and C…Consider the pattern {(xt, yt)} in the never-ending data stream…we first find its nearest memory node, i.e., the best matching unit – BMU” Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added.")
and determining that the encoded task input matches the given one of the vectors when the proximity value for the given one of the vectors is above the predefined threshold. (Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added.")
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 5, Zifeng teaches The method of claim 4, wherein determining the proximity value comprises determining a distance between a key associated with each of the vectors in the orthogonal vector space and a key associated with the encoded task input. (Zifeng Page 5 Paragraph 1; "Denote γ : R Dk × R Dk → R as a function to score the match between the query and prompt key (we find cosine distance works well). Given an input x, we use q(x) to lookup the top-N keys by simply solving the objective: Kx = argmin {si}N i=1⊆[1,M] X N i=1 γ (q(x), ksi ), (3) where Kx represents the a subset of top-N keys selected specifically for x from K" Examiner notes that keys from K are keys associated with each vector in the orthogonal vector space and a cosine distance is determined)
Regarding claim 6, Zifeng does not teach The method of claim 1, wherein dynamically increasing the size of the prompt pool comprises adding a new prompt that represents the encoded task input to the prompt pool.
However, Pourcel does teach The method of claim 1, wherein dynamically increasing the size of the prompt pool comprises adding a new prompt that represents the encoded task input to the prompt pool. (Pourcel Section 3.1 Paragraph 5; "If the to-BMU distance is higher than RT, a new memory node is created and added into the memory space" Examiner notes that the new prompt/new memory node that represents the encoded task input is added into the prompt pool/memory space)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 7, Zifeng does not teach The method of claim 6, further comprising: recalculating the orthogonal vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size, wherein recalculating the orthogonal vector space comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors contained in the prompt pool.
However, Pourcel does teach recalculating the [orthogonal] vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes that the prompt pool/memory space is dynamically increasing and evolves dynamically and continually/recalculating the vector space to account for changes made to the prompt pool)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Zifeng in view of Pourcel does not teach wherein recalculating the orthogonal vector space comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors contained in the prompt pool.
However, Amit does teach wherein recalculating the orthogonal vector space comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors contained in the prompt pool. (Amit Section "Splitting vector spaces" Paragraph 1; "Principal subspace is a subspace of vector , where each of the vector are perpendicular to the orthogonal compliment subspace." Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that the orthogonal vector space is recalculated with the dynamically increased prompt pool wherein it comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors in the prompt pool)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Regarding claim 8, Zifeng does not teach The method of claim 1, wherein automatically retraining the ML model comprises using data aligned with all the vectors in the prompt vector space as input for the retraining.
However, Pourcel does teach The method of claim 1, wherein automatically retraining the ML model comprises using data aligned with all the vectors in the prompt vector space as input for the retraining. (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner evolves dynamically and continually means the model is automatically retrained to comprise data aligned with all vectors in the prompt vector space/data added from the non-stationary data streams)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 9, Zifeng does not teach The method of claim 1, wherein automatically retraining the ML model comprises using data aligned with the vector of the orthogonal vector space that matched the encoded task input.
However, Pourcel does teach The method of claim 1, wherein automatically retraining the ML model comprises using data aligned with the vector of the [orthogonal] vector space that matched the encoded task input. (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner evolves dynamically and continually means the model is automatically retrained to comprise data aligned with the vector of the vector space that matched the task input/data added from the non-stationary data streams)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Zifeng in view of Pourcel does not teach orthogonal vector space
However, Amit does teach orthogonal vector space (Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that orthogonal complement is recalculated with the matched encoded task input added to the vector space)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Regarding claim 10, Zifeng teaches and attaching a new prompt of the [dynamically] increased prompt pool that represents the encoded task input and that matches the encoded task input to an embedding of the input when performing the task. (Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that the selected prompt is selected from an increased prompt pool for ML model to perform the task associated with the encoded task input)
Zifeng does not teach The method of claim 1, further comprising: deploying the automatically retrained ML model, the retrained ML model including the dynamically increased prompt pool;
using the dynamically increased prompt pool when the ML model performs the task associated with the encoded task input;
However, Pourcel does teach The method of claim 1, further comprising: deploying the automatically retrained ML model, the retrained ML model including the dynamically increased prompt pool; (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes that the ML model is constantly redeployed to include the dynamically increased prompt pool)
using the dynamically increased prompt pool when the ML model performs the task associated with the encoded task input; (Pourcel Section 3.1 Paragraph 4; "This update strategy is similar to that of competitive learning proposed by [28] to adapt models to the current data distribution." Examiner notes that when the model performs the task associated with the encoded task input, it uses/adapts to the dynamically increased prompt pool/current data distribution)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 11, Zifeng teaches deploying in a machine-learning (ML) model a prompt vector space, the prompt vector space comprising a prompt pool including vectors comprising T number of prompts representing T number of tasks that are performed by the ML model, each prompt encoding information about a specific one of the tasks; (Zifeng Figure 1 Overview; "L2P uses a single
backbone model and learns a prompt pool to instruct the model conditionally. Task-specific knowledge is stored inside a prompt pool," Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that the prompt pool/prompt vector space is deployed into the ML model/single backbone model; the prompt vector space comprises any number of prompts representing any number of tasks that are performed by the ML model; each prompt stores encoding information/input embeddings about a specific task)
monitoring an encoded task input of the ML model [to determine if the encoded task input matches a given one of the vectors in the orthogonal vector space]; (Zifeng Page 2 Paragraph 1; "pre-trained language model can process parameterized inputs in order to perform prompt-specific prediction" Examiner notes that ML model is monitoring the encoded task input/parameterized input to process it)
Zifeng does not teach calculating an orthogonal vector space as an orthogonal complement to the prompt vector space, the orthogonal vector space including vectors that are orthogonal to the vectors comprising the prompts of the prompt pool, wherein the orthogonal vector space represents that have not been seen previously;
However, Amit does teach calculating an orthogonal vector space as an orthogonal complement to the prompt vector space, the orthogonal vector space including vectors that are orthogonal to the vectors comprising the prompts of the prompt pool, wherein the orthogonal vector space represents that have not been seen previously; (Amit Section "Splitting vector spaces" Paragraph 1; "Principal subspace is a subspace of vector , where each of the vector are perpendicular to the orthogonal compliment subspace." Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that cited section discusses calculating an orthogonal vector space/orthogonal compliment subspace as an orthogonal complement where orthogonal vector space includes vectors that are orthogonal to the vectors in the prompt pool/principal subspace; orthogonal complement of a vector space represents information that is perpendicular and uncorrelated meaning it represents that have not been seen previously)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Zifeng in view of Amit does not teach A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
determine if the encoded task input matches a specific vector of the vectors in the orthogonal vector space;
and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool and automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size.
However, Pourcel does teach A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising: (Pourcel Section 4.1 Paragraph 1; “We ran each experiment five times on a computer with one NVIDIA RTX 2060 GPU and report the average Top-1 classification results.” Examiner notes that computer contains hardware processors to perform stored instructions)
determine if the encoded task input matches a specific vector of the vectors in the orthogonal vector space; (Pourcel Section 2.3 Paragraph 2; “When storing a pattern, the network computes the Hamming distance between the input address vector and each hard address. Address nodes within a preset Hamming distance will be activated (i.e., set to 1).” Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added." Examiner notes that a distance metric is used to determine if the input/input address vector matches a given one of the vectors/ address vector in the orthogonal vector space)
and when it is determined that the encoded task input matches the given one of the vectors in the orthogonal vector space, dynamically increasing a size of the prompt pool by adding the specific vector to the prompt pool and automatically retraining the ML model to account for changes made to the prompt pool by increasing the prompt pool’s size. (Pourcel Section 2.3 Paragraph 2; “When storing a pattern, the network computes the Hamming distance between the input address vector and each hard address. Address nodes within a preset Hamming distance will be activated (i.e., set to 1).” Pourcel Section 3 Paragraph 1; "In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space." Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added." Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes when the input is determined to match the given one of the vectors in the orthogonal vector space by the BMU metric, the prompt pool/memory space is dynamically increasing and evolves dynamically and continually/automatically retraining the ML model to account for changes made to the prompt pool; the prompt pool’s/memory space size is increased by adding the specific vector/input address vector to the prompt pool)
And operating the ML model in an environment; (Pourcel Section 4.1 Paragraph 1; “We ran each experiment five times on a computer with one NVIDIA RTX 2060 GPU and report the average Top-1 classification results.” Examiner notes that the ML model is operating in an environment/computer)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 12, Zifeng does not teach The non-transitory storage medium of claim 11, further comprising the following operations: recalculating the [orthogonal] vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size.
However, Pourcel does teach The non-transitory storage medium of claim 11, further comprising the following operations: recalculating the [orthogonal] vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size. (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes that evolves dynamically and continually is recalculating to model the distribution of non-stationary data streams/account for changes made to prompt pool by increasing prompt pool’s size)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Zifeng does not teach orthogonal vector space
However, Amit does teach orthogonal vector space (Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that orthogonal complement is recalculated with updated prompt pool)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Regarding claim 13, Zifeng teaches using the prompt pool without dynamically increasing its size when the ML model performs the task associated with the encoded task input; (Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that the selected prompt is selected from an unchanged prompt pool for ML model to perform the task associated with the encoded task input)
and attaching one of the prompts of the prompt pool that is not dynamically increased that matches the encoded task input to an embedding of the input when performing the task. (Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that prompt is attached to the input embeddings when performing the task)
Zifeng does not teach The non-transitory storage medium of claim 11, further comprising the following operations: when it is determined that encoded task input does not match one of the vectors in the orthogonal vector space
However, Pourcel does teach The non-transitory storage medium of claim 11, further comprising the following operations: when it is determined that encoded task input does not match one of the vectors in the orthogonal vector space (Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added." Examiner notes that the BMU distance is used to determine if the task input matches one of the vectors in the orthogonal vector space; matching to one vector in the orthogonal vector space means not matching to one of the other vectors in the orthogonal vector space)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 14, Zifeng does not teach The non-transitory storage medium of claim 11, wherein determining if the encoded task input matches a given one of the vectors in the orthogonal vector space comprises: determining if a proximity value which is a function of a distance between the encoded task input and each of the vectors in the orthogonal vector space is above a predefined threshold;
and determining that the encoded task input matches the given one of the vectors when the proximity value for the given one of the vectors is above the predefined threshold.
However, Pourcel does teach The non-transitory storage medium of claim 11, wherein determining if the encoded task input matches a given one of the vectors in the orthogonal vector space comprises: determining if a proximity value which is a function of a distance between the encoded task input and each of the vectors in the orthogonal vector space is above a predefined threshold; (Pourcel Section 3.1 Paragraph 2; "Our memory M is represented by two matrices A and C…Consider the pattern {(xt, yt)} in the never-ending data stream…we first find its nearest memory node, i.e., the best matching unit – BMU” Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added.")
and determining that the encoded task input matches the given one of the vectors when the proximity value for the given one of the vectors is above the predefined threshold. (Pourcel Section 3.1 Paragraph 3; "they compare the to-BMU distance or similar metric to a pre-defined threshold to decide whether a new node or layer is added.")
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 15, Zifeng teaches The non-transitory storage medium of claim 14, wherein determining the proximity value comprises determining a distance between a key associated with each of the vectors in the orthogonal vector space and a key associated with the encoded task input. (Zifeng Page 5 Paragraph 1; "Denote γ : R Dk × R Dk → R as a function to score the match between the query and prompt key (we find cosine distance works well). Given an input x, we use q(x) to lookup the top-N keys by simply solving the objective: Kx = argmin {si}N i=1⊆[1,M] X N i=1 γ (q(x), ksi ), (3) where Kx represents the a subset of top-N keys selected specifically for x from K" Examiner notes that keys from K are keys associated with each vector in the orthogonal vector space and a cosine distance is determined)
Regarding claim 16, Zifeng does not teach The non-transitory storage medium of claim 11, wherein dynamically increasing the size of the prompt pool comprises adding a new prompt that represents the encoded task input to the prompt pool.
However, Pourcel does teach The non-transitory storage medium of claim 11, wherein dynamically increasing the size of the prompt pool comprises adding a new prompt that represents the encoded task input to the prompt pool. (Pourcel Section 3.1 Paragraph 5; "If the to-BMU distance is higher than RT, a new memory node is created and added into the memory space" Examiner notes that the new prompt/new memory node that represents the encoded task input is added into the prompt pool/memory space)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 17, Zifeng does not teach The non-transitory storage medium of claim 16, further comprising the following operations: recalculating the orthogonal vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size, wherein recalculating the orthogonal vector space comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors contained in the prompt pool.
However, Pourcel does teach The non-transitory storage medium of claim 16, further comprising the following operations: recalculating the [orthogonal] vector space to account for the changes made to the prompt pool by increasing the prompt pool’s size (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes that the prompt pool/memory space is dynamically increasing and evolves dynamically and continually/recalculating the vector space to account for changes made to the prompt pool)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Zifeng in view of Pourcel does not teach wherein recalculating the orthogonal vector space comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors contained in the prompt pool.
However, Amit does teach wherein recalculating the orthogonal vector space comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors contained in the prompt pool. (Amit Section "Splitting vector spaces" Paragraph 1; "Principal subspace is a subspace of vector , where each of the vector are perpendicular to the orthogonal compliment subspace." Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that the orthogonal vector space is recalculated with the dynamically increased prompt pool wherein it comprises adding a new vector to the orthogonal vector space that is orthogonal to the vectors in the prompt pool)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Regarding claim 18, Zifeng does not teach The non-transitory storage medium of claim 11, wherein automatically retraining the ML model comprises using data aligned with the vectors in the prompt vector space as input for the retraining.
However, Pourcel does teach The non-transitory storage medium of claim 11, wherein automatically retraining the ML model comprises using data aligned with the vectors in the prompt vector space as input for the retraining. (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner evolves dynamically and continually means the model is automatically retrained to comprise data aligned with all vectors in the prompt vector space/data added from the non-stationary data streams)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Regarding claim 19, Zifeng does not teach The non-transitory storage medium of claim 11, wherein automatically retraining the ML model comprises using data aligned with the vector of the orthogonal vector space that matched the encoded task input.
However, Pourcel does teach The non-transitory storage medium of claim 11, wherein automatically retraining the ML model comprises using data aligned with the vector of the [orthogonal] vector space that matched the encoded task input. (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner evolves dynamically and continually means the model is automatically retrained to comprise data aligned with the vector of the vector space that matched the task input/data added from the non-stationary data streams)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Zifeng in view of Pourcel does not teach orthogonal vector space
However, Amit does teach orthogonal vector space (Amit Section "PCA Setting" Paragraph 1; "to find the projection of X with the orthogonal compliment, we are taking a inner product of the ith component of orthogonal compliment with ith component of X vector." Examiner notes that orthogonal complement is recalculated with the matched encoded task input added to the vector space)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng and Amit. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. One of ordinary skill would have motivation to combine Zifeng and Amit to better represent the changing data/tasks for the ML model “eigen-values and orthogonal projections help us to find a low dimensional projection of a high dimensional vector with minimum information loss.” (Amit Section “PCA Setting” Paragraph 4).
Regarding claim 20, Zifeng teaches and attaching a new prompt of the [dynamically] increased prompt pool that represents the encoded task input and that matches the encoded task input to an embedding of the input when performing the task. (Zifeng Page 2 Paragraph 4; "The selected prompts are then prepended to the input embeddings (Figure 2), which implicitly add task-relevant instruction to pre-trained models" Examiner notes that the selected prompt is selected from an increased prompt pool for ML model to perform the task associated with the encoded task input)
Zifeng does not teach The non-transitory storage medium of claim 11, further comprising the following operations: deploying the automatically retrained ML model, the retrained ML model including the dynamically increased prompt pool;
using the dynamically increased prompt pool when the ML model performs the task associated with the encoded task input;
However, Pourcel does teach The non-transitory storage medium of claim 11, further comprising the following operations: deploying the automatically retrained ML model, the retrained ML model including the dynamically increased prompt pool; (Pourcel Section 5 Paragraph 1; "DSDM is an associative content-addressable memory model that evolves dynamically and continually to model the distribution of non-stationary data streams." Examiner notes that the ML model is constantly redeployed to include the dynamically increased prompt pool)
using the dynamically increased prompt pool when the ML model performs the task associated with the encoded task input; (Pourcel Section 3.1 Paragraph 4; "This update strategy is similar to that of competitive learning proposed by [28] to adapt models to the current data distribution." Examiner notes that when the model performs the task associated with the encoded task input, it uses/adapts to the dynamically increased prompt pool/current data distribution)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Zifeng, Amit, and Pourcel. Zifeng teaches using prompts stored in a memory space to instruct a pre-trained model to learn tasks sequentially under different task transitions. Amit teaches orthogonal compliment subspaces. Pourcel teaches an efficient semi-distributed associative memory algorithm for learning and evaluating to be carried out at any point of time. One of ordinary skill would have motivation to combine Zifeng, Amit, and Pourcel to improve performance when dealing with correlated, i.e., nonrandom, input data “Conventional SDM networks have a fixed number of memory nodes whose ’hard’ addresses are randomly distributed in a binary memory space. This tends to lower performance when dealing with correlated, i.e., nonrandom, input data [23]. In our algorithm, we start with an empty memory space, and new address nodes are incrementally added depending on the considered input patterns and the current state of memory space.” (Pourcel Section 3 Paragraph 1).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL DUC TRAN whose telephone number is (571)272-6870. The examiner can normally be reached Mon-Fri 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.D.T./Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights