DETAILED ACTION
This action is in response to the filing on 11/25/2025. Claims 1–20, are pending and have been considered below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
a reinforcement learning module configured to train a machine learning model in claim 16.
an agent module configured to generate, … , a plurality of generated sequential patterns in claim 16.
an environment module configured to, … obtain, …, at least one recorded sequence … and determine, … , a reward in claim 16.
a pattern-extracting module configured to extract a plurality of high-utility sequential patterns in claim 16.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward satisfies a threshold according to a predetermined utility measure”, on lines 21-24, the specification lacks description to support a predicted reward satisfying a threshold according to a utility measure. At best, the specification supports applying a utility measure to calculate rewards [see para. 15]. For the purpose of examination, it will be interpreted as "wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward is calculated according to a predetermined utility measure".
Claim 11 recites “wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward satisfies a threshold according to a predetermined utility measure”, on lines 23-26, the specification lacks description to support a predicted reward satisfying a threshold according to a utility measure. At best, the specification supports applying a utility measure to calculate rewards [see para. 15]. For the purpose of examination, it will be interpreted as "wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward is calculated according to a predetermined utility measure".
Claim 16 recites “wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward satisfies a threshold according to a predetermined utility measure”, on lines 21-24, the specification lacks description to support a predicted reward satisfying a threshold according to a utility measure. At best, the specification supports applying a utility measure to calculate rewards [see para. 15]. For the purpose of examination, it will be interpreted as "wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward is calculated according to a predetermined utility measure".
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1–20 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent Claims 1, 11, and 16
Step 1:
Claims 1, 11, and 16 recite a method, manufacture, and system respectively; therefore, they are directed to one of the four categories of statutory subject matter (process/method, machine/product/apparatus, manufacture, or composition of matter).
Step 2A Prong 1:
Claims 1, 11, and 16 recite a method, manufacture, and system comprising:
generating, by an agent module and based on a plurality of next-actions predicted by the machine learning model, a plurality of generated sequential patterns, wherein each of the plurality of generated sequential patterns describes a corresponding sequence of system interaction events — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)). Or a mathematical concept (see MPEP § 2106.04(a)(2)(I)), specifically organizing information and manipulating information through mathematical correlations.
for each generated sequential pattern of the plurality of generated sequential patterns: determining, by the environment module and based on the at least one recorded sequence, a reward for the generated sequential pattern — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)). Or a mathematical concept (see MPEP § 2106.04(a)(2)(I)), specifically mathematical calculations.
updating the machine learning model, by the agent module and based on the reward — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)). Or a mathematical concept (see MPEP § 2106.04(a)(2)(I)), specifically organizing information and manipulating information through mathematical correlations.
whose predicted reward satisfies a threshold according to a predetermined utility measure (interpreted as whose predicted reward is calculated according to a predetermined utility measure per 35 U.S.C. 112(a) rejection above) — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)). Or a mathematical concept (see MPEP § 2106.04(a)(2)(I)), specifically organizing information and manipulating information through mathematical correlations.
Step 2A Prong 2:
This judicial exception is not integrated into a practical application.
Claim 1 recites the additional elements of:
training, by a reinforcement learning server, a machine learning model to learn a state-action map that contains high-value sequential patterns and obtain a trained machine learning model, comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to training a machine learning model.
for each generated sequential pattern of the plurality of generated sequential patterns: obtaining, by an environment module and from among a plurality of recorded sequences of system interaction events, at least one recorded sequence that matches the generated sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
extracting, by a pattern-extracting server, at least one high-value sequential pattern from the trained machine learning model, wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
causing a user interface of a computing environment to be modified based on information from the at least one high-value sequential pattern, wherein the user interface is modified to promote the at least one high-value sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity in view of Jiming Liu et al., "An adaptive user interface based on personalized learning" (pg. 1, Section "The adaptive user interface", para. 1, discloses a history of adaptive user interfaces monitoring user-activity sequences and adapting the UI based on the sequences).
Claim 11 recites the additional elements of:
A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to generic computer components.
training a machine learning model to learn a state-action map that contains high-value sequential patterns and obtain a trained machine learning model, comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to training a machine learning model.
wherein each of the plurality of generated sequential patterns describes a corresponding sequence of system interaction events and ends with a stop action — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to sequential patterns that end with a particular action.
for each generated sequential pattern of the plurality of generated sequential patterns: obtaining, from among a plurality of recorded sequences of system interaction events, at least one recorded sequence that matches the generated sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
extracting, by a pattern-extracting server, at least one high-value sequential pattern from the trained machine learning model, wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
causing a user interface of a computing environment to be modified based on information from the at least one high-value sequential pattern, wherein the user interface is modified to promote the at least one high-value sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity in view of Jiming Liu et al., "An adaptive user interface based on personalized learning" (pg. 1, Section "The adaptive user interface", para. 1, discloses a history of adaptive user interfaces monitoring user-activity sequences and adapting the UI based on the sequences).
Claim 16 recites the additional elements of:
a reinforcement learning module configured to train a machine learning model to learn a state-action map that contains high-utility sequential patterns and obtain a trained machine learning model, comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to training a machine learning model.
wherein each of the plurality of generated sequential patterns describes a corresponding sequence of system interaction events and ends with a stop action — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to sequential patterns that end with a particular action.
an environment module configured to, for each generated sequential pattern of the plurality of generated sequential patterns: obtain, from among a plurality of recorded sequences of system interaction events stored in a data store, at least one recorded sequence that matches the generated sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
a pattern-extracting module configured to extract a plurality of high-utility sequential patterns from the trained machine learning model, wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
an interface-modification server configured to cause a user interface of a computing environment to be modified based on information from the plurality of high-utility sequential patterns, wherein the user interface is modified to promote the at least one high-value sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity in view of Jiming Liu et al., "An adaptive user interface based on personalized learning" (pg. 1, Section "The adaptive user interface", para. 1, discloses a history of adaptive user interfaces monitoring user-activity sequences and adapting the UI based on the sequences).
Step 2B:
The claims do not contain significantly more than the judicial exception.
Claim 1 recites the additional elements of:
training, by a reinforcement learning server, a machine learning model to learn a state-action map that contains high-value sequential patterns and obtain a trained machine learning model, comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to training a machine learning model.
for each generated sequential pattern of the plurality of generated sequential patterns: obtaining, by an environment module and from among a plurality of recorded sequences of system interaction events, at least one recorded sequence that matches the generated sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
extracting, by a pattern-extracting server, at least one high-value sequential pattern from the trained machine learning model, wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
causing a user interface of a computing environment to be modified based on information from the at least one high-value sequential pattern, wherein the user interface is modified to promote the at least one high-value sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity in view of Jiming Liu et al., "An adaptive user interface based on personalized learning" (pg. 1, Section "The adaptive user interface", para. 1, discloses a history of adaptive user interfaces monitoring user-activity sequences and adapting the UI based on the sequences).
Claim 11 recites the additional elements of:
A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to generic computer components.
training a machine learning model to learn a state-action map that contains high-value sequential patterns and obtain a trained machine learning model, comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to training a machine learning model.
wherein each of the plurality of generated sequential patterns describes a corresponding sequence of system interaction events and ends with a stop action — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to sequential patterns that end with a particular action.
for each generated sequential pattern of the plurality of generated sequential patterns: obtaining, from among a plurality of recorded sequences of system interaction events, at least one recorded sequence that matches the generated sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
extracting, by a pattern-extracting server, at least one high-value sequential pattern from the trained machine learning model, wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
causing a user interface of a computing environment to be modified based on information from the at least one high-value sequential pattern, wherein the user interface is modified to promote the at least one high-value sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity in view of Jiming Liu et al., "An adaptive user interface based on personalized learning" (pg. 1, Section "The adaptive user interface", para. 1, discloses a history of adaptive user interfaces monitoring user-activity sequences and adapting the UI based on the sequences).
Claim 16 recites the additional elements of:
a reinforcement learning module configured to train a machine learning model to learn a state-action map that contains high-utility sequential patterns and obtain a trained machine learning model, comprising — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to training a machine learning model.
wherein each of the plurality of generated sequential patterns describes a corresponding sequence of system interaction events and ends with a stop action — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to sequential patterns that end with a particular action.
an environment module configured to, for each generated sequential pattern of the plurality of generated sequential patterns: obtain, from among a plurality of recorded sequences of system interaction events stored in a data store, at least one recorded sequence that matches the generated sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
a pattern-extracting module configured to extract a plurality of high-utility sequential patterns from the trained machine learning model, wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
an interface-modification server configured to cause a user interface of a computing environment to be modified based on information from the plurality of high-utility sequential patterns, wherein the user interface is modified to promote the at least one high-value sequential pattern — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity in view of Jiming Liu et al., "An adaptive user interface based on personalized learning" (pg. 1, Section "The adaptive user interface", para. 1, discloses a history of adaptive user interfaces monitoring user-activity sequences and adapting the UI based on the sequences).
As such claims 1, 11, and 16 are not patent eligible.
Dependent Claims 2–10, 12–15, and 17–20
Step 1:
Claims 2–10, 12–15, and 17–20 recite a method, manufacture, and system respectively; therefore, they are directed to one of the four categories of statutory subject matter (process/method, machine/product/apparatus, manufacture, or composition of matter).
Step 2A Prong 1:
Claims 2–10, 12–15, and 17–20 merely narrow the previously cited abstract idea limitations. For the reasons described above with respect to independent claims 1, 11, and 16 this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. The claim(s) disclose similar limitations described for the independent claim(s) above and do not provide anything more than the abstract idea.
Claims 4, 13, and 18 recite a method, manufacture, and system comprising:
wherein, for each generated sequential pattern of the plurality of generated sequential patterns, determining the reward is based on a utility measure that indicates an average value across a plurality of products — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)). Or a mathematical concept (see MPEP § 2106.04(a)(2)(I)), specifically mathematical calculations.
Claim 3 recites a method comprising:
wherein, for each generated sequential pattern of the plurality of generated sequential patterns, determining the reward is based on a utility measure that is not anti-monotonic — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)). Or a mathematical concept (see MPEP § 2106.04(a)(2)(I)), specifically mathematical calculations.
Claims 5, 14, and 19 recite a method, manufacture, and system comprising:
wherein, for at least one generated sequential pattern of the plurality of generated sequential patterns, generating the generated sequential pattern comprises selecting a next-action of the generated sequential pattern at random — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)).
Claims 8, 15, and 20 recite a method, manufacture, and system comprising:
wherein extracting the at least one high-value sequential pattern from the trained machine learning model comprises searching the trained machine learning model using a depth-first-search algorithm — Under its broadest reasonable interpretation, this limitation encompasses the abstract idea of a mental process, or a concept that can be performed in the human mind with the use of a physical aid (e.g. pen and paper), including observation, evaluation, judgement or opinion (see MPEP § 2106.04(a)(2)(III)). Or a mathematical concept (see MPEP § 2106.04(a)(2)(I)), specifically organizing information and manipulating information through mathematical correlations.
Step 2A Prong 2:
This judicial exception is not integrated into a practical application.
Claims 2, 12, and 17 recite the additional element of:
wherein the system interaction events among the plurality of recorded sequences of system interaction events include click events or entity-initiated communications — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to particular system interaction events.
Claim 6 recites the additional element of:
wherein each generated sequential pattern of the plurality of generated sequential patterns ends with a stop action — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to sequential patterns that end with a particular action.
Claim 7 recites the additional element of:
wherein the machine learning model comprises a deep neural network — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to a deep neural network.
Claim 9 recites the additional element of:
wherein, for each of the plurality of generated sequential patterns, each of the corresponding sequence of system interaction events comprises a plurality of attributes — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
Claim 10 recites the additional element of:
wherein the machine learning model includes a long short-term memory network and a Q-network — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to an LSTM network and a Q-network.
Step 2B:
The claims do not contain significantly more than the judicial exception.
Claims 2, 12, and 17 recite the additional element of:
wherein the system interaction events among the plurality of recorded sequences of system interaction events include click events or entity-initiated communications — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to particular system interaction events.
Claim 6 recites the additional element of:
wherein each generated sequential pattern of the plurality of generated sequential patterns ends with a stop action — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to sequential patterns that end with a particular action.
Claim 7 recites the additional element of:
wherein the machine learning model comprises a deep neural network — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to a deep neural network.
Claim 9 recites the additional element of:
wherein, for each of the plurality of generated sequential patterns, each of the corresponding sequence of system interaction events comprises a plurality of attributes — This element amounts to no more than insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is well-understood, routine, conventional activity (see MPEP § 2106.05(d)(II), storing and retrieving information in memory).
Claim 10 recites the additional element of:
wherein the machine learning model includes a long short-term memory network and a Q-network — This element amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). This element merely limits the use of the abstract idea to an LSTM network and a Q-network.
As such claims 2–10, 12–15, and 17–20 are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1–7, 9, 11–14, and 16–19 are rejected under 35 U.S.C. 103 as being unpatentable over Mnih et al. (US 9,679,258 B2), hereinafter Mnih, in view of Cramer (US 8,117,197 B1), hereinafter Cramer.
Regarding claim 1, Mnih teaches A method comprising (According to the present invention there is therefore provided a method of reinforcement learning [see Mnih, Col. 2, lines 36–38]):
training, by a reinforcement learning server (FIG. 5a shows a schematic block diagram of a data processor 100 configured to implement a neural network-based reinforcement learning procedure as described above. [see Mnih, Col. 15, line 66–Col. 16, line 1; FIG. 5a]), a machine learning model to learn a state-action map that contains high-value sequential patterns and obtain a trained machine learning model, comprising (Mnih discloses using neural networks to learn an action-value for a respective action for all possible actions, and to learn an action selector to maximize the quality values when selecting actions [see Mnih, Col. 3, lines 53–59; Col. 7, lines 11–49]. The optimum learnt action-value is the maximum expected return following a strategy given a sequence and taking an action corresponding to the function Q(s,a) [see Mnih, Col. 9, lines 24–61]);
generating, by an agent module and based on a plurality of next-actions predicted by the machine learning model (Mnih discloses a software agent as the reinforcement learning agent that generates actions from up to 18 actions at each frame [see Mnih, Col. 10, lines 42–67; FIG. 2]), a plurality of generated sequential patterns, wherein each of the plurality of generated sequential patterns describes a corresponding sequence of system interaction events (Mnih discloses using the deep neural network to estimate a set Q-values with one for each action [see Mnih, Col. 10, lines 65–67; Col. 15, lines 61–65; FIG. 2] when selecting actions for every kth frame [see Mnih, Col. 16, lines 54–56]);
for each generated sequential pattern of the plurality of generated sequential patterns:
obtaining, by an environment module and from among a plurality of recorded sequences of system interaction events, at least one recorded sequence that matches the generated sequential pattern (Mnih discloses the data processor to implement [see Mnih, Col. 15, line 66–Col. 16, line 1] the agent interacts with the environment E in a sequence of actions, observations and rewards to select an action at each time-step [see Mnih, Col. 8, lines 64–67]. Mnih further discloses storing the before and after states, the action taken, and the reward taken as experience data and drawing a transition from the stored experience to the 'after' state of the transition [see Mnih, Col. 11, lines 28–52] at each time-step in a data-set pooled over many episodes into a replay memory which can be sampled for experience replay during training [see Mnih, Col. 13, lines 53–65]);
determining, by the environment module and based on the at least one recorded sequence, a reward for the generated sequential pattern (Mnih discloses the data processor to implement [see Mnih, Col. 15, line 66–Col. 16, line 1] the agent interacts with the environment E in a sequence of actions, observations and rewards to select an action at each time-step [see Mnih, Col. 8, lines 64–67]. Mnih further discloses storing the before and after states, the action taken and the reward taken as experience data [see Mnih, Col. 11, lines 28–52]);
determining, by the environment module and based on the at least one recorded sequence, a reward for the generated sequential pattern (Mnih discloses the data processor to implement [see Mnih, Col. 15, line 66–Col. 16, line 1] the agent interacts with the environment E in a sequence of actions, observations and rewards to select an action at each time-step [see Mnih, Col. 8, lines 64–67]. Mnih further discloses storing the before and after states, the action taken and the reward taken as experience data [see Mnih, Col. 11, lines 28–52]);
updating the machine learning model, by the agent module and based on the reward (Mnih discloses the software agent is enabled for reinforcement learning using a deep neural network to estimate Q-values [see Mnih, Col. 10, lines 42–67] which is trained using the reward to update the target Q-value [see Mnih Col. 11, lines 28–65])
extracting, by a pattern-extracting system, at least one high-value sequential pattern from the trained machine learning model, wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward satisfies a threshold according to a predetermined utility measure (wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward satisfies a threshold according to a predetermined utility measure interpreted as wherein the at least one high-value sequential pattern comprises a time-ordered sequence of system interaction events whose predicted reward is calculated according to a predetermined utility measure per 35 U.S.C. 112(a) above)(Mnih discloses the machine learning model selecting an action that corresponds to an optimum parameter, generally the maximized expected reward [see Mnih, Col. 3, lines 53-59], wherein the reward is calculated according to a predetermined utility measure that depends on the whole sequence [see Mnih, Col. 9, lines 6-9], and the optimal action function is defined as the maximum expected return achievable by following any strategy after seeing some sequence and taking some action [see Mnih, Col. 9, lines 28-34]).
However, Mnih fails to teach extracting, by a pattern-extracting server; and causing a user interface of a computing environment to be modified based on information from the at least one high-value sequential pattern, wherein the user interface is modified to promote the at least one high-value sequential pattern.
In the same field of endeavor, Cramer teaches:
extracting, by a server (Cramer discloses software on a server to produce updating rankings of search results wherein each set of recommendations displayed are displayed in a different area to set them apart from previously displayed results [see Cramer, Col. 2, lines 45–65]. Cramer further discloses a real-time model for ranking search results that takes into account recent user actions for the recommended results [see Cramer, Col. 8, line 52–Col. 9, line 7]);
causing a user interface of a computing environment to be modified based on information from at least one result, wherein the user interface is modified to promote the at least one result (Cramer discloses software on a server to produce updating rankings of search results wherein each set of recommendations displayed are displayed in a different area to set them apart from previously displayed results [see Cramer, Col. 2, lines 45–65]).
Mnih and Cramer both address the issue of machine learning models that adjust weights, accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate extracting, by a server as suggested by Cramer into Mnih, with a reasonable expectation of success, such that the neural network utilizes a server to produce action sequence patterns that maximize an expected reward and to further incorporate causing a user interface of a computing environment to be modified based on information from at least one result, wherein the user interface is modified to promote the at least one result as suggested by Cramer into Mnih, with a reasonable expectation of success, such that the system of Mnih could provide action sequence results in the form of sequential patterns with at least one high-value sequential pattern being selected, that could then be displayed on the search results ranking of Cramer with the high-value pattern being ranked higher than lower value patterns such to promote the high-value sequential pattern based on it being ranked higher. This modification would have been motivated by the desire to address the need to develop a UI capable of delivering the benefits of real-time re-ranking while being as unobtrusive as possible to the user's search experience (see Cramer, Col. 2, lines 36–38).
Regarding claim 2, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein the system interaction events among the plurality of recorded sequences of system interaction events include click events or entity-initiated communications (Mnih discloses that the states are defined by image data as a sequence of images [see Mnih, Col. 5, lines 20–21] which is obtained from a system emulating the game frames [see Mnih, Col. 11, lines 1–4], and that the states are included in the recorded sequences [see Mnih, Col. 11, lines 29–31]).
Regarding claim 3, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein, for each generated sequential pattern of the plurality of generated sequential patterns, determining the reward is based on a utility measure that is not anti-monotonic (In addition it receives a reward r.sub.t representing the change in game score. Note that in general the game score may depend on the whole prior sequence of actions and observations [see Mnih, Col. 9, lines 6–9]).
Regarding claim 4, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein, for each generated sequential pattern of the plurality of generated sequential patterns, determining the reward is based on a utility measure that indicates an average value across a plurality of products (the behavior distribution is averaged over many of its previous states, smoothing out learning and helping to avoid oscillations or divergence in the parameters [see Mnih, Col. 14, lines 15–18]).
Regarding claim 5, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein, for at least one generated sequential pattern of the plurality of generated sequential patterns, generating the generated sequential pattern comprises selecting a next-action of the generated sequential pattern at random (Mnih discloses selecting a next-action randomly including randomly with probability ε [see Mnih, Col. 5, lines 6–10; Col. 11, lines 18–21]).
Regarding claim 6, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein each generated sequential pattern of the plurality of generated sequential patterns ends with a stop action (At step S206, the procedure draws a transition from the stored experience data, either randomly or according to a prioritised strategy, and provides the end, ‘after’ state of the transition to the first neural network (neural network 0). [see Mnih, Col. 11, lines 31–35]).
Regarding claim 7, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein the machine learning model comprises a deep neural network (Preferably the first and second neural networks are deep neural networks [see Mnih, Col. 5, lines 36–37]).
Regarding claim 9, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein, for each of the plurality of generated sequential patterns, each of the corresponding sequence of system interaction events comprises a plurality of attributes (The experience data comprises, for each action, data defining the starting state, the action taken, and the subsequent state of the system. [see Mnih, Col. 4, lines 6–8]).
Regarding claim 11, claim 11 contains substantially similar limitations to those found in claim 1. Therefore it is rejected for the same reason as claim 1 above. Additionally, the combination of Mnih and Cramer further teaches:
A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising (The invention also provides processor control codes and/or data (for example learnt weights) to implement embodiments of the invention, in particular on a physical (non-transitory) data carrier such as a disk, programmed memory, for example on non-volatile memory such as Flash or in Firmware. Code and/or data to implement embodiments of the invention may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or code for a hardware description language. [see Mnih, Col. 6, lines 52–61]);
wherein each of the plurality of generated sequential patterns describes a corresponding sequence of system interaction events and ends with a stop action (Mnih discloses that the states are defined by image data as a sequence of images [see Mnih, Col. 5, lines 20–21] which is obtained from a system emulating the game frames [see Mnih, Col. 11, lines 1–4], and that the states are included in the recorded sequences [see Mnih, Col. 11, lines 29–31]);
for each generated sequential pattern of the plurality of generated sequential patterns:
determining, based on the at least one recorded sequence and a utility measure that is not anti-monotonic, a reward for the generated sequential pattern (Mnih discloses the data processor to implement [see Mnih, Col. 15, line 66–Col. 16, line 1] the agent interacts with the environment E in a sequence of actions, observations and rewards to select an action at each time-step [see Mnih, Col. 8, lines 64–67]. Mnih further discloses storing the before and after states, the action taken and the reward taken as experience data [see Mnih, Col. 11, lines 28–52]; In addition it receives a reward r.sub.t representing the change in game score. Note that in general the game score may depend on the whole prior sequence of actions and observations [see Mnih, Col. 9, lines 6–9]).
Regarding claim 16, claim 16 contains substantially similar limitations to those found in claim 11. Therefore it is rejected for the same reason as claim 11 above. Additionally, the combination of Mnih and Cramer further teaches A system comprising:
for each generated sequential pattern of the plurality of generated sequential patterns:
obtain, from among a plurality of recorded sequences of system interaction events stored in a data store, at least one recorded sequence that matches the generated sequential pattern (Mnih discloses the data processor to implement [see Mnih, Col. 15, line 66–Col. 16, line 1] the agent interacts with the environment E in a sequence of actions, observations and rewards to select an action at each time-step [see Mnih, Col. 8, lines 64–67]. Mnih further discloses storing the before and after states, the action taken, and the reward taken as experience data and drawing a transition from the stored experience to the 'after' state of the transition [see Mnih, Col. 11, lines 28–52] at each time-step in a data-set pooled over many episodes into a replay memory which can be sampled for experience replay during training [see Mnih, Col. 13, lines 53–65]; As previously mentioned, in preferred embodiments of the approach an experience data store records experience data for some or all of the actions taken. [see Mnih, Col. 4, lines 4–6]);
an interface-modification server (Cramer discloses software on a server to produce updating rankings of search results wherein each set of recommendations displayed are displayed in a different area to set them apart from previously displayed results [see Cramer, Col. 2, lines 45–65].);
Regarding claims 12 and 17, claims 12 and 17 contains substantially similar limitations to those found in claim 2 above. Consequently, claims 12 and 17 are rejected for the same reasons.
Regarding claims 13 and 18, claims 13 and 18 contains substantially similar limitations to those found in claim 4 above. Consequently, claims 13 and 18 are rejected for the same reasons.
Regarding claims 14 and 19, claims 14 and 19 contains substantially similar limitations to those found in claim 5 above. Consequently, claims 14 and 19 are rejected for the same reasons.
Claims 8, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Mnih et al. (US 9,679,258 B2), hereinafter Mnih, in view of Cramer (US 8,117,197 B1), hereinafter Cramer, as applied in claim 1 above, and further in view of Joy et al. (US 2009/0037403 A1), hereinafter Joy.
Regarding claim 8, the combination of Mnih and Cramer as applied in claim 1 above teaches all the limitations of claim 1 and further teaches:
wherein extracting the at least one high-value sequential pattern from the trained machine learning model comprises searching the trained machine learning model (Cramer discloses software on a server to produce updating rankings of search results wherein each set of recommendations displayed are displayed in a different area to set them apart from previously displayed results [see Cramer, Col. 2, lines 45–65]. Cramer further discloses a real-time model for ranking search results that takes into account recent user actions for the recommended results [see Cramer, Col. 8, line 52–Col. 9, line 7]).
However, the combination of Mnih and Cramer fails to teach searching the trained machine learning model using a depth-first-search algorithm.
In the same field of endeavor, Joy teaches:
searching the trained machine learning model using a depth-first-search algorithm (The search engine can rank search results or employ in its search various heuristics [see Joy, para. 24]; According to FIG. 4, a server computing device 400 receives a query string 402, and produces ranked search results 422. … The Interpretation Finder component can apply various search techniques (e.g., depth first search), and can employ various heuristics (e.g., guided search based on spatial overlap of matched attribute footprints from the Spatial Footprint Index 413) to make the search for interpretations more efficient. [see Joy, para. 33]).
It would have been obvious to one of ordinary skill, in the art at the time before the effective filing date of the invention to incorporate searching the trained machine learning model using a depth-first-search algorithm as suggested in Joy into the combination of Mnih and Cramer because both methods have a search engine to produce ranked results (see Cramer, Col. 5, lines 2–9; see Joy, para. 24). Incorporating the techniques of Joy into the combination of Mnih and Cramer would make the search for interpretations more efficient (see Joy, para. 33).
Regarding claims 15 and 20, claims 15 and 20 contains substantially similar limitations to those found in claim 8 above. Consequently, claims 15 and 20 are rejected for the same reasons.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Mnih et al. (US 9,679,258 B2), hereinafter Mnih, in view of Cramer (US 8,117,197 B1), hereinafter Cramer, as applied in claim 9 above, and further in view of Ritter et al. (US 11,423,300 B1), hereinafter Ritter.
Regarding claim 10, the combination of Mnih and Cramer as applied in claim 9 above teaches all the limitations of claim 9 and further teaches:
wherein the machine learning model includes a Q-network (In a related aspect the invention provides a method of Q-learning wherein Q values are determined by a neural network and used to select actions to be performed on a system to move the system between states, wherein a first neural network is used to generate a Q-value for a target for training a second neural network used to select said actions. [see Mnih, Col. 6, lines 14–19]; Thus, the first neural network can be Q-network used to generate the Q-value for the second neural network).
However, the combination of Mnih and Cramer fails to teach wherein the machine learning model includes a long short-term memory network.
In the same field of endeavor, Ritter teaches:
wherein the machine learning model includes a long short-term memory network (Generally, the action selection network 112 can have any appropriate neural network architecture. In this specification, the action selection network 112 should be understood to be a recurrent neural network. ... In one example, the action selection network 112 may be a long short-term memory (LSTM) recurrent neural network [see Ritter, Col. 7, lines 30–42]).
It would have been obvious to one of ordinary skill, in the art at the time before the effective filing date of the invention to incorporate wherein the machine learning model includes a long short-term memory network as suggested in Ritter into the combination of Mnih and Cramer to teach wherein the machine learning model includes a long short-term memory network and a Q-network because both methods perform Q-learning for (see Mnih, Col. 6, lines 14–19; see Ritter, Col. 7, lines 45–55). Thus, the first neural network can be a Q-network as taught by Mnih, and the second neural network can be the LSTM network taught by Ritter. Incorporating the techniques of Ritter into the combination of Mnih and Cramer would enable the system to revert back to a previously learned action selection policy (i.e., based on the remembered value of the hidden state of the action selection network) rather than relearning information about the task (see Ritter, Col. 3, lines 36–40).
Response to Arguments
Applicant’s arguments, filed 11/25/2025, traversing the rejections of claims 1-20 under 35 U.S.C. 112(b) have been fully considered and are persuasive, the rejections are respectfully withdrawn.
Applicant’s arguments, filed 11/25/2025, traversing the rejection of claims 1-20 under 35 U.S.C. 101 have been fully considered and are not persuasive. Applicant argues that the claims do not recite an abstract idea, and further even if they recite an abstract idea that the claim practically integrates the judicial exception into a practical application, Examiner respectfully disagrees.
Applicant argues, on pg. 8-10, that the human mind cannot possibly perform the steps recited in claims 1, 11, and 16, nor are claims 1, 11, and 16 directed to mathematical concepts. Specifically, Applicant argues the human mind cannot train a machine learning model to learn a state-action map that contains high-value sequential patterns. However, while the claim limitation recites training a machine learning model, the claims are still directed to the abstract idea of learning a state-action map that contains a high-value sequential pattern which is reasonably performable in the human mind. Thus, the training step is directed to a machine learning model performing an abstract idea that is reasonably performable in the human mind, and thus, is directed to an abstract idea. Applicant further argues that the training feature also includes for each generated sequential pattern, determining a reward for the generated sequential pattern and updating the machine learning model based on the reward. However, determining a reward is also an abstract idea that can be performed in the human mind, or a mathematical concept. Further, updating the machine learning model based on the reward is also a mathematical concept as a reward is a mathematical value which is used to organize and manipulate the machine learning model through mathematical correlations. Applicant further argues that generating a plurality of sequential patterns wherein each of the sequential patterns describes a corresponding sequence of system interaction events is not directed to a mathematical concept. However, one of the ordinary skill in the art before the effective filing date would understand that a sequential pattern of interaction events would be a time ordered sequence as defined per para. 24 of the specification. Thus, the sequential pattern organizes and manipulates a system through mathematical correlations, specifically, through actions that are ordered by the time at which they are taken, such that one action is taken after another sequentially.
Applicant argues, on pg. 10-11, that claims 1, 11, and 16 integrate the abstract idea into a practical application, and cites to para. 16-18 of the specification. Para. 17 is pertinent to reconfigure or modify a user interface to an interactive computing environment to operate more efficiently and/or to optimize one or more identified performance metrics, with specific modifications outlined to increase network throughput, or reduce network bandwidth consumption. However, with respect to the broad assertion of reconfiguring/modifying a user interface to operate more efficiently and/or to optimize one or more performance metrics, just an assertion of an improvement without a technical explanation of how the improvement is realized is insufficient to practically integrate the judicial exception (see MPEP 2106.05(a)). Further, the specific modifications that lead to the improved network throughput and bandwidth consumption are not reflected in the claims, there is no limitation to reflect modifying a website to reduce average response time, nor a limitation to reflect to reduce the number of web pages a user may visit. Thus, the claim does not reflect the purported improvement to the function of a machine.
For at least the aforementioned reasons, the independent claims recite abstract ideas and do not practically integrate the judicial exceptions, thus, the rejection of claims 1-20 under 35 U.S.C. 101 is respectfully maintained.
Applicant’s arguments, filed 11/25/2025, traversing the rejection of claims 1-20 under 35 U.S.C. 103 have been fully considered and are not persuasive. Applicant argues that none of the cited references disclose or make obvious "extracting, by a pattern-extracting server, at least one high-value sequential pattern from the trained machine learning model" or "causing a user interface of a computing environment to be modified based on information from the at least one high-value sequential pattern", Examiner respectfully disagrees.
With respect to the extracting step, Applicant’s argument that Cramer fails to teach high-value sequential patterns is persuasive, however, Mnih renders obvious extracting high-value sequential patterns as the machine learning model can extract high-value actions (actions which maximize the expected reward) to form sequential patterns of actions thus forming high-value sequential patterns. Further, while Mnih fails to teach a pattern-extracting server and instead discloses a pattern-extracting system, Cramer teaches a result ranking server; thus, It would have been obvious to one of ordinary skill, in the art at the time before the effective filing date of the invention that the pattern extracting system of Mnih could be implemented on a server, such as the result ranking server disclosed by Cramer. Further, the high-value sequential patterns of Mnih are also time-ordered as each action is taken sequentially with the previous actions and observations used as input for predicting the current action to take.
With respect to the user interface, Cramer discloses causing a user interface of a computing environment to be modified based on information from the results (i.e., ranking the results on the user interface), which could be combined with the resulting sequential patterns of Mnih to rank sequential patterns instead of search results. Applicant argues that Cramer does not disclose modifying a user interface of a computing environment, however, Cramer discloses the user interface running on a computer (see Cramer, Col. 2, lines 45-40) which is a computing environment. Applicant further argues that ranking recommended results is different than promoting the sequential pattern. However, the combination of Mnih and Cramer would rank sequential patterns, which would in turn promote the sequential patterns based on their high-values (i.e., their rewards). Thus, the promoted sequential pattern being the sequential pattern which is ranked the highest.
For at least the aforementioned reasons, the independent claims are obvious in view of Mnih and Cramer and thus the rejections of claims 1-20 under 35 U.S.C. 103 are respectfully maintained.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
VAN DE KERKHOF JAN et al. (WO 2020091829 A1) teaches a machine learning model which could be an LSTM or deep Q-network, may take as input sequential actions taken by a user to detect or predict sequential application usage, and a reinforcement learning model that learns actions in an environment to maximize rewards.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAKE BREEN whose telephone number is (571)272-0456. The examiner can normally be reached Monday - Friday, 7:00 AM - 3:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.B./Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143