DETAILED ACTION
Status of Claims
Applicant has amended claims 1, 4-5, 10-12, 14, 16-17 and 21-22 . No claims have been added. Claim 8-9 and 19-20 have been canceled. Claims 3, 6-7, 15 and 18 were canceled prior to previous office action. Thus, claims 1-2, 4-5, 10-14, 16-17 and 21-22 remain pending in this application. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments and amendments filed on 11 May 2026 with respect to:
objection to claims 1, 2, 4, 5, 8-14, 16, 17 and 19-22,
rejections of claims 1, 10 and 21 under U.S.C. § 112(a),
rejections of claims 1, 2, 4, 5, 8-14, 16, 17 and 19-22 under U.S.C. § 112(b),
rejection to claims 1, 2, 4, 5, 8-14, 16, 17 and 19-22 under U.S.C. § 101,
have been fully considered. Amendments to claims have been entered.
Examiner acknowledges amendments to claims to overcome claim objections and 35 U.S.C. § 112(a) and § 112(b) rejections. However, amendments are not totally effective.
Applicant argues terms such as “defining an action-blocking environmental condition” and “determining whether the current environmental state data satisfies the action-blocking environmental condition” [remarks page 7] remain as broad terms which do not clearly convey an inventive concept. “Blocking something from happening” does not clearly convey a method step. Further, in view of subject matter eligibility, such “actions” remain abstract.
Applicant presents examples of application of the automated agent as including self-driving vehicles and trading securities vis-à-vis “trading securities” [remarks page 7]. However, as recited in the previous office action:
“the specification discusses, inter alia, “limit price”, “market price”, “trading platform” and “tasks to be performed in relation to securities (e.g., stocks, bonds, options or other negotiable financial instruments)”, “market spread features”, “market quote” and “normalized market trade”. However, the claim language is devoid of any such language. Examiner suggests amending claims to recite something along the lines of ‘securities trading’ ’” (emphasis added).
Examiner acknowledges amendments to, and arguments regarding claims to overcome 35 U.S.C. § 101 rejection. However, arguments are not persuasive.
Applicant argues subject matter eligibility under Step 2A – Prong One contending that the claims are not directed to the abstract idea of disabling a training cycle, grouped under a fundamental economic practice of mitigating risk, but are directed to a specific manner of improving training of a reinforcement learning neural network of an automated agent [remarks page 8]. However, Examiner maintains, as discussed supra, the claim language is sufficiently broad to be interpreted as a fundamental economic practice of mitigating risk.
Applicant argues subject matter eligibility under Step 2A, Prong 2, in that the claims are integrated into a practical application of a judicial exception because the claimed subject matter provides an improvement to the training of a reinforcement learning neural network which in turn provides improvements in the operation of the automated agent that includes the reinforcement neural network [remarks page 9]. Examiner respectfully disagrees in as much as
“to learn from rewards provided to the reinforcement learning neural network and determines and performs actions within an environment under exploration by the automated agent during one or more of the training cycles”
Is at best a statement of intended use which does not further limit the claim. Terminology such as “rewards” and “an environment under exploration” is not definitive: the metes and bounds of these terms are not clear. As such, a practical application can not be conveyed.
Applicant argues subject matter eligibility analogizing the claims of Desjardins with the present claims which provide an improvement to a method of training a machine earning model and therefore are directed to improvements to a technology or technical field [remarks page 10]. Applicant further analogizing the present claims with those of claims of Desjardins, in that the present claims provide improvements as to how the reinforcement learning neural network itself operates, since the reinforcement learning neural network can generate better action signals, by being trained on less noisy data [remarks page 10]. Examiner respectfully disagrees as discussed supra.
Applicant makes similar arguments regarding subject matter eligibility in the light of Step 2B contending that the claims contain “significantly more” in that:
“in response to determining the action-blocking environmental condition is satisfied, generating a disable signal causing training of the automated agent to be disabled for at least the given training cycle when the action is not taken due to the action-blocking environmental condition; and upon receiving the disable signal, disabling processing of a reward for the reinforcement learning neural network for the given training cycle or disabling providing the current environmental state data to the reinforcement learning neural network for the given training cycle, thereby reducing noise in the training of the reinforcement learning neural network”
[remarks page 10]. Examiner respectfully disagrees in that receiving data, determining a “next” action, determining whether some “data” satisfies “some” condition, and disabling “processing something” are recited at a high level of generality, such that they do not convey an improvement to technology
As stated in previous office action, as with determining a practical application to an abstract idea, types of limitations indicative of an inventive concept (aka “significantly more”) – subject matter eligibility under Step 2B - include:
Improvements to the functioning of a computer, or to any other technology or technical field - see MPEP 2106.05(a)
Applying the judicial exception with, or by use of, a particular machine - see MPEP 2106.05(b),
Effecting a transformation or reduction of a particular article to a different state or thing - see MPEP 2106.05(c),
Applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception - see MPEP 2106.05(e) and Vanda Memo
Further, limitations also indicative of an inventive concept include:
Adding a specific limitation other than what is well-understood, routine, conventional activity in the field - see MPEP 2106.05(d).
Examiner maintains that the claimed invention does not contain any of these “types” of aforementioned limitations.
Limitations that are not indicative of an inventive concept include:
Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f),
Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g),
Generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h),
Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception - see MPEP 2106.05(d) and Berkheimer Memo.
Examiner maintains that the claimed invention merely appends well-understood, routine, conventional activities previously known to the industry. Such activities are recited at a high level of generality, such that they do not convey an improvement to technology.
Rejections have been clarified herein in view of the claim amendments and the January 2019 Patent Subject Matter Eligibility Guidance – 2019 PEG.
Priority
This application, filed on 23 August 2022 claims priority from provisional application 63/236,429, filed on 24 August 2021.
Claim Objections
Claims 1-2, 4-5, 10-14, 16-17 and 21-22 are objected to because the claim language is so broad that it is not clear what the Applicant tis claiming as his invention.
For example, the specification discusses, inter alia, “limit price”, “market price”, “trading platform” and “tasks to be performed in relation to securities (e.g., stocks, bonds, options or other negotiable financial instruments)”, “market spread features”, “market quote” and “normalized market trade”. However, the claim language is devoid of any such language. Examiner suggests amending claims to recite something along the lines of “securities trading”.
Claims 1, 10 and 21 are objected to because the representative limitation:
determine whether the current environmental state data satisfies the action-blocking environmental condition
It is not clear what the metes and bounds are of “determining whether the current environmental state data satisfies the action-blocking environmental condition”.
Correction is required. See related § 112 rejections.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 1, 10 and 21 rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor at the time the application was filed, had possession of the claimed invention.
Regarding claims 1, 10 and 21, the representative limitation recites:
receive an input defining an action-blocking environmental condition during which a condition of the environment prevents the automated agent from taking an action;
However, Applicant's specification does not say what defines an action-blocking environmental condition.
For example, the Applicant’s specification does recite of “determining a next action” such as in paragraph [0013] and [0024] of US Pub. 20230061752 A1 of the Applicant’s specification. Paragraph [0024] recites:
[0024] In another aspect, there is provided a non-transitory computer-readable storage medium storing instructions which when executed adapt at least one computing device to: instantiate an automated agent that includes a reinforcement learning neural network that is trained over a plurality training cycles and provides a policy for generating resource task requests; detect a learning condition that is expected to impede training of the automated agent during a given training cycle of the plurality of training cycles; and in response to the detecting, generating a disable signal to disable training of the automated agent for at least the given training cycle. (emphasis added)
However, detecting something that is “expected to impede something” is vague and indefinite in as much as the term “expected" is a relative term. “Expected” does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
Regarding claims 1, 10 and 21, the representative limitation recites:
determine whether the current environmental state data satisfies the action-blocking environmental condition;
However, Applicant's specification does not say what satisfies an action-blocking environmental condition. For example, paragraph [0085] of US Pub. 20230061752 A1 of the Applicant’s specification recites:
[0085] An automated agent 180 is restricted from taking actions in accordance with its policy when a user imposes a restriction on the automated agent 180. In one example, a user may substitute its own action selection in place of the action selection of the automated agent 180. In one example, a user may substitute its own policy in place of the policy of the automated agent 180. In one example, a user may prohibit the automated agent 180 from taking a certain action or prohibit the automated agent 180 from selecting a certain parameter value of an action. In one example, a user may prohibit the automated agent 180 from entering a certain portion of the environment being explored by the automated agent 180.
However, “being restricted from taking actions” does not convey “how” to determine satisfying an action-blocking environmental condition. Therefore, Applicant has not described to one of ordinary skill how Applicant intended the function of "determining whether the current environmental state data satisfies the action-blocking environmental condition" is to be performed. As per MPEP:
MPEP 2163 Guidelines for the Examination of Patent Applications Under the 35 U.S.C. 112(a) or Pre-AIA 35 U.S.C. 112, first paragraph, "Written Description" Requirement [R-10.2019]
35 U.S.C. 112(a) and the first paragraph of pre-AIA 35 U.S.C. 112 require that the "specification shall contain a written description of the invention ...." This requirement is separate and distinct from the enablement requirement. Ariad Pharm., Inc. v. Eli Lilly & Co., 598 F.3d 1336, 1340, 94 USPQ2d 1161, 1167 (Fed. Cir. 2010) (en banc); Vas-Cath, Inc. v. Mahurkar, 935 F.2d 1555, 1560, 19 USPQ2d 1111, 1114 (Fed. Cir. 1991); see also Univ. of Rochester v. G.D. Searle & Co., 358 F.3d 916, 920-23, 69 USPQ2d 1886, 1890-93 (Fed. Cir. 2004) (discussing the history and purpose of the written description requirement); In re Curtis, 354 F.3d 1347, 1357, 69 USPQ2d 1274, 1282 (Fed. Cir. 2004) ("conclusive evidence of a claim’s enablement is not equally conclusive of that claim’s satisfactory written description"). The written description requirement has several policy objectives. "[T]he ‘essential goal’ of the description of the invention requirement is to clearly convey the information that an applicant has invented the subject matter which is claimed." In re Barker, 559 F.2d 588, 592 n.4, 194 USPQ 470, 473 n.4 (CCPA 1977). Another objective is to convey to the public what the applicant claims as the invention. See Regents of the Univ. of Cal. v. Eli Lilly, 119 F.3d 1559, 1566, 43 USPQ2d 1398, 1404 (Fed. Cir. 1997), cert. denied, 523 U.S. 1089 (1998). "The ‘written description’ requirement implements the principle that a patent must describe the technology that is sought to be patented; the requirement serves both to satisfy the inventor’s obligation to disclose the technologic knowledge upon which the patent is based, and to demonstrate that the patentee was in possession of the invention that is claimed." Capon v. Eshhar, 418 F.3d 1349, 1357, 76 USPQ2d 1078, 1084 (Fed. Cir. 2005). Further, the written description requirement promotes the progress of the useful arts by ensuring that patentees adequately describe their inventions in their patent specifications in exchange for the right to exclude others from practicing the invention for the duration of the patent’s term.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-2, 4-5, 10-14, 16-17 and 21-22 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1, 10 and 21, the representative limitation:
determine whether the current environmental state data satisfies the action-blocking environmental condition;
is vague and indefinite for several reasons. First, it is not clear what the metes and bounds are of “determining whether the current environmental state data satisfies the action-blocking environmental condition”. Second, satisfying “something” is not a clear method step.
Claims 2, 4-5, 11-14, 16-17 and 22 are rejected by way of dependency on a rejected independent claim.
The art rejections below are in view of the 112(b) rejections stated above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4-5, 10-14, 16-17 and 21-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
In the instant case, claims 1, 2, 4, 5 and 22 are directed to a “system” which is one of the four statutory categories of invention. Claims are directed to the abstract idea of disabling a training cycle which is grouped under a fundamental economic practice of mitigating risk.
in prong one of step 2A (See 2019 Revised Patent Subject Matter Eligibility Guidance (Federal Register, Vol. 84, No. 5, p.p. 50-57 (Jan. 7, 2019))). Claims recite:
instantiate an automated agent that includes the reinforcement learning neural network;
receive an input defining an action-blocking environmental condition during which a condition of the environment prevents the automated agent from taking an action;
obtain, during a given training cycle of the plurality of cycles, current environmental state data;
determine a next action for the agent, based in part on the current environmental state data;
determine whether the current environmental state data satisfies the action-blocking environmental condition;
in response to determining the action-blocking environmental condition is satisfied,generate a disable signal causing training of the automated agent to be disabled for at least the given training cycle when the action is not taken due to the action-blocking environmental condition; and
upon receiving the disable signal, disable processing of a reward for the reinforcement learning neural network for the given training cycle or disable providing the current environmental state data to the reinforcement learning neural network for the given training cycle.
Limitations such as:
that is trained over a plurality training cycles to learn from rewards provided to the reinforcement learning neural network and determines and performs actions within an environment under exploration by the automated agent during one or more of the training cycles
are merely a description of data and do not impose any meaningful limit on the computer implementation of the abstract idea.
Limitation such as:
thereby reducing noise in the training of the reinforcement learning neural network
are merely statements of intended use and do not impose any meaningful limit on the computer implementation of the abstract idea.
Accordingly, the claim recites an abstract idea (See 2019 Revised Patent Subject Matter Eligibility Guidance).
This judicial exception is not integrated into a practical application because, when analyzed under prong two of step 2A (See 2019 Revised Patent Subject Matter Eligibility Guidance), the additional elements of the claim such as
a communication interface,
at least one processor, and
memory in communication with the at least one processor and software code stored in the memory executing the software,
represent the use of a computer as a tool to perform an abstract idea and/or do no more than generally link the abstract idea to a particular field of use. Similarly, “determining a next action for the agent, based in part on the current environmental state data” is a broad statement which also represents the use of a computer as a tool to perform an abstract idea. Therefore, the additional elements do not integrate the abstract idea into a practical application as they do no more than represent a computer performing functions that correspond to (i.e. automate) the acts of “collecting information, analyzing the information and providing the results of the analysis”.
When analyzed under step 2B (See 2019 Revised Patent Subject Matter Eligibility Guidance), the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception itself because the ordered combination does not offer substantially more than the sum of the functions of the elements when each is taken alone.
The computer and computer program instructions are recited at a high level of generality and are recited as performing generic computer functions routinely used in computer applications. The elements together execute in routinely and conventionally accepted coordinated manners and interact with their partner elements to achieve an overall outcome which, similarly, is merely the combined and coordinated execution of generic computer functionalities. These functionalities are well-understood, routine and conventional activities previously known to the industry. Therefore, the use of these additional elements does no more than employ a computer as a tool to automate and/or implement the abstract idea, which cannot provide significantly more than the abstract idea itself (MPEP 2106.05(I)(A)(f) & (h)).
Thus, viewed as a whole, the combination of elements recited in the claims merely describe the concept of generating a disable signal using computer technology (e.g. the processor).
Hence, claims are not patent eligible.
Dependent claims 2, 4-5 and 22 when analyzed as a whole are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to a judicial exception (Step 2A- Prong One). Nor are the claims directed to a practical application to a judicial exception (Step 2A- Prong Two).
Accordingly, none of the dependent claims add a technological solution to the fundamental economic practice in the independent claim.
Note: The analysis above applies to all statutory categories of invention. As such, the presentment of claims 10-14 and 16-17 otherwise styled as a method, and claim 21 styled as a non-transitory computer readable storage medium would be subject to the same analysis.
Conclusion
The claims as a whole do not amount to significantly more than the abstract idea itself. This is because the claims do not affect an improvement to another technology or technical field; the claims do not amount to an improvement to the functioning of a computer system itself; and the claims do not move beyond a general link of the use of an abstract idea to a particular technological environment.
Accordingly, there are no meaningful limitations in the claims that transform the judicial exception into a patent eligible application such that the claims amount to significantly more than the judicial exception itself.
Additional Comments
Regarding claims 1-2, 4-5, 10-14, 16-17 and 21-22, in view of pending rejections, the Examiner is unable to locate prior art references that anticipate the claimed invention or renders it obvious.
Conclusion
The prior art of record and not relied upon is considered pertinent to Applicant’s disclosure:
GETTINGS et al: “ENVIRONMENTAL MONITORING DEVICE”, (US Pub. No. 20150310720 A1).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDWARD J BAIRD whose telephone number is (571)270-3330. The examiner can normally be reached 7 am to 3:30 pm M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Donlon can be reached at 571-270-3602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EDWARD J BAIRD/Primary Examiner, Art Unit 3692
Prosecution Insights