DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-3 and 5-7 have been amended. Claims 1-8 are pending and have been examined.
Response to Amendment
In light of the amendments made to claim 3, the rejection under 35 USB 112(b) has been withdrawn.
The amendments made on claims 1-3 and 5-7 do not overcome the rejection under 35 USC 101. See updated rejection below.
Response to Arguments
Applicants’ arguments on pages 7-8 regarding the rejection under 35 USB 112(a) have been fully considered and are persuasive. The rejection under 35 USB 112(a) has been withdrawn.
Applicant’s arguments regarding the rejection under 35 USC 101 have been fully considered but are not persuasive.
Regarding applicant’s arguments on pages 8-9:
While individual steps (e.g., predicting, inferring) may involve abstract concepts, the claim as a whole is not directed to an abstract idea. The invention integrates a multi-character RL model with episodic future thinking to predict future states and decide behaviors, improving multi-agent decision-making (title, abstract, [0005]-[0006], [0055]-[0060], Fig. 5). This is a technical solution, not a mental process, as it requires processor-executed RL algorithms.
Examiner’s response:
Examiner agrees that the steps include abstract concepts. As stated in the rejection, predicting behavior of an agent is a mental process. A person could look at the behavior or any available information of an agent (which paragraph 36 of the specification indicates could be humans) and predict a future behavior of a particular agent relative to other agents or elements in its environment. There is nothing in the claims themselves that foreclose them from being performed mentally. Therefore, any improvement would be in the abstract idea, not on the functioning of a computer or in any other technological field. The argument conflates a mental process performed using a computer (processor-executed RL models) with a technological solution. Many mental processes may be performed using a computer but are not technological.
In addition, claiming the use of processor-executed RL algorithms to implement the abstract idea of predicting behavior of an agent is not sufficient to provide a practical application or amount to significantly more than the abstract idea. Predicting behavior of an agent is not an improvement on RL algorithms. As per MPEP 2106.04(a)(2)(III)(C)), a claim that requires a computer may still recite a mental process. In the instant case, the computer is merely used as a tool to implement the mental process. In the instant case, the computer is merely used as the tool to implement the abstract idea.
Regarding applicant’s arguments on page 9:
Step 2A, Prong 2: The claims integrate the judicial exception into a practical application. The amendments specify using a trained RL model, classical physics for inference, and decision-making to improve safety by avoiding collisions, validated by higher reinforcement learning rewards (Fig. 5, [0058]-[0060]). This enhances computer functionality in dynamic, multi- agent environments (e.g., autonomous vehicles, [0004], [0053]), not merely applying an abstract idea. Data collection is integral, not extra-solution, as it feeds the RL model. The units (e.g., behavior predicting unit) are not generic but specialized for this process, and "agents" define a technological context, not just a field of use (Diamond v. Diehr, 450 U.S. 175, 192 n.14).
Examiner’s response:
Examiner respectfully disagrees. Decision making is a mental process. Improving safety is not an improvement in the functionality of a computer or in any other technological field. Any improvement would be in the abstract idea of decision making. It is also noted that the claim does not define the agents as autonomous vehicles. It is further noted that paragraph 36 of the specification indicates that the agents could be humans.
Regarding the “data collection” argument, the claim does not describe any particular way of collecting the data in such a way that is not a generic way of gathering data. Therefore, it is insignificant extra-solution activity since every computer program must be fed with the data to process.
As to the argument regarding the agents, the claims do not define these agents to any technological context. Therefore, these agents could be anything, including humans as per paragraph 36 of the specification.
Regarding applicant’s arguments on page 9:
Step 2B: The claims include an inventive concept. The combination of character
inference (via maximum likelihood) with episodic future thinking is novel and non-obvious, as evidenced by the performance gains over prior art (Fig. 5, [0058]). This is not a well-understood, routine, or conventional application of RL, especially without cited prior art. Dependent claims (e.g., Claim 3's equation) add specific, non-abstract implementations.
Examiner’s response:
Examiner respectfully disagrees. The reinforcement learning model was not rejected as well understood ,routine or conventional activity, it has been rejected as a generic computer component to implement the abstract idea. Applicant has not provided any evidence that the RL model is not generic. In addition just because there is no art applied does not mean that the claims are eligible under 35 USC 101. The question of whether a particular claimed invention is novel or obvious is "fully apart" from the question of whether it is eligible. Diamond v. Diehr, 450 U.S. 175, 190, 209 USPQ 1, 9 (1981).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-8 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 was amended to include: “controlling an operation of the main agent to avoid collision of the main agent with the surrounding agent based on a result of the deciding the behavior of the main agent such that the behavior of the main agent is decided to improve safety in a multi- agent environment by considering predicted future collisions, as validated by reinforcement learning rewards. There is no support in the specification for this limitation. Therefore, this limitation constitute new matter.
Claims 2-8 depend from claim 1 but fail to cure the deficiency set forth above. They are rejected on the same basis.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101 because they are directed to an abstract idea without significantly more.
Step 1 analysis for all claims:
In the instant case, claims 1-4 are directed to a method (process) and claims 5-8 are directed to an apparatus (manufacture). Thus, each of the claims falls within one of the four statutory categories of invention.
Regarding Claim 1:
Step 2A, Prong 1 analysis:
The claim(s) recite(s) in part:
determining a character coefficient of the surrounding agent using the maximum likelihood method by inputting the observation information of the surrounding agent. As drafted, this limitation encompasses a mathematical concept.
predicting a behavior of the surrounding agent by inputting the observation information of the main agent and the surrounding agent at the first time point and the character coefficient of the surrounding agent. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and. For example, this limitation encompasses analyzing the observation information of the main agent and the surrounding agent along with the character coefficient to predict the action that the surrounding agent should or will take.
inferring expected observation information of the main agent including the surrounding agent and the environment at a second time point corresponding to the behavior prediction result of the surrounding agent to estimate positions and speeds at the second time point. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and. For example, this limitation encompasses analyzing the prediction obtained to determine factors (observation information) that could lead to a future action and evaluating that to estimate the position and the speed of the agent at a future point in time.
deciding a behavior of the main agent at the first time point to maximize a reward by avoiding a collision indicated by the expected observation information of the main agent including the surrounding agent and the environment at the second time point. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and. For example, this limitation encompasses deciding what action the main agent should take based on what is expected to happen in the future.
Step 2A, Prong 2 analysis:
The claim recites in part:
collecting observation information and behavior information of a surrounding agent. As drafted, this limitation amounts to extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself and cannot integrate a judicial exception into a practical application.
collecting observation information of a main agent and the surrounding agent at a first time point. As drafted, this limitation amounts to extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself and cannot integrate a judicial exception into a practical application.
The “one or more processors”, “first information collecting unit”, “character inferring unit”, “second information collecting unit”, “by inputting…to the multi-character reinforcement learning model trained in advance” and “behavior predicting unit” are computer elements for performing the limitations which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
The “main agent” and “surrounding agent” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate the judicial exception into a practical application (MPEP § 2106.05(h)).
controlling an operation of the main agent to avoid collision of the main agent with the surrounding agent based on a result of the deciding the behavior of the main agent such that the behavior of the main agent is decided to improve safety in a multi- agent environment by considering predicted future collisions, as validated by reinforcement learning rewards. This limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)). In addition, this limitation also amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (the claimed agent). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, Prong 2 the additional elements individually or in combination do not integrate a judicial exception into a practical application.
Step 2B:
As discussed above, the additional elements of “collecting observation information and behavior information of a surrounding agent” and “collecting observation information of a main agent and the surrounding agent at a first time point” amount to extra-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
As discussed above, the “first information collecting unit”, “character inferring unit”, “second information collecting unit” and “behavior predicting unit” are computer elements for performing the limitations which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
As discussed above, the “main agent” and “surrounding agent” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself (MPEP § 2106.05(h)).
As discussed above, controlling an operation of the main agent to avoid collision of the main agent with the surrounding agent based on a result of the deciding the behavior of the main agent such that the behavior of the main agent is decided to improve safety in a multi- agent environment by considering predicted future collisions, as validated by reinforcement learning rewards. This limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)). In addition, this limitation also amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (the claimed agent). As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the exception itself.
Regarding Claim 2
Step 2A, Prong 1 analysis:
The claim recites in part:
randomly initializing an estimated character coefficient of the surrounding agent. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and. For example, this limitation encompasses thinking of a random number for the character coefficient.
sampling a behavior of the surrounding agent using the estimated character coefficient, the observation information, and a multi-character reinforcement learning model. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and. For example, this limitation encompasses determining a behavior of the surrounding agent by analyzing the estimated character coefficient, the observation information and the result of a multi-character reinforcement learning model.
determining the character coefficient of the surrounding agent by comparing the sampled behavior and the behavior information. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and. For example, this limitation encompasses performing an analysis of the sampled behavior vs the behavior information do determine a character coefficient.
Step 2A, Prong 2 analysis:
The clam recites using a multi-character reinforcement learning model. This limitation is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Step 2B
As set forth above, the clam recites using a multi-character reinforcement learning model. This limitation is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding Claim 3
Step 2A, Prong 1 analysis:
The claim recites “wherein the estimated character coefficient of the surrounding agent is updated by the following Equation 1…”. This limitation encompasses a mathematical concept. Therefore, the claim is directed to an abstract idea.
Step 2A, Prong 2 analysis:
The claim does not recite any additional elements that integrate the judicial exception into a practical application.
Step 2B
The claim does not recite any additional elements that amount to significantly more than the judicial exception.
Claim 4
Step 2A, Prong 1 analysis:
The claim recites “wherein in the predicting of a behavior of the surrounding agent, the behavior of the surrounding agent is predicted using observation information of the surrounding agent excluding the observation information of the main agent, among observation information”. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and. For example, this limitation encompasses making the prediction of the behavior of the surrounding agent without considering the observation information of the main agent.
Step 2A, Prong 2 analysis:
The claim does not recite any additional elements that integrate the judicial exception into a practical application.
Step 2B
The claim does not recite any additional elements that amount to significantly more than the judicial exception.
Regarding Claim 5
The clam recites substantially the same limitations as claim 1 and is rejected on the same basis. The claim further recites “one or more processors which execute an instruction”.
Step 2A, Prong 2 analysis:
The claim recites “one or mor processors” which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Step 2B
As set forth above, the claim recites “one or mor processors” are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding Claims 6-8
Claims 6-8 recite that same limitations as claims 2-4 and are rejected on the same basis.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Omar F Fernandez Rivas whose telephone number is (571)272-2589. The examiner can normally be reached Mon-Fri 5:30-3:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Wiley can be reached at (571) 272-4150. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128