DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is made final.
This office action is made in response to the amendments filed on April 20 2026.
Claims 1-3, 6-8, and 10-12 have been amended.
Priority
Receipt is acknowledged of certified copies, retrieved on October 28, 2022, of papers required by 37 CFR 1.55.
Response to Amendment
The amendments filed on April 20 2026, has been entered. Claims 1-13 remain pending in the application.
Response to Arguments
Response to Specification/Abstract
Applicant’s arguments, see pages 13-14, filed April 20, 2026, with respect to the amended abstract have been fully considered and are persuasive. The objection of September 11, 2025 has been withdrawn.
Response to Double Patenting
Applicant’s arguments, see page 14, filed April 20, 2026, with respect to the Double Patenting have been fully considered and are persuasive. The rejection of September 11, 2025 has been withdrawn.
Response to 112(b)
Applicant’s arguments, see page 14, filed April 20, 2026, with respect to the 112(b) have been fully considered and are persuasive. The rejection of September 11, 2025 has been withdrawn.
Response to 101
Applicant's arguments filed April 20,2026 have been fully considered but they are not persuasive.
Applicant Argues on pages 14-17
Claims 1-3, 5-11, 13-17, and 19-20 are rejected under 35 USC § 101 because the claimed invention is allegedly being directed to non-statutory subject matter. Applicant respectfully disagrees and requests reconsideration.
The first step ("Step 1") is to determine whether the claim is directed to one of the four patent-eligible subject matter categories. MPEP § 2106. Regarding Step 1, there is no dispute that the claims are directed to a patent-eligible subject matter category of 35 USC § 101.
The Office, however, alleges that the claims do not qualify under the second step ("Step 2") because they are allegedly directed to mental processes.
Applicant respectfully disagrees for at least the following reasons.
Step 2 is to determine whether the claims are wholly directed to subject matter
encompassing a judicially recognized exception (e.g., law of nature, natural phenomena, abstract idea). MPEP. § 2106. Step 2 is divided into two parts that determine whether the claim is directed to a judicial exception ("Step 2A") and, if so, whether the claim recites additional elements that amount to significantly more than the judicial exception ("Step 2B"). Step 2A asks whether a claim is "directed to" a judicial exception. MPEP § 2106. Specifically, under a first prong of Step 2A, the MPEP groups abstract ideas into three enumerated categories: mathematical concepts, certain methods of organizing human activity, and mental processes. Id. A claim is treated as abstract if it falls into one of mathematical concepts, certain methods of organizing human activity, and mental processes. Id. In addition, under a second prong of Step 2A, a claim is not abstract if the alleged abstract idea is integrated into a practical application. Id.
Applicant respectfully submits that claim 1 is not directed to a law of nature, a natural
phenomenon, or an abstract idea under the first prong of Step 2A. With respect to the first prong of Step 2A, the Office asserts that claim 1 is directed to mental processes.
Regarding mental processes, the Federal Circuit held that "claims do not recite a mental process when they do not contain limitations that can practically be performed in the human mind, for instance when the human mind is not equipped to perform the claim limitations." CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 1375-76 (Fed. Cir. 2011) (distinguishing Research Corp. Techs. Inc. v. Microsoft Corp., 627 F.3d 859 (Fed. Cir. 2010)).
Without conceding to the appropriateness of the rejection, claim 1 is amended to recite, in part, "storing decision making history data including a plurality of items each indicating an actual change applied in the past to a train operation diagram under constant parameter values of the train operation schedule;" "computing an objective function which is a linear function of a feature vector whose elements are indices of train operation derived from a train operation diagram, the objective function being generated in advance in accordance with a maximum entropy inverse reinforcement learning method based on the decision making history data;" "outputting as a second target, an optimized train operation diagram obtained by changing the first target by optimization using the objective function so that a value of the objective function is increased within a set of feasible train operation diagrams under constant parameter values of the train operation schedule;" "presenting the second target to a user and accept, from the user, a change instruction regarding the second target;" "outputting as a third target, a train operation diagram obtained by further changing the second target based on the change instruction;" "determining, as decision making history data, actual change information indicating an actual change from the second target to the third target, and add the actual change information to the decision making history data;" and "learning the objective function by maximum entropy inverse reinforcement learning by updating a parameter vector of the objective function so as to increase a likelihood that the decision making history data including the actual change information is generated, the likelihood being defined, for each item of the decision making history data, by a probability of a corresponding train operation diagram which is proportional to an exponential of a value of the objective function and normalized by a sum of exponentials of objective function values for feasible train operation diagrams under the constant parameter values."
Applicant respectfully submits that the claimed features cannot be practically
performed in the human mind, even with a paper and pencil. For example, a human mind cannot practically update a parameter vector to increase a likelihood that is defined by a probability of a corresponding train operation diagram which is proportional to an exponential of a value of the objective function and normalized by a sum of exponentials of objective function values for feasible train operation diagrams under the constant parameter values. Also, the human mind cannot practically compute an objective function which is a linear function of a feature vector whose elements are indices of train operation derived from a train operation diagram, the objective function being generated in advance in accordance with a maximum entropy inverse reinforcement learning method based on the decision making history data. Rather, claim 1 is directed to achieving a specific functional improvement over train schedule scheduling systems.
Examiner Response
The rejection is not maintained merely on the basis that the claim recites a mental process. The claim expressly recites mathematical concepts, including computing a linear objective function, updating a parameter vector, increasing a likelihood, and defining probabilities using exponentials and normalization. Therefore, even assuming the recited calculations are not practically performed in the human mind, claim 1 still recites mathematical concepts under Step 2A, Prong One.
Applicant argues on pages 17-20
Even assuming for the sake of the argument that the claims are directed to an abstract
idea, which Applicant does not concede, Applicant respectfully submits that the alleged abstract idea is integrated into a practical application under the second prong of Step 2A. The MPEP states that the Office should "evaluate integration into a practical application by: (a) identifying whether there are any additional elements recited in the claim beyond the judicial exception; and (b) evaluating those additional elements individually and in combination to determine whether they integrate the exception into a practical application." MPEP § 2106.04. The Office asserts that the other elements of claim 1 (e.g., processor) are "additional elements" that do not integrate the claim into a practical application.
Applicant respectfully disagrees. The claim as a whole integrates the alleged exception into a practical application, as evidenced by the Specification. One way to demonstrate such integration is when the claimed invention improves the functioning of a computer or improves another technology or technical field. MPEP § 2106.04(d)(1).
Also, the memorandum ("Memo") issued on December 5, 2025, following the Appeals
Review Panel decision in Ex Parte Desjardins, Appeal No. 2024-000567 (PTAB Sept. 26, 2025) ("Appeals Review Panel Decision"), states that the Office should evaluate and determine "if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement in the functioning of a computer, or an improvement to other technology or a technical field." Memo, at p. 2. In Ex Parte Desjardins, the specification identified the "technical problem of 'catastrophic forgetting' in continual learning systems." Appeals Review Panel Decision, at p. 7. The specification also noted improvements in "training the machine learning model itself' and described further improvements such as "effectively learn[ing] new tasks in succession whilst protecting knowledge about previous tasks," "us[ing] less of their storage capacity," and enabling "reduced system complexity." Id., at pp. 8-9. It was held that the "claim as a whole ... independent claim 1 ... reflects the improvement." Id., at p. 9. Also, "when evaluating a claim as a whole, examiners should not dismiss additional elements as mere 'generic computer components' without considering whether such elements confer a technological improvement to a technical problem, especially as to improvements to computer components or the computer system." Memo, at p. 4.
Similar to Ex Parte Desjardins, the claimed invention provides an improvement in the functioning of a computer, or an improvement to other technology or a technical field. For example, the Specification describes the technical problem, which is, in the machine learning space, difficulties in collecting sufficient decision-making history data for relearning objective functions by inverse reinforcement learning, which makes it challenging to develop objective functions that accurately capture users' current intentions as circumstances evolve over time. See, e.g., paragraphs [0007]-[0009]. In other words, the objective function learned from past data may deviate from current times and the operator's intentions.
Claim 1, as a whole and when read in light of the Specification, solves this technical
problem by at least presenting the second target to a user and accepting, from the user, a change instruction regarding the second target; outputting, as a third target, a train operation diagram obtained by further changing the second target; and learning the objective function by maximum entropy inverse reinforcement learning. As a result, an objective function can be learned that reflects the user's intention. See id. at paragraphs [0013], [0046], [0070], and [0074]. That is, incorporating the actual difference between the system-optimal schedule and the user-modified schedule into the framework of maximum entropy IRL continuously improves the objective function learned by inverse reinforcement learning or inverse optimization. Therefore, Applicant respectfully submits that claim 1 as a whole is directed to an improvement in the functioning of a computer or an improvement to other technology or a technical field.
Accordingly, for at least the foregoing reasons, claim 1 is not directed to a judicial exception to patentability and is therefore patent-eligible under 35 USC § 101.
Independent claims 6 and 10 each include recitations which are similar, though not
identical, to those of claim 1 discussed above. Accordingly, claims 6 and 10 are also directed to patent-eligible subject matter for similar reasons as claim 1.
Claims 2-5, 7-9, and 11-13 each depend from one of claims 1, 6, and 10. Claims 2-5, 7- 9, and 11-13 are also directed to patent-eligible subject matter at least due to their respective dependencies and the additional features recited therein.
Withdrawal of the pending rejections of these claims is, therefore, respectfully requested.
Examiner Response
The amended claim applies mathematical optimization and maximum entropy inverse reinforcement learning to train operation diagram data. The additional elements recite receiving/storing diagram and history data, presenting/outputting diagram data, accepting user input, and adding user change information to stored history data. These limitations limit the mathematical concepts to the field of train operation scheduling and provide data used in the mathematical learning process, but they do not recite a specific technological improvement to train control hardware, signaling equipment, dispatching infrastructure, computer functionality, or machine learning architecture.
The alleged improvement is that the objective function better reflects the user’s current intention. That may improve the result of the mathematical model, but it does not integrate the mathematical concepts into a practical application Required under Step 2A, Prong Two.
Applicants reliance on Desjardins is also not persuasive. In Desjardins, The claims reflected an improvement to the operation of the machine learning model itself, Such as preserving prior task knowledge and reducing storage or system complexity. Claim one here does not recite a comparable improvement to computer or machine learning system operation; rather, it uses maximum entropy inverse reinforcement learning to update an objective function for train operation diagram data.
Response to 103
Applicant’s arguments with respect to claim(s) 1-13 have been considered but are moot in view of the new ground(s) of rejection necessitated by the amendment(s).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
To determine if a claim is directed to patent ineligible subject matter, the Court has guided the Office to apply the Alice/Mayo test, which requires:
Step 1: Determining if the claim falls within a statutory category.
Step 2A: Determining if the claim is directed to a patent ineligible judicial exception consisting of a law of nature, a natural phenomenon, or abstract idea; and Step 2A is a two prong inquiry. MPEP 2106.04(II)(A). Under the first prong, examiners evaluate whether a law of nature, natural phenomenon, or abstract idea is set forth or described in the claim. Abstract ideas include mathematical concepts, certain methods of organizing human activity, and mental processes. MPEP 2104.04(a)(2). The second prong is an inquiry into whether the claim integrates a judicial exception into a practical application. MPEP 2106.04(d).
Step 2B: If the claim is directed to a judicial exception, determining if the claim recites limitations or elements that amount to significantly more than the judicial exception. (See MPEP 2106).
Claims 1-13 are rejected under 35 U.S.C. 101 because the claimed invention is directed an abstract idea without significantly more.
Step 1: Claims 1-5 are directed to a learning device (a machine), Claims 6-9 are directed to a method (a process), and Claims 10-13 are directed to a non-transitory computer-readable medium (a manufacture). Therefore, claims 1-13 are directed to a process, machine, manufacture or composition of matter.
Regarding Claim 1
Step 2A, Prong 1
Claim 1 recites the following mathematical concepts, that in each case under the broadest reasonable interpretation, covers performance of mathematical relationships, mathematical formulas or equations, and mathematical calculations but for recitation of generic computer components (e.g., “learning device”, “memory”, “ processor”, and “inverse reinforcement learning”) [see MPEP 2106.04(a)(2)(III)].
“compute an objective function which is a linear function of a feature vector whose elements are indices of train operation derived from a train operation diagram, the objective function being generated in advance in accordance with a maximum entropy inverse reinforcement learning method based on the decision making history data” (e.g., calculating a score by applying a linear formula to numerical operation features/indices)
“an optimized train operation diagram obtained by changing the first target by optimization using the objective function so that a value of the objective function is increased within a set of feasible train operation diagrams under constant parameter values of the train operation schedule” (e.g., optimizing/scoring candidate train operation diagrams)
“learn the objective function by maximum entropy inverse reinforcement learning by updating a parameter vector of the objective function so as to increase a likelihood that the decision making history data including the actual change information is generated, the likelihood being defined, for each item of the decision making history data, by a probability of a corresponding train operation diagram which is proportional to an exponential of a value of the objective function and normalized by a sum of exponentials of objective function values for feasible train operation diagrams under the constant parameter values” (e.g., mathematical concept by defining learning as updating a parameter vector to increase a likelihood/probability calculated using exponential objective function values and normalization)
Claim 1 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “learning device”, “memory”, “ processor”, and “inverse reinforcement learning”) [see MPEP 2106.04(a)(2)(III)].
“determine, as decision making history data, actual change information indicating an actual change from the second target to the third target” (e.g., a human can identify changes form one set of information to another similar to see what has changed)
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A, Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of a “learning device”, “memory”, “ processor”, and “inverse reinforcement learning”, which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)). In particular, the recited “inverse reinforcement learning” is merely a generic computer component, because it is merely recited to perform the function of implementing “objective function” and “optimization result for a first target” the claims do not recite any particular structure for how such “inverse reinforcement learning” is implemented.
Regarding the “accept, as a first target, diagram data representing a train operation schedule to be changed” limitation, this additional element is recited at a high-level of generality and amounts to extra-solution activity of accepting input, i.e. pre-solution activity of gathering data for use in the claimed system (see MPEP 2106.05(g)).
Regarding “store decision making history data including a plurality of items each indicating an actual change applied in the past to a train operation diagram under constant parameter values of the train operation schedule” which is recited at a high-level of generality, of storing information on a computer, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding the “output as a second target, an optimized train operation diagram obtained by changing the first target by optimization using the objective function so that a value of the objective function is increased within a set of feasible train operation diagrams under constant parameter values of the train operation schedule” limitation, this additional element is recited at a high-level of generality and amounts to extra-solution activity of outputting information, i.e. post-solution activity of data outputting for use in the claimed system (see MPEP 2106.05(g)).
Regarding “present the second target to a user and accept, from the user, a change instruction regarding the second target” which is recited at a high-level of generality, of displaying information on a computer, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding the “output as a third target, a train operation diagram obtained by further changing the second target based on the change instruction” limitation, this additional element is recited at a high-level of generality and amounts to extra-solution activity of generating output, i.e. post-solution activity of outputting data for use in the claimed system (see MPEP 2106.05(g)).
Regarding “add the actual change information to the decision making history data” limitation, this additional element is recited at a high-level of generality and amounts to extra-solution activity of adding information to the database, i.e. post-solution activity of gathering data for use in the claimed system (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “learning device”, “memory”, “ processor”, and “inverse reinforcement learning”, which are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)). In particular, the recited “inverse reinforcement learning” is merely a generic computer component, because it is merely recited to perform the function of implementing “objective function” and “optimization result for a first target” the claims do not recite any particular structure for how such “inverse reinforcement learning” is implemented.
Regarding the “accept, as a first target, diagram data representing a train operation schedule to be changed” limitation, as discussed above, this additional element of accepting input is recited at a high-level of generality and amounts to extra-solution activity of pre-solution activity of gathering data. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Regarding “store decision making history data including a plurality of items each indicating an actual change applied in the past to a train operation diagram under constant parameter values of the train operation schedule” which is recited at a high-level of generality, of storing information on a computer, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding the “output as a second target, an optimized train operation diagram obtained by changing the first target by optimization using the objective function so that a value of the objective function is increased within a set of feasible train operation diagrams under constant parameter values of the train operation schedule” limitation, as discussed above, of outputting information, this additional element is recited at a high-level of generality and amounts to extra-solution activity of post-solution activity of data outputting. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Regarding “present the second target to a user and accept, from the user, a change instruction regarding the second target” which is recited at a high-level of generality, of displaying information on a computer, such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Regarding the “output as a third target, a train operation diagram obtained by further changing the second target based on the change instruction” limitation, as discussed above, of generating output, this additional element is recited at a high-level of generality and amounts to extra-solution activity of post-solution activity of outputting data. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Regarding “add the actual change information to the decision making history data” limitation, as discussed above, of adding data to a database, this additional element is recited at a high-level of generality and amounts to extra-solution activity of post-solution activity of gathering data. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Regarding Claim 2
Step 2A, Prong 1
Claim 2 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “memory”, “ processor”, and “inverse reinforcement learning”) [see MPEP 2106.04(a)(2)(III)].
“accept the change instruction from the user for the output second target, and output the resulting target based on the accepted change instruction as the third target” (e.g. a human update information of a table according to edits)
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A Prong 2
In accordance with Step 2A, Prong 2, the claim does not include any additional elements and the judicial exception is not integrated into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding Claim 3
Step 2A, Prong 1
Claim 3 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “memory”, “ processor”, and “inverse reinforcement learning”) [see MPEP 2106.04(a)(2)(III)].
“accept the change instruction from the user for weights of explanatory variables included in the objective function represented by a linear expression” (e.g. a human update weights of values for factors by using mathematical functions)
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A, Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of a “output a third target as a result of changing the second target by optimization using the changed objective function” limitation, this additional element is recited at a high-level of generality and amounts to extra-solution activity of generating output, i.e. post-solution activity of outputting data for use in the claimed system (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “output a third target as a result of changing the second target by optimization using the changed objective function” limitation, as discussed above, the additional element of generating output is recited at a high level of generality and amounts to extra-solution activity of post-solution activity of outputting data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Regarding Claim 4
Step 2A, Prong 1
Claim 4 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “memory”, “ processor”, and “inverse reinforcement learning”) [see MPEP 2106.04(a)(2)(III)].
“accept the change instruction from the user to add an explanatory variable to the objective function” (e.g. a human update weights of values for factors by using mathematical functions)
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A, Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of a “output a third target as a result of changing the second target by optimization using the changed objective function” limitation, this additional element is recited at a high-level of generality and amounts to extra-solution activity of generating output, i.e. post-solution activity of outputting data for use in the claimed system (see MPEP 2106.05(g)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “output a third target as a result of changing the second target by optimization using the changed objective function” limitation, as discussed above, the additional element of generating output is recited at a high level of generality and amounts to extra-solution activity of post-solution activity of outputting data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Regarding Claim 5
Step 2A, Prong 1
Claim 5 does not introduce any new abstract ideas, but recites the abstract idea identified in claim 1.
Accordingly, at Step 2A, prong one, the claim recites an abstract idea.
Step 2A, Prong 2
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of a “learn the objective function including the added explanatory variable” limitation, this additional element is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Step 2B
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional element of a “learn the objective function including the added explanatory variable” limitation, this additional element is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)).
Accordingly, at Step 2B, the additional elements individually or in combination do not integrate the judicial exception into a practical application.
Regarding claims 6-9
Claims 6-9 recites a method. Which corresponds directly to the system steps recited in claims 1-4, respectively, with the addition of instructions and computer-executable instructions which are insufficient to render the claims subject matter eligible for the same reasons described above.
Specifically:
Claim 6 corresponds to claim 1, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 1.
Claim 7 corresponds to claim 2, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 2.
Claim 8 corresponds to claim 3, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 3.
Claim 9 corresponds to claim 4, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 4.
Regarding claims 10-13
Claims 10-13 recites a non-transitory computer readable information recording medium. Which corresponds directly to the system steps recited in claims 1-4, respectively, with the addition of instructions and computer-executable instructions which are insufficient to render the claims subject matter eligible for the same reasons described above.
Specifically:
Claim 10 corresponds to claim 1, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 1.
Claim 11 corresponds to claim 2, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 2.
Claim 12 corresponds to claim 3, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 3.
Claim 13 corresponds to claim 4, with the added recitation of method steps executing instructions to perform the same abstract system steps of claim 4.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1, 6, and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ichihashi et al. (WO2014064788A1, referred to as Ichihashi), in view of Ziebart et al. (“Maximum Entropy Reinforcement Learning”, referred to as Ziebart).
Regarding claim 1, Ichihashi teaches, a learning device comprising:
a memory storing instructions; and
one or more processors configured to execute the instructions to (Pages 5-6, 9-10, and 16: Describes a train operation management device/system including processing units, display control terminals, a central unit, timetable management device, prediction calculation device, driving support device, and database storing timetable/route/vehicle/parameter data.):
accept, as a first target, diagram data representing a train operation schedule to be changed (Page 9, and 11-13: Describes receiving a planned timetable representing a train operation plan and current train location information, monitoring a discrepancy between the planned timetable and the current train location, and modifying the train operation timetable as needed. The planned timetables, actual timetables, and predicted timetables share a diamond data format including train arrival times, departure times , train numbers, intermediate positions between stations, and corresponding times. );
output as a second target, an optimized train operation diagram obtained by changing the first target by optimization using the objective function so that a value of the objective function is increased within a set of feasible train operation diagrams under constant parameter values of the train operation schedule (Pages 6-7, 21-23 and 25-26: Describes comparing candidate predicted timetables based on total cost, including power consumption and delay impact, and proposing/displaying the changed predicted timetable when the total cost is lower. It calculates the predicted timetable using predefined constants, including minimum travel time between stations and minimum operating intervals.);
present the second target to a user and accept, from the user, a change instruction regarding the second target (Pages 6-7, 10, and 18-20: Describes displaying planned, actual, and predicted timetables on a single screen, allowing the operator to change the displayed timetable using an operation control panel, and transmitting the operators change information to the timetable management device.);
output as a third target, a train operation diagram obtained by further changing the second target based on the change instruction (Pages 25-27, and 29-30:Describes that, when the operator selects the time priority driving mode, the arrival order of trains is changed and the revised predicted timetable is used/transmitted to the timetable management device.);
determine, as decision making history data, actual change information indicating an actual change from the second target to the third target, and add the actual change information to the decision making history data (Page 10, and 13: Describes that the operator may change the displayed timetable, and that information about the operator’s timetable changes is transmitted to the timetable management device, which manages train operation records. It calculates a predicted timetable based on timetable changes received from the operation management desk.);
compute an objective function which is a linear function of a feature vector whose elements are indices of train operation derived from a train operation diagram, the objective function being generated in advance in accordance with a maximum entropy inverse reinforcement learning method based on the decision making history data (Pages 6-8, 16-18, and 24-25: Describes computing an evaluation/cost value for a train operation timetable based on train operation values derived from timetable/diagram data. It compares first and second predicted timetables based on total cost including power consumption and delay impact. The delay impact may be calculated by multiplying delay time values by weighting factors associated with station, line, time of day, train type, passenger volume, and other train operation factors. It uses station information such as arrival order, arrival time, departure time, direction, and delay, and performing predefined weighting calculations based on multiple predicted timetables and power consumption.);
Although Ichihashi teaches computing train operation cost/evaluation values based on weighted train operation factors derived from timetable/diagram data, it does not teach generating or learning the objective function in advance using maximum entropy inverse reinforcement learning based on decision making history data.
Ziebart teaches, compute an objective function which is a linear function of a feature vector whose elements are indices of train operation derived from a train operation diagram, the objective function being generated in advance in accordance with a maximum entropy inverse reinforcement learning method based on the decision making history data( Page 1433-1435 , Background and Maximum Entropy IRL: Describes maximum entropy inverse reinforcement learning, where demonstrated behavior is used to recover an unknown reward/objective function, the reward is assumed to be linear in features, and reward weights are found to make demonstrated behavior appear near optimal. The reward/objective is a weighted feature expression.)
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the train timetable data of Ichihashi with the objective reward function of ZIebart. Doing so would have enabled the system to propose future timetable changes based on prior operator scheduling preferences.
store decision making history data including a plurality of items each indicating an actual change applied in the past to a train operation diagram under constant parameter values of the train operation schedule (Pages 1433-1435, Background and Learning from Demonstrated Behavior: Describes using observed/demonstrated decision history as training data for inverse reinforcement learning, wherein the reward/objective parameters are learned so that the demonstrated behavior is likely under the learned model.);
learn the objective function by maximum entropy inverse reinforcement learning by updating a parameter vector of the objective function so as to increase a likelihood that the decision making history data including the actual change information is generated, the likelihood being defined, for each item of the decision making history data, by a probability of a corresponding train operation diagram which is proportional to an exponential of a value of the objective function and normalized by a sum of exponentials of objective function values for feasible train operation diagrams under the constant parameter values (Pages 1433-1435, Background, Maximum Entropy IRL and Learning from Demonstrated Behavior: Describes recovering a utility/reward function from demonstrated behavior, where the reward/objective is linear in feature values and parameterized by reward weights θ, e.g., function 1 on page 1434. IT assigns a probability to a candidate path/plan according to the function 2 on page 1434, such that higher reward plans are exponentially preferred and normalized by a partition function. It updates θ by maximizing the likelihood of the observed demonstrated behavior.).
Regarding claim 6, which recites substantially the same limitations claim 1. Claim 6 further cites a learning method(Ichihashi pages 9-13: Describes an operation/process performed by the train operation system, corresponding to a method to be executed on computer hardware.) to perform the system steps of claim 1, and is therefore rejected on the same premise.
Regarding claim 10, which recites substantially the same limitations claim 1. Claim 10 further cites a non-transitory computer readable information recording medium (Ichihashi pages 5-6: Describes a computerized train operation system with computer hardware and software.) to perform the system steps of claim 1, and is therefore rejected on the same premise.
Claim(s) 2 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ichihashi et al. (WO2014064788A1, referred to as Ichihashi), in view of Ziebart et al. (“Maximum Entropy Reinforcement Learning”, referred to as Ziebart), in view of Fails et al. ("Interactive machine learning.", referred to as Fails).
Regarding claim 2, Ichihashi, in view of Ziebart, teaches the learning device according to claim 1.
Although Ichihashi, in view of Ziebart teach the device according to claim 1, they do not teach, accept the direct change instruction from the user for the output second target, and output the resulting target based on the accepted change instruction as the third target.
Fails teaches accept the direct change instruction from the user for the output second target, and output the resulting target based on the accepted change instruction as the third target (, Page 2, Image Processing With Crayons: Describes the interactive loop with the user edits, displays results which the system re-trains then it will further display the updated feedback immediately, and will repeat this process.).
It would have been obvious to one of ordinary skill int the art at the time of the claimed invention to have incorporated the device of Ichihashi, in view of Ziebart, with the interactive user correction technique of Fails’. Doing so would have enabled the system to improve later objective function learning and producing future train operation diagrams to better reflect the operators intent.
Regarding claim 7, which recites substantially the same limitations claim 2. Claim 7 further cites a learning method (Ichihashi pages 9-13: Describes an operation/process performed by the train operation system, corresponding to a method to be executed on computer hardware.) to perform the system steps of claim 2, and is therefore rejected on the same premise.
Claim(s) 3-5, 8-9, and 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ichihashi et al. (WO2014064788A1, referred to as Ichihashi), in view of Ziebart et al. (“Maximum Entropy Reinforcement Learning”, referred to as Ziebart), in view of Li et al. (“Interactive Machine learning by Visualization: A Small Data Solution”, referred to as Li).
Regarding claim 3, Ichihashi, in view of Ziebart, teaches the learning device according to claim 1.
Although Ichihashi, in view of Ziebart, teaches the learning device, they do not teach wherein the processor is configured to execute the instructions to accept the change instruction from the user for the weights of explanatory variables included in the objective function represented by a linear expression.
Li teaches, wherein the processor is configured to execute the instructions to
accept the change instruction from the user for the weights of explanatory variables included in the objective function represented by a linear expression (Page 6 Equation 3: “We may express the reward function as a linear sum of weighted features” showing that the reward function/ objective function is a linear equation using weighted variables.; Page 12, Algorithm 1: Describes using these to facilitate training to produce sequential target outputs.), and output a third target as a result of changing the second target by optimization using the changed objective function(The examiner notes that this is a combination of previous teachings of claims 1 and 2 with the added teaching of Li.).
Regarding claim 4, Ichihashi, in view of Ziebart, teaches, the learning device according to claim 1.
Although Ichihashi, in view of Ziebart t teaches the learning device, they do not teach wherein the processor is configured to execute the instructions to accept the change instruction from the user for the weights of explanatory variables included in the objective function represented by a linear expression.
Li teaches, wherein the processor is configured to execute the instructions to
accept the change instruction from the user to add an explanatory variable to the objective function (Page 3, Section 3: Describes how a user can input new features or other details into a training system to create optimal training sets, to generate optimized training outputs.), and output a third target as a result of changing the second target by optimization using the changed objective function (The examiner notes that this is a combination of previous teachings of claims 1 and 2 with the added teaching of Li.).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to incorporate the inverse reinforcement system of Ichihashi, in view of Ziebart, with the function changes of Li. Doing so would allow for the user to update model parameters during runs of the model with the outputs generated, giving the user the ability to generate optimized results faster.
Regarding claim 5, Ichihashi, in view of Ziebart, in view of Li teaches, the learning device according to claim 4, wherein the processor is configured to execute the instructions to
learn the objective function (Ziebart pages 1433-1435, Background, and Learning form Demonstrated Behavior: Describes maximizing entropy inverse reinforcement learning wherein an objective/reward function is parameterized by weights and learned from demonstrated behavior by updating the weights to increase the likelihood of the observed demonstration.).
Although Ziebart teaches learning the objective function, it does not include an added explanatory variable.
Li teaches, including the added explanatory variable(Page 3, Section 3: Describes how a user can input new features or other details into a training system to create optimal training sets, to generate optimized training outputs.).
Regarding claim 8, which recites substantially the same limitations claim 3. Claim 8 further cites a learning method(Ichihashi, as discussed above.) to perform the system steps of claim 3, and is therefore rejected on the same premise.
Regarding claim 9, which recites substantially the same limitations claim 4. Claim 9 further cites a learning method (Ichihashi, as discussed above.) to perform the system steps of claim 4, and is therefore rejected on the same premise.
Regarding claim 12, which recites substantially the same limitations claim 3. Claim 12 further cites a non-transitory computer readable information recording medium (Ichihashi, as discussed above.) to perform the system steps of claim 3, and is therefore rejected on the same premise.
Regarding claim 13, which recites substantially the same limitations claim 4. Claim 13 further cites a non-transitory computer readable information recording medium (Ichihashi, as discussed above.) to perform the system steps of claim 4, and is therefore rejected on the same premise.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DONALD T RODEN whose telephone number is (571)272-6441. The examiner can normally be reached Mon-Thur 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.T.R./Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights