DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendment filed on 01/12/2026. Claims 1,4,6,8 and 10 are pending in the case. This action is Final.
Applicant Response
3. In Applicant’s response dated 01/12/2026, Applicant amended Claims 1, 4, 6, 8, and 10, cancelled claims 2, 3, 5, 7, 9 and 11 and argued against all objections and rejections previously set forth in the Office Action dated 10/10/2025.
Claim Rejections - 35 USC § 101
4. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1, 4, 6, 8 and 10 are rejected under 35 U.S.C. 101 because the claimed invention is directed towards an abstract idea, without significantly more.
Regarding Claim 1, 8 and 10
Step 1
According to the first part of the analysis, in the instant case, claims 1-16 are directed to a system claim and claims 17-20 are directed to method claim. Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).).
At step 2A, Prong 1, Does the claim recite a judicial exception?
Claim 1 recites the steps of:
receive, as a first target, an operation schedule of a train or an aircraft to be changed (This step involves data gathering which falls under the mental process concept abstract idea group);
calculate, based on decision making history data, a likelihood indicating plausibility of each of a plurality of objective functions estimated from data used for learning the plurality of objective functions and generated in advance by inverse reinforcement learning based on the decision making history data (This step involves mathematical optimization which falls under the mathematical concept abstract idea group);
select one or more objective functions from the plurality of objective functions based on the likelihood (This step involves mathematical optimization which falls under the mathematical concept abstract idea group);
exclude, from being optimized, an objective function of the plurality of objective functions whose likelihood is lower than a predetermined threshold (This step involves mathematical comparison which falls under the mathematical concept abstract idea group);
output a plurality of second targets, which are optimization results for the first target, using the one or more objective functions wherein the decision making history data indicates an actual change to one or more of the first target or the plurality of second targets, wherein each of the plurality of second targets is output by optimization using a respective selected objective function of the one or more objective functions (This step involves mathematical optimization which falls under the mathematical concept abstract idea group);
output, in association with each of the plurality of second targets, each respective selected objective function (This step involves training a mathematical model (IRL) and is understood to be a mathematical concept abstract idea group.)
accept a selection instruction from a user to accept a second target of a plurality of the output second targets;
output an actual change from the first target to the accepted second target as the decision making history data;
accept, from the user, a change instruction regarding the accepted second target, the change instruction including at least one of:
output a third target indicating an operation schedule resulting from further changing of the accepted second target based on the change instruction (This step involves selecting action form the user and is understood to be a mental process that can be performed with pen and paper, i.e., judgment.);
output an actual change from the accepted second target to the third target as additional decision making history data (This step involves selecting action form the user and is understood to be a mental process that can be performed with pen and paper, i.e., judgment.);
learn the respective selected objective function by relearning using the decision making history data including the additional decision making history data (This step involves mathematical optimization which falls under the mathematical concept abstract idea group);
The claim recites a judicial exception, a mathematical concept and mathematical applied in the field of machine learning which falls within the “Mental Processes” groupings of abstract ideas. Accordingly, the claims recite an abstract idea.
Step 2A prong 2: Does the claim recite additional elements? Do those additional elements, individually and in combination, integrate the judicial exception into a practical application?
Further, the claim does not recite any additional element which could integrate this abstract idea into a practical application, because the additional elements recited of consist of:
… generated in advance by inverse reinforcement learning] based on decision making history data indicating an actual change to a target, i.e. inverse reinforcement learning and using a ML model to make decision that is not meaningful technical improvement.” as a tool to perform the abstract idea step of generating an output (see MPEP 2106.05(f)), and \
changing an operation time, changing an operation flight, changing weights of explanatory variables included in the respective selected objective function, or adding an explanatory variable to the respective selected objective function;
the one or more processors being further configured to execute the instructions to:
store the additional decision making history data in the memory; and
A system comprising: a processor; and a storage medium storing instructions which, when executed by the processor, cause the system to which is a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f));
A non-transitory computer-readable storage medium recording which, when executed by a computing device, cause the computing device to perform acts comprising (claim 10) which is a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f));
The additional elements as disclosed above alone or in combination do not
integrate the judicial exception into practical application as they are generic computer functions in combination with limitations that are generally linking the use of the judicial exception to a particular technological environment or field of use that are implemented to perform the disclosed abstract idea above. Thus, the claim is directed towards the abstract idea.
Step 2B: Do the additional elements, considered individually and in combination, amount to significantly more than the judicial exception?
No, As shown above with respect to integration of the abstract idea into a practical application, the additional element of:
… generated in advance by inverse reinforcement learning] based on decision making history data indicating an actual change to a target, i.e. inverse reinforcement learning and using a ML model to make decision that is not meaningful technical improvement.” as a tool to perform the abstract idea step of generating an output (see MPEP 2106.05(f)), and
output the actual change from the first target to the accepted second target as the decision making history data. i.e. providing or displaying information
changing an operation time, changing an operation flight, changing weights of explanatory variables included in the respective selected objective function, or adding an explanatory variable to the respective selected objective function;
the one or more processors being further configured to execute the instructions to:
store the additional decision making history data in the memory; and
A system comprising: a processor; and a storage medium storing instructions which, when executed by the processor, cause the system to which is a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f));
A no transitory computer-readable storage medium recording which, when executed by a computing device, cause the computing device to perform acts comprising (claim 10) which is a generic computer components on which to implement the abstract idea (see MPEP 2106.05(f));
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are generic computer functions in combination with limitations that are generally linking the use of the judicial exception to a particular technological environment or field of use that are implemented to perform the disclosed abstract idea above. Thus, the claim is directed towards the abstract idea
Thus, the claims are not patent eligible. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Neither can insignificant extra-solution activity. All of these additional elements as generically claimed are thus considered well-understood, routine, and conventional. Therefore, these limitations, taken alone or in combination, do not integrate the abstract idea into a practical application or recite significantly more that the abstract idea.
Thus, these independent claims are not patent eligible.
The dependent claims respectively recite a judicial exception in limitations of: “ select a predetermined top objective function with a high likelihood among the one or more objective functions whose derivative of the parameter is zero (claims 4), “wherein select a solution with a higher likelihood than a predetermined threshold among the plurality of second target, and relearns by adding decision making history data including the selected solution.” (claims 6),
These additional limitations (in claims 4 and 6) also constitute concepts performed in the human mind which fall within the “Mental Processes” groupings of abstract ideas.
This judicial exception is not integrated into a practical application. Additional elements “computer readable medium comprising: computer program code (in claims 4 and 6) all amount to no more than adding insignificant extra-solution activity/specifications related to data gathering, data input, or data transmittal. These additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The dependent claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of non-transitory computer readable medium comprising: computer program code are again insignificant extra-solution activity steps that cannot provide an inventive concept. All of these additional elements as generically claimed are considered well-understood, routine, and conventional.
Therefore, these limitations, taken alone or in combination, do not integrate the abstract idea into a practical application or recite significantly more that the abstract idea. Thus, all of the dependent claims are also not patent eligible.
Examiner Comments
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. Claims 1, 4, 6, 8 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Tang (US 20210256403 A1, 2021-08-19) in view of Freund (Pat. No.: US 10394408 B1, Pub. Date: 2019-08-27) in further view of LOBETE (Pub. No.: US 20190318254 A1, Pub. Date: 2019-10-17)
Regarding independent Claim 1,
Tang teaches a learning device comprising: a memory storing instructions; and one or more processors (see Tang: Fig.12, [0208], “the training apparatus includes at least one processor 1201, at least one memory 1202, and at least one communications interface 1203. The processor 1201, the memory 1202, and the communications interface 1203 are connected and communicate with each other by using a communications bus.”), configured to execute the instructions to:
receive, as a first target, an operation schedule of a train or an aircraft to be changed (see Tang: Fig.1, [0083], “The recommendation system architecture includes only the database 130, and does not include the data storage system 150. After the user equipment 140 receives the target to-be-recommended object that is output by the execution device 110 through the I/O interface 112, the user equipment 140 stores the target to-be-recommended object and the user behavior for the target to-be-recommended object into the database 130, to train the status generation model/selection policy 101.”)
calculate, based on decision making history data, a likelihood indicating plausibility of each of a plurality of objective functions estimated from data used for learning the plurality of objective functions and generated in advance by inverse reinforcement learning based on the decision making history data (see Tang: Fig.1, [0081], “After obtaining the status generation model and the selection policy, the execution device 110 obtains the plurality of historical recommended objects and the user behavior for each historical recommended object from the data storage system 150; calculates a reward value of each historical recommended object based on the user behavior for each historical recommended object”)
select one or more objective functions from the plurality of objective functions based on the likelihood (see Tang: Fig.2, [0119], “the recommendation apparatus inputs the recommendation system status parameter into the selection policy corresponding to the target set, to obtain a probability distribution of a plurality of to-be-recommended objects included in the target set, and then the recommendation apparatus randomly selects one of the plurality of to-be-recommended objects as the target to-be-recommended object based on the probability distribution of the plurality of to-be-recommended objects included in the target set”)
exclude, from being optimized, an objective function of the plurality of objective functions whose likelihood is lower than a predetermined threshold (see Tang: Fig.2, [0121], “The recommendation apparatus inputs the recommendation system status parameter into a selection policy (that is, a selection policy 1) corresponding to the level-1 set, to obtain a probability distribution (that is, a probability distribution 1) of the level-2 set 1 and the level-2 set 2, and then the recommendation apparatus randomly selects one of the level-2 set 1 and the level-2 set 2 as a target level-2 set based on the probability distribution of the level-2 set 1 and the level-2 set 2”)
output a plurality of second targets, which are optimization results for the first target, using the one or more objective functions ( see Tang: Fig.3, [0094], “e recommendation apparatus inputs the first spliced vector v.sub.1 in the t−1 spliced vectors into the status generation model for an operation, to obtain a calculation result j.sub.1; inputs the calculation result j.sub.1 and the second spliced vector v.sub.2 in the t−1 spliced vectors into the status generation model, to obtain a calculation result j.sub.2; and then inputs the calculation result j.sub.2 and the third spliced vector in the t−1 spliced.”), wherein the decision making history data indicates an actual change to one or more of the first target or the plurality of second targets data (see Tang: Fig.3, [0095], “a result v.sub.3 obtained by splicing a vector of the third historical recommended object in the three historical recommended objects and a corresponding reward vector of the third historical recommended object is (1, 0, 0, 5, 6).), wherein each of the plurality of second targets is output by optimization using a respective selected objective function of the one or more objective functions;
output, in association with each of the plurality of second targets, each respective selected objective function (see Tang: Fig.2, [0095], “The recommendation apparatus determines a target set from lower-level sets based on the recommendation system status parameter and according to a selection policy corresponding to an upper-level set.”
accept a selection instruction from a user to accept a second target of a plurality of the output second targets (see Tang: Fig.2, [0066], “determines a recommended object (for example, a to-be-recommended item) based on the recommendation system status parameter, and sends the selected recommended object to the user. After receiving the recommended object, the user performs a specific behavior (such as click or download) on the recommended object.
output an actual change from the first target to the accepted second target as the decision making history data (see Tang: Fig.2, [00119], “recommendation apparatus inputs the recommendation system status parameter into the selection policy corresponding to the target set, to obtain a probability distribution of a plurality of to-be-recommended objects included in the target set, and then the recommendation apparatus randomly selects one of the plurality of to-be-recommended objects as the target to-be-recommended object based on the probability distribution of the plurality of to-be-recommended objects included in the target set.”)
accept, from the user, a change instruction regarding the accepted second target, the change instruction (see Tang: Fig.2, [0066], “determines a recommended object (for example, a to-be-recommended item) based on the recommendation system status parameter, and sends the selected recommended object to the user. After receiving the recommended object, the user performs a specific behavior (such as click or download) on the recommended object. The recommendation system generates a value based on the behavior performed by the user. The value is referred to as a system reward value”… [0091], “or another example, an article is recommended to a user, and if the user clicks and reads the article, a reward of the article is 1, or if the user does not click or read the article, a reward of the article.”, i.e. the user performs a selection of single item from the list of recommended items ) including at least one of: changing an operation time, changing an operation flight, changing weights of explanatory variables included in the respective selected objective function, or adding an explanatory variable to the respective selected objective function (see Tang: Fig.2, [0109], “The following specifically describes implementing the status generation model in a weighting manner. After splicing the t−1 historical recommended object vectors and the corresponding reward vectors of the t−1 historical recommended object vectors to obtain the t−1 spliced vectors (that is, the spliced vectors v.sub.1, v.sub.2, . . . , and v.sub.t−1), the recommendation apparatus obtains a weighting result V according to a formula V=α.sub.1xv.sub.1+α.sub.2xv.sub.2+ . . . +α.sub.t−1xv.sub.t−1, where a.sub.1, a.sub.2, . . . , and a.sub.t−1 are weights”)
the one or more processors being further configured to execute the instructions (see Tang: Fig.12, [0208], “the training apparatus includes at least one processor 1201, at least one memory 1202, and at least one communications interface 1203. The processor 1201, the memory 1202, and the communications interface 1203 are connected and communicate with each other by using a communications bus.”), to:
output a third target indicating an operation schedule resulting from further changing of the accepted second target based on the change instruction ( see Tang: Fig.3, [0094], “e recommendation apparatus inputs the first spliced vector v.sub.1 in the t−1 spliced vectors into the status generation model for an operation, to obtain a calculation result j.sub.1; inputs the calculation result j.sub.1 and the second spliced vector v.sub.2 in the t−1 spliced vectors into the status generation model, to obtain a calculation result j.sub.2; and then inputs the calculation result j.sub.2 and the third spliced vector in the t−1 spliced vectors into the status generation model, to obtain a calculation result j.sub.3.”);
output an actual change from the accepted second target to the third target as additional decision making history data (see Tang: Fig.3, [0095], “a result v.sub.3 obtained by splicing a vector of the third historical recommended object in the three historical recommended objects and a corresponding reward vector of the third historical recommended object is (1, 0, 0, 5, 6).)”;
store the additional decision making history data in the memory (see Tang: Fig.1, [0083], “the recommendation system architecture includes only the database 130, and does not include the data storage system 150. After the user equipment 140 receives the target to-be-recommended object that is output by the execution device 110 through the I/O interface 112, the user equipment 140 stores the target to-be-recommended object and the user behavior for the target to-be-recommended object into the database 130, to train the status generation model/selection policy 101.”);
As shown above, Tang teaches the recommendation-providing method in the field of artificial intelligence, an apparatus for generating recommendations obtains a recommendation system status parameter based on a plurality of historical recommended objects and a user behavior for each historical recommended object without using inverse reinforcement learning technique.
Tang does not teach a learning device comprising:
accept a selection instruction from a user for a plurality of the output second targets
performing inverse reinforcement learning based on decision making history data indicating an actual change to a target,
learn the respective selected objective function by relearning using the decision making history data including the additional decision making history data.
However, Freund teaches accept a selection instruction from a user for a plurality of the output second targets (see Freund: Fig.8, Col.19. Line 30 -37, “At 802, input indicting an interest in two or more videos (i.e. second targets) included in a first subset of media items recommended to a user is received (e.g., using selection component 108) ( i.e. a selection from a user). For example, expressed input of user interest can be received, such request to have a media item re-recommend, a request to play a media item, or a request to have a media item saved in a watch later file.”)
Because both Tang and Freund address the same/similar issue of plurality of recommended items selection by a user, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to modify teaching of Tang to include the system that accept a selection instruction from a user for a plurality of the output second target as taught by Freund. After modification of Tang, the recommendation system that allow user to select a single item form plurality of recommended items can also incorporate the mechanism of selection plurality of recommended items as taught by Freund. One would be motivated to make such a combination in order to improve accuracy and personalization of the optimization results.
Tang and FREUND does not teach the learning system comprising:
performing inverse reinforcement learning based on decision making history data indicating an actual change to a target,
learn the respective selected objective function by relearning using the decision making history data including the additional decision making history data.
However, LOBETE teaches the system wherein:
performing inverse reinforcement learning based on decision making history data indicating an actual change to a target (see LOBETE: Fig.1 [0031] “t provide a Cloud-based ML heuristic that learns from multiple, e.g. millions, of devices/sources and can optimize itself and the devices under it. Embodiments can be based on the widely accepted Q-Learning (Reinforcement Learning) Heuristic, or other types of ML), and
learn the respective selected objective function by relearning using the decision making history data including the additional decision making history data (see LOBETE: Fig.4 [0045] “generating 401 a decision-making data structure using a machine learning data structure; transmitting 403, to a second electronic device, the decision-making data structure; receiving 405, from the electronic device, result data regarding a result of performing a selected action selected from the decision-making data structure; and updating 407 the machine learning data structure using the result data.”)
Because Tang, FREUND and LOBETE are in the same/similar field of endeavor of adaptive computing systems that provides recommendations based on user interaction, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to modify teaching of Tang to include the system that apply inverse reinforcement learning based reward learning as taught by LOBETE. After modification of Tang, the recommendation system that is based on a plurality of historical recommended objects and a user behavior for each historical recommended object can incorporate the IRL based reward learning recommendation teaching of as taught by LOBETE. One would be motivated to make such a combination in order to improve recommendation accuracy and prediction efficiency by using advance machine learning techniques.
Regarding Claim 4,
As shown above, Tang, FREUND and LOBETE teach all the limitations of Claim 2. Tang further teaches the system configures to:
select a predetermined top objective function with a high likelihood among the one or more objective functions whose derivative of the parameter is zero (see Tang: Fig.5, [0121], “The recommendation apparatus inputs the recommendation system status parameter into a selection policy (that is, a selection policy 1) corresponding to the level-1 set, to obtain a probability distribution (that is, a probability distribution 1) of the level-2 set 1 and the level-2 set 2, and then the recommendation apparatus randomly selects one of the level-2 set 1 and the level-2 set 2 as a target level-2 set based on the probability distribution of the level-2 set 1 and the level-2 set 2.”)
Regarding Claim 6,
As shown above, Tang, FREUND and LOBETE teach all the limitations of Claim 1. Tang further teaches the system configures to:
select a solution with a higher likelihood than a predetermined threshold among the plurality of second targets, and relearn by adding decision making history data including the selected solution (see Tang: Fig.2, [0113], “the recommendation apparatus inputs the recommendation system status parameter into the selection policy of the upper-level set, to obtain a probability distribution of the plurality of lower-level sets of the upper-level set, and the recommendation apparatus randomly selects one of the plurality of lower-level sets as the target set based on the probability distribution of the plurality of lower-level sets.”)
Regarding independent Claim 8,
Claim 8 is directed to a method claim and has similar/same claim limitation as Claim 1 and is rejected under the same rationale.
Regarding independent Claim 10,
Claim 10 is directed to a non-transitory computer readable information recording medium and has similar claim limitation as claim 1 and is rejected under the same rationale.
Response to Arguments
Claim Rejections - 35 U.S.C. § 103,
Applicant’s arguments with respect to claim amendments have been considered but are moot considering the new combination of references being used in the current rejection. The new combination of references was necessitated by Applicant’s claim amendments. Therefore, the claims are rejected under the new combination of references as indicated above.
Claim Rejections - 35 U.S.C. § 112(f),
The rejection to the claims as being indefinite under - 35 U.S.C. § 112(f), has been withdrawn based on applicant amendment.
Claim Rejections - 35 U.S.C. § 101,
Regarding the 35 U.S.C. 101 rejection for being directed non-statutory subject matter for claims 1,4,6,8 and 10 has been sustained and updated based on applicant amendments and. Therefore, the 35 U.S.C. 101 rejection has been sustained.
Regarding the 35 U.S.C. 101 rejection the claimed recording medium can include signal per se for claim 10 has been withdrawn based on the amendment.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PGPUB
NUMBER:
INVENTOR-INFORMATION:
TITLE / DESCRIPTION
US 20190035040 A1
Goel; Kumar
Title: Method, Apparatus And System For Dynamic Analysis And Recommendations Of Options And Choices Based On User Provided Inputs
Description: he present invention relates generally to informed consumer choice and buying. More specifically, the present invention proposes a method, apparatus and system to provide a framework (framework system) to a user to input his or her preference and options on available health insurance for providing him or her the best set of options and plans
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZELALEM W SHALU whose telephone number is (571)272-3003. The examiner can normally be reached M- F 0800am- 0500pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Zelalem Shalu/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights