DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice for all US Patent Applications filed on or after March 16, 2013
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of the Claims
This communication is in response to communications received on 7/10/24. Claim(s) 1, 5, 6, 9, 11, 13, 15, 16, and 23 is/are amended, claim(s) 19-22 is/are cancelled, claim(s) 24 is/are new, and applicant does not provide any information on where support for the amendments can be found in the instant specification. Therefore, Claims 1-18 and 23-24 is/are pending and have been addressed below.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 7/10/24 and 4/3/26 was/were considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. PCT/SE2023/050031, filed on January 11, 2023.
Response to Arguments
There are no arguments.
Claims Without Prior Art Rejections
Claim(s) 6-14 do/does not have prior art rejections. The remaining rejections are 101 as noted below.
Closest prior art to the invention include
Kimura (US 2024/0086714 A1) in view of Yoon et al. (US 2020/0167611 A1) for claim(s) 6-14.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-18 and 23-24 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter as noted below.
The limitation(s) below for representative claim(s) 1, 23, and 24 that, under its broadest reasonable interpretation, is directed to decomposing a total reward into reward components.
Step 1: The claim(s) as drafted, is/are a process (claim(s) 1-18 recites a series of steps) and system (claim(s) 23-240 recites a series of components).
Step 2A – Prong 1: The claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) (emphasis added):
Claim 1: obtaining first correlation values indicating correlations between input features and reward components;
obtaining reward weights for the reward components, wherein each of the reward weights indicates a contribution of each of the reward components to a total reward; and
applying the reward weights to the first correlation values, thereby generating weighted correlation values which indicate weighted correlations between the input features and the reward components.
Claim(s) 23-24: same analysis as claim(s) 1.
Dependent claims 2-18 recite the same or similar abstract idea(s) as independent claim(s) 1, 23, and 24 with merely a further narrowing of the abstract idea(s): .
The identified limitations of the independent and dependent claims above fall well-within the groupings of subject matter identified by the courts as being abstract concepts of:
a method of organizing human activity (commercial or legal interactions including advertising, marketing or sales activities or behaviors, or business relations) because the invention is directed to economic and/or business relationships as they are associated with decomposing a total reward into reward components.
Step 2A – Prong 2: This judicial exception is not integrated into a practical application because:
The additional elements unencompassed by the abstract idea include a computer (claim(s) 1), computing device, memory, processing circuitry (claim(s) 23), computer program product comprising a non-transitory computer readable medium, processing circuitry of a system (claim(s) 24), neural network (claim(s) 4, 16, 17).
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements as described above with respect to Step 2A Prong 2 fails to describe:
Improvements to the functioning of a computer, or to any other technology or technical field - see MPEP 2106.05(a)
Applying or using a judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition – see Vanda Memo
Applying the judicial exception with, or by use of, a particular machine – see MPEP 2106.05(b)
Effecting a transformation or reduction of a particular article to a different state or thing - see MPEP 2106.05(c)
Applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception - see MPEP 2106.05(e) and Vanda Memo.
Thus the additional elements as described above with respect to Step 2A Prong 2 are merely (as additionally noted by instant specification [0127]) invoked as a tool and/or general purpose computer to apply instructions of an abstract idea in a particular technological environment, and/or mere application of an abstract idea in a particular technological environment and merely limiting the use of an abstract idea to a particular technological field do not integrate an abstract idea into a practical application (MPEP 2106.05(f)&(h)).
Step 2B: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus the additional elements as described above with respect to Step 2A Prong 2 are merely (as additionally noted by instant specification [0127]) invoked as a tool and/or a general purpose computer to apply instructions of an abstract idea in a particular technological environment, and/or mere application of an abstract idea in a particular technological environment and merely limiting the use of an abstract idea to a particular technological field do not integrate an abstract idea into a practical application and thus similarly the combination and arrangement of the above identified additional elements when analyzed under Step 2B also fails to necessitate a conclusion that the claims amount to significantly more than the abstract idea for the same reasons as set forth above (MPEP 2106.05(f)&(h)).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, 15-18, and 23-24 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kiruma (US 2024/0086714 A1).
Regarding claim 1, 23, and 24 (Currently Amended), Kimura teaches a computer-implemented method, the method comprising:
{a computing device, the computing device comprising: a memory; and processing circuitry coupled to the memory, wherein the computing device is configured to perform the method of any one of processing circuitry coupled to the memory, wherein the computing device is configured to - claim 23}
{a computer program product comprising a non-transitory computer readable medium storing instructions which when executed by processing circuitry of a system causes the system to perform a process that comprises: - claim 24}
obtaining first correlation values indicating correlations between input features and reward components [see at least Figs. 1 and 3 and [0079, 0100] computer;
Figs. 5, 9, and 10 and [0113, 0176-0182, 0188-0189] plots that reflect the correlation of the features of the training data and reward components, e.g. R3, R5 that correspond to weights W3, W5, respectively, allowing to determine that the two clusters that belong to the professional driver C CL32-1 and CL32-2 relate with the differences in light amount of the data, and moreover, an implied value -considered as correlation value- for each of the input features of the data that allows the later derivation of the correlation value alpha of Figure 10];
obtaining reward weights for the reward components, wherein each of the reward weights indicates a contribution of each of the reward components to a total reward [see at least [0076] wherein the weights w1 to w5 of the corresponding rewards R1 to R5 indicate a contribution of each of the reward components to a total reward R]; and
applying the reward weights to the first correlation values, thereby generating weighted correlation values which indicate weighted correlations between the input features and the reward components [see at least Fig. 10 and [0188-0189] correlation value of the prior art reflects the application of the reward weights, i.e. the reward ratio W3 on each of the data].
Regarding claim 2, Kimura teaches the computer-implemented method of claim 1, comprising:
obtaining current state information indicating a current state of an environment;
obtaining a first set of quality values associated with a first reward component included in the reward components; and
obtaining a second set of quality values associated with a second reward component included in the reward components,
wherein each quality value included in the first set of quality values and the second set of quality values indicates a quality of an action to be performed by an agent given the current state of the environment [for the limitations above, see at least [0006, 0068, 0069, 0131] reinforcement learning is applied based on states and actions, wherein the rewards relate with the actions;
Fig. 8 and [0071, 0072, 0171, 0171] training data of professional drivers are linked with reward ratios for each reward of interest, wherein the reward ratios are considered as quality values that are associated with the rewards R1-R5].
Regarding claim 3, Kimura teaches the computer-implemented method of claim 2, wherein obtaining the first correlation values comprises generating the first correlation values based at least on the current state information, the first set of quality values, and the second set of quality values [see at least [0006, 0068, 0069, 0131] the correlation values between the reward components R1 to R5 and the corresponding features is based on the state information and the features mapped to quality values as elaborated in the scope of claim 2].
Regarding claim 4, Kimura teaches the computer-implemented method of claim 3, comprising using a neural network (NN) in a reinforcement learning (RL), determining an action to be performed by an agent given the current state of the environment,
wherein the first correlation values are generated based at least on the determined action to be performed by the agent [see at least [0062, 0063, 0081, 0119, 0144] ].
Regarding claim 5 (Currently Amended), Kimura teaches the computer-implemented method of claim 2, comprising:
obtaining a third set of quality values associated with the total reward, wherein each quality value included in the third set of quality values indicates a quality of an action to be performed by an agent given the current state of the environment, wherein
obtaining the first set of quality values comprises generating the first set of quality values based at least on the third set of quality values and the reward weights, and
obtaining the second set of quality values comprises generating the second set of quality values based at least on the third set of quality values and the reward weights [see at least Fig. 13 and [0209-0218] wherein the snow related data to be deleted are considered as a third set of quality values related with each of the rewards R1-R5].
Regarding claim 15 (Currently Amended), Kimura teaches the computer-implemented method of claim 1, further comprising transmitting towards a user or a network node the generated weighted correlation values for updating a machine learning, ML model [see at least [0099, 0102, 0161, 0061] ].
Regarding claim 16 (Currently Amended), Kimura teaches the computer-implemented method of claim 1, further comprising, based on the generated weighted correlation values, revising a neural network configured to determine an action to be performed by an agent [see at least [0192-0193] wherein retraining implies revising the corresponding neural network for determining an action to be performed by an agent;
additionally [0062, 0063, 0081, 0119, 0144] ].
Regarding claim 17, Kimura teaches the computer-implemented method of claim 16, wherein revising the neural network comprises removing at least some of the input features from being used as inputs of the neural network [see at least [0192-0193] wherein retraining implies revising the corresponding neural network for determining an action to be performed by an agent;
[0194] “With this configuration, the information processing apparatus 100 can provide information regarding the correlation between the weight of each of rewards estimated based on each training data group and each piece of environmental information included in each training data group, making it possible to provide information necessary for achieving learning with higher accuracy. For example, the information processing apparatus 100 can provide information regarding a reward necessary (lacking) for achieving learning with higher accuracy.”;
additionally [0062, 0063, 0081, 0119, 0144] ].
Regarding claim 18, Kimura teaches the computer-implemented method of claim 15, wherein updating the ML model comprises removing at least one input feature from the input features and/or adjusting at least one reward weights for at least one reward component in the reward components [see at least [0099, 0102, 0161, 0061];
[0192-0193] wherein retraining implies revising the corresponding neural network for determining an action to be performed by an agent;
[0194] “With this configuration, the information processing apparatus 100 can provide information regarding the correlation between the weight of each of rewards estimated based on each training data group and each piece of environmental information included in each training data group, making it possible to provide information necessary for achieving learning with higher accuracy. For example, the information processing apparatus 100 can provide information regarding a reward necessary (lacking) for achieving learning with higher accuracy.”;
additionally [0062, 0063, 0081, 0119, 0144] ].
Conclusion
When responding to the office action, any new claims and/or limitations should be accompanied by a reference as to where the new claims and/or limitations are supported in the original disclosure.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kiruma – WO 2021/075107 A1 (relevant because it teaches same as US 2024/0086714 A1) as noted in IDs dated 7/10/24
Shrikumar et al. – Learning Important Features Through Propagating Activiation Differences (relevant because it teaches DeepLIFT) as noted in IDS dated 7/10/24
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES WEBB whose telephone number is (313)446-6615. The examiner can normally be reached on M-F 10-3.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O’Connor can be reached on (571) 272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES WEBB/Examiner, Art Unit 3624
Prosecution Insights