DETAILED ACTION
Status of Claims
The action is in reply to the Response to Restriction/Election Requirement filed on 06/25/2026.
Claim 11 has been withdrawn.
Claims 1-10 and 13-14 have been elected and are hereby entered.
Claims 1-11 and 13-14 are currently pending and Claims 1-10 and 13-14 have been examined.
The action is made NON-FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Claims 1-10, 13, and 14 in the reply filed on 06/25/2026 is acknowledged.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-10, 13, and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 13, and 14 recite the limitation "wherein real-time data comprise data from databases, data lakes, data warehouses, cloud infrastructures, blockchain". It is unclear if these data sources are alternatives or are all part of the data used. For examining purposes, the limitation is interpreted to be as alternatives which is “wherein the real-time data comprise data from databases, data lakes, data warehouses, cloud infrastructures, or blockchain”. Claims 2-10 depend from Claim 1 and inherit the deficiency of their parent claim. Therefore, Claims 1-10, 13, and 14 are rejected under 35 U.S.C. 112(b) as being indefinite.
Claim 8 recites the limitations "wherein the supply chain nodes comprise internet of things, industrial internet of things, cyberphysical systems, wherein the supply chain nodes data exchange processes are data from databases, data lakes, data warehouses, cloud infrastructures, blockchain". It is unclear if these data sources and nodes are alternatives or are all part of the data used and nodes. For examining purposes, the limitations are interpreted to be as alternatives which are “wherein the supply chain nodes comprise internet of things, industrial internet of things, or cyberphysical systems, wherein the supply chain nodes data exchange processes are data from databases, data lakes, data warehouses, cloud infrastructures, or blockchain”. Therefore, Claim 8 is rejected under 35 U.S.C. 112(b) as being indefinite.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 and 13-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1-10 and 13-14 are directed to one of the four statutory categories (process, machine, article of manufacture, or composition of matter) since the claimed invention falls into “a process” (a method for optimizing a multi-distribution-level supply chain), “a machine” (a system for optimizing a multi-distribution-level supply chain), and “an article of manufacture” (a computer-readable storage medium for optimizing a multi-distribution-level supply chain) categories.
Regarding Claims 1-10 and 13-14, the claim invention is directed to a judicial exception to patentability, an abstract idea.
Claim 1 recites the following limitations:
A computer-implemented method for optimizing a multi-distribution-level supply chain, wherein the method comprises:
a. obtaining a state of the said supply chain (602), wherein the state is determined based on real-time data from the supply chain, wherein real-time data comprise data from …;
b. calculating a value of a reward function (604), wherein the value is associated to the state of the said supply chain;
c. providing the state of the said supply chain with the calculated value of the reward function to a … model (606);
d. receiving from the … model an optimal action (608), wherein the optimal action is associated with the provided state of the said supply chain and the calculated value of the reward function;
e. updating the state of the said supply chain with the optimal action (610).
Step 2A, Prong 1: The limitations for Claim 1 described above fall within “Certain Methods of Organizing Human Activity” for commercial interactions such as business relations. Accordingly, this claim recites an abstract idea.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. Claim 1 recites additional elements – “databases, data lakes, data warehouses, cloud infrastructures, blockchain” and “trained model”. The claim as a whole merely describes how to generally “apply” the abstract idea by using generic computer components. The claimed computer components are recited at high level of generality and merely invoked as a tool to perform a process for optimizing a multi-distribution-level supply chain (See MPEP 2106.05(f)). Simply implementing the abstract idea on a generic computer component is not a practical application. Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. This claim is directed to an abstract idea.
Step 2B: Claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of using a computer system to perform a process for optimizing a multi-distribution-level supply chain amount to no more than how to generally “apply” the exception using a generic computer component (See MPEP 2106.05(f)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. As a result, this claim is not patent eligible.
Claims 2-3 and 5-6 are directed to substantially the same abstract idea as Claim 1 and are rejected for substantially the same reasons. The additional recited limitations of the dependent claims fail to establish that the claims do not recite an abstract idea because the additional recited limitations of the claims further narrow the abstract idea. These dependent claims further narrow the abstract idea of Claim 1 such as by defining “wherein the multi-distribution-level supply chain is a multi-echelon supply chain” in Claim 2, by defining “wherein the reward function is a target cost function” in Claim 3, by defining “wherein the optimal action comprises a sequence of one or more steps of a reconfiguration of the supply chain” in Claim 5, and by defining “wherein the reconfiguration of the supply chain comprises a reorder policy” in Claim 6.
Step 2A, Prong 2: These dependent claims do not integrate the abstract idea into practical application because they do not recite additional elements.
Step 2B: These dependent claims do not amount to significantly more than the abstract idea because they do not recite additional elements. Therefore, these claims are not patent eligible.
Claims 4 and 8-10 are directed to substantially the same abstract idea as Claim 1 and are rejected for substantially the same reasons. The additional recited limitations of the dependent claims fail to establish that the claims do not recite an abstract idea because the additional recited limitations of the claims further narrow the abstract idea. These dependent claims further narrow the abstract idea of Claim 1 such as by defining “wherein the model comprises … model” in Claim 4, by defining “wherein the supply chain nodes comprise …, wherein the supply chain nodes data exchange processes are data from …” in Claim 8, by defining “wherein the simulated virtual environment is …” in Claim 9, and by defining “wherein the virtual representation is …” in Claim 10.
Step 2A, Prong 2: Claims 4 and 8-10 do not integrate the abstract idea into practical application. Claim 4 recites an additional element – “a deep reinforcement learning”, Claim 8 recites additional elements – “internet of things, industrial internet of things, [or] cyberphysical systems” and “databases, data lakes, data warehouses, cloud infrastructures, [or] blockchain”, Claim 9 recites an additional element – “Markovian”, and Claim 10 recites an additional element – “a digital twin”. The additional elements of “a deep reinforcement learning”, “Markovian”, and “a digital twin” represent mere generally linking of the use of the judicial exception (the abstract idea) to a particular technological environment or field of use (See MPEP 2106.05(h)). Other additional elements amount to no more than mere instructions to apply the exception using generic computer components (See MPEP 2106.05(f)). The limitations of these dependent claims do not integrate an abstract idea into a practical application because individually or in combination, these additional elements do not impose any meaningful limits on practicing the abstract idea.
Step 2B: Claims 4 and 8-10 do not amount to significantly more than the abstract idea. Claims 4 and 8-10 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of using a computer system to perform a process for optimizing a multi-distribution-level supply chain amount to no more than how to generally “apply” the exception using a generic computer component (See MPEP 2106.05(f)) and representing mere generally linking of the use of the abstract idea to a particular technological environment or field of use (See MPEP 2106.05(h)). Mere instructions to apply an exception using a generic computer component and representing mere generally linking of the use of the abstract idea to a particular technological environment or field of use cannot provide an inventive concept. Therefore, these claims are not patent eligible.
Claim 7 recites the following limitations:
The method of claim 1, further comprising training the model, the training comprising the steps of:
a. receiving a virtual representation of the state of the said supply chain (702), wherein the virtual representation of the state of the said supply chain comprises … of the said supply chain updated with real-world data from the said supply chain, wherein said … comprises supply chain nodes and their data exchange processes, and wherein real-world data comprise data from …;
b. calculating a value of a reward function (704), wherein the value is associated to the obtained virtual representation of the state of the said supply chain;
c. providing said virtual representation of the state of the said supply chain with the calculated value of the reward function to the model (706), wherein the model comprises …;
d. running … (708), wherein running … comprises:
i. obtaining at least one modified representation of the state of the said supply chain by taking at least one action on the provided virtual representation of the state of the said supply chain;
ii. calculating at least one value of a reward function, wherein the value is associated to the at least one modified representation of the state of the said supply chain and to the at least one said action;
iii. selecting from the said one or more actions an optimal action based on the value of the reward function associated to the said optimal action;
e. outputting the … model (710), wherein the … model comprises the selected optimal action.
Claim 7 is directed to substantially the same abstract idea as Claim 1 and is rejected for substantially the same reasons. The additional recited limitations of the dependent claim fail to establish that the claim does not recite an abstract idea because the additional recited limitations of the claim further narrow the abstract idea.
Step 2A, Prong 2: Claim 7 does not integrate the abstract idea into practical application. Claim 7 recites additional elements – “a simulated environment”, “databases, data lakes, data warehouses, cloud infrastructures, [or] blockchain”, “an optimizer”, and “trained model”. These additional elements amount to no more than mere instructions to apply the exception using generic computer components. The limitations of this dependent claim do not integrate an abstract idea into a practical application because individually or in combination, these additional elements do not impose any meaningful limits on practicing the abstract idea and amount to no more than mere instructions to apply the exception using a generic computer component. (See MPEP 2106.05(f)).
Step 2B: Claim 7 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of using a computer system to perform a process a process for optimizing a multi-distribution-level supply chain amount to no more than how to generally “apply” the exception using a generic computer component (See MPEP 2106.05(f)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Therefore, this dependent claim is not patent eligible.
Claim 13 recites the following limitations:
… the steps of:
a. obtaining a state of the said supply chain, wherein the state is determined based on real-time data from the supply chain, wherein real-time data comprise data from …;
b. calculating a value of a reward function, wherein the value is associated to the state of the said supply chain;
c. providing the state of the said supply chain with the calculated value of the reward function to a … model;
d. receiving from the … model an optimal action, wherein the optimal action is associated with the provided state of the said supply chain and the calculated value of the reward function;
e. updating the state of the said supply chain with the optimal action.
Step 2A, Prong 1: The limitations for Claim 1 described above fall within “Certain Methods of Organizing Human Activity” for commercial interactions such as business relations. Accordingly, this claim recites an abstract idea.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. Claim 13 recites additional elements – “a computer-readable storage medium comprising instructions which, when the program is executed by a computer, cause the computer to carry out”, “databases, data lakes, data warehouses, cloud infrastructures, blockchain” and “trained model”. The claim as a whole merely describes how to generally “apply” the abstract idea by using generic computer components. The claimed computer components are recited at high level of generality and merely invoked as a tool to perform a process for optimizing a multi-distribution-level supply chain (See MPEP 2106.05(f)). Simply implementing the abstract idea on a generic computer component is not a practical application. Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. This claim is directed to an abstract idea.
Step 2B: Claim 13 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of using a computer system to perform a process for optimizing a multi-distribution-level supply chain amount to no more than how to generally “apply” the exception using a generic computer component (See MPEP 2106.05(f)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. As a result, this claim is not patent eligible.
Claim 14 recites the following limitations:
A system comprising:
a. … (202) configured to receive data, wherein data comprises real-time data;
b. … (204) configured to perform the steps of:
i. obtaining a state of the said supply chain, wherein the state is determined based on real-time data from the supply chain, wherein real-time data comprise data from …;
ii. calculating a value of a reward function, wherein the value is associated to the state of the said supply chain;
iii. providing the state of the said supply chain with the calculated value of the reward function to a … model;
iv. receiving from the … model an optimal action, wherein the optimal action is associated with the provided state of the said supply chain and the calculated value of the reward function;
v. updating the state of the said supply chain with the optimal action.
Step 2A, Prong 1: The limitations for Claim 14 described above fall within “Certain Methods of Organizing Human Activity” for commercial interactions such as business relations. Accordingly, this claim recites an abstract idea.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. Claim 14 recites additional elements – “an input/output (I/O) unit”, “a processor”, “databases, data lakes, data warehouses, cloud infrastructures, blockchain” and “trained model”. The claim as a whole merely describes how to generally “apply” the abstract idea by using generic computer components. The claimed computer components are recited at high level of generality and merely invoked as a tool to perform a process for optimizing a multi-distribution-level supply chain (See MPEP 2106.05(f)). Simply implementing the abstract idea on a generic computer component is not a practical application. Accordingly, alone and in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. This claim is directed to an abstract idea.
Step 2B: Claim 14 does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of using a computer system to perform a process for optimizing a multi-distribution-level supply chain amount to no more than how to generally “apply” the exception using a generic computer component (See MPEP 2106.05(f)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. As a result, this claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3, 5, and 13-14 are rejected under 35 U.S.C. 102(a)(1) as being clearly anticipated by Devarakonda et al. (US PG Pub. No. 2020/0210947 A1; hereinafter "Devarakonda").
Regarding Claim 1, Devarakonda teaches a computer-implemented method for optimizing a multi-distribution-level supply chain, wherein the method comprises: a. obtaining a state of the said supply chain (602), wherein the state is determined based on real-time data from the supply chain, wherein real-time data comprise data from databases, data lakes, data warehouses, cloud infrastructures, [or] blockchain (See “The method 500 commences at operation 600, ingesting supply chain data 306 is ingested (e.g., received) from a complex supply chain. For example, a data analyst may ingest the supply chain data 306 from the supply chain 100. The ingested supply chain data 306 relates to operational plans and observed (e.g., measured) supply chain operational metrics across the supply chain 100. The supply chain data 306 may be received from various external data sources 202, such as those described above with reference to FIG. 2. In addition to ingestion (e.g., in batches), the supply chain data 306 may be streamed, in real time, from the external data sources 202.” in Paragraph [0082] and “The external data sources 202 may receive and process information pertaining to a supply chain, such as the supply chain 100 shown in FIG. 1. These external data sources 202 include planning systems 208 including an enterprise resource planning (ERP) system 204 and a customer resource planning (CRM) system 206. Various supply chain information items, which will be discussed in further detail below, are extracted from the external data sources 202, and stored in an intermediate staging database 210, before being transferred to the relational database system 212. Here, the supply chain information is staged within the input staging database 214, before being transferred to the production database 216.” in Paragraph [0060]); b. calculating a value of a reward function (604), wherein the value is associated to the state of the said supply chain; c. providing the state of the said supply chain with the calculated value of the reward function to a trained model (606) (See “An optimization may be performed on the objective of interest/maximize the reward (e.g., value at risk, which may be a combination of lost sales, inventory holding costs, and expedite costs) at a network node or at a global level. The possible action sequences that maximize the desired objective function (based on a scored cost function/reward function) are then presented, within respective user interfaces, visually to equip a user with a set of choices based on which they may make decisions, and based on which the user, at operation 516, can authorize a specific workflow that is to be executed by the system described by interfacing and exchanging data with the real-time execution system within the user domain.” in Paragraph [0094]); d. receiving from the trained model an optimal action (608), wherein the optimal action is associated with the provided state of the said supply chain and the calculated value of the reward function (See “The option to operate in fully-autonomous mode may include the optimal action sequence being executed automatically by the machine-learning (ML) application 218. For example, the machine-learning (ML) application 218 may automatically identify the first action recommendation as an optimal recommended action by identifying a predicted value impact as the highest predicted value impact from a set of predicted value impacts each associated with an action recommendation for a product in the supply chain 100. The delivery date of a product may be initially received as part of an operational plan (e.g., a planned delivery date) but then changed in the fully-autonomous mode.” in Paragraph [0097]); e. updating the state of the said supply chain with the optimal action (610) (See “In addition, the production run 1522 is associated with updated predicted supply chain operational metrics. For example, the updated predicted supply chain operational metrics include an updated value at risk ranking for the production run 1522 from “HIGH” to “LOW.” an updated capacity metric in the capacity panel 1524 in the alternate production panel 1604 indicating an increase in the utilization of the production resource “X998AB” (see three histogram bars), and an updated capacity metric in the capacity panel 1524 in the primary production panel 1602 indicating a decrease in utilization of the production resource “X999AB” (see three histogram bars).” in Paragraph [0186]).
Claim 13 is a product claim corresponding to method Claim 1. All of the limitations in Claim 13 are found reciting the same scopes of the respective limitations in Claim 1. Accordingly, Claim 13 is considered anticipated (rejection) by the same rationales presented in the rejection of Claim 1, respectively set forth above.
Claim 14 is a system claim corresponding to method Claim 1. All of the limitations in Claim 14 are found reciting the same scopes of the respective limitations in Claim 1. Accordingly, Claim 14 is considered anticipated (rejection) by the same rationales presented in the rejection of Claim 1, respectively set forth above.
Regarding Claim 3, Devarakonda teaches all the limitations of Claim 1 as described above. Devarakonda also teaches wherein the reward function is a target cost function (See “An optimization may be performed on the objective of interest/maximize the reward (e.g., value at risk, which may be a combination of lost sales, inventory holding costs, and expedite costs) at a network node or at a global level. The possible action sequences that maximize the desired objective function (based on a scored cost function/reward function) are then presented, within respective user interfaces, visually to equip a user with a set of choices based on which they may make decisions, and based on which the user, at operation 516, can authorize a specific workflow that is to be executed by the system described by interfacing and exchanging data with the real-time execution system within the user domain.” in Paragraph [0094]).
Regarding Claim 5, Devarakonda teaches all the limitations of Claim 1 as described above. Devarakonda also teaches wherein the optimal action comprises a sequence of one or more steps of a reconfiguration of the supply chain (See “The utilization of the manual mode, and/or the semi-autonomous mode, and/or the fully-autonomous mode may cause the execution of one or more actions (e.g., sequence of actions). The one or more actions may impact a delivery date of a product. For example, the one or more actions may include a sequence of actions impacting the product (e.g., delivery date of the product) including a sequence of actions to expedite a shipment of the product (e.g., first action), a sequence of actions to reallocate quantities of products being shipped in outbound shipments (e.g., second action), a sequence of actions to reschedule a production (e.g., production run) of a product (e.g., lipstick, red) on a first production resource (e.g., third action), a sequence of actions to reschedule a production of a product from the first production resource to a second production resource (e.g., fourth action), and a sequence of actions to increase a quantity of the production of the product on the first production resource (e.g., fifth action). The action sequence may be described by an action description describing a set of actions including one or more actions impacting a delivery date of the product, such as those described above.” in Paragraph [0095]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2, 4, and 6-10 are rejected under 35 U.S.C. 103 as being unpatentable over Devarakonda in view of Yan et al. (Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities; hereinafter "Yan").
Regarding Claim 2, Devarakonda teaches all the limitations of Claim 1 as described above. Devarakonda does not explicitly teach; however, Yan teaches wherein the multi-distribution-level supply chain is a multi-echelon supply chain (See “multi-echelon SC model” in Section 3.4.3).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the process of Devarakonda to include wherein the multi-distribution-level supply chain is a multi-echelon supply chain, as taught by Yan since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding Claim 4, Devarakonda teaches all the limitations of Claim 1 as described above. Devarakonda does not explicitly teach; however, Yan teaches wherein the model comprises a deep reinforcement learning model (See “reinforcement learning” in Section 2.2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the process of Devarakonda to include wherein the model comprises a deep reinforcement learning model, as taught by Yan since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding Claim 6, Devarakonda teaches all the limitations of Claims 1 and 5 as described above. Devarakonda does not explicitly teach; however, Yan teaches wherein the reconfiguration of the supply chain comprises a reorder policy (See “They propose a neuro-DP approach to deriving policies and improving actions, with the goal of minimising the expected total discounted cost.” in Section 3.4.1 and “Reorder rule” in Table 14).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the process of Devarakonda to include wherein the reconfiguration of the supply chain comprises a reorder policy, as taught by Yan since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding Claim 7, Devarakonda teaches all the limitations of Claim 1 as described above. Devarakonda also teaches training the model, the training comprising the steps of (See “A machine-learning application trainer 224 operates to train the machine-learning application 218, as described further in FIG. 3.” in Paragraph [0065]): … d. running the optimizer (708), wherein running the optimizer comprises: … iii. selecting from the said one or more actions an optimal action based on the value of the reward function associated to the said optimal action (See “The semi-autonomous mode receives user input causing selection of an action recommendation from a set of action recommendation actions. The action recommendations may be prioritized according to a value at risk. Each action recommendation includes an action sequence including one or more actions that, if executed, eliminates or significantly reduces the value at risk. Each action recommendation is executed responsive to its selection in accordance with real-time information. The semi-autonomous mode may be utilized to select an action recommendation including action sequences effecting the movement of products in the supply chain or effecting production of products in the supply chain. For example, the semi-autonomous mode may be utilized to change to a means of a shipment of a product, or to change a quantity of product shipped, or to change a utilization of one or more production resources for producing the product.” in Paragraph [0052]); e. outputting the trained model (710), wherein the trained model comprises the selected optimal action (See “With the supply chain training data 304 and the identified features 302, the machine-learning application 218 (e.g., model) is trained by machine-learning application trainer 224. The machine-learning application 218 appraises the value of the features 302 as they correlate to the supply chain training data 304. The result of the training is the machine-learning application 218.” in Paragraph [0077] and “The option to operate in fully-autonomous mode may include the optimal action sequence being executed automatically by the machine-learning (ML) application 218. For example, the machine-learning (ML) application 218 may automatically identify the first action recommendation as an optimal recommended action by identifying a predicted value impact as the highest predicted value impact from a set of predicted value impacts each associated with an action recommendation for a product in the supply chain 100. The delivery date of a product may be initially received as part of an operational plan (e.g., a planned delivery date) but then changed in the fully-autonomous mode.” in Paragraph [0097]).
Devarakonda does not explicitly teach; however, Yan teaches a. receiving a virtual representation of the state of the said supply chain (702), wherein the virtual representation of the state of the said supply chain comprises a simulated environment of the said supply chain updated with real-world data from the said supply chain, wherein said simulated environment comprises supply chain nodes and their data exchange processes, and wherein real-world data comprise data from databases, data lakes, data warehouses, cloud infrastructures, [or] blockchain; b. calculating a value of a reward function (704), wherein the value is associated to the obtained virtual representation of the state of the said supply chain; c. providing said virtual representation of the state of the said supply chain with the calculated value of the reward function to the model (706), wherein the model comprises an optimizer (See Section 2.4.1 and 2.4.2 for the training steps of Reinforcement Learning model, “Digital twin-enabled logistics and supply chain applications: Most of the RL frameworks reviewed in this paper rely on a simulation model to represent the environment (e.g., Mo and Ohmori, 2021; Zeng et al., 2011). Recently, digital twin technologies, which construct real-time virtual representations of physical assets or systems to model the physical interactions in the digital space (Mikell and Clark, 2018), have become mature and gained significant adoptions in various types of real-world applications (Kuo et al., 2021). Enabled by the Internet of Things, the real-time physical system state can be captured. Applications which leverage digital twin technologies to incorporate the real environment for the RL agents to interact with are expected to provide more effective and realistic model training and practical decisions.” in Section 4.3, “Finally, the use of real-world data would further demonstrate the practicality of the research and provide non-trivial insights into addressing realistic problems.” in Section 3.3.7); and i. obtaining at least one modified representation of the state of the said supply chain by taking at least one action on the provided virtual representation of the state of the said supply chain; ii. calculating at least one value of a reward function, wherein the value is associated to the at least one modified representation of the state of the said supply chain and to the at least one said action (See “Digital twin-enabled logistics and supply chain applications: Most of the RL frameworks reviewed in this paper rely on a simulation model to represent the environment (e.g., Mo and Ohmori, 2021; Zeng et al., 2011). Recently, digital twin technologies, which construct real-time virtual representations of physical assets or systems to model the physical interactions in the digital space (Mikell and Clark, 2018), have become mature and gained significant adoptions in various types of real-world applications (Kuo et al., 2021). Enabled by the Internet of Things, the real-time physical system state can be captured. Applications which leverage digital twin technologies to incorporate the real environment for the RL agents to interact with are expected to provide more effective and realistic model training and practical decisions.” in Section 4.3, “With online methods, the approximations are calculated and updated every time when a state is revealed and a decision is made. Despite the computational effort required for online training, the advantage of an online method over an offline one is its adaptiveness to prevailing system conditions, in particular when there is a significant change in system characteristics before and after the control process starts.” in Section 2.2.4, and “On the other hand, Bayesian inference can also be applied in a model-free setting, in which value function or policy are updated via new sample transitions. In particular, Gaussian Process Temporal Difference Learning (GPTD) applies Gaussian Processes to evaluate value function for any given policy. Building upon the same idea as GPTD, GPSARSA estimates the 𝑄 function using Gaussian Processes and determines the action at each time step via sampling and estimating the 𝑄 function over action space. Other than methods that are based on value function, Bayesian PG, and Bayesian AC adopt the ideas from PG methods in nominal RL settings.” in Section 2.6).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the process of Devarakonda to include the steps of receiving, calculating, and obtaining described above, as taught by Yan since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding Claim 8, Devarakonda in view of Yan teaches all the limitations of Claims 1 and 7 as described above. Devarakonda also teaches wherein the supply chain nodes comprise internet of things, industrial internet of things, [or] cyberphysical systems, wherein the supply chain nodes data exchange processes are data from databases, data lakes, data warehouses, cloud infrastructures, [or] blockchain (See “The example supply chain includes nodes in the form of a materials supplier 102, a plant 104, a hub 106 and a distribution center 108. The material suppliers 102 fulfill material purchase orders (POs) received from a plant 104, such fulfillment including a specific lead time (e.g., five weeks). The plant 104 executes a process order (e.g., production) to produce products (e.g., stock), fulfils stock transfer orders to the hub 106 and the distribution center 108, each of these fulfillments in turn having their respective lead times.” in Paragraph [0056], “The external data sources 202 may receive and process information pertaining to a supply chain, such as the supply chain 100 shown in FIG. 1. These external data sources 202 include planning systems 208 including an enterprise resource planning (ERP) system 204 and a customer resource planning (CRM) system 206. Various supply chain information items, which will be discussed in further detail below, are extracted from the external data sources 202, and stored in an intermediate staging database 210, before being transferred to the relational database system 212. Here, the supply chain information is staged within the input staging database 214, before being transferred to the production database 216.” in Paragraph [0060], and “The method 1660 commences, at operation 1662, with the supply chain server machine 205 communicating an electronic user interface over a network (e.g., Internet) to the client machine 203. For example, the machine-learning application 218 may communicate the user interface, which may include the production run user interface 1500, as illustrated in FIG. 14A.” in Paragraph [0200]).
Regarding Claim 9, Devarakonda in view of Yan teaches all the limitations of Claims 1 and 7 as described above. Devarakonda does not explicitly teach; however, Yan teaches wherein the simulated virtual environment is Markovian (See “Markov decision process” in Section 2.1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the process of Devarakonda to include wherein the simulated virtual environment is Markovian, as taught by Yan since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding Claim 10, Devarakonda in view of Yan teaches all the limitations of Claims 1 and 7 as described above. Devarakonda does not explicitly teach; however, Yan teaches also teaches wherein the virtual representation is a digital twin (See “Digital twin-enabled logistics and supply chain applications” in Section 4.3).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the process of Devarakonda to include wherein the virtual representation is a digital twin, as taught by Yan since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TAYAR M KYU whose telephone number is (571)272-3419. The examiner can normally be reached Mon-Fri 9:00 am - 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeffrey Zimmerman can be reached at 571-272-4602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.M.K./Examiner, Art Unit 3628
/GEORGE CHEN/Primary Examiner, Art Unit 3628
Prosecution Insights