Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
101 Subject Matter Eligibility Analysis
Step 1: Claims 1-20 are within the four statutory categories (a process, machine, manufacture or composition of matter).
Step 2A Prong One, Step 2A Prong Two, and Step 2B Analysis:
Step 2A Prong One asks if the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon). If the claim recites a judicial exception, analysis proceeds to Step 2A Prong Two, which asks if the claim recites additional elements that integrate the abstract idea into a practical application. If the claim does not integrate the judicial exception, analysis proceeds to Step 2B, which asks if the claim amounts to significantly more than the judicial exception. If the claim does not amount to significantly more than the judicial exception, the claim is not eligible subject matter under 35 U.S.C. 101.
None of the claims represent an improvement to technology.
Claims 1-16 are directed to a method consisting of a series of steps, meaning that it is directed to the statutory category of process. Claims 17-20 are directed to storage mediums and computers which are machines.
Regarding claim 1, the following claim elements are abstract ideas:
generating synthetic network disruption data including sampling from one or more second travel time distributions corresponding respectively to one or more simulated network disruptions (This is an abstract idea of a mental process. The limitation involves performing mathematical estimation to model hypothetical disruption scenarios by selecting representative travel-time values from estimated distributions. A person could estimate how a disruption such as a storm or labor shortage would affect delivery times, mathematically approximate those delays, and record example values to represent the disruption. Such mathematical estimation and selection of representative values can be performed in the human mind or with the air of pen and paper, and therefore falls within the mental process grouping of abstract ideas. See MPEP 2106.04(a)(2)(III).);
generating a second dataset having the synthetic network disruption data (This is an abstract idea of a mental process. The limitation involves organizing and compiling previously generated hypothetical disruption information into a collection of data. A person could mentally record estimated disruption scenarios and corresponding values into a table or list, thereby forming a dataset. Such organization and compilation of information based on observation and recording can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process groupings of abstract ideas.);
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
receiving first dataset representing movement of objects through a plurality of entities in a supply chain network, wherein the first dataset includes (i) travel time data representing how long it took for each object to be transported between at least two entities in the supply chain network and (ii) one or more features representing respective conditions of the supply chain network when the objects were transported (The step of “receiving” a dataset is merely a generic data operation that amounts to obtaining or transmitting information for use in conjunction with an abstract idea. The limitation represents insignificant extra-solution activity and constitutes well-understood, routine, and conventional activity, such as receiving or transmitting data over a network or from memory. See MPEP 2106.05(d)(II)(i) and MPEP 2106.05(g).);
obtaining data representing one or more first travel time distributions between at the at least two entities in the supply chain network (The step of “obtaining” data is merely a generic data operation that amounts to retrieving or accessing information from memory or another data source in conjunction the abstract idea. Such obtaining of data has been recognized as well-understood, routine, and conventional activity.);
training a network policy agent using the second dataset (This limitation are merely instructions to apply the abstract idea using a generic machine learning model and does not provide a meaningful limitation.)
Regarding claim 2, the rejection of claim 1 is incorporated herein. Further, claim 2 recites the following abstract ideas:
generating the first travel time distributions from the first dataset (This is an abstract idea of a mental process. The limitation involves performing mathematical estimation to summarize observed travel time into representative groupings or ranges. A person could review historical delivery records, estimate typical travel durations between entities, and organize those values into distributions based on observation and judgement. Such mathematical estimation and abstraction of real-world behavior can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process grouping of abstract ideas.).
Regarding claim 3, the rejection of claim 1 is incorporated herein. Further, claim 3 recites the following abstract ideas:
generating the second travel time distributions from elements of the first dataset having conditions meeting one or more disruption criteria (The limitation involves identifying data that satisfies specified disruption conditions and performing mathematical estimation to form representative travel-time disruption for those conditions. A person could review historical records, identify instances that meet disruption criteria (e.g., severe weather or delays), and estimate corresponding travel-time ranges. Such selection, evaluation, and mathematical estimation based on observed conditions can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process grouping of abstract ideas.).
Regarding claim 4, the rejection of claim 1 is incorporated herein. Further, claim 4 recites the following abstract ideas:
maintaining a mapping between different types of network disruptions and corresponding second travel time distributions (The limitation involves associated different disruption types with estimated travel-time effects. A person could mentally or manually maintain such associations by noting, for example, that a snowstorm typically causes moderate delays while a labor strike causes longer delays, and recording those relationships in a table or a list. Such classification and association conditions with estimated outcomes based on human judgement can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process groupings of abstract ideas.);
implementing, using the mapping, a modification to a first travel time distribution to generate a second travel time distribution (This is an abstract idea of a mental process. The limitation involves applying known disruption effects to an existing estimate of travel times to produce a new estimated travel time distribution that reflects the disruption. A person could start with typical travel-time ranges, use the mapping to adjust those ranges based on a selected disruption (e.g., adding delay for a snowstorm or strike), and thereby form a modified set of expected travel times, i.e., a second travel time distribution. Such mathematical estimation and adjustment to create a new distribution can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process grouping of abstract ideas.);
sampling from the second travel time distribution that corresponds to the user-selected type of network distribution to generate the synthetic network disruption data (The limitation involves selecting representative travel-time values from an estimated distribution based on a chosen disruption type. A person could review the modified travel-time range, pick example delay values that reflect the disruption (e.g., choosing longer delivery times for a storm scenario), and use those selected values to form hypothetical disruption scenarios. Such selection and estimation based on judgement can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process grouping of abstract ideas.).
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
receiving user input selecting one or more of the different types of network disruptions (This limitation is merely directed to receiving data from a user and amounts to a generic data input operation. Such receiving of user input is a well-understood, routine, and conventional activity for a computer system and constitutes insignificant extra-solution activity.);
Regarding claim 5, the rejection of claim 4 is incorporated herein. Further, claim 5 recites the following abstract ideas:
selecting the one or more of the different types of network disruptions comprises natural language input indicating the one or more of the different types of network disruptions (The limitation involves a user identifying and expressing, in natural language, which types of disruptions are of interest. A person could mentally decide that certain disruptions (e.g., hurricanes or labor strikes) are relevant and communicate that selection using ordinary language. Such selection and expression of choices based on human judgement can be performed in the human mind and therefore falls within the mental process grouping of abstract ideas.).
Regarding claim 6, the rejection of claim 5 is incorporated herein. Further, claim 6 recites the following abstract ideas:
identifying a label corresponding to the one or more of the different types of network disruptions based on processing the user input with natural language processing techniques (The limitation involves interpreting user-provided language to determine an associated category or label. A person could read a natural language description (e.g., “we are concerned about hurricanes”), understand its meaning, and assign a corresponding label (e.g., “weather-related disruption”) based on comprehension and judgement. Such interpretation and categorization of language can be performed in the human mind and therefore falls within the mental process grouping of abstract ideas.).
Regarding claim 7, the rejection of claim 5 is incorporated herein. Further, claim 7 recites the following abstract ideas:
identifying a label corresponding to a nature of the one or more of the different types of network disruptions, a distribution of the one or more of the different types of network disruptions, and a severity of the one or more of the different types of network disruptions (The limitation involves categorizing disruptions by their type, associated distribution characteristics, and degree of impact. A person could review information describing a disruption, determine its general nature (e.g., weather-related or labor-related), consider how the disruption affects estimated values reflected in a distribution, and judge the severity of its impact. Such categorization and judgement based on observation and reasoning can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process grouping of abstract ideas.).
Regarding claim 8, the rejection of claim 1 is incorporated herein. Further, claim 8 recites the following abstract ideas:
that implements a machine learned policy that grants rewards exceeding a threshold reward level for more severe disruptions to the supply chain network (This is abstract idea of a mental process. The limitation involves evaluating the severity of disruptions and assigning higher rewards when the severity exceeds a threshold. A person could review disruption scenarios, judge which disruptions are more severe, compare the severity to a threshold level, and determine that greater credit or priority should be given to more severe disruptions. Such evaluation, comparison, and threshold-based decision-making can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process grouping of abstract ideas.),
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
obtaining an adversarial agent (This limitation merely recites obtaining a software component, which is a well-understood, routine, and conventional activity.)
wherein sampling from the one or more second travel time distributions comprises causing the adversarial agent to sample from the second travel time distributions (This limitation merely recites instructing an adversarial agent to perform the sampling step, which amounts to a mere instruction to apply the abstract idea and insignificant extra-solution activity.).
Regarding claim 9, the rejection of claim 1 is incorporated herein. Further, claim 9 recites the following abstract ideas:
wherein generating the synthetic network disruption data comprises generating a plurality of correlated modified travel time distributions (This limitation involves mathematically estimating and correlating modified travel time distributions. A person could use basic calculations – such as adjusting values and aligning distributions based on assumed relationships – using the human mind with basic computational tools, pen, and paper. Such mathematical estimation and correlation fall within the mathematical concepts and mental process grouping of abstract ideas. See MPEPE 2105.04(a)(2)(I) and 2106.04(a)(2)(III).).
Regarding claim 10, the rejection of claim 1 is incorporated herein. Further, claim 10 recites the following abstract ideas:
wherein generating the synthetic network disruption data further comprises generating a plurality of correlated (i) modified supply, (ii) modified demand, and (iii) modified traffic costs for the supply chain network (This is an abstract idea of a mental process. The limitation involves estimating and adjusting supply levels, demand levels, and traffic costs and determining their relationships among those adjustments. A person could, using judgement and basic calculations, consider how changes in one factor affect the others and assign corresponding modified values using the human mind with basic computational tools, pen and paper. Such estimation and correlation based on reasoning fall within the mental process grouping of abstract ideas.).
Regarding claim 11, the rejection of claim 1 is incorporated herein. Further, claim 11 recites the following abstract ideas:
to identify potential disruptions to the supply chain network (The limitation involves observing information related to supply chain operations and determining whether certain conditions may constitute potential disruptions. Such identification based on observation and reasoning can be performed in the human mind or with basic computation tools such as pen and paper or a calculator, and therefore falls within the mental process groupings of abstract ideas.)
generating, based on execution of the trained machine learning network policy agent and identification of the potential disruptions to the supply chain network, recommendations to reorganize operations of the supply chain network (This is an abstract idea of a mental process. The limitation involves determining suggested changes to operations based on identified disruption conditions. A person could review information indicating potential disruptions and, through reasoning and judgement, decide what operational changes should be made in response. Such evaluation and recommendation-making based on observed conditions can be performed in the human mind or with basic computational tools such as pen and paper or a calculator, and therefore falls within the mental process grouping of abstract ideas.).
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
executing the trained network policy agent (This limitation merely instructions execution of a trained model to apply the abstract and does not provide a meaningful limitation.)
Regarding claim 12, the rejection of claim 11 is incorporated herein. Further, claim 12 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the identified potential disruptions include a storm, a shipment delay, an object production delay, or a labor strike (This limitation merely provides examples of types of disruptions and does not add any additional processing or meaningful limitation to the abstract idea. Listing exemplary disruption scenarios constitutes insignificant extra-solution activity that does not integrate the judicial exception into a practical application.).
Regarding claim 13, the rejection of claim 11 is incorporated herein. Further, claim 13 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the generated recommendations apply to more than one of the identified potential disruptions, wherein each of the identified potential disruptions is a different type of disruption to the supply chain network (This limitation amounts to insignificant extra-solution activity and does add a meaningful limitation. See MPEP 2106.05(g).).
Regarding claim 14, the rejection of claim 1 is incorporated herein. Further, claim 14 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the first dataset includes historic supply chain data associated with the supply chain network (This limitation amounts to insignificant extra-solution activity and does add a meaningful limitation. See MPEP 2106.05(g).).
Regarding claim 15, the rejection of claim 1 is incorporated herein. Further, claim 15 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
wherein the first dataset includes historic supply chain data associated with at least one other supply chain network, wherein the at least one other supply chain network is different than the supply chain network (This limitation amounts to insignificant extra-solution activity and does add a meaningful limitation. See MPEP 2106.05(g).).
Regarding claim 16, the rejection of claim 1 is incorporated herein. Further, claim 16 recites the following additional elements, which taken alone or in combination with other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
further comprising returning the trained network policy agent for runtime use in the supply chain network (This limitation merely instructs applying the abstract idea by providing the trained agent for use and does not add a meaningful limitation.).
Regarding claim 17, the following claim elements are abstract ideas:
generating synthetic network disruption data including sampling from one or more second travel time distributions corresponding respectively to one or more simulated network disruptions (This is an abstract idea of a mental process. The limitation involves performing mathematical estimation to model hypothetical disruption scenarios by selecting representative travel-time values from estimated distributions. A person could estimate how a disruption such as a storm or labor shortage would affect delivery times, mathematically approximate those delays, and record example values to represent the disruption. Such mathematical estimation and selection of representative values can be performed in the human mind or with the air of pen and paper, and therefore falls within the mental process grouping of abstract ideas. See MPEP 2106.04(a)(2)(III).);
generating a second dataset having the synthetic network disruption data (This is an abstract idea of a mental process. The limitation involves organizing and compiling previously generated hypothetical disruption information into a collection of data. A person could mentally record estimated disruption scenarios and corresponding values into a table or list, thereby forming a dataset. Such organization and compilation of information based on observation and recording can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process groupings of abstract ideas.);
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
one or more computers and one or more storage devices storing instructions that are operable (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
receiving first dataset representing movement of objects through a plurality of entities in a supply chain network, wherein the first dataset includes (i) travel time data representing how long it took for each object to be transported between at least two entities in the supply chain network and (ii) one or more features representing respective conditions of the supply chain network when the objects were transported (The step of “receiving” a dataset is merely a generic data operation that amounts to obtaining or transmitting information for use in conjunction with an abstract idea. The limitation represents insignificant extra-solution activity and constitutes well-understood, routine, and conventional activity, such as receiving or transmitting data over a network or from memory. See MPEP 2106.05(d)(II)(i) and MPEP 2106.05(g).);
obtaining data representing one or more first travel time distributions between at the at least two entities in the supply chain network (The step of “obtaining” data is merely a generic data operation that amounts to retrieving or accessing information from memory or another data source in conjunction the abstract idea. Such obtaining of data has been recognized as well-understood, routine, and conventional activity.);
training a network policy agent using the second dataset (This limitation are merely instructions to apply the abstract idea using a generic machine learning model and does not provide a meaningful limitation.)
Regarding claim 18, the rejection of claim 17 is incorporated herein. The claim recites similar limitations corresponding to claim 3. Therefore, the same subject matter analysis that was utilized for claim 3, as described above, is equally applicable to claim 18.
Therefore, claim 18 is ineligible.
Regarding claim 19, the rejection of claim 17 is incorporated herein. The claim recites similar limitations corresponding to claim 4. Therefore, the same subject matter analysis that was utilized for claim 4, as described above, is equally applicable to claim 19.
Therefore, claim 19 is ineligible.
Regarding claim 20, the following claim elements are abstract ideas:
generating synthetic network disruption data including sampling from one or more second travel time distributions corresponding respectively to one or more simulated network disruptions (This is an abstract idea of a mental process. The limitation involves performing mathematical estimation to model hypothetical disruption scenarios by selecting representative travel-time values from estimated distributions. A person could estimate how a disruption such as a storm or labor shortage would affect delivery times, mathematically approximate those delays, and record example values to represent the disruption. Such mathematical estimation and selection of representative values can be performed in the human mind or with the air of pen and paper, and therefore falls within the mental process grouping of abstract ideas. See MPEP 2106.04(a)(2)(III).);
generating a second dataset having the synthetic network disruption data (This is an abstract idea of a mental process. The limitation involves organizing and compiling previously generated hypothetical disruption information into a collection of data. A person could mentally record estimated disruption scenarios and corresponding values into a table or list, thereby forming a dataset. Such organization and compilation of information based on observation and recording can be performed in the human mind or with the aid of pen and paper, and therefore falls within the mental process groupings of abstract ideas.);
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
One or more non-transitory computer storage media encoded with computer program instructions (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
one or more computers (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
receiving first dataset representing movement of objects through a plurality of entities in a supply chain network, wherein the first dataset includes (i) travel time data representing how long it took for each object to be transported between at least two entities in the supply chain network and (ii) one or more features representing respective conditions of the supply chain network when the objects were transported (The step of “receiving” a dataset is merely a generic data operation that amounts to obtaining or transmitting information for use in conjunction with an abstract idea. The limitation represents insignificant extra-solution activity and constitutes well-understood, routine, and conventional activity, such as receiving or transmitting data over a network or from memory. See MPEP 2106.05(d)(II)(i) and MPEP 2106.05(g).);
obtaining data representing one or more first travel time distributions between at the at least two entities in the supply chain network (The step of “obtaining” data is merely a generic data operation that amounts to retrieving or accessing information from memory or another data source in conjunction the abstract idea. Such obtaining of data has been recognized as well-understood, routine, and conventional activity.);
training a network policy agent using the second dataset (This limitation are merely instructions to apply the abstract idea using a generic machine learning model and does not provide a meaningful limitation.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4 and 8-20 are rejected under the 35 U.S.C. 103 as being unpatentable over Kumar at al., (Pub. No.: US 20230129665 A1 (Filed: 2021)). in view of Gutierrez et al., (Pub. No.: US 20220215141 A1 (Filed: 2021)).
Regarding claim 1, Kumar discloses:
receiving first dataset representing movement of objects through a plurality of entities in a supply chain network, wherein the first dataset includes (i) travel time data representing how long it took for each object to be transported between at least two entities in the supply chain network (Kumar, paragraph [0022] “The training data 20 may be organized into a plurality of training timesteps 58 that include time series data for a plurality of training-phase agents 22. The plurality of training-phase agents 22 may be included in a training supply chain graph 50 in which the plurality of training-phase agents 22 may be connected by training graph edges 56 that represent transactions performed between the training-phase agents 22…The directions of the training graph edges 56 may indicate directions in which materials and products move through the supply chain between training-phase agents 22.” – Kumar teaches receiving training data organized as time series data across training timesteps for agents with a supply chain graph. The training-phase agents correspond to entities in the supply chain network, and that training graph edges represent movement of materials and products between those entities. Because the movement occurs across time series timesteps, the training data includes temporal information representing how long it took for materials and products to move between at least two entities. Under the broadest reasonable interpretation, such time-based movement data constitutes travel time data representing transportation between entities.) and
(ii) one or more features representing respective conditions of the supply chain network when the objects were transported (Kumar, paragraph [0024] “the training data 20 may include a plurality of training forecast states 24. Each training forecast state of the plurality of training forecast states 24 may be associated with a respective training-phase agent 22…the plurality of training forecast states 24 may include a plurality of training upstream price forecasts 24A and a plurality of training downstream demand forecasts 24B. The plurality of training upstream price forecasts 24A may be predictions of respective prices charged by one or more other training-phase agents… The plurality of training downstream demand forecasts 24B may be predictions of respective quantities of a product…that are demanded by one or more other training-phase agents.” – teaches that the received training data includes training forecast states associated with agents in a supply chain network. These forecast states include upstream price forecasts and downstream demand forecasts, which are variables that describe the operational state of the supply chain. Under BRI, such price and demand variables constitute features representing respective conditions of the supply chain network. Accordingly, Kuman teaches features representing network conditions associated with the transportation of objects through the supply chain.);
obtaining data representing one or more first travel time distributions between at the at least two entities in the supply chain network (Kumar, paragraph [0022] “The training data 20 may be organized into a plurality of training timesteps 58… training graph edges 56 that represent transactions performed between the training-phase agents 22… the training supply chain graph 50 may be a directed graph in which the training graph edges 56 respective directions. The directions of the training graph edges 56 may indicate directions in which materials and products move through the supply chain between training-phase agents 22.” – Kumar teaches obtaining training data that represents movement of materials and products between entities (training-phased agents) in a supply chain network, where such movement occurs over time as defined by training timesteps. Because transactions between agents occur across successive timesteps, the elapsed time associated with movement between entities constitutes travel time data under the broadest reasonable interpretation. When such time-based movement data is collected across multiple transactions, it represents one or more travel time distributions between entities in the supply chain network.);
training a network policy agent using the second dataset (Kumar, paragraph [0098] “At step 204, the method 200 may further include training a reinforcement learning simulation of the training supply chain graph using the training data via policy gradient reinforcement learning. For example, the reinforcement learning simulation may be trained via actor-critic reinforcement learning… training the reinforcement learning simulation may include computing a training action output based at least in part on a plurality of actor network weights of an actor network and computing a value of an actor network objective function based at least in part on the actor network weights and the training action output. Performing actor-critic reinforcement learning may further include computing an estimated actor network gradient at a critic network based at least in part on the value of the actor network objective function, critic network weights of the critic network, and the value of a critic network loss function of the critic network. Gradient descent may then be performed using the estimated actor network gradient.”).
However, Kumar does not teach but Kumar in view of Gutierrez teaches the following limitations:
generating synthetic network disruption data including sampling from one or more second travel time distributions corresponding respectively to one or more simulated network disruptions (Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset… Some events are rare because they are uncommon…it may be beneficial to simulate different types of fraud (both factual synthetic datasets and counterfactual datasets) and add the fraud-related synthetic datasets to an existing dataset.” [0121] “applications of the synthetic data generated by the ABM may include the generation of data with a distribution that changes over time… using an ABM, the variations in spending habits may be obtained by simulating probability distributions while enabling arbitrary complexity to be included in the definition of agents and/or behaviors” – Gutierrez teaches generating synthetic data using ABMs where agent behaviors follow probabilistic functions over time. Under BRI, such time-based probabilistic functions constitute temporal distributions, which correspond travel time distributions in a modeled network. Gutierrez also teaches that ABMs simulate rare or previously unexperienced events. Under BRI, these rare events are simulated disruptions because they represent abnormal, non-baseline network behavior.);
generating a second dataset having the synthetic network disruption data (Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset…it may be beneficial to simulate… (both factual synthetic datasets and counterfactual datasets) and add the… synthetic datasets to an existing dataset.” – Gutierrez discloses generating synthetic data that includes simulated rare or unexperienced events. Under BRI, such simulated rare events constitute network disruptions. Therefore, Gutierrez teaches generating a dataset that contains synthetic network disruption data, satisfying the claim.); and
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having a combination of Kumar and Gutierrez before them, to incorporate the generation of synthetic training data representing rare or unobserved network conditions, as taught by Gutierrez, into the reinforcement-learning-based supply chain simulation of Kumar. One would have been motivated to make such a combination in order to improve the robustness and completeness of the training data used to train a network policy agent, particularly in scenarios where certain supply chain disruptions are infrequent, underrepresented, or absent from historical data. This would allow the reinforcement learning simulation to be trained on a broader range of network conditions, including simulated disruption scenarios, thereby enabling the learned policy agent to more reliably predict and respond to supply chain delays and disruptions across different operating conditions.
Regarding claim 2, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
generating the first travel time distributions from the first dataset (Kumar, paragraph [0022] “The training data 20 may be organized into a plurality of training timesteps 58 that include time series data for a plurality of training-phase agents 22.” [0024] “the training data 20 may include a plurality of training forecast states 24.” [0025] “the training forecast states 24 included in the training data 20 may be collected from a plurality of real-world agents included in a real-world supply chain.” [0026] “at least in part by performing regression on empirical training forecast states 24 collected from real-world agents included in a real-world supply chain.” – Kumar discloses generating statistical distributions from a dataset that includes time-series data organized across training timesteps and forecast states associated with agents in a supply-chain graph. These forecast states are derived from real-world-supply-chain interactions and are generated using regression on empirical movement-related data. Under BRI, a travel-time distribution encompasses any statistical distribution derived from time-dependent interactions between entities in a network. Because Kumar generates regression-based distributions from time-indexed supply-chain data, this corresponds to generating first travel-time distributions from the first dataset.).
Regarding claim 3, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
generating the second travel time distributions from elements of the first dataset having conditions meeting one or more disruption criteria (Gutierrez, paragraph [0049] “as some datasets may be partitioned in time, geographical region, and other criteria, the new dataset may be created from a first set of rows from a first table and a second set of rows from a second table.” [0120] “ applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset.” [0121] “applications of the synthetic data generated by the ABM may include the generation of data with a distribution that changes over time.” – Gutierrez teaches selecting subsets of dataset elements based on time-based partitioning, which under BRI corresponds to selecting elements relevant to travel-time behavior. Gutierrez further teaches generating data representing rare events, which constitute disruption conditions under BRI. Gutierrez also discloses generating data with distributions that change over time, which corresponds to second travel-time distributions from elements of the first dataset that meet disruption-related criteria.).
Regarding claim 4, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
maintaining a mapping between different types of network disruptions and corresponding second travel time distributions (Gutierrez, paragraph [0061] “Further the generation of the synthetic datasets may be repeatedly tuned to customize the synthetic datasets to be statistically closer to or statistically farther from actual data.“ [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset.” [0121] “applications of the synthetic data generated by the ABM may include the generation of data with a distribution that changes over time.” [0146] “In step 1304, statistical parameters and correlation parameters may be determined for the fields in the scrubbed dataset… a machine learning model evaluates the scrubbed true-source data to learn its patterns and distributions both within a field and by evaluating dependencies across fields (for example, income may be influenced by age). Using the ability to determine correlations between fields of steps 303 and/or 403, dependencies between various fields may be determined. Based on those dependencies, the relationships between the fields may be mapped within the dataset to improve the accuracy of the output.”);
receiving user input selecting one or more of the different types of network disruptions (Gutierrez, paragraph [0027] “FIG. 7 depicts a user interface for selecting and/or modifying parameters of a probabilistic graphical model;” [0073] “FIG. 7 depicts a possible representation of a user interface, permitting modification of a generative model. The user interface 701 may comprise one or more regions 702 permitting a user to select and/or modify statistical parameters of the generative model… one or more regions 703 permitting a user to select and/or modify correlation parameters… Another region 719 may allow a user to identify how many generated datasets are to be generated and sent to the user.”);
implementing, using the mapping, a modification to a first travel time distribution to generate a second travel time distribution (Gutierrez, paragraph [0065] “Various generative models may encode the distribution of a dataset by capturing both the individual variations of a variable in the dataset as well as the covariances of pairs of variables.” [0073] “The user interface 701 may comprise one or more regions 702 permitting a user to select and/or modify statistical parameters of the generative model and one or more regions 703 permitting a user to select and/or modify correlation parameters of the generative model.” [0146] “Based on those dependencies, the relationships between the fields may be mapped within the dataset to improve the accuracy of the output.” [0147] “In step 1306, based on the generative model created in step 1305, a synthetic dataset is generated.” – discloses a generative model that encodes statistical distributions of data and maintains mapped relationships between fields within the dataset. Gutierrez further discloses modifying statistical and correlation parameters of the generative model based on user input. Implementing these modifications alters the distributions encoded by the generative model, and synthetic data is subsequently generated based on the modified model. Under BRI, modifying a first distribution using mapped relationships to generate a new distribution corresponds to implementing, using the mapping, a modification to a first travel time distribution to generate a second travel time distribution.);
sampling from the second travel time distribution that corresponds to the user-selected type of network distribution to generate the synthetic network disruption data (Gutierrez, paragraph [0027] “FIG. 7 depicts a user interface for selecting and/or modifying parameters of a probabilistic graphical model;” [0125] “generate, using the simulation specification, a simulation state of an agent-based model, the generate comprising instantiating, via sampling using a random number generator to sample probability distribution definitions… simulate…via sampling using the random number generator to sample a probability distribution definition of the one or more behaviors associated with the agent instance… generate, based on the stored simulation step, a synthetic dataset; and output the synthetic dataset.” – teaches that a user may select or modify parameters of a probabilistic graphical model, and that the system samples from the resulting probability distributions using a random number generator to generate synthetic data. Under BRI, sampling from a distribution defined by user-selected parameters corresponds to sampling from a second travel-time distribution associated with a user-selected network disruption type to generate synthetic network disruption data.).
Regarding claim 8, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
obtaining an adversarial agent that implements a machine learned policy that grants rewards exceeding a threshold reward level for more severe disruptions to the supply chain network (Kumar, paragraph [0087] “During training of the reinforcement learning simulations 30 in the simulations performed by the inventors, the training-phase agents 22 were simulated for 15000 episodes of 40 epochs each.” [0094] “setting w.sub.i=s_.sub.i during both training and inferencing allowed the rewards for both of the training-phase agents 22 to reach higher, more stable values” [0095] “At step 202, the method 200 may include receiving training data including, for each of a plurality of training timesteps, a plurality of training forecast states associated with a respective plurality of training-phase agents included in a training supply chain graph. The training-phase agents may each have respective reward functions.” [0096] “The plurality of training-phase agents included in the training supply chain graph may have a plurality of respective reward functions that are neither fully correlated nor fully anticorrelated with each other. Thus, the interaction of the training-phase agents may be a mixed-motive game. In some examples, the reward for each of the training-phase agents may be a profit earned by that training-phase agent.” [0097] “step 202 may include, at step 202A, generating at least a portion of the plurality of training forecast states at least in part by sampling the portion of the plurality of training forecast states from a simulated training forecast state distribution.” [0098] “training a reinforcement learning simulation of the training supply chain graph using the training data via policy gradient reinforcement learning. For example, the reinforcement learning simulation may be trained via actor-critic reinforcement learning.” – Kumar teaches training agents within a simulated supply-chain graph using reinforcement learning techniques such as policy-gradient and actor-critic learning, resulting in machine-learned policies associated with respective reward functions. Kumar further discloses conditions under which agent rewards reach higher and more stable values, which under BRI corresponds to rewards exceeding a threshold reward level. Kumar also describes agents interacting in a mixed-motive game, which under BRI constitutes adversarial behavior, and training these agents across varying simulated supply-chain forecast states, which correspond to differing severities of supply chain conditions. Accordingly, Kumar teaches obtaining an adversarial agent that implements a machine-learning policy that grants rewards exceeding a threshold reward level for more severe supply-chain conditions.),
wherein sampling from the one or more second travel time distributions comprises causing the adversarial agent to sample from the second travel time distributions (Kumar, paragraph [0097] “generating at least a portion of the plurality of training forecast states at least in part by sampling the portion of the plurality of training forecast states from a simulated training forecast state distribution.” [0096] “the training supply chain graph may be simulated as a Markov decision process. The plurality of training-phase agents included in the training supply chain graph may have a plurality of respective reward functions that are neither fully correlated nor fully anticorrelated with each other. Thus, the interaction of the training-phase agents may be a mixed-motive game.” [0101] “generating a respective runtime action output associated with a corresponding runtime forecast state of the runtime agent for a current runtime step.” – Kumar teaches that forecast states used by reinforcement-learning agents are generated by sampling from simulated forecast-state distributions, and that these sampled states are provided to the agents during both training and runtime to determine their actions. Under BRI, causing an agent to operate using forecast states produced by sampling from a distribution constitutes causing the agent to sample from that distribution. Because the second travel time distributions correspond, under BRI, to the simulated forecast-state distributions in Kumar, the reference teaches causing the adversarial agent to sample from the second travel-time distributions.).
Regarding claim 9, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
wherein generating the synthetic network disruption data comprises generating a plurality of correlated modified travel time distributions (Gutierrez, paragraph [0146] “Using the ability to determine correlations between fields of steps 303 and/or 403, dependencies between various fields may be determined. Based on those dependencies, the relationships between the fields may be mapped within the dataset to improve the accuracy of the output. The output of this process is a generative model (step 1305) that is generates realistic generated datasets similar to the scrubbed true-source dataset and may be used to generate synthetic data that follows the distributions of that scrubbed data.” – Gutierrez teaches determining correlations between multiple fields of a dataset and mapping the relationships between those fields within a generative model. Gutierrez further discloses generating synthetic data using a generative model such that the generated data follows the learned distributions and correlations. Under BRI, generating synthetic data based on correlated distribution parameters corresponds to generating a plurality of correlated modified travel-time distributions, where the distributions are modified relative to the original data and correlated according to the learned dependencies.).
Regarding claim 10, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
wherein generating the synthetic network disruption data further comprises generating a plurality of correlated (i) modified supply, (ii) modified demand, and (iii) modified traffic costs for the supply chain network (Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset.” [0146] “ statistical parameters and correlation parameters may be determined for the fields in the scrubbed dataset…a machine learning model evaluates the scrubbed true-source data to learn its patterns and distributions both within a field and by evaluating dependencies across fields (for example, income may be influenced by age). Using the ability to determine correlations between fields of steps 303 and/or 403, dependencies between various fields may be determined. Based on those dependencies, the relationships between the fields may be mapped within the dataset to improve the accuracy of the output…and may be used to generate synthetic data that follows the distributions of that scrubbed data.” – Gutierrez teaches simulating rare events by generating synthetic data that follows learned statistical distributions and correlation parameters derived from a scrubbed dataset. Under BRI, the “fields” and “dependencies” in Gutierrez encompass quantitative supply-chain variables such as supply, demand, and traffic costs. By determining correlations among these fields and generating synthetic data that follows the correlated distributions, Gutierrez generates modified values for each field that preserve their inter-field dependencies. Accordingly, Gutierrez teaches generating a plurality of correlated modified supply, demand, and traffic cost values corresponding to simulating a network-disruption scenario.).
Regarding claim 11, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
executing the trained network policy agent for the supply chain network to identify potential disruptions to the supply chain network (Kumar, paragraph [0098] “training a reinforcement learning simulation of the training supply chain graph using the training data via policy gradient reinforcement learning.” [0100] “an example method 300 for use at a computing system during an inferencing phase. The example method 300 utilizes a trained reinforcement learning simulation, which may be the reinforcement learning simulation trained according to the method…The runtime supply chain graph may be a graph representation of a real-world supply chain” [0101] “the method 300 may further include, at the trained reinforcement learning simulation, generating a respective runtime action output associated with a corresponding runtime forecast state of the runtime agent for a current runtime step.” Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events” – Kumar teaches training a reinforcement-learning policy agent for a supply chain network and executing the trained policy during an inferencing phase to generate runtime action outputs based on forecast states of the network. Kumar further discloses that the runtime supply chain graph may represent a real-world supply chain. Gutierrez teaches simulating rare events corresponding to abnormal network conditions. Under BRI, executing a trained network policy agent under simulated rare event conditions enables identification of potential disruptions, as the agent’s runtime actions and resulting network behavior reflect the effects of such abnormal conditions. Accordingly, the combination of Kumar and Gutierrez teaches executing a trained network policy agent for a supply chain network to identify potential disruptions to the supply chain network.); and
generating, based on execution of the trained machine learning network policy agent and identification of the potential disruptions to the supply chain network, recommendations to reorganize operations of the supply chain network (Kumar, paragraph [0080] “the runtime agent 122 may be displayed to the user as recommended actions. The one or more runtime action outputs 126 may include recommendations of one or more raw material purchase quantities for the runtime agent 122 to purchase from one or more respective upstream runtime agents 122. The one or more runtime action outputs 126 may further include a price per unit product for the runtime agent 122 to charge one or more downstream runtime agents 122.”)
Regarding claim 12, Kumar in view of Gutierrez teaches all the elements of claim 11, therefore is rejected for the same reasons as those presented for claim 11. Kumar in view of Gutierrez further teaches:
wherein the identified potential disruptions include a storm, a shipment delay, an object production delay, or a labor strike (Gutierrez, paragraph [0085] “a set of behaviors may be modified to account for possible economic events before adding in existing agents…the distribution is generally regarded as varying per simulation step following the distribution pattern identified for that first instance's attribute.” – Gutierrez teaches modifying agent behaviors to account for possible economic events prior to simulation and generating behavior values that vary across simulation steps. Under BRI, economic events may lead to disruptions in supply chain operations, including shipment delays, production delays, or labor-related interruptions. By modifying agent behaviors to account for such economic events and simulating their effects over time, Gutierrez teaches identifying potential disruptions that include shipment delays, object production delays, or labor strikes.).
Regarding claim 13, Kumar in view of Gutierrez teaches all the elements of claim 11, therefore is rejected for the same reasons as those presented for claim 11. Kumar in view of Gutierrez further teaches:
wherein the generated recommendations apply to more than one of the identified potential disruptions, wherein each of the identified potential disruptions is a different type of disruption to the supply chain network (Gutierrez, paragraph [0085] “a set of behaviors may be modified to account for possible economic events before adding in existing agents.” [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events” Kumar, paragraph [0080] “ the one or more runtime action outputs 126 generated at the trained reinforcement learning simulation 30 for the runtime agent 122 may be displayed to the user as recommended actions. The one or more runtime action outputs 126 may include recommendations of one or more raw material purchase quantities.” – because the policy agent in Kumar is trained using a plurality of simulation states and executed at runtime to generate action outputs, the trained policy inherently maps multiple operating conditions to corresponding actions. When the trained policy agent is applied to the different disruption scenarios disclosed by Gutierrez, including rare events and economic events, the resulting runtime action outputs correspond to recommendations applicable across more than one type of disruption. Accordingly, the generated recommendation are not limited to a single disruption type but apply to multiple different disruption conditions within the supply chain.).
Regarding claim 14, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
wherein the first dataset includes historic supply chain data associated with the supply chain network (Kumar, paragraph [0003] “ receive training data including, for each of a plurality of training timesteps, a plurality of training forecast states associated with a respective plurality of training-phase agents included in a training supply chain graph.” – teaches receiving training data associated with a training supply chain graph, where the data includes forecast states of supply chain agents across a plurality of training timesteps. Because the training data reflects prior observed or generated system states used during the training phase before inferencing, such data constitutes historical data under BRI.).
Regarding claim 15, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches/does not teach:
wherein the first dataset includes historic supply chain data associated with at least one other supply chain network, wherein the at least one other supply chain network is different than the supply chain network (Kumar, paragraph [0003] “receive training data including, for each of a plurality of training timesteps, a plurality of training forecast states associated with a respective plurality of training-phase agents included in a training supply chain graph…During an inferencing phase, the processor may be further configured to receive a plurality of runtime forecast states associated with a respective plurality of runtime agents included in a runtime supply chain graph.” – Kumar distinguishes between a training supply chain graph used during a training phase and a runtime supply chain graph used during the inferencing phase. The training supply chain graph used to generate training data prior to deployment of the trained policy, while runtime supply chain graph represents a supply chain to which the trained policy is applied. Under BRI, the training supply chain graph constitutes a different supply chain network than the runtime supply chain network, as the two graphs are separately defined and used for different purposes. Accordingly, the training data received during the training phase represents historic supply chain data associated with at least one other supply chain network that is different from the runtime supply chain network.).
Regarding claim 16, Kumar in view of Gutierrez teaches all the elements of claim 1, therefore is rejected for the same reasons as those presented for claim 1. Kumar in view of Gutierrez further teaches:
returning the trained network policy agent for runtime use in the supply chain network (Kumar, paragraph [0003] “train a reinforcement learning simulation of the training supply chain graph using the training data via policy gradient reinforcement learning…During an inferencing phase, the processor may be further configured to receive a plurality of runtime forecast states associated with a respective plurality of runtime agents included in a runtime supply chain graph…at the trained reinforcement learning simulation, based at least in part on the plurality of runtime forecast states, the processor may be further configured to generate a respective runtime action output” – Kumar teaches training a reinforcement-learning simulation during a training phase and subsequently executing the trained reinforcement-learning simulation during an inferencing phase to generate runtime action output for agents in a runtime supply chain graph. Under BRI, making the trained reinforcement-learning simulation available for execution during the inferencing phase constitutes returning the trained network-policy agent for runtime use in the supply chain network.).
Regarding claim 17, Kumar teaches the following limitations:
A system comprising: one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising (Kumar, paragraph [0108] “computing system 400 may be included in one or more personal computers” [0109] “Computing system 400 includes a logic processor 402 volatile memory 404, and a non-volatile storage device 406” [0110] “Logic processor 402 includes one or more physical devices configured to execute instructions.”):
receiving first dataset representing movement of objects through a plurality of entities in a supply chain network, wherein the first dataset includes (i) travel time data representing how long it took for each object to be transported between at least two entities in the supply chain network (Kumar, paragraph [0022] “The training data 20 may be organized into a plurality of training timesteps 58 that include time series data for a plurality of training-phase agents 22. The plurality of training-phase agents 22 may be included in a training supply chain graph 50 in which the plurality of training-phase agents 22 may be connected by training graph edges 56 that represent transactions performed between the training-phase agents 22…The directions of the training graph edges 56 may indicate directions in which materials and products move through the supply chain between training-phase agents 22.” – Kumar teaches receiving training data organized as time series data across training timesteps for agents with a supply chain graph. The training-phase agents correspond to entities in the supply chain network, and that training graph edges represent movement of materials and products between those entities. Because the movement occurs across time series timesteps, the training data includes temporal information representing how long it took for materials and products to move between at least two entities. Under the broadest reasonable interpretation, such time-based movement data constitutes travel time data representing transportation between entities.) and
(ii) one or more features representing respective conditions of the supply chain network when the objects were transported (Kumar, paragraph [0024] “the training data 20 may include a plurality of training forecast states 24. Each training forecast state of the plurality of training forecast states 24 may be associated with a respective training-phase agent 22…the plurality of training forecast states 24 may include a plurality of training upstream price forecasts 24A and a plurality of training downstream demand forecasts 24B. The plurality of training upstream price forecasts 24A may be predictions of respective prices charged by one or more other training-phase agents… The plurality of training downstream demand forecasts 24B may be predictions of respective quantities of a product…that are demanded by one or more other training-phase agents.” – teaches that the received training data includes training forecast states associated with agents in a supply chain network. These forecast states include upstream price forecasts and downstream demand forecasts, which are variables that describe the operational state of the supply chain. Under BRI, such price and demand variables constitute features representing respective conditions of the supply chain network. Accordingly, Kuman teaches features representing network conditions associated with the transportation of objects through the supply chain.);
obtaining data representing one or more first travel time distributions between at the at least two entities in the supply chain network (Kumar, paragraph [0022] “The training data 20 may be organized into a plurality of training timesteps 58… training graph edges 56 that represent transactions performed between the training-phase agents 22… the training supply chain graph 50 may be a directed graph in which the training graph edges 56 respective directions. The directions of the training graph edges 56 may indicate directions in which materials and products move through the supply chain between training-phase agents 22.” – Kumar teaches obtaining training data that represents movement of materials and products between entities (training-phased agents) in a supply chain network, where such movement occurs over time as defined by training timesteps. Because transactions between agents occur across successive timesteps, the elapsed time associated with movement between entities constitutes travel time data under the broadest reasonable interpretation. When such time-based movement data is collected across multiple transactions, it represents one or more travel time distributions between entities in the supply chain network.);
training a network policy agent using the second dataset (Kumar, paragraph [0098] “At step 204, the method 200 may further include training a reinforcement learning simulation of the training supply chain graph using the training data via policy gradient reinforcement learning. For example, the reinforcement learning simulation may be trained via actor-critic reinforcement learning… training the reinforcement learning simulation may include computing a training action output based at least in part on a plurality of actor network weights of an actor network and computing a value of an actor network objective function based at least in part on the actor network weights and the training action output. Performing actor-critic reinforcement learning may further include computing an estimated actor network gradient at a critic network based at least in part on the value of the actor network objective function, critic network weights of the critic network, and the value of a critic network loss function of the critic network. Gradient descent may then be performed using the estimated actor network gradient.”).
However, Kumar does not teach but Kumar in view of Gutierrez teaches the following limitations:
generating synthetic network disruption data including sampling from one or more second travel time distributions corresponding respectively to one or more simulated network disruptions (Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset… Some events are rare because they are uncommon…it may be beneficial to simulate different types of fraud (both factual synthetic datasets and counterfactual datasets) and add the fraud-related synthetic datasets to an existing dataset.” [0121] “applications of the synthetic data generated by the ABM may include the generation of data with a distribution that changes over time… using an ABM, the variations in spending habits may be obtained by simulating probability distributions while enabling arbitrary complexity to be included in the definition of agents and/or behaviors” – Gutierrez teaches generating synthetic data using ABMs where agent behaviors follow probabilistic functions over time. Under BRI, such time-based probabilistic functions constitute temporal distributions, which correspond travel time distributions in a modeled network. Gutierrez also teaches that ABMs simulate rare or previously unexperienced events. Under BRI, these rare events are simulated disruptions because they represent abnormal, non-baseline network behavior.);
generating a second dataset having the synthetic network disruption data (Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset…it may be beneficial to simulate… (both factual synthetic datasets and counterfactual datasets) and add the… synthetic datasets to an existing dataset.” – Gutierrez discloses generating synthetic data that includes simulated rare or unexperienced events. Under BRI, such simulated rare events constitute network disruptions. Therefore, Gutierrez teaches generating a dataset that contains synthetic network disruption data, satisfying the claim.); and
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having a combination of Kumar and Gutierrez before them, to incorporate the generation of synthetic training data representing rare or unobserved network conditions, as taught by Gutierrez, into the reinforcement-learning-based supply chain simulation of Kumar. One would have been motivated to make such a combination in order to improve the robustness and completeness of the training data used to train a network policy agent, particularly in scenarios where certain supply chain disruptions are infrequent, underrepresented, or absent from historical data. This would allow the reinforcement learning simulation to be trained on a broader range of network conditions, including simulated disruption scenarios, thereby enabling the learned policy agent to more reliably predict and respond to supply chain delays and disruptions across different operating conditions.
Regarding claim 18, Kumar in view of Gutierrez teaches all the elements of claim 17, therefore is rejected for the same reasons as those presented for claim 17. The claim recites similar limitations corresponding to claim 3 and is rejected for similar reasons as claim 3 using similar teachings and rationale.
Regarding claim 19, Kumar in view of Gutierrez teaches all the elements of claim 17, therefore is rejected for the same reasons as those presented for claim 17. The claim recites similar limitations corresponding to claim 4 and is rejected for similar reasons as claim 4 using similar teachings and rationale.
Regarding claim 20, Kumar teaches the following limitations:
One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising (Kumar, paragraph [0113] “Non-volatile storage device 406 may include physical devices that are removable and/or built-in. Non-volatile storage device 406 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., ROM, EPROM, EEPROM, FLASH memory, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.)… that non-volatile storage device 406 is configured to hold instructions even when power is cut to the non-volatile storage device 406.” [0108] “computing system 400 may be included in one or more personal computers,”):
receiving first dataset representing movement of objects through a plurality of entities in a supply chain network, wherein the first dataset includes (i) travel time data representing how long it took for each object to be transported between at least two entities in the supply chain network (Kumar, paragraph [0022] “The training data 20 may be organized into a plurality of training timesteps 58 that include time series data for a plurality of training-phase agents 22. The plurality of training-phase agents 22 may be included in a training supply chain graph 50 in which the plurality of training-phase agents 22 may be connected by training graph edges 56 that represent transactions performed between the training-phase agents 22…The directions of the training graph edges 56 may indicate directions in which materials and products move through the supply chain between training-phase agents 22.” – Kumar teaches receiving training data organized as time series data across training timesteps for agents with a supply chain graph. The training-phase agents correspond to entities in the supply chain network, and that training graph edges represent movement of materials and products between those entities. Because the movement occurs across time series timesteps, the training data includes temporal information representing how long it took for materials and products to move between at least two entities. Under the broadest reasonable interpretation, such time-based movement data constitutes travel time data representing transportation between entities.) and
(ii) one or more features representing respective conditions of the supply chain network when the objects were transported (Kumar, paragraph [0024] “the training data 20 may include a plurality of training forecast states 24. Each training forecast state of the plurality of training forecast states 24 may be associated with a respective training-phase agent 22…the plurality of training forecast states 24 may include a plurality of training upstream price forecasts 24A and a plurality of training downstream demand forecasts 24B. The plurality of training upstream price forecasts 24A may be predictions of respective prices charged by one or more other training-phase agents… The plurality of training downstream demand forecasts 24B may be predictions of respective quantities of a product…that are demanded by one or more other training-phase agents.” – teaches that the received training data includes training forecast states associated with agents in a supply chain network. These forecast states include upstream price forecasts and downstream demand forecasts, which are variables that describe the operational state of the supply chain. Under BRI, such price and demand variables constitute features representing respective conditions of the supply chain network. Accordingly, Kuman teaches features representing network conditions associated with the transportation of objects through the supply chain.);
obtaining data representing one or more first travel time distributions between at the at least two entities in the supply chain network (Kumar, paragraph [0022] “The training data 20 may be organized into a plurality of training timesteps 58… training graph edges 56 that represent transactions performed between the training-phase agents 22… the training supply chain graph 50 may be a directed graph in which the training graph edges 56 respective directions. The directions of the training graph edges 56 may indicate directions in which materials and products move through the supply chain between training-phase agents 22.” – Kumar teaches obtaining training data that represents movement of materials and products between entities (training-phased agents) in a supply chain network, where such movement occurs over time as defined by training timesteps. Because transactions between agents occur across successive timesteps, the elapsed time associated with movement between entities constitutes travel time data under the broadest reasonable interpretation. When such time-based movement data is collected across multiple transactions, it represents one or more travel time distributions between entities in the supply chain network.);
training a network policy agent using the second dataset (Kumar, paragraph [0098] “At step 204, the method 200 may further include training a reinforcement learning simulation of the training supply chain graph using the training data via policy gradient reinforcement learning. For example, the reinforcement learning simulation may be trained via actor-critic reinforcement learning… training the reinforcement learning simulation may include computing a training action output based at least in part on a plurality of actor network weights of an actor network and computing a value of an actor network objective function based at least in part on the actor network weights and the training action output. Performing actor-critic reinforcement learning may further include computing an estimated actor network gradient at a critic network based at least in part on the value of the actor network objective function, critic network weights of the critic network, and the value of a critic network loss function of the critic network. Gradient descent may then be performed using the estimated actor network gradient.”).
However, Kumar does not teach but Kumar in view of Gutierrez teaches the following limitations:
generating synthetic network disruption data including sampling from one or more second travel time distributions corresponding respectively to one or more simulated network disruptions (Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset… Some events are rare because they are uncommon…it may be beneficial to simulate different types of fraud (both factual synthetic datasets and counterfactual datasets) and add the fraud-related synthetic datasets to an existing dataset.” [0121] “applications of the synthetic data generated by the ABM may include the generation of data with a distribution that changes over time… using an ABM, the variations in spending habits may be obtained by simulating probability distributions while enabling arbitrary complexity to be included in the definition of agents and/or behaviors” – Gutierrez teaches generating synthetic data using ABMs where agent behaviors follow probabilistic functions over time. Under BRI, such time-based probabilistic functions constitute temporal distributions, which correspond travel time distributions in a modeled network. Gutierrez also teaches that ABMs simulate rare or previously unexperienced events. Under BRI, these rare events are simulated disruptions because they represent abnormal, non-baseline network behavior.);
generating a second dataset having the synthetic network disruption data (Gutierrez, paragraph [0120] “applications of the synthetic data generated by the ABM may include the simulation of rare events to augment an existing dataset…it may be beneficial to simulate… (both factual synthetic datasets and counterfactual datasets) and add the… synthetic datasets to an existing dataset.” – Gutierrez discloses generating synthetic data that includes simulated rare or unexperienced events. Under BRI, such simulated rare events constitute network disruptions. Therefore, Gutierrez teaches generating a dataset that contains synthetic network disruption data, satisfying the claim.); and
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having a combination of Kumar and Gutierrez before them, to incorporate the generation of synthetic training data representing rare or unobserved network conditions, as taught by Gutierrez, into the reinforcement-learning-based supply chain simulation of Kumar. One would have been motivated to make such a combination in order to improve the robustness and completeness of the training data used to train a network policy agent, particularly in scenarios where certain supply chain disruptions are infrequent, underrepresented, or absent from historical data. This would allow the reinforcement learning simulation to be trained on a broader range of network conditions, including simulated disruption scenarios, thereby enabling the learned policy agent to more reliably predict and respond to supply chain delays and disruptions across different operating conditions.
Claims 5-7 are rejected under the 35 U.S.C. 103 as being unpatentable over Kumar at al., (Pub. No.: US 20230129665 A1 (Filed: 2021)). in view of Gutierrez et al., (Pub. No.: US 20220215141 A1 (Filed: 2021)) further in view of Dantuluri et al., (Pub. No.: US 20240119237 A1 (Filed: 2022)).
Regarding claim 5, Kumar in view of Gutierrez teaches all the elements of claim 4, therefore is rejected for the same reasons as those presented for claim 4. Kumar in view of Gutierrez does not teach but Kumar in view of Gutierrez further in view of Dantuluri teaches the following limitation:
wherein the user input selecting the one or more of the different types of network disruptions comprises natural language input indicating the one or more of the different types of network disruptions (Dantuluri, paragraph [0016] “receiving, from an end-point device, a user input, wherein the user input comprises natural language data; [0068] “implement natural language processing algorithms and techniques to extract content information from the user input. By implementing natural language processing algorithms and techniques, the system focuses on understanding the user input…the system may extract content and intent associated with the user input as attributes. For example, semantic information retrieval techniques may be used to extract whether the user has had a positive or a negative experience, identify specific issues experienced by the user”).
Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the user of natural-language user input and natural-language processing, as taught by Dantuluri, into the data-driven simulation and synthetic data generation techniques of Kumar and Gutierrez. One would have been motivated to make such a combination in order to allow a user to specify desired scenario conditions using natural-language input, while enabling the system to automatically extract intent and attributes from that input and apply them to the selection and modification of probability distributions used to generate synthetic data. This would allow the users to specify scenario conditions in a more intuitive manner, without requiring detailed technical knowledge of the underlying model parameters.
Regarding claim 6, Kumar in view of Gutierrez further in view of Dantuluri teaches all the elements of claim 5, therefore is rejected for the same reasons as those presented for claim 5. Kumar in view of Gutierrez further in view of Dantuluri further teaches the following limitation:
identifying a label corresponding to the one or more of the different types of network disruptions based on processing the user input with natural language processing techniques (Gutierrez, paragraph [0072] “In step 601, a system receives a labeled true-source dataset. In step 602, the system (e.g., processor 213 or other processors) creates a data model object of the dataset using the labels of the true-source dataset. The data model object may be stored as metadata 207. In step 603, the system generates a user interface based on the metadata of step 602.” Dantuluri, [0068] “ implement natural language processing algorithms and techniques to extract content information from the user input.”).
Regarding claim 7, Kumar in view of Gutierrez further in view of Dantuluri teaches all the elements of claim 5, therefore is rejected for the same reasons as those presented for claim 5. Kumar in view of Gutierrez further in view of Dantuluri further teaches the following limitation:
identifying a label corresponding to a nature of the one or more of the different types of network disruptions, a distribution of the one or more of the different types of network disruptions, and a severity of the one or more of the different types of network disruptions (Gutierrez, paragraph [0072] “In step 601, a system receives a labeled true-source dataset. In step 602, the system (e.g., processor 213 or other processors) creates a data model object of the dataset using the labels of the true-source dataset. The data model object may be stored as metadata 207. In step 603, the system generates a user interface based on the metadata of step 602.” [0073] “The user interface 701 may comprise one or more regions 702 permitting a user to select and/or modify statistical parameters of the generative model… The one or more regions 702 permitting selection/modification of statistical parameters may comprise one or more of a node (in the case of a PGM)/field selection/deselection (represented by region 704), a distribution modification option (represented by region 705), a mean modification option (represented by region 706), a mode modification option (represented by region 707), a maximum modification option (represented by region 708), a minimum modification option (represented by region 709), a standard deviation modification option (represented by region 710), a symmetry modification option (represented by region 711), a skewness modification option (represented by region 712), and/or a kurtosis modification option (represented by region 713). Other regions may be added as desired to permit modification of other statistical parameters. The one or more regions 703 permitting selection/modification of correlation parameters may comprise one or more of edge selection (in the case of a PGM) (represented by region 714) and/or the ability to select fields directly, e.g., first field 715 and second field 716, a type of correlation option (represented by region 717), and a degree of correlation option (represented by region 718).” – discloses identifying labels associated with a dataset and using those labels as metadata for controlling a generative model. Gutierrez further discloses defining and modifying statistical parameters that characterize probability distributions, including numerical parameters such as mean, maximum, minimum, and standard deviation. Under BRI, labels identifying the nature or type of model condition, the statistical parameters define the distribution of that condition, and numerical parameters reflecting magnitude or variability correspond to severity of the modeled condition.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daravanh Phakousonh whose telephone number is (571)272-6324. The examiner can normally be reached Mon - Thurs 7 AM - 5 PM, Every other Friday 7 AM - 4PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at 571-272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daravanh Phakousonh/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121