Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
MPEP § 2111.01, subsection I, provides that, under a broadest reasonable interpretation (BRI), words of the claim must be given their plain meaning, unless such meaning is inconsistent with the specification. The plain meaning of a term means the ordinary and customary meaning given to the term by those of ordinary skill in the art at the relevant time. The ordinary and customary meaning of a term may be evidenced by a variety of sources, including the words of the claims themselves, the specification, drawings, and prior art. However, the best source for determining the meaning of a claim term is the specification.
Applicant has not provided a clear definition for the term “network” recited in claims 1-8, 10-20 and 22 within the specification. Therefore, the Examiner will interpret the above by its plain meaning. See MPEP § 2111.01.
Examiner will interpret the meaning of claim term “task allocation strategy” found at paragraph [0095] within the disclosure as “In the implementation, a task allocation strategy may represent a strategy with which a task to be processed is allocated to and implemented by at least one IoT device. In other words, at least one IoT device may be determined through a task allocation strategy, and a task to be processed may be implemented by the at least one IoT device as instructed by the task allocation strategy. In one example, a task allocation strategy may also be referred to as one of a task allocation method, a task allocation mode, a task scheduling strategy, a task scheduling method, a task scheduling mode, a task arrangement strategy, a task arrangement method, a task arrangement mode, etc.”
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3-8 and 12-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 3 and 12-13 recite “one task allocation strategy”. It is unclear whether this recitation refers to the previously recited “at least one task allocation strategy”.
Claims 5 and 14 recite “a computation graph”. However, it is unclear whether this is the same or different from the previously recited “computation graph”.
Claims 5-6 and 14-15 recite “each resource sub-graph”. Claims 7 and 16 recite “the resource sub-graph”. There is insufficient antecedent basis for these elements. Examiner will assume that claims 5 and 14 depend from claims 4 and 13 respectively as such would give these elements proper antecedent basis in these claims.
Appropriate corrections are required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 10-13, 17-20 and 22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US 20210027197 A1 to Foerster et al. (“Foerster”).
Regarding claim 1, Foerster taught a method for network training based on an internet of things (IoT) device (“smart devices; consider paragraph 0062-0063), comprising:
determining a training dataset, wherein the training dataset comprises at least one task allocation strategy and an actual performance (“real usage”) of each task allocation strategy of the at least one task allocation strategy, and the actual performance is acquired by implementing a task to be processed based on the task allocation strategy; and training a first network (consider paragraphs 0067-0069 regarding “machine learning models” which may include “networks” as claimed) based on the training dataset, wherein the first network is used to predict performance of each task allocation strategy. (consider paragraphs 0065-0066, specifically “The system combines the computational graph data, the context information, and optionally the set of optimization constraints, to generate a model input for the system (230). Generating the model input includes transforming computational graph data and context information into an input of the type that the machine learning model is configured to receive…The system then processes the model input using a machine learning model to generate an output defining placement assignments of the operations of the computational graph to computing devices in the computational environment (240). That is, as described above, the machine learning model has been trained to generate placement assignments for the operations of the computational graph that satisfy one or more optimization goals.”) (consider further paragraph 0076, specifically “In order to provide optimal placement assignments, the system trains the machine learning model to predict placement assignments, based on given input.”) (consider further paragraphs 0078-0080, “To train the machine learning model, the system initializes the values of a set of parameters the model, e.g., to randomly assigned or pre-determined values. The system may determine current environment conditions (410) including the context information from the computational environment…The system also identifies computational graph data to assign to devices (415). In some implementations, the system generates weights for one or more optimization goals when the model expects weights as input. The system generates a model input from the current environment conditions and the computational graph data. The environmental conditions and the graph to be processed should ideally be from real usage—e.g., a model running in a camera application under a set of conditions on real devices. These conditions can be logged anonymously. Then the system can take the model/graph and conditions and run simulations to train the operation placement model.”) (consider further paragraph 0086, specifically “The system then updates the current values of the model parameters based on the reward using a reinforcement learning algorithm (440). That is, the system updates, using the reinforcement learning algorithm, the current values of the model parameters so that the model generates placements that result in an increased reward being generated.”) (consider further paragraph 0088, “The system repeats the training process 400 many times for different environment conditions and computational graphs to train the model to effectively account for numerous computational graph tasks being executed in a variety of computational environments.”) (consider further paragraph 0090, specifically “The trained model can then predict placement assignments for any computational graph task in any given computational environment.”)
Regarding claim 2, Foerster taught the method of claim 1, further comprising:
determining a computation graph corresponding to the task to be processed (“computational graph”; consider paragraph 0016) and a resource graph corresponding to the IoT device (“context information”; consider paragraph 0012), and generating the at least one task allocation strategy based on the computation graph and the resource graph. (consider paragraph 0005, specifically “In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of assigning computational graph operations to a plurality of computing devices in a computing environment according to placement assignments that are defined by a machine learning model.”) (consider further paragraph 0007, specifically “An example implementation includes obtaining data characterizing a computational graph comprising a plurality of nodes representing operations and directed edges representing data dependencies. Context information for a computational environment in which to perform the operations of the computational graph is received. The context information includes data representing a network connecting a plurality of computing devices in the computational environment. Model input is generated which includes at least the context information and the data characterizing the computational graph.”) (consider further paragraph 0016, “The computational graph or sub-graph may represent a particular task. After determining placement assignments for the operations of the computational graph or sub-graph representing the particular task, a policy may be created that defines placement assignments of the operation of the particular task from the determination of placement assignments for the operations. A second computational graph or sub-graph representing the same particular task may be received and the placement assignments of the operations of the second computational graph or sub-graph may be determined from the created policy.”) (consider further paragraph 0022-0023, “By dynamically determining optimal computational operation placement, the operation placement assignment system ensures the most efficient use of resources and placement of operations that will achieve defined optimization goals. That is, because the system can distribute different computational graphs or even the same computational graphs to different devices when faced with different computational environment characteristics, the system can effectively utilize the computational capacity of the various devices in the computational environment to effectively execute the computational graph. The operation placement system learns, by training, a mapping from computational capabilities and optimization goals to optimal placement. By using the system, computational graph operations can be assigned to computational devices in a way that increases execution speed for the operations. The system can also quickly and easily change the placement of operations based on changes in the computational environment to ensure optimal execution even with these changes in the environment.”)
Regarding claim 3, Foerster taught the method of claim 2, wherein generating the at least one task allocation strategy based on the computation graph and the resource graph comprises:
generating at least one resource sub-graph based on the computation graph and the resource graph, wherein each resource sub-graph of the at least one resource sub-graph comprises one task allocation strategy, the task allocation strategy is used to allocate at least one node of the resource graph to each node of the computation graph, a node in the resource sub-graph represents at least part of capability of the IoT device, and an edge connecting two adjacent nodes in the resource sub-graph represents a relationship between at least parts of the capability of the IoT device. (again, consider paragraph 0016, “The computational graph or sub-graph may represent a particular task. After determining placement assignments for the operations of the computational graph or sub-graph representing the particular task, a policy may be created that defines placement assignments of the operation of the particular task from the determination of placement assignments for the operations. A second computational graph or sub-graph representing the same particular task may be received and the placement assignments of the operations of the second computational graph or sub-graph may be determined from the created policy.”) (consider paragraph 0033, “Each machine learning task can be in the form of a computational graph that includes nodes connected by directed edges. Each node in a computational graph represents an operation. An incoming edge to a node represents a flow of an input into the node, i.e., an input to the operation represented by the node. An outgoing edge from a node represents a flow of an output of the operation represented by the node to be used as an input to an operation represented by another node. Thus, a directed edge connecting a first node in the graph to a second node in the graph indicates that an output generated by the operation represented by the first node is used as an input to the operation represented by the second node. For example, a computational graph can represent the operations performed by a machine learning model to determine an output for a received input. Thus, for example, the directed edges may represent dependencies of a neural network. Activations can flow in the direction of the edges. As another example, the computational graph can represent the operations performed to train a machine learning model on training data.”)
Regarding claim 10, Foerster taught the method of claim 1, further comprising:
updating the training dataset, by at least one of:
generating at least one resource sub-graph using at least one of heuristic, graph search, graph optimization, or sub-graph matching based on a computation graph and a resource graph, acquiring an actual performance corresponding to each resource sub-graph of the at least one resource sub-graph by implementing a task allocation strategy corresponding to the resource sub-graph, and adding, to the training dataset, the computation graph, each resource sub-graph, and the actual performance corresponding to the resource sub-graph;
generating the at least one resource sub-graph using at least one of heuristic, graph search, graph optimization, or sub-graph matching based on the computation graph and the resource graph, acquiring a predicted performance corresponding to each resource sub-graph of the at least one resource sub-graph through the first network, selecting a resource sub-graph with a best predicted performance from the at least one resource sub-graph, acquiring an actual performance of a task allocation strategy corresponding to the resource sub-graph with the best predicted performance by implementing the task allocation strategy, and adding, to the training dataset, the computation graph, the resource sub-graph with the best predicted performance, and the actual performance corresponding to the resource sub-graph with the best predicted performance; or
generating at least one resource sub-graph through random walk based on the computation graph and the resource graph, acquiring an actual performance of a task allocation strategy corresponding to each resource sub-graph of the at least one resource sub-graph by implementing the task allocation strategy, and adding, to the training dataset, the computation graph, the at least one resource sub-graph, and the actual performance corresponding to each resource sub-graph. (consider paragraph 0016, “The computational graph or sub-graph may represent a particular task. After determining placement assignments for the operations of the computational graph or sub-graph representing the particular task, a policy may be created that defines placement assignments of the operation of the particular task from the determination of placement assignments for the operations. A second computational graph or sub-graph representing the same particular task may be received and the placement assignments of the operations of the second computational graph or sub-graph may be determined from the created policy.”) (consider further paragraphs 0086-0088, specifically “The system then updates the current values of the model parameters based on the reward using a reinforcement learning algorithm (440). That is, the system updates, using the reinforcement learning algorithm, the current values of the model parameters so that the model generates placements that result in an increased reward being generated…In some implementations, in order to ensure that the space of possible assignments is sufficiently explored during the training of the model, the system incorporates an exploration policy into the training that ensures that assignments other than those that the model currently predicts would be the best assignment can be selected. For example, in certain iterations of the training process 400, the system may randomly select an assignment rather than selecting the assignment generated by the model. As another example, the system may include a term in the reward function that increases the reward when a new or rarely seen assignment is selected. The system repeats the training process 400 many times for different environment conditions and computational graphs to train the model to effectively account for numerous computational graph tasks being executed in a variety of computational environments.”)
Regarding claim 11, Foerster taught a method for task allocation based on an internet of things (IoT) device, comprising:
determining a computation graph corresponding to a task to be processed (“computational graph”; consider paragraph 0016) and a resource graph corresponding to the IoT device (“context information”; consider paragraph 0012); generating at least one task allocation strategy based on the computation graph and the resource graph; (consider paragraph 0005, specifically “In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of assigning computational graph operations to a plurality of computing devices in a computing environment according to placement assignments that are defined by a machine learning model.”) (consider further paragraph 0007, specifically “An example implementation includes obtaining data characterizing a computational graph comprising a plurality of nodes representing operations and directed edges representing data dependencies. Context information for a computational environment in which to perform the operations of the computational graph is received. The context information includes data representing a network connecting a plurality of computing devices in the computational environment. Model input is generated which includes at least the context information and the data characterizing the computational graph.”) (consider further paragraph 0016, “The computational graph or sub-graph may represent a particular task. After determining placement assignments for the operations of the computational graph or sub-graph representing the particular task, a policy may be created that defines placement assignments of the operation of the particular task from the determination of placement assignments for the operations. A second computational graph or sub-graph representing the same particular task may be received and the placement assignments of the operations of the second computational graph or sub-graph may be determined from the created policy.”) (consider further paragraph 0022-0023, “By dynamically determining optimal computational operation placement, the operation placement assignment system ensures the most efficient use of resources and placement of operations that will achieve defined optimization goals. That is, because the system can distribute different computational graphs or even the same computational graphs to different devices when faced with different computational environment characteristics, the system can effectively utilize the computational capacity of the various devices in the computational environment to effectively execute the computational graph. The operation placement system learns, by training, a mapping from computational capabilities and optimization goals to optimal placement. By using the system, computational graph operations can be assigned to computational devices in a way that increases execution speed for the operations. The system can also quickly and easily change the placement of operations based on changes in the computational environment to ensure optimal execution even with these changes in the environment.”)
acquiring a predicted performance of each task allocation strategy of the at least one task allocation strategy by inputting the at least one task allocation strategy into a first network, the first network being trained according to the method of claim 1; determining a task allocation strategy with a best predicted performance, and performing task allocation based on the determined task allocation strategy. (consider paragraphs 0065-0066, specifically “The system combines the computational graph data, the context information, and optionally the set of optimization constraints, to generate a model input for the system (230). Generating the model input includes transforming computational graph data and context information into an input of the type that the machine learning model is configured to receive…The system then processes the model input using a machine learning model to generate an output defining placement assignments of the operations of the computational graph to computing devices in the computational environment (240). That is, as described above, the machine learning model has been trained to generate placement assignments for the operations of the computational graph that satisfy one or more optimization goals.”) (consider further paragraph 0076, specifically “In order to provide optimal placement assignments, the system trains the machine learning model to predict placement assignments, based on given input.”) (consider further paragraphs 0078-0080, “To train the machine learning model, the system initializes the values of a set of parameters the model, e.g., to randomly assigned or pre-determined values. The system may determine current environment conditions (410) including the context information from the computational environment…The system also identifies computational graph data to assign to devices (415). In some implementations, the system generates weights for one or more optimization goals when the model expects weights as input. The system generates a model input from the current environment conditions and the computational graph data. The environmental conditions and the graph to be processed should ideally be from real usage—e.g., a model running in a camera application under a set of conditions on real devices. These conditions can be logged anonymously. Then the system can take the model/graph and conditions and run simulations to train the operation placement model.”) (consider further paragraphs 0086-0088, specifically “The system then updates the current values of the model parameters based on the reward using a reinforcement learning algorithm (440). That is, the system updates, using the reinforcement learning algorithm, the current values of the model parameters so that the model generates placements that result in an increased reward being generated…In some implementations, in order to ensure that the space of possible assignments is sufficiently explored during the training of the model, the system incorporates an exploration policy into the training that ensures that assignments other than those that the model currently predicts would be the best assignment can be selected. For example, in certain iterations of the training process 400, the system may randomly select an assignment rather than selecting the assignment generated by the model. As another example, the system may include a term in the reward function that increases the reward when a new or rarely seen assignment is selected. The system repeats the training process 400 many times for different environment conditions and computational graphs to train the model to effectively account for numerous computational graph tasks being executed in a variety of computational environments.”) (consider further paragraph 0090, specifically “The trained model can then predict placement assignments for any computational graph task in any given computational environment.”)
Claims 12 and 13 recite substantially the same limitations as recited in claims 3 and 4 respectively and are also rejected under 35 USC § 102(a)(1) as being anticipated by the same teachings of Foerster.
Regarding claim 17, Foerster taught the method of claim 11, further comprising: after performing the task allocation, acquiring an actual performance of the task allocation strategy when the task to be processed is implemented according to the task allocation strategy; and storing the task allocation strategy and the actual performance in a training dataset, the training dataset being used to update the first network. (again, consider further paragraphs 0078-0080, “To train the machine learning model, the system initializes the values of a set of parameters the model, e.g., to randomly assigned or pre-determined values. The system may determine current environment conditions (410) including the context information from the computational environment…The system also identifies computational graph data to assign to devices (415). In some implementations, the system generates weights for one or more optimization goals when the model expects weights as input. The system generates a model input from the current environment conditions and the computational graph data. The environmental conditions and the graph to be processed should ideally be from real usage—e.g., a model running in a camera application under a set of conditions on real devices. These conditions can be logged anonymously. Then the system can take the model/graph and conditions and run simulations to train the operation placement model.”) (again, consider further paragraphs 0086-0088, specifically “The system then updates the current values of the model parameters based on the reward using a reinforcement learning algorithm (440). That is, the system updates, using the reinforcement learning algorithm, the current values of the model parameters so that the model generates placements that result in an increased reward being generated…In some implementations, in order to ensure that the space of possible assignments is sufficiently explored during the training of the model, the system incorporates an exploration policy into the training that ensures that assignments other than those that the model currently predicts would be the best assignment can be selected. For example, in certain iterations of the training process 400, the system may randomly select an assignment rather than selecting the assignment generated by the model. As another example, the system may include a term in the reward function that increases the reward when a new or rarely seen assignment is selected. The system repeats the training process 400 many times for different environment conditions and computational graphs to train the model to effectively account for numerous computational graph tasks being executed in a variety of computational environments.”) (again, consider further paragraph 0090, specifically “The trained model can then predict placement assignments for any computational graph task in any given computational environment.”)
Claims 18-20 recite an apparatus that contains substantially the same limitations as recited in claims 1-3 respectively and are also rejected under 35 USC § 102(a)(1) as being anticipated by the same teachings of Foerster.
Claim 22 recites an apparatus that contains substantially the same limitations as recited in claim 11 and are also rejected under 35 USC § 102(a)(1) as being anticipated by the same teachings of Foerster.
Allowable Subject Matter
Claims 4 and 13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to G. C. Neurauter, Jr. whose telephone number is (571)272-3918. The examiner can normally be reached Monday-Friday 9am-5pm Eastern Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tonia Dollinger, can be reached at 571-272-4170. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/G. C. Neurauter, Jr./Primary Examiner, Art Unit 2459
Prosecution Insights