DETAILED ACTION
Claims 1-17 are pending and have been examined.
--
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 3 and 10-17 are objected to because of the following informalities:
In claim 3, “said algorithm module uses information from fed back said data labeling and said memory simulation as a basis for said machine learning” should be “said algorithm module uses information from said data labeling and said memory simulation as a basis for said machine learning”
In claims 10-17, “An intelligent dispatching method through machine learning…” should be “The intelligent dispatching method through machine learning….” (claims 10-17 are dependent claims.)
In claim 11, “between said predict optimized runtime and said actual runtime” should be “between said predicted optimized runtimes and said actual runtime”
In claim 13, “information from fed back said data labeling and said memory simulation as a basis are used as a basis for said machine learning algorithms” should be “information from said data labeling and said memory simulation as a basis are used as a basis for said machine learning algorithms”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 6 recites the limitation “wherein said basic information comprises tool constraint information, flow information and lot schedule information.” There is insufficient antecedent basis for “said basic information” in the claim. For examination purposes examiner has interpreted “said basic information” to be “said basic data.”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
-
Claims 1-17 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more
Step 1:
Claims 1-8 recite a machine learning intelligent dispatching system that is directed to software per se, i.e. no structure is recited in the claim, therefore is ineligible subject matter. (In light of specification [0017] “… program code or data relevant to the computer-implemented methods of the present invention and load them into the memory.”) Therefore, claims 1-8 fail step 1 analysis.
Claims 9-17 recite an intelligent dispatching method. Therefore, claims 9-17 are directed to a process.
With respect to claim 1:
2A Prong 1: The claim recites a judicial exception.
an algorithm module, working out predicted runtimes and predicted switching times of specific said recipe groups when specific said lots are processed in specific said tools based on said history data and said basic data… (mental process – evaluation or judgement, generating runtimes and switching times)
a robot module, working out an optimized schedule result based on said history data, said basic data, said predicted runtimes and said predicted switching times of specific said recipe groups (mental process – evaluation or judgement, generating a schedule result)
2A Prong 2: The judicial exception is not integrated into a practical application.
a history information module, storing various history data relevant to tools, recipe groups and lots (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting: acquiring history data)
a basic information module, storing various basic data relevant to said tools, said recipe groups and said lots (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting: acquiring basic data)
through machine learning (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
a dispatching module, dispatching said lots according to said optimized schedule result to obtain an actual production result and feed said actual production result back to said robot module (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
a history information module, storing various history data relevant to tools, recipe groups and lots (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting: acquiring history data, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
a basic information module, storing various basic data relevant to said tools, said recipe groups and said lots (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering: and outputting: acquiring basic data, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
through machine learning (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
a dispatching module, dispatching said lots according to said optimized schedule result to obtain an actual production result and feed said actual production result back to said robot module (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 2:
2A Prong 1: The claim recites a judicial exception.
wherein said robot module compares said actual production result and said optimized schedule result and (mental process – evaluation or judgement, comparing the actual result and the optimized result)
2A Prong 2: The judicial exception is not integrated into a practical application.
feeds a comparison result back to said algorithm module as a basis for said machine learning (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
feeds a comparison result back to said algorithm module as a basis for said machine learning (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 3:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said robot module performs data labeling and memory simulation to said history data and said basic data based on said comparison result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
said algorithm module uses information from fed back said data labeling and said memory simulation as a basis for said machine learning (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said robot module performs data labeling and memory simulation to said history data and said basic data based on said comparison result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
said algorithm module uses information from fed back said data labeling and said memory simulation as a basis for said machine learning (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 4:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein algorithm models adopted in said machine learning comprises decision tree, random forest, artificial neural network or Bayesian network (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein algorithm models adopted in said machine learning comprises decision tree, random forest, artificial neural network or Bayesian network (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 5:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said history data comprises tool information, production capacity information and lot information (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting: Claim 1 recites “storing(acquiring) history data,” which is insignificant extra-solution activity.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said history data comprises tool information, production capacity information and lot information (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i)). Claim 1 recites “storing(acquiring) history data,” which is insignificant extra-solution activity.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 6:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said basic information comprises tool constraint information, flow information and lot schedule information (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting: Claim 1 recites “storing(acquiring) various basic data,” which is insignificant extra-solution activity.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said basic information comprises tool constraint information, flow information and lot schedule information (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i)). Claim 1 recites “storing(acquiring) various basic data,” which is insignificant extra-solution activity.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 7:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said dispatching module comprises active management system and real-time dispatching system, and said active management system executes actions of dispatching said lots and said real-time dispatching system feeds said actual production result back to said robot module in real time (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said dispatching module comprises active management system and real-time dispatching system, and said active management system executes actions of dispatching said lots and said real-time dispatching system feeds said actual production result back to said robot module in real time (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 8:
2A Prong 1: The claim recites a judicial exception.
wherein said optimized schedule result comprises predicted optimized runtimes of multiple said lots processed in multiple said tools using multiple said recipe groups (mental process – evaluation or judgement: claim 1 recites “working out an optimized schedule result” which is an abstract idea. The specifics (predicted optimized runtimes) do not change the scope of the claim.)
2A Prong 2: The judicial exception is not integrated into a practical application.
said actual production result comprises actual runtime of said lots dispatched according to said optimized schedule result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception: claim 1 recites “dispatching said lots… to obtain an actual production result” which is mere instructions to apply an exception.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
said actual production result comprises actual runtime of said lots dispatched according to said optimized schedule result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception: claim 1 recites “dispatching said lots… to obtain an actual production result” which is mere instructions to apply an exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 9:
2A Prong 1: The claim recites a judicial exception.
working out predicted runtimes and predicted switching times of specific said recipe groups when specific said lots are processed in specific said tools based on said history data and said basic data… (mental process – evaluation or judgement, generating runtimes and switching times)
working out an optimized schedule result based on said history data, said basic data, said predicted runtimes and said predicted switching times of specific said recipe groups (mental process – evaluation or judgement, generating an optimized result)
comparing said actual production result and said optimized schedule result and feeding back a comparison result as a basis for said machine learning algorithms (mental process – evaluation or judgement, comparing the actual result and the optimized result to provide feedback)
2A Prong 2: The judicial exception is not integrated into a practical application.
acquiring various history data and basic data relevant to tools, recipe groups and lots from database (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting: receiving history data)
through different machining learning algorithms (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
dispatching said lots according to said optimized schedule result to obtain an actual production result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
acquiring various history data and basic data relevant to tools, recipe groups and lots from database (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting: receiving history data, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
through different machining learning algorithms (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
dispatching said lots according to said optimized schedule result to obtain an actual production result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 10:
2A Prong 1: The claim recites a judicial exception.
wherein said optimized schedule result comprises predicted optimized runtimes of multiple said lots processed in multiple said tools using multiple said recipe groups (mental process – evaluation or judgement: claim 9 recites “working out an optimized schedule result” which is an abstract idea. The specifics (predicted optimized runtimes) do not change the scope of the claim.)
2A Prong 2: The judicial exception is not integrated into a practical application.
said actual production result comprises actual runtime of said lots dispatched according to said optimized schedule result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception: claim 9 recites “dispatching said lots… to obtain an actual production result” which is mere instructions to apply an exception.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
said actual production result comprises actual runtime of said lots dispatched according to said optimized schedule result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception: claim 9 recites “dispatching said lots… to obtain an actual production result” which is mere instructions to apply an exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 11:
2A Prong 1: The claim recites a judicial exception.
wherein comparing said actual production result and said optimized schedule result comprises working out a difference between said predict optimized runtime and said actual runtime, and (mental process – evaluation or judgement)
if said difference exceeds a set value, determining there is a disparity between said optimized schedule result and said actual production result, and (mental process – evaluation or judgement)
if said difference is smaller than said set value, determining there is no disparity between said optimized schedule result and said actual production result (mental process – evaluation or judgement)
With respect to claim 12:
2A Prong 1: The claim recites a judicial exception.
wherein when it is determined that there is a disparity between said optimized schedule result and said actual production result, feedback said comparison result as a basis for said machine learning algorithms (mental process – evaluation or judgement, determining a disparity to provide feedback)
With respect to claim 13:
2A Prong 2: The judicial exception is not integrated into a practical application.
further comprising performing data labeling and memory simulation to said history data and said basic data based on said comparison result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
information from fed back said data labeling and said memory simulation are used as a basis for said machine learning algorithms (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
further comprising performing data labeling and memory simulation to said history data and said basic data based on said comparison result (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
information from fed back said data labeling and said memory simulation are used as a basis for said machine learning algorithms (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 14:
2A Prong 1: The claim recites a judicial exception.
further comprising using or excluding specific said machine learning algorithms based on said comparison result (mental process – evaluation or judgement,--- using or excluding algorithms based on the result)
With respect to claim 15:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein said machine learning algorithms comprise decision tree, random forest, artificial neural network or Bayesian network(mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein said machine learning algorithms comprise decision tree, random forest, artificial neural network or Bayesian network(mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 16:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein acquiring said history data and said basic data comprises said machine learning algorithm decides whether to process said history data and said basic data with average, median or weighted treatment (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception, in light of specification [0022] “algorithm module may self-determine”)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein acquiring said history data and said basic data comprises said machine learning algorithm decides whether to process said history data and said basic data with average, median or weighted treatment (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception, in light of specification [0022] “algorithm module may self-determine”)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 17:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein comparing said actual production result and said optimized schedule result comprises auto-regulating algorithms and variation factors in information cross-validation based on said recipe groups or products (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein comparing said actual production result and said optimized schedule result comprises auto-regulating algorithms and variation factors in information cross-validation based on said recipe groups or products (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 4-7 rejected under 35 U.S.C. 103 as being unpatentable over Priore ("Dynamic scheduling of manufacturing systems using machine learning: An updated review" 2014) in view of Schelthoff (US 20230260056 A1, filed on 20230216)
In regard to claim 1, Priore teaches: A machine learning intelligent dispatching system, comprising: a history information module, storing various history data… a basic information module, storing various basic data relevant to… (Priore, p. 83, Abstract "a scheduling approach that uses machine learning can be used. Analyzing the previous performance of the system (training examples) [storing/acquiring history data and basic data] by means of this technique, knowledge is obtained that can be used to decide which is the most appropriate dispatching rule at each moment in time."; p. 83 "based on the artificial intelligence field, a set of earlier system simulations (training examples) [history data and basic data] is used to determine which rule is the best for each possible system state."; p. 84, 2. KNOWLEDGE-BASED SYSTEMS "An overview of a scheduling system that uses machine learning is shown in Figure 1... Depending on the manufacturing system’s performance, the knowledge may need to be refined by generating further training examples."; training examples are generated based on previous performance of the system [history data, basic data relevant to the history data])
PNG
media_image1.png
400
750
media_image1.png
Greyscale
an algorithm module, (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "The machine learning algorithm [an algorithm module] acquires the knowledge necessary to make future scheduling decisions from the training examples.")
a robot module, working out an optimized schedule result based on said history data, said basic data… (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "The real-time control system [a robot module] using the 'scheduling knowledge,' the manufacturing system’s state and performance, determines the best dispatching rule [an optimized schedule result] for job scheduling."; the best dispatching rule is based on output of machine learning algorithm, which is based on history data and basic data, therefore the best dispatching rule is based on said history data and said basic data)
a dispatching module, dispatching said lots according to said optimized schedule result to obtain an actual production result and feed said actual production result back to said robot module. (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "The real-time control system using the 'scheduling knowledge,' the manufacturing system’s state and performance, determines the best dispatching rule [according to said optimized schedule result] for job scheduling. [dispatching said lots] Depending on the manufacturing system’s performance [obtain an actual production result], the knowledge may need to be refined by generating further training examples."; manufacturing system is [a dispatching module]; see Fig. 1, 'System sate and performance' are provided back to the real-time control system [feed said actual production result back to said robot module])
Priore does not teach, but Schelthoff teaches: … storing various history data relevant to tools, recipe groups and lots; … storing various basic data relevant to said tools, said recipe groups and said lots; (Schelthoff, Abstract "The method further includes, for each production operation in the sorted list, (i) sampling feature values for a plurality of features by sampling from a database of collected feature values... "; [0016] "the machine learning system take as inputs different feature sets. [acquiring (storing) history/basic data]"; [0054] "In FIG. 1, the table shows the feature set of our approach.") (Schelthoff, [0064] "G. Availability of machines [tools] (a): the availability is defined by the number of available machines which are able to execute the operation. Preferably, we obtain the number of machines in each equipment state ('available', 'repair', 'maintenance', 'setup', and 'shutdown') as features...") (Schelthoff, [0072] "O. Amount of similar operations in the queue (ql_sim): Similar operations are of the same operation type (independent of its product) and can be therefore produced in batches, [recipe groups (batches with similar operations)] if the equipments are capable of processing batches. Hence, a lot could be preferred if a lot of similar operations is waiting for execution to create full batches.") (Schelthoff, [0056]-[0057] "A. Lot priority [lots] (P): Each lot is assigned a priority at fab entry… B. Work-in-progress (WIP): The WIP is defined as the number of lots currently in operation in a machine group and the number of lots currently waiting"; [0071]-[0074])
working out predicted runtimes and predicted switching times (Schelthoff, [0018] "the trained machine learning system [machine learning] or the plurality of trained machine learning systems are configured to additionally output the expected processing times. [predicted runtimes]"; [0100] "The training system 500 comprises a provider system 51, which provides input features from a training data base. Input features are fed to the machine learning system 52 to be trained, which determines expected waiting time [predicted switching times] from them.") of specific said recipe groups when specific said lots are processed in specific said tools based on said history data and said basic data through machine learning; (Schelthoff, [0016] "the machine learning system take as inputs different feature sets. [said history/basic data]"; [0054] "In FIG. 1, the table shows the feature set of our approach."; [0072] "O. Amount of similar operations in the queue (ql_sim): Similar operations are of the same operation type (independent of its product) and can be therefore produced in batches, [said recipe groups (batches with similar operations)] if the equipments [said tools] are capable of processing batches. Hence, a lot could be preferred if a lot of similar operations [said lots] is waiting for execution to create full batches.")
an optimized schedule result based on… said predicted runtimes and said predicted switching times of specific said recipe groups; and (Schelthoff, [0018] "the trained machine learning system or the plurality of trained machine learning systems are configured to additionally output the expected processing times. [said predicted runtimes]"; [0100] "Input features are fed to the machine learning system 52 to be trained, which determines expected waiting time [said predicted switching times] from them."; Priore teaches the real-time control system generates the best dispatching rule [an optimized schedule result] based on the output of the machine learning algorithm, and Schelthoff teaches the output of the machine learning model is predicted runtimes and switching times, therefore the best dispatching rule is based on said predicted runtimes and said predicted switching times of specific said recipe groups. See above limitation for said recipe groups.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Priore to incorporate the teachings of Schelthoff by including different feature sets as inputs. Doing so would allow feature set to be reduced significantly, while the prediction performance remains equal. (Schelthoff, [0016] the machine learning system take as inputs different feature sets. This means that the inputs of the respective machine learning system can be actively reduced to a set of necessary features."; [0022] "The advantage thereof is that the feature set can be reduced significantly, while the prediction performance remains equal.")
In regard to claim 4, Priore teaches: wherein algorithm models adopted in said machine learning comprises decision tree, random forest, artificial neural network or Bayesian network. (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "machine learning techniques, such as inductive learning or neural networks (NNs), are used"; p. 85, 3. REVIEW OF KNOWLEDGE-BASED APPROACHES "Depending on the type of machine learning algorithm used, these approaches can be divided into the following categories: inductive learning, NNs…")
In regard to claim 5, Priore does not teach, but Schelthoff teaches: wherein said history data comprises tool information, (Schelthoff, [0016] "the machine learning system take as inputs different feature sets. [said history data]"; [0054] "In FIG. 1, the table shows the feature set of our approach."; [0064] "G. Availability of machines [e.g. tool information] (a): the availability is defined by the number of available machines which are able to execute the operation. Preferably, we obtain the number of machines in each equipment state ('available', 'repair', 'maintenance', 'setup', and 'shutdown') as features...")
production capacity information and (Schelthoff, [0062]-[0063] "F. The utilization (upreH)... The utilization of the equipment's indicates the available capacity for the process execution [e.g. production capacity information]...")
lot information. (Schelthoff, [0056]-[0057] "A. Lot priority [e.g. lot information] (P): Each lot is assigned a priority at fab entry… B. Work-in-progress (WIP): The WIP is defined as the number of lots currently in operation in a machine group and the number of lots currently waiting"; [0071]-[0074])
The rationale for combining the teachings of Priore and Schelthoff is the same as set forth in the rejection of claim 1.
In regard to claim 6, Priore does not teach, but Schelthoff teaches: wherein said basic information comprises tool constraint information, (Schelthoff, [0016] "the machine learning system take as inputs different feature sets. [said basic information]"; [0054] "In FIG. 1, the table shows the feature set of our approach."; [0064] "G. Availability of machines [e.g. tool constraint information] (a): the availability is defined by the number of available machines which are able to execute the operation. Preferably, we obtain the number of machines in each equipment state ('available', 'repair', 'maintenance', 'setup', and 'shutdown') as features...")
flow information and (Schelthoff, [0066]-[0068]"I. Product mix in the fab (pmfab)… The complexity of a product can be measured by the amount of layers [e.g. flow information] necessary for its completion... J. Number of tool loops (l): This feature indicates whether an operation is executed for the first time, or is repeated as a rework step..."; [0078] "U. Number of total stages necessary (Sttotal) for completion: This feature shall indicate how complex the respective lot is..."; in light of psecification [0023] "Flow information may include... specific process steps to be performed for various recipe groups")
lot schedule information. (Schelthoff, [0056]-[0057] "A. Lot priority [e.g. lot schedult information] (P): Each lot is assigned a priority at fab entry… B. Work-in-progress (WIP): The WIP is defined as the number of lots currently in operation in a machine group and the number of lots currently waiting"; [0071]-[0074])
The rationale for combining the teachings of Priore and Schelthoff is the same as set forth in the rejection of claim 1.
In regard to claim 7, Priore teaches: wherein said dispatching module comprises active management system and real-time dispatching system, and (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "A real-time scheduling system that modifies dispatching rules dynamically should fulfill two contradictory characteristics... The real-time control system using the 'scheduling knowledge,' the manufacturing system’s state and performance, determines the best dispatching rule [real-time dispatching system] for job scheduling. [active management system]"; see Fig. 1, manufacturing system is [a dispatching module], [real-time dispatching system], and [active management system])
said active management system executes actions of dispatching said lots and said real-time dispatching system feeds said actual production result back to said robot module in real time. (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "... determines the best dispatching rule for job scheduling. [executes actions of dispatching said lots] Depending on the manufacturing system’s performance [said actual production result], the knowledge may need to be refined by generating further training examples."; p. 87 "The scheduling system chooses rules for the machines and the automated material handling systems. [executes actions on machines or AMHS]"; see Fig. 1, 'System sate and performance' are provided back to the real-time control system [feed said actual production result back to said robot module])
Claims 2-3 and 8 rejected under 35 U.S.C. 103 as being unpatentable over Priore and Schelthoff as applied to claim 1, and in further view of Park ("A Reinforcement Learning Approach to Robust Scheduling of Semiconductor Manufacturing Facilities" 20191231)
In regard to claim 2, Priore and Schelthoff do not teach, but Park teaches: wherein said robot module compares said actual production result and said optimized schedule result and (Park, p. 1422, III. Scheduling by RL [a robot module] "The proposed scheduling method based on the Q -network is presented in this section."; p. 1424, D. Training and Test Phases "L = ... f(yu, qu) (3) where f(yu [said actual production result], qu [said optimized schedule result]) is the loss function given by f(yu, qu) = ...yu-qu [said comparison result, a difference] ... (4)")
PNG
media_image2.png
283
774
media_image2.png
Greyscale
feeds a comparison result back to said algorithm module as a basis for said machine learning. (Park, p. 1424, D. Training and Test Phases "In line 22, the learning algorithm performs a gradient descent step with respect to θ on the Huber loss L [42], denoted as L…"; the loss L is provided back to the RL model [feeds a comparison result back])
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Priore and Schelthoff to incorporate the teachings of Park by including reinforcement learning (RL) and a centralized policy by sharing a neural network. Doing so would minimize the makespan for an MCP scheduling problem. (Park, p. , Abstract "To minimize the makespan for an MCP scheduling problem, we propose a setup change scheduling method using reinforcement learning (RL) in which each agent determines setup decisions in a decentralized manner and learns a centralized policy by sharing a neural network among the agents to deal with the changes in the number of machines.")
In regard to claim 3, Priore and Schelthoff do not teach, but Park teaches: wherein said robot module performs data labeling and (Park, p. 1422, III. Scheduling by RL [a robot module] "The proposed scheduling method based on the Q -network is presented in this section."; p. 1424, D. Training and Test Phases "Given the sampled transitions, a loss is calculated from a Q-value and target value [35] [data labeling] (lines 19–21)."; p. 1424, Algorithm 1 "19: Set qu=Q(su,au;θ) 20: Set yu = ru+γ1F(su+1)maxa′Q^(su+1,a′;θ^) 21: Calculate loss L from (3)–(4)"; yu is the "synthetic label" for the specific state-action input)
memory simulation (Park, p. 1422, III. Scheduling by RL "we adopt the replay buffer [e.g. memory simulation, sample is memory] that contains the set of transitions each of which consists of state, action, reward, and next state... The weights of the Q -network [e.g. memory simulation, weights is memory] are periodically replicated to the target Q -network, which improves the convergence stability of the Q -network [35]."; in light of [0024] "The memory simulation may include static memory like parametric model (i.e. model parameter is memory), non-parametric model (i.e. sample is memory)... or neural network model (i.e. weights in network connectivity is memory), etc.";) to said history data and said basic data based on said comparison result, and said algorithm module uses information from fed back said data labeling and said memory simulation as a basis for said machine learning. (Park, p. 1424, Algorithm 1 Scheduling with Q-Network "1:… replay buffer B [history/basic data] to size NB… Calculate loss L from (3)-(4) 22: Perform a gradient descent step on L [said comparison result] w.r.t. θ "; RL uses the yu [said data labeling] and stored samples in the replay buffer or weights θ [memory simulation] for learning)
The rationale for combining the teachings of Priore, Schelthoff and Park is the same as set forth in the rejection of claim 2.
In regard to claim 8, Priore teaches: of said lots dispatched according to said optimized schedule result. (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "The real-time control system using the 'scheduling knowledge,' the manufacturing system’s state and performance, determines the best dispatching rule [according to said optimized schedule result] for job scheduling. [dispatching said lots]")
Priore does not teach, but Schelthoff teaches: of multiple said lots processed in multiple said tools using multiple said recipe groups, and (Schelthoff, [0072] "O. Amount of similar operations in the queue (ql_sim): Similar operations are of the same operation type (independent of its product) and can be therefore produced in batches, [multiple recipe groups (batches with similar operations)] if the equipments are capable of processing batches. Hence, a lot could be preferred if a lot of similar operations is waiting for execution to create full batches.")
The rationale for combining the teachings of Priore and Schelthoff is the same as set forth in the rejection of claim 1.
Priore and Schelthoff do not teach, but Park teaches: wherein said optimized schedule result comprises predicted optimized runtimes… said actual production result comprises actual runtime (Park, p. 1421, II. Problem Definition "Oj,k can be processed on any machine among its alternatives, and the processing time of Oj,k is denoted as pj,k."; p. 1423, B. State, Action, and Reward "If the previous setup type is Oj′,k′ and ai is Oj,k, τ(si+1) is calculated as the sum of τ(si), pj,k, and σj′,k′,j,k... As a state transition takes place from si to si+1 , ri is observed... ri is defined as follows: ri = ... τ(si+1)… (1) [reward is based on τ(si+1), which is based on the processing time (said runtime)]"; p. 1424, Algorithm 1 "19: Set qu=Q(su,au;θ) 20: Set yu = ru+γ...max...Q^(su+1,a′;θ^) [yu is based on ru, which is based on the processing time (said runtime)]"; yu is actual accumulated rewards and processing time at time i+1 [actual runtime] , qu is predicted accumulated rewards and processing time at time i [predicted optimized runtimes])
The rationale for combining the teachings of Priore, Schelthoff and Park is the same as set forth in the rejection of claim 2.
Claims 9-10 and 13-15 rejected under 35 U.S.C. 103 as being unpatentable over Priore and Schelthoff and in further view of Park
In regard to claim 9, Priore teaches: An intelligent dispatching method through machine learning, comprising: acquiring various history data and basic data (Priore, p. 83, Abstract "a scheduling approach that uses machine learning can be used. Analyzing the previous performance of the system (training examples) [storing/acquiring history data and basic data] by means of this technique, knowledge is obtained that can be used to decide which is the most appropriate dispatching rule at each moment in time."; p. 83 "based on the artificial intelligence field, a set of earlier system simulations (training examples) [history data and basic data] is used to determine which rule is the best for each possible system state."; p. 84, 2. KNOWLEDGE-BASED SYSTEMS "An overview of a scheduling system that uses machine learning is shown in Figure 1... Depending on the manufacturing system’s performance, the knowledge may need to be refined by generating further training examples."; training examples are generated based on previous performance of the system [history data, basic data relevant to the history data])
… working out an optimized schedule result based on said history data, said basic data… (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "The real-time control system using the 'scheduling knowledge,' the manufacturing system’s state and performance, determines the best dispatching rule [an optimized schedule result] for job scheduling."; the best dispatching rule is based on output of machine learning algorithm, which is based on history data and basic data, therefore the best dispatching rule is based on said history data and said basic data)
… dispatching said lots according to said optimized schedule result to obtain an actual production result; and (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "The real-time control system using the 'scheduling knowledge,' the manufacturing system’s state and performance, determines the best dispatching rule [according to said optimized schedule result] for job scheduling. [dispatching said lots] Depending on the manufacturing system’s performance [obtain an actual production result], the knowledge may need to be refined by generating further training examples.")
Priore does not teach, but Schelthoff teaches: relevant to tools, (Schelthoff, [0064] "G. Availability of machines [tools] (a): the availability is defined by the number of available machines which are able to execute the operation. Preferably, we obtain the number of machines in each equipment state ('available', 'repair', 'maintenance', 'setup', and 'shutdown') as features...") recipe groups and (Schelthoff, [0072] "O. Amount of similar operations in the queue (ql_sim): Similar operations are of the same operation type (independent of its product) and can be therefore produced in batches, [recipe groups (batches with similar operations)] if the equipments are capable of processing batches. Hence, a lot could be preferred if a lot of similar operations is waiting for execution to create full batches.") lots (Schelthoff, [0056]-[0057] "A. Lot priority [lots] (P): Each lot is assigned a priority at fab entry… B. Work-in-progress (WIP): The WIP is defined as the number of lots currently in operation in a machine group and the number of lots currently waiting"; [0071]-[0074]) from database; (Schelthoff, Abstract "The method further includes, for each production operation in the sorted list, (i) sampling feature values for a plurality of features by sampling from a database of collected feature values... [database]")
working out predicted runtimes and predicted switching times (Schelthoff, [0018] "the trained machine learning system or the plurality of trained machine learning systems are configured to additionally output the expected processing times. [predicted runtimes]"; [0100] "The training system 500 comprises a provider system 51, which provides input features from a training data base. Input features are fed to the machine learning system 52 to be trained, which determines expected waiting time [predicted switching times] from them.") of specific said recipe groups when specific said lots are processed in specific said tools based on said history data and said basic data through different machining learning algorithms; (Schelthoff, Abstract "The method further includes, for each production operation in the sorted list, (i) sampling feature values for a plurality of features by sampling from a database of collected feature values... "; [0016] "the machine learning system take as inputs different feature sets. [said history/basic data]"; [0054] "In FIG. 1, the table shows the feature set of our approach."; [0072] "O. Amount of similar operations in the queue (ql_sim): Similar operations are of the same operation type (independent of its product) and can be therefore produced in batches, [said recipe groups (batches with similar operations)] if the equipments [said tools] are capable of processing batches. Hence, a lot could be preferred if a lot of similar operations [said lots] is waiting for execution to create full batches."; [0005] There are approaches on waiting time predictions by forecasting models, wherein the forecasting models can be neural networks or data mining models [different machining learning algorithms] to forecast cycle times.")
working out an optimized schedule result based on… said predicted runtimes and said predicted switching times of specific said recipe groups; (Schelthoff, [0018] "the trained machine learning system or the plurality of trained machine learning systems are configured to additionally output the expected processing times. [said predicted runtimes]"; [0100] "Input features are fed to the machine learning system 52 to be trained, which determines expected waiting time [said predicted switching times] from them."; Priore teaches the real-time control system generates the best dispatching rule [an optimized schedule result] based on the output of the machine learning algorithm, and Schelthoff teaches the output of the machine learning model is predicted runtimes and switching times, therefore the best dispatching rule is based on said predicted runtimes and said predicted switching times of specific said recipe groups.)
The rationale for combining the teachings of Priore and Schelthoff is the same as set forth in the rejection of claim 1.
Priore and Schelthoff do not teach, but Park teaches: comparing said actual production result and said optimized schedule result and (Park, p. 1424, D. Training and Test Phases "L = ... f(yu, qu) (3) where f(yu [said actual production result], qu [said optimized schedule result]) is the loss function given by f(yu, qu) = ...yu-qu [said comparison result] ... (4)")
feeding back a comparison result as a basis for said machine learning algorithms. (Park, p. 1424, D. Training and Test Phases "In line 22, the learning algorithm performs a gradient descent step with respect to θ on the Huber loss L [42], denoted as L…"; the loss L is provided back to the RL model [feeds a comparison result back])
The rationale for combining the teachings of Priore, Schelthoff and Park is the same as set forth in the rejection of claim 2.
In regard to claim 10, Priore teaches: of said lots dispatched according to said optimized schedule result. (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "The real-time control system using the 'scheduling knowledge,' the manufacturing system’s state and performance, determines the best dispatching rule [according to said optimized schedule result] for job scheduling. [dispatching said lots]")
Priore does not teach, but Schelthoff teaches: … of multiple said lots processed in multiple said tools using multiple said recipe groups, and… (Schelthoff, [0072] "O. Amount of similar operations in the queue (ql_sim): Similar operations are of the same operation type (independent of its product) and can be therefore produced in batches, [multiple recipe groups (batches with similar operations)] if the equipments are capable of processing batches. Hence, a lot could be preferred if a lot of similar operations is waiting for execution to create full batches.")
The rationale for combining the teachings of Priore and Schelthoff is the same as set forth in the rejection of claim 1.
Priore and Schelthoff do not teach, but Park teaches: wherein said optimized schedule result comprises predicted optimized runtimes… said actual production result comprises actual runtime… (Park, p. 1421, II. Problem Definition "Oj,k can be processed on any machine among its alternatives, and the processing time of Oj,k is denoted as pj,k."; p. 1423, B. State, Action, and Reward "If the previous setup type is Oj′,k′ and ai is Oj,k, τ(si+1) is calculated as the sum of τ(si), pj,k, and σj′,k′,j,k... As a state transition takes place from si to si+1 , ri is observed... ri is defined as follows: ri = ... τ(si+1)... (1) [reward is based on τ(si+1), which is based on the processing time (said runtime)]"; p. 1424, Algorithm 1 "19: Set qu=Q(su,au;θ) 20: Set yu = ru+γ...max...Q^(su+1,a′;θ^) [yu is based on ru, which is based on the processing time (said runtime)]"; yu is actual accumulated rewards and processing time at time i+1 [actual runtime] , qu is predicted accumulated rewards and processing time at time i [predicted optimized runtimes])
The rationale for combining the teachings of Priore, Schelthoff and Park is the same as set forth in the rejection of claim 2.
In regard to claim 13, Priore and Schelthoff do not teach, but Park teaches: further comprising performing data labeling and (Park, p. 1424, D. Training and Test Phases "Given the sampled transitions, a loss is calculated from a Q-value and target value [35] [data labeling] (lines 19–21)."; p. 1424, Algorithm 1 "19: Set qu=Q(su,au;θ) 20: Set yu = ru+γ1F(su+1)maxa′Q^(su+1,a′;θ^) 21: Calculate loss L from (3)–(4)"; yu is the "synthetic label" for the specific state-action input)
memory simulation (Park, p. 1422, III. Scheduling by RL "we adopt the replay buffer [e.g. memory simulation, sample is memory] that contains the set of transitions each of which consists of state, action, reward, and next state... The weights of the Q -network [e.g. memory simulation, weights is memory] are periodically replicated to the target Q -network, which improves the convergence stability of the Q -network [35]."; in light of [0024] "The memory simulation may include static memory like parametric model (i.e. model parameter is memory), non-parametric model (i.e. sample is memory)... or neural network model (i.e. weights in network connectivity is memory), etc.";) to said history data and said basic data based on said comparison result, and information from fed back said data labeling and said memory simulation are used as a basis for said machine learning algorithms. (Park, p. 1424, Algorithm 1 Scheduling with Q-Network "1:… replay buffer B [history/basic data] to size NB… Calculate loss L from (3)-(4) 22: Perform a gradient descent step on L [said comparison result] w.r.t. θ "; RL uses the yu [said data labeling] and stored samples in the replay buffer or weights θ [memory simulation] for learning)
The rationale for combining the teachings of Priore, Schelthoff and Park is the same as set forth in the rejection of claim 2.
In regard to claim 14, Priore and Schelthoff do not teach, but Park teaches: further comprising using or excluding specific said machine learning algorithms based on said comparison result. (Park, p. 1424, D. Training and Test Phases "In line 32, the weights of the target Q-network, denoted as θ^ , are periodically replaced to those for the Q-network [using or excluding specific said machine learning algorithms (periodically synced the weights)] for ensuring stable convergence as in [35]. The update period is denoted as NU."; p. 1422, III. SCHEDULING BY RL "The weights of the Q-network are periodically replicated to the target Q-network, which improves the convergence stability of the Q-network [35]."; p. 1424, D. Training and Test Phases "L = ... (yu, qu) (3) where f(yu,qu) is the loss function given by f(yu, qu) = ...yu-qu [said comparison result] ... (4)"; the weights of the target Q-network θ^ and the Q-network are periodically synced, [using or excluding specific said machine learning algorithms] and the weights of Q-network are calculated based on the loss, which comprising said comparison result, yu-qu [based on said comparison result])
The rationale for combining the teachings of Priore, Schelthoff and Park is the same as set forth in the rejection of claim 2.
In regard to claim 15, Priore teaches: wherein said machine learning algorithms comprise decision tree, random forest, artificial neural network or Bayesian network. (Priore, p. 84, 2. KNOWLEDGE-BASED SYSTEMS "machine learning techniques, such as inductive learning or neural networks (NNs), are used"; p. 85, 3. REVIEW OF KNOWLEDGE-BASED APPROACHES "Depending on the type of machine learning algorithm used, these approaches can be divided into the following categories: inductive learning, NNs…")
Claims 11-12 rejected under 35 U.S.C. 103 as being unpatentable over Priore, Schelthoff and Park as applied to claim 10, and in further view of Sun (CN 110084375 A, 20190802)
In regard to claim 11, Priore and Schelthoff do not teach, but Park teaches: wherein comparing said actual production result and said optimized schedule result comprises working out a difference between said predict optimized runtime and said actual runtime, and (Park, p. 1424, D. Training and Test Phases "L = ... f(yu, qu) (3) where f(yu [said actual production result, said actual runtime], qu [said optimized schedule result, said predict optimized runtime]) is the loss function given by f(yu, qu) = ...yu-qu [said comparison result, a difference] ... (4)"; TD Error = (actual performance/target q-value) - (predicted performance/current q-value)) (Park, p. 1421, II. Problem Definition "Oj,k can be processed on any machine among its alternatives, and the processing time of Oj,k is denoted as pj,k."; p. 1423, B. State, Action, and Reward "If the previous setup type is Oj′,k′ and ai is Oj,k, τ(si+1) is calculated as the sum of τ(si), pj,k, and σj′,k′,j,k... As a state transition takes place from si to si+1 , ri is observed... ri is defined as follows: ri = ... τ(si+1)... (1) [reward is based on τ(si+1), which is based on the processing time (said runtime)]"; p. 1424, Algorithm 1 "19: Set qu=Q(su,au;θ) 20: Set yu = ru+γ...max...Q^(su+1,a′;θ^) [yu is based on ru, which is based on the processing time (said runtime)]"; yu is actual accumulated rewards and processing time at time i+1 [said actual runtime] , qu is predicted accumulated rewards and processing time at time i [said predict optimized runtime], therefore yu-qu is [a difference between said predict optimized runtime and said actual runtime])
The rationale for combining the teachings of Priore, Schelthoff and Park is the same as set forth in the rejection of claim 2.
Priore, Schelthoff and Park do not teach, but Sun teaches: if said difference exceeds a set value, determining there is a disparity between said optimized schedule result and said actual production result, and (Sun p. 4 "using TDerror (deviation between the estimated value and the current value of that time sequence difference learning method ) method for updating its network parameter until reaching the maximum number or loss value less than the given threshold value")
if said difference is smaller than said set value, determining there is no disparity between said optimized schedule result and said actual production result. (Sun p. 4 "using TDerror (deviation between the estimated value and the current value of that time sequence difference learning method ) method for updating its network parameter until reaching the maximum number or loss value less than the given threshold value")
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Priore, Schelthoff and Park to incorporate the teachings of Sun by including using the loss as Temporal Difference (TD) loss in RL learning to update its network parameter. Doing so would enable optimal learning from each step and adapt quickly to the current environments. (Sun, p. 4 "evaluating the actor network according to the current environment state and the self-state selecting proper action, and according to the critic network using the policy gradient method to update its network parameter, so as to obtain the optimal control strategy")
In regard to claim 12, Priore, Schelthoff and Park do not teach, but Sun teaches: wherein when it is determined that there is a disparity between said optimized schedule result and said actual production result, (Sun p. 4 "using TDerror (deviation between the estimated value and the current value of that time sequence difference learning method ) method for updating its network parameter until reaching the maximum number or loss value less than the given threshold value")
feedback said comparison result as a basis for said machine learning algorithms. (Sun p. 4 "using TDerror (deviation between the estimated value and the current value of that time sequence difference learning method ) method for updating its network parameter [feedback said comparison result] until reaching the maximum number or loss value less than the given threshold value")
The rationale for combining the teachings of Priore, Schelthoff, Park and Sun is the same as set forth in the rejection of claim 12.
Claim 16 rejected under 35 U.S.C. 103 as being unpatentable over Priore, Schelthoff and Park as applied to claim 9, and in further view of Kozakowski ("Q-Value Weighted Regression: Reinforcement Learning with Limited Data" 20210212)
In regard to claim 16, Priore, Schelthoff and Park do not teach, but Kozakowski teaches: wherein acquiring said history data and said basic data comprises said machine learning algorithm decides whether to process said history data and said basic data with average, median or weighted treatment. (Kozakowski, p. 2, 2. Q-Value Weighted Regression "Improvement is achieved by weighting the actor loss by exponentiated advantage Aμ(s, a) of an action, skewing the regression towards the better-performing actions."; p. 3, 2.3 Q-Value Weighted Regression, a) Counterfactual Action Sampling "we calculate the advantage of the sampling policy μ based on a learned Q-function: Aμ(s, a)=Qμ(s, a)−Vμ(s)"; RL selects high-value experiences from a dataset, effectively "weighting" past data [weighted treatment] based on learned Q-values to improve decision-making, especially with limited samples, rather than just processing data randomly)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Priore, Schelthoff and Park to incorporate the teachings of Kozakowski by including Q-Value Weighted Regression (QWR) with the sampling policy μ. Doing so would achieve good performance in the RL learning (Kozakowski, p. 1, Abstract "We introduce Q-Value Weighted Regression (QWR), a simple RL algorithm that excels in these aspects. QWR is an extension of Advantage Weighted Regression (AWR)… We also verify that QWR performs well in the offline RL setting")
Claim 17 rejected under 35 U.S.C. 103 as being unpatentable over Priore, Schelthoff and Park as applied to claim 9, and in further view of Chau (US 20220171373 A1)
In regard to claim 17, Priore, Schelthoff and Park do not teach, but Chau teaches: wherein comparing said actual production result and said optimized schedule result comprises auto-regulating algorithms and (Chau, [0155] "FIG. 11 shows an example of high bias and high variance. High variance can indicate over-fitting. Over-fitting can be prevented using various methods. For example, regularization [auto-regulating algorithms] can be used, where large weights can be penalized using penalties or constraints on their squared values (L2 penalty) or absolute values (L1 penalty). [comparing said actual production result and said optimized schedule result]")
variation factors in information cross-validation based on said recipe groups or products. (Chau, [0148] Other validation methods can be used to validate the model. For example, an N-fold cross-validation [variation factors in information cross-validation] method may be used. In this method, the total dataset [based on said products] is divided into one final test set and N other subsets, where N is an integer greater than one."; [0135] "At 504, configuration and recipe data [input datasets includes recipe data of products] for the one or more tools are received."; [0005] "The processing chambers in the substrate processing tools usually repeat the same task on multiple substrates. The processing chambers operate based on a recipe that defines process parameters."; recipe data are sequencing information for the substrates/wafers [products]; also see Schelthoff teaches feature set (input data) includes recipe group or products (lots, wafers)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Priore, Schelthoff and Park to incorporate the teachings of Chau by including regularization and cross-validation. Doing so would prevent over-fitting and achieve lower error rate. (Chau, [0155] "Over-fitting can be prevented using various methods. For example, regularization can be used, where large weights can be penalized using penalties or constraints..."; [0148] "The model with the lowest validation error rate is deployed for use.")
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SU-TING CHUANG whose telephone number is (408)918-7519. The examiner can normally be reached Monday - Thursday 8-5 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SU-TING CHUANG/Examiner, Art Unit 2146