DETAILED ACTION
Claims 1-12 and 15-16 (filed 12/04/2025) have been considered in this action. Claims 1 and 15 have been amended. Claims 2-12 and 16 have been presented in the same format as previously presented. Claims 13, 14 and 17 have been canceled.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/04/2025 has been entered.
Response to Arguments
Applicant's arguments, see page 5 paragraph 2, filed 12/04/2025, with respect to the rejection of claims 1-12, 15 and 16 under 35 U.S.C. 112(b) have been fully considered but they are not persuasive.
While the examiner agrees the issue of “relearning” when no initial learning has been performed has been overcome, the scope of the amended limitation is still indefinite. The claim has been amended to recite:
wherein a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required for learning via the neural network.
Again, it is noted that there is no required learning in the claim, nor is the time required for adjusting the parameter value described in any fashion in the specification. This limitation appears to be functional language in the form of an intended result of the invention. As noted in MPEP 2173.05(g) “When a claim limitation employs functional language, the examiner’s determination of whether the limitation is sufficiently definite will be highly dependent on context (e.g., the disclosure in the specification and the knowledge of a person of ordinary skill in the art)”. PHOSITA would recognize there is no clear cut scope to this claim limitation because the execution time of a learning of a neural network is ambiguous, as there is no limit to the complexity of the neural network, the form of the neural network, the parameters of the neural network, the amount of training data provided for the learning, etc.. PHOSITA would recognize that different neural networks would have different learning times based on all of the above criteria. PHOSITA would further recognize that in order for a parameter adjustment time to be shorter than a neural network learning time, it must be clear how long the duration of both are, or clearly establish a process for determining these durations. However, the provided disclosure provides no means of discerning how long either of these processes take, nor any specifics about the neural network such that PHOSITA would be able to experimentally determine such criteria. In fact, aside from other recitations of this functional language/intended result of learning time being longer than adjustment time (see [0037] of US 20230176548), the only concrete provided example is in [0064] which states that a deep neural network (not claimed, only regular neural network is claimed) with 1,545 parameters (not claimed) has a learning time that is shorter than that of the corrector (arithmetic express) with only 3 parameters (again, not claimed). None of these details are claimed, and in fact this single example seems to be the only functional example provided for an instance in which parameter adjustment time is shorter than learning time. How the intended result was determined is non-existent in the disclosure. The claims are broad and cover every solution in which parameter adjustment time is shorter than the learning time, without claiming the particular arrangement that is supported by paragraph [0064]. In other words, the provided disclosure only has evidence for when the parameter adjustment time is shorter than the learning time in the singular example provided by paragraph [0064], but broadly covers every solution whereby this criteria is true.
This limitation is further confusing and unclear because the limitation states “determine a parameter value of the adjustable coefficient”, however the final limitation then refers to “a time required to adjust the parameter value is shorter than a time required for learning”. The final limitation is referring to the fact that the time to “adjust the parameter value” is less than that for learning, however nowhere in the claim is the parameter value ever required to be adjusted. While the claim makes reference to an adjustable coefficient (meaning capable of adjustment), it never requires the adjustable coefficient to ever be adjusted or changed. When looking at the claim language under the BRI, all that is required is for a frequency analysis to occur, determine a frequency to be improved, and determine a parameter value, however that determined parameter value is never actually required to be used as a new value for the adjustable coefficient. When considering that the final limitation requires “a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required for learning via the neural network” but the claim never requires any adjustment of the parameter value, nor any learning by the neural network, it is considered that the claim is indefinite. It is unclear how either of these concepts can relate to the other limitations in the claim, when their required functionality is not actually claimed.
Applicant's arguments, see page 5 paragraph 5, filed 12/04/2025, with respect to the rejection of claims 1-12, 15 and 16 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant has alleged that Asano (WO 2019069649, hereinafter Asano) fails to teach:
perform frequency analysis on the control deviation;
determine a frequency to be improved based on an analysis result of the frequency analysis;
and determine a parameter value of the adjustable coefficient of the arithmetic expression so that the control deviation at the determined frequency does not exceed a prescribed value;
…
and generate the control signal to control the position of the control object based on the first signal and the second signal;
Asano teaches these concepts in that a frequency analysis in performed and a frequency to be improved is determined ([page 6] “if the controller 12 has a high-pass stage deviation and the performance is degraded, a low-pass filter can be used to drop the high-pass from the band in which the controller 12 operates. Similarly, if a particular frequency range degrades the control performance by controller 12, a notch filter can be used”) because the controller is recognizing the frequencies at which performance has degraded and adjusting the filter in accordance with those frequencies at which performance is degraded while allowing those frequencies at which performance is not degraded. The claim limitations relating to determining a parameter value so that the control frequency at the determined frequency does exceed a prescribed value is also taught by Asano ([page 6] if a particular frequency range degrades the control performance by controller 12, a notch filter can be used. If the low-pass performance of the controller 11 is sufficient, a low-pass filter can be used as the band-pass filter 41 because the low-pass is left to the controller 11. These filters may be used in combination. Further, the band-pass filter 41 to be used may be changed (switched) according to the job of the imprint apparatus. In this case, since learning of the neural network is performed in accordance with the type of the band pass filter 41, control is performed by switching to parameters of the controller 12 corresponding to the band pass filter 41). This portion is teaching that when the performance has degraded, the filter parameters of the notch filter can be changed to those in which the performance degradation is reduced to an acceptable amount (prescribed value), thus teaching this concept. Finally, when Asano teaches adder 14 ([page 4]) which combines the signal from the first control unit (11) and the second control unit (12) to produce an output (14) which is used to control the positioning of a stage in a lithography apparatus ([page 2]). Therefore, all of the above limitations are taught by Asano.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-12 and 15-16 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1 and 15 recite “wherein a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required for learning via the neural network”. There is insufficient support from the original disclosure to support such broad intended results based functional limitation. As noted in the MPEP 2161.01:
“original claims may lack written description when the claims define the invention in functional language specifying a desired result but the specification does not sufficiently describe how the function is performed or the result is achieved. For software, this can occur when the algorithm or steps/procedure for performing the computer function are not explained at all or are not explained in sufficient detail (simply restating the function recited in the claim is not necessarily sufficient). In other words, the algorithm or steps/procedure taken to perform the function must be described with sufficient detail so that one of ordinary skill in the art would understand how the inventor intended the function to be performed”
The claim fails to establish what particular neural network is being utilized, and thus is covering every form of neural network. The provided disclosure provides no means of discerning how long either of the “time required to adjust the parameter value” or “time required for learning via the neural network” processes take, nor any specifics about the neural network such that PHOSITA would be able to experimentally determine such criteria. In fact, aside from other recitations of this functional language/intended result of learning time being longer than adjustment time (see [0037] of US 20230176548), the only concrete provided example is in [0064] of US 20230176548 which states that a deep neural network (not claimed, only regular neural network is claimed) with 1,545 parameters (not claimed) has a learning time that is shorter than that of the corrector (arithmetic express) with only 3 parameters. None of these details are claimed, and in fact this single example seems to be the only functional example provided for an instance in which parameter adjustment time is shorter than learning time. The claims are broad and cover every solution in which parameter adjustment time is shorter than the learning time, without claiming the particular arrangement that is supported by paragraph [0064]. In other words, the provided disclosure only has evidence for when the parameter adjustment time is shorter than the learning time in the singular example provided by paragraph [0064], but the claim broadly covers every form of parameter adjustment and neural network learning timing. The claims are rejected on the basis that the supporting disclosure only provides a single example, yet claims every conceivable solution to the intended result (i.e. every solution where the parameter adjustment time is less than the neural network learning time). Such an intended result would be highly dependent upon the complexity of the neural network used. Additionally, there is no discussion on how the time taken to adjust a parameter value is determined or what is encompassed by such a limitation.
Accordingly, claims 2-12 and 16 are dependent upon claim 1, and thus inherit the rejection of claims 1 and 15 respectively under 35 U.S.C. 112(a).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-12 and 15-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 15 recite the limitation: “wherein a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required via learning of the neural network”. When looking to MPEP 2173.05(g), the examiner considers this limitation indefinite because the there is no clear cut indication of the scope of the claims because it merely establishes an intended result (systems in which parameter adjustment time is less than neural network learning) without any clear boundary on the invention aside from that result. A person having ordinary skill in the art would not know the bounds of the claim because it seemingly covers every possible combination that reaches that intended result, without any limits. As noted in the above rejection of claims under 112(a), there is no description for any other solution that reaches this intended result aside from the singular example at [0064] of US 20230176548.
Additionally, there is no discussion on how the time taken to adjust a parameter value is determined or what is encompassed by such a limitation. Is the parameter value adjustment time just the time required for a processor to query and write to memory the parameter value? The time required for the controller to actually implement the adjusted parameter value? This step is unclear on what is required in order for a timing of a parameter value adjustment to be determined for comparison. Accordingly, this limitation is ambiguous and fails the requirements of 35 U.S.C. 112(b).
Claims 2-12 and 16 are dependent upon claims 1 and 15 respectively, and thus inherit the rejection of claims 1 and 15 under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-12 and 15-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 15 each recite “a time required to adjust the parameter value of the adjustable coefficient”. This step is unclear because neither claim 1 nor 15 require the adjustment of a parameter value of the adjustable coefficient. The claim recites that a parameter value is determined, but never actually uses/sets that parameter value as the adjustable coefficient value in order to generate a correction signal on the basis of a determined parameter value. It is unclear how “a time required to adjust the parameter value of the adjustable coefficient” is germane to the invention or limits its scope when there is no claimed adjustment of a parameter value.
Claims 2-12 and 16 are dependent upon claims 1 and 15, and thus inherit the rejection of claims 1 and 15 under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 12, 15 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Asano et al. (WO 2019069649, hereinafter Asano) in view of Havlena et al. (US 20210341895, hereinafter Havlena).
In regards to Claim 1, Asano teaches “A control apparatus for generating a control signal for controlling a position of a control object, the control apparatus comprising:” ([page 3] The imprint apparatus 100 has a mechanical structure and a control system 200. The main body structure 101 of the imprint apparatus 100 is installed on the floor via a tripod or four legs vibration isolation mechanism 102 using an air spring or the like. The wafer (substrate) 103 is held on a wafer stage (substrate stage) 7 by a wafer chuck (not shown). The wafer stage 7 moves the wafer 103 with a sufficient X direction and Y direction stroke in order to perform imprint processing on each shot region on the entire surface of the wafer 103) “at least one processor or circuit configured to: generate a first signal based on a control deviation of the control object;” ([page 3] The control system 200 includes an arithmetic processing unit such as a CPU or an FPGA, and a storage device such as a memory. [page 4] In the control unit 1, the difference between the stage position (measured value) sent from the position measurement unit 2 by the deviation calculation unit 13 and the target value of the stage position sent from the position command unit 3 (control deviation, hereinafter called stage deviation) ) And sends the stage deviation to the controller 11 and the controller 12. The controller 11 (first control unit) uses a PID control system, and the controller 12 (learning control unit) is formed of a control system including a neural network) “generate a correction signal by correcting the control deviation in accordance with an arithmetic expression including an adjustable coefficient, wherein in generating the correction signal, the at least one processor or circuit is configured to” ([page 4] The controller 11 (first control unit) uses a PID control
system, and the controller 12 (learning control unit) is formed of a control system including a neural network....The network parameters of the neural network 16 in the controller 12 need to be adjusted in some way in advance. As the adjustment method, for example, a network parameter adjustment method by reinforcement learning can be used, but any method may be used to adjust the network parameter. Further, the neural network may be a network (policy network) that directly outputs one corresponding to the dimension of the instruction value, or may be a network (action value network) for calculating the value of the instruction value. In the case of the action value network, the controller 12 adds a selection unit for selecting an action with the highest value to the subsequent stage of the neural network 16, and the instruction value selected by the selection unit becomes the output of the controller 12. The network parameter may be machine-learning in a state where the range of the second operation amount that can be output from the second control unit is limited by the output limiter 21) “perform frequency analysis on the control deviation; determine a frequency to be improved based on an analysis result of the frequency analysis; and determine a parameter value of the adjustable coefficient of the arithmetic expression so that the control deviation at the determined frequency does not exceed a prescribed value;” ([page 6] A third embodiment will now be described with reference to FIG. FIG. 5 is a block diagram of the control unit 1, which differs from the first embodiment in that a band pass filter 41 for reducing (cut off) a predetermined band is applied to a stage deviation which is an input of the controller 12. . The signal from the band pass filter 41 is input to the learning control unit. The band pass filter 41 can use a high pass filter, a low pass filter, a band pass filter, or a notch filter. Each filter may have different bands to be reduced. For example, if the controller 12 has a high-pass stage deviation and the performance is degraded, a low-pass filter can be used to drop the high-pass from the band in which the controller 12 operates. Similarly, if a particular frequency range degrades the control performance by controller 12, a notch filter can be used. If the low-pass performance of the controller 11 is sufficient, a low-pass filter can be used as the band-pass filter 41 because the low-pass is left to the controller 11. These filters may be used in combination. Further, the band-pass filter 41 to be used may be changed (switched) according to the job of the imprint apparatus. In this case, since learning of the neural network is performed in accordance with the type of the band pass filter 41, control is performed by switching to parameters of the controller 12 corresponding to the band pass filter 41. That is, the parameters of the controller 12 have the first parameter determined by machine learning using the deviation of the first band as an input when the first filter is used. It also has a second band deviation different from the first band when the second filter is used and a second parameter determined by machine learning used as an input) “generate a second signal by a neural network based on the correction signal” ([page 4] In the control unit 1, the difference between the stage position (measured value) sent from the position measurement unit 2 by the deviation calculation unit 13 and the target value of the stage position sent from the position command unit 3 (control deviation, hereinafter called stage deviation) ) And sends the stage deviation to the controller 11 and the controller 12. The controller 11 (first control unit) uses a PID control system, and the controller 12 (learning control unit) is formed of a control system including a neural network. The controller 11 receives the information on the stage deviation and outputs the operation amount to the wafer stage 7) “and generate the control signal to control the position of the control object based on the first signal and the second signal” ([page 4] The adder 14 outputs the output value U1 (first operation amount) generated by the controller 11 (first control unit) and the output value U2 generated by the controller 12 and limited by the output limiter 21 (second operation Output the sum (sum) of the quantities).
Asano fails to teach “wherein a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required for relearning of the neural network”.
Havlena teaches “an arithmetic expression including an adjustable coefficient” ([0015] There have been numerous attempts to develop a PID auto tuner based on time at frequency domain models. However, a reliable solution seems to be still missing. One may see here a good approach using historical or real time data and machine learning techniques, particularly reinforcement learning. [0022] A PID controller may be running in a so-called DDC engine (direct digital control) and the Q-learning algorithm for controller tuning may be running in an Edge Analytics/ML Engine. A PID controller may provide its input and output data to the auto tuner, and the auto tuner may provide updated values of controller parameters. The communication between the two blocks may be via an internal memory (e.g., a memory mapped file or shared memory)) “wherein a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required for relearning of the neural network” ([0028] Published approaches for applying the reinforcement learning to PID controller auto tuning may therefore consider a fixed controller structure, e.g., PID, and consider the PID parameters as optimized parameters which are subject to a machine learning technique. In this way, the integral action may be enforced by setting the controller structure in advance....Buffer 94 may update PID parameters using a schedule approach (e.g., periodically, on request, when control performance deteriorates, and so forth); [0037] The controller may also incorporate other measured variables, for example, a preview of the next reference value and a set of measurable disturbances. Then the control may be equivalent to a PID with feedforward from measurable disturbances and some predictive functionality. Unlike the “brute force” applications of machine learning techniques in control, that may require millions of data samples and long learning periods (weeks or months), the Q-learning method of the proposed auto tuner may be modified by incorporation of control-relevant prior information, that can guarantee faster convergence (about one to two days) and acceptable control performance during the learning phase, so that the algorithm can be running on real-time process data; wherein the adjustment of the variable to real-time data is inherently faster than one to two days of learning, and PID are inherently arithmetic expressions).
It would have been obvious to a person having ordinary skill in the art before the effective file date of the claimed invention to have modified the system with a PID and neural network control block with adjustable parameters for determining control actions of a position command as taught by Asano, with the use of the learning methods of Havlena which adjust the parameters of a PID using machine learning in which the learning occurs slower than a parameter adjustment, because by incorporating such a feature of auto-tuning parameters of a PID, it would gain the stated benefit of Havlena, notably optimized control when poor performance is detected ([0015]). By incorporating these elements, it can be considered taking the known control system with a PID and neural network based control outputs, and modify the controllers to utilize the learning methods of Havlena for learning optimized parameters of the PID expressions in a known way that achieves predictable results.
In regards to Claim 2, the combination of Asano and Havlena teaches the control apparatus as incorporated by claim 1 above. Asano further teaches “The control apparatus according to claim 1, wherein the arithmetic expression contains a term proportional to the control deviation” ([page 4] The controller 11 (first control unit) uses a PID control system; wherein the P in PID stands for proportional).
In regards to Claim 3, the combination of Asano and Havlena teaches the control apparatus as incorporated by claim 1 above. Asano further teaches “The control apparatus according to claim 1, wherein the arithmetic expression contains a term that integrates the control deviation not less than once” ([page 4] The controller 11 (first control unit) uses a PID control system; wherein the I in PID stands for integration).
In regards to Claim 4, the combination of Asano and Havlena teaches the control apparatus as incorporated by claim 1 above. Asano further teaches “The control apparatus according to claim 1, wherein the arithmetic expression contains a term that differentiates the control deviation not less than once” ([page 4] The controller 11 (first control unit) uses a PID control system; wherein the D in PID stands for derivative/differentiation).
In regards to Claim 5, the combination of Asano and Havlena teaches the control apparatus as incorporated by claim 1 above. Asano further teaches “The control apparatus according to claim 1, wherein the arithmetic expression contains at least one of a term proportional to the control deviation, a term that integrates the control deviation not less than once, or a term that differentiates the control deviation not less than once” ([page 4] The controller 11 (first control unit) uses a PID control system; wherein the D in PID stands for derivative/differentiation).
In regards to Claim 6, the combination of Asano and Havlena teaches the control apparatus as incorporated by claim 1 above. Havlena further teaches “The control apparatus according to claim 1, wherein the at least one processor or circuit is further configured to set the arithmetic expression” ([0025] To this end, the control action supplied to the reinforcement learning module is the time-difference 93 of the control action 83 applied to the controlled process module 81. An output 86 from reinforcement learning module 92 may go to PID controller 78 and optimize performance of PID controller 78. Control action may be an input 83 from PID controller 78 to controlled process module 81 for an optimized control process of module 81. PID controller 78 and machine learning module 84 may be employed on the same hardware of a microcontroller 88).
In regards to Claim 12, the combination of Asano and Havlena teaches the control apparatus as incorporated by claim 1 above. Asano further teaches “The control apparatus according to claim 1, the at least one processor or circuit include a learning model configured to determine a parameter value of the neural network by machine learning” ([page 4] The network parameters of the neural network 16 in the controller 12 need to be adjusted in some way in advance. As the adjustment method, for example, a network parameter adjustment method by reinforcement learning can be used, but any method may be used to adjust the network parameter. Further, the neural network may be a network (policy network) that directly outputs one corresponding to the dimension of the instruction value, or may be a network (action value network) for calculating the value of the instruction value. In the case of the action value network, the controller 12 adds a selection unit for selecting an action with the highest value to the subsequent stage of the neural network 16, and the instruction value selected by the selection unit becomes the output of the controller 12. The network parameter may be machine-learning in a state where the range of the second operation amount that can be output from the second control unit is limited by the output limiter 21).
In regards to Claim 15, Asano teaches “A lithography apparatus for transferring a pattern of an original to a substrate, the lithography apparatus comprising” ([page 2] In the present embodiment, an imprint apparatus will be described as an example of a lithography apparatus that forms a pattern on a substrate) “a control apparatus for generating a control signal for controlling a position of a control object, which is one the substrate or the original, the control apparatus comprising at least one processor or circuit configured to:” ([page 3] The imprint apparatus 100 has a mechanical structure and a control system 200. The main body structure 101 of the imprint apparatus 100 is installed on the floor via a tripod or four legs vibration isolation mechanism 102 using an air spring or the like. The wafer (substrate) 103 is held on a wafer stage (substrate stage) 7 by a wafer chuck (not shown). The wafer stage 7 moves the wafer 103 with a sufficient X direction and Y direction stroke in order to perform imprint processing on each shot region on the entire surface of the wafer 103… [page 3] The control system 200 includes an arithmetic processing unit such as a CPU or an FPGA, and a storage device such as a memory.) “generate a first signal using a control deviation of the control object;” ([page 3] The control system 200 includes an arithmetic processing unit such as a CPU or an FPGA, and a storage device such as a memory. [page 4] In the control unit 1, the difference between the stage position (measured value) sent from the position measurement unit 2 by the deviation calculation unit 13 and the target value of the stage position sent from the position command unit 3 (control deviation, hereinafter called stage deviation) ) And sends the stage deviation to the controller 11 and the controller 12. The controller 11 (first control unit) uses a PID control system, and the controller 12 (learning control unit) is formed of a control system including a neural network) “generate a correction signal by correcting the control deviation in accordance with an arithmetic expression including an adjustable coefficient, wherein in generating the correction signal, the at least one processor or circuit is configured to” ([page 4] The controller 11 (first control unit) uses a PID control
system, and the controller 12 (learning control unit) is formed of a control system including a neural network....The network parameters of the neural network 16 in the controller 12 need to be adjusted in some way in advance. As the adjustment method, for example, a network parameter adjustment method by reinforcement learning can be used, but any method may be used to adjust the network parameter. Further, the neural network may be a network (policy network) that directly outputs one corresponding to the dimension of the instruction value, or may be a network (action value network) for calculating the value of the instruction value. In the case of the action value network, the controller 12 adds a selection unit for selecting an action with the highest value to the subsequent stage of the neural network 16, and the instruction value selected by the selection unit becomes the output of the controller 12. The network parameter may be machine-learning in a state where the range of the second operation amount that can be output from the second control unit is limited by the output limiter 21) “perform frequency analysis on the control deviation; determine a frequency to be improved based on an analysis result of the frequency analysis; and determine a parameter value of the adjustable coefficient of the arithmetic expression so that the control deviation at the determined frequency does not exceed a prescribed value;” ([page 6] A third embodiment will now be described with reference to FIG. FIG. 5 is a block diagram of the control unit 1, which differs from the first embodiment in that a band pass filter 41 for reducing (cut off) a predetermined band is applied to a stage deviation which is an input of the controller 12. . The signal from the band pass filter 41 is input to the learning control unit. The band pass filter 41 can use a high pass filter, a low pass filter, a band pass filter, or a notch filter. Each filter may have different bands to be reduced. For example, if the controller 12 has a high-pass stage deviation and the performance is degraded, a low-pass filter can be used to drop the high-pass from the band in which the controller 12 operates. Similarly, if a particular frequency range degrades the control performance by controller 12, a notch filter can be used. If the low-pass performance of the controller 11 is sufficient, a low-pass filter can be used as the band-pass filter 41 because the low-pass is left to the controller 11. These filters may be used in combination. Further, the band-pass filter 41 to be used may be changed (switched) according to the job of the imprint apparatus. In this case, since learning of the neural network is performed in accordance with the type of the band pass filter 41, control is performed by switching to parameters of the controller 12 corresponding to the band pass filter 41. That is, the parameters of the controller 12 have the first parameter determined by machine learning using the deviation of the first band as an input when the first filter is used. It also has a second band deviation different from the first band when the second filter is used and a second parameter determined by machine learning used as an input) “generate a second signal by a neural network based on the correction signal” ([page 4] In the control unit 1, the difference between the stage position (measured value) sent from the position measurement unit 2 by the deviation calculation unit 13 and the target value of the stage position sent from the position command unit 3 (control deviation, hereinafter called stage deviation) ) And sends the stage deviation to the controller 11 and the controller 12. The controller 11 (first control unit) uses a PID control system, and the controller 12 (learning control unit) is formed of a control system including a neural network. The controller 11 receives the information on the stage deviation and outputs the operation amount to the wafer stage 7) “and generate the control signal to control the position of the control object based on the first signal and the second signal” ([page 4] The adder 14 outputs the output value U1 (first operation amount) generated by the controller 11 (first control unit) and the output value U2 generated by the controller 12 and limited by the output limiter 21 (second operation Output the sum (sum) of the quantities).
Asano fails to teach “wherein a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required for relearning of the neural network”.
Havlena teaches “an arithmetic expression including an adjustable coefficient” ([0015] There have been numerous attempts to develop a PID auto tuner based on time at frequency domain models. However, a reliable solution seems to be still missing. One may see here a good approach using historical or real time data and machine learning techniques, particularly reinforcement learning. [0022] A PID controller may be running in a so-called DDC engine (direct digital control) and the Q-learning algorithm for controller tuning may be running in an Edge Analytics/ML Engine. A PID controller may provide its input and output data to the auto tuner, and the auto tuner may provide updated values of controller parameters. The communication between the two blocks may be via an internal memory (e.g., a memory mapped file or shared memory)) “wherein a time required to adjust the parameter value of the adjustable coefficient is shorter than a time required for relearning of the neural network” ([0028] Published approaches for applying the reinforcement learning to PID controller auto tuning may therefore consider a fixed controller structure, e.g., PID, and consider the PID parameters as optimized parameters which are subject to a machine learning technique. In this way, the integral action may be enforced by setting the controller structure in advance....Buffer 94 may update PID parameters using a schedule approach (e.g., periodically, on request, when control performance deteriorates, and so forth); [0037] The controller may also incorporate other measured variables, for example, a preview of the next reference value and a set of measurable disturbances. Then the control may be equivalent to a PID with feedforward from measurable disturbances and some predictive functionality. Unlike the “brute force” applications of machine learning techniques in control, that may require millions of data samples and long learning periods (weeks or months), the Q-learning method of the proposed auto tuner may be modified by incorporation of control-relevant prior information, that can guarantee faster convergence (about one to two days) and acceptable control performance during the learning phase, so that the algorithm can be running on real-time process data; wherein the adjustment of the variable to real-time data is inherently faster than one to two days of learning, and PID are inherently arithmetic expressions).
It would have been obvious to a person having ordinary skill in the art before the effective file date of the claimed invention to have modified the system with a PID and neural network control block with adjustable parameters for determining control actions of a position command as taught by Asano, with the use of the learning methods of Havlena which adjust the parameters of a PID using machine learning in which the learning occurs slower than a parameter adjustment, because by incorporating such a feature of auto-tuning parameters of a PID, it would gain the stated benefit of Havlena, notably optimized control when poor performance is detected ([0015]). By incorporating these elements, it can be considered taking the known control system with a PID and neural network based control outputs, and modify the controllers to utilize the learning methods of Havlena for learning optimized parameters of the PID expressions in a known way that achieves predictable results.
In regards to Claim 16, the combination of Asano and Havlena teaches the control apparatus as incorporated by claim 15 above. Asano further teaches “An article manufacturing method comprising: a transfer step of transferring a pattern of an original to a substrate using the lithography apparatus according to claim 15” ([page 2] The imprint apparatus forms a pattern of a cured product to which a concavo-convex pattern of a mold is transferred by bringing an imprint material supplied on a substrate into contact with the mold and applying energy for curing to the imprint material) “a processing step of processing the substrate having undergone the transfer step” ([page 2] Then, the pattern of the mold can be transferred to the imprint material on the substrate by widening the space between the mold and the substrate and peeling (releasing) the mold from the cured imprint material. Such a series of processes is called an imprint process and is performed for each of a plurality of shot areas on the substrate. That is, when the imprint process is performed on each of a plurality of shot areas in one substrate, the imprint process is repeatedly performed as many as the number of shot areas in the one substrate) “obtaining an article from the substrate having undergone the processing step” ([page 3] the imprinting process is sequentially performed while changing the position of the shot area, and when the imprinting process is completed for all the shot areas of one wafer, the wafer stage 7 moves to the wafer exchange position. Then, the wafer after imprinting is collected by a wafer exchange hand (not shown), and the next new wafer is supplied).
Claims 7-11 are rejected under 35 U.S.C. 103 as being unpatentable over Asano and Havlena as applied to claim 6 above, and further in view of Kraus (US 4602326, hereinafter Kraus).
In regards to Claim 7, the combination of Asano and Havlena teach the control apparatus as incorporated by claim 6 above.
The combination of Asano and Havlena fail to teach “The control apparatus according to claim 6, wherein the at least one processor or circuit is configured to reset the adjustable coefficient of the arithmetic expression in a case where a predetermined condition is met” ([col 4 line 9] If the magnitude of disturbance 26 is sufficiently large to cause the process 12 to make appreciable changes in the value of the process controlled variable 14, the control loop 8 will respond accordingly with corrective action to remove the effects of the disturbance 26...a comparator 30 operates to produce an error signal 32 which represents the difference between the value of the process controlled variable 14 and the desired value. An adapter 34 (to be described in greater detail hereinafter) is coupled to the comparator 30 for receiving the error signal 32. The adapter 34 then produces a processor signal 40 which is subsequently applied to an input of a conventional proportional-integral-derivative (PID) controller 42. Being also coupled to comparator 30 for receiving the error signal 32, the PID controller 42 operates to produce the controller signal 20 (representing corrective action) which is proportional to a three-term sum of the error signal 32 plus a time integral of the error signal plus a time derivative of the error signal. The relative contributions of each of the three terms are determined by constants that are known respectively as proportional (P), integral (I) and derivative (D) coefficients. The controller 42 responds to the processor signal 40 for setting the values for the PID coefficients. [col 12 line 6] In the preferred embodiment of applicant's invention, provision is also made for a pre-adapt mode wherein various process characteristics are automatically identified so that initial settings of the P, I and D coefficients, the maximum wait time, and the size of the noise band NB can be determined...When the process 12 reaches its new steady state condition or the process controlled variable 14 changes 10%, the process is returned to its initial steady state value).
It would have been obvious to a person having ordinary skill in the art before the effective file date of the claimed invention to have modified the system which determines adjustable coefficients for a neural network controller and PID controller as taught by Asano and Havlena, with the use of monitoring the process and resetting the parameters when a condition is detected as taught by Kraus, because it would then gain the stated benefits of Kraus, namely “[col 1] controller operating parameters are changed automatically as required in response to differences occurring between the actual and desired states of the process so that controller behavior substantially matches process dynamics”. In other words, the controller would be more suitable for matching control signaling to process dynamics of said system, because it is able to detect when those don’t match and make parameter changes accordingly. By combining these elements, it can be considered taking the known method of resetting parameters when a condition is detected as taught by Kraus, and implement them in the controller of Asano and Havlena in a known way that achieves predictable results.
In regards to Claim 8, the combination of Asano, Havlena and Kraus teach the control apparatus as incorporated by claim 7 above. Kraus further teaches “The control apparatus according to claim 7, wherein the predetermined condition includes the control deviation exceeding a prescribed value” ([col 4 line 17] Generally, the operation of self-tuning controller system 10 can be described in nine states...the self-tuning controller system 10 operates with the PID coefficients set in the PID controller 42 and performs the functions described in block 64 of FIG. 3A. So long as the value of the error signal 32 remains between the upper and lower levels (corresponding to the situation depicted in FIG. 2 where trace 50 is left of time T.sub.q), no decisions are made regarding changes in the PID coefficients [col 8 line 35] After calculating the two intermediate times, the adapter 34 enters a ninth state known as the ADAPT MODE of block 83 shown in FIG. 7. Based on the measured peak amplitudes E1, E2 and E3, the adapter calculates the previously defined damping and overshoot performance measures which are identified as DMP (MEASURED) and OVR (MEASURED) respectively. In general, three steps are taken in the ADAPT MODE for determining the new PID coefficients).
In regards to Claim 9, the combination of Asano and Havlena teach the control apparatus as incorporated by claim 6 above. Kraus further teaches “The control apparatus according to claim 6, wherein in a case where a predetermined condition is met, the at least one processor or circuit resets the coefficient of the arithmetic expression in a state in which a parameter value of the neural network is maintained in a previous state” ([col 6 line 28] the self-tuning controller system 10 operates with the PID coefficients set in the PID controller 42 and performs the functions described in block 64 of FIG. 3A. So long as the value of the error signal 32 remains between the upper and lower levels (corresponding to the situation depicted in FIG. 2 where trace 50 is left of time T.sub.q), no decisions are made regarding changes in the PID coefficients [ [col 12 line 6] In the preferred embodiment of applicant's invention, provision is also made for a pre-adapt mode wherein various process characteristics are automatically identified so that initial settings of the P, I and D coefficients, the maximum wait time, and the size of the noise band NB can be determined...When the process 12 reaches its new steady state condition or the process controlled variable 14 changes 10%, the process is returned to its initial steady state value).
In regards to Claim 10, the combination of Asano, Havlena and Kraus teach the control apparatus as incorporated by claim 9 above. Kraus further teaches “The control apparatus according to claim 9, wherein the predetermined condition includes a condition that the control deviation exceeds a prescribed value” ([col 6 line 28] the self-tuning controller system 10 operates with the PID coefficients set in the PID controller 42 and performs the functions described in block 64 of FIG. 3A. So long as the value of the error signal 32 remains between the upper and lower levels (corresponding to the situation depicted in FIG. 2 where trace 50 is left of time T.sub.q), no decisions are made regarding changes in the PID coefficients).
In regards to Claim 11, the combination of Asano, and Havlena teach the control apparatus as incorporated by claim 6 above. Kraus further teaches “The control apparatus according to claim 6, wherein the at least one processor or circuit resets the arithmetic expression based on a disturbance suppression characteristic” ([col 5 line 17] As can be understood, the closed-loop response of the control loop 8 (as represented by the behavior of the trace 50) can be specified in terms of damping, overshoot and time period which are performance measures that are well known to control engineers for describing the behavior of a control loop. In particular, damping DMP, overshoot OVR and the time period T.sub.0 can be defined as follows....As a result, the desired performance for the control loop 8 can be specified in terms of prescribed values for damping, overshoot and time period which in turn can be used to describe an ideal pattern for use to tune the self-tuning controller system 10. In other words, if the trace 50 matched the ideal pattern, the controller settings that produced the trace would be optimum. In applicant's invention, the ideal pattern preferably includes three peaks arranged so that the second peak occurs midway in the time interval between the occurrences of the first and third peaks; wherein damping is considered disturbance suppression).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN M SKRZYCKI whose telephone number is (571)272-0933. The examiner can normally be reached M-Th 7:30-3:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ken Lo can be reached at 571-272-9774. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN MICHAEL SKRZYCKI/Examiner, Art Unit 2116