Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is a non-final First Office Action.
This action is in response to communications filed on 04/17/2023.
Claims 1-15 are pending and have been considered.
Claims 13 is rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter, “software per se”.
Claims 1- 15 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter, a judicial exception, an abstract idea, without significantly more.
Claims 1-7, 10-15 are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez-Molina et al, Multi-objective meta-heuristic optimization in intelligent control: A survey on the controller tuning problem, Applied Soft Computing Journal 93 (2020) 106342 (“ROD”) in view of Arcelli et al, EASIER: an Evolutionary Approach for multi-objective Software archItecturE Refactoring, 2018 IEEE International Conference on Software Architecture (“ARC”)
Claims 8(1), 9(5) are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez-Molina et al, Multi-objective meta-heuristic optimization in intelligent control: A survey on the controller tuning problem, Applied Soft Computing Journal 93 (2020) 106342 (“ROD”) in view of Arcelli et al, EASIER: an Evolutionary Approach for multi-objective Software archItecturE Refactoring, 2018 IEEE International Conference on Software Architecture (“ARC”) in further view of Hein et al Interpretable policies for reinforcement learning by genetic programming” Engineering Applications of Artificial Intelligence Volume 76, November 2018, Pages 158-169 (“HEI”)
Priority
The application claims priority to the Application European Patent EP20202826.2 filed 10/20/2020. A 371 of PCT/EP2021/074918 with filing date0 9/10/2021. Foreign priority under 35 U.S.C. 119 (a)-(d) is acknowledged.
Information Disclosure Statement (IDS)
The information disclosure statement (IDS) submitted on 04/17/2023, 07/01/2025, 10/14/2025 is/are in compliance with the provisions of 37 CFR 1.97.
Notations, Abbreviations and Conventions used.
The number in the parenthesis, following next to a claim number, when used, is the number of the parent claim.
The following abbreviations are used:
BRI = Broadest Reasonable Interpretation
POSITA = Person of Ordinary Skill in The Art
101 - 35 USC § 101
102 or 103 = 35 USC § 102 or 35 USC § 103
(S1)/(S2A1)/(S2A2) (S2B) = Steps 1, 2AProng1 , 2AProng2, and 2B of the multi-step eligibility analysis in the Alice/Mayo framework
WURC = Well Understood, Routine, Conventional
{ } text from the reference
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to a configuration system which is interpreted in BRI to be a computer program. The claim’s scope is therefore software per se – which is non-statutory subject matter (MPEP 2106.03). One way to overcome this rejection, is to amend the claim to recite a non-transitory computer-readable medium. This will be the interpretation adopted for advancing prosecution. Claim 14, in view of the specification, is interpreted as being directed to a non-transitory computer readable medium.
Claims directed to an ineligible judicial exception.
Claims are analyzed under the Alice/Mayo framework to determine whether the claims are directed to an ineligible judicial exception. Recitation of judicial exceptions are highlighted in bold font. Paraphrased language, shown in italics, is used to simplify reference. Claims with similar limitations, although not verbatim identical, that share the same rationale under Alice/Mayo steps Step 1 (S1) and Steps 2 Prongs A1, A2 and B (S2A1, S2A2, S2B) are grouped. The analysis is performed on a representative claim of each group. An additional analysis is performed if any claim in the group includes additional limitations.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter, a judicial exception (abstract idea, mental process) without significantly more.
(S1) Prima facie, claims 1-15 are each directed to a statutory category of invention: process (Claims 1-12 directed to a method), machine (claims X-Y directed to an computer product) and manufacture (claims 13, 15 directed to a non-transitory computer readable medium). This is based on the interpretation of claim 13 directed to a NTCRM – see the “software per se” rejection above.
INDEPENDENT CLAIMS
Regarding claims 1, 13, 14, 15
(S2A1)
Claim 1, representative for claims 13, 14, 15, recites recite an abstract idea, shown in bold below:
a) reading in a predefined default configuration data set for the control device;
b) generating a plurality of test configuration data sets;
c) for a respective test configuration data set:
determining a deviation value quantifying a deviation from the default configuration data set, and determining a performance value quantifying a performance for controlling the technical system based on the respective test configuration data set
d) performing a Pareto optimization for the plurality of test configuration data sets, wherein the deviation as well as the performance are used as Pareto objective criteria, and
e)selecting a configuration data set resulting from the Pareto optimization for configuring the control device
According to the MPEP 2106.04 II. B guidance when multiple limitations reciting abstract ideas these should be combined for analysis as a single abstract idea. In broadest reasonable interpretation and in view of the specification the combination of abstract ideas in the bolded limitations claim recites a process aimed at: “determine configurations that are Pareto optimal in respect to deviation from default configuration and performance ”.
This is a combination that, under its broadest reasonable interpretation covers performance of limitations expressing observation, evaluation, judgement and decision-making. It can be practically performed mentally or manually by a human. These are Mental Processes – Concepts Performed in the Human Mind (MPEP § 2106.04(a)(2), subsection III. The combination also covers performance of limitations expressing mathematical concepts like [mathematical relationships, mathematical formulas or equations, mathematical calculations. These are Mathematical Concepts (see MPEP 2106.04(a)(2) subsection I) ). See https://en.wikipedia.org/wiki/Multi-objective_optimization “is an area of multiple-criteria decision making that is concerned with mathematical optimization problems”
Accordingly, claims 1, 13, 14, 15 recite an abstract idea.
(S2A2)
The identified abstract idea is not integrated into a practical application because the additional elements in the claims only amount to Mere Instructions to Apply the judicial Exception on a computer (MPEP 2106.05(f)), an Insignificant Extra-Solution Activity (MPEP 2106.05(g)), or to a general link to a particular technological environment or field of use (MPEP 2106.05(h).
The additional elements “computer-implemented” recite computing elements at a high level of generality, which is equivalent to instructions to implement the abstract idea “by a computer” or “on a computer” (used as a tool to implement the judicial exception (MPEP § 2106.05(f))
The additional claim elements also recite: reading a configuration [a], generating test configurations[b]. When considered individually, each amounts to nothing more than “Insignificant Extra-Solution (Pre-Solution and/or Post-Solution) Activity”, i.e. activities incidental to the primary process or product that are merely a nominal or tangential addition to the claims. Specifically, the claim elements in [a], [b] are considered pre-solution activity because they are mere gathering or pre-processing data/information in conjunction with the abstract idea, or post-solution activity because they are mere outputting or post-processing results from executing the abstract idea (see MPEP §2106.05(d); which the courts have identified did not integrate a judicial exception into a practical application.
The additional elements, taken individually or in combination, fail to integrate the recited judicial exception into a practical application when evaluated using the considerations in MPEP §§ 2106.04(d), 2106.05(a)-(c), (e)-(h) because these do not impose any meaningful limits on practicing the abstract idea, nor do they effect an improvement to any technology or technical field. Therefore, the claim remains directed to a judicial exception.
(S2B) Claims 1, 13-15 do not include additional elements, which individually or in combination amount to significantly more than the judicial exception. As analyzed in step S2A2 the additional elements recite Mere Instructions to Apply the judicial Exception on a computer (MPEP 2106.05(f)), and the Insignificant Extra (Pre-Solution and/or Post-Solution) Activity (MPEP 2106.05(g)), which for situations substantially similar to those here, these additional elements, including data gathering, data manipulation, and data transmission, data outputting recited at high level of generality were found by the courts to be Well-Understood, Routine and Conventional (see MPEP § 2106.05(d)(ll)).
When considered as a whole, with additional elements in an ordered combination, the additional elements in the claim only amount to instructions to apply the abstract idea on a computer and Insignificant Extra Activities. Additional elements elaborate on the identified abstract idea but do not practically or significantly alter how the identified abstract idea would be performed. Moreover, as noted above, there is nothing about the computing environment or the additional steps that is significant or meaningful to the underlying judicial exception because the identified abstract idea “ determine configurations that are Pareto optimal in respect to deviation from default configuration and performance ” could have been reasonably performed when provided with the relevant data and/or information. There is no inventive concept beyond the judicial exception, and thus the claim as a whole does not amount to significantly more than the judicial exception itself.
Therefore, it is concluded that claims 1, 13-15 are ineligible.
Dependent claims further recites as follows. As drafted the claims are all ineligible. However, underlined elements in claims 7, 8 and 11 could make the claims eligible with minor amendments, for example as suggested after the analysis.
2(1)
wherein data elements of the default configuration data set are selected, and in that the test configuration data sets are generated on a basis of the default configuration data set, wherein a change to the selected data elements is suppressed.
The claim elements provide further details and restrictions on data elements.
3(1)
wherein a Pareto front is determined by the Pareto optimization within the generated test configuration data sets, and in that a configuration data set is selected from the Pareto front for configuring the control device.
The claim elements provide further details and restrictions on the Pareto front.
4(1).
wherein the Pareto optimization is performed by means of a genetic optimization method, a method of genetic programming, a gradient-based optimization method, a stochastic gradient method, a particle swarm optimization method, a Metropolis optimization method, and/or another machine learning method.
The limitation reciting performing a Pareto optimization is no longer a mental process. At least some of the recited methods (e.g. gradient based) recite mathematical concepts. Even if it were considered not abstract, but an insignificant extra solution activity, it recites classical methods that are WURC.
5(1)
wherein new configuration data sets generated when performing the Pareto optimization are used as test configuration data sets
The claim elements provide further details and restrictions on the new configuration data sets.
6(5)
wherein the new configuration data sets are generated as part of performance-driven optimization.
The claim elements provide further details and restrictions on the new configuration data sets.
7(1)
wherein for determining the performance value for a respective test configuration data set, the technical system and/or a simulation model of the technical system is/are controlled on a basis of the respective test configuration data set and a resulting performance of the technical system is measured in the process.
As drafted (with the “OR”) the claim includes the possibility that the control is only of a simulation and not of a physical system. If it were drafted such that the control would certainly include a technical system, with or without the simulation, the step may be interpreted as a limitation that integrates the abstract idea into a practical application.
8(1)
wherein for determining the performance value for a respective test configuration data set, a deviation of a response behavior of the control device, configured with the respective test configuration data set, from a response behavior of the control device, configured with a performance-optimized configuration data set, is determined.
As drafted, the active step remains determining a deviation based on received information, where the deviation may be determined mentally and if quantitatively determined would in essence describe a difference (mathematical concept). If it were drafted for example to specify controlling the technical system and determining a response, controlling with the optimized configuration data set and determining a response, determining the deviation etc then the claim may be interpreted as reciting elements that integrate the abstract idea in a practical application.
9(5)
wherein the performance-optimized configuration data set is determined by means of a method of reinforcement learning.
Similar interpretation as for Claim 4. Reinforcement learning is a well known method. The limitation does not bring an improvement or significantly more.
10(1)
wherein for determining the deviation value for a respective test configuration data set, a deviation of a component representation of the respective test configuration data set from a component representation of the default configuration data set is determined.
The claim elements provide further details and restrictions on the new configuration data sets.
11(1) wherein in order to determine the deviation value for a respective test configuration data set, a deviation of a response behavior of the control device, configured with the respective test configuration data set, from a response behavior of the control device, configured with the default configuration data set-(L-), is determined.
Similar to claim 8.
12(1)
wherein the technical system is a traffic signal system, a turbine, a manufacturing system, a robot, a motor, another machine, another device or another system.
The claim elements provide further details on technical system and indicate field of use.
13 A configuration system for configuring a control device for a technical system, configured for carrying out the method according to claim 1.
14. A computer program product comprising a computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement a method
claim 1.
15. The computer-readable storage medium comprising a-the computer program product according to claim14.
Each of these claims continues to recite, and further reinforce/elaborate on the abstract idea in the parent claim. The analysis for each of them is similar to that of the parent claim. The bolded claim elements recite mental processes. The additional elements further recited by the claim are of the same nature as those identified in the parent claim, specifically Insignificant Extra (Pre-Solution and/or Post-Solution) Activity (MPEP 2106.05(g)) limitations of data gathering, data manipulation, and data transmission, data outputting recited at high level of generality were found by the courts to be Well-Understood, Routine and Conventional (see MPEP § 2106.05(d)(ll)). data manipulation and mere instructions to apply an exception ((MPEP 2106.05(f) ).
For each of the above claims, when considered individually or in combination, the additional elements do not provide any specific improvements and do not practically or significantly alter how the identified abstract idea would be performed. Therefore, these claim elements fail to integrate the judicial exception into a practical application. The claim is thus directed to a judicial exception.
For each of the above claims, when considered individually and in combination, the claim as a whole, the additional elements do not provide an inventive concept beyond the judicial exception, and thus the claim as a whole does not amount to significantly more than the judicial exception itself. Claims are thus found ineligible under 35 USC 101.
Elements of amendments that would make the claims eligible are included, for example as discussed in regards to claims 7, 8 and 11. For example, an amendment to recite controlling the technical system and measuring its behavior response, in the optimization loop would move the claim towards eligibility. Another modality would be to recite sufficient complexity in the processing steps that performing mentally would be impractical. In BRI, the limitations as recited, do not prevent performance in the mind. The dependent claims reciting various algorithms would not be practical to be performed in the mind, however those recite mathematical concepts, i.e. still abstract ideas.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
i. Determining the scope and contents of the prior art.
ii. Ascertaining the differences between the prior art and the claims at issue.
iii. Resolving the level of ordinary skill in the pertinent art.
iv. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims that share substantially similar limitations (even though not verbatim) are grouped and analyzed together; the analysis is done on the claim with most comprehensive limitations. The parenthesis following a claim number indicates the parent claim.
Claims 1 directed to method, 13 directed to a configuration system (interpreted in BRI as a computer) 14 and 15 as amended to recite statutory category under 101 , 15 -directed to computer-readable storage recite essentially similar limitations.
In BRI the independent claim 1 is interpreted as a method that takes an initial configuration/solution (setting, programming bits etc) that determines the behavior of a system, and searches around it, evaluating multiple alternative solutions to determine if these are better than the initial one, using a multi-objective search/optimization algorithmic steps, with objectives both the deviation from the initial configuration and the quality of the response of the configured system, the optimization using solutions from a Pareto front, from which one is selected at the end to control the system. The method described is a classic population-based multi-objective optimization using a Pareto front; improving system response behavior is commonly one of the objectives (maximizing a value function or minimizing the distance function to an ideal response). The deviation from initial configuration is also sometimes used as one of the objectives, especially in tuning applications, searching around a known solution, when a good/working solution is known, and improvements around that set point are possible, yet avoiding too large deviations that may affect stability, trustworthiness, robustness etc. The population search is often an evolutionary algorithm of some flavor (including for example genetic programming, genetic algorithms, evolutionary strategies etc). The limitations of dependent claims interpreted in BRI also refer to well known aspects, such as keeping fixed some bits of a configuration (chromosome in genetic/evolutionary representations) or allowing minimal and selective mutations (both corresponding to suppression of changes to some configuration data) using the population at next step based on the Pareto front ( based on the results of the multi-objective function). The applications have an extremely broad range, from technical systems to drug synthesis, business cases etc. Re deviation from a default/initial configuration/solution one should also mentioned that well known to a practitioner in the art are Taguchi loss functions, applied both in business and technical processes, in which one minimizes the deviation /distance from a default/initial configuration, and combinations of genetic searches (including with Pareto fronts) uses Taguchi loss function as one of the objective function functions. Genetic optimizations with Pareto fronts are well known for controllers. Also determining a solution (based on quality of response, i.e. performance) using reinforcement learning is a well known in the art. This preliminary background is provided to as context, and the claims are analyzed individually in the following.
Claims 1-7, 10-15 are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez-Molina et al, Multi-objective meta-heuristic optimization in intelligent control: A survey on the controller tuning problem, Applied Soft Computing Journal 93 (2020) 106342 (“ROD”) in view of Arcelli et al, EASIER: an Evolutionary Approach for multi-objective Software archItecturE Refactoring, 2018 IEEE International Conference on Software Architecture (“ARC”)
Re claims 1, 13, 14, 15 ROD teaches
reading in a predefined default configuration data set for the control device; {see at least 5.2.1. For many applications, a fine controller tuning is required. This necessity can be satisfied by performing a local search around a given controller configuration; .p12 right col - the generation of the initial population based on tuning methods from control engineering. See Table 8 for initial population choices e.g. [118] MaPSO External archive, corner archive with extremes of the Pareto front, constraint-handling based on penalization, initial population generated with stable closed-loop system configurations } reading predefined default configuration data is interpreted as (giving) a given controller configuration around which a tuning search is performed – the given controller configuration forms the initial population.
generating a plurality of test configuration data sets; {p 12 right col middle generate the next population; p13 right col middle At the end of each generation, the fittest solution between each original individual and its offspring forms the next population. } plurality of test configuration data sets interpreted as population of an evolutionary/genetic algorithm.
for a respective test configuration data set: {p 12 right col middle generate the next population } plurality of test configuration data sets interpreted as population of an evolutionary/genetic algorithm. P.13 Differential Evolution [53] like a GA, is a MOMHO based on the process of natural evolution. This algorithm includes a population of individuals (randomly initialized), which are evaluated to determine their fitness. .. At the end of each generation, the fittest solution between each original individual and its offspring forms the next population. The above process is repeated until a given stop condition is met. }
Candidate solutions corresponding to individuals of the population are evaluated.
determining a deviation value quantifying a deviation from the default configuration data set, and determining a performance value quantifying a performance for controlling the technical system based on the respective test configuration data set { p6 left col sec4. In this, a MOP is stated based on different performance criteria, and a MOMHO is used to find the controller parameters with the best trade-offs. Each parameter configuration must be tested through a dynamic simulation to measure the controller performance and select the most promising alternatives. After the best trade-offs are found, the decision-maker is responsible for choosing a single parameter configuration to be implanted in the real controller; Table 3 ; P12 left col As a reminder, the solution of a MOP is a Pareto optimal set P∗, that mapped in the objective function space, generates a true Pareto front PF∗ that contains all possible trade-offs
among objectives. In the MOMHOCTP, P∗ refers to all feasible controller parameter configurations, while PF∗ includes all their performance trade-offs (see Table 6).
PNG
media_image1.png
177
755
media_image1.png
Greyscale
} a performance value quantifying a performance for controlling the technical system based on the respective test configuration data set interpreted as Integral Absolute Error (as example, see fragment from Table3) which quantifies performance for controlling the system based on configuration of design variables such as PID gains.
selecting a configuration data set resulting from the Pareto optimization for configuring the control device {p6 left col, Section 4. Each parameter configuration must be tested through a dynamic simulation to measure the controller performance and select the most promising alternatives. After the best trade-offs are found, the decision-maker is responsible for choosing a single parameter configuration to be implanted in the real controller; p12 right col top 4.5. Multi -objective meta-heuristic optimizer The main goal of the tuning process is to find proper controller configurations that met all performance specifications established in the MOP. Then, the MOMHO in Fig. 7-D is responsible for solving the controller tuning MOP (Fig. 7-C) to find the best controller parameters….NSGA-II includes a nondominated sorting operator to determine the rank of each individual based on Pareto dominance.}
ROD thus teaches optimizing configurations of a controller based on improvement on controller performance. It teaches the population based search, stating with a given configuration. ROD does not explicitly disclose the use of a deviation from the default configuration set and use with performance value as objective criteria in the Pareto optimization. However, ARC teaches a multi-objective Pareto optimization method in which both improvement in performance and deviation from initial /default configuration are objective criteria:
determining a deviation value quantifying a deviation from the default configuration data set, and determining a performance value quantifying a performance for controlling the technical system based on the respective test configuration data set {ARC: Abstract In this paper we introduce EASIER (Evolutionary Approach for multi-objective Software archItecturE Refactoring), that is an approach for optimizing architecture refactoring based on performance and on the intensity of changes. Fig1, Fig 2; p108 left col bottom: After the (custom) generation of the initial population, solutions are evaluated according to a customized fitness function that, in EASIER, considers three objectives to optimize, namely ArchDist, PerfQand #PAs, which are defined in the following. PerfQ(to maximize). It represents a performance quality indicator aimed at quantifying the relative performance improvement induced by a refactoring w.r.t. an initial architecture; p. 109 left col, middle, ArchDist (to minimize). It quantifies the distance of an architectural alternative A from the initial one}
performing a Pareto optimization for the plurality of test configuration data sets,
wherein the deviation as well as the performance are used as Pareto objective criteria {ARC: Fig1, Fig 2; p108 left col bottom: solutions are evaluated according to a customized fitness function that, in EASIER, considers three objectives to optimize, namely ArchDist, PerfQand #PAs,... PerfQ(to maximize). It represents a performance quality indicator aimed at quantifying the relative performance improvement induced by a refactoring w.r.t. an initial architecture; p. 109 left col, middle, ArchDist (to minimize). It quantifies the distance of an architectural alternative A from the initial one p109 right col, middle- After epo iterations, the available solutions are once more compared each other and the set of non-dominated ones, namely the Pareto frontier, is returned.}
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of ROD and ARC . Both are in the area of Pareto multi-objective optimization of technical systems. One would have been motivated to in, in addition to system performance, use deviation from default configuration set, in order to obtain the advantage of tuning a given configuration favoring configurations that do not deviate too far, as further deviations introduce more unknowns that can decrease robustness but also further configuration may require more changes (in the process of reconfiguration itself) which come with additional cost.
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
2(1) Regarding claim 2(1) ROD/ARC teaches the limitations of the parent claim. ROD further teaches
wherein data elements of the default configuration data set are selected, and in that the test configuration data sets are generated on a basis of the default configuration data set, wherein a change to the selected data elements is suppressed.{ Determine the tunable controller parameters: No matter the selected controller structure, it has a set of parameters
that directly compromises the controller operation and the plant response. The designer must determine which of these parameters are used for tuning and which remain fixed to a predetermined value.}
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
3(1) Regarding claim 3(1) ROD/ARC teaches the limitations of the parent claim. ROD further teaches
wherein a Pareto front is determined by the Pareto optimization within the generated test configuration data sets, and in that a configuration data set is selected from the Pareto front for configuring the control device. {see at least Table 10 [131] IMOSA Generation mechanism based on orthogonal experimental solutions, Pareto based scoring function to
measure the performance of candidate solutions and exploit its neighborhood. P20 left col sec 5.1.7 For each of the reviewed works, the selected parameter configuration from the Pareto front approximation is validated in simulation. In 24% of the reviewed works, the tuned controller is also compared in simulation with controllers from state-of the- art tuning methods or other control structures. }
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
4(1). Regarding claim 4(1) ROD/ARC teaches the limitations of the parent claim. ROD further teaches
wherein the Pareto optimization is performed by means of a genetic optimization method, a method of genetic programming, a gradient-based optimization method, a stochastic gradient method, a particle swarm optimization method, a Metropolis optimization method, and/or another machine learning method. {p4 right col bottom - Some representative MOMHOs under this approach are the Multi-objective Genetic Algorithm (MOGA) [59], Non-dominated Sorting Genetic Algorithm II (NSGA-II) [60], Multi-objective Particle Swarm Optimization (MOPSO) [61] and Multi-objective Differential Evolution (MODE) [62].}
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
5(1) Regarding claim 5(1) ROD/ARC teaches the limitations of the parent claim. ROD further teaches
wherein new configuration data sets generated when performing the Pareto optimization are used as test configuration data sets {p4 2.4. Meta-heuristic search approaches Dominance-based: The search of the PFA is guided based on the Pareto dominance, i.e., Pareto optimality is used to find the best trade-off among candidate solutions. In this case, a solution that dominates another or a solution that is less dominated by a solution set is preferred and must persist. p12 right col Multi-objective Genetic Algorithms: 51% of works use variants of Multi-objective Genetic Algorithms. In genetic algorithms [52], the fitness of individuals in the population (usually randomly initialized at the beginning) is evaluated. Next, the best individuals are selected according to their fitness. Then, the best individuals have better chances to generate the next population through crossover and mutation operations. The process is repeated until a stop condition is satisfied. Among the approaches found in Table 7, the Non-dominated Sorting Genetic Algorithm II (NSGA-II) [167] looks like a popular choice. This state-of-the-art MOMHO has exhibited a high performance [168]. NSGA-II includes a nondominated sorting operator to determine the rank of each individual based on Pareto dominance, crowding of individuals and feasibility. Although a considerable number of works use the original NSGA-II, some improvements have been proposed to solve the controller tuning MOP…. Another remarkable feature found in some of these MOMHOs, is the inclusion of elitism through the use of external archives. The above is used to retain the nondominated solutions found on each iteration.). used as test configuration data sets is interpreted here as must persist (in further generation of search). Also the iterative process further detailed, and NSGA-2 determining rank of individual based on Pareto dominance.
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
6(5) Regarding claim 6(5) ROD/ARC teaches the limitations of the parent claim. ROD further teaches
wherein the new configuration data sets are generated as part of performance-driven optimization. { p6 left col Fig. 7 shows the general MOMHOCTP. In this, a MOP is stated based on different performance criteria, and a MOMHO is used to find the controller parameters with the best trade-offs. Each parameter configuration must be tested through a dynamic simulation
to measure the controller performance and select the most promising alternatives.}
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
7(1) Regarding claim 7(1) ROD/ARC teaches the limitations of the parent claim. ROD further teaches
wherein for determining the performance value for a respective test configuration data set, the technical system and/or a simulation model of the technical system is/are controlled on a basis of the respective test configuration data set and a resulting performance of the technical system is measured in the process. {{ p6 left col Fig. 7 shows the general MOMHOCTP. In this, a MOP is stated based on different performance criteria, and a MOMHO is used to find the controller parameters with the best trade-offs. Each parameter configuration must be tested through a dynamic simulation
to measure the controller performance and select the most promising alternatives.}
PNG
media_image2.png
250
354
media_image2.png
Greyscale
Fragment from Fig 7
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
11(1) Regarding claim 12(1) ROD/ARC teaches the limitations of the parent claim. ARC further teaches:
wherein in order to determine the deviation value for a respective test configuration data set, a deviation of a response behavior of the control device, configured with the respective test configuration data set, from a response behavior of the control device, configured with the default configuration data set, is determined. {p108 PerfQ (to maximize). It represents a performance quality indicator aimed at quantifying the relative performance
improvement induced by a refactoring w.r.t. an initial architecture}
default configuration interpreted as the initial architecture; deviation of the response behavior interpreted as relative performance improvement.
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of ROD and ARC . Both are in the area of Pareto multi-objective optimization of technical systems. One would have been motivated to do so, in order to obtain the advantage of evaluating progress comparted to the starting point so that solutions that make a progress are rewarded..
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
12(1) Regarding claim 12(1) ROD/ARC teaches the limitations of the parent claim. ROD further teaches
wherein the technical system is a traffic signal system, a turbine, a manufacturing system, a robot, a motor, another machine, another device or another system. { p6 right col top - It is observed that about of 70% of the works use linear models, and most of them correspond to benchmarked or proposed test plants. The rest of the adopted linear models are related to well-known plants such as voltage regulators, power generation systems, gas production systems, distillation systems, and aerial vehicles. On the other hand, the remaining 30% of works uses nonlinear models related to vehicles, robotic manipulators, and mechanical
systems. }
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
10(1) Regarding claim 10(1) ROD/ARC teaches the limitations of the parent claim. ARC further teaches
wherein for determining the deviation value for a respective test configuration data set, a deviation of a component representation of the respective test configuration data set from a component representation of the default configuration data set is determined. { ArchDist (to minimize). It quantifies the distance of an architectural alternative A from the initial one, in terms of intensity of refactoring changes. The distance of A from an initial one is defined as the sum of the distance induced by each RefactoringAction ai in the corresponding
genome.} A deviation of a component representation interpreted as the RefactoringAction in the corresponding genome.
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of ROD and ARC . Both are in the area of Pareto multi-objective optimization of technical systems. One would have been motivated to do so, in order to obtain the advantage of a fine level of granularity in observing deviations – such as in the components of a vector that represents the configuration, a finer granularity offering more flexibility in estimating deviation in specific directions (Ad vector components) .
Accordingly, the claimed subject matter would have been obvious over RDO/ARC.
Claims 8(1), 9(5) are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez-Molina et al, Multi-objective meta-heuristic optimization in intelligent control: A survey on the controller tuning problem, Applied Soft Computing Journal 93 (2020) 106342 (“ROD”) in view of Arcelli et al, EASIER: an Evolutionary Approach for multi-objective Software archItecturE Refactoring, 2018 IEEE International Conference on Software Architecture (“ARC”) in further view of Hein et al Interpretable policies for reinforcement learning by genetic programming” Engineering Applications of Artificial Intelligence Volume 76, November 2018, Pages 158-169 (“HEI”)
8(1) Regarding claim 8(1) ROD/ARC teaches the limitations of the parent claim. ROD/ARC does not teach however HEI teaches
wherein for determining the performance value for a respective test configuration data set, a deviation of a response behavior of the control device, configured with the respective test configuration data set, from a response behavior of the control device, configured with a performance-optimized configuration data set, is determined. { p161 Genetic programming reinforcement learning (GPRL) right col middle -. Given that GPRL is able to find rather short (non-complex) equations, we expect to reveal substantial knowledge about underlying coherencies between available state variables and well-performing control policies with respect to a certain RL problem. Fig 2. In contrast to both other policy training approaches, the GP regression policy mimics an already existing policy by learning to minimize an error with respect to the existing policy’s action. p166 right col, 3rd para The Pareto front results of the GP regression experiments are presented in Fig. 7(b). Here, the fitness value driving the GP optimization was the regression error with respect to the NN policy}
Deviation interpreted as error response behavior of the control device configured with a performance-optimized configuration data set is interpreted as well-performing policy.
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of ROD/ARC with HEI, researches in the area of multi-objective optimization of technical systems. One would have been motivated to do so, in order to obtain the advantage of using a fitness function that measures the approach to a very good target solution.
Accordingly, the claimed subject matter would have been obvious over RDO/ARC/HEI.
9(5) Regarding claim 9(5) ROD/ARC teaches the limitations of the parent claim. ROD/ARC does not teach however HEI teaches
wherein the performance-optimized configuration data set is determined by means of a method of reinforcement learning.{ The performance of GPRL is compared to a rather straightforward approach, which utilizes GP to conduct symbolic regression on a data
set D^ generated by a well-performing but non-interpretable RL policy.}
In addition, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the invention, to combine the teachings of ROD/ARC with HEI, researches in the area of multi-objective optimization of technical systems. One would have been motivated to do so, in order to obtain the advantage of using a validated method to obtained a desired behavior which would act as target behavioral solution.
Accordingly, the claimed subject matter would have been obvious over RDO/ARC/HEI.
Prior art made of record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 20100057410 A1
Cai Q. Reinforcement Learning Driven Heuristic Optimization, DRL4KDD-2019 arXiv:1906.06639, 2019
Reynoso-Meza et al Preference driven multi-objective optimization design procedure for industrial controller tuning
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADRIAN STOICA whose telephone number is (571) 272-3428. The examiner can normally be reached Monday to Friday, 9 a.m. -5 p.m. PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached on (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.S./Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights