DETAILED ACTION
This nonfinal action is in response to application 18/778,714 filed on 07/19/2024 with priority to provisional application 63/528,180 filed on 07/21/2023.
Claims 1-20 are pending in the application. Claims 1 and 11 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected because the claimed invention is directed to non- statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because they are directed towards software per se.
As recited in MPEP § 2106.03(I), claims reciting “products that do not have a physical or tangible form, such as information (often referred to as "data per se") or a computer program per se (often referred to as "software per se") when claimed as a product without any structural recitations” are not directed to any of the statutory categories (i.e., are directed to non-statutory subject matter).
Claim 1 recites, inter alia, “A system for generating realistic defective data samples comprising:…a scenario generator unit….a prompt generator unit…a simulator unit...a digital twins data bank unit…a generative machine learning unit”. The recited functional “units”, based on their plain meaning, are interpretable as encompassing purely software implementations of the recited functionalities. While the specification does state that “The scenario generator unit, generative machine learning unit, digital twins data bank unit, prompt generator unit, and simulator unit may take on the form of a digital hardware unit or a number of different digital hardware units or one or more digital signal processor units, one or more processors, one or more microprocessors, and/or digital logic, such as discrete digital logic, programmable logic, or other circuitry” [¶ 0026], further description (“Furthermore, as will be apparent to one of ordinary skill in the relevant art, the modules, routines, features, attributes, methodologies and other aspects of the present invention can be implemented as software, hardware, firmware or any combination of the three. Also, wherever a component, an example of which is a module, of the present invention is implemented as software, the component can be implemented as a standalone program, as part of a larger program, as a plurality of separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and/or in every and any other way known now or in the future to those of ordinary skill in the art of computer programming” [¶ 0046]) fails to limit the recited units to a hardware implementation. The remaining limitations in claim 1, and further limitations in dependent claims 2-10, also do not recite any further structural elements that would provide a physical or tangible form to the claimed subject matter, and are likewise also interpretable as reciting purely software elements within the claimed system.
As such, under a broadest reasonable interpretation, claims 1-10 are directed towards software per se (i.e., non-statutory subject matter).
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
Independent Claims (Claim 1, Claim 11):
Step 1: Claim 1 could be amended to fall within a statutory category (e.g., be drawn to an apparatus), and claim 11 is drawn to a method. Therefore, each of these claims either falls under one of the four categories of statutory subject matter (process/method, machine/apparatus, manufacture/product, or composition of matter) or could be amended to fall within a statutory category.
Step 2A Prong 1: Claims 1 and 11 each recite a judicially recognized exception of an abstract idea.
Claim 1 recites, inter alia:
A system for generating realistic defective data samples comprising: generat[ing] defect scenario data; generat[ing] prompts based on the defect scenario data; generat[ing] simulation data based on the defect scenario data and digital twins data; and generat[ing] a realistic defective data sample based on the simulation data and the prompts – These limitations recite a series of data observation and processing steps that amount to brainstorming a possible defect scenario, logically applying the scenario to a real-world object, and using reasoning to simulate, or predict, how the defect would realistically manifest for the given object. They therefore recites a process of evaluation that a human could reasonably perform in the mind or using pen and paper.
Claim 11 recites substantially similar abstract idea limitations to those recited in claim 1, and therefore recites the same judicial exception.
Step 2A Prong 2: The following additional elements recited in claims 1 and 11 also do not integrate the recited judicial exceptions into a practical application.
Claim 1 additionally recites:
A system comprising: a scenario generator unit configured to [generate]; a prompt generator unit configured to [generate]; a simulator unit configured to [generate]; a generative machine learning unit configured to [generate] – These limitations amount to mere instructions to implement abstract procedural steps on a computer or computer components.
a digital twins data bank unit configured to store digital representations of real-world entities – This limitation amounts to a tangential intermediate step of simply storing data that is utilized for analysis, and therefore recites insignificant extra-solution activity.
Claim 11 recites substantially similar abstract idea limitations to those recited in claim 1, and therefore does not integrate the recited judicial exceptions into a practical application.
Step 2B: The additional elements recited in claims 1 and 11, viewed individually or as an ordered combination, do not provide an inventive concept or otherwise amount to significantly more than the recited abstract ideas themselves.
Claim 1 additionally recites:
A system comprising: a scenario generator unit configured to [generate]; a prompt generator unit configured to [generate]; a simulator unit configured to [generate]; a generative machine learning unit configured to [generate] – Mere instructions to implement abstract procedural steps on a computer or computer components do not provide an inventive concept or significantly more to the recited abstract idea.
a digital twins data bank unit configured to store digital representations of real-world entities – Storing data in memory is well-understood, routine, and conventional activity (see MPEP § 2106.05(d); “Storing and retrieving information in memory”) and therefore does not provide an inventive concept or significantly more to the recited abstract idea.
Claim 11 recites substantially similar abstract idea limitations to those recited in claim 1, and therefore does not provide an inventive concept or significantly more to the recited abstract idea.
As such, claims 1 and 11 are not patent eligible.
Dependent Claims (Claims 2-10, Claims 12-20)
Dependent claims 2-10 and 12-20 narrow the scope of independent claims 1 and 11, and likewise narrow the recited judicial exceptions. They recite abstract idea limitations that are similar to those recited within the independent claims (i.e., mental processes and/or mathematical concepts), and thereby merely expand on the already recited exceptions. The dependent claims also do not recite any further additional elements that successfully integrate the recited judicial exceptions into a practical application or provide significantly more than the recited abstract ideas themselves. Consequently, claims 2-10 and 12-20 are also rejected under 35 U.S.C. 101.
Step 1: Claims 2-10 could be amended to fall within a statutory category (e.g., be drawn to an apparatus), and claims 12-20 are drawn to a method. Therefore, each of these claims either falls under one of the four categories of statutory subject matter (process/method, machine/apparatus, manufacture/product, or composition of matter) or could be amended to fall within a statutory category.
Step 2A Prong 1: Claims 2-10 and 12-20 each recite a judicially recognized exception of an abstract idea.
Claim 2 recites the same judicial exception as claim 1.
Claim 3 recites the same judicial exception as claim 1.
Claim 4 recites the same judicial exception as claim 1.
Claim 5 recites the same judicial exception as claim 1.
Claim 6 recites the same judicial exception as claim 5.
Claim 7 recites the same judicial exception as claim 1.
Claim 8 recites, inter alia:
generat[ing] prompts based on historical defect data, allowing for the simulation of scenarios that replicate past occurrences – This limitation amounts to considering past observations when brainstorming or refining defect scenarios, and therefore recites a process of evaluation that a human could reasonably perform in the mind or using pen and paper.
Claim 9 recites, inter alia:
run[ning] multiple simulations with varying parameters and conditions to generate a diverse range of simulation data for different defect scenarios – This limitation amounts to considering a variety of extenuating factors when simulating, or predicting, how a defect would realistically manifest for a given object, and therefore recites a process of evaluation that a human could reasonably perform in the mind or using pen and paper.
Claim 10 recites, inter alia:
evaluat[ing] the realism and relevance of the generated defective data samples using an evaluation that provides feedback to refine subsequent simulations – This limitation amounts to using reasoning to evaluate and provide feedback on a predicted result, and therefore recites a process of evaluation that a human could reasonably perform in the mind or using pen and paper.
Claim 12 recites the same judicial exception as claim 11.
Claim 13 recites the same judicial exception as claim 11.
Claim 14 recites the same judicial exception as claim 11.
Claim 15 recites the same judicial exception as claim 11.
Claim 16 recites the same judicial exception as claim 15.
Claim 17 recites the same judicial exception as claim 11.
Claim 18 recites, inter alia:
wherein the prompts are dynamically adjusted based on real-time feedback from the simulation data to create more realistic and varied defective data scenarios – This limitation amounts to considering feedback when refining brainstormed defect scenarios, and therefore recites a process of evaluation that a human could reasonably perform in the mind or using pen and paper
Claim 19 recites, inter alia:
incorporate environmental factors such as lighting, camera position, and vibration in the simulation data, to mimic real-world conditions that may contribute to the defects appearance – This limitation amounts to considering a variety of possible extenuating (e.g., environmental) factors when simulating, or predicting, how a defect would realistically manifest for a given object, and therefore recites a process of evaluation that a human could reasonably perform in the mind or using pen and paper.
Claim 20 recites, inter alia:
compares the generated realistic defective data sample against a repository of known defect patterns to assess the accuracy and reliability of the generated data – This limitation amounts to using reasoning to evaluate and provide feedback on a predicted result via comparison to prior known examples, and therefore recites a process of evaluation that a human could reasonably perform in the mind or using pen and paper.
Step 2A Prong 2: Claims 8-9 and 18-19 do not recite any further additional elements besides those recited in the independent claims, and the following additional elements recited in claims 2-7, 10, 12-17 and 20 also do not integrate the recited judicial exceptions into a practical application.
Claim 2 additionally recites:
generat[ing] defect scenario data based on user input – This limitation does no more than specify a source from which data is provided, and therefore recites insignificant extra-solution activity.
Claim 3 additionally recites:
generat[ing] defect scenario data automatically without user input – This limitation amounts to mere instructions to implement an abstract idea on a computer or computer components.
Claim 4 additionally recites:
wherein the defect scenario data is in the form of natural language or a set of numbers indicating a set of scenario parameters – This limitation amounts to merely specifying data as being in multiple possible formats/types, and therefore recites insignificant extra-solution activity.
Claim 5 additionally recites:
wherein the digital twins data bank unit stores digital representations in forms including but not limited to images, videos, volumetric scans, 3D models, point clouds, or neural networks – This limitation amounts to a tangential intermediary step of specifying data storage types, and therefore recites insignificant extra-solution activity.
Claim 6 additionally recites:
wherein the digital twins data bank unit further includes metadata associated with each digital representation, including information such as manufacturing date, materials, and past maintenance history – This limitation further amounts to a tangential step specifying a means by which data is stored, and therefore recites insignificant extra-solution activity.
Claim 7 additionally recites:
wherein the generative machine learning unit is a latent diffusion neural network model or a generative adversarial network – This limitation does no more than generally link a judicial exception to application on a neural model without any significant details of implementation, and therefore amounts to mere instructions to “apply” an exception.
Claim 10 additionally recites:
[evaluate] using an evaluation module – This limitation amounts to mere instructions to implement an abstract idea on a computer or computer components.
Claims 12-15 recite substantially similar additional elements to those recited in claims 2-5, and therefore also do not integrate the recited judicial exceptions into a practical application.
Claim 16 additionally recites:
wherein the digital twins data bank unit is further configured to periodically update the stored digital representations with real-world performance data, to ensure that the digital twins accurately reflect the current state of the real-world entities – This limitation amounts to a tangential intermediary step specifying the means by which data is continually stored and processed, and therefore recites insignificant extra-solution activity.
Claim 17 recites substantially similar additional elements to those recited in claims 7, and therefore also does not integrate the recited judicial exceptions into a practical application.
Claim 20 additionally recites:
wherein the generative machine learning unit includes a validation component [that compares] – This limitation amounts to mere instructions to implement an abstract idea on a computer or computer components.
Step 2B: The additional elements recited in claims 2-7, 10, 12-17 and 20, viewed individually or as an ordered combination, do not provide an inventive concept or otherwise amount to significantly more than the recited abstract ideas themselves.
Claim 2 additionally recites:
generat[ing] defect scenario data based on user input – Receiving data is well-understood, routine, and conventional activity (see MPEP § 2106.05(d); “Storing and retrieving information in memory”, “Receiving or transmitting data over a network”) and therefore does not provide an inventive concept or significantly more to the recited abstract idea.
Claim 3 additionally recites:
generat[ing] defect scenario data automatically without user input – Mere instructions to implement an abstract idea on a computer or computer components do not provide an inventive concept or significantly more to the recited abstract idea.
Claim 4 additionally recites:
wherein the defect scenario data is in the form of natural language or a set of numbers indicating a set of scenario parameters – Receiving and transmitting data is well-understood, routine, and conventional activity (see MPEP § 2106.05(d); “Receiving or transmitting data over a network”) and therefore does not provide an inventive concept or significantly more to the recited abstract idea.
Claim 5 additionally recites:
wherein the digital twins data bank unit stores digital representations in forms including but not limited to images, videos, volumetric scans, 3D models, point clouds, or neural networks – Storing data is well-understood, routine, and conventional activity (see MPEP § 2106.05(d); “Storing and retrieving information in memory”) and therefore does not provide an inventive concept or significantly more to the recited abstract idea.
Claim 6 additionally recites:
wherein the digital twins data bank unit further includes metadata associated with each digital representation, including information such as manufacturing date, materials, and past maintenance history – Storing data is well-understood, routine, and conventional activity (see MPEP § 2106.05(d); “Storing and retrieving information in memory”) and therefore does not provide an inventive concept or significantly more to the recited abstract idea.
Claim 7 additionally recites:
wherein the generative machine learning unit is a latent diffusion neural network model or a generative adversarial network – Mere instructions to “apply” an exception on a neural model without any significant details of implementation do not provide an inventive concept or significantly more to the recited abstract idea.
Claim 10 additionally recites:
[evaluate] using an evaluation module – Mere instructions to implement an abstract idea on a computer or computer components do not provide an inventive concept or significantly more to the recited abstract idea.
Claims 12-15 recite substantially similar additional elements to those recited in claims 2-5, and therefore also do not provide an inventive concept or significantly more to the recited abstract idea.
Claim 16 additionally recites:
wherein the digital twins data bank unit is further configured to periodically update the stored digital representations with real-world performance data, to ensure that the digital twins accurately reflect the current state of the real-world entities – Receiving and transmitting data is well-understood, routine, and conventional activity (see MPEP § 2106.05(d); “Receiving or transmitting data over a network”) and therefore does not provide an inventive concept or significantly more to the recited abstract idea.
Claim 17 recites substantially similar additional elements to those recited in claims 7, and therefore also does not provide an inventive concept or significantly more to the recited abstract idea.
Claim 20 additionally recites:
wherein the generative machine learning unit includes a validation component [that compares] – Mere instructions to implement an abstract idea on a computer or computer components do not provide an inventive concept or significantly more to the recited abstract idea.
As such, claims 2-10 and 12-20 also are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-17 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (“Deep Transfer Fault Diagnosis Using Digital Twin and Generative Adversarial Network”, available online 10/19/2021), hereinafter Wang, in view of Mirza et al. (“Conditional Generative Adversarial Nets”, available arXiv 6 Nov 2014), hereinafter Mirza.
Regarding claim 1, Wang teaches A system for generating realistic defective data samples (“Instead of directly training diagnosis models on hard-to-collected sensory signals, training on simulated data generated from digital twin is feasible since simulated data with annotation is easy to access. However, simulated samples are often not realistic enough and domain gap exists between simulated and real sensory samples. The fault diagnosis model may lack of generalization if the model is trained only on simulated samples. In this paper, we propose a deep transfer fault diagnosis approach which generalizes fault diagnosis knowledge from digital twin to perform fault diagnosis task in real-world. First, a digital twin model with capability of bearing dynamics is composed using numerical simulation method, from which the simulated faulty samples are generated under various faulty depth and operation conditions. Second, a generative adversarial network is utilized to perform domain adaptation and eliminate the domain gap. Specifically, the generator component conditioned on simulated samples as well as random noise and produces target-like samples as realistic as possible to fool the discriminator” [Wang Abstract]; “The framework consists of three steps, namely constructing the digital twin model, generating the simulating signals and training a generative domain adaptation network” [Wang page 188 Models]) comprising:
a scenario generator unit configured to generate defect scenario data; (“Suppose there is a scratch fault existing in the inner raceway or the outer raceway. As shown in Fig. 3 the fault width is wd and the running speed of rollers is v. An additional deformation
PNG
media_image1.png
37
41
media_image1.png
Greyscale
will be generated once the roller runs into the faulty area…where wd is the fault width” [Wang pages 188-189 Construct the digital twin model]; “The test rig is shown in Fig. 4., and the characteristics of bearings in the test rig are shown in Table I” [Wang page 189 Generating simulation signals]; see TABLE I. Bearing Properties of CWRU including Type, Ball Number, Pitch Diameter, Ball Diameter, Contact Angle, Mass of Outer Ring [Wang page 189]; In the digital twin model construction phase, a defect and scenario are respectively determined, and defect-specific (e.g., fault width) parameters, i.e., defect scenario data, and scenario-specific (e.g., test rig bearing characteristics) parameters, i.e., digital twins data, are generated prior to the generation of simulating signals)
a simulator unit configured to generate simulation data based on the defect scenario data and digital twins data; (“The inner and outer race fault signals as well as healthy signals of bearings are simulated under working conditions 1, 2, and 3. All the faults have dept 0.011 inches but different fault diameters of 0.007, 0.014, and 0.021 inches. The simulated vibration acceleration signals in x-axis for CWRU dataset are shown in Fig. 5” [Wang pages 189-190 Generating simulation signals]; see Fig. 5 [Wang page 190] depicting simulated sample)
a digital twins data bank unit configured to store digital representations of real-world entities; (“The widely used CWRU dataset is chosen to verify the simulation model and to generate the missing faulty bearing signals” [Wang page 189 Generating simulation signals]; The recited dataset is used for storing digital twin representations (e.g., test rig) and scenario-specific parameters (e.g., bearing properties)) and
a generative machine learning unit configured to generate a realistic defective data sample based on the simulation data (“To handle the issues of large domain gap and faulty samples missing in target domain, we propose a domain adaptation method utilizing the generative adversarial network to generate realistic samples from simulation samples while keeping the annotation information of simulation samples” [Wang page 190 Generating synthetic signals and perform domain adaptation]).
However, Wang does not expressly teach a prompt generator unit configured to generate prompts based on the defect scenario data, and generating a realistic defective data sample based on the prompts.
In the same field of endeavor, Mirza teaches a generative adversarial framework (“In this work we introduce the conditional version of generative adversarial nets, which can be constructed by simply feeding the data, y, we wish to condition on to both the generator and discriminator. We show that this model can generate MNIST digits conditioned on class labels. We also illustrate how this model could be used to learn a multi-modal model, and provide preliminary examples of an application to image tagging in which we demonstrate how this approach can generate descriptive tags which are not part of training labels” [Mirza Abstract]) that comprises a prompt generator unit configured to generate prompts based on the defect scenario data, (“One way to help address the first issue is to leverage additional information from other modalities: for instance, by using natural language corpora to learn a vector representation for labels in which geometric relations are semantically meaningful” [Mirza page 2 Multi-modal learning for Image Labelling]; “We trained a conditional adversarial net on MNIST images conditioned on their class labels, encoded as one-hot vectors” [Mirza page 3 Unimodal]; From initial data labels, vector representations can be generated and leveraged as additional information by the generative network) and generating a realistic samples based on the prompts (“Generative adversarial nets can be extended to a conditional model if both the generator and discriminator are conditioned on some extra information y. y could be any kind of auxiliary information, such as class labels or data from other modalities. We can perform the conditioning by feeding y into the both the discriminator and generator as additional input layer” [Mirza page 2 Conditional Adversarial Nets]; In addition to taking initial data as input, the generated vectors (i.e., prompts) can be leveraged as additional auxiliary input for the adversarial network).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated a prompt generator unit configured to generate prompts based on the defect scenario data, and generating a realistic defective data sample based on the prompts as taught by Mirza into Wang because they are both directed towards generative adversarial frameworks. Incorporating the teachings of Mirza would support leveraging of additional information provided by source sample annotations by the generative adversarial network [Wang page 190 Generating synthetic signals and perform domain adaptation], therefore further optimizing realism of generated samples.
Regarding claim 2, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the scenario generator unit generates defect scenario data based on user input (“The widely used CWRU dataset is chosen to verify the simulation model and to generate the missing faulty bearing signals” [Wang page 189 Generating simulation signals]; Scenario data can implicitly be manually selected from CWRU dataset, e.g., via the demonstrated example)
Regarding claim 3, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the scenario generator unit generates defect scenario data automatically without user input. (“The widely used CWRU dataset is chosen to verify the simulation model and to generate the missing faulty bearing signals” [Wang page 189 Generating simulation signals]; Scenario data can implicitly be selected from CWRU dataset automatically for testing, i.e., without human intervention)
Regarding claim 4, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the defect scenario data is in the form of natural language or a set of numbers indicating a set of scenario parameters ([Wang page 189] as detailed in claim 1 above; Scenario data comprises defect-specific (e.g., fault width) numerical parameters)
Regarding claim 5, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the digital twins data bank unit stores digital representations in forms including but not limited to images, videos, volumetric scans, 3D models, point clouds, or neural networks. (“The widely used CWRU dataset is chosen to verify the simulation model and to generate the missing faulty bearing signals” [Wang page 189 Generating simulation signals]; see Fig. 4 – image depiction of physical test rig apparatus [Wang page 189])
Regarding claim 6, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the digital twins data bank unit further includes metadata associated with each digital representation, including information such as manufacturing date, materials, and past maintenance history (see Table I. Bearing Properties of CWRU including bearing type, ball number, pitch diameter, contact angle, outer ring mass (i.e., manufacturing/materials information) associated with physical representation (Fig. 4) [Wang page 189])
Regarding claim 7, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the generative machine learning unit is a latent diffusion neural network model or a generative adversarial network (“To handle the issues of large domain gap and faulty samples missing in target domain, we propose a domain adaptation method utilizing the generative adversarial network to generate realistic samples from simulation samples while keeping the annotation information of simulation samples” [Wang page 190 Generating synthetic signals and perform domain adaptation]).
Regarding claim 8, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Mirza further teaches wherein the prompt generator unit is further configured to generate prompts from provided data (e.g., defect data) ([Mirza page 2 Multi-modal learning for Image Labelling] and ” [Mirza page 3 Unimodal] as detailed above). Wang further teaches generating signals based on historical defect data, allowing for the simulation of scenarios that replicate past occurrences (“The widely used CWRU dataset is chosen to verify the simulation model and to generate the missing faulty bearing signals” [Wang page 189 Generating simulation signals]; “We evaluate our method for bearing fault diagnosis task on the CWRU dataset. The experimental dataset is composed of simulation data and real sensory data” [Wang page 191 Dataset]).
Regarding claim 9, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the simulator unit is configured to run multiple simulations with varying parameters and conditions to generate a diverse range of simulation data for different defect scenarios. (“The inner and outer race fault signals as well as healthy signals of bearings are simulated under working conditions 1, 2, and 3. All the faults have dept 0.011 inches but different fault diameters of 0.007, 0.014, and 0.021 inches” [Wang pages 189-190 Generating simulation signals]).
Regarding claim 10, the combination of Wang and Mirza teaches the limitations of parent claim 1, and Wang further teaches wherein the generative machine learning unit is further configured to evaluate the realism and relevance of the generated defective data samples using an evaluation module that provides feedback to refine subsequent simulations (“The discriminator is trained to distinguish between synthetic samples from generator and real samples from target domain. The adversarial training is motivated between the generator network and discriminator network” [Wang page 190 Generating synthetic signals and perform domain adaptation]).
Regarding claims 11-15, they are method claims that substantially correspond to the apparatuses of claims 1-5. Consequently, they are rejected for the same reasons.
Regarding claim 16, the combination of Wang and Mirza teaches the limitations of parent claim 15, and Wang further teaches wherein the digital twins data bank unit is further configured to periodically update the stored digital representations with real-world performance data, to ensure that the digital twins accurately reflect the current state of the real-world entities (“Wang et al. [20] proposed a digital twin method for rotating machinery fault diagnosis. The virtual object established by finite element method is updated to reflect the status of the physical system using parameter sensitive analysis” [Wang page 187 Related Work]; It is suggested that digital twin data can be periodically updated to reflect state of its real-world counterpart).
Regarding claim 17, it is a method claim that substantially corresponds to the apparatus of claim 7. Consequently, it is rejected for the same reasons.
Regarding claim 19, the combination of Wang and Mirza teaches the limitations of parent claim 11, and Wang further teaches wherein the simulator unit is further configured to incorporate environmental factors such as lighting, camera position, and vibration in the simulation data, to mimic real-world conditions that may contribute to the defects appearance (see, e.g., rotating speed
PNG
media_image2.png
32
26
media_image2.png
Greyscale
in equation (2) and radial force
PNG
media_image3.png
40
27
media_image3.png
Greyscale
in equation (9) [Wang page 189])
Regarding claim 20, the combination of Wang and Mirza teaches the limitations of parent claim 11, and Wang further teaches wherein the generative machine learning unit includes a validation component that compares the generated realistic defective data sample against a repository of known defect patterns to assess the accuracy and reliability of the generated data (“The discriminator is trained to distinguish between synthetic samples from generator and real samples from target domain. The adversarial training is motivated between the generator network and discriminator network…The discriminator network D takes the generated samples G(si) and target samples ti as inputs, and outputs the probability of which domain the samples come from” [Wang page 190 Generating synthetic signals and perform domain adaptation.)
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Wang and Mirza, as applied to claim 11 above, further in view of Louppe et al. (“Adversarial Variational Optimization of Non-Differentiable Simulators”, available arXiv 04/16/2020), hereinafter Louppe.
Regarding claim 18, the combination of Wang and Mirza teaches the limitations of parent claim 11, and Mirza further teaches prompts generated by the prompt generator, as detailed above.
However, the combination does not expressly teach wherein the prompts generated by the prompt generator unit are dynamically adjusted based on real-time feedback from the simulation data to create more realistic and varied defective data scenarios.
In the same field of endeavor, Louppe teaches a generative adversarial learning framework (“We introduce Adversarial Variational Optimization (AVO), a likelihood-free inference algorithm for fitting a non-differentiable generative model incorporating ideas from generative adversarial networks, variational optimization and empirical Bayes. We adapt the training procedure of generative adversarial networks by replacing the differentiable generative network with a domain-specific simulator. We solve the resulting non-differentiable minimax problem by minimizing variational upper bounds of the two adversarial objectives” [Louppe Abstract]) that dynamically adjust[s] generated parameters based on real-time feedback from simulation data to create more realistic and varied data scenarios (“As for many likelihood-free inference algorithms, AVO is intimately tied to a class of algorithms that can be framed as density estimation-by-comparison, as reviewed in (Mohamed and Lakshminarayanan, 2016). In most cases, these inference algorithms are formulated as an iterative two-step process where the model distribution is first compared to the true data distribution and then updated to make it more comparable to the latter” [Louppe page 8 Related Work]; “Given a function f to minimize, these techniques are based on the observation that
PNG
media_image4.png
45
310
media_image4.png
Greyscale
where
PNG
media_image5.png
22
61
media_image5.png
Greyscale
is a proposal distribution with parameters
PNG
media_image6.png
22
23
media_image6.png
Greyscale
over input values θ” [Louppe page 2 Variational optimization]; “We can view the optimization of
PNG
media_image7.png
27
53
media_image7.png
Greyscale
with respect to
PNG
media_image6.png
22
23
media_image6.png
Greyscale
through the lens of empirical Bayes, where the data are used to optimize a prior within the family
PNG
media_image5.png
22
61
media_image5.png
Greyscale
” [Louppe page 3 Parameter Point Estimation]; The parameters that were used to generate simulation data are adjusted based on feedback to produce subsequent simulation data more representative of the observed data)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have incorporated dynamically adjusting generated parameters (incl. generated encoded vectors) based on real-time feedback from simulation data to create more realistic and varied data scenarios as taught by Louppe into the combination because both Wang and Louppe are directed towards generative adversarial learning frameworks. Incorporating the teachings of Louppe would generate subsequent simulation scenarios that more closely correspond to observed data.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Burkhardt et al. (Pub. No. US 20210142467 A1, “Systems and Methods of Generating Datasets for Training Neural Networks”, published 05/13/2021) discloses a system of generating images related to manufacturing processes and using the generated images to train a neural network to identify defects in the manufacturing process.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VIJAY M BALAKRISHNAN whose telephone number is (571) 272-0455. The examiner can normally be reached 10am-5pm EST Mon-Thurs.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JENNIFER WELCH can be reached on (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.M.B./
Examiner, Art Unit 2143
/JENNIFER N WELCH/
Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights