Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to the amendments filed 03 September 2025. Claims 1, 6, and 7 are amended. Claims 1, 3-7, and 9-14 are pending and have been examined.
Response to Arguments
Applicant's arguments, see page 9, filed 03 September 2025, with respect to the rejection of Claims 1, 3-7, and 9-14 under 35 U.S.C. 101 have been fully considered but they are not persuasive.
APPLICANT'S ARGUMENT: Applicant argues (page 9, paragraph 5) that "The claims are now explicitly directed to a process for 'acquiring design parameters for an electronic circuit.' This is not merely a mathematical calculation, but a specific method for improving the technology of electronic circuit design-a tangible, physical system. The process results in an 'optimum combination of the design parameters,' which can be used to manufacture an improved electronic circuit with superior physical properties, such as higher power efficiency or lower signal loss."
EXAMINER'S RESPONSE: Examiner respectfully disagrees that amended Claim 1 currently recites an improvement in technology. The claim limitation "for acquiring design parameters for an electronic circuit" of amended Claim 1 recites an intended use of the recited process, and therefore carries no patentable weight. Examiner further notes that amended Claim 1 does not positively recite a step of manufacturing. As indicated in the rejection of amended Claim 1 under 35 U.S.C. 101 below, the claim does not recite any additional elements that integrate the abstract ideas into a practical application or provide significantly more.
APPLICANT'S ARGUMENT: Applicant argues (page 9, paragraph 6) that "The invention as claimed provides a significant improvement to the technical field of computer-aided design by solving a technical problem: how to efficiently and accurately find optimal design parameters in a complex, multi-modal solution space without being misled by regions where a model's predictions are unreliable."
EXAMINER'S RESPONSE: Examiner respectfully disagrees that amended Claim 1 currently recites a solution to a technical problem. As indicated in the rejection of amended Claim 1 under 35 U.S.C. 101 below, the claim does not recite any additional elements that integrate the abstract ideas into a practical application or provide significantly more.
Applicant’s arguments, see pages 9-10, filed 03 September 2025, with respect to the rejection of Claims 1, 3, 4, 6, 7, 9, 10, 12, and 13 under 35 U.S.C. 103 have been considered but they are not persuasive.
APPLICANT'S ARGUMENT: Applicant argues (page 10, paragraph 1) that "By correlating the density of training data with the inference accuracy of the VAE's decoder, unreliable inferences from sparse, poorly-learned regions of the latent space may be avoided. The 'avoiding regions where inference is unreliable' is a purposeful, technical step, not a mere consequence of sampling from a concentrated area. The recognition that data density can be used as a proxy for the decoder's inference accuracy, and the subsequent use of this principle to deliberately restrict the search space for an optimization algorithm is new and not obvious."
EXAMINER'S RESPONSE: Examiner respectfully disagrees that the recited feature of amended Claim 1 is not obvious. In the rejection made under 35 U.S.C. 103 below, amended Claim 1 has been found to be obvious in view of Mandt in view of Ohta. The argued feature is taught by Ohta.
Amended Claim 1 has now also been rejected under 35 U.S.C. 103 on a new ground of rejection in view of Das in view of O'Shea in view of Tripp in view of Chen. The argued feature is taught by Chen.
APPLICANT'S ARGUMENT: Applicant argues (page 10, paragraph 2) that "Mandt not teach or suggest using a VAE to solve a general, black-box optimization problem, such as finding an optimal set of design parameters for a physical system. ... Ohta does not teach the critical insight of correlating the density of training data with the inference accuracy of the VAE's decoder."
EXAMINER'S RESPONSE: In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., "a general, black-box optimization problem") are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
In the rejection of amended Claim 1 made under 35 U.S.C. 103 below in view of Mandt in view of Ohta, the argued feature is taught by Ohta.
APPLICANT'S ARGUMENT: Applicant argues (page 11, paragraph 1) that "A POSITA seeking to improve Ohta's optimization method would have no reason to tum to Mandt, a signal compression reference, for guidance. ... [T]he combination would fail to teach the specific limitations now recited in the amended independent claims. ... The combination, therefore, does not teach or suggest this causal link between density, inference accuracy, and search space definition."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. The rejection of amended Claim 1 below has been made in view of Mandt in view of Ohta. Mandt teaches a medium and program for learning a VAE that further determines sparseness and density of regions of the latent space. Mandt does so in the context of neural network learning methods used for the purpose of reduction of hardware complexity or efficient processing. Per 2141.01(a), art is proper for use in an obviousness rejection under 35 U.S.C. 103 when it is "from the same field of endeavor as the claimed invention (even if it addresses a different problem)."
APPLICANT'S ARGUMENT: Applicant argues (page 12, paragraph 4) that "Akima does not teach first defining a high-confidence search region based on the density of training data distribution as recited in the independent claims. but follows a search trajectory through the latent space."
EXAMINER'S RESPONSE: Examiner notes that Akima is not relied on to teach the argued feature of amended Claim 1 below.
Applicant’s arguments, see pages 9-10, filed 03 September 2025, with respect to the rejection of Claims 1, 3, 4, 6, 7, 9, 10, 12, and 13 under 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
APPLICANT'S ARGUMENT: Applicant argues (page 12, paragraph 1) that "Das teaches determining a density model in a latent space and that Chen suggests a correlation between data density and reconstruction error (a measure of accuracy). However, neither reference, alone or in combination, teaches or suggests the crucial step recited in the amended claims: deciding a search region ... based on a density ... corresponding to a region of high inference accuracy ... to acquire an optimum combination of... design parameters."
EXAMINER'S RESPONSE: Examiner notes that Applicant's arguments are now moot. Amended Claim 1 has now been rejected under 35 U.S.C. 103 on a new ground of rejection in view of Das in view of O'Shea in view of Tripp in view of Chen. The argued features are taught by the proposed combination for the motivations set forth below.
APPLICANT'S ARGUMENT: Applicant argues (page 12, paragraph 2) that "Das uses its density model and classifier for a generative task: to sample new data points that conform to certain attributes. Chen uses its clustering loss for an analytical task: to better distinguish out-of-distribution samples. Neither reference is concerned with a black-box optimization task, such as finding the single best set of parameters for a physical system."
EXAMINER'S RESPONSE: Examiner notes that Applicant's arguments are now moot. Amended Claim 1 has now been rejected under 35 U.S.C. 103 on a new ground of rejection in view of Das in view of O'Shea in view of Tripp in view of Chen. The argued features are taught by the proposed combination for the motivations set forth below.
APPLICANT'S ARGUMENT: Applicant argues (page 12, paragraph 3) that "A person of ordinary skill in the art, when faced with the problem of optimizing a circuit design, would not have been motivated to combine a data generation method (Das) with an out-of-distribution analysis method (Chen) to arrive at the Applicant's specific solution of using density to define a high-confidence search boundary for optimization. ... A POSITA would not have considered O'Shea's use of a standard reconstruction error objective as a motivation to apply the data generation methods of Das and Chen to a complex, external optimization problem like circuit design."
EXAMINER'S RESPONSE: Examiner notes that Applicant's arguments are now moot. Amended Claim 1 has now been rejected under 35 U.S.C. 103 on a new ground of rejection in view of Das in view of O'Shea in view of Tripp in view of Chen. The argued features are taught by the proposed combination for the motivations set forth below.
APPLICANT'S ARGUMENT: Applicant argues (page 12, paragraph 4) that "Tripp is directed continually re-learning the model, not to finding the optimum solution within the reliable bounds of an already-learned model as recited in the independent claims."
EXAMINER'S RESPONSE: Examiner notes that Tripp is not relied on to anticipate amended Claim 1 below, but rather to teach the indicated features in combination.
Claim Objections
Claim 7 is objected to for informalities. Examiner believes that, as recited, Claim 7 mistakenly includes the previously claimed element "solution of a desired objective function by using the pieces of training data included in the search range," which has been struck from independent Claims 1 and 6. Appropriate correction and/or clarification is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 6, and 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term "unreliable" in Claim 1 is a relative term which renders the claim indefinite. The term "unreliable" is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Evaluating performance of the claimed decoder has been rendered indefinite by the recited use of the term "unreliable." Claims 6 and 7 are rejected under the same rationale.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-7, and 9-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1
Step 1
Claim 1 recites a non-transitory computer-readable storage medium storing a program that causes a computer to execute a process for acquiring design parameters for an electronic circuit, and thus the claimed manufacture falls within a statutory category of invention.
Step 2A Prong 1
The claim recites learning a variational autoencoder (VAE), which is a mental process. The claim recites mapping the plurality of pieces of training data to a latent space of the learned VAE to identify a data distribution over the latent space, which is a mental process. The claim recites determining sparseness or denseness of the distribution of the plurality of pieces of training data over the latent space, which is a mental process. The claim recites deciding a search region in which a density of the distribution of the plurality of pieces of training data over the latent space is equal to or greater than a threshold, the search region being a region of high data density corresponding to a region of high inference accuracy of a decoder of the learned VAE, thereby avoiding sparse regions of the distribution where inference by the decoder is unreliable, which is a mental process. The claim recites acquiring an optimum combination of the design parameters for the electronic circuit by performing a search for an optimum solution of the objective function exclusively within the decided search region, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element using a plurality of pieces of training data including ... an objective function, which invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element a combination of design parameters for the electronic circuit and a corresponding circuit characteristic serving as an objective function does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 3
Step 1
Regarding Claim 3, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites wherein in the latent space, the inference accuracy of a decoder of the learned VAE is higher in a region having a higher density of training data than in a region having a lower density of training data, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 4
Step 1
Regarding Claim 4, the rejection of Claim 3 is incorporated.
Step 2A Prong 1
The claim recites generating a sampling set of latent variables generated from the pieces of training data belonging to the region in which the density is equal to or greater than the threshold, which is a mental process. The claim recites acquiring the optimum solution of the desired objective function by inputting the sampling set to the decoder of the learned VAE, which is a mental process. Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 5
Step 1
Regarding Claim 5, the rejection of Claim 3 is incorporated.
Step 2A Prong 1
The claim recites selecting one piece among the pieces of training data belonging to the region in which the density is equal to or greater than the threshold, which is a mental process. The claim recites acquiring the optimum solution of the desired objective function based on a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 6
Step 1
Claim 6 recites an optimum solution acquisition method, for acquiring design parameters for an electronic circuit, executed by a computer, and thus the claimed method falls within a statutory category of invention.
Step 2A Prong 1
The claim recites learning a variational autoencoder (VAE), which is a mental process. The claim recites mapping the plurality of pieces of training data to a latent space of the learned VAE to identify a data distribution over the latent space, which is a mental process. The claim recites determining sparseness or denseness of the distribution of the plurality of pieces of training data over the latent space, which is a mental process. The claim recites deciding a search region in which a density of the distribution of the plurality of pieces of training data over the latent space is equal to or greater than a threshold, the search region being a region of high data density corresponding to a region of high inference accuracy of a decoder of the learned VAE, thereby avoiding sparse regions of the distribution where inference by the decoder is unreliable, which is a mental process. The claim recites acquiring an optimum combination of the design parameters for the electronic circuit by performing a search for an optimum solution of the objective function exclusively within the decided search region, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element using a plurality of pieces of training data including ... an objective function, which invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element a combination of design parameters for the electronic circuit and a corresponding circuit characteristic serving as an objective function does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claims 12-14, dependent on Claim 6, incorporate the rejection of Claim 6. Claims 12-14 incorporate substantively all the limitations of Claims 3-5, respectively, and are rejected under the same
rationales.
Regarding Claim 7
Step 1
Claim 7 recites an optimum solution acquisition apparatus for acquiring design parameters for an electronic circuit, comprising: a memory; and a processor coupled to the memory, and thus the claimed machine falls within a statutory category of invention.
Step 2A Prong 1
The claim recites learn a variational autoencoder (VAE), which is a mental process. The claim recites map the plurality of pieces of training data to a latent space of the learned VAE to identify a data distribution over the latent space, which is a mental process. The claim recites determining sparseness or denseness of the distribution of the plurality of pieces of training data over the latent space, which is a mental process. The claim recites deciding a search region in which a density of the distribution of the plurality of pieces of training data over the latent space is equal to or greater than a threshold, the search region being a region of high data density corresponding to a region of high inference accuracy of a decoder of the learned VAE, thereby avoiding sparse regions of the distribution where inference by the decoder is unreliable, which is a mental process. The claim recites acquire an optimum combination of the design parameters for the electronic circuit by performing a search for an optimum solution of the objective function exclusively within the decided search region, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element using a plurality of pieces of training data including ... an objective function, which invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element a combination of design parameters for the electronic circuit and a corresponding circuit characteristic serving as an objective function does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claims 9-11, dependent on Claim 7, incorporate the rejection of Claim 7. Claims 9-11 incorporate substantively all the limitations of Claims 3-5, respectively, and are rejected under the same rationales.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 4, 6, 7, 9, 10, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Mandt, et al. (US 2019/0393903 A1, hereinafter "Mandt"), in view of Ohta, et al. (US 2021/0097402 A1, hereinafter "Ohta").
Regarding Claim 1, Mandt teaches:
a non-transitory computer-readable storage medium storing a program that causes a computer to execute a process ... (Mandt, Claim 10: "A non-transitory computer-readable medium storing program instructions that, when executed by a processor, cause the processor to decode") the process comprising:
learning a variational autoencoder (VAE) by using a plurality of pieces of training data ... (Mandt, [0071]-[0073]: "In some embodiments, the method 600 may be performed by a training engine 140 that executes a neural network algorithm 210 that works in conjunction with a neural network 105 to train an encoder model 155 and a decoder model 165... The method 600 begins when the training engine 140 receives (at step 610) an objective function 220 that defines the overall objective for the neural network algorithm 210.... The training engine 140 then receives (at step 640) training input data" where [0040]: "In some embodiments, the neural network algorithm 210 comprises a variational auto-encoder {VAE)");
mapping the plurality of pieces of training data (Mandt, [0065]: "the encoder comprises the mappings that assign
x
to its most likely encoding
f
and
z
under the encoder," where Mandt's
x
corresponds to the instant pieces of training data and where Mandt's
f
and
z
correspond to the instant each latent variable, as in [0061]: "the reconstruction of frame xt at time t depends on variable f and corresponding variable zt") to a latent space of the learned VAE (Mandt, [0065]: "The fact that the probabilistic encoder model is a probability distribution over
f
and
z
means that a given sequence
x
is assigned to multiple possible encodings
f
and
z
, each weighted with a probability value," where Mandt's variables
f
and
z
correspond to the instant latent space, as in [0061]: "the reconstruction of frame xt at time t depends on variable f and corresponding variable zt"), to identify a data distribution over the latent space (Mandt, [0065]: "the encoder model 155 comprises an optimized Gaussian distribution over variables
f
and
z
that receives variable
x
as an input. ... this probability distribution is learned in the training phase");
determining sparseness or denseness of the distribution of the plurality of pieces of training data over the latent space (Mandt, [0065]: "The encoder and decoder models are each represented by an equation that expresses a Gaussian distribution that is determined by neural network parameters that are determined via neural network training. ... In general, the neural network training is used to parameterize the means and variances of the Gaussian distributions of the encoder and decoder models," where instant determining sparseness or denseness of the distribution corresponds to learning the variance/dispersion of distributions of
f
and
z
learned during training in Mandt)
Mandt teaches a process of training and optimizing a VAE that according to the distribution of training data in the latent space of the VAE. Mandt does so in the context of AI-based optimization of audio/video signal transmission over communications networks.
Mandt does not explicitly teach deciding, as the search range of the optimum solution , a region in which a density of the distribution of the plurality of pieces of training data over the latent space is equal to or greater than a threshold and acquiring an optimum solution of a desired objective function.
However, Ohta teaches in the context of AI-based optimization of circuit design:
a process for acquiring design parameters for an electronic circuit (Ohta, [0090]: "In this environment, according to the specific example, a simulation is executed by randomly extracting arbitrary points in the latent space and adopting the inferred circuit parameter combination as design parameters, and the failure/no-failure of the circuit parts optimization is checked");
training data including a combination of design parameters for the electronic circuit (Ohta, [0041]: "The VAE learns characteristic amounts of input data by performing dimension compression of the input data to a latent space. ... With focus on the characteristic, the VAE is learned by giving objective functions, variables and characteristic values corresponding to correct solution information to training data of the VAE the information processing apparatus" and [0056]: "A specific example of the aforementioned training data generation will be described with reference to FIGS. 4 to 8. As an example, optimization of design parameters in a circuit design will be described") and a corresponding circuit characteristic serving as an objective function (Ohta, [0060]: "As illustrated in FIG. 7, the training data generating unit 21 images each of n power efficiencies 1 to n, which is one of the objective functions, by setting an image density in accordance with the value of the objective function. The power loss, which is the other objective function, is imaged in the same manner");
deciding a search region ... of the plurality of pieces of training data over the latent space (Ohta, [0067]: "the set generating unit 23 gives (inputs) training data ... to the learned VAE ... and generates a solution space in which objective functions with higher similarities are placed in a concentrated manner (parts with higher objective functions and parts with low objective functions are concentrated). The set generating unit 23 generates a sampling set similar to the objective function desired by a user in the generated solution space," where Ohta's concentrated functions in the solution space corresponds to the instant search region over the latent space, where [0041]: "The VAE learns characteristic amounts of input data by performing dimension compression of the input data to a latent space. This is characterized in that data pieces with higher similarities are placed by concentrating the data pieces into arbitrary points in the latent space") ... in which a density of the distribution ... is equal to or greater than a threshold (Ohta, [0119]: "the information processing apparatus ... learns a VAE by using training data including an objective function, inputs training data to the learned VAE (encoder) and places objective functions having higher similarities over a latent space," where Ohta's categorical higher similarities corresponds to the instant greater than a threshold), the search region being a region of high data density corresponding to a region of high inference accuracy of a decoder of the learned VAE (Ohta, [0065]: "Because of minimization of the regularization loss, images having a higher similarity are decoded to close points in the latent space" and [0081]: "the set generating unit 23 gives a set of training data to the learned VAE and calculates a set of mean values of latent variables (S301). For example, the set generating unit 23 inputs a set of training data ... to the encoder in the learned VAE and obtains a set ... of mean values of latent variables" and [0082]: "The set generating unit 23 generates a sampling set from the range of latent variables (S303). For example, the set generating unit 23 generates a sampling set M of the range corresponding to the objective function desired by a user. ... After that, the acquiring unit 24 decodes the sampling set (S304) and acquires an optimum solution (S305)" where [0063]: "the VAE has an encoder and a decoder. When input data (vector X) is input to the encoder, the encoder generates parameters
μ
(vector) and
Σ
(vector) having normal distributions followed by a latent variable
z
. In other words, for example, the encoder compresses a characteristic of the input data (vector X), outputs a mean
μ
and a distribution
Σ
of an N-dimensional Gaussian distribution and, based on the two, acquires a latent variable Z by sampling"), thereby avoiding sparse regions of the distribution where inference by the decoder is unreliable (Ohta, [0114]: "FIG. 27 is a diagram for explaining a comparison in power efficiency between estimated values and simulation values. FIG. 27 illustrates simulation values and estimated values using the learned VAE for the power efficiency of the As illustrated in FIG. 27, the validation data with absolute errors of ±0.002 covers 95.5% of the interpolation area of the learning data, which is 62.5% of the whole data. The validation data with absolute errors of ±0.003 covers 100% of the interpolation area of the learning data, which is 82.0% of the whole data. The data with absolute errors of ±0.003 or lower contains 82% of the validation data (validated with random 200 points in the characteristic amount distribution), and combination candidates of parameter variables which maximize the design index were acquired")
acquiring an optimum combination of the design parameters for the electronic circuit (Ohta, [0043]: "the information processing apparatus 10 acquires an optimum value for the desired objective function by inference by using a decoder of the learned VAE based on the generated sampling set and acquires variables and characteristic values which give the optimum value for the objective function by inference by using the decoder of the learned VAE") by performing a search for an optimum solution of the objective function exclusively within the decided search region (Ohta, [0082]: "the set generating unit 23 calculates a range (lowest and highest) of the latent variables from the set of mean values of the latent variables (S302). The set generating unit 23 generates a sampling set from the range of latent variables (S303). For example, the set generating unit 23 generates a sampling set M of the range corresponding to the objective function desired by a user," where Ohta's samples generated from the decided range around mean values corresponds to the instant exclusively from the decided region).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Mandt regarding determining sparseness or denseness of the distribution of the training data over the latent space of a VAE with those of Ohta regarding training data including an objective function, deciding the search range of an optimum solution a region in which a density of the distribution of the plurality of pieces of training data over the latent space is equal to or greater than a threshold, and acquiring an optimum solution of a desired objective function by using the pieces of training data included in the search range.
The motivation to do so would be to improve the efficiency of determining a solution to an optimization problem using a VAE (Ohta, [0120]: "the information processing apparatus 10 may acquire an optimum solution rapidly even when the solution space of a problem is complex. The information processing apparatus 10 may configure a solution space in which objective functions are placed in a concentrated manner, and an optimum solution may be acquired with a lower number of data pieces by interpolation of the learned VAE").
Regarding Claim 6, Mandt teaches:
an optimum solution acquisition method executed by a computer (Mandt, [0071]: "FIG. 6 illustrates a flow diagram of method steps for training and generating encoder and decoder models using a neural network, according to various embodiments of the present invention" and [0073]: "the training engine 140 executes the neural network algorithm 210 to perform neural network training for determining an optimized neural network parameter phi (φ) for the encoder model and an optimized neural network parameter theta (θ) for the decoder model that, in conjunction, best approximates the objective function 220.") the method comprising: precisely those steps recited by the process of Claim 1. Claim 6 is rejected under the same rationale as Claim 1.
Regarding Claim 7, Mandt teaches:
an optimum solution acquisition apparatus, comprising: a memory; and a processor coupled to the memory (Mandt, Claim 19: "A computing system ... comprising: a memory ... and a processor that is coupled to the memory") the processor configured to: perform precisely those steps recited by the process of Claim 1. Claim 7 is rejected under the same rationale as Claim 1.
Regarding Claim 3, the Mandt/Ohta combination teaches the non-transitory computer-readable storage medium according to claim 1, and thus the rejection of Claim 1 is incorporated.
Ohta further teaches:
wherein in the latent space, the inference accuracy of a decoder of the learned VAE is higher in a region having a higher density of training data than in a region having a lower density of training data (Ohta, [0065]: "Because of minimization of the regularization loss, images having a higher similarity are decoded to close points in the latent space. (4) in FIG. 9 indicates a mean squared error, a cross entropy error or the like between the input X and the output X' as an approximation of the reproduction loss" and [0066]: "In the VAE designed as described above, the parameters in the encoder and the decoder are learned such that Loss is minimized with respect to a set
ξ
=
{
X
1
,
X
2
,
…
X
n
}
of the training data," where the minimization of Ohta's joint regularization/reproduction loss corresponds to the instant higher inference accuracy of the decoder).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Mandt/Ohta combination regarding identifying a distribution of the plurality of pieces of training data over a latent space of the learned VAE with the further teachings of Ohta regarding the inference accuracy of the VAE decoder being higher in a region having a higher density of training data than in a region having a lower density of training data.
The motivation to do so would be to facilitate a more efficient optimization process (Ohta, [0058]: "By searching proper values of the latent variables within the latent space by using the characteristic of the VAE 325, the structure of the objective function in the optimization problem of the circuit design changes, and the number of changes of circuit parameters in the searching processing may be reduced. Therefore, the circuit parameters may be rapidly and easily optimized").
Regarding Claim 4, the Mandt/Ohta combination teaches the non-transitory computer-readable storage medium according to claim 3, and thus the rejection of Claim 3 is incorporated. The Mandt/Ohta combination teaches:
wherein the acquiring includes: generating a sampling set of latent variables generated from the pieces of training data belonging to the region in which the density is equal to or greater than the threshold; and acquiring the optimum solution of the desired objective function by inputting the sampling set to the decoder of the learned VAE (given the training method in Mandt [0048]: "Maximizing the objective function 220 shown in equation (1) also produces a minimized reconstruction error achieved by the encoder model and the decoder model when reconstructing (encoding then decoding) the training input sequences," the instant pieces of training data correspond to the final pieces of training data of Mandt's training input sequence, which pieces are handled as the reconstruction error is sufficiently minimized; and where the instant belonging to the region in which the density is equal to ... the threshold corresponds to Mandt's method of decoding latent variables, where all pieces of training data belong to the threshold region, which is the mean of the distributions of the latent variables, as in [0065]: "The encoder and decoder models are each represented by an equation that expresses a Gaussian distribution that is determined by neural network parameters that are determined via neural network training.... After this probability distribution is learned in the training phase ... the decoder model 165 comprises an optimized Gaussian distribution over x, whose Gaussian mean and variance are neural network transformations of f and z. After training, the Gaussian means are taken as the deterministic decoder model," where Mandt's x represents the input to the encoder and f and z represent latent variables input to the decoder").
Claims 9 and 10 incorporate the steps of the process of Claims 3 and 4, respectively, in apparatus form, and are rejected under the same rationale.
Claims 12 and 13 incorporate the steps of the process of Claims 3 and 4, respectively, in method form, and are rejected under the same rationales.
Claims 1, 3-7, and 9-14 are rejected under 35 U.S.C. 103 as being unpatentable over Das, et al. (US 2021/0110255 A1, hereinafter "Das") in view of O'Shea, et al. (US 2019/0274108 A1, hereinafter "O'Shea") in view of Tripp, et al., "Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining" (hereinafter "Tripp") in view of Chen, et al., "Skip The Question You Don’t Know: An Embedding Space Approach" (hereinafter "Chen").
Regarding Claim 1, Das teaches:
a non-transitory computer-readable storage medium storing a program that causes a computer to execute a process ... (Das, [0013]: "a computer program product for generating attribute-based samples includes a computer readable storage medium having program instructions embodied therewith, where the computer readable storage medium is not a transitory signal per se, and where the program instructions are executable by a processor to cause the processor to perform a method"), the process comprising:
learning a variational autoencoder (VAE) (Das, [0098]: "unsupervised learning of meaningful continuous latent representation z is performed. The method is agnostic to latent variable model (VAE/WAE/ALI/GAN/ ... )" and [0076]: "a method is presented to generate new samples that are similar to a dataset, while controlling a set of attributes. ... The method may be presented in the 'latent variable model' paradigm, which includes VAE, WAE, GANs, ALI, etc. The latent space will be written as 'z'") by using a plurality of pieces of training data ... (Das,[0097]: "FIG. 6 illustrates an exemplary training of a latent variable model 600, according to one exemplary aspect. As shown, training data 602 is input into an encoder 604 of the latent variable model 600, which creates a latent space representation 606 of the training data 602. A decoder 608 is then trained to convert the latent space representation 606 into reconstructed data 610. A regularization loss 612 and a reconstruction loss 614 are also calculated");
mapping the plurality of pieces of training data to a latent space of the learned VAE to identify a data distribution over the latent space (Das, [0060]: "training the classifier may include embedding every labeled data point in the latent space representation in order to create the explicit density model. In another example, the explicit density model may explicitly capture how the labeled data is arranged within the latent space representation," where Das's labeled data points corresponds to the instant latent variables and Das's embedded data points correspond to the instant latent variables mapped to a latent space, and where Das's density model corresponds to the instant data distribution, as in [0062]: "the encoder may map data points to a latent variable z. In another example, the classifier may then predict a label for a data point based on its latent variable (z) representation").
determining sparseness or denseness of the distribution of the plurality of pieces of training data over the latent space (Das, [0073]: "method 500 may proceed with operation 506, where an explicit density model for the data set is determined utilizing the latent space representation for the data set. In addition, method 500 may proceed with operation 508, where a set of classifiers is determined to identify which regions of the latent space representation are consistent with a predetermined set of labels," where Das's determined density model corresponds to the instant determining denseness);
deciding a search region in which a density of the distribution of the plurality of pieces of training data over the latent space is equal to or greater than a threshold (Das, [0123]: "Rejection sampling is performed through the proposal distribution:
g
z
=
Q
ξ
z
which can be directly sampled" and [0126]: "the sample is accepted with a probability equal to the product of the classifiers' scores. In order to accept any samples a region in z space needs to exist where
Q
ξ
z
>
0
and the classifiers assign nonzero probability to all desired attributes, i.e. the combination of attributes has to be realizable in z-space," where Das's
Q
ξ
corresponds to the instant density of the distribution, as in [0088]: "Q_xi(z) is a simple explicit density estimator with parameters xi, e.g. Gaussian, mixture of gaussians, normalizing flow density estimator, etc." and where the regions are search regions per Fig. 5, block 508, "Determine a set of classifiers to identify which regions of the latent space representation are consistent with a predetermined set of labels," where Das's to identify consistent regions corresponds to the instant search space).
Das teaches a process of learning a VAE using training data and mapping the training data to the latent space of the VAE to learn a distribution over the training data.
Das does not explicitly teach a process for acquiring design parameters for an electronic circuit, training data including a combination of design parameters for the electronic circuit and a corresponding circuit characteristic serving as an objective function, or acquiring an optimum combination of the design parameters for the electronic circuit by performing a search for an optimum solution of the objective function.
However, O'Shea teaches:
a process for acquiring design parameters for an electronic circuit (O'Shea, [0100]: "by optimizing the objectives of an approximated communications channel and information encoding for the approximated communications channel, the disclosed system enables the design of communications systems that can account for specific hardware devices, channel types, channel impairments, or other constraints, which are traditionally hard to model ( or result in suboptimal performance when making simplifying assumptions about the channel effects)");
training data including a combination of design parameters for the electronic circuit (O'Shea, [0193]: "An encoder machine-learning network is used to process this first information.... [I]n some implementations the first information is represented by training data, in which case the encoder machine-learning network processes the training data representing the first information. Furthermore, as discussed above, the generated first RF signal may represent an analog RF waveform that is transmitted over a channel, or may be an intermediate representation (e.g., samples, basis coefficients, distributions over RF waveforms, etc.)" and [0117]: "In scenarios of approximated channel training ... the transmitted signal 130 may be compared with the received signal 140, and the channel machine-learning network of the approximated channel may be trained (updated) based on ... factors, such as ... transmission bandwidth or power used to communicate over the channel 108, or various combinations thereof and other metrics") and a corresponding circuit characteristic serving as an objective function (O'Shea, [0222]: "Channel input 702 may be a transmitted signal (e.g., transmitted signal 130 as shown in FIG. 1), created by an encoder (e.g., encoder 904 as shown in FIG. 9). Channel input 702 may be the actual RF waveform in analog form, or may be a series of radio samples in time, frequency, or any other signal representation basis, or may be an intermediate representation (e.g., RF samples, basis coefficients, distributions over RF waveform values, etc.).... For example, the channel input 702 and the channel output 712 may represent distributions over RF waveform values"); ...; and
acquiring an optimum combination of the design parameters for the electronic circuit (O'Shea, [0072]: "by optimizing the objectives of an approximated communications channel and information encoding for the approximated communications channel, the disclosed system enables the design of communications systems that can account for specific hardware devices, channel types, channel impairments, or other constraints, which are traditionally hard to model ... or which may vary widely depending on hardware and environmental factors upon deployment") by performing a search for an optimum solution of the objective function ... (O'Shea, [0162]: "Training the encoder 204 and decoder 214 may involve optimizing over a set of basis functions or over different sets of basis functions, for example using greedy search or other optimization-type algorithm").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Das regarding learning a VAE using training data and mapping the training data to the latent space of the VAE to learn a distribution over the training data with those of O'Shea regarding a process for acquiring design parameters for an electronic circuit, training data including a combination of design parameters for the electronic circuit and a corresponding circuit characteristic serving as an objective function, or acquiring an optimum combination of the design parameters for the electronic circuit by performing a search for an optimum solution of the objective function.
The motivation to do so would be to facilitate design of communication systems with otherwise challenging constraints to model (O'Shea, [0100]: "by optimizing the objectives of an approximated communications channel and information encoding for the approximated communications channel, the disclosed system enables the design of communications systems that can account for specific hardware devices, channel types, channel impairments, or other constraints, which are traditionally hard to model (or result in suboptimal performance when making simplifying assumptions about the channel effects)").
The Das/O'Shea combination teaches training data for a VAE including a combination of design parameters for an electronic circuit and acquiring an optimum combination of the design parameters for the electronic circuit by performing a search for an optimum solution of the objective function.
The Das/O'Shea combination does not explicitly teach the search region being a region of high data density corresponding to a region of high inference accuracy of a decoder of the learned VAE, or a search for an optimum solution of the objective function exclusively within the decided search region.
However, Tripp teaches:
the search region being a region of high data density corresponding to a region of high inference accuracy of a decoder of the learned VAE (Tripp, p. 3, 3 Failure Modes of Latent Space Optimization: "This means that although the resulting function
g
:
Z
↦
X
is defined over the entire latent space
Z
, it is effectively only trained on points in regions of
Z
with high probability under
p
. Importantly, even if
Z
is an unbounded space with infinite volume such as
R
n
, because
p
has finite volume, there must exist a finite subset
Z
'
⊂
Z
that contains virtually all the probability mass of p. We call
Z
'
the feasible region of
Z
. Although in principle optimization can be performed over all of
Z
'
, it has been widely observed that optimizing outside of the feasible region tends to give poor results, yielding samples that are low-quality, or even invalid ... [A]ll LSO methods known to us employ some sort of measure to restrict the optimization to near or within the feasible region .... This means that LSO should be treated as a bounded optimization problem, whose feasible region is determined by
p
" and p. 3, note 2: "For example, a
d
dimensional Gaussian distribution has non-zero probability density everywhere, but almost all the probability mass in in a spherical shell of radius
d
" where Tripp's feasible region for optimization corresponds to the instant dense region of high accuracy); and
... a search for an optimum solution of the objective ... exclusively within the decided search region (Tripp, p. 5, 4.3 Weighted Retraining Combined: "in the first iteration in Fig. 1b the high-scoring pink point is given a higher weight, causing the feasible region to extend farther into the green region at the expense of the red region. This allows a better first point (orange) to be chosen relative to Fig. 1a. In the second iteration in Fig. 1b, weighted retraining on the orange point reshapes the latent space again, bringing the global optimum into the feasible region, where it is ultimately reached," where Tripp's global optimum corresponds to the optimum solution).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Das/O'Shea combination regarding training data for a VAE including a combination of design parameters for an electronic circuit and acquiring an optimum combination of the design parameters for the electronic circuit by performing a search for an optimum solution of the objective function with those of Tripp regarding the search region being a region of high data density corresponding to a region of high inference accuracy of a decoder of the learned VAE, or a search for an optimum solution of the objective function exclusively within the decided search region.
The motivation to do so would be to facilitate model training in circumstances where training data is scarce (Tripp, p. 4, 4 Latent Space Optimization with Weighted Retraining, 4.1 Training a Generative Model with a Weighted Training Objective: "One obvious but inadequate method of achieving this to simply discard all low-scoring points, e.g. by keeping only the top 10% of the data set, and use this reduced dataset to train the DGM [deep generative model]. While this strategy could be feasible if data is plentiful, when data is scarce this option may not be viable because state of the art neural networks need a large amount of training data to avoid overfitting").
The Das/O'Shea/Tripp combination teaches acquiring design parameters for an electronic circuit by training a VAE and acquiring an optimum solution of an objective function in a dense region of the latent space of the VAE.
The Das/O'Shea/Tripp combination does not explicitly teach avoiding sparse regions of the distribution where inference by the decoder is unreliable.
However, Chen teaches:
avoiding sparse regions of the distribution where inference by the decoder is unreliable (Chen, p. 7, Fig. 5, "Three types of samples and their positions at embedding space" and p. 7, IV D. Embedding Visualization: "our model has much cleaner embedding space. For the OOD samples with label 0 and 1, we can observe that they are well separated from other clusters, and the distance between different label clusters are much larger than the previous two models. As a result, our model will give outliers a much higher reconstruction loss," where Chen's well-separated clusters and higher reconstruction loss of outliers corresponds to the instant higher density and higher decoder inference accuracy, respectively).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the prior teachings of the Das/Chen/O'Shea combination regarding determining sparseness or denseness of the distribution of the plurality of pieces of training data over the latent space with the further teachings of Chen regarding in the latent space, the inference accuracy of a decoder of the learned VAE is higher in a region having a higher density of training data than in a region having a lower density of training data.
The motivation to do so would be to facilitate improved identification of out-of-distribution samples in a dataset (Chen, p. 7, IV D. Embedding Visualization: "The cluster penalization term in our method will make the distributions of normal samples at the embedding layer more representative. When an outlier comes, the difference of distributions at embedding layer will give higher reconstruction error, which helps us to distinguish the OOD samples").
Regarding Claim 3, the rejection of Claim 1 is incorporated.
Chen further teaches:
wherein: in the latent space, the inference accuracy of a decoder of the learned VAE is higher in a region having a higher density of training data than in a region having a lower density of training data (Chen, p. 7, Fig. 5, "Three types of samples and their positions at embedding space" and p. 7, IV D. Embedding Visualization: "our model has much cleaner embedding space. For the OOD samples with label 0 and 1, we can observe that they are well separated from other clusters, and the distance between different label clusters are much larger than the previous two models. As a result, our model will give outliers a much higher reconstruction loss," where Chen's well-separated clusters and higher reconstruction loss of outliers corresponds to the instant higher density and higher decoder inference accuracy, respectively).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the prior teachings of the Das/O'Shea/Tripp/Chen combination regarding determining sparseness or denseness of the distribution of the plurality of pieces of training data over the latent space with the further teachings of Chen regarding in the latent space, the inference accuracy of a decoder of the learned VAE is higher in a region having a higher density of training data than in a region having a lower density of training data.
The motivation to do so would be to facilitate improved identification of out-of-distribution samples in a dataset (Chen, p. 7, IV D. Embedding Visualization: "The cluster penalization term in our method will make the distributions of normal samples at the embedding layer more representative. When an outlier comes, the difference of distributions at embedding layer will give higher reconstruction error, which helps us to distinguish the OOD samples").
Claims 9 and 12 incorporate substantively the limitations of Claim 3 in apparatus and method forms, respectively, and are rejected under the same rationale.
Regarding Claim 4, the rejection of Claim 3 is incorporated. The Das/O'Shea/Tripp/Chen combination teaches:
wherein the acquiring includes: generating a sampling set of latent variables generated from the pieces of training data belonging to the region in which the density is equal to or greater than the threshold (Das, Fig. 5, 510: "Sample data points within the latent space representation for the data set that are consistent with the predetermined set of labels, utilizing rejection sampling"); and
inputting the sampling set to the decoder of the learned VAE (Das, Fig. 5, 512: "Convert the sampled data points from a latent space representation to a data representation, utilizing the trained decoder").
Chen further teaches:
acquiring the optimum solution of the desired objective function by inputting the sampling set to the decoder (Chen, p. 5, Fig. 3, depicting joint training of clustered data, and p. 5, C. Optimization on embedding loss: "We optimize equation III-B by stochastic gradient descent(SGD) with learning rate 0.01," where Chen's equation III-B corresponds to the instant objective function, and D. Training: "after pretraining reconstruction part of the neural network with some clean data, ... we jointly train the classifier and reconstruction neural network.... Since we continue updating our cluster center and target distribution ... the model can still converge to incorporate these points").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the prior teachings of the Das/Chen/O'Shea combination regarding generating a sampling set of latent variables generated from the pieces of training data belonging to the region dense region and inputting the sampling set to the decoder of the learned VAE with the further teachings of Chen regarding acquiring the optimum solution of the desired objective function by inputting the sampling set to the decoder.
The motivation to do so would be to facilitate training a model with greater discriminative power (Chen, p. 7, IV D. Embedding Visualization: "the model without clustering loss has poor ability to separate between cluster 4 and cluster 9, the inter-cluster distance between 3 and 5 is also very small. In contrast, our model has much cleaner embedding space. For the OOD samples with label 0 and 1, we can observe that they are well separated from other clusters, and the distance between different label clusters are much larger than the previous two models").
Claims 10 and 13 incorporate substantively the limitations of Claim 4 in apparatus and method form, respectively, and are rejected under the same rationale.
Regarding Claim 5, the rejection of Claim 3 is incorporated. The Das/O'Shea/Tripp/Chen combination teaches:
selecting ... pieces of training data belonging to the region in which the density is equal to or greater than the threshold (Das, [0123]: "Rejection sampling is performed through the proposal distribution:
g
z
=
Q
ξ
z
which can be directly sampled" and [0126]: "In order to accept any samples a region in z space needs to exist where
Q
ξ
z
>
0
and the classifiers assign nonzero probability to all desired attributes, i.e. the combination of attributes has to be realizable in z-space," where Das's
Q
ξ
corresponds to the instant density of the distribution, as in [0088]: "Q_xi(z) is a simple explicit density estimator with parameters xi, e.g. Gaussian, mixture of gaussians, normalizing flow density estimator, etc."); and
acquiring ... a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE (Das, [0097]: "training data 602 is input into an encoder 604 of the latent variable model 600, which creates a latent space representation 606 of the training data 602. A decoder 608 is then trained to convert the latent space representation 606 into reconstructed data 610. A regularization loss 612 and a reconstruction loss 614 are also calculated," where Das's reconstructed data corresponds to the instant restoration result).
Tripp futher teaches:
selecting one piece among the pieces of training data ... (Tripp, p. 6, Algorithm 1 Latent Space Optimization with Weighted Retraining, lines 6-7, "optimize
h
to obtain new latent query point
z
~
" and "Obtain corresponding input
x
~
=
g
z
~
," where Tripp's corresponding input corresponds to the instant one piece); and
acquiring the optimum solution of the desired objective function based on ... the latent variable generated from the selected training data ... (Tripp, p. 2, Figure 1: "Our proposed approach weights data points according to their objective function value and retrains g to incorporate newly queried data. ... Red/green regions correspond to points with low/high objective function values, respectively. The yellow star is the global optimum in
X
," where Tripp's model may be a VAE, as in p. 7, 6 Empirical Evaluation, 2D Shape Area Maximization Toy Task: "Model: A convolutional VAE with
Z
=
R
2
, as a standard neural network architecture for image modelling").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Das/O'Shea/Tripp/Chen combination regarding selecting pieces of training data belonging to the region in which the density is equal to or greater than the threshold and acquiring a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE with the further teachings of Tripp regarding selecting one piece among the pieces of training data and acquiring the optimum solution of the desired objective function based on the latent variable generated from the selected training data.
The motivation to do so would be to facilitate efficient use of a VAE for optimization (Tripp, p. 5, 4.3 Weighted Retraining Combined: "When put together, data weighting and periodic retraining complement each other elegantly, transforming the generative model from a passive decoding function into an active participant in the optimization process, whose role is to ensure that the latent manifold is constantly occupied by the most updated and relevant points for optimization").
Claims 11 and 14 incorporate substantively all limitations of Claim 5 in apparatus and method form, respectively, and are rejected under the same rationale.
Regarding Claim 6, Das teaches:
an optimum solution acquisition method executed by a computer (Das, [0011]: "a computer-implemented method includes training an encoder and decoder of a latent variable model (LVM) ... and converting the sampled data points from a latent space representation to a data representation, utilizing the trained decoder") the method comprising precisely those steps recited by the process of Claim 1. Claim 6 is rejected under the same rationale as Claim 1.
Regarding Claim 7, Das teaches:
an optimum solution acquisition apparatus, comprising: a memory; and a processor coupled to the memory (Das, [0013]: "the program instructions are executable by a processor to cause the processor to perform a method including training, by the processor, an encoder and decoder of a latent variable model (LVM) ... and converting, by the processor, the sampled data points from a latent space representation to a data representation, utilizing a trained decoder of the LVM" and [0137]: "The processor may be of any configuration as described herein, such as a discrete processor or a processing circuit that includes many components such as processing hardware, memory, I/O interfaces, etc.") and the processor configured to perform precisely those steps recited by the process of Claim 1. Claim 7 is rejected under the same rationale as Claim 1.
Claims 5, 11, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Mandt, et al. (US 2019/0393903 A1, hereinafter "Mandt"), in view of Ohta, et al. (US 2021/0097402 A1, hereinafter "Ohta"), in further view of Akima (US 2020/0285690 A1, hereinafter "Akima").
Regarding Claim 5, the Mandt/Ohta combination teaches the non-transitory computer-readable storage medium according to claim 3, and thus the rejection of Claim 3 is incorporated. The Mandt/Ohta combination teaches:
selecting ... pieces of training data belonging to the region in which the density is equal to or greater than the threshold; and acquiring the ... solution of the desired objective function based on a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE (Mandt, [0048]: "Maximizing the objective function 220 shown in equation (1) also produces a minimized reconstruction error achieved by the encoder model and the decoder model when reconstructing (encoding then decoding) the training input sequences," the instant one piece among the pieces of training data corresponds to the final piece of training data of Mandt's training input sequence, which piece is handled as the reconstruction error is sufficiently minimized; and where the instant belonging to the region in which the density is equal to ... the threshold corresponds to Mandt's method, where the final piece of training data belongs to the threshold region, which is the mean of the distributions of the latent variables, as in [0065]: "The encoder and decoder models are each represented by an equation that expresses a Gaussian distribution that is determined by neural network parameters that are determined via neural network training.... After this probability distribution is learned in the training phase, the encoder comprises the mappings that assign x to its most likely encoding f and z under the encoder model 155, making the encoder a deterministic mapping. The most likely value is the mean of the Gaussian distribution, which prevents the neural networks from overfitting during training. ").
The Mandt/Ohta combination does not explicitly teach selecting one piece among the pieces of training data and acquiring the optimum solution of the desired objective function based on a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE.
However, Akima teaches:
selecting one piece among the pieces of training data ... (Akima, [0051]: "If the difference between the calculated characteristic value and the target value is higher than a threshold value, the image generating unit 313 generates an image representing the generated time-series data. Next, the searching unit 316 acquires an average value of each of the latent variables from the generated image by using the VAE 325. An average value of each of the plurality of latent variables represents a current search point within the latent space" and [0053]: "The simulator 312, the image generating unit 313 and the searching unit 316 repeat the processing of changing the values of the variables until the difference between the characteristic value and the target value gets lower than the threshold value. When the difference gets lower than the threshold value, the searching unit 316 stores the values of the variables in the storage unit 311 as proper parameters 330, and the output unit 317 outputs the proper parameters 330," where Akima's stored variable values corresponds to the instant one piece among training data); and
acquiring the optimum solution of the desired objective function (Akima, Fig. 9, step 908, and [0099]: "the data processing apparatus 301 repeats the processing in step 902 and subsequent steps... If
Δ
η
is equal to or lower than TH (YES in step 903) ... the output unit 317 outputs the proper parameters 330 (step 908)," where Akima's proper parameters correspond to the instant optimum solution) based on a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE (Akima, [0069]: "The first term of the right side of Expression (11) represents a regularized loss, and the second term represents a reconstruction loss" and [0070]: "The learning unit 314 learns parameters of the encoder 601 and the decoder 602 for the image set 1; in Expression (7) such that the loss L in Expression (11) is minimized," where Akima's decoder learned based on minimized reconstruction loss corresponds to the instant based on a restoration result).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Das/O'Shea/Tripp/Chen combination regarding selecting pieces of training data belonging to the region in which the density is equal to or greater than the threshold and acquiring a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE with the further teachings of Akima regarding selecting one piece among the pieces of training data and acquiring the optimum solution of the desired objective function based on a restoration result obtained by inputting the latent variable generated from the selected training data to the decoder of the learned VAE.
The motivation to do so would be to facilitate more efficient learning of an optimal solution to an objective function based on sampling from the latent space of a VAE (Akima, [0058]: "By searching proper values of the latent variables within the latent space by using the characteristic of the VAE 325, the structure of the objective function in the optimization problem of the circuit design changes, and the number of changes of circuit parameters in the searching processing may be reduced. Therefore, the circuit parameters may be rapidly and easily optimized").
Claims 11 and 14 incorporate substantively all limitations of Claim 5 in method form and are rejected under the same rationale.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT N DAY whose telephone number is (703)756-1519. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.N.D./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122