DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-14 are pending.
Priority
This application claims benefit of application 2022-050434, filed 03/25/2022, in Japan. The instant application has the effective filing date of 25 March 2022.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/12/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Drawings
The drawings, submitted on 01/12/2023, are accepted by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitations are: “storage unit”, “modulation unit”, “generation unit”, and “output unit” in claim 1; “control unit” in claim 3; and “evaluation unit” in claim 6.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The disclosure states the storage unit can include hardware devices such as read only memory (ROM), a random-access memory (RAM), a hard disk drive (HDD), and a flash memory (page 8, para. 1); the input unit can include hardware devices such as a keyboard, a mouse, a touch panel, a numeric keypad, and a scanner (page 8, para. 1); the output unit can include hardware devices such as a display and a printer (page 8, para. 1); and the control unit can include hardware devices such as a controller that (page 55, para. 2). As such, the units qualify as hardware components, clearly associated with embodiments per MPEP 2181 II (C).
The disclosure states other units have access to the processor (fig. 1), including the generation unit, which may equate to a spectral generator, that calculates signal positions with equations (page 25, para. 2); and the evaluation unit, which may equate to an evaluator, that calculates the reward and trains a regression model using equations (page 51, para. 2). As such, the units qualify as algorithms for performing computer-implemented functions per MPEP 2181 II (B).
The disclosure states the modulation unit may be a modulator, which executes identification modulation and receives variables from the user (page 23, para. 2).The metes and bounds of the structure of the modulator are unclear and rendered indefinite per MPEP 2181.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 6-10 are rejected under 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph, because claim 6 purports to invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, with the use of “modulation unit,” but fails to recite a combination of elements as required by that statutory provision and thus cannot rely on the specification to provide the structure, material or acts to support the claimed function.
As such, the claim recites a function that has no limits and covers every conceivable means for achieving the stated function, while the specification discloses at most only those means known to the inventor. Accordingly, the disclosure is not commensurate with the scope of the claim. Claims dependent on claim 6, that also fail to remedy the aforementioned issue are rejected on the same grounds.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The “modulation unit” limitation of claim 6 has been evaluated under the three-prong test set forth in MPEP § 2181, subsection I, but the result is inconclusive. Thus, it is unclear whether this limitation should be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because there does not appear to be sufficient structure within the disclosure for one of ordinary skill in the art to confidently ascertain its function. The boundaries of this claim limitation are ambiguous; therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph. Claims dependent on claim 6, that also fail to remedy the aforementioned issue are rejected on the same grounds.
In response to this rejection, applicant must clarify whether this limitation should be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Mere assertion regarding applicant’s intent to invoke or not invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph is insufficient. Applicant may:
(a) Amend the claim to clearly invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, by reciting “means” or a generic placeholder for means, or by reciting “step.” The “means,” generic placeholder, or “step” must be modified by functional language, and must not be modified by sufficient structure, material, or acts for performing the claimed function;
(b) Present a sufficient showing that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, should apply because the claim limitation recites a function to be performed and does not recite sufficient structure, material, or acts to perform that function;
(c) Amend the claim to clearly avoid invoking 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, by deleting the function or by reciting sufficient structure, material or acts to perform the recited function; or
(d) Present a sufficient showing that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, does not apply because the limitation does not recite a function or does recite a function along with sufficient structure, material or acts to perform that function.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 are rejected under U.S.C 101 because the claimed invention is directed to abstract ideas without significantly more, as detailed in the analysis below.
Eligibility Step 1: Subject matter eligibility evaluation in accordance with MPEP § 2106:
Claims 1-12 are directed to a statutory category (apparatus).
Claim 13 is directed to a statutory category (method).
Claim 14 is directed to a statutory category (product).
Therefore, in accordance with MPEP § 2106.03 all claims have patent eligible subject matter.
[Eligibility Step 1: YES]
Eligibility Step 2A: This step determines whether a claim is directed to a judicial exception in accordance with MPEP § 2106.
Eligibility Step 2A -- Prong One: Limitations are analyzed to determine if the claims recite any concepts that could equate to a judicial exception (i.e. abstract idea, law of nature, or natural phenomenon). Possible judicial exceptions are explored below.
Recitations of Judicial Exceptions:
Claims 1 and 13-14: a generation unit configured to generate a first multispectral signal obtained by classifying the first signal modulated by the modulation unit for each analysis target into a first spectral signal for each value of the objective variable; (mathematical concept, mental process)
an output unit configured to generate, based on the first multispectral signal, a signal distribution obtained by one-dimensionally arranging a distribution of the first signal based on the value of the objective variable (mathematical concept, mental process)
a modulation unit configured to generate, based on the action history information, a first signal obtained by modulating the analysis target data for each analysis target; (mental process, mathematical concept)
Claim 2: wherein the modulation unit combines the actions in the action history information to create an expression, and outputs the first signal that is a calculation result of the expression for each analysis target. (mental process, mathematical concept)
Claim 3: a control unit configured to select a first action from the pattern information, and add the first action to the action history information. (mental process)
Claim 4: the control unit randomly selects the first action from the pattern information. (mental process)
Claim 5: wherein the control unit generates, based on a training parameter and the first multispectral signal, a first array indicating a value for each action, selects the first action corresponding to a specific value in the first array, and adds the first action to the action history information. (mathematical concept, mental process)
Claim 6: an evaluation unit configured to generate a training model based on the first signal for each analysis target and the value of the objective variable, calculate a predicted value for each analysis target by inputting the first signal for each analysis target to the training model, and calculate, based on the predicted value for each analysis target and the value of the objective variable, a reward for evaluating the value of the first action, wherein the modulation unit generates, based on action history information to which the first action is added by the control unit, a second signal obtained by modulating the analysis target data for each analysis target, (mental process, mathematical concept)
the generation unit generates a second multispectral signal obtained by classifying the second signal modulated by the modulation unit for each analysis target into a second spectral signal based on the value of the objective variable, (mental process)
the control unit generates, based on the reward, a training parameter, and the second multispectral signal, a second array indicating a value for each action, selects a specific value in the second array, and updates the training parameter. (mathematical concept, mental process)
Claim 7: wherein the reward increases as prediction accuracy of the training model increases. (mathematical concept, mental process)
Claim 8: wherein the value of the objective variable is an identification value related to the analysis target, the output unit generates, based on the first multispectral signal, the signal distribution obtained by one-dimensionally arranging a plurality of distributions of the first signal for each value of the objective variable, (mathematical concept, mental process)
and the reward increases as the number of overlapping portions of the plurality of distributions decreases. (mathematical concept, mental process)
Claim 9: wherein the value of the objective variable is an identification value related to the analysis target, the output unit generates, based on the first multispectral signal, the signal distribution obtained by one-dimensionally arranging a plurality of distributions of the first signal for each value of the objective variable, (mental process)
and the reward increases as an interval between the plurality of distributions increases. (mathematical concept, mental process)
Claim 11: wherein the value of the objective variable is a predicted value indicating a regression result related to the analysis target, the generation unit calculates a loss function for each analysis target based on the value of the objective variable and the first signal modulated by the modulation unit for each analysis target, (mathematical concept)
generates the first multispectral signal obtained by classifying a calculation result of the loss function into a first spectral signal for each value of the objective variable, and the output unit generates, based on the first multispectral signal, the signal distribution indicating the calculation result of the loss function for the first signal arranged in order of the value of the objective variable (mathematical concept, mental process)
Claim 12: wherein the generation unit generates the first multispectral signal for each loss function when a plurality of the loss functions are set, and the output unit generates, based on the first multispectral signal for each loss function, one signal distribution including calculation results of the plurality of loss functions for the first signal arranged in order of the value of the objective variable, (mathematical concept, mental process)
Step 2A – Prong One Analysis:
Analysis techniques such as identifying, selecting, classifying, arranging, and making determinations of data, requiring nothing more than the human mind and pen/paper, read on observations, evaluations, judgments, and opinions, and thus fall under the mental process grouping of abstract ideas.
Analysis techniques such as loss functions, calculations, regression, and distributing data recite mathematical calculations and relationships that fall under the mathematical concept grouping of abstract ideas.
Therefore, the claims are found to recite judicial exceptions.
[Eligibility Step 2A – Prong One: YES]
Eligibility Step 2A – Prong Two: A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. If the claim contains no additional claim elements beyond the abstract idea, the claim fails to integrate the abstract idea into a practical application (MPEP 2106.04(d)). Additional elements are recited, categorized, and analyzed below.
Data Gathering/Outputting Elements:
Claims 1, 8-9, and 11-14: output the signal distribution in a displayable manner
Claims 1 and 13-14: store an analysis target data group including, for each analysis target, analysis target data that includes a value of an explanatory variable and a value of an objective variable for the analysis target, and action history information retaining one or more actions which are either the explanatory variable or a modulation method for modulating the explanatory variable;
Claim 2: acquiring the value of the explanatory variable included in the expression from the analysis target data
Claim 3: wherein the storage unit stores pattern information including one or more explanatory variables and one or more modulation methods
Claim 10: outputs the interval between the plurality of distributions in a displayable manner.
Computer Components Elements:
Claims 1, 3, and 13-14: storage unit
Claims 1 and 8-14: output unit
Step 2A – Prong Two Analysis:
The data gathering elements perform mere data gathering and outputting necessary to complete the judicial exception. Elements of this manner are classified as insignificant extra solution-activity per MPEP 2106.05(g).
Generic computer components (storage unit, output unit) and implementations of a method onto a generic computer environment provide mere instructions to implement the abstract ideas onto a technological environment per Alice Corp., 573 U.S. at 223, 110 USPQ2d at 1983. See also 573 U.S. at 224, 110 USPQ2d at 1984.
As such, the additional elements, when viewed separately and in the context of a whole claimed invention, do not integrate the judicial exceptions into practical application.
[Eligibility Step 2A – Prong Two: NO]
Eligibility Step 2B: Claim elements are probed for inventive concept equating to significantly more than the judicial exception (MPEP 2106.04(II)).
Step 2B Analysis:
Gathering and selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display is found to be well-understood, routine, and conventional per Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354-55, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016).
The computer components are further found to be well-understood, routine, and conventional per Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93 for storing and retrieving information in memory and Alice Corp., 573 U.S. at 225, 110 USPQ2d at 1984 (MPEP 2106.05 (a)); and MPEP 2106.05 (g) for necessary, generic outputting.
As such, the additional elements are further found to lack inventive concept.
[Eligibility Step 2B: NO]
Therefore, claims 1-14 are directed to judicial exceptions without significantly more and are rejected under 35 U.S.C 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Wen et al. (ACS Sensors; Vol. 6: 10; 2021).
Claims 1 and 13-14 are directed to an apparatus, method, and computer readable medium with instructions that store an analysis target data group that includes an explanatory variable, an objective variable for the analysis target, and action history information in the form of an explanatory variable or a modulation method for modulating the explanatory variable.
Wen et al. describes a guide to signal processing algorithms for nanopore sensors.
Wen et al. teaches signal processing is an inseparable component of sensing in order to identify the hidden features in the signals and analyze them (page 1, column 1); signal processing technologies for identification of nanopore biomolecules include both software algorithms and hardware readout circuits/systems (page 2, column 1).
Wen et al. teaches by processing the signal and analyzing the features of the spikes such as amplitude, width, occurrence frequency, and waveform, the properties of the analytes can be inferred, including size, shape, charge, dipole moment, and concentration (page 1, column 1); and outputs will be associated with historical input or the distribution of input by “learning” from the history/distribution and exploiting the hidden relations/patterns carried in the input data via a ML-based algorithm (page 3, column 1).
Claims 1 and 13-14 are further directed to the optional modulation method including: generating, based on the action history information, a first signal obtained by modulating the analysis target data for each analysis target.
Wen et al. teaches generating a steady ionic current, which constitutes the baseline of the signal, by applying a bias voltage across the membrane (page 2 column 2).
Claims 1 and 13-14 are further directed to generating a first multispectral signal by classifying the first signal into a first spectral signal for each objective variable.
Wen et al. teaches prior to classification, an ensemble of empirical mode decomposition, variational mode decomposition, inherent time scale decomposition, and Hilbert transform has been designed to extract multispectral features of nanopore electrical signals, for example according to their respective translocation spike signal (page 9, column 1) based on user-defined thresholds of amplitude (page 3, column 1).
Claims 1 and 13-14 are further directed to one-dimensionally arranging a distribution of the first signal, based on the objective variable to generate and output a signal distribution in a displayable manner.
Wen et al. teaches selecting features related to the shape of the translocation spikes; applying Expectation Maximization algorithm to estimate the statistical distribution parameters of the spikes in a seven-dimensional feature space (page 11, column 1); and analyzing the rationality and consistency of the outputs with the assistance of related physical models (page 14, column 2).
Claim 2 is directed to combining the actions in the action history information to create an expression; acquiring the value of the explanatory variable, within the expression; and outputting the first signal as a calculation result of the expression for each analysis target.
Wen et al. teaches dynamically adjusting key parameters of the Kalman filter, according to the historical inputs; acquiring and representing stochastic properties of the signal by these dynamic parameters (page 4, column 1); and implementing threshold functions to select the boundary between large and small magnitude and keep the large magnitude components as the outcome of the wavelet transform of the main features of the signal (page 4, column 1).
Claim 3 is directed to storing pattern information including one or more explanatory variables and one or more modulation methods; and the signal processing apparatus including a control unit that can select an action from the pattern information, and add the first action to the action history information.
Wen et al. teaches learning regular patterns as shapelets from the training data set to maximize the discriminative features among the spikes from different analytes (page 7, column 2). Wen et al. teaches the regular patterns include characteristics of the translocating analytes as well as their interactions with the nanopore, such as tiny fluctuations of the ionic current in the blockage state (page 7, column 2); selecting several details of the translocation spikes as features (page 5, column 2); and exploiting the hidden relations/patterns from the input data (page 3, column 2).
Claim 4 is directed to the control unit randomly selecting the first action from the pattern information.
Wen et al. teaches a RF algorithm constructs the bagging ensemble architecture by involving a random selection mechanism in the training data selection for each DT (page 9, column 1), where the training data sets are artificially generated by a simulator on the foundation of a set of physical models, describing open-pore current, blockage spikes, background noise, and baseline variations.
Claim 5 is directed to the control unit generating, based on a training parameter and the first multispectral signal, a first array indicating a value for each action; selecting the first action corresponding to a specific value in the first array; and adding the first action to the action history information.
Wen et al. teaches an ensemble of empirical mode decomposition, variational mode decomposition, inherent time scale decomposition, and Hilbert transform designed to extract multispectral features of nanopore electrical signals (page 9, column 2); and by combining ResNet with SVM, adeno-associated viruses carrying different genetic cargos are discriminated according to their respective translocation spike signal through a SiNx nanopore. Wen et al. teaches ResNet extracts abstract “features” of the signal traces, although these features are not describable and cannot be directly correlated to physical meanings, and delivers them to a SVM for classification (page 9, column 2), where the algorithm “learns” from the history/distribution and exploits the hidden relations/patterns carried in the input data (page 3, column 1).
Wen et al. does not explicitly teach one-dimensionally arranging a distribution of the first signal (claims 1 and 13-14).
However, Wen et al. teaches by training on lower dimensional data and comparing different strategies, such as fully connected DNN, CNN, and LSTM, a high accuracy up to 94% on average is reached (page 9, column 2).
Therefore Wen et al. teaches using objective variables (amplitude, width, occurrence frequency, and waveform); explanatory variables (size, shape, charge, dipole moment, and concentration); and action history to generate and arrange signal distributions.
Though Wen et al. teaches generating a seven-dimensional signal, instead of one, it teaches that training on lower dimensional datasets can lead to higher accuracies. Therefore, the method is capable of generating at least a univariate signal distribution, and provides motivation for one of ordinary skill in the art to generate lower dimensional distributions with a reasonable expectation of success.
Claims 6 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Wen et al. (ACS Sensors; Vol. 6: 10; 2021), as applied to claims 1-5 and 13-14 above, in view of Eysenbach et al. (Berkely Artificial Intelligence Research; 2020).
Claim 6 is directed to generating a training model based on the first signal for each analysis target and the value of the objective variable; calculating a predicted value for each analysis target by inputting the first signal for each analysis target to the training model; and calculating a reward for evaluating the value of the first action, based on the predicted value for each analysis target and the value of the objective variable.
Wen et al. teaches dynamically adjusting the parameters and structures of the ML-based algorithms in the training process according to the input data (page 3, column 1); and during the training process, sending segments of time sequence traces to the NN (page 7, column 2); predicting values of spike number, average amplitude, and duration of the appearing spikes; comparing them with the respective ground truths to obtain deviations (page 7, column 2); and evaluating the training performance in each epoch (page 7, column 2) using validation metrics (page 13, column 2).
Claim 6 is further directed to modulating the analysis target data for each analysis target using action history information including a first action, to obtain a second signal; and classifying the second signal modulated for each analysis target into a spectral signal based on the value of the objective variable.
Wen et al. teaches generating signals for nanopore sensing, by functionalizing the nanopore surface with a probe molecule to generate a specific interaction with target analytes resulting in characteristic signals on the monitoring ionic current trace (page 3, column 1); inferring the analyte properties and identify/classifying the analytes based on the extracted features (page 5, column 2); identifying new spikes by synthetically considering the three features in a three-dimensional space, (page 5, column 2), or applying systematic modifications to the original training data in a way that it creates new samples (page 15, column 1).
Claim 6 is further directed to the control unit generating a second array indicating a value for each action, based on the reward, a training parameter, and the second multispectral signal; selecting a specific value in the second array; and updating the training parameter.
Wen et al. teaches using an Expectation Maximization (EM) algorithm to estimate values of the latent variables from the parameters of stochastic schemes; generating a vector of probabilities with one value for each possible class (page 10, table 2); selecting seven features related to the shape of the translocation spikes, by applying the EM iteration (page 11, column 1); and updating the stochastic parameters according to the observed variables from the data set and latent variables (page 11, column 1).
Claim 8 is directed to the value of the objective variable being an identification value related to the analysis target; the output unit generating, based on the first multispectral signal, the signal distribution obtained by one-dimensionally arranging a plurality of distributions of the first signal for each value of the objective variable; and outputs the signal distribution in a displayable manner, and the reward increasing as the number of overlapping portions of the plurality of distributions decreases.
Wen et al. teaches by processing the signal and analyzing the features of the spikes, the properties of the analytes can be inferred, include size, shape, charge, dipole moment, and concentration (page 1, column 1); and the SVM classifier does not perform well when the data points at different target classes severely overlap (page 8, column 1).
Wen et al. does not explicitly teach using the reward to generate a second array (claim 6).
Eysenbach et al. describes how reinforcement learning acts as supervised learning on optimized data.
Eysenbach et al. teaches the simplified notation of Expectation Maximization can be represented using πθ(τ) as the probability that policy πθ produces trajectory τ; q(τ) to denote the data distribution that we will optimize; considering the log of the expected reward objective, logJ(θ); and since log function is monotonic increasing, maximizing this as an equivalence to maximizing the expected reward (page 3, column 1).
Eysenbach et al. further teaches this lower bound is useful as it allows us to optimize a policy using data sampled from a different policy; makes explicit the fact that reinforcement learning (RL) is a joint optimization problem over the policy and experience (page 3, column 1); and when optimizing the lower bound with respect to the policy, the objective is exactly equivalent to supervised learning, such as behavior cloning (page 4, column 1).
Eysenbach et al. teaches this observation is exciting because supervised learning is generally much more stable than RL algorithms; and suggests that prior RL methods that use supervised learning as a subroutine might actually be optimizing a lower bound on expected reward (page 4, column 1).
Therefore Eysenbach et al. teaches a method of utilizing the maximization of a reward within an Expectation Maximization algorithm, in order to yield more stable results. As such, it would be obvious to one of ordinary skill in the art to apply the technique taught by Eysenbach et al. to the method of Wen et al. with a reasonable expectation of success and improvement to the system.
Claims 7 and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Wen et al. (ACS Sensors; Vol. 6: 10; 2021), in view of Eysenbach et al. (Berkely Artificial Intelligence Research; 2020) as applied to claims 1-6, 8, and 13-14 above, and in further view of Wang et al. (npj2D Materials and Applications; Vol. 5: 66; 2021).
Wen et al. in view of Eysenbach et al. teach a method, apparatus, and computer readable medium with instructions for using objective variables, explanatory variables, and action history to generate and arrange signal distributions; calculate rewards; and output results.
Claim 7 is directed the reward increasing as prediction accuracy of the training model increases.
Wen et al. does not explicitly teach reward increasing as prediction accuracy of the training model increases.
Wang et al. describes efficient water desalination with graphene nanopores obtained using artificial intelligence
Wang et al. teaches the reward of DRL agent in our model was calculated based on the water flux/ion rejection prediction of performance predictor (page 2, column 2); and since the accuracy of performance predictor directly influence how accurately the DRL agent is rewarded/penalized during training, the model with the least MSE and highest R2 values was chosen to be used for reward estimation, using ResNet50 (page 3, column 1).
Claim 9 is directed to the value of the objective variable being an identification value related to the analysis target, the output unit generating, based on the first multispectral signal, the signal distribution by one-dimensionally arranging a plurality of distributions of the first signal for each value of the objective variable; outputting the signal distribution in a displayable manner; and the reward increasing as an interval between the plurality of distributions increases.
Wen et al. teaches by processing the signal and analyzing the features of the spikes, the properties of the analytes can be inferred, include size, shape, charge, dipole moment, and concentration (page 1, column 1); selecting features related to the shape of the translocation spikes; applying Expectation Maximization algorithm to estimate the statistical distribution parameters of the spikes in a seven-dimensional feature space (page 11, column 1); and analyzing the rationality and consistency of the outputs with the assistance of related physical models (page 14, column 2).
Wang et al. teaches at each timestep, the DRL agent generates an updated nanopore by removing at most one atom from the graphene (page 2, column 1); the CNN-based performance predictor network predicts the water flux/ion rejection rate of the nanopore; and the DRL agent generates the nanopore which brings a positive reward at each timestep (page 5, column 1).
Claim 10 is directed to the output unit outputting the interval between the plurality of distributions in a displayable manner.
Wang et al. teaches displaying the summation of reward in each timestep vs. episode, where the red line is the running average of the reward with window size 21 and the blue shadow represents the standard deviation; b) summation of reward in each timestep vs. timestep; d) predicted water flux vs. timestep; and e) predicted ion rejection vs. timestep (page 4, fig. 3).
Therefore Wen et al. teaches a method of processing signals using supervised learning techniques, analogous to the claimed invention, and provides motivation that reinforcement learning is an optimized-data alternative (page 15, column 1). Eysenbach et al. suggests that RL methods that use supervised learning as a subroutine might optimize a lower bound on expected reward (page 4, column 1); and Wang et al. teaches a data-driven artificial intelligence framework for discovering efficient graphene nanopores for desalination via a combination of deep reinforcement learning and convolutional neural network (page 1, abstract).
As such, it would be obvious to one of ordinary skill in the art to combine the methods and systems of Wang et al. that utilizes supervised learning as a subroutine, into the method of signal process taught by Wen et al. with a reasonable expectation of success, based on the teachings of Eysenbach et al.
Claims 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Wen et al. (ACS Sensors; Vol. 6: 10; 2021), as applied to claims 1-5 and 13-14 previously, in view of Wang et al. (npj2D Materials and Applications; Vol. 5: 66; 2021).
Wen et al. teaches a method, apparatus, and computer readable medium with instructions for using objective variables, explanatory variables, and action history to generate and arrange signal distributions; and output results.
Claim 11 is directed to the value of the objective variable being a predicted value indicating a regression result related to the analysis target; and calculating a loss function for each analysis target based on the objective variable and first signal modulated by the modulation unit for each analysis target.
Wen et al. teaches using an SVM-based regressor to establish the correspondence between specific peptide features inside the pore and the generated signal; and a method of implementing the RF regression for translocation waveform prediction (page 9, column 1).
Wen et al. teaches the resulting RF becomes more robust to outliers; exhibits less overfitting (page 9, column 1); and uses the derivatives between of the predicted values and the ground truths, chain rule, and Stochastic Gradient Descent (SGD) algorithm to move the optimal point of the network progressively, in order to find some local minimum in a loss function that the system seeks to minimize (page 12, column 2).
Claim 11 is further directed to generating the first multispectral signal obtained by classifying a result of the loss function into a first spectral signal for each value of the objective variable. Claim 11 is further directed to the output unit generating, based on the first multispectral signal, the signal distribution indicating the result of the loss function for the first signal arranged in order of the value of the objective variable; and outputting the signal distribution in a displayable manner.
Wen et al. teaches using the loss function result as valuable knowledge that aids in collecting the relevant data in order to understand the target requirements (page 13, column 1).
Wen et al. does not explicitly teach classifying and outputting the result of the loss function, according to the remaining limitations of claim 11.
Wang et al. describes efficient water desalination with graphene nanopores obtained using artificial intelligence, as taught above.
Wang et al. teaches making predictions through an MLP regression model (page 7, column 2); using a loss function to measure the difference between the target Q value and the prediction of current Q-network Q (page 8, column 1); the mean squared error (MSE) and coefficient of determination (R2) as metrics to evaluate the performance predictions of models (page 3, column 1); choosing the model with the least MSE and highest R2 values for reward estimation (page 3, column 1); and leveraging the predicted flux ft and ion rejection it to compute the reward signal rt (page 3, column 2).
Wang et al. teaches the model with the least MSE and highest R2 values was chosen to be used for reward estimation (page 3, column 1); and shows the output water flux and ion rejection rate distribution of the final training dataset for predictive CNN model (page 3, fig. 2d), where a reverse sigmoid function was fitted to the distribution of samples to show the general relationship between the water flux and ion rejection rates, represented by a blue dashed line (page 2, column 2).
Claim 12 is directed to generating the first multispectral signal for each loss function, when a plurality of the loss functions are set; generating, based on the first multispectral signal for each loss function, one signal distribution including calculation results of the plurality of loss functions for the first signal arranged in order of the value of the objective variable; and outputting the signal distribution in a displayable manner.
Wang et al. teaches benchmarking the model based on the MSE and R2 of their resulting water flux/ion rejection rate prediction (page 7, column 2); feeding the processed geometrical features into a CNN; extracting a feature vector with the dimension of 1000 output from the CNN model, so finally, the MLP was able to make predictions of flux and ion rejection rates, shown in in fig. 3d (page 5, column 1).
Wang et al. further shows output in the form of 2D t-SNE embedding of features extracted from water flux prediction CNN model, where each point is colored by its predicted water flux; and b) 2D t-SNE embedding of features extracted from ion rejection rate prediction CNN model, where each point is colored by its predicted ion rejection, and each axis represents a dimension of the t-SNE embedding (page 5, fig. 4).
Therefore, Wang et al. teaches methods of using reward and loss functions in the form of validation metrics, such as mean-squared error, in order to classify and arrange the results via a combination of deep reinforcement learning (DRL) and convolutional neural network (page 1, column 1) for water desalination.
Wen et al. teaches a base product of signal processing, using mostly supervised learning. Wen et al. further teaches reinforcement learning is an optimized-data alternative to supervised learning, since the sample complexity does not depend on preexisting data, but rather on the actions that an agent takes in the dynamics of an environment (page 15, column 1); and nanopore technology holds great promise for a wide range of applications such as biomedical sensing, chemical detection, and desalination (page 1, column 1).
As such, it would be obvious to one of ordinary skill in the art to combine the teachings of Wang et al. with the method of Wen et al. in order to optimize the supervised learning process with reinforcement learning, with a reasonable expectation of success in the field of signal processing for water desalination.
Conclusion
No claims are currently allowed.
Correspondence
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Milana Thompson whose telephone number is (571)272-8740. The examiner can normally be reached Monday - Friday, 9:00-6:00 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Karlheinz Skowronek can be reached at (571) 272-1113. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.K.T./Examiner, Art Unit 1687
/Karlheinz R. Skowronek/Supervisory Patent Examiner, Art Unit 1687
Prosecution Insights