DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 9, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
The drawings are objected to because in Figure 3, “device” and “interface” should be “devices” and “interfaces”, respectively. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Objections
Claim 14 is objected to because of the following informalities: “the training” should be “wherein the training”.
Claim 19 is objected to because of the following informalities: an “and” should follow the penultimate limitation and “configured to encoding” in the first limitation should be “configured to encode”. Claim 20 is objected to for dependency on claim 19.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 and 19-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 1: The claim recites a method; therefore, it is directed to the statutory category of processes.
Step 2A, Prong 1: The claim recites, inter alia:
[D]etermining cadence-velocity data and micro range-Doppler data from the radar data: This limitation could encompass the mental determination of the two types of data.
[E]ncoding the cadence-velocity data to obtain a cadence-velocity feature vector … and encoding the micro range-Doppler data to obtain a range-Doppler feature vector …: This limitation could encompass mentally encoding the two types of data to obtain feature vectors.
[D]ecoding the cadence-velocity feature vector to obtain reconstructed cadence-velocity data … and decoding the range-Doppler feature vector to obtain reconstructed range-Doppler data …: This limitation could encompass mentally decoding the two types of data to obtain reconstructed data.
[D]etermining first reconstruction loss information based on the cadence-velocity data and the reconstructed cadence-velocity data and determining second reconstruction loss information based on the micro range-Doppler data and the reconstructed range-Doppler data: This limitation could encompass mentally determining loss information based on the claimed types of data. Determining a loss is also a mathematical concept.
[C]lassifying the radar data based on the first reconstruction loss information and the second reconstruction loss information: This limitation could encompass mentally classifying the radar data using the listed data.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “obtaining radar data from the scene”, which is directed to the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g). The claim further recites that certain operations are performed using a “first trained autoencoder”, “second trained autoencoder”, “first trained decoder”, and “second trained decoder”. However, these limitations amount to mere instructions to apply the judicial exception using a generic computer programmed with generic classes of algorithms. MPEP § 2106.05(f).
Step 2B: The claim does not contain elements that amount to significantly more than the judicial exception. The obtaining limitation, in addition to being insignificant extra-solution activity, also recites the well-understood, routine, and conventional activity of receiving and transmitting data over a network. MPEP § 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015). Otherwise, the analysis at this step mirrors that of step 2A, prong 2. As an ordered whole, the claim is directed to a mentally performable process of classifying radar data based on other prepared types of data. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 2
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites, inter alia, that “classifying the radar data further comprises combining the range-Doppler feature vector and the cadence-velocity feature vector to obtain a combined feature vector”. This limitation could encompass mentally combining the vectors.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the combination is performed “using multihead-attention”. However, this is a mere instruction to apply the judicial exception using a generic computer programmed with a generic class of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain elements that amount to significantly more than the judicial exception. The claim further recites that the combination is performed “using multihead-attention”. However, this is a mere instruction to apply the judicial exception using a generic computer programmed with a generic class of computer algorithm. MPEP § 2106.05(f).
Claim 3
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites that “combining the range-Doppler feature vector and the cadence-velocity feature vector comprises using different weightings of different elements of the range-Doppler feature vector and the cadence-velocity feature vector to obtain the combined feature vector.” This limitation could encompass mentally combining the vectors in the claimed manner.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. See claim 2 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 2 analysis.
Claim 4
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites that “classifying the radar data comprises providing information on whether the radar data lie[] in a data distribution used to train the first trained autoencoder, the second trained autoencoder, the first trained decoder, and the second trained decoder.” This limitation could encompass mentally comparing the radar data distribution to those used to train the networks.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 5
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites that “classifying the radar data comprises providing information on whether one or more humans are present in the scene.” This limitation could encompass mentally determining whether humans are present in the scene.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 6
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites that “classifying the radar data comprises determining an energy score based on the range-Doppler feature vector and the cadence-velocity feature vector and using the energy score to classify the radar data, wherein the energy score indicates a compatibility of the range-Doppler feature vector and the cadence-velocity feature vector with data used for training the classifying.” This limitation could encompass mentally determining an energy score and mentally using that energy score to classify the data.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. See claim 1 analysis.
Step 2B: The claim does not contain significantly more than the judicial exception. See claim 1 analysis.
Claim 7
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites the same judicial exceptions as in claim 1.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that “the classifying the radar data comprises using a trained neural network.” However, this is a mere instruction to apply the judicial exception using a generic computer programmed with a generic class of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that “the classifying the radar data comprises using a trained neural network.” However, this is a mere instruction to apply the judicial exception using a generic computer programmed with a generic class of computer algorithm. MPEP § 2106.05(f).
Claim 8
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites the same judicial exceptions as in claim 1.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that “the first trained autoencoder and the first trained decoder form a first trained generative autoencoder; and the second trained autoencoder and the second trained decoder form a second generative autoencoder.” However, this is a mere instruction to apply the judicial exception using a generic computer programmed with a generic class of computer algorithm. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that “the first trained autoencoder and the first trained decoder form a first trained generative autoencoder; and the second trained autoencoder and the second trained decoder form a second generative autoencoder.” However, this is a mere instruction to apply the judicial exception using a generic computer programmed with a generic class of computer algorithm. MPEP § 2106.05(f).
Claim 9
Step 1: A process, as above.
Step 2A, Prong 1: The claim recites, inter alia, that “obtaining the radar data from the scene comprises … sampling a radar signal reflected from the scene.” This limitation could encompass mentally sampling the signal.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “receiving … a radar signal reflected from the scene.” This limitation is directed to the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites “receiving … a radar signal reflected from the scene.” This limitation is directed to the well-understood, routine, and conventional activity of receiving and transmitting data over a network. MPEP § 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015).
Claim 10
Step 1: The claim recites a non-transitory storage medium; therefore, it is directed to the statutory category of articles of manufacture.
Step 2A, Prong 1: The claim recites the same judicial exceptions as in claim 1.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “[a]n apparatus for classifying radar data from a scene, the apparatus comprising: one or more interfaces configured to receive radar data from the scene; and one or more processing devices configured to perform the method”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites “[a]n apparatus for classifying radar data from a scene, the apparatus comprising: one or more interfaces configured to receive radar data from the scene; and one or more processing devices configured to perform the method”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claim 11
Step 1: The claim is directed to an apparatus comprising processing devices; therefore, it is directed to the statutory category of machines.
Step 2A, Prong 1: The claim recites the same judicial exceptions as in claim 1.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites “[a] non-transitory storage medium with instructions stored thereon, where the instructions, when executed by a computer, a processor or a programmable hardware component enable the computer, the processor or the programmable hardware component to perform the method”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites “[a] non-transitory storage medium with instructions stored thereon, where the instructions, when executed by a computer, a processor or a programmable hardware component enable the computer, the processor or the programmable hardware component to perform the method”. However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claim 19
Step 1: The claim recites a system comprising processors and memories; therefore, the claim is directed to the statutory category of machines.
Step 2A, Prong 1: The claim recites, inter alia:
[E]encoding cadence velocity data to obtain a cadence-velocity feature vector: This limitation could encompass mentally encoding the cadence velocity data into a feature vector.
[E]ncod[ing] micro range-Doppler data to obtain a range-Doppler feature vector: This limitation could encompass mentally encoding the range-Doppler data into a feature vector.
[D]ecod[ing] the cadence-velocity feature vector to obtain reconstructed cadence-velocity data: This limitation could encompass mentally decoding the cadence-velocity feature vector.
[D]ecod[ing] the range-Doppler feature vector to obtain reconstructed range-Doppler data: This limitation could encompass mentally decoding the range-Doppler feature vector.
[C]lassify[ing] radar data based on first reconstruction loss information and second reconstruction loss information: This limitation could encompass mentally classifying radar data based on reconstruction loss information.
[D]etermining the cadence-velocity data and the micro range-Doppler data from the radar data: This limitation could encompass mentally determining the cadence-velocity and micro range-Doppler data.
[D]etermining the first reconstruction loss information based on the cadence-velocity data and the reconstructed cadence-velocity data and determining the second reconstruction loss information based on the micro range-Doppler data and the reconstructed range-Doppler data: This limitation could encompass mentally determining the reconstruction losses based on the claimed data.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that certain operations are performed with “a first trained autoencoder”, “a second trained autoencoder”, “a first trained decoder”, “a second trained decoder”, “a trained neural network”, “one or more processors”, and “one or more memories coupled to the one or more processors with instructions stored thereon, wherein the instructions, when executed by the one or more processors, enable the one or more processors to perform the steps”. However, these are mere instructions to apply the judicial exception using a generic computer programmed with generic classes of computer algorithm. MPEP § 2106.05(f). The claim further recites “obtaining the radar data from a radar sensor”, which recites the insignificant extra-solution activity of mere data gathering and output. MPEP § 2106.05(g).
Step 2B: The claim does not contain elements that amount to significantly more than the judicial exception. The obtaining limitation, in addition to being insignificant extra-solution activity, also recites the well-understood, routine, and conventional activity of receiving or transmitting data over a network. MPEP § 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015). As an ordered whole, the claim is directed to a mentally performable process of encoding, decoding, and classifying radar data. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Claim 20
Step 1: A machine, as above.
Step 2A, Prong 1: The claim recites the same judicial exceptions as in claim 19.
Step 2A, Prong 2: This judicial exception is not integrated into a practical application. The claim further recites that the system “further compris[es] the radar sensor.” However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Step 2B: The claim does not contain significantly more than the judicial exception. The claim further recites that the system “further compris[es] the radar sensor.” However, this is a mere instruction to apply the judicial exception using a generic computer. MPEP § 2106.05(f).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 5, 7, 9-13, 15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Seyfioglu et al., “Deep Convolutional Autoencoder for Radar-Based Classification of Similar Aided and Unaided Human Activities,” in 54.4 IEEE Transactions on Aerospace and Electronic Sys. 1709-23 (2018) (“Seyfioglu”) in view of Jia et al., “Human Activity Classification with Radar Signal Processing and Machine Learning,” in Int’l Conf. UK-China Emerging Technologies (2020) (“Jia”).
Regarding claim 1, Seyfioglu discloses “[a] method for classifying radar data from a scene, the method comprising:
obtaining radar data from the scene (radar measurements of human activity were made in an indoor laboratory environment spanning a range of 1-5 m using an NI-USRP 2922 model software-defined radio platform programmed to transmit a continuous wave signal at 4 GHz – Seyfioglu, sec. II, first paragraph);
determining cadence-velocity data and micro range-Doppler data from the radar data (spectrograms were used to represent the time-frequency distribution of the measured micro-Doppler signature – Seyfioglu, sec. II(A), first paragraph; physical features are direct measurements of properties of the spectrogram or cadence velocity diagram – id. at sec. III(A)); …
encoding the micro range-Doppler data to obtain a range-Doppler feature vector using a second trained autoencoder (physical features include the overall Doppler frequency and transform-based features computed from the discrete cosine transform of the micro-Doppler frequency – Seyfioglu, sec. III(A); for a given input vector, the encoder of an autoencoder computes a nonlinear mapping of the inputs [to produce a feature vector, which is a range-Doppler feature vector when range-Doppler inputs are given] – id. at sec. IV(A), second paragraph; see also sec. V(A), second paragraph (disclosing that a convolutional autoencoder is used)); …
decoding the range-Doppler feature vector to obtain reconstructed range-Doppler data using a second trained decoder (encoded features are then decoded to reconstruct the given input vector x – Seyfioglu, sec. IV(A), second paragraph; see also sec. III(A) (disclosing that the features input, and therefore the features reconstructed, are Doppler-related features)); …
determining second reconstruction loss information based on the micro range-Doppler data and the reconstructed range-Doppler data (during unsupervised pretraining, the network tries to minimize the reconstruction error [loss information] by adjusting its weights and biases – Seyfioglu, sec. IV(A), third paragraph; see also sec. III(A) (disclosing that the features input, and therefore the features reconstructed, are Doppler-related features)); and
classifying the radar data based on … the second reconstruction loss information (after unsupervised pretraining [using the reconstruction loss information], the decoder is removed and the remaining encoder components are trained in a supervised manner by adding a softmax classifier with 12 neurons after the encoder – Seyfioglu, sec. IV(A), p. 1714, first full paragraph; see also sec. I, first paragraph (disclosing that the classification is of radar data)).”
Seyfioglu appears not to disclose explicitly the further limitations of the claim. However, Jia discloses “encoding the cadence-velocity data to obtain a cadence-velocity feature vector using a first trained autoencoder (autoencoder consists of an encoder [first trained autoencoder] and a decoder; the aim of the stacked autoencoder is compressing [encoding] the input into a middle layer and then decompressing it back, and make the output as similar to the input as possible – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)) …; …
decoding the cadence-velocity feature vector to obtain reconstructed cadence-velocity data using a first trained decoder (autoencoder consists of an encoder and a decoder [first trained decoder]; the aim of the stacked autoencoder is compressing the input into a middle layer and then decompressing [decoding] it back, and make the output as similar to the input as possible – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)) …; …
determining first reconstruction loss information based on the cadence-velocity data and the reconstructed cadence-velocity data (autoencoder consists of an encoder and a decoder; the aim of the stacked autoencoder is compressing the input into a middle layer and then decompressing it back, and make the output as similar to the input as possible [i.e., minimize first reconstruction loss information] – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)) …; and
classifying the radar data based on the first … information (after training [i.e., based on the minimization of the reconstruction loss], the middle layer of the stacked autoencoder containing the compressed representation of the input is drawn out and connected to a softmax layer to realize classification using a Bayes optimizer – Jia, sec. V(A)) ….”
Jia and the instant application both relate to machine learning using radar-related data and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Seyfioglu to classify radar data using an autoencoder accepting cadence-velocity data, as disclosed by Jia, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the classification to protect individual privacy by avoiding the use of cameras. See Jia, sec. I, first paragraph.
Regarding claim 5, Seyfioglu, as modified by Jia, discloses that “classifying the radar data comprises providing information on whether one or more humans are present in the scene (proposed convolutional autoencoder is used to discriminate 12 classes of aided and unaided human activities [which implicitly classifies whether a human is present] – Seyfioglu, sec. I, penultimate paragraph).”
Regarding claim 7, Seyfioglu, as modified by Jia, discloses that “the classifying the radar data comprises using a trained neural network (convolutional autoencoder is fine-tuned [trained] by optimizing a cost function after the decoding part of the network is removed and fully connected layers and a softmax classifier are added at the end of the network – Seyfioglu, sec. IV(C), penultimate paragraph; see also sec. I, first paragraph (disclosing that the classification is of radar data)).”
Regarding claim 9, Seyfioglu, as modified by Jia, discloses that “obtaining the radar data from the scene comprises receiving and sampling a radar signal reflected from the scene (radar measurements of human activity were made in an indoor laboratory environment spanning a range of 1-5 m using an NI-USRP 2922 model software-defined radio platform programmed to transmit a continuous wave signal at 4 GHz; measurements were taken such that the direction of motion was directly aligned with the center of the antenna beam pattern [i.e., the radar data are gathered by reflection of the transmission off of the human] – Seyfioglu, sec. II, first paragraph; see also sec. III(A) (disclosing that some of the features are computed from coefficients of the discrete-cosine transform, i.e., the data are sampled)).”
Regarding claim 10, Seyfioglu, as modified by Jia, discloses “[a]n apparatus for classifying radar data from a scene, the apparatus comprising:
one or more interfaces configured to receive radar data from the scene (radar measurements of human activity were made in an indoor laboratory environment spanning a range of 1-5 m using an NI-USRP 2922 model software-defined radio platform programmed to transmit a continuous wave signal at 4 GHz; measurements were taken such that the direction of motion was directly aligned with the center of the antenna beam pattern [implying the existence of an interface to receive the measurements] – Seyfioglu, sec. II, first paragraph); and
one or more processing devices configured to perform the method of claim 1 (deep learning networks were run on a system with an NVIDIA TESLA K 80 GPU with 24 GB of RAM and an INTEL XEON E5 2683 processor – Seyfioglu, sec. V(C), first paragraph).”
Regarding claim 11, Seyfioglu, as modified by Jia, discloses [a] non-transitory storage medium with instructions stored thereon, where the instructions, when executed by a computer, a processor or a programmable hardware component enable the computer, the processor or the programmable hardware component to perform the method of claim 1 (deep learning networks were run on a system with an NVIDIA TESLA K 80 GPU with 24 GB of RAM [non-transitory storage medium with instructions] and an INTEL XEON E5 2683 processor – Seyfioglu, sec. V(C), first paragraph).”
Regarding claim 12, Seyfioglu discloses “[a] method for training one or more neural networks to classify radar data, the method comprising:
obtaining classified radar data from a scene (radar measurements of human activity were made in an indoor laboratory environment spanning a range of 1-5 m using an NI-USRP 2922 model software-defined radio platform programmed to transmit a continuous wave signal at 4 GHz – Seyfioglu, sec. II, first paragraph; see also sec. IV(A), penultimate paragraph (indicating that the samples are labeled, i.e., classified));
determining classified cadence-velocity data and classified micro range-Doppler data from the radar data (spectrograms were used to represent the time-frequency distribution of the measured micro-Doppler signature – Seyfioglu, sec. II(A), first paragraph; physical features are direct measurements of properties of the spectrogram or cadence velocity diagram – id. at sec. III(A)); …
training a second autoencoder-decoder pair based on the classified range Doppler data to obtain range-Doppler feature vectors from the second trained autoencoder (physical features include the overall Doppler frequency and transform-based features computed from the discrete cosine transform of the micro-Doppler frequency – Seyfioglu, sec. III(A); for a given input vector, the encoder of an autoencoder computes a nonlinear mapping of the inputs [to produce a feature vector, which is a range-Doppler feature vector when range-Doppler inputs are given]; encoded features are then decoded to reconstruct the given input vector x – id. at sec. IV(A), second paragraph; see also sec. V(A), second paragraph (disclosing that a convolutional autoencoder is used), sec. III(A) (disclosing that the features input, and therefore the features reconstructed, are Doppler-related features)); and
training a classifier based on the classified radar data … and the range-Doppler feature vectors (after unsupervised pretraining [using the reconstruction loss information], the decoder is removed and the remaining encoder components are trained in a supervised manner by adding a softmax classifier with 12 neurons after the encoder – Seyfioglu, sec. IV(A), p. 1714, first full paragraph; see also sec. I, first paragraph (disclosing that the classification is of radar data)).”
Seyfioglu appears not to disclose explicitly the further limitations of the claim. However, Jia discloses “training a first autoencoder-decoder pair based on the classified cadence-velocity data to obtain cadence-velocity feature vectors from the first trained autoencoder (autoencoder consists of an encoder [autoencoder] and a decoder [decoder]; the aim of the stacked autoencoder is compressing the input into a middle layer and then decompressing [decoding] it back, and make the output [i.e., train the network to produce output] as similar to the input as possible – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)); [and] …
training a classifier based on … the cadence-velocity feature vectors (after training [i.e., based on the minimization of the reconstruction loss], the middle layer of the stacked autoencoder containing the compressed representation of the input is drawn out and connected to a softmax layer to realize classification using a Bayes optimizer – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Seyfioglu to classify radar data using an autoencoder accepting cadence-velocity data, as disclosed by Jia, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the classification to protect individual privacy by avoiding the use of cameras. See Jia, sec. I, first paragraph.
Regarding claim 13, the rejection of claim 12 is incorporated. Seyfioglu further discloses that “training the second autoencoder-decoder pair comprises training with respect to optimized reconstruction losses … at a decoder of the second autoencoder-decoder pair (during unsupervised pretraining [training], the network [including the decoder] tries to minimize [optimize] the reconstruction error [loss] by adjusting its weights and biases – Seyfioglu, sec. IV(A), third paragraph).”
Seyfioglu appears not to disclose explicitly the further limitations of the claim. However, Jia discloses that “training the first autoencoder-decoder pair … comprises training … at a decoder of the first autoencoder-decoder pair (autoencoder consists of an encoder and a decoder; the aim of the stacked autoencoder (SAE) is compressing the input into a middle layer and then decompressing it back, and make the output as similar to the input as possible; after optimization [training], the SAE yields 92% accuracy – Jia, sec. V(A)) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Seyfioglu to train an autoencoder with a first trained encoder and decoder, as disclosed by Jia, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would help the system to yield tolerable accuracy on classification tasks. See Jia, sec V(A).
Regarding claim 15, Seyfioglu, as modified by Jia, discloses that “the classified radar data comprise[] information on whether there is a human present in the scene (proposed convolutional autoencoder is used to discriminate 12 classes of aided and unaided human activities [which implicitly classifies whether a human is present] – Seyfioglu, sec. I, penultimate paragraph); and
training of the classifier comprises training the classifier to output information on whether a human is present in the scene (proposed convolutional autoencoder is used to discriminate 12 classes of aided and unaided human activities [which implicitly classifies whether a human is present] – Seyfioglu, sec. I, penultimate paragraph).”
Regarding claim 17, Seyfioglu, as modified by Jia, discloses “[a]n apparatus for training one or more neural networks to classify radar data, the apparatus comprising:
one or more interfaces configured to receive radar data from the scene (radar measurements of human activity were made in an indoor laboratory environment spanning a range of 1-5 m using an NI-USRP 2922 model software-defined radio platform programmed to transmit a continuous wave signal at 4 GHz; measurements were taken such that the direction of motion was directly aligned with the center of the antenna beam pattern [implying the existence of an interface to receive the measurements] – Seyfioglu, sec. II, first paragraph); and
one or more processing devices configured to perform the method of claim 12 (deep learning networks were run on a system with an NVIDIA TESLA K 80 GPU with 24 GB of RAM and an INTEL XEON E5 2683 processor – Seyfioglu, sec. V(C), first paragraph).”
Regarding claim 18, Seyfioglu, as modified by Jia, discloses “[a] non-transitory storage medium with instructions stored thereon, where the instructions, when executed by a computer, a processor or a programmable hardware component enable the computer, the processor or the programmable hardware component to perform the method of claim 12 (deep learning networks were run on a system with an NVIDIA TESLA K 80 GPU with 24 GB of RAM [non-transitory storage medium with instructions] and an INTEL XEON E5 2683 processor – Seyfioglu, sec. V(C), first paragraph).”
Regarding claim 19, Seyfioglu discloses “[a] radar system comprising: …
a second trained autoencoder configured to encode micro range-Doppler data to obtain a range-Doppler feature vector (physical features include the overall Doppler frequency and transform-based features computed from the discrete cosine transform of the micro-Doppler frequency – Seyfioglu, sec. III(A); for a given input vector, the encoder of an autoencoder computes a nonlinear mapping of the inputs [to produce a feature vector, which is a range-Doppler feature vector when range-Doppler inputs are given] – id. at sec. IV(A), second paragraph; see also sec. V(A), second paragraph (disclosing that a convolutional autoencoder is used)); …
a second trained decoder configured to decode the range-Doppler feature vector to obtain reconstructed range-Doppler data (encoded features are then decoded to reconstruct the given input vector x – Seyfioglu, sec. IV(A), second paragraph; see also sec. III(A) (disclosing that the features input, and therefore the features reconstructed, are Doppler-related features));
a trained neural network configured to classify radar data based on … second reconstruction loss information (after unsupervised pretraining [using the reconstruction loss information], the decoder is removed and the remaining encoder components are trained in a supervised manner by adding a softmax classifier with 12 neurons after the encoder – Seyfioglu, sec. IV(A), p. 1714, first full paragraph; see also sec. I, first paragraph (disclosing that the classification is of radar data));
one or more processors (deep learning networks were run on a system with an NVIDIA TESLA K 80 GPU with 24 GB of RAM and an INTEL XEON E5 2683 processor – Seyfioglu, sec. V(C), first paragraph); and
one or more memories coupled to the one or more processors with instructions stored thereon (deep learning networks were run on a system with an NVIDIA TESLA K 80 GPU with 24 GB of RAM [memory with instructions] and an INTEL XEON E5 2683 processor – Seyfioglu, sec. V(C), first paragraph), wherein the instructions, when executed by the one or more processors, enable the one or more processors to perform the steps of:
obtaining the radar data from a radar sensor (radar measurements of human activity were made in an indoor laboratory environment spanning a range of 1-5 m using an NI-USRP 2922 model software-defined radio platform programmed to transmit a continuous wave signal at 4 GHz – Seyfioglu, sec. II, first paragraph),
determining the cadence-velocity data and the micro range-Doppler data from the radar data (spectrograms were used to represent the time-frequency distribution of the measured micro-Doppler signature – Seyfioglu, sec. II(A), first paragraph; physical features are direct measurements of properties of the spectrogram or cadence velocity diagram – id. at sec. III(A)), [and] …
determining the second reconstruction loss information based on the micro range-Doppler data and the reconstructed range-Doppler data (during unsupervised pretraining, the network tries to minimize the reconstruction error [loss information] by adjusting its weights and biases – Seyfioglu, sec. IV(A), third paragraph; see also sec. III(A) (disclosing that the features input, and therefore the features reconstructed, are Doppler-related features)).”
Seyfioglu appears not to disclose explicitly the further limitations of the claim. However, Jia discloses “a first trained autoencoder configured to encod[e] cadence velocity data to obtain a cadence-velocity feature vector (autoencoder consists of an encoder [first trained autoencoder] and a decoder; the aim of the stacked autoencoder is compressing [encoding] the input into a middle layer and then decompressing it back, and make the output as similar to the input as possible – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)); …
a first trained decoder configured to decode the cadence-velocity feature vector to obtain reconstructed cadence-velocity data (autoencoder consists of an encoder and a decoder [first trained decoder]; the aim of the stacked autoencoder is compressing the input into a middle layer and then decompressing [decoding] it back, and make the output as similar to the input as possible – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)); …
first reconstruction loss information (autoencoder consists of an encoder and a decoder; the aim of the stacked autoencoder is compressing the input into a middle layer and then decompressing it back, and make the output as similar to the input as possible [i.e., minimize first reconstruction loss information] – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)) …; … [and]
determining the first reconstruction loss information based on the cadence-velocity data and the reconstructed cadence-velocity data (autoencoder consists of an encoder and a decoder; the aim of the stacked autoencoder is compressing the input into a middle layer and then decompressing it back, and make the output as similar to the input as possible [i.e., minimize first reconstruction loss information] – Jia, sec. V(A); see also sec. III(B) (disclosing that the input feature vector is handcrafted from, inter alia, a cadence velocity diagram)) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Seyfioglu to classify radar data using an autoencoder accepting cadence-velocity data, as disclosed by Jia, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the classification to protect individual privacy by avoiding the use of cameras. See Jia, sec. I, first paragraph.
Regarding claim 20, Seyfioglu, as modified by Jia, discloses that the system “further compris[es] the radar sensor (radar measurements of human activity were made in an indoor laboratory environment spanning a range of 1-5 m using an NI-USRP 2922 model software-defined radio platform programmed to transmit a continuous wave signal at 4 GHz [implying the existence of a radar sensor] – Seyfioglu, sec. II, first paragraph).”
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Seyfioglu in view of Jia and further in view of Lyng et al. (US 20240095591) (“Lyng”).
Regarding claim 2, the rejection of claim 1 is incorporated. Seyfioglu further discloses “radar data” and “the range-Doppler feature vector and the cadence-velocity [data]”, as shown by the rejection of claim 1.
Neither Seyfioglu nor Jia appears to disclose explicitly the further limitations of the claim. However, Lyng discloses that “classifying the … data further comprises combining the [first] feature vector and the [second] feature vector to obtain a combined feature vector using multihead-attention (multi-head attention layer generates and/or outputs a combined vector representing an attention-driven vector of the multiple inputs [feature vectors including a first and second] – Lyng, paragraph 131)1.”
Lyng and the instant application both relate to multihead attention and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu and Jia to combine the vectors using a multi-head attention layer, as disclosed by Lyng, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to exploit parallelism by computing multiple vectors simultaneously and combining them. See Lyng, paragraph 40.
Regarding claim 3, the rejection of claim 2 is incorporated. Seyfioglu further discloses “the range-Doppler feature vector and the cadence-velocity [data]”, as shown by the rejection of claim 1.
Neither Seyfioglu nor Jia appears to disclose explicitly the further limitations of the claim. However, Lyng discloses that “combining the [first] feature vector and the [second] feature vector comprises using different weightings of different elements of the [first] feature vector and the [second] feature vector to obtain the combined feature vector (combined vector comprises a weighted average of each time-by-code vector in the set of time-by-code vectors [feature vectors including a first and second], wherein the weighted average [i.e., using different weights of different elements] is determined based on a set of weights generated utilizing the attention model comprising a multi-head attention layer – Lyng, claim 10).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu and Jia to perform a weighted combination of the feature vectors using multi-head attention, as disclosed by Lyng, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to exploit parallelism by computing multiple vectors simultaneously and combining them. See Lyng, paragraph 40.
Claims 4, 8, 14, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Seyfioglu in view of Jia and further in view of Hilprecht et al. (US 20200193298) (“Hilprecht”).
Regarding claim 4, the rejection of claim 1 is incorporated. Seyfioglu further discloses “the radar data”, “the second trained autoencoder”, and “the second trained decoder”, as shown in the rejection of claim 1.
Seyfioglu appears not to disclose explicitly the further limitations of the claim. However, Jia discloses a “first trained autoencoder” and a “first trained decoder”, as shown in the rejection of claim 1. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Seyfioglu to use a first trained autoencoder and decoder, as disclosed by Jia, for substantially the same reasons as given in the rejection of claim 1.
Neither Seyfioglu nor Jia appears to disclose explicitly the further limitations of the claim. However, Hilprecht discloses that “classifying the … data comprises providing information on whether the … data lie[] in a data distribution used to train the [models] (processor computes a reconstruction error by determining a value of a function associated with a distance between the reconstructed data and the data; processor can generate an indication indicating the confidence with which data records [data] are assumed to be present in the training data [data used to train the model] – Hilprecht, paragraph 19; see also paragraphs 18 and 20 (disclosing that the autoencoder minimizes a KL divergence between a distribution of the latent variables and a continuous probability distribution and that the indication is based on the reconstruction error [i.e., the indication of whether the data records are in the training data is an indication of whether each data record lies within the training data’s distribution])) ….”
Hilprecht and the instant application both relate to variational autoencoders and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu and Jia to determine whether a data point lies within the training data’s distribution, as disclosed by Hilprecht, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to generate new instances that are similar to the original training dataset, thereby augmenting the training dataset and making it more useful for training. See Hilprecht, paragraph 17.
Regarding claim 8, the rejection of claim 1 is incorporated. Seyfioglu further discloses that “the second trained autoencoder and the second trained decoder form a second … autoencoder (for a given input vector, the encoder of an autoencoder computes a nonlinear mapping of the inputs; encoded features are then decoded to reconstruct the given input vector x – Seyfioglu, sec. IV(A), second paragraph [encoder + decoder = second autoencoder]).”
Seyfioglu appears not to disclose explicitly the further limitations of the claim. However, Jia discloses that “the first trained autoencoder and the first trained decoder form a first trained … autoencoder (autoencoder consists of an encoder and a decoder; the aim of the stacked autoencoder is compressing [encoding] the input into a middle layer and then decompressing [decoding] it back, and make the output as similar to the input as possible – Jia, sec. V(A)) ….” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Seyfioglu to form an autoencoder with a first trained encoder and decoder, as disclosed by Jia, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would help the system to yield tolerable accuracy on classification tasks. See Jia, sec V(A).
Neither Seyfioglu nor Jia appears to disclose explicitly the further limitations of the claim. However, Hilprecht discloses a “generative autoencoder (unlike other autoencoders, VAEs are generative autoencoders, meaning that they can generate new instances that look similar to the original dataset used for training – Hilprecht, paragraph 17) ….”
Hilprecht and the instant application both relate to generative autoencoders and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu and Jia to use a generative autoencoder, as disclosed by Hilprecht, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to generate new instances that are similar to the original training dataset, thereby augmenting the training dataset and making it more useful for training. See Hilprecht, paragraph 17.
Regarding claim 14, the rejection of claim 12 is incorporated. Seyfioglu further discloses “feature vector[s]”, a “ range-Doppler feature vector” and “cadence-velocity [data]”, as shown in the rejection of claim 12.
Neither Seyfioglu nor Jia appears to disclose explicitly the further limitations of the claim. However, Hilprecht discloses that “the training the classifier further comprises training the classifier to determine an energy score based on a pair of … [data points] (processor computes a reconstruction error by determining a value of a function associated with a distance between the reconstructed data and the data; processor can generate an indication indicating the confidence [energy score] with which data records [including at least two data points] are assumed to be present in the training data – Hilprecht, paragraph 19; see also paragraphs 18 and 20 (disclosing that the autoencoder minimizes a KL divergence between a distribution of the latent variables and a continuous probability distribution and that the indication is based on the reconstruction error)).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu and Jia to determine an energy score based on the data points, as disclosed by Hilprecht, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to generate new instances that are similar to the original training dataset, thereby augmenting the training dataset and making it more useful for training. See Hilprecht, paragraph 17.
Regarding claim 16, the rejection of claim 12 is incorporated. Seyfioglu further discloses “radar data”, as shown in the rejection of claim 12.
Neither Seyfioglu nor Jia appears to disclose explicitly the further limitations of the claim. However, Hilprecht discloses that the “training of the classifier further comprises training the classifier to provide information whether input … data lie[] within a distribution of the classified … data (processor computes a reconstruction error by determining a value of a function associated with a distance between the reconstructed data and the data; processor can generate an indication indicating the confidence with which data records [classified data] are assumed to be present in the training data [input data] – Hilprecht, paragraph 19; see also paragraphs 18 and 20 (disclosing that the autoencoder minimizes a KL divergence between a distribution of the latent variables and a continuous probability distribution and that the indication is based on the reconstruction error [i.e., the indication of whether the data records are in the training data is an indication of whether each data record lies within the training data’s distribution])).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu and Jia to determine whether a data point lies within the training data’s distribution, as disclosed by Hilprecht, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to generate new instances that are similar to the original training dataset, thereby augmenting the training dataset and making it more useful for training. See Hilprecht, paragraph 17.
Claim 6 is are rejected under 35 U.S.C. 103 as being unpatentable over Seyfioglu in view of Jia and further in view of Hilprecht and Hu (US 20220301689) (“Hu”).
Regarding claim 6, the rejection of claim 1 is incorporated. Seyfioglu further discloses “feature vector[s]”, “radar data” and “the range-Doppler feature vector and the cadence-velocity [data]”, as shown in the rejection of claim 1.
Neither Seyfioglu nor Jia appears to disclose explicitly the further limitations of the claim. However, Hilprecht discloses that “classifying the … data comprises determining an energy score based on the … [two data points] …, wherein the energy score indicates a compatibility of the … [two data points] with data used for training the classifying (processor computes a reconstruction error by determining a value of a function associated with a distance between the reconstructed data and the data; processor can generate an indication indicating the confidence [energy score] with which data records [including at least two data points] are assumed to be present in the training data [data used for training the classifying] – Hilprecht, paragraph 19; see also paragraphs 18 and 20 (disclosing that the autoencoder minimizes a KL divergence between a distribution of the latent variables and a continuous probability distribution and that the indication is based on the reconstruction error [i.e., the indication of whether the data records are in the training data is an indication of whether each data record is compatible with the training data])).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu and Jia to determine an energy score based on the data points, as disclosed by Hilprecht, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to generate new instances that are similar to the original training dataset, thereby augmenting the training dataset and making it more useful for training. See Hilprecht, paragraph 17.
Neither Seyfioglu, Jia, nor Hilprecht appears to disclose explicitly the further limitations of the claim. However, Hu discloses “using the energy score to classify the … data (energy-based out-of-distribution detection [i.e., classifying data as in- or out-of-distribution] is used to map samples to an energy-based anomaly score, yielding AUC scores that are analogous to the anomaly scores in the one-class classification approach – Hu, paragraph 46) ….”
Hu and the instant application both relate to machine learning and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Seyfioglu, Jia, and Hilprecht to use an energy score to classify the data, as disclosed by Hu, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the system to detect anomalies in data, thereby preventing modeling errors due to their inclusion in the data. See Hu, paragraph 46.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN C VAUGHN whose telephone number is (571)272-4849. The examiner can normally be reached M-R 7:00a-5:00p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at 571-272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN C VAUGHN/ Primary Examiner, Art Unit 2125
1 Examiner has checked to ensure that all portions of Lyng relied upon in the rejection of claims 2-3 find support in provisional application 63/376,460, to which Lyng claims priority.
Prosecution Insights