DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 12/13/2024 is/are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is/are being considered by the examiner.
Claim Objections
Claim(s) 2, 10 is/are objected to because of the following informalities:
In Claim 2, the word “comprises” should be “comprising”
In Claim 10, the word “comprises” should be “comprising”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims
particularly pointing out and distinctly claiming the subject matter which the
inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out
and distinctly claiming the subject matter which the applicant regards as his
invention.
Claim(s) 1-2, 11, and 19 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 1, the claim recites the limitation “generating a first range-Doppler map … based on the radar data and a second range-Doppler map.” It is unclear whether this limitation means that the first range-Doppler map is generated based on both the radar data and the second range-Doppler map, or that a first range-Doppler map is generated based on the radar data and that a second range-Doppler map is also generated. For examination purposes, the limitation is interpreted as meaning that a first range-Doppler map is generated based on the radar data and that a second range-Doppler map is also generated.
Regarding Claim 2, the claim recites the limitation “the first range-Doppler map of each of a plurality of frames.” There is insufficient antecedent basis for this limitation in the claim. Additionally, it is unclear whether the limitation means that the first range-Doppler map contains a plurality of frames, or that the first range-Doppler map comprises a plurality of first range-Doppler maps which correspond to a plurality of frames. Based on the specification ([00115]), the limitation is interpreted as meaning the first range-Doppler map comprises a plurality of first range-Doppler maps which correspond to a plurality of frames.
Regarding Claim 11, the claim recites the limitation “classify the gesture probability above a predetermined value.” It is unclear whether this means each gesture probability is assigned a value above a predetermined value, or that probabilities above a predetermined value are identified. Based on the specification ([00132]), the limitation is interpreted as meaning probabilities above a predetermined value are identified. This rejection also applies to the corresponding limitation in Claim 19.
Regarding Claim 11, the claim recites the limitation “count the gesture probability.” It is unclear whether this limitation means that the probability values are counted/summed, or that occurrences of probabilities above the predetermined value are counted. For examination purposes, the limitation is interpreted as meaning that occurrences of probabilities above the predetermined value are counted. This rejection also applies to the corresponding limitation in Claim 19.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C.
102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the
statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a
new ground of rejection if the prior art relied upon, and the rationale supporting the rejection,
would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the
claimed invention is not identically disclosed as set forth in section 102, if the
differences between the claimed invention and the prior art are such that the
claimed invention as a whole would have been obvious before the effective filing
date of the claimed invention to a person having ordinary skill in the art to which
the claimed invention pertains. Patentability shall not be negated by the manner in
which the invention was made.
Claim(s) 1, 9, 12-14, 17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Wang et al., “Adaptive framework towards radar-based diversity gesture recognition with range-Doppler signatures,” 2022) in view of Choi (Choi et al., “Short-Range Radar Based Real-Time Hand Gesture Recognition Using LSTM Encoder,” 2019).
Regarding Claim 1, Wang teaches: A gesture sensing device comprising:
a preprocessing unit receiving radar data from a sensor ([p. 1543]: “consecutive radar signals”; “A public gesture data set collected by the Soli radar is adopted in this study for evaluation”), and generating a first range-Doppler map including a distance to an object and information about a relative speed based on the radar data and a second range-Doppler map ([p. 1539]: “trajectory RDM (t‐RDM)”; [p. 1541]: “RDMs characterise the targets uniquely with respect to the range and velocity”; Figure 2);
a motion sensing unit receiving the first range-Doppler map and generating a motion information … based on the first range-Doppler map ([p. 1539]: “a trajectory RDM (t‐RDM) … can effectively record the movement trajectory”; [p. 1541]: “records a trajectory about range and velocity in t‐RDM”); and
a gesture sensing unit receiving the second range-Doppler map and the motion information, and generating a gesture probability by classifying gesture features of the object based on the second range-Doppler map ([p. 1539]: “a two‐pathway convolutional neural network (CNN) is proposed with dual‐channel inputs especially for the t‐RDM and enhanced t‐RDM”; [p. 1542]: “a module composed of convolutional layers in each pathway can be regarded as a feature extractor”; “the last fully connected layer with n classes of neurons is adopted as a classification layer and outputs the predicted probabilities”),
wherein the second range-Doppler map is generated in the preprocessing unit using the first range-Doppler map and the motion information ([p. 1539]: “an enhanced t‐RDM is proposed to highlight the trajectory in the raw t‐RDMs”; [p. 1541-1542]; Figure 2).
Wang does not explicitly teach:
generating motion information including a motion start and a motion end based on a signal strength calculated based on the first range-Doppler map.
However, Choi is in the field of radar gesture recognition (Choi [Abstract]) and teaches:
generating motion information including a motion start and a motion end based on a signal strength calculated based on a range-Doppler map (Choi [p. 33612]: “xt represents the sum of all pixel values on the RDM of all four channels”; “detected intervals”; Equation (8)); and
selecting range-Doppler map data, based on the motion information, as an input to a gesture recognition algorithm (Choi [p. 33612]: “The RDM data contained in the detected interval is used as an input to the gesture recognition algorithm”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and generate the motion information including a motion start and a motion end based on a signal strength calculated based on a range-Doppler map, as taught by Choi, with a reasonable expectation of success. Applying Choi’s motion interval detection to Wang’s gesture recognition system yields the predictable result of selecting relevant motion-related data in Wang’s range-Doppler map and discarding unnecessary data in order to improve gesture recognition.
Regarding Claim 14, Wang teaches: A gesture sensing method comprising:
generating a first range-Doppler map including a distance to an object and information about a relative speed based on radar data ([p. 1539]: “trajectory RDM (t‐RDM)”; [p. 1541]: “RDMs characterise the targets uniquely with respect to the range and velocity”; Figure 2);
generating motion information … based on the first range-Doppler map ([p. 1539]: “a trajectory RDM (t‐RDM) … can effectively record the movement trajectory”; [p. 1541]: “records a trajectory about range and velocity in t‐RDM”);
generating a second range-Doppler map different from the first range-Doppler map based on the first range-Doppler map and the motion information ([p. 1539]: “an enhanced t‐RDM is proposed to highlight the trajectory in the raw t‐RDMs”; [p. 1541-1542]; Figure 2); and
generating a gesture probability by classifying gesture features of the object based on the second range-Doppler map ([p. 1539]: “a two‐pathway convolutional neural network (CNN) is proposed with dual‐channel inputs especially for the t‐RDM and enhanced t‐RDM”; [p. 1542]: “a module composed of convolutional layers in each pathway can be regarded as a feature extractor”; “the last fully connected layer with n classes of neurons is adopted as a classification layer and outputs the predicted probabilities”).
Wang does not explicitly teach:
generating motion information including a motion start and a motion end based on a signal strength calculated based on the first range-Doppler map.
However, Choi is in the field of radar gesture recognition (Choi [Abstract]) and teaches:
generating motion information including a motion start and a motion end based on a signal strength calculated based on a range-Doppler map (Choi [p. 33612]: “xt represents the sum of all pixel values on the RDM of all four channels”; “detected intervals”; Equation (8)); and
selecting range-Doppler map data, based on the motion information, as an input to a gesture recognition algorithm (Choi [p. 33612]: “The RDM data contained in the detected interval is used as an input to the gesture recognition algorithm”).
The rationale to modify Wang with the teachings of Choi persist from Claim 1.
Regarding Claims 9 and 17, Wang as modified teaches: wherein the gesture sensing unit includes a convolution neural network (CNN) ([p. 1539]: “a two‐pathway convolutional neural network (CNN)”).
Wang as modified does not explicitly teach – but Choi teaches: wherein the gesture sensing unit includes a long short-term memory (LSTM) (Choi [p. 33611]: “LSTM”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and use an LSTM to output a gesture probability, as taught by Choi, with a reasonable expectation of success. The combination of Wang and Choi yields the predictable result of using both a CNN and an LSTM to detect and output gesture probabilities, and the LSTM is beneficial for extracting global temporal features of the radar data in order to improve gesture recognition (Choi [p. 33611]).
Regarding Claims 12 and 20, Wang as modified teaches: wherein the radar data is a signal received in a millimeter wave ([p. 1539]: “millimetre‐wave radar”; “Soli radar”; [p. 1543]: “60GHz”).
Regarding Claim 13, Wang as modified teaches: wherein the preprocessing unit performs a fast Fourier transform on the radar data to generate the first range-Doppler map ([p. 1541]: “RDMs … can be obtained by 2‐D Fourier transform on a certain number of consecutive radar signals”).
Claim(s) 5, 8, 10, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Wang et al., “Adaptive framework towards radar-based diversity gesture recognition with range-Doppler signatures,” 2022) in view of Choi (Choi et al., “Short-Range Radar Based Real-Time Hand Gesture Recognition Using LSTM Encoder,” 2019), as applied to Claims 1 and 14 above, and further in view of Qiu (US 2020/0356178).
Regarding Claim 5, Wang as modified teaches: wherein the preprocessing unit includes:
a range-Doppler map generating unit configured to generate the first range-Doppler map ([p. 1541]: “RDMs … can be obtained by 2‐D Fourier transform on a certain number of consecutive radar signals”); and
a range-Doppler map conversion unit configured to generate the second range-Doppler map ([p. 1542]: “The corresponding enhanced t‐RDM is obtained by Equation (2)”).
Wang as modified does not explicitly teach:
a peak sensing unit configured to sense the distance based on a peak of the signal strength; or
a beamforming unit configured to calculate an angle to the object based on the first range-Doppler map.
However, Qiu is in the field of radar gesture recognition (Qiu [Abstract]) and teaches:
a peak sensing unit configured to sense the distance based on a peak of the signal strength (Qiu [0117]: “the tap detector 514 selects the tap(s) with the largest (or maximum) power in the in RDM 512”; [0118]: “The range information 518 can be based directly from the index of the detected taps.”); and
a beamforming unit configured to calculate an angle to the object based on the first range-Doppler map (Qiu [0118]: “Beamforming 520 generates the Azimuth and Elevation angle feature 522”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and use a peak sensing unit to determine a distance based on a peak of the signal strength, and use a beamforming unit to calculate an angle to the object based on the first range-Doppler map, as taught by Qiu, with a reasonable expectation of success. Applying Qiu’s peak sensing and beamforming to Wang’s gesture recognition system yields the predictable result of determining distance and angle of the object in order to improve gesture recognition (Qiu [0104]).
Regarding Claim 8, Wang as modified does not explicitly teach – but Qiu teaches: wherein the preprocessing unit further includes a motion log unit (Qiu [0098]: “feature buffer 53”), and
wherein the motion log unit receives the motion information, the distance and the angle (Qiu [0104]), and records and outputs the distance and the angle during a period between the motion start and the motion end (Qiu [0098]: “The feature buffer 530 stores a history of the features that are extracted by the feature extractor 510.”; [0104]).
angle feature 522”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and use a motion log to receive, record, and output distance and angle information during a period between the motion start and the motion end, as taught by Qiu, with a reasonable expectation of success. Applying Qiu’s feature buffering to Wang’s gesture recognition system yields the predictable result of storing the object’s distance and angle history in order to improve gesture recognition (Qiu [0104]).
Regarding Claims 10 and 18, Wang as modified does not explicitly teach – but Qiu teaches: a postprocessing unit configured to receive the motion information, the distance, the angle and the gesture probability, and to output a gesture of the object (Qiu [0104]: “The metrics 550 include a movement range 552, a Doppler shift range 554, an angle change range 556, and a power level change range 558. The various metric 550 can be used by the logical gate 570 to determine whether the output of the gesture classifier 560 meets a logical baseline.”; [0133]: “When the identified gesture corresponds to the calculated metrics, an output (similar to the output 575 of FIGS. 5A, 5B, and 5C) is generated in step 598.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and use a motion log to receive the motion information, the distance, the angle and the gesture probability, and to output a gesture of the object, as taught by Qiu, with a reasonable expectation of success. Applying Qiu’s gesture validation to Wang’s gesture recognition system yields the predictable result of using the motion information, the distance, the angle and the gesture probability to determine if the gesture output is probable in order to improve gesture detection (Qiu [0104]).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang (Wang et al., “Adaptive framework towards radar-based diversity gesture recognition with range-Doppler signatures,” 2022) in view of Choi (Choi et al., “Short-Range Radar Based Real-Time Hand Gesture Recognition Using LSTM Encoder,” 2019) and Qiu (US 2020/0356178), as applied to Claim 5 above, and further in view of Shin (US 2022/0050175).
Regarding Claim 6, Wang as modified does not explicitly teach: wherein the preprocessing unit further includes a filter unit configured to receive the radar data, and wherein the filter unit includes an infinite impulse response filter.
However, Shin is in the field of radar (Shin [Abstract]) and teaches:
A radar system that may be used in gesture detection (Shin [0068]: “Gesture detection may be performed by other hardware or software components that use the output of radar subsystem”) includes a filter unit configured to receive the radar data, and wherein the filter unit includes an infinite impulse response filter (Shin [0114]: “an infinite impulse response (IIR) filter is incorporated as part of movement filter 211. Specifically, a single-pole IIR filter may be implemented to filter out raw waveform data that is not indicative of movement.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wang and use an infinite impulse response filter configured to filter the radar data, as taught by Shin, with a reasonable expectation of success. Applying Shin’s infinite impulse response filtering to Wang’s gesture recognition system yields the predictable result of removing data that is not indicative of movement in order to improve gesture recognition.
Allowable Subject Matter
Claims 2-4, 7, 11, 15-16, and 19 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and/or to include all of the limitations of the base claim and any intervening claims.
Regarding Claims 2, 7, 11, 15-16, and 19, the prior art does not appear to teach the combined limitations in the claims. Claims 3-4 would be allowable by virtue of dependence on Claim 2.
Conclusion
The cited references made of record in the contemporaneously filed PTO-892 form and not relied upon in the instant office action are considered pertinent to Applicant’s disclosure, and may have one or more of the elements in Applicant’s disclosure and at least Claims 1 and 14.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH Y. ZHU whose telephone number is (571) 270-0170. The examiner can normally be reached Monday-Friday, 8AM-4PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vladimir Magloire, can be reached on (571) 270-5144. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH YI MIN ZHU/Examiner, Art Unit 3648
/BRADY W FRAZIER/Primary Examiner, Art Unit 3648
Prosecution Insights