Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/13/2024 is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 6-8, 12-15, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hayashi (WO 2024072466 A1), in view of Baur (US 20240125920 A1) and further in view of DuHadway (US 20120310504 A1).
Regarding claims 1, 8 and 15, Hayashi discloses a processor ([0004] when executed by at least one processor,), configured to:
receiving motion capture data of a target from a camera ("[0014] The computing device 102 may detect, based on one or more radar-receive signals reflected off the user’s hand, that this motion is a gesture, rather than a non-gesture movement, based on radarsignal characteristics of the radar-receive signals.
[0032] Other devices may also be used, such as a security camera, a baby monitor, a Wi-Fi® router, a drone, a trackpad, a drawing pad, a netbook, an e-reader, other forms of a home-automation and control systems, a wall display, a virtual-reality headset, another vehicle (e.g., an e-bike or plane), and other home appliances, to name just a few examples.");
generating a first set of motion trajectories from the motion capture data ([0252] In this example, the confidence threshold criteria is not met, and as a result the ambiguous gesture 3402 cannot be recognized as a particular gesture but is instead associated with one or more known gestures (e.g., a first gesture 3404 and a second gesture 3406).);
generating a first set of augmented motion trajectories using a set of data augmentation functions on the first set of motion trajectories ([0128] Radar-augmentation techniques described in this disclosure include determination of random or predetermined phase rotations and/or magnitude scaling of data corresponding to one or more radar-signal characteristics. By implementing these radar-augmentation techniques, the computing device 102 may reduce the amount of gesture training needed to accurately recognize gestures to a desired level of confidence.);
generating a radar cross-section of the target using the motion capture data to perform at least one of gesture recognition or activity detection ([0140] Examples of positive data may include radar-signal characteristics correlated with a gesture class (e.g., tap, swipe, flick, point) or a specific user based on, for instance, radar cross section (RCS) data. Negative data, on the other hand, may include radar-signal characteristics that do not correlate, to the desired level of confidence, to one or more stored radar-signal characteristics of a gesture or user 104.). Hayashi does not explicitly disclose generating one or more synthetic electromagnetic (EM) signatures of one or more activities of the target
using the first set of augmented motion trajectories and the radar cross-section; and
training a machine learning model configured for EM signature-based gesture recognition or activity detection with a domain adaptation process using the one or more synthetic EM signatures.
In a similar field of endeavor of synthetic radio frequency (RF) images, Baur teaches generating one or more synthetic electromagnetic (EM) signatures of one or more activities of the target ([0041] In its most generic implementation, the process 1 comprises three steps, namely: having 11 a three-dimensional (3D) body model of a human; sampling 12 the 3D body model; and generating 13 the at least one synthetic RF image in accordance with an electromagnetic (EM) simulation of the 3D body sample.)
training a machine learning model configured for EM signature-based gesture recognition or activity detection with a domain adaptation process using the one or more synthetic EM signatures ("[0016] Mobilizing the 3D body model may comprise: retargeting the given motion capture data onto the body skeleton model of the 3D body model; and skinning the body surface mesh model of the retargeted 3D body model in accordance with a Linear Blend Skinning, LBS, algorithm and given mobilization weights.
[0017] The imaging transformation may comprise one of: a physical optics based electromagnetic, EM, simulation, and an inference by a machine learning, ML, algorithm being trained for RF imaging of 3D body samples."). Baur implicitly teaches using the first set of augmented motion trajectories and the radar cross-section ("[0417] Example 53: The method as recited by any of examples 47-52, the method further comprising: responsive to the determining that the performed gesture is the known gesture, augmenting the radar-signal characteristic of the performed gesture prior to storing the radarsignal characteristic, the augmenting including an interpolation or extrapolation of the radar-signal characteristic without requiring one or more additional radar-receive signals.
[0418] Example 54: The method as recited by example 53, wherein the radar-signal characteristic is associated with at least one complex-range Doppler map, the at least one complex range Doppler map comprising: a range dimension corresponding to a displacement of the performed gesture relative to the radar system, the displacement taken at a scattering center of the performed gesture; and a Doppler dimension corresponding to a velocity of the performed gesture relative to the radar system.").
It would have been obvious to one of ordinary skill in the art to combine the known system of gesture detection through em signatures, as disclosed by Hayashi, with the known teaching of synthetic em signature generation, as taught by Baur, in order to yield the predictable results of bypassing line-of-sight limitations, enhance spatial resolution without bulky hardware, and improve algorithm training for touchless human-computer interaction (HCI) and wearables.
Hayashi and Baur do not explicitly disclose but in a similar field of endeavor of Radar And GPS Localization, DuHadway teaches, in better detail, using the first set of augmented motion trajectories and the radar cross-section ([0039] In some embodiments, multiple thresholds may be useful in better classifying the detected objects. By way of example, a detected object that is moving at 8 MPH may be a bicycle, a running pedestrian, a deer, or a motorized vehicle. A detected object that is moving at 45 MPH is almost certainly a motorized vehicle. Additional data, such as the strength of the detected radar signal, may be used to augment the velocity data in classifying a detected object. Thus, a detected object with a radar cross-section above a certain threshold may be classified as a motorized vehicle even when the detected object is moving at a very low speed.).
It would have been obvious to one of ordinary skill in the art to combine the known system of gesture detection through synthetic em signatures, as disclosed by Hayashi and Baur, with the known teaching of trajectory detection, as taught by DuHadway, in order to yield the predictable results of mapping movement over time rather than just capturing isolated static poses with enhanced privacy and to provide sub-millimeter spatial resolution and highly accurate temporal mapping.
Regarding claims 2 and 13, Hayashi discloses wherein the domain adaptation process uses ("[0060] During operation, the transmitter 502 may pass electrical signals to the antenna 214, which may emit one or more radar-transmit signals 402-Y to probe the proximate region 106 for user presence and/or gestures. In particular, the waveform generator 504 may generate electrical signals with a specified waveform (e.g., specified amplitude, phase, frequency). The waveform generator 504 may additionally communicate information regarding the electrical signals to the system processor 218 for digital signal processing. If the radar-transmit signal 402- Y interacts with a user 104, then the radar system 108 may receive the radar-receive signals 404- Z on the receive channel 508. The radar-receive signal 404-Z (or multiple versions thereof) may be sent to the system processor 218 to enable user detection (using the user module 222 of the system media 220) and/or gesture detection (using the gesture module 224).
[0128] Radar-augmentation techniques described in this disclosure include determination of random or predetermined phase rotations and/or magnitude scaling of data corresponding to one or more radar-signal characteristics. By implementing these radar-augmentation techniques, the computing device 102 may reduce the amount of gesture training needed to accurately recognize gestures to a desired level of confidence.").
In a similar field of endeavor of synthetic radio frequency (RF) images, Baur teaches simulated EM signatures ([0041] In its most generic implementation, the process 1 comprises three steps, namely: having 11 a three-dimensional (3D) body model of a human; sampling 12 the 3D body model; and generating 13 the at least one synthetic RF image in accordance with an electromagnetic (EM) simulation of the 3D body sample.).
It would have been obvious to one of ordinary skill in the art to combine the known system of gesture detection through em signatures, as disclosed by Hayashi, with the known teaching of synthetic em signature generation, as taught by Baur, in order to yield the predictable results of bypassing line-of-sight limitations, enhance spatial resolution without bulky hardware, and improve algorithm training for touchless human-computer interaction (HCI) and wearables.
Regarding claims 6, 12 and 19, Hayashi discloses the radar cross-section is a time-varying radar cross-section of the target ([0075] A logic system may determine that the low confidence is below an allowed threshold criterion (e.g., limit) and instead prompt the radar system 108 to send out a second radar-transmit signal 402-2 (or set of signals transmitted over a period of time) to probe the proximate region 106 again. The user module 222 may also include one or more machine-learned models to improve user distinction, as further described with respect to FIG. 7.), and
the time-varying radar cross-section is input into an EM scattering signal model to generate the one or more synthetic EM signatures ([0020] For example, the computing device 102 sends a first radar-transmit signal into the proximate region 106 and then receives a first radar-receive signal (e.g., a reflected radar-transmit signal) associated with the presence of an object (e.g., a first user 104-1). This first radar-receive signal includes one or more radar-signal characteristics (e.g., radar cross-section (RCS) data, motion signatures, gesture performances, and so forth) that may be used to distinguish the first user 104-1 from other users 104.).
Regarding claims 7, 14 and 20, Hayashi discloses grouping markers to create at least one marker group, wherein the at least one marker group is approximated as at least one planar target and each marker includes a position in the at least one planar target ([0173] An example of a pausable sustained gesture can be a voice listing gesture, wherein the user can start moving their hand in a rolling circular motion within a generally vertical plane that passes through themself and the device (hereinafter “rolling their hand” - intuitively, one can envision an analogy to a “keep the film rolling” gesture made by a film director to a cameraman).);
determining a unit normal vector of the at least one planar target based on the positions of the markers in the at least one planar target ([0294] To increase the accuracy of detections of user engagement, any number of factors may be used to determine user engagement, including, for example, projected proximity of the user relative to the computing device 102 or body orientation of the user relative to the computing device 102. Projected proximity of the user relative to the computing device 102 may include determining the rate at which the proximity of the user 104 to the computing device 102 is changing. In some instances, determining the projected proximity may include determining a pathway 3704 indicative of a direction in which the user 104 is moving. The pathway 3704 may be represented as a vector that indicates the direction and in some cases magnitude (e.g., speed) of the user’s movement relative to the computing device. By determining a pathway of the user 104, in contrast to a simple change in proximity, the computing device 102 may more accurately determine if the user 104 is intending or not intending to engage with the computing device 102.);
determining a projected angle of the at least one planar target with respect to a radar axis of a virtual radar direction ([0133] In an example, a first user 104-1 (e.g., a new, unregistered person) is detected at a distance of two meters (2 m) from the computing device 102 and at an angle of 90 degrees relative to the front (0- degree orientation) of the device. At the user’s position, the computing device 102 detects one radar-signal characteristic of the first user 104-1, which may be used to distinguish them from another user. Before the radar system 108 can determine a second radar-signal characteristic, however, the first user 104-1 leaves the proximate region 106 of the computing device 102. For some devices, one radar-signal characteristic may not be sufficient to distinguish the presence of the first user 104-1 at a future time to a high level of confidence. The computing device 102 of this disclosure, however, may augment this first radar-signal characteristic to enable an accurate distinction of the first user 104-1 at the future time. In particular, the augmentation may include rotation phase angles, 0, of 0, 180, and 270 degrees along with magnitudes, s, that correspond to linear displacements of 0.5, 1, and 4 m.); and
generating the radar cross-section of the target using the projected angle as input ([0133] In particular, the computing device 102 may augment a set of one or more radarsignal characteristics used to distinguish a user 104 and to improve detection of user presence. In an example, a first user 104-1 (e.g., a new, unregistered person) is detected at a distance of two meters (2 m) from the computing device 102 and at an angle of 90 degrees relative to the front (0- degree orientation) of the device. At the user’s position, the computing device 102 detects one radar-signal characteristic of the first user 104-1, which may be used to distinguish them from another user. Before the radar system 108 can determine a second radar-signal characteristic, however, the first user 104-1 leaves the proximate region 106 of the computing device 102.).
Allowable Subject Matter
Claims 3, 9 and 16 (and claims 4 and 5 which are dependent to claim 3) objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Prior art does not disclose or teach the unique combinations of mapping each data augmentation function of the set of data augmentation functions to a function token to create a set of function tokens;
using the set of function tokens to generate a data augmentation transformation pool; and
applying the data augmentation transformation pool to the first set of motion trajectories..
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 10332265 B1 With regards to claim 3: col 8, lines 31-55: FIG. 6 illustrates the process flow for recognition and detection according to embodiments of the invention. First, based on a video frame sequence 600 (obtained with a video camera) and motion sensor data 602, the ego-motion 604 of the platform is estimated, as described above. Alternatively, the ego-motion 604 may be estimated based only on the video frame sequence 600 or only on the motion sensor data 602. The ego-motion 604 defines the 6 degree-of-freedom velocity 312 of the platform (6=3 degrees of translation+3 degrees of rotation). Given this velocity 312, the transformation-consistency map is computed (element 606) by quantifying how the predicted change of the image (estimated from the geometric relationship between the relative velocity of an obstacle (e.g., traffic cone) and the optical flow on the image plane (see FIG. 4) is consistent with the actual input. The image areas affected by nuisance factors, such as rain, sun-glare, dirt, or snow, will not likely follow this consistent transformation, so lower consistency values would be generated for these image areas. In addition, radar data 608 may be used to improve the estimate of the transformation-consistency map (element 606). Radar may be more reliable than vision for estimating depth and velocity in the depth direction, but has poor angular resolution. So it may augment vision to improve the estimate of location and velocity of the surrounding objects..
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED A NASHER whose telephone number is (571)272-1885. The examiner can normally be reached Mon - Fri 0800 - 1700.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED A NASHER/Examiner, Art Unit 2675
/ANDREW M MOYER/Supervisory Patent Examiner, Art Unit 2675
Prosecution Insights