DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 9, 11-16, 21-22, 25 and 101 are rejected under 35 U.S.C. 103 as being unpatentable over Halim et al. (“Human action recognition based on 3D skeleton part-based pose estimation and temporal multi-resolution analysis”; IEEE; 2016; hereinafter Halim), in view of Yamanaka et al. (US 2019/0065872; hereinafter Yamanaka).
Regarding claim 1, Halim discloses a human action recognition based on 3D skeleton part-based pose estimation. Halim shows a computer-implemented method (see abstract) comprising: causing a plurality of image capture devices to capture a plurality of video feeds (see RGB-D cameras” in “introduction” on page 3041), wherein the plurality of video feeds captures movements of a subject from different view (see “introduction” on page 3041); receiving video data from the plurality of image captures devices (see “introduction” on page 3041), wherein the video data comprises the plurality of video feeds (see “introduction” on page 3041); determining, using the video data, first point data identifying a location of a first body part of the subject for a first frame during a first time period (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining, using the video data, second point data identifying a location of a second body part of the subject for the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining, using the first point data and the second point data, first distance data representing a distance between the first body part and the second body part (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), wherein the distance between the first body part and the second body part is a first distance frame feature in the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining a first feature vector corresponding to at least the first frame and a second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the first feature vector including at least the first distance data and second distance data (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); processing to identify subject exhibiting a behavior during the first time period (see page 3044 and table 1 on page 3044); and determining, based on the processing of at least the first feature vector, a first label corresponding to the first time period (see pages 3042-3044), the first label identifying a first behavior of the subject during the first time period (see pages 3042-3044).
But, Halim fails to explicitly state processing, using a trained model, trained model is machine learning model; the trained model is trained using a plurality of training frames, wherein the training frames were annotated to indicate first behavior; the trained model configured to identify a likelihood of the subject exhibiting a behavior during the first time period.
Yamanaka discloses a behavior recognition apparatus. Yamanaka also teaches using a computer and plural of image capture devices to capture images feed from plural views (see abstract; par. [0051]; [0078]; fig. 1A). Furthermore, Yamanaka further teaches using a trained model (see fig. 2; par. [0013]), trained model is machine learning model (see par. [0012], [0074]); the trained model is trained using a plurality of training frames, wherein the training frames were annotated to indicate first behavior (see fig. 2; par. [0018], [0029], [0049]-[0055], [0063], [0065], [0074]-[0075]; the trained model configured to identify a likelihood of the subject exhibiting a behavior during the first time period (see par. [0018], [0029], [0077]-[0084]; fig. 2 and 11).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing of the claimed invention, to have utilized the teaching of using a trained model, trained model is machine learning model; the trained model is trained using a plurality of training frames, wherein the training frames were annotated to indicate first behavior; the trained model configured to identify a likelihood of the subject exhibiting a behavior during the first time period in the invention of Halim, as taught by Yamanaka, to provide an efficient classifier to be able to correctly determine a correct behavior of the subject.
Regarding claim 2, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining, using the video data, third point data identifying a location of a third body part of the subject for the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2).
Regarding claim 3, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining, using the first point data and the third point data, second distance data representing a distance between the first body part and the third body part (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), wherein the distance between the first body part and the third body part is a second distance frame feature in the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining a second feature vector corresponding to the first frame to include at least the second distance data (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); and processing the first feature vector and the second feature vector (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and Yamanaka teaches the trained model (see fig. 2; par. [0013]).
Regarding claim 4, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining, using the first point data, the second point data and the third point data, first angle data representing an angle corresponding to the first body part, the second body part and the third body part, wherein the angle data is a first angle frame feature in the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining a second feature vector corresponding to at least the first frame, the second feature vector including at least the first angle data see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2; and processing the first feature vector and the second feature vector (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and Yamanaka teaches the trained model (see fig. 2; par. [0013]).
Regarding claim 5, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining, using the video data, fourth point data identifying a location of the first body part for a second frame during the first time period (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining, using the video data, fifth point data identifying a location of the second body part for the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining, using the video data, sixth point data identifying a location of the third body part for the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining, using the fourth point data and the fifth point data, third distance data representing a distance between the first body part and the second body part for the second frame, wherein the distance between the first body part and the second body part is a first distance frame feature in the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining, using the fourth point data and the sixth point data, fourth distance data representing a distance between the first body part and the third body part for the second frame, wherein the distance between the first body part and the third body part is a fourth distance frame feature in the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); determining, using the fourth point data, the fifth point data and the sixth point data, second angle data representing an angle corresponding to the first body part, the second body part and the third body part for the second frame, wherein the angle data is a second angle frame feature in the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); and determining the second feature vector to include at least the third distance data, the fourth distance data, and the second angle data (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2).
Regarding claim 6, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows calculating metric data corresponding to the first frame using at least the first distance data and the second distance data (see “full body approach” on page 3042-3043), wherein the first feature vector includes the metric data, and wherein the metric data represents statistical analysis corresponding to at least the first distance data and the second distance data (see “full body approach” on page 3042-3043), the statistical analysis being at least a mean (see “full body approach” on page 3042-3043).
Regarding claim 9, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Yamanaka teaches processing the video data using an additional trained model to determine the first point data, wherein the first point data includes pixel data representing the location of the first body part (see par. [0018], [0029], [0077]-[0084]; fig. 2 and 11).
Regarding claim 11, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining, using the video data, additional point data identifying locations of at least 12 portions of the subject for the first frame, wherein the 12 portions include at least the first body part and the second body part (see fig. 1-2).
Regarding claim 12, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows Burner shows determining additional distance data representing distances between a plurality of body portion-pairs for the first frame(see “full body approach” on page 3042; fig. 1-2), the plurality of body portion-pairs formed using pairs of the 12 portions of the subject (see “full body approach” on page 3042; fig. 1-2), wherein each distance between the plurality of body portion-pairs is a distance frame feature in the first frame, and wherein the first feature vector includes the additional distance data (see “full body approach” on page 3042; fig. 1-2).
Regarding claim 13, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining additional angle data representing angles corresponding to a plurality of body-portion trios for the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the plurality of body portion-trios formed by selecting three of the 12 portions of the subject (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), wherein each angle corresponding to the plurality of body- portion trios is an angle frame feature in the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and wherein the first feature vector includes the additional angle data (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2).
Regarding claim 14, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining additional feature vectors corresponding to six frames during the first time period (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the six frames including at least the first frame and the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), wherein the six frames are a window of frames surrounding at least the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); calculating metric data using the additional feature vectors, the metric data representing at least one of a mean (see “full body approach” on page 3042-3043); and processing the metric data using the to determine the first label data (see “full body approach” on page 3042-3043), and Yamanaka teaches the trained model (see fig. 2; par. [0013]).
Regarding claim 15, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining location data representing 12 portions of the subject for the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the location data including at least the first point data (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the second point data and the third point data (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and wherein processing the metric data includes processing the location data (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and Yamanaka teaches the trained model (see fig. 2; par. [0013]), and Yamanaka teaches the trained model (see fig. 2; par. [0013]) and pixel coordinates of portions of the subject (see par. [0055], [0056]).
Regarding claim 16, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining additional feature vectors corresponding to 11 frames during the first time period (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the 11 frames including at least the first frame and the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), wherein the 11 frames are a window of frames surrounding at least the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); calculating metric data using the additional feature vectors (see “full body approach” on page 3042-3043), the metric data representing at least one of a mean (see “full body approach” on page 3042-3043); and processing the metric data to determine the first label (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and Yamanaka teaches the trained model (see fig. 2; par. [0013]).
Regarding claim 21, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows configured to process feature data corresponding to video frames to determine a behavior exhibited by the subject represented in the video frames (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the feature data corresponding to portions of the subject (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and Yamanaka teaches the trained model is a classifier (see abstract).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing of the claimed invention, to have utilized the teaching that the trained model is a classifier in the invention of Halim, as taught by Yamanaka, to provide an efficient classifier to be able to correctly determine a correct behavior of the subject.
Regarding claim 22, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows the first body part is a mouth of the subject (see fig. 1); the second body part is right hind foot of the subject (see fig. 1), the first label indicates the first frame represents contact between the first body part and the second body part (see page 3042-3044; the examiner notes that claim does not require direct contact) and Yamanaka teaches the trained model is configured to identify a likelihood of the subject exhibiting contact between the first body part and the second body part (see par. [0018], [0029], [0077]-[0084]; fig. 2 and 11; the examiner notes that claim does not require direct contact). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing of the claimed invention, to have utilized the teaching of using the trained model is configured to identify a likelihood of the subject exhibiting contact between the first body part and the second body part in the invention of Halim, as taught by Yamanaka, to provide an efficient classifier to be able to correctly determine a correct behavior of the subject.
Regarding claim 25, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows wherein the subject is a mammal, and wherein the mammal is primate (see fig. 1).
Regarding claim 101, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows wherein the first behavior is one of shaking and flicking (see pages 3041 and 3043 and table 1).
Claims 10 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Halim et al. (“Human action recognition based on 3D skeleton part-based pose estimation and temporal multi-resolution analysis”; IEEE; 2016; hereinafter Halim), in view of Yamanaka et al. (US 2019/0065872; hereinafter Yamanaka) as applied to claim 1 above, and further in view of Russ et al. (US 2022/0237808; hereinafter Russ).
Regarding claim 10, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Yamanaka teaches processing the video data using an additional trained model to determine a likelihood that a pixel coordinate corresponds to the first body part (see par. [0018], [0029], [0058], [0077]-[0084]; fig. 2 and 11) and determining the first point data based at least in part on the likelihood that a pixel coordinate corresponds to the first body part satisfying a threshold, the first point data including the pixel coordinate (see par. [0018], [0029], [0058], [0077]-[0084]; fig. 2 and 11), but fails to explicitly satisfying a threshold.
Russ discloses a motion analysis and tracking and teaches satisfying a threshold (see par. [0041], [0053], [0080]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing of the claimed invention, to have utilized the teaching of satisfying a threshold in the invention of Halim and Yamanaka, as taught by Russ, to provide a more accurate image and body part detection by removing unwanted pixels.
Regarding claim 24, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Yamanaka teaches video capturing top view (see fig. 3), but fails to explicitly state a side view of the subject.
Russ discloses a motion analysis and tracking and teaches side view of the subject (see fig. 5). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing of the claimed invention, to have utilized the teaching of side view of the subject in the invention of Halim and Yamanaka, as taught by Russ, to provide a more accurate image and body part detection by capturing a side view of the subject.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Halim et al. (“Human action recognition based on 3D skeleton part-based pose estimation and temporal multi-resolution analysis”; IEEE; 2016; hereinafter Halim), in view of Yamanaka et al. (US 2019/0065872; hereinafter Yamanaka) as applied to claim 1 above, and further in view of Brunner et al. (US 2005/0163349; hereinafter Brunner).
Regarding claim 18, Halim and Yamanaka disclose the invention substantially as described in the 103 rejection above, furthermore, Halim shows determining additional feature vectors corresponding to several frames during the first time period (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), the several frames including at least the first frame and the second frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), wherein the several frames are a window of frames surrounding at least the first frame (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2); calculating metric data using the additional feature vectors (see “full body approach” on page 3042-3043), the metric data representing at least one of a mean (see “full body approach” on page 3042-3043) ; and processing the metric data to determine the first label (see “introduction” on page 3041 and “full body approach” on page 3042; fig. 1-2), and Yamanaka teaches the trained model (see fig. 2; par. [0013]), but fails to explicitly states 21 frames.
Brunner discloses a system and method for assessing motor and locomotor defects and teaches using 21 frames (see par. [0038]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing of the claimed invention, to have utilized the teaching of Halim and Yamanaka, as taught by Brunner, to be able to provide a better-quality video feed.
Response to Arguments
Applicant’s arguments with respect to prior art rejection of claim 1 have been considered but are moot because the new ground of rejection does not rely on any rejection applied in the prior office action of record for any teaching or matter specifically challenged in the argument. The examiner has provided new prior arts Halim and Yamanaka to address the newly added claim limitations in claim 1.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHDEEP MOHAMMED whose telephone number is (571)270-3134. The examiner can normally be reached Monday to Friday, 9am to 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anne M Kozak can be reached at (571)270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHDEEP MOHAMMED/ Primary Examiner, Art Unit 3797
Prosecution Insights