Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 22 November, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 8, 9, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”).
Regarding claim 1, Ren teaches a computer-implemented method for real-time analysis of video images, comprising:
- receiving visual data comprising video images captured by at least one camera (¶ 0019: FIG. 1 is a diagram of a sequence of video images 101, 102, 103 according to some embodiments. The video images 101, 102, 103 may be referred to collectively as "the video images 101-103" and may be a subset of a larger sequence of video images such as frames captured by a video camera or surveillance camera trained on a scene including one or more people.);
- detecting anonymous body regions of individuals in the video images by performing a body key point analysis on the video images (¶ 0024: FIG. 4 is a diagram that illustrates using keypoints to identify a human 105 in bounding boxes 305, 315 from different video images according to some embodiments. Keypoints 405 (only one indicated by a reference numeral in the interest of clarity) may be identified using the image of the human 105 within the bounding box 305.);
- extracting subregions from the video images containing the anonymous body regions (¶ 0022: define bounding boxes 305,306,307,308 ( collectively referred to as "the bounding boxes 305-308") that define the portions of the foreground image 301 that include the corresponding humans 105-108.;); and
Ren does not explicitly teach a first neural network; and processing the extracted subregions using a second neural network to classify the anonymous body regions contained in the extracted subregions according to demographic information.
However, Perlin does teach a first neural network (Figure 1: The dashed rectangle represents the feature extractor layers.; Page 252, Col. 1, ¶ 2: The architecture of an CNN is designed to include both the feature extractor and the classifier, as shown in Fig. 1.); and
processing the extracted subregions using a second neural network (Figure 1: The dashed rectangle represents the feature extractor layers, and the continuous one, the classifier layers.; Page 252, Col. 1, ¶ 2: The architecture of an CNN is designed to include both the feature extractor and the classifier, as shown in Fig. 1.) to classify the anonymous body regions contained in the extracted subregions according to demographic information (Page 252, Col. 2, Section 4: In this work we aim at extracting and classifying automatically soft biometric information from images containing a person. The classification includes the gender, as well as the upper and lower clothes wearing by the individual, based on their appearance.)
Perlin is considered to be analogous art as it pertains to human biometric data classification. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for extracting human attributes using a convolutional neural network approach (as taught by Perlin) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Perlin improves performance and eliminates possible noise by using a threshold to cut the length of the ranked feature vectors. (See page 254, Col. 1, ¶ 8).
This motivation for the combination of Ren and Perlin is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 2, the Ren and Perlin combination teaches the method according to claim 1.
Additionally, Ren teaches wherein each subregion is extracted to contain body key points corresponding to one individual (Figures 3 – 5; ¶ 0022: define bounding boxes 305,306,307,308 ( collectively referred to as "the bounding boxes 305-308") that define the portions of the foreground image 301 that include the corresponding humans 105-108.).
Regarding claim 8, the Ren and Perlin combination teaches the method according to claim 1.
Additionally, Perlin teaches wherein the first neural network comprises a convolutional neural network (CNN) comprising a plurality of convolutional layers (Figure 1: Layers (I) represent the raw input image. Layers (C) represent the convolution operations....; Page 252, Col. 1, ¶ 2: The architecture of an CNN is designed to include both the feature extractor and the classifier, as shown in Fig. 1.)
Additionally, Ren teaches detecting key points of a body region (Figure 4 and 5; ¶ 0024: Key points 405 ( only one indicated by a reference numeral in the interest of clarity) may be identified using the image of the human 105 within the bounding box 305.)
Perlin is considered to be analogous art as it pertains to human biometric data classification. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for extracting human attributes using a convolutional neural network approach (as taught by Perlin) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Perlin improves performance and eliminates possible noise by using a threshold to cut the length of the ranked feature vectors. (See page 254, Col. 1, ¶ 8).
Regarding claim 9, the Ren and Perlin combination teaches the method according to claim 8.
Additionally, Perlin teaches wherein the first neural network is trained on a first training dataset comprising images including torso, limbs, and accessories associated with the torso and limbs (Page 255, Col 1. Section 6.3: The H3D includes a large variety of people, wearing different clothes, standing in different poses, sometimes subject to occlusion and cluttered background. Therefore, this dataset is challenging for pattern recognition methods. The original dataset contains 4013 images annotated with 9 binary attributes (is male, has long hair, has glasses, has hat, has t-shirt, has long sleeves, has shorts, has jeans, has long pants).).
Perlin is considered to be analogous art as it pertains to human biometric data classification. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for extracting human attributes using a convolutional neural network approach (as taught by Perlin) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Perlin improves performance and eliminates possible noise by using a threshold to cut the length of the ranked feature vectors. (See page 254, Col. 1, ¶ 8).
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of device claim 19 while noting that the rejection above cites to both method and device disclosures.
For the device limitations of claim 19 see Ren’s teaching on:
A computing system for real-time analysis of video images, the computing system comprising one or more processor and memory, the memory having instructions stored thereon which, when executed by the one or more processors, cause the computing system to (¶ 0041: Some embodiments of the processors 1025 may identify or track images of humans in the video images by executing instructions stored in the memory 1020.):
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of manufacture claim 20 while noting that the rejection above cites to both method and manufacture disclosures.
For the manufacture limitations of claim 20 see Ren’s teaching on:
A non-transitory computer-readable medium having instructions stored thereon which, when executed by one or more processors of a computing system, cause the computing systems to (¶ 0042: The software comprises one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above):
Claims 3 – 6 are rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”) and further in view of Kaberg Johard et al (2021/0319134 A1, hereinafter “Johard”).
Regarding claim 3, the Ren and Perlin combination teaches the method according to claim 1.
Ren does not explicitly teach wherein the subregions are extracted to exclude a head and/or a face of individuals associated with the anonymous body regions.
However, Johard teaches wherein the subregions are extracted to exclude a head and/or a face of individuals associated with the anonymous body regions (¶ 0059: A 'destructive hash function' or 'one-way function' herein is used in the general sense as any hash function that is non-injective in the space of the collected biometric values. In other words, the function destroys information in the input… The output of this function is called a biometric key. The set of people with faces that result in a certain biometric key is called a 'hash group'.; In the first step it is possible to divide the space into hyperrectangles and destroy the information about within which hyperrectangle the input lies while preserving information about the location within the hyperrectangle.).
Johard is considered to be analogous art as it pertains to anonymous biometric data detection. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for handling anonymous biometric and/or behavioral data (as taught by Johard) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Johard performs division into smaller hyperrectangles which creates a simple locality-sensitive hashing, which increases the probability of any two noisy biometric measurements from the same subject being assigned to the same biometric key. (See ¶ 0065).
This motivation for the combination of Ren, Perlin, and Johard is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 4, the Ren and Perlin combination teaches the method according to claim 1.
Additionally, Johard teaches wherein the subregions are extracted to exclude uniquely identifying biometric information associated with the anonymous body regions (¶ 0059: A 'destructive hash function' or 'one-way function' herein is used in the general sense as any hash function that is non-injective in the space of the collected biometric values. In other words, the function destroys information in the input… The output of this function is called a biometric key. The set of people with faces that result in a certain biometric key is called a 'hash group'.; Examiner’s note: The examiner understands this claim to be similar to that of claim 3, only more broad in its language. As such, by erasing the face using a destructive hash function of Johard, this is understood to read on the limitation of excluding uniquely identifying biometric information.; ¶ 0064: In the first step it is possible to divide the space into hyperrectangles and destroy the information about within which hyperrectangle the input lies while preserving information about the location within the hyperrectangle.).
Johard is considered to be analogous art as it pertains to anonymous biometric data detection. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for handling anonymous biometric and/or behavioral data (as taught by Johard) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Johard performs division into smaller hyperrectangles which creates a simple locality-sensitive hashing, which increases the probability of any two noisy biometric measurements from the same subject being assigned to the same biometric key. (See ¶ 0065).
Regarding claim 5, the Ren and Perlin combination teaches the method according to claim 1.
Additionally, Ren teaches wherein the body key point analysis is configured to identify body key points corresponding to torso and limbs (Figure 4 and 5; ¶ 0024: FIG. 4 is a diagram that illustrates using keypoints to identify a human 105 in bounding boxes 305, 315 from different video images according to some embodiments; Examiner’s note: Figure 4 displays the identification of keypoints on the limbs, head, shoulders (torso), waist (torso), and arms of the human.), and
Additionally, Johard teaches the subregions are extracted to include anonymous body regions comprising the torso, limbs and accessories associated with the torso and limbs (¶ 0059: A 'destructive hash function' or 'one-way function' herein is used in the general sense as any hash function that is non-injective in the space of the collected biometric values. In other words, the function destroys information in the input… The output of this function is called a biometric key. The set of people with faces that result in a certain biometric key is called a 'hash group'.; ¶ 0064: In the first step it is possible to divide the space into hyperrectangles and destroy the information about within which hyperrectangle the input lies while preserving information about the location within the hyperrectangle.; Examiner’s note: As Johard erases the identifiable face of the human from the input images (¶ 0059) while leaving the rest of the subject intact, this is understood by the examiner to mean the regions of the image including the torso, limbs, and limb accessories (such as jewelry, watches, armbands, leg bands etc.) would remain.).
Johard is considered to be analogous art as it pertains to anonymous biometric data detection. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for handling anonymous biometric and/or behavioral data (as taught by Johard) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Johard performs division into smaller hyperrectangles which creates a simple locality-sensitive hashing, which increases the probability of any two noisy biometric measurements from the same subject being assigned to the same biometric key. (See ¶ 0065).
Regarding claim 6, the Ren and Perlin combination teaches the method according to claim 1.
Additionally, Johard teaches wherein extracting a subregion from the video images comprises fitting a boundary that contains all body key points associated with one individual, and extracting a subregion defined by the boundary (¶ 0022: define bounding boxes 305,306,307,308 ( collectively referred to as "the bounding boxes 305-308") that define the portions of the foreground image 301 that include the corresponding humans 105-108.; ¶ 0024: FIG. 4 is a diagram that illustrates using keypoints to identify a human 105 in bounding boxes 305, 315 from different video images according to some embodiments).
Johard is considered to be analogous art as it pertains to anonymous biometric data detection. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for handling anonymous biometric and/or behavioral data (as taught by Johard) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Johard performs division into smaller hyperrectangles which creates a simple locality-sensitive hashing, which increases the probability of any two noisy biometric measurements from the same subject being assigned to the same biometric key. (See ¶ 0065).
Claims 7 is rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”) in view of Kaberg Johard et al (2021/0319134 A1, hereinafter “Johard”) and further in view of Ning et al (U.S. Patent Publication No. 2021/0090284 A1, hereinafter “Ning”).
Regarding claim 7, the Ren, Perlin, and Johard combination teaches the method according to claim 6.
Ren does not explicitly teach wherein extracting the subregion from the video images comprises fitting the boundary to contain all body key points associated with one individual in addition to a predefined margin around the body key points.
However, Ning teaches wherein extracting the subregion from the video images comprises fitting the boundary to contain all body key points associated with one individual in addition to a predefined margin around the body key points (¶ 0092: The pose tracking module 160 is then configured to infer an inferred bounding box for each object from the determined keypoints. In certain embodiments, the inferred bounding box is inferred by: defining an enclosing box (or namely minimum bounding box or smallest bounding box) with the smallest measure within which all the points lie, and enlarging the rectangular box by 20% in horizontal direction and vertical direction to obtain the inferred bounding box.).
Ning is considered to be analogous art as it pertains to keypoint extraction. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and system for online top-down human pose tracking (as taught by Ning) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Ning uses easily replaceable and extendable components, making the framework faster and more accurate. (See ¶ 0172).
This motivation for the combination of Ren, Perlin, Johard, and Ning is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”) and further in view of Martin et al (U.S. Patent Publication No. 2020/0005060 A1, hereinafter “Martin”).
Regarding claim 10, the Ren and Perlin combination teaches the method according to claim 9.
Ren does not explicitly teach wherein the images in the first training dataset do not include an identifiable head and/or face of individuals associated with the torso, limbs and accessories.
However, Martin teaches wherein the images in the first training dataset do not include an identifiable head and/or face of individuals associated with the torso, limbs and accessories (¶ 0007: The training data may include a plurality of images that includes and/or excludes the face.).
Martin is considered to be analogous art as it pertains to human detection and identification. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system of machine learning based driver assistance (as taught by Martin) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Martin is trained such that the facial detection is capable of detecting a person in images in which landmarks may be obscured, for example, by poor lighting, physical objects, head pose, or the like. (See ¶ 0040).
This motivation for the combination of Ren, Perlin, and Martin is supported by KSR exemplary rationale (A) Combining prior art elements according to known methods to yield predictable results. MPEP 2141 (III).
Claim 11 and 12 is rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”) and further in view of Wang et al (U.S. Patent Publication No. 2018/0150684 A1, hereinafter “Wang”).
Regarding claim 11, the Ren and Perlin combination teaches the method according to claim 1.
Additionally, Perlin teaches wherein the second neural network comprises a convolutional neural network (CNN) comprising a plurality of convolutional layers (Figure 1: Layers (I) represent the raw input image. Layers (C) represent the convolution operations....; Page 252, Col. 1, ¶ 2: The architecture of an CNN is designed to include both the feature extractor and the classifier, as shown in Fig. 1.)
Ren does not explicitly teach classifying images according to an age and a gender.
However, Wang teaches wherein the second neural network is a CNN comprising a plurality of convolutional layers to classify images according to at least an age and a gender (Figure 1B; Abstract: Embodiments described herein provide various examples of an age and gender estimation system capable of performing age and gender classifications on face images having sizes greater than the maximum number of input pixels supported by a given small-scale hardware convolutional neutral network (CNN) module.).
Wang is considered to be analogous art as it pertains to biometric data detection. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the age and gender estimation using small-scale convolutional neural network modules (as taught by Wang) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Wang identifies facial landmark points of each detected face which allows the system to track each face and select the best pose image, which improves real-time face detection efficiency. (See ¶ 0064).
This motivation for the combination of Ren, Perlin, and Wang is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 12, the Ren, Perlin, and Wang combination teaches the method according to claim 11.
Additionally, Perlin teaches wherein the second neural network is trained on a second training dataset comprising images including torso, limbs, and accessories associated with the torso and limbs (Page 255, Col 1. Section 6.3: The H3D includes a large variety of people, wearing different clothes, standing in different poses, sometimes subject to occlusion and cluttered background. Therefore, this dataset is challenging for pattern recognition methods. The original dataset contains 4013 images annotated with 9 binary attributes (is male, has long hair, has glasses, has hat, has t-shirt, has long sleeves, has shorts, has jeans, has long pants).).
Perlin is considered to be analogous art as it pertains to human biometric data classification. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for extracting human attributes using a convolutional neural network approach (as taught by Perlin) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Perlin improves performance and eliminates possible noise by using a threshold to cut the length of the ranked feature vectors. (See page 254, Col. 1, ¶ 8).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”) and further in view of Wang et al (U.S. Patent Publication No. 2018/0150684 A1, hereinafter “Wang”) and Martin et al (U.S. Patent Publication No. 2020/0005060 A1, hereinafter “Martin”).
Regarding claim 13, the Ren, Johard, Wang, and Topiwala combination teaches the method according to claim 12.
Ren does not explicitly teach wherein the images in the second training dataset do not include an identifiable head and/or face of individuals associated with the torso, limbs and accessories.
However, Martin teaches wherein the images in the second training dataset do not include an identifiable head and/or face of individuals associated with the torso, limbs and accessories (¶ 0007: The training data may include a plurality of images that includes and/or excludes the face.).
Martin is considered to be analogous art as it pertains to human detection and identification. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system of machine learning based driver assistance (as taught by Martin) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Martin is trained such that the facial detection is capable of detecting a person in images in which landmarks may be obscured, for example, by poor lighting, physical objects, head pose, or the like. (See ¶ 0040).
This motivation for the combination of Ren, Perlin, Wang, and Martin is supported by KSR exemplary rationale (A) Combining prior art elements according to known methods to yield predictable results. MPEP 2141 (III).
Claims 14 - 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”) and further in view of Ning et al (U.S. Patent Publication No. 2021/0090284 A1, hereinafter “Ning”).
Regarding claim 14, the Ren and Perlin combination teaches the method according to claim 1.
Ren does not explicitly teach wherein performing the body key point analysis comprises identifying a plurality of body key points each with a corresponding confidence; and extracting the subregions from the video images comprises defining anonymous body regions containing key points that meet a predefined minimum confidence threshold, and extracting subregions from the video images containing the defined anonymous body regions.
Additionally, Ning teaches wherein performing the body key point analysis comprises identifying a plurality of body key points each with a corresponding confidence (¶ 0092: Each keypoint corresponds to a confidence score based on the heatmap, indicating the possibility that the keypoint is located in the frame.); and
extracting the subregions from the video images comprises defining anonymous body regions containing key points that meet a predefined minimum confidence threshold, and extracting subregions from the video images containing the defined anonymous body regions (¶ 0093: When the object state is "tracked," the pose tracking module 160 is configured to generate an enclosing box using four keypoints from the second frame, and infer an inferred bounding box from the enclosing box.).
Ning is considered to be analogous art as it pertains to keypoint extraction. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and system for online top-down human pose tracking (as taught by Ning) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Ning uses easily replaceable and extendable components, making the framework faster and more accurate. (See ¶ 0172).
This motivation for the combination of Ren, Perlin, and Ning is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim 15, the Ren, Perlin, and Ning combination teaches the method according to claim 14.
Additionally, Ning teaches wherein defining the anonymous body regions comprises selecting a minimum set of body key points that meet the predefined minimum confidence threshold (¶ 0092: Each keypoint corresponds to a confidence score based on the heatmap, indicating the possibility that the keypoint is located in the frame. The keypoints defines pose of the object. In certain embodiments, the number of the keypoints for each object is 15, and the 15 keypoints are, for example, "right knee," "left knee," "right pelvis," "left pelvis," "right wrist," "left wrist," "right ankle," "left ankle," "right shoulder," "left shoulder," "left elbow," "right elbow," "neck," "nose," "head."; ¶ 0093: The object state is "tracked" in the second frame if the average confidence score is greater than a threshold value… When the object state is "tracked," the pose tracking module 160 is configured to generate an enclosing box using four keypoints from the second frame, and infer an inferred bounding box from the enclosing box.).
Ning is considered to be analogous art as it pertains to keypoint extraction. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and system for online top-down human pose tracking (as taught by Ning) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Ning uses easily replaceable and extendable components, making the framework faster and more accurate. (See ¶ 0172).
Regarding claim 16, the Ren and Perlin combination teaches the method according to claim 1.
Additionally, Ning teaches wherein detecting anonymous body regions comprises determining a number of body key points identified via the body key point analysis (¶ 0092: Each keypoint corresponds to a confidence score based on the heatmap, indicating the possibility that the keypoint is located in the frame. The keypoints defines pose of the object. In certain embodiments, the number of the keypoints for each object is 15, and the 15 keypoints are, for example, "right knee," "left knee," "right pelvis," "left pelvis," "right wrist," "left wrist," "right ankle," "left ankle," "right shoulder," "left shoulder," "left elbow," "right elbow," "neck," "nose," "head."), and performing the extracting and processing of subregions only if the number of body key points identified is above a predetermined threshold (¶ 0092: The enclosing box is also called minimum bounding box or smallest bounding box. In certain embodiments, the enclosing box is provided by picking up the top-most keypoint, the lowest keypoint, the leftmost keypoint, and the rightmost keypoint from the 15 keypoints, and draw a rectangular box.; Examiner’s note: It is understood that in order for the bounding box to be formed (extraction) the system must identify 4 keypoints in order to create the bounding rectangle.).
Ning is considered to be analogous art as it pertains to keypoint extraction. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and system for online top-down human pose tracking (as taught by Ning) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Ning uses easily replaceable and extendable components, making the framework faster and more accurate. (See ¶ 0172).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Ren et al (U.S. Patent Publication No. 2016/0092727 A1, hereinafter “Ren”) in view of Perlin et al (Hugo Alberto Perlin, "Extracting human attributes using a convolutional neural network approach", Pattern Recognition Letters, Volume 68, Part 2, 2015, Pages 250-259, ISSN 0167-8655, hereinafter “Perlin”) and further in view of Renkis et al (U.S. Patent Publication No. 2015/0381946 A1, hereinafter “Renkis”).
Regarding claim 17, the Ren and Perlin combination teaches the method according to claim 1.
Ren does not explicitly teach further comprising generating track entries associated with the anonymous body regions and corresponding demographic information, and processing the track entries to output anonymized traffic data.
However, Renkis teaches further comprising generating track entries associated with the anonymous body regions and corresponding demographic information, and processing the track entries to output anonymized traffic data (¶ 0134: In one embodiment, the cloud-based analytics comprises counting the number of people passing through each entrance and exit, counting people in a certain area within the retail store, track customers trajectories within the store, estimate age and gender of customers in the retail store. With preliminary customer demographic and behavioral data, the cloud-based analytics platform can provide more advanced analytics functions. In one embodiment, the cloud-based analytics platform is operable to compare customer numbers during different periods of time in a day, week, month, and/or year in order to optimize promotional activities; to generate a heat map based on customer density within the retail store in order to inform marketing and management decisions such as optimizing shelf arrangement and staff shifts; to compare data across different stores in order to identify successful business strategies and improve under-performed stores.).
Renkis is considered to be analogous art as it pertains to extracting demographics of images of humans. Therefore, it would have been obvious to one of ordinary skill in the art to combine the system for tracking humans in video images (as taught by Ren) and the system for automated cloud-based analytics (as taught by Renkis) before the effective filing date of the claimed invention. The motivation for this combination of references would be the system of Renkis produces analytical data which can be used across different stores to identify successful business strategies and improve under-performed stores. (See ¶ 0134).
This motivation for the combination of Ren, Perlin, and Renkis is supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Allowable Subject Matter
Claim 18 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 18, the prior art, neither alone or in combination teaches wherein the receiving visual data, detecting anonymous body regions, extracting subregions, and processing the extracted subregions are performed by at least one computing system on a secure customer network, and the track entries are stored and processed on at least one cloud computing system outside the secure customer network.
Renkis teaches wherein the processing and storing of the track entries are performed on a cloud computing system, however, it does not teach wherein this cloud computing system is separated from a secure customer network and none of the prior art appears to teach a secure customer network which performs the receiving visual data, detecting anonymous body regions, extracting subregions, and processing of the extracted subregions.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW JONES whose telephone number is (703)756-4573. The examiner can normally be reached Monday - Friday 8:00-5:00 EST, off Every Other Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW B. JONES/ Examiner, Art Unit 2667
Prosecution Insights