DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claims 1-10, 12-17, and 19 are amended. Claim 12-20 are withdrawn. Claims 1-11 are pending in this application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ning et al., US 2021/0090284 in view of Akbas et al., US 11,182,924.
Regarding claim 1, Ning discloses one or more processors comprising one or more processing units (fig. 1, element 112; para 0084-0085; the processor may be a central processing unit (CPU) which is configured to control operation of the computing device) to:
detect one or more first positions of one or more human keypoints based at least on processing a first frame of sensor data using one or more neural networks (figs. 1 and 6; para 0092, 0094 and 0129; first frame of the video…detect objects (or targets, or candidates) from the first frame…include coordinates of the bounding box in the first frame…estimate pose for each object…determine keypoints in the bounding box, for example using a convolutional neural network (CNN));
detect one or more second positions of the one or more human keypoints based at least on processing a second frame of sensor data using the one or more neural networks (figs. 1 and 6; para 0093-0094 and 0130; process the second frame…the location and pose of an object are generally similar in two sequential frames…perform CNN using the inferred bounding box and the second frame, generate heatmaps, and determine keypoints using the heatmaps, where all the keypoints in the second frame are located in the area that is enclosed by the inferred bounding box).
Ning discloses claim 1 as enumerated above, but Ning does not explicitly disclose
generate a representation of one or more identified faults based at least on executing one or more checks that evaluate physiological plausibility of at least one of the one or more first positions or the one or more second positions of the one or more human keypoints as claimed.
However, Akbas discloses one of these models may be a “plausible physics” model. This model determined the plausibility of the estimated pose in the physical domain. In addition, this model may consider the temporal parameters of the physics, including: (i) body inertia, (ii) ground/floor contact in regards to foot position, (iii) body segment lengths, (iv) body segment angular velocities, and (v) joint ranges of motion. In the illustrative embodiment, an additional CNN may be applied for allowable human poses. This is a general model which will prevent unrealistic body representations and 3D reconstructions (col. 17, lines 10-35).
Therefore, taking the combined disclosures of Ning and Akbas as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate one of these models may be a “plausible physics” model. This model determined the plausibility of the estimated pose in the physical domain. In addition, this model may consider the temporal parameters of the physics, including: (i) body inertia, (ii) ground/floor contact in regards to foot position, (iii) body segment lengths, (iv) body segment angular velocities, and (v) joint ranges of motion. In the illustrative embodiment, an additional CNN may be applied for allowable human poses. This is a general model which will prevent unrealistic body representations and 3D reconstructions as taught by Akbas into the invention of Ning for the benefit of predicting three dimensional (3D) human poses from a single image (Akbas: col. 3, lines 7-14).
Regarding claim 2, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on comparing a designated threshold to a spatial displacement of the one or more human keypoints from the one or more first positions to the one or more second positions (Ning: para 0093, 0112, and 0130).
Regarding claim 3, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on applying a designated threshold to an angular displacement of a joint represented by a plurality of human keypoints comprising the one or more human keypoints from a plurality of first positions comprising the one or more first positions to a plurality of second positions comprising the one or more second positions (Ning: fig. 5; para 0079-0080, 0093, and 0130).
Regarding claim 4, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on applying a designated threshold to a difference between a detected limb length represented by a plurality of human keypoints comprising the one or more human keypoints at a plurality of first positions comprising the one or more first positions and at a plurality of second positions comprising the one or more second positions (Ning: fig. 5; para 0092-0093, and 0130).
Regarding claim 5, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on applying a designated threshold to a detected joint angle represented by at least one of: a plurality of human keypoints comprising the one or more human keypoints at a plurality of first positions comprising the one or more first positions, or the plurality of human keypoints at a plurality of second positions comprising the one or more second positions (Ning: fig. 5; para 0079-0080, 0093, and 0130).
Regarding claim 6, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on applying a designated threshold to a detected limb length represented by at least one of: a plurality of human keypoints comprising the one or more human keypoints at a plurality of first positions comprising the one or more first positions, or the plurality of human keypoints at a plurality of second positions comprising the one or more second positions (Ning: fig. 5; para 0092-0093, and 0130).
Regarding claim 7, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on applying a designated threshold associated with one or more spatial constraints that are external to a human body represented by the one or more human keypoints (Ning: para 0093, 0112, and 0130).
Regarding claim 8, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on applying a designated threshold on relative positions of keypoints within a detected instance of a plurality of human keypoints comprising the one or more human keypoints (Ning: figs. 1 and 6; para 0093-0094 and 0130).
Regarding claim 9, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein first frame represents a first modality of sensor data from a first time slice, the second frame represents a second modality of sensor data from the first time slice, and the one or more checks evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) based at least on applying a designated threshold to a spatial displacement of the one or more human keypoints from the one or more first positions detected using the first modality of sensor data from the first time slice to the one or more second positions detected using the second modality of sensor data from the first time slice (Ning: para 0093-0094, 0112, and 0130).
Regarding claim 10, the one or more processors of claim 1, Ning and Akbas in the combination further disclose wherein the one or more processing units are further to evaluate the physiological plausibility (Akbas: col. 17, lines 10-23) on first hardware rated for a first safety or reliability level, and to execute the one or more neural networks on second hardware that is unrated or rated for a lower safety or reliability level than the first safety or reliability level (Ning: para 0004 and 0172-0175).
Regarding claim 11, the one or more processors of claim 1, Ning in the combination further discloses wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations (fig. 4; para 0115; Siamese Graph Convolutional Network (SGCN)); a system for performing remote operations; a system for performing real-time streaming (para 0091; an online video or a live video); a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (para 0084; a cloud computer).
Response to Arguments
Applicant's arguments with respect to claims 1-11 have been considered but are moot in view of the new ground(s) of rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VAN D HUYNH whose telephone number is (571)270-1937. The examiner can normally be reached 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen R Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN D HUYNH/Primary Examiner, Art Unit 2665
Prosecution Insights