Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 12 recites the limitation “the first prediction result” in page 3 of the Claims. There is insufficient antecedent basis for this limitation in the claim. Claims 20 and 29 are also objected to for reciting a similar limitation.
Claims 17-18 recite the limitation “the at least one task” in page 4 of the Claims. There is insufficient antecedent basis for this limitation in the claim. It should be “the at least one prediction task” as disclosed in claim 12. Claims 25-26 are also objected to for reciting similar limitations.
Claim 18 recites the limitations of “a third loss” and “a third prediction result”. There is insufficient antecedent basis for this limitation in the claim, as claim 12 does not disclose a “second loss” nor a “second prediction result”. Appropriate correction is required. Claim 26 is also objected to for reciting a similar limitation.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 12-26 and 29 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental and mathematical process) without significantly more. The flow chart in MPEP 2106, Subject Matter Eligibility Test for Products and Processes, will be referred to establish ineligible subject matter.
Regarding claim 1, in Step 1 of the 101 analysis set forth in MPEP 2016, the claim recites “inputting an original image into a predetermined model; outputting a prediction result for at least one prediction task of the original image by the predetermined model; wherein the at least one prediction task comprises a key point prediction task, and wherein a loss item of the predetermined model in a training process comprises a first loss constructed based on an error distribution between the first prediction result of the key point prediction task and a key point position label.” which would be categorized as a process under the four statutory categories. In Step 2A Prong One, the claim is further directed to abstract ideas (mental and mathematical processes) of:
outputting a prediction result for at least one prediction task of the original image by the predetermined model (outputting a prediction can be performed mentally. The use of a machine model is only a substitute for a human mind, see MPEP 2106.04(a)(2))
wherein the at least one prediction task comprises a key point prediction task, and wherein a loss item of the predetermined model in a training process comprises a loss item of the predetermined model in a training process comprises a first loss constructed based on an error distribution between the first prediction result of the key point prediction task and a key point position label (constructing a loss using keypoints can be performed mathematically. The task of a predetermined model is a technological environment or field of use and does not amount to significantly more)
Step 2A Prong Two: Additional elements include “inputting an original image into a predetermined model…” This limitation generally links the use of the judicial exception to a particular technological environment or field of use as stated in MPEP 2106(h). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Step 2B: The additional claim elements do not amount to significantly more than the judicial exception. With regards to, “inputting an original image into a predetermined model…” This limitation generally links the use of the judicial exception to a particular technological environment and is claimed in a way that is well understood, routine, and conventional. With regards to, “outputting a prediction result for at least one prediction task of the original image by the predetermined model” these are mere mental processing with the use of a technological environment or field of use, and is claimed in a way that is well understood, routine, and conventional. With regards to, “wherein the at least one prediction task comprises a key point prediction task, and wherein a loss item of the predetermined model in a training process comprises a loss item of the predetermined model in a training process comprises a first loss constructed based on an error distribution between the first prediction result of the key point prediction task and a key point position label,” this is mere mathematical processing, and is claimed in a way that is well understood, routine, and conventional. Claims 20 and 29 have similar limitations and are not eligible for similar reasons.
Regarding claim 13, it is dependent upon claim 12 and thereby incorporates the limitations of and corresponding analysis applied to claim 12. Further, claim 13 recites constructing a flow model based on the first prediction result and the key point position label; determining an error distribution between the first prediction result and the key point position label based on the constructed flow model. Additional limitations are to judicial exception (mathematical process) of creating an error distribution between a prediction and a label in the flow model (in which constructing a flow model is another mathematical process). The addition of a judicial exception does not amount to significantly more. Therefore, this claim is ineligible. Claim 21 has similar limitations and is not eligible for similar reasons.
Regarding claim 14, it is dependent upon claim 13 and thereby incorporates the limitations of and corresponding analysis applied to claim 13. Further claim 14 recites obtaining a first sample and a second sample respectively by sampling a first predetermined distribution and an error between the first prediction result and the key point position label; constructing a flow model based on the first and the second samples. Additional limitations are to judicial exception (mathematical process) of obtaining a first and second sample by sampling a distribution and error. The construction of a flow model is also a mathematical process. The addition of a judicial exception does not amount to significantly more. Therefore, this claim is ineligible. Claim 22 has similar limitations and is not eligible for similar reasons.
Regarding claim 15, it is dependent upon claim 14 and thereby incorporates the limitations of and corresponding analysis applied to claim 14. Further claim 15 recites determining an initial flow model based on the first and the second samples iteratively; obtaining the flow model by iteratively updating the initial flow model until a likelihood estimation of the initial flow model meets a predetermined condition. Additional limitations are to judicial exception (mathematical process) of determining an initial flow model and iteratively updating the flow model until a likelihood estimation. The addition of a judicial exception does not amount to significantly more. Therefore, this claim is ineligible. Claim 23 has similar limitations and is not eligible for similar reasons.
Regarding claim 16, it is dependent upon claim 12 and thereby incorporates the limitations of and corresponding analysis applied to claim 12. Further, claim 16 recites performing a log-likelihood estimation of a residual between the error distribution and a second predetermined distribution, to use an obtained residual likelihood estimation loss as the first loss. Additional limitations are to judicial exception (mathematical process) of log-likelihood estimation of a residual. The addition of a judicial exception does not amount to significantly more. Therefore, this claim is ineligible. Claim 24 has similar limitations and is not eligible for similar reasons.
Regarding claim 17, it is dependent upon claim 12 and thereby incorporates the limitations of and corresponding analysis applied to claim 12. Further, claim 17 recites the loss item of the predetermined model in the training process further comprises: a second loss constructed based on a second prediction result of the gesture classification task and a gesture classification label. Additional limitations are to judicial exception (mathematical process) of constructing a second loss based on a second prediction result. The addition of a judicial exception does not amount to significantly more. Therefore, this claim is ineligible. Claim 25 has similar limitations and is not eligible for similar reasons.
Regarding claim 18, it is dependent upon claim 12 and thereby incorporates the limitations of and corresponding analysis applied to claim 12. Further, claim 18 recites the loss item of the predetermined model in the training process further comprises: a third loss constructed based on a third prediction result of the left and right hand classification task and a left and right hand classification label. Additional limitations are to judicial exception (mathematical process) of constructing a second loss based on a second prediction result. The addition of a judicial exception does not amount to significantly more. Claim 26 has similar limitations and is not eligible for similar reasons.
The examiner does note claim 19 is patent eligible as they disclose limitation that are significantly more than an abstract idea. Claims 27-28 recite similar limitations and are eligible for similar reasons.
Regarding claim 19, additional elements include generating a gesture control instruction based on a prediction result of the at least one prediction task, to cause a target application to perform a corresponding action based on the gesture control instruction. This is significantly more than an abstract idea.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 13 recites the limitation “an error distribution” in page 3 of the Claims. It is unclear if this is the same “error distribution” of claim 12 or a different error distribution. Claim 21 is also rejected for reciting a similar limitation. Appropriate correction is required.
Claim 14 recites the limitation “wherein the constructing a flow model…” in page 3 of the Claims. It is unclear if this is attempted to draw antecedence to “a flow model” of claim 13. In addition, claim 14 recites “construction a flow model based on the first and the second samples”. It is additionally unclear if this flow model is different from the flow model of claim 13. Claim 22 is also rejected for reciting a similar limitation. Appropriate correction is required.
Claim 15 recites “wherein the constructing a flow model…” in page 3 of the Claims. It is unclear if this is attempted to draw antecedence to “a flow model” of claim 13 or 14. In addition, “obtaining the flow model” lacks clear antecedent basis as both claims 13 and 14 comprise the limitation of “a flow model”. Therefore, it is unclear which flow model claim 15 is attempting to draw antecedence from. Claim 23 is also rejected for reciting a similar limitation. Appropriate correction is required.
Claim 19 recites the limitation “a prediction result” in page 4 of the claims. It is unclear if this is the same prediction result as disclosed in claim 12 or a new prediction result. In addition, claim 19 is dependent on both claim 12 and 17. It is unclear if “a prediction result” of claim 19 is the “first prediction result” of claim 12, the “second prediction result” of claim 17, or a new prediction result altogether. Claims 27-28 are rejected for reciting similar limitations. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 12, 20, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Peng ZHU CN-112699837-A, hereinafter ZHU.
As per claim 12, ZHU discloses a method of multi-task prediction comprising:inputting an original image into a predetermined model (see ZHU page 6/37 step 3, wherein the image is input into a gesture recognition network);outputting a prediction result for at least one prediction task of the original image by the predetermined model (see ZHU page 6/37 step 3, wherein the prediction results are output. The gesture and posture classification branch outputs the similarity of the gesture and posture, the hand region localization outputting the prediction results, as well as the hand key point heat maps);wherein the at least one prediction task comprises a key point prediction task (see ZHU page 6/37, wherein one of the multi-task branches of the network, wherein the prediction tasks comprise of a posture and gesture classification, as well as a hand key point detection branch to find hand key points, i.e., the key point prediction task), and wherein a loss function of the predetermined model in a training process (see ZHU page 18/37, wherein the loss function
L
p
t
s
(
p
i
,
p
i
*
)
is disclosed. This loss function is within the hand key point detection branch in the network) comprises a first loss (see ZHU page 20/37, comprising the cross entropy loss
L
s
G
1
p
i
,
p
i
*
[corresponding to a first loss] which serves as a loss between the actual value and the predicted value) constructed based on an error distribution (see ZHU page 21/37, wherein the formula for
L
s
of the G1 mode [
L
s
G
1
p
i
,
p
i
*
] utilizes a Gaussian distribution based function S [i.e., an error distribution] between point p under the structure of g) between the first prediction result of the key point prediction task and a key point position label (see ZHU page 20/37, wherein the cross entropy loss of G1 {the formula for
L
s
of the G1 mode [
L
s
G
1
p
i
,
p
i
*
]} is between the predicted value
p
i
, i.e., the prediction result, and the actual value
p
i
*
, i.e., key point [corresponding to the key point position label], as similarly disclosed in the Applicant’s originally filed specification ¶72).
While ZHU does disclose a loss function, it does not explicitly disclose a loss item. However, it would have been obvious for one of ordinary skill in the art to use ZHU to disclose a loss item. The reason is because the first loss
L
s
G
1
p
i
,
p
i
*
within ZHU is within the loss function of
L
p
t
s
. Under the broadest reasonable interpretation, one of ordinary skill in the art could use the loss function of
L
p
t
s
as the loss item, as both ZHU and the disclosure use a loss in a prediction task in order to acquire a prediction result. Thus, it would have been obvious to one of ordinary skill in the art at the time of the invention by the applicant to use the loss function
L
p
t
s
of ZHU as the loss item needed to find the prediction.
As per claim 20, the rationale provided in claim 12 is used herein. In addition, the electronic device of claim 20 (see ZHU page 22/37, wherein a device including a memory and processor is disclosed) corresponds to the method of claim 12.
As per claim 29, the rationale provided in claim 12 is used herein. In addition, the electronic device of claim 29 corresponds to the method of claim 12.
Claims 13 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over ZHU, in further view of Sadegh Aliakbarian et. al. FLAG: Flow-based 3D Avatar Generation from Sparse Observations, hereinafter Sadegh.
As per claim 13, while ZHU teaches the error distribution between the first prediction result and the key point position label is determined (see ZHU page 11/37, wherein
L
s
G
1
p
i
,
p
i
*
and
L
s
G
2
p
i
,
p
i
*
are disclosed, which serve as the first loss as components within the loss function of
L
p
t
s
(
p
i
,
p
i
*
)
. It is further disclosed on ZHU page 21/37 that
L
s
, which is the combination of
L
s
G
1
and
L
s
G
2
, contains the Gaussian distribution, or error distribution, of the predicted and true values between two key points), it fails to explicitly disclose where Sadegh teaches: constructing a flow model (see Sadegh page 3/10, in the context of training a model that uses an error distribution, the flow-based model is developed) based on the first prediction result and the key point position label (see Sadegh pages 4-5/10 and FIG. 4, wherein the predicted joint points in the flow model is disclosed.
L
m
j
p
serves as the prediction result from the predicted joint points and
L
r
e
c
serves as the key point position label using the categorical latent space of the joint points);determining an error distribution between the first prediction result and the key point position label based on the constructed flow model (see Sadegh page 5/10, wherein the likelihood, as in probability or error, distribution of each sub region of the full body in the loss function is calculated.
L
m
j
p
contains the prediction results and
L
r
e
c
contains the key point position label. These are used within the likelihood distribution of the loss function
L
. It is further disclosed in section 4.5 on page 5/10 that the flow-based model is used in order to optimize the likelihood).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHU’s by using Sadegh’s teaching by including the flow model to the prediction result and key point label in order to find more accurate predicted joint points.
As per claim 21, the rationale provided in claim 13 is used herein. In addition, the electronic device of claim 21 corresponds to the method of claim 13.
Claims 16 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over ZHU, in further view of J. Li et al., “Human Pose Regression with Residual Log-likelihood Estimation,” hereinafter Li.
As per claim 16, while ZHU discloses performing a log-likelihood estimation (see ZHU page 21/37, wherein the log-likelihood estimation of
S
*
(
p
|
g
)
log
S
^
(
p
|
g
)
is disclosed within
L
s
), it fails to explicitly disclose where Li teaches:performing a log-likelihood estimation of a residual between the error distribution and a second predetermined distribution (see Li page 5/10 and FIG. 2, wherein a residual log-likelihood estimation is disclosed with a Gaussian distribution and a flow model distribution), to use an obtained residual likelihood estimation loss as the first loss (see Li page 5/10, wherein the residual log-likelihood is used for the loss function).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHU’s method by using Li’s teaching by including a residual to the log-likelihood estimation in order to acquire a better point distribution mapping for the predicted values.
As per claim 24, the rationale provided in claim 16 is used herein. In addition, the electronic device of claim 24 (see ZHU page 22/37, wherein a device including a memory and processor is disclosed) corresponds to the method of claim 16.
Claims 17 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over ZHU, in further view of Tu ZHAO CN-111126339-A, hereinafter ZHAO.
As per claim 17, ZHU fails to explicitly disclose where ZHAO teaches:The method of claim 12, wherein if the key point prediction task is a prediction task of a key point of a hand, the at least one task further comprises a gesture classification task (see ZHAO page 6/31, wherein the gesture classification task in the key point prediction deep learning network is disclosed);and wherein the loss item of the predetermined model in the training process further comprises:a second loss (see ZHAO page 7/31, wherein a second loss is calculated) constructed based on a second prediction result of the gesture classification task (see ZHAO page 7/31, wherein the second loss is calculated from the first standard gestures and the corresponding predicted gestures, i.e., second prediction) and a gesture classification label (see ZHAO page 6/31, wherein the category labels used in the gesture recognition model is disclosed).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHU’s method by using ZHAO’s teaching by including a second loss to the key point prediction in order to further hone in on specific predictions within a gesture in the image.
As per claim 25, the rationale provided in claim 17 is used herein. In addition, the electronic device of claim 25 corresponds to the method of claim 17.
Claims 19 and 27-28 are rejected under 35 U.S.C. 103 as being unpatentable over ZHU, in further view of BIN GAO WO-2023051706-A1, hereinafter GAO.
As per claim 19, while ZHU, in combination with ZHAO, discloses outputting a prediction result for at least one prediction task of the original image by using a predetermined model (see ZHAO page 7/31, wherein the second loss is calculated from the first standard gestures and the corresponding predicted gestures, i.e., second prediction in the gesture recognition model. Further, on page 8/31, the recognition result is output), it fails to explicitly disclose where GAO teaches:generating a gesture control instruction based on a prediction result of the at least one prediction task (see GAO 3/31, wherein a grasping control is created based on the predicted grasping point and predicted grasping posture), to cause a target application to perform a corresponding action based on the gesture control instruction (see GAO page 4/31 Step 103, wherein the grasping task is completed according to the adjusted predicted grasping posture and predicted grasping point).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify ZHU, in combination with ZHAO, by using GAO’s teaching by including a gesture control instruction to the gesture recognition model in order to further create a predicted gesture using the predicted key points acquired from the recognition model.
As per claim 27, the rationale provided in claim 19 is used herein. In addition, the electronic device of claim 27 corresponds to the method of claim 19.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Bradley Obas Felix whose telephone number is (703)756-1314. The examiner can normally be reached M-F 8-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 5712728243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRADLEY O FELIX/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights