DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 14 recites, “A computer-readable storage medium having a computer program stored thereon . . .” The broadest reasonable interpretation of a claim drawn to a computer-readable storage medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. 101 as covering non-statutory subject matter. The USPTO recognizes that applicants may have claims directed to computer readable media that cover signals per se, which the USPTO must reject under 35 U.S.C. 101 as covering both non-statutory subject matter and statutory subject matter. A claim drawn to such a computer readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. $ I01 by adding the limitation "non-transitory" to the claim. Such an amendment would typically not raise the issue of new matter, even when the specification is silent because the broadest reasonable interpretation relies on the ordinary and customary meaning that includes signals per se.
Applicant’s specification in paragraph [0098] recites, “Computing device 1300 typically includes a plurality of computer storage media. Such media may be any available media accessible by the computing device 1300, including, but not limited to, volatile and non-volatile media, removable and non-removable media.” Since Applicant’s disclosure does not limit the definition of a “computer storage media”, it could be a signal based on the plain meaning and interpretation of the claim limitation of “a computer-readable storage medium”. Further, Applicant’s specification in paragraph [0102] recites, “According to example implementations of the present disclosure, a computer program product is further provided. The computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions.” Even though the specification does recites a “non-transitory” CRM, the specification specifically notes that this only an example implementation, and once again does not limited what forms the CRM can be, thus meaning it can include signals per se.
As an additional note, a non-transitory computer-readable storage medium having a computer program stored thereon is considered statutory as non-transitory computer-readable storage medium excludes transitory data signals.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 recites the limitation "a corresponding image key point in the second set of image key points" in Line 5. None of claim limitations of Claim 10 or the base claims recite a second set of image key points. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-11 and 13-21 are rejected under 35 U.S.C. 103 as being unpatentable over Zikos et al. (US 20230008567 A1) (Hereinafter referred to as Zikos) in view of Chao et al. (CN 111589138 B) (Hereinafter referred to as Chao).
Regarding Claim 1, Zikos discloses A method for recognizing a human action, comprising: (See Abstract, “The present invention relates to a method for deriving a 3D data from image data comprising . . .”)
in response to receiving an input image comprising a human object, (See [0011], “receiving, from at least one camera, image data representing an environment; detecting, from the image data, at least one object within the environment; classifying the at least one detected object,” Also see [0014], “Further advantageously, classifying the at least one detected object comprises classifying a first detected object of the at least one detected object as a human object,”)
determining a target action associated with the human object in the input image based on a neural network (See [0061], “the method 200 further comprises determining 205 a 2D skeleton 206a of the classified object 205a by implementing a neural network to identify features of the classified object 205a in the image data 204a corresponding to the classified object 205a.
See [0112], “Advantageously, for each classified object, mapping the determined 2D skeleton to 3D comprises . . . also helps to map detected movements upon well-recognised movements (e.g. that motion of kicking a ball has a well-observed pattern of movement, which can be fitted to time-dependent placements of 3D features).”
Here, Zikos teaches determining a 2D skeleton of the classified object (target action associated with the human object in the input image) as well as showing that the 2D skeleton would be based on a detected movement of the object.)
describing an association relationship between an image comprising a human object and an action of the human object in the image, (See [0061] teaching to determine a 2D skeleton of the object, noting that the object can be a human object. Also see [0112] teaching that the 2D skeleton would be based on the detected movement of the object and mapping it to a well-recognised movement. The 2D skeleton can also be considered as being representative of an association relationship between an image comprising a human object and an action of the human object in the image.)
the action being represented by a set of spatial points in a virtual human object corresponding to the human object, and a dimension of the virtual human object matching a dimension of the human object; and (See [0061] teaching to determine a 2D skeleton of the object. See Fig. 4B shows a 2D skeleton of the human object with an arrangement of vertices and lines. Also see [0066], “As illustrated, the object class ‘human’ can be described by a 2D arrangement of vertices 402a, 402b, 402c, etc. and lines 403a, 403b, etc. that connect the vertices.”
In this case, the 2D skeleton (virtual human object corresponding to the human object) contains a set of vertices (a set of spatial points). Thus the arrangement of the vertices of the 2D skeleton can be considered as the action.
See [0061], “Correspondingly, the number of 2D elements in a 2D skeleton may also vary as needed to fully describe the dimensions and pose of the classified object.”
Although Zikos does not explicitly detail an exact dimension matching between the virtual human object and the human object, since it does note that it takes into account the dimension information and that the 2D skeleton can vary as needed, then an person of ordinary skill in the art would easily derive having matching dimensions.)
updating the input image with the virtual human object performing the target action. (See Fig. 4B shows a 2D skeleton of the human object with an arrangement of vertices and lines. In this case, the derived 2D skeleton (virtual human object performing the target action) can be considered as “updating the input image” as the input image would be processed and thus updated with the 2D skeleton.)
However, Zikos fails to explicitly disclose determining a target action associated with the human object in the input image based on an action model,
the action model describing an association relationship between an image comprising a human object and an action of the human object in the image,
Chao teaches determining a target action associated with the human object in the input image based on an action model, (Page 8 Paragraph 1, “. . . when detecting the human body generating action, the terminal can start the motion capture system captures a period of time in the multi-frame bone data, the multi-frame bone data in the period of time according to the time sequence, to obtain the reference action sequence.” See Page 7 Paragraph 6, “Each frame of bone data is used to represent the pose of the human body at the time stamp corresponding to the frame.”
See Page 2 Paragraph 10 - 11, “obtaining unit, for obtaining the reference action sequence and action prediction model, the reference action sequence comprises at least two frames of continuous bone data, the action prediction model comprises a target prediction model and a target correction model; a prediction unit, for inputting the reference action sequence into the target prediction model, predicting to obtain the initial prediction action sequence corresponding to the reference action sequence through the target prediction model, the preliminary prediction action sequence comprises at least one frame preliminary prediction bone data;”
In this case, the action prediction model (an action model) takes an input reference action sequence (input image) which comprises a human body (human object) and produces an initial prediction action sequence (a target action).)
the action model describing an association relationship between an image comprising a human object and an action of the human object in the image, (See Page 8 Paragraph 1, “. . . when detecting the human body generating action, the terminal can start the motion capture system captures a period of time in the multi-frame bone data, the multi-frame bone data in the period of time according to the time sequence, to obtain the reference action sequence.”
Page 8 Paragraph 9 – Page 9 Paragraph 1, “inputting the reference action sequence into the target prediction model, the target prediction model can obtain the characteristic representation of each joint by learning each frame bone data in the reference action sequence at the same time, so as to predict the state of each joint in the future one frame or several frame bone data to obtain the primary prediction action sequence.”
Here, when a human object is performing an action, that action is captured and then inputted to the action prediction model (action model) which comprises the target prediction model that learns the joint and bone data of the input in order to predict the state of the joints in a future frame. This learning, mapping, and predicting of joint/bone data can be considered as the action model describing the action of the human object in the image and associating it with the human object.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zikos with Chao to include an action model.
The motivation to combine Zikos with Chao would have been obvious as both arts comprise processing input image data and to the 2D skeleton/bone data of the human object within the image, with the action of the human object being represented by vertices/joints (See Chao Page 2 Paragraph 10 – 11 and Chao Page 8 Paragraph 9 – Page 9 Paragraph 1). It should further be noted that the action prediction model is trained model, similar to Zikos teaching a neural network (See Chao Page 8 Paragraph 3). The benefit of including specifically an action model is that it can improve the accuracy of the action prediction (See Chao Page 10 Paragraph 6).
Regarding Claim 2, Zikos in view of Chao disclose The method of claim 1, wherein updating the input image comprises: determining a target virtual image of the virtual human object based on the target action, the virtual human object being configured as the target action; and (See Zikos Fig. 4B shows an image of the 2D skeleton of the human object with an arrangement of vertices and lines. The image of the 2D skeleton can be considered as the determined “target virtual image of the virtual human object based on the target action, the virtual human object being configured as the target action;”)
replacing a portion associated with the human object in the input image with the target virtual image. (See Zikos Fig. 4B shows an image of the 2D skeleton. Note that a portion can be any amount, and thus the examiner’s interpretation of a replacing a portion would be that it includes replacing the entirety of the human object in the input image with the target virtual image. With this interpretation, Zikos teaching the image of the 2D skeleton is sufficient to teach the above limitations.)
Regarding Claim 3, Zikos in view of Chao disclose The method of claim 2, wherein determining the target virtual image comprises: determining a body part of the human object in the input image; (See Zikos [0067], “As illustrated in FIG. 4C, the human object 304 can be described by a combination of constituent sub-objects 404a, 404b and 404c, each representing a constituent portion of the human. In the illustrated example, the sub-objects include objects representing the head 404a, the neck 404b and the torso 404c of the human 400. The relative placement and alignment of these constituent sub-objects can describe an alternative 2D skeleton—for example, one can identify of the ‘head’ object 404a at a first point in 2D space, the ‘neck’ object 404b at a second point in 2D space and the ‘torso’ object 404c at a third point in 2D space.”)
updating a rendering direction of a material of the virtual human object based on the body part; and (See Zikos [0073], “The method 200 further comprises constructing a 3D skeleton 208 for the classified object, comprising mapping 207 the determined 2D skeleton 206a to 3D.”
See Zikos [0093], “Advantageously, the method may also comprise constructing a 3D avatar 210 for the classified object 205a comprising integrating the constructed 3D skeleton 208 with a 3D model corresponding to the classified object.”
See Zikos [0135], “As discussed above, a processing speed advantage is obtained by not generating 3D textures ‘on the fly’, but rather mapping existing textures to 3D skeleton. However, the method can nevertheless be refined by generation of some 3D textures for combination with the 3D model applied to the 3D skeleton. In particular, the method may comprise generating a 3D surface mesh from the received image data, and mapping the derived 3D surface mesh to the 3D skeleton in combination with the 3D model. An example of a surface mesh may be a mesh with a surface describe clothing features, or a mesh with a surface describing facial features. The mesh may be derived from image data for an object, or from image data corresponding to a sub-object of an object.”
Here, Zikos teaches that one can use the 2D skeleton to 3D to generate a 3D skeleton as well as a 3D model integrated with the 3D skeleton. If one considers the 3D model to correspond to the “virtual human object”, then, one can map or generate 3D textures (updating a rendering direction of a material of the virtual human object) for the 3D model, in which the texture can be derived from image data corresponding to a sub-object of an object (based on the body part). Although not shown, obviously, for different body parts of the body, there would be different textures or different directions of the same texture which are mapped onto the 3D model. Considering an example of the texture of the left and right hand of the imaged object would usually be mirrored to each other, and thus Zikos teaching to derive the textures based on that image can be considered as “updating a rendering direction of a material”, since the derived textures would likely be in different directions to one another.)
generating the target virtual image based on the updated rendering direction. (See Zikos [0024], “Further advantageously, the method further comprises integrating the time-varying 3D skeleton with the 3D environment model to construct a time-varying integrated environment.”
See Zikos [0167], “For example, the model can be used to render new 2D images or videos of the integrated environment that were not represented in the image data. For example, the rendered images or videos may represent the environment from a new perspective (e.g. on the pitch of a football game itself, or at any requested point in the stands of the crowd).”
Here, Zikos teaches that the 3D skeleton, which would also be integrated with a 3D model, can be used to construct an integrated environment. Said integrated environment can be rendered (generating the target virtual image) and this would include the 3D model which has the mapped and generated textures (the updated rendering direction).)
Regarding Claim 4, Zikos in view of Chao disclose The method of claim 1, further comprising obtaining the action model by: representing the dimension of the human object based on a first set of image key points in a first training image comprising the human object;(See Zikos [0061], “Correspondingly, the number of 2D elements in a 2D skeleton may also vary as needed to fully describe the dimensions and pose of the classified object.”
See Chao Page 11 Paragraph 2, “training data set is used for training to obtain the action prediction model of the data set. application embodiment of the training data set source is not limited, illustratively, training data set can be derived from H3.6m (Human 3.6Million, data amount is 3 D human body posture) . . .”
Here, Chao teaches that the action prediction model (action model) is trained on a dataset such as H3.6m, a large-scale collection of 3D human poses. 3D human poses can be considered as “representing the dimension of the human object based on a first set of image key points”. A training image from this dataset can be considered as “a first training image comprising the human object”.)
generating the virtual human object based on the dimension of the human object, the virtual human object comprising the set of spatial points respectively corresponding to the first set of image key points; (See Chao Page 11 Paragraph 2 teaching training data for the action prediction model which includes 3D human poses, which would contain bone data (the first set of image key points).
See Zikos [0061], “the method 200 further comprises determining 205 a 2D skeleton 206a of the classified object 205a by implementing a neural network to identify features of the classified object 205a in the image data 204a corresponding to the classified object 205a.”
See Zikos [0061], “Correspondingly, the number of 2D elements in a 2D skeleton may also vary as needed to fully describe the dimensions and pose of the classified object.”
Here Zikos teaches generating a 2D skeleton which contains vertices (the virtual human object comprising the set of spatial points) which can describe the dimensions and pose of the human object (based on the dimension of the human object).)
and determining the action model based on the first set of image key points and the set of spatial points. (See Chao Page 11 Paragraph 2 teaching training data for the action prediction model which includes 3D human poses, which would contain bone data (the first set of image key points).
In combination with Zikos, which teaches generating a 2D skeleton which contains vertices (the set of spatial points), although Chao doesn’t explicitly teach to generate the virtual human object, Chao does teach to predict the state of each joint of bone data. This can considered as being analogous to Zikos teaching the 2D skeleton, as both the 2D skeleton and predicted state of each joint by Chao are both a set of points used to represent the action of the object. Thus, one can have action prediction model be determined by the training dataset of 3D human poses (determining the action model based on the first set of image key points) and be able to generate a 2D skeleton which has vertices representative of the joints of the human object (the set of spatial points). The motivation to combine would have been similar to that of Claim 1 rejection motivation.)
Regarding Claim 5, Zikos in view of Chao disclose The method of claim 4, wherein generating the virtual human object comprises at least one of: determining a position of the set of spatial points based on a position relationship of the first set of image key points; and (See Zikos [0061], “the method 200 further comprises determining 205 a 2D skeleton 206a of the classified object 205a by implementing a neural network to identify features of the classified object 205a in the image data 204a corresponding to the classified object 205a.” Also see Zikos [0072], “As stated above, to determine an appropriate positioning of the two dimensional features of the 2D skeleton, a neural network is used.” Lastly see Zikos [0012], “Advantageously, each of said at least one object comprises a plurality of associated sub-objects, . . . the 2D skeleton constructed from identifying the position of the sub-objects with respect to each other.”)
determining an orientation of the set of spatial points based on an orientation relationship of the first set of image key points. (See Zikos [0067], “The arrangement of the sub-objects will vary depending on the orientation of the object in the image data, such that that the 2D skeleton describes the pose of the object in 2 dimensions, thus representing 2D descriptor information.”
Also see Chap Page 11 Paragraph 2 that the prediction action model is trained using a dataset that includes 3D human poses (the first set of image key points). It is commonly understood that pose implies positioning and orientation information. The motivation to combine would have been similar that of Claim 1 rejection motivation.)
Regarding Claim 6, Zikos in view of Chao disclose The method of claim 5, wherein the first set of image key points respectively corresponds to a set of joint points of the human object, and (See Chao Page 11 Paragraph 2, “training data set is used for training to obtain the action prediction model of the data set. application embodiment of the training data set source is not limited, illustratively, training data set can be derived from H3.6m (Human 3.6Million, data amount is 3D human body posture) data set of 360 thousand, CMU-Mocap (Carnegie Mellon University Motion Capture, Carnegie Mellon University motion capture) data set or 3DPW (3D pose estimation reference) data set.” The 3D human body posture would mean that the “first set of image key points respectively corresponds to a set of joint points of the human object”.)
the set of spatial points comprises a set of joint points respectively corresponding to the virtual human object associated with the set of joint points of the human object. (See Zikos Fig. 4B showing a 2D skeleton (the virtual human object) with vertices (the set of spatial points comprises a set of joint points) and said 2D skeleton would be derived from the classified human object within the image data. The motivation to combine would have been similar to that of Claim 1 rejection motivation.)
Regarding Claim 7, Zikos in view of Chao disclose The method of claim 6, wherein obtaining the action model comprises: determining a predicted action of the human object in a second training image based on the action model and the second training image comprising the human object; (See Chao Page 2 Paragraph 10 - 11, “obtaining unit, for obtaining the reference action sequence and action prediction model, the reference action sequence comprises at least two frames of continuous bone data, the action prediction model comprises a target prediction model and a target correction model; a prediction unit, for inputting the reference action sequence into the target prediction model, predicting to obtain the initial prediction action sequence corresponding to the reference action sequence through the target prediction model, the preliminary prediction action sequence comprises at least one frame preliminary prediction bone data;”
See Chao Page 8 Paragraph 9 – Page 9 Paragraph 1, “inputting the reference action sequence into the target prediction model, the target prediction model can obtain the characteristic representation of each joint by learning each frame bone data in the reference action sequence at the same time, so as to predict the state of each joint in the future one frame or several frame bone data to obtain the primary prediction action sequence.” Here, Chao teaches a prediction action sequence of the human object (determining a predicted action of the human object)
See Chao Page 11 Paragraph 2, “training data set is used for training to obtain the action prediction model of the data set. application embodiment of the training data set source is not limited, illustratively, training data set can be derived from H3.6m (Human 3.6Million, data amount is 3 D human body posture) data set of 360 thousand, CMU-Mocap (Carnegie Mellon University Motion Capture, Carnegie Mellon University motion capture) data set or 3DPW (3D pose estimation reference) data set.” Here, the H3.6m dataset would mean that there exists more than one training image (the second training image comprising the human object).)
determining a predicted virtual image of the virtual human object based on the predicted action; and (See Chao Page 8 Paragraph 9 – Page 9 Paragraph 1 teaching a prediction action sequence which predicts that state of each joint.
See Zikos Fig. 4B showing a 2D skeleton with a set of vertices which would represent the joints of the skeleton. In combination, the 2D skeleton taught by Zikos can be configured based on the prediction action sequence taught by Chao. This combination would result in “a predicted virtual image of the virtual human object based on the predicted action”.
Also see Zikos [0112], “mapping the determined 2D skeleton to 3D comprises applying a recursive estimator across the time-ordered sequence of frames. In this manner, the derived 3D skeleton 508a at a first time instance is used as a further input 509a to the derivation of a second 3D skeleton 508b of the same object at a later time instance.” Here, Zikos teaches the idea of predicting the future state of the skeleton, which contains similarity to Chao.)
updating the action model based on a comparison of the predicted virtual image and a portion associated with the human object in the second training image. (See Chao Page 3 Paragraph 5, “In one possible implementation, the training unit is further used for the first sample action sequence in the training data set input initial prediction model, through the initial prediction model prediction to obtain the first prediction action sequence corresponding to the first sample action sequence; using the prediction loss function between the first prediction action sequence corresponding to the first sample action sequence and the standard action sequence corresponding to the first sample action sequence, reversely updating the parameter of the initial prediction model, obtaining the prediction model after parameter updating;
Here, Chao simply teaches the basic function of training the prediction action model, which takes a first prediction action sequence, using a loss function between the first prediction action sequence and standard action sequence (based on a comparison of the predicted virtual image) and updating the parameters of the model (updating the action model).
See Chao Page 3 Paragraph 2, “In one possible implementation, the obtaining unit is further used for obtaining the training data set, the training data set comprises a sample action sequence and the sample action sequence corresponding to the standard action sequence”
Here, Chao shows that the standard action sequence is a part of the training data set can thus can be considered as “the second training image”, and obviously the comparison would only be between the portion associated with the human object and the prediction action sequence. The motivation to combine would have been similar to that of Claim 1 rejection motivation.)
Regarding Claim 8, Zikos in view of Chao disclose The method of claim 7, wherein determining the predicted action comprises: obtaining a set of joint points of the human object in the second training image; and (See Chao Page 3 Paragraph 5, “using the prediction loss function between the first prediction action sequence corresponding to the first sample action sequence and the standard action sequence corresponding to the first sample action sequence”
See Chao Page 7 Paragraph 6, “each frame of bone data comprises information of a plurality of joints, information of a plurality of joints comprises information of each joint itself, further comprising association information between different joints. The information of each joint itself is used to indicate the state of the joint in the skeletal data of the frame.”
Here, Chao teaches the standard action sequence (the second training image) and that the information used is joint information and subsequently implying that the second training image has a set of joint points of the human object.)
determining the predicted action based on the set of joint points of the human object in the second training image and the action model. (See Chao Page 3 Paragraph 5 teaching to train and update the parameters of the prediction action model based on the loss function of a first prediction action sequence and standard action sequence.
This would mean that the predict action would be “based on the set of joint points of the human object in the second training image and the action model” as the action model is trained with this information. The motivation to combine would have been similar to that of Claim 1 rejection motivation.)
Regarding Claim 9, Zikos in view of Chao disclose The method of claim 7, wherein the predicted action is represented with a set of predicted spatial points of the virtual human object, and determining the predicted virtual image comprises: configuring an orientation of the set of joint points of the virtual human object respectively as an orientation of the set of predicted spatial points; and determining the predicted virtual image of the virtual human object based on the set of configured joint points. (See Chao Page 8 Paragraph 9 – Page 9 Paragraph 1, “inputting the reference action sequence into the target prediction model, the target prediction model can obtain the characteristic representation of each joint by learning each frame bone data in the reference action sequence at the same time, so as to predict the state of each joint in the future one frame or several frame bone data to obtain the primary prediction action sequence.”
See Zikos [0067], “The arrangement of the sub-objects will vary depending on the orientation of the object in the image data, such that that the 2D skeleton describes the pose of the object in 2 dimensions, thus representing 2D descriptor information.”
Here, Zikos teaches that the 2D skeleton (the virtual human object) would be trying to describe the pose of the object, with pose implying both orientation and position. In combination with Chao teaching to predict the state of each joint, the 2D skeleton can configure the orientation of the 2D skeleton to follow the predicted set of spatial points, and the resulting 2D skeleton can be considered “the predicted virtual image of the virtual human object based on the set of configured joint points”. The motivation to combine would have been similar to that of Claim 1 rejection motivation.)
Regarding Claim 10, Zikos in view of Chao disclose The method of claim 9, wherein updating the action model comprises: determining a set of predicted image spatial points in the predicted virtual image that respectively correspond to the set of joint points; and (See Chao Page 8 Paragraph 9 – Page 9 Paragraph 1 teaching a prediction action sequence which predicts that state of each joint. The state of each joint can be considered to correspond to “a set of predicted image spatial points”.
See Zikos Fig. 4B showing a 2D skeleton with a set of vertices which would represent the joints of the skeleton. In combination, the 2D skeleton taught by Zikos can be configured based on the prediction action sequence taught by Chao. This combination would result in “the predicted virtual image” which would include the set of vertices/joints of the 2D skeleton that would correspond to the vertices/joints in the input data (correspond to the set of joint points).)
updating the action model based on a distance between a predicted image spatial point in the set of predicted image spatial points and a corresponding image key point in the second set of image key points. (See Chao Page 3 Paragraph 5 teaching to train and update the parameters of the prediction action model (updating the action model) based on the loss function of a first prediction action sequence and standard action sequence. A loss function is commonly understood to be a distance between the predicted and target outputs, and this can be considered as “a distance between a predicted image spatial point in the set of predicted image spatial points and a corresponding” joint point in the standard action sequence. The motivation to combine would have been similar to that of Claim 1 rejection motivation.)
Regarding Claim 11, Zikos in view of Chao disclose The method of claim 7, further comprising: adjusting the orientation of the set of spatial points of the virtual human object based on the comparison of the predicted virtual image and the portion associated with the human object in the second training image. (See Chao Page 3 Paragraph 5 teaching to train and update the parameters of the prediction action model based on the loss function (comparison) of a first prediction action sequence and standard action sequence (second training image).
See Zikos [0067], “The arrangement of the sub-objects will vary depending on the orientation of the object in the image data, such that that the 2D skeleton describes the pose of the object in 2 dimensions, thus representing 2D descriptor information.” Once again, Zikos teaches that the 2D skeleton (the virtual human object) would be trying to describe the pose of the object, with pose implying both orientation and position. The combination of Chao and Zikos would result in the training of the action prediction model that the produce prediction action sequence (adjusting the orientation of the set of spatial points in the virtual human object) that would more closely resemble the standard action sequence. The motivation to combine would have been similar to that of Claim 1 rejection motivation.)
Regarding Claim 13, Zikos in view of Chao disclose An electronic device, comprising: at least one processing unit; and (See Zikos [0029], “In another aspect of the present invention, there is provided a computer comprising a processor configured to carry out any method described above.”)
at least one memory coupled to the at least one processing unit and storing instructions executable by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform at least: (See Zikos [0173], “The computer program may be maintained on a separate computer storage medium, or the computer program may be stored in the storage 714.”)
in response to receiving an input image comprising a human object, determining a target action associated with the human object in the input image based on an action model, the action model describing an association relationship between an image comprising a human object and an action of the human object in the image, the action being represented by a set of spatial points in a virtual human object corresponding to the human object, and a dimension of the virtual human object matching a dimension of the human object; and updating the input image with the virtual human object performing the target action. (The above limitations are similar to those of Claim 1 and are thus rejected under a similar rationale as that of Claim 1.)
Regarding Claim 14, Zikos in view of Chao disclose A computer-readable storage medium having a computer program stored thereon which, when executed by a processor, causes the processor to perform at least: (See Zikos [0029], “In another aspect of the present invention, there is provided a computer comprising a processor configured to carry out any method described above.” See Zikos [0173], “The computer program may be maintained on a separate computer storage medium, or the computer program may be stored in the storage 714.”)
in response to receiving an input image comprising a human object, determining a target action associated with the human object in the input image based on an action model, the action model describing an association relationship between an image comprising a human object and an action of the human object in the image, the action being represented by a set of spatial points in a virtual human object corresponding to the human object, and a dimension of the virtual human object matching a dimension of the human object; and updating the input image with the virtual human object performing the target action. (The above limitations are similar to those of Claim 1 and are thus rejected under a similar rationale as that of Claim 1.)
Regarding Claim 15, Claim 15 recites similar limitations as to those of Claim 2 and is thus rejected under a similar rationale as that of Claim 2.
Regarding Claim 16, Claim 16 recites similar limitations as to those of Claim 3 and is thus rejected under a similar rationale as that of Claim 3.
Regarding Claim 17, Claim 17 recites similar limitations as to those of Claim 4 and is thus rejected under a similar rationale as that of Claim 4.
Regarding Claim 18, Claim 18 recites similar limitations as to those of Claim 5 and is thus rejected under a similar rationale as that of Claim 5.
Regarding Claim 19, Claim 19 recites similar limitations as to those of Claim 6 and is thus rejected under a similar rationale as that of Claim 6.
Regarding Claim 20, Claim 20 recites similar limitations as to those of Claim 7 and is thus rejected under a similar rationale as that of Claim 7.
Regarding Claim 21, Claim 21 recites similar limitations as to those of Claim 8 and is thus rejected under a similar rationale as that of Claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THANG G HUYNH whose telephone number is (571)272-5432. The examiner can normally be reached Mon-Thu 7:30am-4:30pm EST | Fri 7:30am-11:30am EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.G.H./Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights