Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/26/26 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 10, 12-17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2023/0027234 A1 (hereinafter Wu) in view of “Parametric Model Estimation for 3D Clothed Humans from Point Clouds” by Kangkan Wang, et al. (hereinafter Wang).
Regarding claim 1, the limitation “A method comprising: obtaining, by a volumetric modeling system, a provisional 3D representation of a subject present at a scene, the provisional 3D representation based on image data captured by a set of cameras configured to have different fields of view according to different vantage points the cameras have at the scene, wherein the provisional 3D representation is a point cloud representation of the subject” are taught by Wu (Wu, e.g. abstract, paragraphs 44-105, describes a system for neural rendering of humans using video captured from multiple viewpoints. Wu, e.g. paragraph 55, teaches that the input to the system consists of a synchronized multi-view video sequence towards a human performer which is used to construct an initial point cloud representation of the performer for each frame of the sequence,
PNG
media_image1.png
26
22
media_image1.png
Greyscale
, i.e. the claimed point cloud provisional 3D representation of a subject based on image data captured by a set of cameras having different fields of view from different vantage points.)
The limitations “identifying, by the volumetric modeling system, a deficiency in the provisional 3D representation of the subject … determining, based on the deficiency, a feature area on the provisional 3D representation and a corresponding feature on [a] body model; and generating, by the volumetric modeling system and based on the provisional 3D representation and the … body model, a refined 3D representation of the subject in which the deficiency is mitigated, wherein generating the refined 3D representation comprises increasing, based on the corresponding feature area on the … body model and based on other portions of the provisional 3D representation, a vertex density of the feature area on the provisional 3D representation” are taught by Wu (Wu, e.g. paragraphs 56, 75-87, 99, teaches that following initial construction of the point cloud models for each frame of the sequence, geometry refinement can be performed by generating a Shape-from-Silhouettes (SfS) model for each frame and using the SfS model
PNG
media_image2.png
30
20
media_image2.png
Greyscale
to identify and patch holes in the initial point cloud
PNG
media_image1.png
26
22
media_image1.png
Greyscale
for the frame. That is, Wu, e.g. paragraphs 81-86, teaches identifying the patches necessary for fixing holes,
PNG
media_image3.png
24
26
media_image3.png
Greyscale
, in the initial point cloud model by evaluating distances between the patch points
PNG
media_image4.png
28
24
media_image4.png
Greyscale
of the SfS model
PNG
media_image2.png
30
20
media_image2.png
Greyscale
and the points
PNG
media_image5.png
26
22
media_image5.png
Greyscale
of the initial point cloud model
PNG
media_image1.png
26
22
media_image1.png
Greyscale
, i.e.
PNG
media_image3.png
24
26
media_image3.png
Greyscale
represents both a set of patch points from the SfS model and the location of a hole in the initial point cloud representation, and is determined based on portions of the initial point cloud representation
PNG
media_image5.png
26
22
media_image5.png
Greyscale
which are outside of the hole(s). Wu, e.g. paragraph 81, indicates that the refined geometry comprises the initial point cloud model for the frame and the patch(es) from the SfS model, i.e.
PNG
media_image3.png
24
26
media_image3.png
Greyscale
is added to
PNG
media_image1.png
26
22
media_image1.png
Greyscale
to fill the hole(s). That is, as claimed, Wu’s system identifies a deficiency in the provisional 3D representation of the subject, i.e.
PNG
media_image3.png
24
26
media_image3.png
Greyscale
, wherein
PNG
media_image3.png
24
26
media_image3.png
Greyscale
also corresponds to the claimed feature area on the provisional 3D representation and corresponding feature area on the body model. Further, Wu generates a refined 3D representation in which the deficiency is mitigated, i.e. adding
PNG
media_image3.png
24
26
media_image3.png
Greyscale
to
PNG
media_image1.png
26
22
media_image1.png
Greyscale
fills the hole(s), by increasing the vertex density of the feature area(s) on the provisional 3D representation based on the corresponding feature area(s) on the body model and based on other portions of the provisional 3D representation, i.e. adding the vertices of
PNG
media_image3.png
24
26
media_image3.png
Greyscale
to fill the hole(s) increases vertex density at the hole(s)/feature area(s), wherein the increasing is based on other portions of the provisional 3D representation, i.e. as noted above
PNG
media_image3.png
24
26
media_image3.png
Greyscale
is determined based on portions of the initial point cloud representation
PNG
media_image5.png
26
22
media_image5.png
Greyscale
which are outside of the hole(s).)
The limitations “determining, by the volumetric modeling system, a set of parameters for a body model that is parameterizable to adaptively model a body type of the subject, the determining performed such that an application of the set of parameters to the body model produces a parameterized body model that imitates a pose of the provisional 3D representation of the subject; determining, based on the deficiency, a feature area on the provisional 3D representation and a corresponding feature area on the parameterized body model; generating, by the volumetric modeling system and based on the provisional 3D representation and the parameterized body model, a refined 3D representation of the subject in which the deficiency is mitigated, wherein generating the refined 3D representation comprises increasing, based on the corresponding feature area on the parameterized body model and based on other portions of the provisional 3D representation, a vertex density of the feature area on the provisional 3D representation” are implicitly taught by Wu (As discussed above, Wu generates a refined 3D representation by adding
PNG
media_image3.png
24
26
media_image3.png
Greyscale
to
PNG
media_image1.png
26
22
media_image1.png
Greyscale
to fill the hole(s), where
PNG
media_image3.png
24
26
media_image3.png
Greyscale
comprises points
PNG
media_image4.png
28
24
media_image4.png
Greyscale
of the SfS model
PNG
media_image2.png
30
20
media_image2.png
Greyscale
. That is, Wu’s body model is generated by SfS rather than the claimed parameterized body model imitating a pose of the provisional 3D representation generated by applying determined parameters. Wu, e.g. paragraph 4, 76, teaches that a parametric body model could be used as an alternative to the SfS model, but would have limitations due to being intended to model bare-skinned or tightly clothed humans rather than accounting for real clothes. While not explicitly stated by Wu, one of ordinary skill in the art would have understood that, analogous to Wu’s SfS models which imitate the pose of the initial point cloud in each frame, Wu’s explanation that “The fitted model can then be used to patch the holes while using the originally recovered ones for the rest” indicates that the parametric body model would be fit to the initial point cloud in each frame by determining the appropriate parameters to apply to the parametric body model to imitate the pose of the initial point cloud. It is further noted that although Wu notes a potential drawback of using a parametric body model rather than the SfS model, Wu does not actually teach away from the modification, i.e. as in MPEP 2145 X D 1 paragraph 2, "a reference does not teach away if it merely expresses a general preference for an alternative invention but does not criticize, discredit or otherwise discourage investigation into the invention claimed.", and Wu merely indicates that using a parametric body model may have errors when the human is wearing clothes that are not skin tight. However, in the interest of compact prosecution, Wang is cited for teaching an improved parametric body model which models humans wearing clothing that is not skin tight, thereby negating the potential drawback noted by Wu, along with details of determining the fitting parameters to match the pose of an input point cloud.) However, this limitation is taught by Wang (Wang, e.g. abstract, sections 1, 3-5, describes a system for modeling 3D clothed humans using the SMPL model, i.e. the parametric body model discussed by Wu, with a 3D displacement representing the surface of clothing that is not skin tight. Wang, e.g. section 3.1, describes the parametric body model including the fitting parameters vector
PNG
media_image6.png
14
16
media_image6.png
Greyscale
comprising shape parameters β and pose parameters θ, as well as clothing surface offsets D. Wang, e.g. sections 3.2, 3.3, figure 1, describes using three regression networks to estimate global unclothed model parameters
PNG
media_image6.png
14
16
media_image6.png
Greyscale
0, refined unclothed model parameters Δ
PNG
media_image6.png
14
16
media_image6.png
Greyscale
, and clothing offsets D, where the fitting is based on an input ground truth point cloud having points/vertices
PNG
media_image7.png
20
18
media_image7.png
Greyscale
. Finally, Wang, section 4, figure 9, discusses the results of the system, which successfully models humans wearing clothing that is not skin-tight using the parametric body model.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Wu’s image based modeling and rendering system to substitute Wang’s parametric clothed human body model for Wu’s SfS body model because Wu teaches that a parameterized body model could be used in place of the SfS body model with the potential drawback of that a parametric body model may have errors when the human is wearing clothes that are not skin tight and Wang discloses a parametric clothed human body model which would prevent the potential errors noted by Wu by correctly modeling the surface of clothing that is not skin tight over the parametric body model. In Wu’s modified system substituting Wang’s parametric clothed body model for Wu’s SfS body model, Wang’s fitting procedure to determine fitting parameters, e.g. sections 3.2, 3.3, would be performed in place of Wu’s Mask and Shape Generation step of paragraph 79, outputting a final detail model as in the lower right of Wang, figure 1, i.e. the claimed step of determining the appropriate parameters to apply to the parametric body model to imitate the pose of the provisional 3D representation/point cloud, followed by performing colorization to the fitted parametric clothed body model as in Wu, paragraph 80, followed by identifying the hole(s) and patch(es)
PNG
media_image3.png
24
26
media_image3.png
Greyscale
for patching the initial point cloud model as in Wu, paragraphs 81-86, where the points
PNG
media_image4.png
28
24
media_image4.png
Greyscale
of the patches
PNG
media_image3.png
24
26
media_image3.png
Greyscale
added to the initial point cloud model
PNG
media_image1.png
26
22
media_image1.png
Greyscale
would be points from the fitted parametric clothed body model, i.e. as claimed, generating the refined 3D representation by increasing the vertex density of the feature area(s)/deficiency(ies) of the provisional 3D representation based on the corresponding feature area(s) on the parameterized body model.
Regarding claim 2, the limitations “wherein: the subject is a human subject present at the scene; the provisional 3D representation is a point cloud representation of the human subject; and the refined 3D representation is a textured mesh representation of the human subject” are taught by Wu (Wu, e.g. title, figures 1-8, teaches that the subject is a human present at the scene. As discussed in the claim 1 rejection above, Wu, e.g. paragraph 55, teaches generating initial point clouds from the input multi view video sequence, corresponding to the claimed point cloud provisional 3D representation of the human. Finally, Wu, paragraph 93, teaches that after performing geometry refinement to generate the refined point cloud representation for each frame, Metashape is used to triangulate the point clouds and construct the texture maps for all frames, i.e. the refined 3D representation is converted to the claimed textured mesh representation.)
Regarding claim 3, the limitation “wherein the deficiency in the provisional 3D representation of the subject is identified at a region of the provisional 3D representation that corresponds to a region of the subject that is not located within any of the different fields of view of the set of cameras” is taught by Wu (Wu, e.g. paragraph 75, explains that the holes occur in regions which are not visible to any cameras, i.e. as one of ordinary skill in the art would understand, the multi-view stereo reconstruction technique for generating the initial point cloud representation discussed in paragraph 55 cannot reconstruct points which are not within the field of view of the cameras, which could cause a hole, i.e. the claimed identified deficiency being at a region which was not located within any of the fields of view of the set of cameras.)
Regarding claim 4, the limitation “wherein the deficiency in the provisional 3D representation of the subject is identified at a region of the provisional 3D representation that corresponds to a region of the subject that is occluded, within a particular one of the different fields of view in which the region of the subject is located, by an object present at the scene and further located in the particular one of the fields of view” is taught by Wu (Wu, e.g. paragraph 75, explains that the holes occur in regions which are occluded by other objects within a camera’s field of view, e.g. self occlusion cause by a human arm occluding the torso, inner thigh occluded by outer thigh, i.e. as one of ordinary skill in the art would understand, the multi-view stereo reconstruction technique for generating the initial point cloud representation discussed in paragraph 55 cannot reconstruct points which are occluded even if they are within the field of view of the cameras, which could cause a hole, i.e. the claimed identified deficiency being at a region which was occluded for one or more of the fields of view of the set of cameras. It is noted that while Wu’s examples are self-occlusion examples, one of ordinary skill in the art would understand that holes would also be caused due to occlusion by objects other than the human subject, i.e. the hole is caused by the occlusion, per se, and does not depend on the cause of the occlusion.)
Regarding claim 5, the limitations “wherein the deficiency in the provisional 3D representation of the subject is identified at a region of the provisional 3D representation that corresponds to a region of the subject that: is located within one or more of the different fields of view, and is represented by the image data at a level of detail less than a threshold level of detail” are taught by Wu (Wu, e.g. paragraphs 83-86, describes the mathematical basis for identifying a patch in the body model corresponding to a hole in the initial point cloud, which includes comparing the comparing the distances between the patch points of the body model and the points of the initial point cloud model to the distances between the patch points and other points in the body model, based on the observation that patch points in the body model will generally have larger distances to points in the initial point cloud model in comparison to distances to the points in the body model, as shown visually in figure 5A. That is, the holes/patches correspond to regions of the initial point cloud which have a low point density, i.e. are represented by the image data at a low level of detail. Further, Wu, paragraphs 84-86 uses two thresholds to differentiate hole/patch points from non-hole/patch points, such that hole/patch regions are identified by having a lower point density/level of detail than a threshold point density/level of detail, i.e. as claimed, the identified region(s) of the provisional 3D model are determined at regions which are represented in the image data at a level of detail which is less than a threshold level of detail.)
Regarding claim 10, the limitations “the determining of the set of parameters for the body model is performed prior to the identifying of the deficiency in the provisional 3D representation of the subject; the method further comprises applying, by the volumetric modeling subsequent to the application of the set of parameters to the body model, texture content to the parameterized body model, the texture content based on the image data captured by the set of cameras; and identifying the deficiency in the provisional 3D representation is performed based on an analysis of the parameterized body model to which the texture content has been applied” are taught by Wu in view of Wang (As discussed in the claim 1 rejection above, in Wu’s modified system substituting Wang’s parametric clothed body model for Wu’s SfS body model, Wang’s fitting procedure to determine fitting parameters, e.g. sections 3.2, 3.3, would be performed in place of Wu’s Mask and Shape Generation step of paragraph 79, outputting a final detail model as in the lower right of Wang, figure 1, i.e. the claimed step of determining the appropriate parameters to apply to the parametric body model to imitate the pose of the provisional 3D representation/point cloud, followed by performing colorization to the fitted parametric clothed body model as in Wu, paragraph 80, followed by identifying the hole(s) and patch(es)
PNG
media_image3.png
24
26
media_image3.png
Greyscale
for patching the initial point cloud model as in Wu, paragraphs 81-86. That is, the modified system operates using the sequence recited by the claim, determining the parameters for the body model prior to identifying the deficiency, applying texture content to the parameterized body model from the captured image data after applying the parameters, and identifying the deficiency subsequent to the application of texture content to the parameterized body model.)
Regarding claim 12, the limitation “further comprising providing, by the volumetric modeling system, the refined 3D representation of the subject to an extended reality (XR) presentation device that is configured to provide an extended reality experience to a user and to present the refined 3D representation to the user as part of the extended reality experience” is taught by Wu in view of Wang (Wu, e.g. paragraph 100, indicates that one application of the system is for virtual reality rendering, i.e. synthesizing novel viewpoints which were not captured by the cameras analogous to a bullet-time effect. Further Wang, e.g. section 4.5, suggests recovered human models can be integrated into scenes for VR/AR representations for museum, entertainment, and/or virtual dressing applications.)
Regarding claims 13 and 20, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 1 above, except for the limitations requiring a system/computing device comprising processors executing instructions stored in a non-transitory memory, which one of ordinary skill in the art would have found to be implicit, if not inherent, for implementing Wu’s modified system. That is, while neither Wu or Wang discuss computing platform structures in detail, and Wu only indicates the use of a GPU, e.g. paragraph 89, one of ordinary skill in the art of computer graphics processing would have understood that because the disclosed techniques are directed to applications of machine learning/neural networks it is implicit, if not inherent, that the systems would be implemented using processor(s) executing instructions stored in non-transitory memory.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Wu’s image based modeling and rendering system, substituting Wang’s parametric clothed human body model for Wu’s SfS body model, using processor(s) executing instructions stored in non-transitory memory because one of ordinary skill in the art of computer graphics processing would have understood that because the disclosed techniques are directed to applications of machine learning/neural networks it is implicit, if not inherent, that the systems would be implemented using processor(s) executing instructions stored in non-transitory memory.
Regarding claim 14, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 2 above.
Regarding claim 15, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 3 above.
Regarding claim 16, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 4 above.
Regarding claim 17, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 5 above.
Regarding claim 19, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claim 10 above.
Claims 6-9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2023/0027234 A1 (hereinafter Wu) in view of “Parametric Model Estimation for 3D Clothed Humans from Point Clouds” by Kangkan Wang, et al. (hereinafter Wang) as applied to claims 1 and 13 above, and further in view of “End-to-end Recovery of Human Shape and Pose” by Angjoo Kanazawa, et al. (hereinafter Kanazawa) in view of “Human Pose Estimation with Iterative Error Feedback” by Joao Carreira, et al. (hereinafter Carreira) in view of “Robust Generalized Total Least Squares Iterative Closest Point Registration” by Raul San Jose Estepar, et al. (hereinafter Estepar).
Regarding claim 6, the limitation “the determining of the set of parameters for the body model is performed using a model-fitting technique associated with an error equation that quantifies a fit of the body model to the pose of the provisional 3D representation” is taught by Wu in view of Wang (As discussed in the claim 1 rejection above, in Wu’s modified system substituting Wang’s parametric clothed body model for Wu’s SfS body model, Wang’s fitting procedure to determine fitting parameters, e.g. sections 3.2, 3.3, would be performed in place of Wu’s Mask and Shape Generation step of paragraph 79, outputting a final detail model as in the lower right of Wang, figure 1, i.e. the claimed step of determining the appropriate parameters to apply to the parametric body model to imitate the pose of the provisional 3D representation/point cloud. Wang, e.g. section 3.2, uses the objective loss function L, equation 3, to measure the loss between the posed undressed mesh using estimated parameters
PNG
media_image6.png
14
16
media_image6.png
Greyscale
and the input ground truth point cloud having points/vertices
PNG
media_image7.png
20
18
media_image7.png
Greyscale
, i.e. the claimed error equation quantifying the fit of the body model to the pose of the provisional 3D representation.)
The limitation “the model fitting technique involves iteratively adjusting and reassessing the fit of the body model of the pose of the provisional 3D representation with an objective of minimizing the error equation” is implicitly taught by Wu in view of Wang (Wang’s regression networks, e.g. section 3.2, predict the parameters
PNG
media_image6.png
14
16
media_image6.png
Greyscale
using the objective loss function L, where the term Ladv is a least-square adversarial loss used in the GAN, i.e. generative adversarial network, to discriminate whether the predicted parameters correspond to a real human shape. It is noted that Wang, e.g. section 3.2.1, indicates that the least-square adversarial loss term Ladv used in Wang’s GAN is disclosed by Kanazawa, reference 17. While one of ordinary skill in the art would understand that Wang’s regression networks are generative adversarial networks (GANs) operating by iteratively adjusting and evaluating the parameters
PNG
media_image6.png
14
16
media_image6.png
Greyscale
to minimize the objective function measuring loss/error, Wang does not disclose these details, and Kanazawa does disclose these details, and therefore in the interest of compact prosecution Kanazawa is cited for this limitation.) However, this limitation is taught by Kanazawa (Kanazawa, e.g. abstract, sections 1, 3-5, describes a human pose parameter estimation system which uses 2D images as input for 3D pose reconstruction using the SMPL model. Kanazawa, e.g. section 3, figure 2, teaches that the system relies on a GAN performing iterative 3D regression with feedback in order to estimate the SMPL shape parameters β and pose parameters θ, i.e. Kanazawa’s system operates analogously to Wang’s, except is directed to using 2D images as input rather than 3D point clouds, and is not concerned with modeling clothing details. Kanazawa, sections 3.2, 3.3, describes iterative 3D regression with feedback, which is directed to minimizing the error function comprising terms for reprojection, 3D joint and 3D SMPL parameter losses, equation 3-6, and the adversarial loss, equations 7-8. Further, Kanazawa, e.g. section 3.2, paragraph 2, indicates that the 3D regression operates in a loop, iteratively taking the image features and current estimated parameters to calculate a residual which is added to the current estimated parameters to generate the updated parameters used by the next iteration. That is, as claimed, the GAN performing 3D regression using the error/loss equation iteratively adjusts and reassesses the fit of the body model to the pose of the input representing the body with an objective of minimizing the error equation.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Wu’s image based modeling and rendering system, substituting Wang’s parametric clothed human body model for Wu’s SfS body model, using Kanazawa’s iterative 3D regression with feedback technique for Wang’s regression networks because, as noted above, one of ordinary skill in the art would understand that Wang’s regression networks are generative adversarial networks operating by iteratively adjusting and evaluating the parameters
PNG
media_image6.png
14
16
media_image6.png
Greyscale
to minimize the objective function measuring loss/error, which is explicitly taught by Kanazawa, e.g. section 3.2. It is noted that this does not necessarily require further modification to Wu’s modified system as in the claim 1 rejection, i.e. Wang does indicate the use of the objective loss function L as part of a GAN in implementing the regression network, which would appear to inherently require performing the claimed iteratively adjusting and reassessing operations, but Wang is silent regarding those details. However, in the event that Applicant were able to show that Wang does not necessarily perform the iteratively adjusting and reassessing operations, it would still be an obvious implementation possibility in view of Kanazawa’s disclosure.
The limitation “until the error equation satisfies a predetermined error threshold that represents an acceptable error in the fit of the body model to the pose of the provisional representation” is implicitly taught by Wu in view of Wang and Kanazawa (As discussed above, Kanazawa, e.g. section 3.2, paragraph 2, indicates that the 3D regression operates in a loop, iteratively taking the image features and current estimated parameters to calculate a residual which is added to the current estimated parameters to generate the updated parameters used by the next iteration. While not explicitly stated by Wang or Kanazawa, one of ordinary skill in the art would have understood that an iterative regression loop requires using one or more termination conditions for determining how many iterations to perform, i.e. conventionally a threshold number of iterations or error value threshold are used as termination conditions. However, in the interest of compact prosecution, because neither Wang or Kanazawa explicitly disclose using an error threshold as a termination condition, Carreira and Estepar are cited for explicitly teaching using an error threshold as a termination condition for an iterative regression loop.) However, this limitation is taught by Carreira in view of Estepar (Carreira, e.g. abstract, sections 1-6, describes performing human pose estimation with iterative error feedback, and is cited by Kanazawa as describing a relevant prior art iterative regression system, e.g. Kanazawa, section 3.2, citing reference 7. Carreira, section 1, paragraph 4, describes the iterative error feedback model as iteratively updating estimated output using a correction to generate an updated estimate, where the correction is an error measurement, i.e. if the correction magnitude were 0, the estimated parameters would be identical to the ground truth parameters. Further, Carreira, section 2, paragraphs 1-2, indicates that the optimization loop is performed using a stochastic gradient descent technique which iterates either for a constant number of iterations T, or using a termination condition which is a function of the correction/error value. While Carreira does not explicitly state that the termination condition corresponds to using an error threshold value, one of ordinary skill in the art would have understood that the function of the correction/error value would be used to evaluate whether the error was small enough to be considered acceptable as a final result, i.e. a condition indicating termination of the iteration. Estepar, e.g. sections 1, 1.1, describes conventional iterative closest points used for a rigid transformation which is a form of iterative least-squares regression used for estimating fitting parameters for an input shape. Estepar, e.g. section 1.1, indicates that the process is repeated until a minimum error threshold is achieved or a maximum number of iterations is reached, i.e. the claimed predetermined error threshold based termination condition.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Wu’s image based modeling and rendering system, substituting Wang’s parametric clothed human body model for Wu’s SfS body model, using Kanazawa’s iterative 3D regression with feedback technique for Wang’s regression networks, using an error threshold termination condition for the iterative 3D regression as taught by Carreira in view of Estepar because, as noted above, one of ordinary skill in the art would have understood that an iterative regression loop requires using one or more termination conditions for determining how many iterations to perform, i.e. conventionally a threshold number of iterations or error value threshold are used as termination conditions, and Carreira teaches that the loop for iterative regression with feedback is terminated using a termination condition that may be a function of the correction/error value, and Estepar teaches that the termination condition may be evaluated using a minimum error threshold. In the modified system, Wang’s regression networks would iterate until the error measured by the objective loss function L is below an error threshold value, corresponding to the claimed iterating until the error equation satisfies the predetermined error threshold representing an acceptable error of fit.
Regarding claim 7, the limitations “the error equation includes a plurality of terms representing different aspects of the fit of the body model to the pose of the provisional representation; and the error equation is configured to quantify the fit of the body model to the pose of the provisional 3D representation based on a combination of different assessments of the fit corresponding to the plurality of terms” are taught by Wu in view of Wang (As discussed in the claim 6 rejection above, Wang’s objective loss function L, equation 3, corresponds to the claimed error equation. Further, L is a weighted combination of three different terms representing different aspects of the fit, i.e. 3D correspondence loss between posed model points and ground truth points, parameter loss between estimated and ground truth parameters, and adversarial loss penalizing implausible poses and shapes.)
Regarding claim 8, the limitation “wherein the error equation includes a joint-based term configured to account for position similarity between the joints of the body model and corresponding joints of the subject as represented in the provisional 3D representation” is taught by Wu in view of Wang (As discussed in the claim 6 rejection above, Wang’s objective loss function L, equation 3, corresponds to the claimed error equation. Wang, section 3.2.2, further describes the local regression network, which uses the objective loss function L to perform iterative regression for the local neighborhood around each joint, where L includes the 3D correspondence loss term, equation 4, measuring the distance between corresponding points in the posed parameterized body model and the input point cloud. That is, the 3D correspondence loss term is evaluated for each joint neighborhood in the local regression network, measuring the similarity of the point positions between corresponding points in the neighborhood of each joint in the parameterized body model and input point cloud, corresponding to the claimed joint-based term configured to account for a position similarity between joints of the body model and corresponding joints of the subject as represented in the provisional 3D representation.)
Regarding claim 9, the limitation “wherein the error equation includes a vector-based term to account for pose similarity between vectors extending between particular joints of the body model and corresponding vectors extending between particular joints of the subject as represented in the provisional 3D representation” is taught by Wu in view of Wang (As discussed in the claim 6 rejection above, Wang’s objective loss function L, equation 3, corresponds to the claimed error equation. Wang, section 3.1, indicates that the pose parameters θ correspond to 6 dimensional vectors for 23 joints. Wang, section 3.2.1, equation 5, evaluates the ground truth parameter loss, including the loss based on the pose parameters θ, i.e. the pose parameter loss term of equation 5 corresponds to the claimed vector-based term configured to account for pose similarity between vectors extending between particular joints of the body model and corresponding vectors extending between particular joints of the subject as represented in the provisional 3D representation.)
Regarding claim 18, the limitations are similar to those treated in the above rejection(s) and are met by the references as discussed in claims 6 and 7 above.
Claims 6-9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2023/0027234 A1 (hereinafter Wu) in view of “Parametric Model Estimation for 3D Clothed Humans from Point Clouds” by Kangkan Wang, et al. (hereinafter Wang) as applied to claim 1 above, and further in view of “High-Quality Streamable Free-Viewpoint Video” by Alvaro Collet, et al. (hereinafter Collet).
Regarding claim 11, the limitation “wherein the image data captured by the set of cameras includes both color data and depth data representative of objects present at the scene; and the obtaining of the provisional 3D representation of the subject is performed by generating the provisional 3D representation of the subject based on color data and depth data representative of the subject” is partially taught by Wu (Wu, e.g. paragraph 55, teaches that the input video sequence used to generate the initial point cloud for each frame is in color, i.e. indicating each point in the point cloud is assumed to have a color computed from the input views, corresponding to the claimed image data including color data and the obtained/generated provisional 3D representation being generated based on color data. Wu, paragraph 55, further indicates that the reconstruction process uses multi-view stereo (MVS) to reconstruct the colored point cloud from the input video sequence, but does not disclose details of this process, or the use of depth image data for reconstructing the point cloud for each frame.) However, this limitation is taught by Collet (Collet, e.g. abstract, sections 1, 3-11, describes a free-viewpoint video system which records performances by humans on a stage using a plurality of cameras arranged around the performer, e.g. figure 3. Collet, e.g. sections 4, 5.1, teaches that for each camera pod, depth images are calculated, using a dense stereo algorithm, for each frame along with the captured images, corresponding to the claimed image data captured by the set of cameras including both color data and depth data representative of objects present at the scene, i.e. Applicant’s disclosure, e.g. paragraph 35, indicates that depth maps may be captured using the same video cameras capturing color image data using stereoscopic techniques. Further, Collet, e.g. section 5.3, describes combining the depth maps from all the pods to generate a combined point cloud representation. That is, Collet describes details of the processing corresponding to Wu’s disclosed point cloud reconstruction for each frame using MVS, with Collet indicating that the MVS reconstruction can be performed using the depth image data, as claimed.)
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Wu’s image based modeling and rendering system, substituting Wang’s parametric clothed human body model for Wu’s SfS body model, using Collet’s MVS reconstruction technique for reconstructing the initial point cloud for each frame from the input video sequence because Wu indicates that the reconstruction process uses MVS to reconstruct the colored point cloud from the input video sequence, but does not disclose details of this process and Collet does disclose details of MVS reconstruction, including generating depth images for each camera for each frame in order to generate the point cloud. As noted above, Applicant’s disclosure, e.g. paragraph 35, indicates that depth maps may be captured using the same video cameras capturing color image data using stereoscopic techniques, such that implementing Wu’s system using Collet’s MVS reconstruction technique, including generating the depth images for each camera for each frame in order to generate the point cloud for each frame, corresponds to the claimed image data captured by the cameras including both color data and depth data, as well as generating the provisional 3D representation based on the color data and depth data.
Response to Arguments
Applicant’s arguments, see pages 12-13, filed 1/26/26, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 102(a)(1) and 103(a) in view of Xu and Estepar have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Wu, Wang, Kanazawa, Carreira, Estepar, and Collet.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT BADER whose telephone number is (571)270-3335. The examiner can normally be reached 11-7 m-f.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at 571-272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT BADER/Primary Examiner, Art Unit 2611