Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Yao et al. (CN 112401369 A, hereinafter Yao, see English Translation), in view of Zhou et al. (US 2019/0096050 A1, hereinafter Zhou), and further in view of Diaz-Arias et al. (US 2022/0386942 A1, hereinafter Diaz).
Regarding claim 1, Yao teaches a visual measurement method based on a digital human model, comprising the following steps: ([0007] “To achieve the above objectives, the present invention provides a method for measuring body parameters based on human reconstruction, comprising:”)
S2: matching and aligning the second data with the digital human model, ([0010] “Step 3: Perform non-rigid deformation on the human body model to make it consistent with the template model;”, where “As a preferred option, after the mesh is aligned, the deformation module performs non-rigid deformation registration: each vertex on the template model is rotated and translated so that the position of each vertex is closest to the corresponding point on the human body model.” [0090]. Note: the “template model” is mapped to the second data, and the “human body model” is mapped to the digital human model.)
and establishing a correspondence between the second data and the digital human model through key feature point matching and shape registration, to obtain third data; and ([0073] “More importantly, this embodiment fits the motion through a skeleton-driven approach, but in the shape fitting part, a non-rigid deformation registration is performed on the template model. This optimizes the local geometric details of the template model to be consistent with the original human body model, so that the template model can more accurately represent the shape of the human body model, thereby ensuring the accuracy of the measurement.”. Note: the “rigid body transformation” of the human body model, where the “corresponding points” from “rigid body transformation to make the centroids coincide” is mapped to key feature point matching, and “shape fitting” is mapped to shape registration.)
S3: extracting key feature points from the third data and optimizing the key feature points, ([0017] “Furthermore, in step three, the centroids of the template model and the human body model are aligned, and after extracting the corresponding points, the nearest neighbor point is iteratively optimized to achieve coarse alignment.”, where “When optimizing the error between corresponding points of the human body model and the template model, only these sampled key points are optimized.” [0073]. Note: the “sampled key points” is mapped to key feature points.)
and finally obtaining morphological parameters according to the optimized key feature points. ([0011] “Step 4: Extract the vertex coordinates of the deformed template model and calculate the body parameters.”, where “this embodiment describes a method that fits a human body model containing noise information such as clothing folds using a human body template model, and then measures various body circumference parameters such as waist circumference. This method can measure the length and circumference of multiple parts of the body, is fast and accurate, and is a fully automatic, non-contact measurement method” [0107]. Note: the “body circumference parameters” are mapped to morphological parameters.).
Yao fails to teach constructing a digital human model according to obtained three-dimensional (3D) data, and performing pose estimation on the digital human model to obtain first data. However, this is known in the art as taught by Zhou.
Zhou teaches constructing a digital human model according to obtained three-dimensional (3D) data, ([0018] “a device for three-dimensional reconstruction, comprising: a first obtaining module, configured to obtain a first depth map, which is photographed by a first photographic device, and obtain a second depth map, which is photographed by a second photographic device; a calculation module, configured to merge the first depth map with a first three-dimensional model according to a position of the first photographic device to obtain a second three-dimensional model;”, where “Step 601, obtaining a three-dimensional model of a measured object.” [0062], and “Optionally, the measured object may be a human body.” [0063]).
and performing pose estimation on the digital human model to obtain first data ([0072] “The human body fitting process may be: inputting a human body scanning model, iteratively performing pose fitting and/or shape fitting according to the three-dimensional human body database until convergence”, where “The pose fitting (or pose estimation) may also refer to estimating the pose (comprising three-dimensional spatial position and rotation angle) of the joint point according to the reconstructed three-dimensional model.” [0073]).
Zhou was analogous to the claimed invention, as both relate to generating and measuring three-dimensional human body models. Zhou further teaches that “the method disclosed by the embodiment of the present invention has the advantages that the measurement method based on fitting of the present invention is high in speed, stable and reliable in result and high in expandability.” [0076]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Zhou to Yao in order to have a faster, more reliable three-dimensional reconstruction and measurement using human body models.
The combination of Yao and Zhou fails to teach preprocessing the first data to obtain second data. However, this is known in the art as taught by Diaz.
Diaz teaches preprocessing the first data to obtain second data ([0065] “The set of 3D representations of the skeleton can be denoised based on the at least one pose to produce a set of refined (e.g., with less noise) 3D representations of the skeleton.”, where “the 3D representations of the skeleton (also referred to as the 3D pose estimate)” [0042]. Note: denoising is an example of preprocessing, see Specifications [0013]).
Diaz is analogous to the claimed invention, as both relate to generating 3D representations of a human body. Diaz further teaches that preprocessing after pose estimation allows the invention “to produce a set of refined (e.g., with less noise) 3D representations of the skeleton” [0065]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Diaz to the combination of Yao and Zhou in order to generate a more refined 3D human model.
Regarding claim 10, claim 10 recites substantially similar limitations to claim 1, but in a system form. The combination of Yao, Zhou, and Diaz teaches a data acquisition module, (Zhou; [0021] “The human body modeling module is used to generate 3D human body models based on 3D data.”, and “The human body model processing module is used to preprocess the 3D human body model to obtain an optimized 3D human body model.” [0022])
a data matching module, (Yao; [0023] “The deformation module performs non-rigid deformation on the human body model to make it consistent with the template model.”)
and an extraction and optimization module, (Yao; [0024] “The calculation module extracts the vertex coordinates of the deformed template model and calculates the body parameters.”).
Zhou is analogous to the claimed invention, as both relate to generating 3D human body models from captured 3D data. Zhou further teaches that the invention “provid[es] high-precision 3D human body models.” [0010]. Therefore, it would be obvious for one of ordinary skill in the art before the effective date of the claimed invention to incorporate the teachings of Zhou to Yao in order to generate 3D human models with high precision.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Yao, in view of Zhou and Diaz, and further in view of Li et al. (US 2024/0242430 A1, hereinafter Li) and Du (CN 111145330 A, see English Translation).
The combination of Yao, Zhou, and Diaz teaches adjusting positions and weights of the key feature points in the digital human model during the key feature point binding, (Yao; [0070] “After bone registration, skinning is required, which involves establishing the relationship between bone points and mesh vertices and deforming the mesh surface to fit the current bone pose. Each skeletal point is associated with many vertices, and each vertex has a different weight”, where “First, obtain the 3D coordinates of the skeleton points of the template model, and calculate the weight w of each skeleton point and its surrounding mesh vertices” [0072])
wherein the weight is a parameter for quantifying an influence degree of the key feature point on the digital human model; (Yao; [0070] “Each skeletal point is associated with many vertices, and each vertex has a different weight. When a skeletal point undergoes rotation or translation, its associated mesh vertices will also undergo positional changes to varying degrees”).
The combination of Yao, Zhou, and Diaz fails to teach wherein said constructing a digital human model specifically comprises the following steps: designing a basic image of the digital human model using 3D modeling software, and performing texture mapping processing on the digital human model, wherein the texture mapping processing comprises adding colors, materials, and textures. However, this is known in the art as taught by Li.
Li teaches wherein said constructing a digital human model specifically comprises the following steps: designing a basic image of the digital human model using 3D modeling software, ([0035] “Virtual image: refers to a kind of character image in virtual space that may represent and display characteristics of human image generated through artistic creation or three-dimensional modeling design.”)
and performing texture mapping processing on the digital human model, wherein the texture mapping processing comprises adding colors, materials, and textures; ([0036] “Three-dimensional human body model: a three-dimensional model… may express information such as surface shape, color, and illumination of an object in three-dimensional vision, and generally consists of mesh data and texture data. A three-dimensional model for displaying a human body is a three-dimensional human body model.”, and “Facial reconstruction… The process includes construction of the three-dimensional facial model, as well as construction of a material of the object, color, and information related to it and appearance of the object itself.” [0037]).
and adding expression animation corresponding to the digital human model using 3D animation software ([0120] In one example, an animated skeleton system may be set, the animated skeleton system may include skeleton points of a full-body from a face to a body, and each of the skeleton points may be expressed in a form of a three-dimensional coordinate. The skeleton points may be used for guiding movements of a model, for example, controlling and transforming movements of the model based on the skeleton points.”, where “the three-dimensional human body model constructed by the model reconstruction method provided by the embodiment of the present disclosure may be directly interacted with the 3D engine, which is convenient for a user to carry out animation editing through the 3D engine.” [0126])
Li is analogous to the claimed invention, as both relate to generating 3D human body models. Li further teaches that the model would allow “high-precision human body models are established for performers according to actual needs, which may restore characteristics and artistic effects of characters themselves to a greater extent.” [0003]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Li to the combination of Yao, Zhou, and Diaz in order to generate more accurate human body models with realistic characteristics.
The combination of Yao, Zhou, Diaz, and Li fails to teach performing rendering enhancement on the digital human model using preset rendering settings, wherein the preset rendering settings comprise adjusting lighting, adjusting shadows, adding background, and adding environment. However, this is known in the art as taught by Du.
Du teaches performing rendering enhancement on the digital human model using preset rendering settings, wherein the preset rendering settings comprise adjusting lighting, adjusting shadows, ([0034] “In this technical solution, after acquiring the human body model, direct lighting, indirect lighting, and subsurface scattering are added to the human body model. Since the human body model includes 3D human body model data and skin texture data, adding direct lighting can produce the effect of reflecting highlights and casting shadows on the human body model”, where “The rendering module is used to add direct lighting, indirect lighting, and subsurface scattering to the human body model and then render it.” [0029])
adding background, and adding environment. ([0072] “The specified environment map can refer to the background image of the human body model that the user can select through the above application, and the indirect lighting is added to the human body model based on the background environment image selected by the user relative to the human body model as the environment map.”)
Du is analogous to the claimed invention, as both relate to rendering 3D human body models. Du further teaches that “In this way, the rendered human body model can show the details of skin texture, making the human body model more three-dimensional and more realistic.” [0034]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Du to the combination of Yao, Zhou, Diaz, and Li in order to generate more realistic human body models.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Yao, in view of Zhou and Diaz, and further in view of Bai et al. (US 2022/0270387 A1, hereinafter Bai).
The combination of Yao, Zhao, and Diaz teaches the visual measurement method based on a digital human model according to claim 1, wherein said matching and aligning the second data with the digital human model specifically comprises the following steps: …performing spatial transformation on the digital human model according to a result of the feature point matching, wherein the spatial transformation comprises translation, rotation, (Yao; [0073] “The method for non-rigid deformation registration is to perform rotation and translation transformations on each vertex of the template model so that the position of each vertex is closest to the corresponding point on the human body model, while the entire template model remains smooth locally.”)
adjusting a pose and shape of the digital human model using a 3D deformation method and an optimization algorithm. (Yao; [0072] “During the bone deformation registration stage, the transformation matrix T of each bone point can be obtained. vp is the mesh vertex before transformation, and v’p is the mesh vertex after transformation. Through this formula, the coordinates of each vertex on the mesh after pose transformation can be obtained, thus obtaining the new mesh.”, and “after coarse alignment, the shape differences between the template model and the human body model are optimized to obtain a template model that can be used to represent the human body model after denoising and deformation.”[0018]).
However, the combination of Yao, Zhao, and Diaz fails to teach generating preset feature points through the digital human model, wherein the preset feature points correspond to the key feature points, and performing feature point matching on feature points in the second data using a feature matching algorithm and the preset feature points, wherein the feature point matching represents matching between the feature points in the second data and the preset feature points and wherein the spatial transformation comprises …scaling. However, this is known in the art as taught by Bai.
Bai teaches generating preset feature points through the digital human model, wherein the preset feature points correspond to the key feature points, ([0160] “A reference body surface feature point B2 is arranged near each reference bone point A2. In the three-dimensional coordinate system, the coordinates of the reference bone points A2 and the coordinates of the reference body surface feature points B2 can be converted through a preset conversion relationship.”)
and performing feature point matching on feature points in the second data using a feature matching algorithm and the preset feature points, wherein the feature point matching represents matching between the feature points in the second data and the preset feature points; ([Abstract] “obtaining sizes of body parts of the user; determining to-be-moved target positions of each reference body surface feature point and the reference bone point of each set of reference control points of the reference human body model; adjusting each reference body surface feature point and each reference bone point of the reference human body model to the to-be-moved target positions to obtain the customized human body model of the user.", where “the reference three-dimensional human body model having multiple reference body surface feature points marked on a human body surface and multiple reference bone points marked on a human skeleton” [0019])
and wherein the spatial transformation comprises …scaling ([0025] “Performing a scaling process on the standard human body model according to the human body size to obtain the reference human body model matching the human body size of the user”).
Bai is analogous to the claimed invention, as both relate to generating 3D human body models. Bai further teaches that the invention aims “to provide users with customized human body models to enhance user experience.” [0003], as “the human body model generated by manual modeling has a lower matching degree with the user's real body shape characteristic, lacks reality, and cannot significantly improve the user experience.” [0004]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Bai to the combination of Yao, Zhao, and Diaz in order to generate customized human body models that are realistic to the user’s real body shape.
Allowable Subject Matter
Claims 2, 4-6, and 8-9 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, the closest prior art of Yao teaches the preset data set is a set of digital human model images with preset poses ([0029] “the neural network algorithm model of the reconstruction module is trained using 500 high-precision 3D human body models acquired by hundreds of color camera arrays”)
and training and predicting a ([0029] “The training model enables it to predict the probability of 3D points inside the 3D human body model from the image, and then reconstruct the 3D human body model.”).
However, Yao fails to teach constructing a preset data set based on existing data, wherein the existing data comprises morphological measurement data, facial feature data, and motion and expression data, and the preset data set is a set of digital human model images with preset poses and corresponding pose annotation information; training and predicting a preset model using the constructed preset data set, to obtain a prediction result of the model pose estimation; and after the training, inputting pose information of the digital human model into the trained preset model, and outputting predicted key feature points, to obtain the first data.
Furthermore, no prior art of record either alone or in combination teaches the limitation above as a whole. Therefore, claim 2 is considered allowable.
Regarding claim 4, the closest prior art of Talegaonkar et al. (US 2025/0131647 A1, hereinafter Talegaonkar) teaches denoising, enhancement… the denoising comprises weakening a noise component in the first data using a machine learning algorithm; the enhancement comprises improving contrast, brightness ([0030] “Processing by the ISP can be performed by a plurality of filters or processing blocks being applied to the captured image, such as denoising or noise filtering, edge enhancement, color balancing, contrast, intensity adjustment (such as darkening or lightening), tone adjustment, among others.”, where “For instance, various ISP operations (e.g., ISP processing blocks, including one or more of the processing blocks described above) can be implemented using one or more machine learning networks.” [0051])
However, Talegaonkar fails to teach wherein said preprocessing comprises denoising, enhancement, and correction; the enhancement comprises improving contrast, brightness, and clarity of the first data through sharpening, to increase a proportion of high-frequency components in the first data; and the correction comprises correcting the first data through radiation correction to eliminate distortion and errors of the first data.
Furthermore, no prior art of record either alone or in combination teaches the limitation above as a whole. Therefore, claim 4 is considered allowable.
Claims 5 and 6 contain allowable subject matter because they depend on claim 4, which contains allowable subject matter.
Regarding claim 8, the closest prior art of Yao teaches extracting and optimizing key feature points ([0017] “Furthermore, in step three, the centroids of the template model and the human body model are aligned, and after extracting the corresponding points, the nearest neighbor point is iteratively optimized to achieve coarse alignment.”), however, fails to teach wherein said extracting and optimizing key feature points specifically comprises the following steps: obtaining a preset area and positions of preset feature points using a target detection method, and filtering and correcting the preset feature points with reference to structural information of the digital human model and priori knowledge; extracting the preset feature points again using a convolutional neural network model, and training the digital human model to learn representation and positioning methods of the key feature points to enable recognition of the key feature points by the digital human model; and detecting an overlap degree between the key feature points using a non-maximum suppression method, to remove redundancy and overlap, and verifying and evaluating accuracy and reliability of the extracted key feature points by using a preset data set.
Furthermore, no prior art of record either alone or in combination teaches the limitation above as a whole. Therefore, claim 8 is considered allowable.
Claim 9 contains allowable subject matter because they depend on claim 4, which contains allowable subject matter.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALICIA HA whose telephone number is (571)272-3601. The examiner can normally be reached Mon-Thurs 9:30 AM - 6:30 PM, and Fri 9:30 AM - 1:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALICIA HA/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
