DETAILED ACTION
DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 11-12 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Zhong et al. (WO 2023027712 A1, “Zhong) in view of Beri et al. ( US Patent publication: US 20180315216, “Beri”) and Chen et al. ( US Patent publication: US 20220358675 A1, “Chen”)
Regarding claim 1, Zhong teaches, A method (Method flow Fig,6A) for generating an image of a human using a neural network, the method comprising:
receiving camera parameters describing a view angle; ( “[0043]….In the pose estimation model 550, a feature extraction layer (i.e., the CNN encoder 508) is implemented by an inverted bottleneck module with skip connection, and the regression neural network 510 includes at least two fully connected layers applied to produce camera extrinsic parameter (i.e., camera pose 526 “[0053] The regression neural network 510 generates (608) a camera pose 526 (e.g., a camera position and orientation) of the camera capturing the one or more images 502.”)
receiving pose parameters describing a shape and a pose of a parametric human body model; (“[0042] The CNN encoder 508 is coupled to the human detection module 506, and configured to extract a plurality of features from the image 502 (specifically, from the human area 528 of the image 502). The regression neural network 510 is coupled to the CNN encoder 508, and configured to generate a first set of human model parameters 522 including pose parameters 522A and shape parameters 522B “)
While Zhong teaches geometry information and appearance information from an input image, Zhong doesn’t teach processing the camera parameters and the pose parameters to generate geometry information and appearance information
However Ben teaches processing the camera parameters and the pose parameters to generate appearance information (“[0053]…. In the shape blending stage, the actual shape of the object (i.e., spatial coverage) is used to determine final color to be rendered based on the knockout group.”)
Chen teaches, processing the camera parameters and the pose parameters to generate geometry information (“[0073] In the present embodiment, the camera pose may include the position and pose of the camera. The executing body may determine the viewing angle directions of the spatial points observed by the camera in the human body coordinate system, based on a position and pose of the camera and the coordinates of the spatial points in the human body coordinate system. In particular, the executing body may determine a line connecting a position of the camera and a position of a spatial point in the human body coordinate system; then, based on the pose of the camera, the viewing angle direction of the spatial point observed by the camera is determined. Here, d may be used to represent the viewing angle direction of a spatial point.”)
Zhong, Ben and Chen are analogous as they are from the field of image generation.
Therefore it would have been obvious for an ordinary skilled person the art before the effective filing date of the claimed invention to have modified Zhong to have included processing the camera parameters and the pose parameters to generate geometry information and appearance information as taught by Chen’s teaching of processing the camera parameters and the pose parameters to generate geometry information and Ben’s teaching of processing the camera parameters and the pose parameters to generate appearance information.
The motivation for the above is to sequentially create pose and shape information, geometry and appearance information and final image as an of choice ( alternative to parallel creation of pose and shape information, geometry and appearance information) by designer.
Zhong as modified by Beri and Chen teaches, processing geometry information and the appearance information to generate the image of the human, the image depicting the human viewed from the view angle and with the body of the human having the shape and the pose described by the pose parameters. (Zhong, Geometry information 524A in Fig. 5, appearance information is vertex color 624B “[0044] The 3D human pose estimation module 512 is coupled to the regression neural network 510 and forms a 3D human model 520 (e.g., a skinned multi-person linear (SMPL) model) based on the first and second sets of human model parameters 522 and 524 and camera pose 526. The first and second sets of human model parameters 522 and 524 describe the human body of the SMPL model. The avatar rendering module 516 is coupled to the 3D human pose estimation module 512, and configured to render the avatar 504.”)
Regarding claim 11, Zhong as modified by Beri and Chen teaches, wherein SMPL poses parameters 522A for 24 joints of the human body, (Zhong, [0043]….. the pose parameters include Skinned Multi-Person Linear model parameters.”)
Regarding claim 12, Zhong as modified by Beri and Chen teaches, wherein the geometry information characterizes a 3D geometry of the human, (Chen, [0071] In the present embodiment, when the executing body uses the global rotation parameter and the global translation parameter to convert a human body image frame in the sample video from the camera coordinate system to the human body coordinate system, it may also determine the spatial points of the human body image frame in the human body coordinate system corresponding to the pixels in the human body image frame, based on the global rotation parameter and the global translation parameter. It may be understood that the coordinates of a pixel are two-dimensional, and the coordinates of a spatial point are three-dimensional. Here, the coordinates of a spatial point may be represented by x”). and the appearance information characterizes a RGB appearance of the human. ((“[0053]…. In the shape blending stage, the actual shape of the object (i.e., spatial coverage) is used to determine final color to be rendered based on the knockout group.”)
The motivation is similar to motivation used in claim 1.)
Claim 25 is directed to a system comprising one or more processors and one or more storage devices (Zhong, “ [0005] In another aspect, some implementations include a computer system that includes one or more processors and memory having instructions stored thereon, which when executed by the one or more processors cause the processors to perform any of the above methods” ) and its elements are similar in scope and function of the steps of claim 1 and therefore claim 25 is rejected with same rationales as specified in the rejection of claim 1.
Allowable Subject Matter
Claims 13-19 are allowed.
The following is an examiner’s statement of reasons for allowance:
Claim 13 is allowable because Zhong teaches, A method of training the neural network of any preceding claim, the method comprising:
(a) generating a training image of a human by providing camera parameters describing a view angle; ( “[0043]….In the pose estimation model 550, a feature extraction layer (i.e., the CNN encoder 508) is implemented by an inverted bottleneck module with skip connection, and the regression neural network 510 includes at least two fully connected layers applied to produce camera extrinsic parameter (i.e., camera pose 526 “[0053] The regression neural network 510 generates (608) a camera pose 526 (e.g., a camera position and orientation) of the camera capturing the one or more images 502.”)
providing pose parameters describing a shape and a pose of a parametric human body model; (“[0042] The CNN encoder 508 is coupled to the human detection module 506, and configured to extract a plurality of features from the image 502 (specifically, from the human area 528 of the image 502). The regression neural network 510 is coupled to the CNN encoder 508, and configured to generate a first set of human model parameters 522 including pose parameters 522A and shape parameters 522B “)
While Zhong teaches geometry information and appearance information from an input image, Zhong doesn’t teach processing the camera parameters and the pose parameters by a generator neural network, configured to:
(i) generate geometry information and appearance information, and
However Ben teaches processing the camera parameters and the pose parameters to generate appearance information (“[0053]…. In the shape blending stage, the actual shape of the object (i.e., spatial coverage) is used to determine final color to be rendered based on the knockout group.”)
Chen teaches, processing the camera parameters and the pose parameters to generate geometry information (“[0073] In the present embodiment, the camera pose may include the position and pose of the camera. The executing body may determine the viewing angle directions of the spatial points observed by the camera in the human body coordinate system, based on a position and pose of the camera and the coordinates of the spatial points in the human body coordinate system. In particular, the executing body may determine a line connecting a position of the camera and a position of a spatial point in the human body coordinate system; then, based on the pose of the camera, the viewing angle direction of the spatial point observed by the camera is determined. Here, d may be used to represent the viewing angle direction of a spatial point.”
and (Zhong, Geometry information 524A in Fig. 5, appearance information is vertex color 624B “[0044] The 3D human pose estimation module 512 is coupled to the regression neural network 510 and forms a 3D human model 520 (e.g., a skinned multi-person linear (SMPL) model) based on the first and second sets of human model parameters 522 and 524 and camera pose 526. The first and second sets of human model parameters 522 and 524 describe the human body of the SMPL model. The avatar rendering module 516 is coupled to the 3D human pose estimation module 512, and configured to render the avatar 504.”)
The combination of prior art fails to expressly teach, (b) processing of the training image of the human, by a discriminator neural network module to generate a prediction of whether the training image of a human is an image of a real human or an image of a fake human, and (c) modifying one or more network parameters of the generator neural network and the discriminator neural network based on the prediction.
Claims 14-19 are allowed by virtue of dependency..
Claims 2-10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 2 is objected because the combination of available prior art fails to expressly teach, wherein processing the camera parameters and the pose parameters to generate geometry information and appearance information includes:
processing the camera parameters to generate a representation of a first 3D space comprising a human in a predetermined pose;
obtaining one or more index locations based on the camera parameters and the pose parameters;
generating the geometry information and the appearance information from the representation by sampling the representation at the one or more index locations.
Claims 3-8 are objected by virtue of dependency.
Claim 9 is objected because the combination of available prior art fails to expressly teach, wherein processing the geometry information and the appearance information to generate the image of a human includes:
processing, by a volume rendering module of the neural network, the geometry information and the appearance information to generate a feature image and a RGB image, and
processing, by a decoder module of the neural network, the feature image and the RGB image to generate the image of the human.
Claim 10 is objected by virtue of dependency.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
LU et al. ( CN 113822977 A) teaches generating a trained model for image generation based on the camera parameter, shape parameter. ( Abstract)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tapas Mazumder whose telephone number is (571)270-7466. The examiner can normally be reached M-F 8:00 AM-5:00 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAPAS MAZUMDER/Primary Examiner, Art Unit 2615
Prosecution Insights