DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 11 rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
The claim recites “computer-readable storage medium” but the term “computer-readable storage medium”, given its broadest reasonable interpretation read in light of the specification, does not exclude transitory forms of storage medium such as signals, which are nonstatutory (In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007)). While the specification provides several examples of forms of storage medium, these examples do not serve to form a complete definition of the term, and the meaning of the term to the ordinary artisan potentially includes transitory media such as signals, which are not statutory, as well as non-transitory media. (See also USPTO Official Gazette notice 1351 OG 213.).
Therefore, the claim encompasses nonstatutory subject matter.
Examiner suggest applicant to amendment these claim as “non-transitory storage medium” to overcome 35 U.S.C. 101 rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 21-40 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al (US 20240037827 A1) in view of Makeev et al (US 20220292791 A1).
Regarding claim 21, Zhou discloses A computer-readable storage medium storing instructions ([0058] e one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices),
for training a model to remove garment collisions in a virtual reality headset, the instructions, when executed by a computing system ([0015] digital design system that uses trained neural networks to predict three-dimensional garment deformation due to character body motion with collision handling.), cause the computing system to:
collect multiple images of a dressed avatar, each of the images having at least one known garment collision ([0022] identify information defining a body of a three-dimensional character or figure, including body shape parameters 106, β, and body pose parameters); and
for each particular image of the multiple images:
identify, with the model, a selected area of the particular image that the model estimates contains a garment collision, wherein the model utilizes the images of the dressed avatar with the known garment collision as ground-truth data by ([0023] garment prediction network 112 processes the body shape parameters 106, the body pose parameters 108, and the garment style parameters 110 to generate a set of garment vertices 114, at numeral 4.):
wherein modifying the first feature and the second feature includes defining a loss function based on the first portion or second portion ([0043] the repulsive force prediction network 122 sends the modified set of garment vertices (with a predicted offset for one or more garment vertices) to loss functions 404, as shown at numeral 5).
Makeev discloses modifying a first feature in the model when a first portion of the selected area of particular image is within the known garment collision for the particular image ([0067] , the mapping between feature points of inner and outer cages can be performed using a radial basis function (RBF) interpolation technique); and
modifying second first feature in the model when a second portion of the selected area of particular image is not within the known garment collision for the particular image ([0073] a RBF function with a key-value pair <key, value> has the form of rbf<key, value> for each pair of feature points, wherein key=the inner cage vertex position and value=outer cage vertex position)
Zhou and Makeev are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify digital design system of Zhou to include modifying a first feature in the model when a first portion of the selected area of particular image is within the known garment collision for the particular image; and modifying second first feature in the model when a second portion of the selected area of particular image is not within the known garment collision for the particular image as described by Makeev.
The motivation for doing so would have been to allow any body geometry to be fitted with any clothing geometry, including enabling layers of clothing to be fitted over underlying layer(s) of clothing, thereby providing customization without the limits imposed by pre-defined geometries or requiring complex computations to make a clothing item compatible with arbitrary body shapes of avatars or other clothing items (Makeev, [0037]).
Therefore, it would have been obvious to combine Zhou and Makeev to obtain the invention as specified in claim 21.
Regarding claim 22, Zhou discloses wherein the collecting multiple images of the dressed avatar includes collecting images with different body shapes, different poses, and different viewing angles ([0051] identify parameters defining a character body (e.g., body shape parameters and body pose parameters)).
Regarding claim 23, Zhou discloses wherein the collecting multiple images of the dressed avatar includes projecting the dressed avatar on a two-dimensional plane along a point of view of an observer in an immersive reality application ([0031] repulsive force prediction network 122 generates a modified set of garment vertices 124 using the set of garment vertices 114, the feature vector 116, and the signed distance function data 120, at numeral).
Regarding claim 24, Zhou discloses herein the collecting multiple images of the dressed avatar includes generating a three-dimensional model for a garment of an avatar body that overlaps the avatar body in a finite volume ([0032] neural network trained to determine modified positions of body-penetrating garment vertices to resolve the collision while preserving original wrinkles and other details on the garments).
Regarding claim 25, Zhou discloses wherein the modifying the second feature in the model includes projecting a three- dimensional garment model to form a two-dimensional garment view along a selected point of view to update the at least one pixel in the selected area based on the two-dimensional garment view ([0038] a training system 402 is configured to train neural networks (e.g., garment prediction network 112 and repulsive force prediction network 122) to generate offset distances for body-penetrating garment vertices to move them to positions outside the character body)
Regarding claim 26, Zhou discloses wherein the modifying the first feature in the model includes verifying that at least one pixel of the at least one known garment collision is not included in the selected area when a selected point of view of the dressed avatar is modified ([0052] the neural network of the garment prediction network 606 is trained to predict the garment vertices defining the position and shape of a deformed garment.).
Regarding claim 27, Zhou discloses A computing system for training a model to remove garment collisions in a virtual reality headset, the computing system ([0058] e one or more instructions stored on a computer-readable storage medium and executable by processors of one or more computing devices) comprising:
one or more processors ([0058] computer-readable storage medium and executable by processors of one or more computing devices.); and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to:
collect multiple images of a dressed avatar, each of the images having at least one known garment collision ([0022] dentify information defining a body of a three-dimensional character or figure, including body shape parameters 106, β, and body pose parameters); and
for each particular image of the multiple images:
identify, with the model, a selected area of the particular image that the model estimates contains a garment collision, wherein the model utilizes the images of the dressed avatar with the known garment collision as ground-truth data by ([0023] garment prediction network 112 processes the body shape parameters 106, the body pose parameters 108, and the garment style parameters 110 to generate a set of garment vertices 114, at numeral 4.):
wherein modifying the first feature and the second feature includes defining a loss function based on the first portion or second portion ([0043] the repulsive force prediction network 122 sends the modified set of garment vertices (with a predicted offset for one or more garment vertices) to loss functions 404, as shown at numeral 5).
Makeev discloses modifying a first feature in the model when a first portion of the selected area of particular image is within the known garment collision for the particular image ([0067] , the mapping between feature points of inner and outer cages can be performed using a radial basis function (RBF) interpolation technique); and
modifying second first feature in the model when a second portion of the selected area of particular image is not within the known garment collision for the particular image ([0073] a RBF function with a key-value pair <key, value> has the form of rbf<key, value> for each pair of feature points, wherein key=the inner cage vertex position and value=outer cage vertex position)
Zhou and Makeev are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify digital design system of Zhou to include modifying a first feature in the model when a first portion of the selected area of particular image is within the known garment collision for the particular image; and modifying second first feature in the model when a second portion of the selected area of particular image is not within the known garment collision for the particular image as described by Makeev.
The motivation for doing so would have been to allow any body geometry to be fitted with any clothing geometry, including enabling layers of clothing to be fitted over underlying layer(s) of clothing, thereby providing customization without the limits imposed by pre-defined geometries or requiring complex computations to make a clothing item compatible with arbitrary body shapes of avatars or other clothing items (Makeev, [0037]).
Therefore, it would have been obvious to combine Zhou and Makeev to obtain the invention as specified in claim 27.
Regarding claim 28, Zhou discloses wherein the collecting multiple images of the dressed avatar includes collecting images with different body shapes, different poses, and different viewing angles ([0051] identify parameters defining a character body (e.g., body shape parameters and body pose parameters)).
Regarding claim 29, Zhou discloses wherein the collecting multiple images of the dressed avatar includes projecting the dressed avatar on a two-dimensional plane along a point of view of an observer in an immersive reality application ([0031] repulsive force prediction network 122 generates a modified set of garment vertices 124 using the set of garment vertices 114, the feature vector 116, and the signed distance function data 120, at numeral).
Regarding claim 30, Zhou discloses herein the collecting multiple images of the dressed avatar includes generating a three-dimensional model for a garment of an avatar body that overlaps the avatar body in a finite volume ([0032] neural network trained to determine modified positions of body-penetrating garment vertices to resolve the collision while preserving original wrinkles and other details on the garments).
Regarding claim 31, Zhou discloses wherein the modifying the second feature in the model includes projecting a three- dimensional garment model to form a two-dimensional garment view along a selected point of view to update the at least one pixel in the selected area based on the two-dimensional garment view ([0038] a training system 402 is configured to train neural networks (e.g., garment prediction network 112 and repulsive force prediction network 122) to generate offset distances for body-penetrating garment vertices to move them to positions outside the character body)
Regarding claim 32, Zhou discloses wherein the modifying the first feature in the model includes verifying that at least one pixel of the at least one known garment collision is not included in the selected area when a selected point of view of the dressed avatar is modified ([0052] the neural network of the garment prediction network 606 is trained to predict the garment vertices defining the position and shape of a deformed garment.).
Regarding claim 33, Zhou discloses A computer-implemented method, comprising:
wherein the dressed avatar includes a first 3D mesh for a body of the dressed avatar and at least one second 3D mesh for clothing of the dressed avatar ([0015] a digital design system that uses trained neural networks to predict three-dimensional garment deformation due to character body motion with collision handling. Predicting how a three-dimensional garment deforms in response to the underlying three-dimensional character body motion is essential for many applications, including realistically dressed human body re-construction, interactive garment design, virtual try-on, and robotics control.);
identifying, for the projection of the dressed avatar, an area that includes a garment collision ([0016] robust collision handling based on collision detection and response computation is a critical component of cloth or garment simulation);
cropping a portion of the first 3D mesh, for the body of the dressed avatar, that includes the garment collision ([0019] digital design system uses multiple machine learning models to detect body-penetrating garment vertices and determine offsets to move the body-penetrating garment vertices outside the character body); and
replacing the cropped portion of the first 3D mesh with part of the at least one second 3D mesh for clothing of the dressed avatar, to form a new projection of the dressed avatar ([0019] ReFU further applies a flexible offset to move them, improving the overall collision handling performance. For example, this can avoid additional edge-edge (EE) collisions which normally cannot be detected by the signed distance of the vertices and overcome the artifacts in the estimated implicit functions of the human body.).
Makeev discloses forming a projection of a dressed avatar in an immersive reality application running in a headset ([0039] . Other implementations of the cage-to-cage layering techniques described herein may be applied to other types of 3D environments, such as a virtual reality (VR) conference, a 3D session (e.g., an online lecture or other type of presentation involving 3D avatars), an augmented reality (AR) session, or in other types of 3D environments that may include one or more users that are represented in the 3D environment by one or more 3D avatars.)
Zhou and Makeev are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify digital design system of Zhou to include forming a projection of a dressed avatar in an immersive reality application running in a headset as described by Makeev.
The motivation for doing so would have been to allow any body geometry to be fitted with any clothing geometry, including enabling layers of clothing to be fitted over underlying layer(s) of clothing, thereby providing customization without the limits imposed by pre-defined geometries or requiring complex computations to make a clothing item compatible with arbitrary body shapes of avatars or other clothing items (Makeev, [0037]).
Therefore, it would have been obvious to combine Zhou and Makeev to obtain the invention as specified in claim 33.
Regarding claim 34, Zhou is silent to wherein the forming the projection of the dressed avatar includes selecting a two-dimensional projection along a direction of view of a user of the headset.
Makeev discloses wherein the forming the projection of the dressed avatar includes selecting a two-dimensional projection along a direction of view of a user of the headset. ([0092] RBF interpolations are chained together in sequence to get from any level to any other level in an upward layering direction (which in the case of an avatar cage being draped with layers of clothing may correspond to an outward direction,)
Zhou and Makeev are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify digital design system of Zhou to include wherein the forming the projection of the dressed avatar includes selecting a two-dimensional projection along a direction of view of a user of the headset.as described by Makeev.
The motivation for doing so would have been to allow any body geometry to be fitted with any clothing geometry, including enabling layers of clothing to be fitted over underlying layer(s) of clothing, thereby providing customization without the limits imposed by pre-defined geometries or requiring complex computations to make a clothing item compatible with arbitrary body shapes of avatars or other clothing items (Makeev, [0037]).
Therefore, it would have been obvious to combine Zhou and Makeev to obtain the invention as specified in claim 34.
Regarding claim 35, Zhou discloses wherein the identifying the area that includes the garment collision includes running a model trained to identify a garment collision on projections of dressed avatars ([0023] a neural network includes deep learning architecture for learning representations of real-world data. A neural network may include a machine-learning model that can be tuned (e.g., trained) based on training input to approximate unknown functions).
Regarding claim 36, Zhou is silent to wherein the identifying the area that includes the garment collision includes applying a 2D supervised pixel classification model based on a viewpoint of a user of the headset.
Makeev discloses wherein the identifying the area that includes the garment collision includes applying a 2D supervised pixel classification model based on a viewpoint of a user of the headset ([0092] RBF interpolations are chained together in sequence to get from any level to any other level in an upward layering direction (which in the case of an avatar cage being draped with layers of clothing may correspond to an outward direction,)
Zhou and Makeev are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify digital design system of Zhou to include wherein the identifying the area that includes the garment collision includes applying a 2D supervised pixel classification model based on a viewpoint of a user of the headset as described by Makeev.
The motivation for doing so would have been to allow any body geometry to be fitted with any clothing geometry, including enabling layers of clothing to be fitted over underlying layer(s) of clothing, thereby providing customization without the limits imposed by pre-defined geometries or requiring complex computations to make a clothing item compatible with arbitrary body shapes of avatars or other clothing items (Makeev, [0037]).
Therefore, it would have been obvious to combine Zhou and Makeev to obtain the invention as specified in claim 36.
Regarding claim 37, Zhou discloses wherein the identifying the area that includes the garment collision includes predicting, for multiple pixels of the dressed avatar, a likelihood that the pixel indicates a body portion based on a pixel value and at least an adjacent pixel value ([0053] determine a distance value for each garment vertex in a set of garment vertices predicted by the garment prediction network 606 to a closet point on the surface of the character body).
Regarding claim 38, Zhou discloses wherein the replacing the cropped portion of the first 3D mesh includes applying coloring according to the at least one second 3D mesh ([0056] defining a character body (e.g., body shape parameters and body pose parameters) and parameters defining a garment (e.g., garment style parameters) received by the digital design system).
Regarding claim 39, Zhou is silent to wherein the identifying the area that includes the garment collision includes propagating a ray along a point of view of a user, in a direction of the area that includes the garment collision; and
wherein the cropping the portion of the first 3D mesh includes removing a portion of first 3D mesh that intersects the ray.
Makeev discloses wherein the identifying the area that includes the garment collision includes propagating a ray along a point of view of a user, in a direction of the area that includes the garment collision ([0092] RBF interpolations are chained together in sequence to get from any level to any other level in an upward layering direction (which in the case of an avatar cage being draped with layers of clothing may correspond to an outward direction,); and
wherein the cropping the portion of the first 3D mesh includes removing a portion of first 3D mesh that intersects the ray ([0056] the feature points of the outer cage of the clothing layer 400 of FIG. 4 are now combined with the feature points of the body cage 300, so as to result in a composite outer cage 500 that is made up of feature points of exposed portions of the body cage 300 and feature points along the exterior surface of the jacket.)
Zhou and Makeev are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify digital design system of Zhou to include wherein the identifying the area that includes the garment collision includes propagating a ray along a point of view of a user, in a direction of the area that includes the garment collision; and wherein the cropping the portion of the first 3D mesh includes removing a portion of first 3D mesh that intersects the ray as described by Makeev.
The motivation for doing so would have been to allow any body geometry to be fitted with any clothing geometry, including enabling layers of clothing to be fitted over underlying layer(s) of clothing, thereby providing customization without the limits imposed by pre-defined geometries or requiring complex computations to make a clothing item compatible with arbitrary body shapes of avatars or other clothing items (Makeev, [0037]).
Therefore, it would have been obvious to combine Zhou and Makeev to obtain the invention as specified in claim 39.
Regarding claim 40, Zhou discloses wherein the identifying the area that includes the garment collision includes resolving an overlap between the first 3D mesh and the at least one second 3D mesh ([0031] he modified set of garment vertices 124 includes both unmodified garment vertices (e.g., the garment vertices with positive signed distance function values and thus determined to be located at a point outside of the character body) and modified garment vertices (e.g., the garment vertices determined to be located at a point inside of the character body or at the surface of the character body)); and
wherein the replacing the cropped portion includes projecting an updated 3D clothing model on the new projection of the dressed avatar ([0031] each of the garment vertices of the modified garment vertices are moved an offset distance from the original location of the garment vertex (e.g., within the character body) to an updated location outside the character body.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIVANG I PATEL whose telephone number is (571)272-8964. The examiner can normally be reached on M-F 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached on (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIVANG I PATEL/Primary Examiner, Art Unit 2615
Prosecution Insights