Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Response to Arguments
Applicant's arguments filed 03/11/2026 have been fully considered but they are not persuasive. For the new limitations to claim 1, the arguments made in how the amendment differs from prior art is not persuasive.
Applicant’s arguments on page 11 discuss amending claim 1 to include the following “the representation of the fit comprising a heat map indicating a looseness or tightness of the clothing item in multiple locations vis-a-vis the measurements of the subject person, the heat map being displayed with a fitting key.” These reflect the limitations of the original claim 10. Pg. 12 of applicant’s arguments discusses the context of each of the references and correctly states that Kanani alone does not teach a heat map indicating looseness or tightness. On pg. 12, applicant asserts Santesteban teaches a deformable mesh indicating garment deformation but that it is not a heat map.
Pg. 13 of applicant’s arguments discusses what Kim teaches and asserts that the heatmap of Kim compares two garments, but does not show how a garment fits on a particular body. On the contrary, the examiner asserts Kim does teach a heatmap indicating the looseness or tightness of different areas of a garment relative to a user’s body, as taught below in the rejection for claim 1. As for most of the amendments in the dependent claims, the original art was used to reject those limitations. However, for claims 7-12, new limitations are amended into them and in some cases, new grounds of rejection were used below.
Pg. 14 of applicant’s arguments asserts that because claims 2-14 and 16-19 depend from claims 1 or 15, the rejections of the dependent claims should be withdrawn. However, the examiner maintains the amended claims 1, 15, and 20 are still unpatentable under 103.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8, 10-12, 15-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kanani (US 20220258049 A1) in view of Kim (Pub No. US 11004133 B1).
As per claim 1, Kanani teaches the claimed:
1. A method comprising: receiving, by a processing device, an image of a subject person and a selection of a clothing item; (Kanani [0014]: “In certain embodiments, the software application may comprise two modules in communication with each other—a retailer (or backend) module, and a user (or frontend) module. The retailer module may be deployed by a clothing retailer to offer a selection of virtual apparels to its customers to try-on and interact with in real-time before placing an order for the desired clothing. Using Augmented Reality (AR) techniques, the retailer module may generate an augmented image (or video frame) of the user in real-time, with a user-selected virtual apparel fitted on the user. The user module, on the other hand, may be installed on the user equipment (UE) to allow the user to capture and send the user's body measurements to the retailer module and also to transmit user interactions for processing by the retailer module. As mentioned before, an apparatus controller may be operated by the user to interact with a specific virtual apparel. The apparatus controller may locally communicate with the user module, for example, via a Bluetooth® link with the UE. Based on the inferred intent of the user interaction, the backend module may modify—in real-time—the AR image of the user to allow the user to control how the virtual apparel looks on the user in real-time and under different poses/movements.”).
determining, using a machine-learning model, measurements of the subject person, the measurements relatable to one or more dimensions of the clothing item; (Kanani [0089]: “FIG. 7A illustrates an exemplary architecture of various software modules in the retailer module 102 of FIG. 4, and FIG. 7B illustrates an exemplary interaction among various software modules of the VCI application 100 as per certain embodiments of the present disclosure. FIG. 7A is a high level illustration of the deep learning architecture of the retailer module 102. It is noted that the interconnections among various modules/units shown in FIG. 7A is exemplary only. In other embodiments, additional or different modules may be deployed and trained, and they may be operatively coupled in a manner different from that illustrated in the embodiment of FIG. 7A. Like any other well-known body mesh generating system, in the retailer module 102, a 3D generative model for learning parameters is fitted on established corpus and discriminative models are trained for regressing the pose and shape of the human body in order to predict the 3D body mesh as well as clothing/apparel mesh. As shown in FIG. 7A, the measurement server 402 may include a deep CNN 700 and an IUV extraction unit 702. At run-time both units may receive measurement data from the user interface unit 408 in the user module 104 (FIG. 4).”).
determining, using the machine-learning model, a fit of the clothing item on the subject person; (Kanani abstract: “The user can interact with the virtual apparel for identifying, defining, and changing the look, fit, and design of the apparel on the user's body. The real-time interaction is with the same virtual apparel. The system defines operations based on user's features, sartorial measurements, intent, gestures, position, pressure values received from a controller operated by the user, and the sensed motion of the user to translate into a set of machine learning inference models that predict a series of states that visually generate the outcome the user anticipates based on the user's interaction with the virtual clothing.”).
and displaying, by the processing device via a display, a representation of the fit of the clothing item on the subject person overlayed on a reproduced image of the subject person wearing the clothing item. (Kanani abstract “intent, gestures, position, pressure values received from a controller operated by the user, and the sensed motion of the user to translate into a set of machine learning inference models that predict a series of states that visually generate the outcome the user anticipates based on the user's interaction with the virtual clothing.” Kanani describes a body mesh based on the person, which is the reproduced image. Kanani then describes mapping a structure for a clothing item onto the body mesh. This is the overlay onto a reproduced image. Kanani [0093]: “The cloth material (which may be defined by colors and fabric) may be inverse UV mapped onto the mesh and reconstructed using a Recurrent CNN (RCNN) with a standard Softmax function. More specifically, the measurement co-ordinates from the garment database (which may be a part of the database 216 a noted before) may be mapped onto the user's UV body mesh generated by the measurement server 402. Thereafter, the applicable garment is considered in context (for example, gender, size, style, texture, and material—as provided by the retailer's images for the front and the back of the garment. As mentioned before, in particular embodiments, such garment-related content (and other retailer-provided data) may be stored in the garment database within the database 216. The shape regressor 710 may transfer the garment's context (in the form of UV images of the garment) to the IUV container 712, which may create an IUV structure for the garment that can be mapped onto the person's UV body mesh.”).
Kanani alone does not explicitly teach the remaining claim limitations.
However, Kanani in combination with Kim teaches the claimed:
the representation of the fit comprising a heat map indicating a looseness or tightness of the clothing item in multiple locations vis-à-vis the measurements of the subject person, the heat map being displayed with a fitting key. (Kim col. 16 line 45-col. 17 line 7: “The graphical representation 700 may include a heat map 706 that visually summarizes the relationship between the two garments. The heat map 706 may leverage various indicators (e.g., colors, hatching, patterns, etc.) to represent data. A key 708 to the heat map is shown in graphical representation 700. For at least one fit characteristic (e.g., leg fit) the heat map may leverage a first indicator to indicate that the score corresponding to the fit characteristic in a first garment (e.g., Brand A jeans) is greater than a corresponding score in a second garment, the garment being used for comparison (e.g., Brand X Modern Curve jeans). For instance, for the leg fit characteristic, the Brand A jeans are slightly looser than the Brand X Modern Curve jeans as shown by the corresponding hatching on the heat map 706. The heat map 706 also includes a textual summary “slightly looser leg.” For other fit characteristics (e.g., backside fit), the heat map may leverage a second indicator to indicate that the score corresponding to the fit characteristic in a first garment (e.g., Brand A) is less than a corresponding score in the second garment, the garment being used for comparison (e.g., Brand X Modern Curve). For instance, for the backside fit characteristic, the Brand A jeans are tighter than the Brand X Modern Curve as shown by the corresponding hatching on the heat map. The heat map 706 also includes a textual summary “tighter backside.” The graphical representation 700 is one example of a graphical representation that visually summarizes fit characteristics associated with two garments, and any other presentation or configuration may be used”).
As per claims 15 and 20, these claims are similar in scope to limitations recited in claim 1, and thus are rejected under the same rationale. Kanani teaches a device. Kanani [0012]: “As a solution, particular embodiments of the present disclosure relate to a system and method that allows a user with a smartphone or tablet or other wearable device (laptop/desktop) to define retail adjustment operations on a virtual apparel/clothing in real-time using an AR-based visual interface and the user's fingertips. ” Kanani also teaches a computer readable medium. Kanani [0017]: “In a further embodiment, the present disclosure is directed to a computer program product comprising a non-transitory computer-usable medium having computer-readable program code embodied therein, wherein the computer-readable program code, when executed by a computing system, causes the computing system to implement a method.”
As per claim 2, Kanani teaches the claimed:
2. The method of claim 1, wherein the selection of the clothing item indicates a selected size of the clothing item. (Kanani concerns trying out different sizes of clothing on a model of a person. Kanani [0009]: “As mentioned before, the virtual try-on of 3D clothing in real-time remains a challenge because of the need to adjust the shape, size, and orientation of the clothing as per the contours of the user's body in real-time. The deformation needed to make the virtual clothing appear realistic on the user may need to be adjusted in real-time as per the user movement/motion. Furthermore, a virtual try-on solution can be truly beneficial only when it allows the user to interact with the 3D virtual clothing in real-time to find the right-fitting apparel and also to control how a designated clothing item looks on him/her in real-time”
Kanani teaches selection based on the size of the virtual apparel. Kanani [0062]: “In some embodiments, a virtual apparel may be selected by the user 212 from those displayed on a retailer's website or recommended by the retailer's system 202 based on a number of attributes (some or all may be selectable by the user)—such as the gender of the user, the style of the virtual apparel (the style also may include the characteristics of an apparel such as stitching, pleating, and so on), the size of the virtual apparel, the material of the virtual apparel, the texture of the virtual apparel, and physical effects (such as gravity) on the virtual apparel. The selected virtual apparel then may be displayed as fitted on the corresponding body portion of the user. In other embodiments,”).
As per claim 3, Kanani teaches the claimed:
3. The method of claim 1, wherein the method further comprises determining or looking up the one or more dimensions of the clothing item and at least one of stretchiness, material, drape, or color of the clothing item as inputs for determining the fit of the clothing item on the subject person. (Kanani teaches a user choosing attributes of the apparel, including the size and material. Kanani claim 6: “The method of claim 1, wherein displaying the real-time image comprises: allowing, by the computing system, the user to choose one or more of the following attributes related to the virtual apparel the user wishes to try on the corresponding body portion of the user: a style of the virtual apparel, a size of the virtual apparel, a material of the virtual apparel, a texture of the virtual apparel, and physics effects on the virtual apparel; selecting, by the computing system, the virtual apparel having the one or more attributes chosen by the user; and displaying, by the computing system, the selected virtual apparel fitted on the corresponding body portion of the user in the real-time image.”).
As per claim 4, Kanani teaches the claimed:
4. The method of claim 3, wherein the one or more dimensions of the clothing item include at least two of shoulder width, waist width, waist circumference, inseam length, hip circumference, sleeve length, collar opening diameter, chest width, or chest diameter. (Kanani teaches choosing clothing items based on at least waist and hip circumference. Kanani [0055]: “Referring now to the flowchart 300 in FIG. 3A, initially, the computing system (for example, the UE 204 and/or the host system 202) may wirelessly obtain sartorial measurements of a human user (block 302). As discussed in more detail later with reference to FIG. 4, the sartorial measurements as per particular embodiments of the present disclosure may include more than typical clothing-related body measurements—such as, for example, waist size, shoulder length, hip measurements, wrist size, neck width, and so on—of the user (such as the user 212 in FIG. 2). In a virtual world, where someone like a human tailor is absent and where all body measurements are generated electronically and remotely/wirelessly, the body measurements alone may not suffice to convey sufficient information necessary to accurately render a virtual apparel for the best fit. For example, even if a user indicates to the VCI application 100 that he/she wears a t-shirt of “small” size, that, in and of itself, does not provide sufficient information to generate sartorial measurements of the user… The size of the virtual apparel may be selected based on the sartorial measurements of the relevant body portion(s) of the user. The real-time display mentioned at block 303 may generate an augmented image of the user in real-time. In particular embodiments, the augmented image may be based on AR techniques and may be displayed to the user as a real-time video frame as discussed later. In other embodiments, the augmented image may be displayed as a combination of video frames, objects, and likeness stacked on top of each other to be viewed as a single video frame to a human eye.”).
As per claim 5, Kanani anticipates the claimed:
5. The method of claim 1, wherein the machine-learning model comprises a parametric model that generates a representation of the subject person using a human mesh model with the measurements of the subject person. (Kanani [0005]: “For example, in case of try-on of a virtual watch, ARTag technology may be used to generate a band printed with specific markers. The band may be worn on a user's wrist to start a virtual try-on of a three-dimensional (3D) watch that is displayed on the user's wrist at the location of the band. In case of virtual footwear, AI's deep learning technologies may be utilized to estimate the pose of a user's foot based on the estimated position of selected 3D keypoints. Thereafter, a parametric 3D model of the user's foot may be created, positioned, and scaled according to the geometric properties of the user's foot.” Kanami teaches that the model is a human mesh. Kanani [0093]: “However, when the IUV maps for garments are inverted to the human mesh, the result may be extremely noisy and disparate, and, hence, non-utilizable for retail applications. To ameliorate this problem, in particular embodiments, the parameters collected from the query translator 409 and query assimilator 404 are fed into the hourglass network of the shape regressor 710. These parameters take into account the real-time user interactions with a virtual garment. The hourglass network may map the location and temporal sequences of human shape and pose to generate three key regressors: (i) the virtual apparel's cloth position in 3D co-ordinates (x,y,z); (ii) the texture and the material of the virtual apparel—for example, a drape shows a hanging effect on a dress and a folding effect on a t-shirt; and (iii) the style (or shape) and size of the virtual apparel for which the convex hull parameters may be estimated to the radius and recalibrated (by the shape regressor 710 and VAE GAN 714) along localized joints of the human body.”).
As per claim 6, Kanani anticipates the claimed:
6. The method of claim 5, wherein the parametric model is a skinned multi-person linear (SMPL) model. (Kanani [0071]: “Variations of Skinned Multi-Person Linear Model (SMPL) for CNN-based deep learning have been proven in the field to take in 2D imagery and reconstruct 3D meshes, vertices and joints. In particular embodiments, the present disclosure utilizes an iteration of SMPL—referred to as SMPL-X—as the base framework to segment and generate 28 body measurements and 73 key points to reconstruct the joints, rigs, and skin texture of the user's body.”).
As per claim 16, this claim is similar in scope to limitations recited in claims 5-6, and thus is rejected under the same rationale.
As per claim 7, Kanani teaches the claimed:
7. The method of claim 5, wherein the machine-learning model further comprises a generative adversarial network that generates the reproduced image of the subject person wearing the clothing item based on the human mesh model (Kanani [0006]: “In the context of virtual try-on of an item of clothing (such as, for example, a shirt, a pair of pants, a t-shirt, a skirt, a dress, and so on), a two-dimensional (2D) image or representation of the clothing item may be “applied” or transferred onto a 2D photo or silhouette of the user. The technologies such as Generative Adversarial Networks (GANs), Human Pose Estimation models, and Human Parsing models may be used for the 2D clothes transferring applications.” This is used to generate a human mesh model to try on clothes. Kanani [0024]: “FIGS. 5A-5C show exemplary screenshots and illustrations depicting various body measurements and subsequent generation of a 3D body mesh and joints of a user for real-time rendering of a virtual apparel as per certain embodiments of the present disclosure.”).
wherein the generative adversarial network comprises: a generator network configured to receive the human mesh model and the clothing item as inputs and create a photorealistic image of the subject person wearing the clothing item;
and (Kanani [0006]: “In the context of virtual try-on of an item of clothing (such as, for example, a shirt, a pair of pants, a t-shirt, a skirt, a dress, and so on), a two-dimensional (2D) image or representation of the clothing item may be “applied” or transferred onto a 2D photo or silhouette of the user. The technologies such as Generative Adversarial Networks (GANs), Human Pose Estimation models, and Human Parsing models may be used for the 2D clothes transferring applications. Generally, the following steps may be performed: (i) Initially, the areas corresponding to the relevant individual body part(s) may be identified in the user's 2D image/photo. For example, legs may be identified for pants, arms and torso may be identified for shirts, and so on. (ii) Then, the position of the identified body parts may be detected. (iii) Based on the detected position of the relevant body part(s), a 2D warped image of a virtual clothing item (which is to be transferred onto the user's image) may be produced. For example, if the user has selected to view a virtual shirt, then the warped image of the shirt may be generated based on the detected position of the relevant body parts—here, the arms and torso of the user. (iv) Finally, the warped image of the virtual clothing item may be applied to the 2D image of the user with minimal artifacts.” The adversarial process is used to generate models of garments on a user. Kanani [0075]: “In particular embodiments of the present disclosure, to reduce computational complexity, the motion discriminators for sequence modeling (of human movements) do not retrain every corpus. Instead, those embodiments use an hourglass network such as, for example, the hourglass network 704 shown in FIG. 7A—to decide on the latent differences between the user's motion with a garment and the garment's own behavior. Any adversarial loss may be back-propagated to Gated Recurrent Units (GRUs) (such as, for example, the GRUs 706 in FIG. 7A) and handled by the adversarial loss function as follows. General details about an hourglass network may be obtained, for example, from A. Newell, K. Yang, and J. Deng, “Stacked Hourglass Networks for Human Pose Estimation,” arXiv® document no. 1603.06937, published on Jul. 26, 2016 and available at https://arxiv.org/pdf/1603.06937.pdf, the disclosure of which is incorporated herein by reference.” The GAN is used to map clothing onto a body mesh. Kanani [0095]: “It is noted that the hourglass network in the shape regressor 710 may deploy a deterministic, query-based autoencoder, which may be configurable or driven by user's actions/queries. Similarly, the discriminator functionality of the measurement server 402 also may be more deterministic. Furthermore, in some embodiments, the GAN 714 may use one or more RCNNs in both generative and discriminative networks. As previously noted, cloth material may be inverse UV mapped onto the user's UV body mesh and reconstructed with a standard Softmax function using an RCNN in the GAN 714 and the hourglass network in the shape regressor 710. In particular embodiments, the combination of the shape regressor 710 and the GAN 714 may modify the effect of how an item of clothing will look on the user based on the outputs from the query translator 409 and/or query assimilator 404. On the other hand, as discussed earlier, the IUV container unit 712 may operate to fit the selected item of clothing to the user's current pose. The rendering unit 716 may operate on the inputs from the units 712, 714 to generate the AR datasets to be sent to the client application module 410, which may contain a 3D/physics visualization engine for accurate, real-time rendering of the virtual apparel on the corresponding body portion of the user. The rendering may be displayed on the display screen of the UE 204 through the UI module 408.” It is intended to generate a realistic image of apparent behavior. Kanani [0057]: “In particular embodiments, these semantics may be used to recreate garment structures using a real-time feedback loop inside a Convolutional Neural Network (CNN) based deep-learning model such as a Recurrent CNN (RCNN) or a deep CNN. This may generate unique outputs for rendering an apparel with AR effects needed to realistically capture the apparel behavior when worn by a human in a real-time try-on environment.”).
a discriminator network configured to receive both real images and generated images and distinguish between the real images and the generated images through an adversarial training process. (Kanani teaches a discriminator network. Kanani [0071]: “Below is a brief outline of technical details pertinent to how sartorial measurements may be generated and of distinctive aspects of the deep learning based model that may be deployed for real-time rendering of virtual garments as per particular embodiments of the present disclosure. The deep learning based garment-rendering model may comprise a number of neural network and Machine Learning (ML) based component modules, as discussed later with reference to FIG. 7A. … The new parameter “Mu” may be introduced with a sequential GAN identifier, where a corpus of human movements are already provisioned to a discriminator (such as the deep CNN 700 in the measurement server 402 in FIG. 7A) to enable the discriminator to estimate the user's look in a “T” side pose and a “Y” pose to understand the anchors of the virtual apparel to be rendered, and the shape transitions and added effects such as texture and material wrinkles to be applied in the rendering for an accurate body fit.”
Kanani teaches a loss function for measuring the difference between real images and the generated model which tries to replicate the image and show the effects of mapping garments from images onto it. Kanani [0082]: “Based on the foregoing, the overall loss function that may be taken into account during training and implementation of the virtual garment rendering methodology as per teachings of the present disclosure may be given as:
L.sub.total=L.sub.iuv+L.sub.ady+L.sub.SMPL-X+L.sub.3D(LiDAR+Kinect corpus)+L.sub.dynamic (4)
In the equation (4) above, the “L.sub.SMPL-X” is given by equation (1), “L.sub.adv” is given by equation (2), and “L.sub.3D” is given by equation (3). As mentioned earlier, the VCI application 100 primarily relates to texture mapping or modeling of cloth behavior with respect to the human user's movements, and not to mesh generation or pose prediction. Thus, although pose and mesh prediction may be implicit in the functionality of the VCI application, 2D mesh or joint map may not need to be generated or accounted for in the loss function. Hence, there is no creation of a 2D mesh or 2D joint map in the present disclosure. Instead, the “L.sub.dynamic” feature is introduced that dynamically updates the earlier-mentioned measurement position “θ” in a user's pose based on user-adjusted joint coordinates (such as, for example, the user's adjustment of shoulder sleeves of a virtual t-shirt that creates an offset for the shoulder anchors and shoulder joints). Furthermore, the “L.sub.inv” loss function minimizes the mapping loss when IUV images of virtual garments are mapped onto a user's 3D UV (ultraviolet) body mesh, as discussed later with reference to the IUV extraction unit 702 and the IUV container unit 712 in the embodiment of FIG. 7A.”).
As per claim 17, this claim is similar in scope to limitations recited in claim 7, and thus is rejected under the same rationale.
As per claim 8, Kanani teaches the claimed:
8. The method of claim 7, wherein the image of the subject person is projected onto the human mesh model to generate the reproduced image and the clothing item is projected onto the reproduced image of the subject person. (Kanani fig. 5c shows construction of a mesh with the features of the human body based on the image. Kanani [0064]: “For ease of explanation, the operation of various modules in FIG. 4 will be discussed with reference to illustrations in FIGS. 5A-5C, which show exemplary screenshots and illustrations depicting various body measurements and subsequent generation of a 3D body mesh and joints of a user for real-time rendering of a virtual apparel as per certain embodiments of the present disclosure.” Kanani fig. 5c shows the mesh conduction of the human. Fig. 8a shows clothing projected onto a model of a person. Kanani [0089]: “FIG. 7A illustrates an exemplary architecture of various software modules in the retailer module 102 of FIG. 4, and FIG. 7B illustrates an exemplary interaction among various software modules of the VCI application 100 as per certain embodiments of the present disclosure. FIG. 7A is a high level illustration of the deep learning architecture of the retailer module 102. It is noted that the interconnections among various modules/units shown in FIG. 7A is exemplary only. In other embodiments, additional or different modules may be deployed and trained, and they may be operatively coupled in a manner different from that illustrated in the embodiment of FIG. 7A. Like any other well-known body mesh generating system, in the retailer module 102, a 3D generative model for learning parameters is fitted on established corpus and discriminative models are trained for regressing the pose and shape of the human body in order to predict the 3D body mesh as well as clothing/apparel mesh. As shown in FIG. 7A, the measurement server 402 may include a deep CNN 700 and an IUV extraction unit 702.” The prediction of the clothing apparel mesh is the projection of clothing onto the mesh to predict what it looks like.).
As per claim 10, Kanani teaches the claimed:
10. The method of claim 5, reproduced image comprises a mannequin image with body proportions based on the human mesh model generated from the image of the subject person. (The examiner is interpreting a “mannequin image” as an image that involves clothing being imposed on a 3D model that resembles a human but is not completely realistic and is artificially generated. Applicant’s specification [0051] describes that a subject representation can show a mannequin image with proportions based on the subject mesh model. Kanani fig. 5C shows mesh construction of a person. Fig. 9A-9C show virtual representations of a person trying on different clothes, and the representation is based on the 3D model. This is the mannequin image.).
As per claim 18, this claim is similar in scope to limitations recited in claim 10, and thus is rejected under the same rationale.
As per claim 11, Kanani teaches the claimed:
11. The method of claim 1, wherein a suggestion that includes an AI size prediction with a recommended best size and a summary of the fit is displayed along with the representation of the fit. (Kanani teaches machine learning that predicts a look of Kanani [0012]: “ As a solution, particular embodiments of the present disclosure relate to a system and method that allows a user with a smartphone or tablet or other wearable device (laptop/desktop) to define retail adjustment operations on a virtual apparel/clothing in real-time using an AR-based visual interface and the user's fingertips. The solution allows the user to interact with the virtual apparel for identifying, defining, and changing the look, fit, and design of the specific apparel on the user's own body in real-time as per individual needs. The real-time interaction is with the same virtual garment, and not a different garment. A user can provide queries based on his/her own body measurements in order to interact with the virtually-generated clothing to fit the clothing to the user's needs in real-time. The system defines operations that utilize a combination of constructs such as user's features (hands, face, legs, and so on), sartorial measurements of the user, intent of the user, gestures of the user, depth of the user's position, pressure values received from a controller operated by the user, and the sensed motion of the user to translate into a set of machine learning (ML) inference models that predict a series of states that visually generate the outcome the user anticipates based on the user's interaction with a virtual piece of clothing” The fit is directly related to the size of the garment. Kanani teaches recommended certain garments of a certain size on the user’s body profile for selected a good clothing choice. Kanani [0127]: “FIGS. 8A-8C show examples of three simple apparel interactions and corresponding translations of these interactions by the query translator 409 for further processing as per particular embodiments of the present disclosure. The pose transitions in FIGS. 8A-8C illustrate how query translator 409 may adapt in real-time to initially “interpret” these transitions. In FIG. 8A, the user 212 is shown to be standing in a T-pose. In FIG. 8B, the user 212 tilts to a wide-pose, whereas in FIG. 8C, the user 212 transitions to a side pose. In FIGS. 8A-8C, a virtual t-shirt and a virtual pair of pants have been selected by the user (or by the system as discussed earlier) for try-on. The illustration 800 in FIG. 8A (which is similar to the illustration 508 in FIG. 5B) indicates that the initial measurement of user's body dimensions is completed (for example, by the measurement server 402) and a medium size t-shirt has been recommended for try-on based on user's body profile and pre-stored dimensions of a medium size t-shirt (for example, as provided by the retailer).” Kanani figs. 8A-8C shows text indicates the nature of the fit and recommends the clothing or not.).
As per claim 12, Kanani teaches the claimed:
12. The method of claim 10, wherein a textual summary of the fit of the clothing item is displayed along with the representation of the fit and the heat map is a grayscale or color scale heat map that visualizes tight, loose, or well-fitting areas for the clothing item on the subject person. (Kim teaches a heatmap of the fit of the clothing and provides a textual summary of the fit as well. The heatmap has different colors for different portions and reflects the looseness or tightness of the clothing. Kim Col. 16 lines 45-68: “(88) The graphical representation 700 may include a heat map 706 that visually summarizes the relationship between the two garments. The heat map 706 may leverage various indicators (e.g., colors, hatching, patterns, etc.) to represent data. A key 708 to the heat map is shown in graphical representation 700. For at least one fit characteristic (e.g., leg fit) the heat map may leverage a first indicator to indicate that the score corresponding to the fit characteristic in a first garment (e.g., Brand A jeans) is greater than a corresponding score in a second garment, the garment being used for comparison (e.g., Brand X Modern Curve jeans). For instance, for the leg fit characteristic, the Brand A jeans are slightly looser than the Brand X Modern Curve jeans as shown by the corresponding hatching on the heat map 706. The heat map 706 also includes a textual summary “slightly looser leg.” For other fit characteristics (e.g., backside fit), the heat map may leverage a second indicator to indicate that the score corresponding to the fit characteristic in a first garment (e.g., Brand A) is less than a corresponding score in the second garment, the garment being used for comparison (e.g., Brand X Modern Curve). For instance, for the backside fit characteristic, the Brand A jeans are tighter than the Brand X Modern Curve as shown by the corresponding hatching on the heat map. The heat map 706 also includes a textual summary “tighter backside.” The graphical representation 700 is one example of a graphical representation that visually summarizes fit characteristics associated with two garments, and any other presentation or configuration may be used.”).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Kanani in view of Kim and further in view of Li (Pub No. US 20190244407 A1).
As per claim 9, Kanani alone does not explicitly teach the claimed limitations.
However, Kanani in combination with Li teaches the claimed:
9. The method of claim 5, wherein the machine-learning model comprises a multi-view convolutional neural network (MVCNN) configured to receive multiple two-dimensional images of the human mesh model from different viewpoints as input and generate a new image of the human mesh model wearing the clothing item from a specified viewpoint. (Li [0037]: “For 3D deep learning, the recent success of deep neural networks for tasks such as classification and regression may be explained in part by the deep neural network's effectiveness in converting data into a high-dimensional feature representation. Because convolutional neural networks may be designed to process images, 3D shapes may often be converted into regular grid representations to enable convolutions. Multiview convolutional neural networks (CNNs) render 3D point clouds or meshes into depth maps and then apply 2D convolutions to them. Volumetric CNNs apply 3D convolutions directly onto the voxels, which may be converted from a 3D mesh or point cloud. One example may present a unified architecture that may directly take point clouds as input. Another example applies 3D CNNs to a variational autoencoder to embed 3D volumetric objects into a compact subspace. The methods may be limited to very low resolutions (e.g., 32×32×32) and focus on man-made shapes, while an example goal may be to encode high-resolution (128×192×128) 3D orientation fields as well as volumes of hairstyles. An example may infer a 3D face shape in the image space via direct volumetric regression from a single-view input while a volumetric representation may be embedded in an example hairstyle representation that may use a 3D direction field in addition to an occupancy grid. Furthermore, the embedding may be learned in a canonical space with fixed head size and position, which may allow for the handling of cropped images, as well as head models in arbitrary positions and orientations.” Li teaches a network used to generate depictions for trying fashion elements including clothing. Li [0070]: “Due to the proposed 3D hair synthesis framework potentially having minimal storage requirements and superior robustness compared to existing methods, an example 3D hair synthesis framework may be particularly well-suited for next generation avatar digitization solutions. While the application of 3D hair digitization is the focus, an example volumetric VAE-based synthesis algorithm may be extended to reconstruct a broad range of nontrivial shapes such as clothing, furry animals, facial hair, or other graphical constructs.” Kanani teaches viewing the garment rendering from a specific perspective. This would be the set viewpoint generated by the network. Kanani [0167]: “It is noted that the sartorial measurements performed by the VCI application 100 are dynamic measurements that can handle random poses, occlusions, projections and the like. Furthermore, the measurements are performed on real humans and in real-time, and not on mannequins or other samples. The VCI application 100 deals with garment level interactions (between the human subject and garments), and with its own methodology based on query assimilator, inference selector, apparatus controller, and other operators. In some embodiments, the VCI application offers measurement customization (from end user perspective) along with garment rendering.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the multi-view convolutional neural network on as taught by Li with the system of Kanani in order to be able to use images from multiple perspectives to reconstruct a model of a human and determine fit for garments, since clothing is a complex shape and is intended to look good from many different perspectives.
Claims 13-14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kanani in view of Kim and further in view of Wiesel (Pub No. US 20190244407 A1).
As per claim 13, Kanani alone does not explicitly teach the claimed limitations.
However, Kanani in combination with Wiesel teaches the claimed:
13. The method of claim 10, wherein a suggestion for a different size of the clothing item or a different clothing item with a better fit is displayed along with the representation of the fit. (Wiesel [0088]: “The system may process user image(s) of himself or herself (or, of another person, if the user would like to check how a particular clothing article would look on another person), and may optionally extract or determine or estimate, for example, user gender, user height, user dimensions; without necessarily requiring the user to manually input such data; and optionally by utilizing machine learning to increase the type or number of images that the system is able to accurately process. Additionally, analysis of “big data” repositories and data-mining may enable the system to suggest to the user similar items, or items that the system estimates that may be of interest to the user, based on his past history of virtual dressing.” Wiesel [0473]: “Reference is made to FIG. 23, which is a schematic illustration of enhanced search results 2300 that are generated by the system of the present invention. It demonstrates how the system enables to generate a user-tailored or user-personalized catalog of products, each one of them depicted as virtually-dressed on: (a) the actual user herself, based on her uploaded image; or, (b) a particular other person (e.g., a celebrity, a singer, a fashion model, a television figure) that the user selects from a pre-defined set of images of such other persons, or that the user uploads or otherwise provides to the system (e.g., via a link or pointer to an image of that celebrity or model or third party).” The recommendations are based on the image uploaded to be analyzed for the fit of the clothing. It would be obvious to display the recommendation next to the model showing the fit.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the suggestions for other products of different fits as taught by Wiesel with the system of Kanani in order to give suggestions for different garments easily based on the results of the mesh model fit of Kanani.
As per claim 14, Kanani alone does not explicitly teach the claimed limitations.
However, Kanani in combination with Wiesel teaches the claimed:
14. The method of claim 1, wherein the reproduced image is three-dimensional (3D) and configured to allow rotation of the reproduced image. (Wiesel [0460]: “For demonstrative purposes, some portions of the discussion herein may relate to modifying or generating a single two-dimensional image (e.g., of a leather jacket) that is modified based on a user-specific context (e.g., the leather jacket being virtually dressed on an image of user Bob). However, the system of the present invention may similarly generate, modify, augment and/or serve other suitable formats of search results, for example: a set or batch or group of multiple contextually-modified images of the product (e.g., front side of the leather jacket virtually dressed on user Bob; right side of the leather jacket virtually dressed on user Bob; left side of the leather jacket virtually dressed on user Bob); a video clip or animation in which the item is modified based on the context (e.g., a video clip or animation that shows the leather jacket being virtually dressed on user Adam), optionally allowing to view 360 degrees around the product, or to see zoomed-in or zoomed-out features, depth features, shading features, or the like); a three-dimensional model that may be generated, showing the product (e.g., black leather jacket) being virtually dressed on a three-dimensional model of user Bob, enabling the user to rotate or spin the three-dimensional model on his screen;”.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the rotation of the human model with the clothing tried on as taught by Wiesel with the system of Kanani in order to offer the user a full view of the human model being fitted in the clothing.
As per claim 19, this claim is similar in scope to limitations recited in claims 12 and 13, and thus is rejected under the same rationale.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS JOHN FOSTER whose telephone number is (571)272-5053. The examiner can normally be reached Mon, Fri 8:30-6. Tues-Thurs 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS JOHN FOSTER/Examiner, Art Unit 2616
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights