DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-4, and 6-21 are currently pending in the present application, with claims 1, 18, and 20 being independent.
Response to Amendments / Arguments
Applicant’s arguments with respect to claim(s) 1-4, and 6-21 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Regarding the remaining arguments: Applicant argues with respect to the amended claim language, which is fully addressed in the prior art rejections set forth below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 10, 13-14, 18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Majithia et al. "Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems", Arxiv.Org, Cornell University Library, 201 Olin Library Cornell University Ithaca, NY, (30 November 2021), hereinafter referred to as “Majithia”, in view of Chen et al. (US 20180240280 A1), hereinafter referred to as “Chen,” and in further view of Kolen et al. (US 20210275925), hereinafter referred to as “Kolen”.
Regarding claim 1, Majithia discloses a method comprising: receiving, by one or more processors, an image depicting a set of fashion items (Pg. 3, Right Column, Section 3; generic e-commerce catalog images having body pose variations…known types of garments (e.g. T-shirts) and perform mapping of high-quality texture from an input catalog image…Fig. 2; Input (T-shirt, Trouser). Fig. 5; (a)T-shirts and (b) Trousers),
identifying a set of three-dimensional (3D) avatar fashion item assets corresponding to the set of fashion items depicted in the image (Pg. 3, Right Column, Section 3; …fixed topology parametric template mesh models for known types of garments (e.g., T-shirts) UV map panels corresponding to the parametric mesh model of the garment…Fig. 2; 3D garment digitization. Pg. 5, Left Column, Section 4.1; We have taken template 3D garment meshes),
replacing textures of the set of 3D avatar fashion item assets with target textures generated using the set of fashion items depicted in the image to generate a set of target avatar fashion item assets (Pg. 3, Right Column, Section 3; perform mapping of high-quality texture from an input catalog image to UV map panels…Pg. 4-5, Section 3.2; module takes predicted landmarks a input and maps the relevant regions of the 2D image onto the UV map of the template garment mesh for texture transfer…TPS enables us to transfer high-frequency texture details and provides a pixel-level accurate mask required for texture inpainting…recover consistent UV maps, we employ automated texture inpainting network MADFNet… Fig. 5; sample images…highlighting diverse textures and poses for (a) T-shirts and (b) Trousers),
and generating an avatar using the set of target avatar fashion item assets (Fig. 2; Draping & Rendering Module drapes the template mesh…on the target 3D human avatar. Pg. 5, Left Column, Section 3.3.; Given the fixed template for the T-shirt and trousers alongside a human avatar, each garment is aligned with the avatar…We individually apply the deformations to each template and perform a collision resolution to visualize both garments together on a human avatar).
Majithia does not disclose the identifying of the set of 3D avatar fashion item assets comprising, overlaying a segmentation of a given fashion item on a candidate 3D avatar fashion item asset; determining a percentage of overlap between the segmentation and the candidate 3D avatar fashion item asset; and selecting the candidate 3D avatar fashion item asset in response to determining that the segmentation overlaps the candidate 3D avatar fashion item asset without extending more than a threshold distance away from pixels of the candidate 3D avatar fashion item asset; processing, using a generative machine learning model, the received image depicting the set of fashion items together with the set of 3D avatar fashion item assets to generate target textures.
In the same art of image-based 3D garment reconstruction and garment-template fitting, Chen discloses the identifying of the set of 3D avatar fashion item assets comprising, overlaying a segmentation of a given fashion item on a candidate 3D avatar fashion item asset (Fig. 4 and Par. 0169-0176; create a 3D garment model from all the texture cut-outs of the garment in different camera views…parametric template model fitting and shape registration framework…Given an initial 3D garment template model…and image silhouettes…we generate a 3D garment model…that matches those image silhouettes in all of the V views);
determining a percentage of overlap between the segmentation and the candidate 3D avatar fashion item asset (Par. 0183-0188; The normalized overlapping area in all views, formulated as…Eq (4)…where G.sub.v is the projection of the garment template in view v, and T.sub.v is the silhouette of the garment texture image in view v. The score in each view will be a number between 0 and 1…);
and selecting the candidate 3D avatar fashion item asset in response to determining that the segmentation overlaps the candidate 3D avatar fashion item asset (Fig. 4, Fig. 11, and Par. 0182-0191; Once we have defined the garment template for a garment type, in the first stage of the template fitting, we change the shape morph parameters m and the pose parameters p of the 3D morphable garment template to achieve an approximate agreement in shape with the garment texture images in all V views…By adjusting morph and pose parameters of the garment template, we obtain a garment model which approximately matches the image silhouettes in all camera views…) without extending more than a threshold distance away from pixels of the candidate 3D avatar fashion item asset (Par. 0188; A large discrepancy in the bounding box of the silhouette will cause penalty. Par. 0197-0199; Data term F.sub.data. This term regulates the distance between the projected silhouette of the 3D garment template and the silhouette of the garment texture image I.sub.ν in each corresponding view ν. It enforces the silhouette consistency in all V camera views as the following equation shows…Eq. (6)…DT.sub.S.sub.ν.sup.2(.Math.) denotes the Euclidean distance transform of the silhouette S.sub.ν of the garment texture image in view ν, which returns the closest Euclidean distance from the point to the silhouette for an arbitrary image position x…Eq. (7)…Examiner's note: determines the degree of overlap between the projected garment template and the garment-image silhouette, penalizes large silhouette discrepancies, and further regulates the closest Euclidean distance between the projected garment-template silhouette vertices and the observed garment silhouette.);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Chen’s silhouette-based 3D garment template matching technique into Majithia’s garment digitization system. Doing so provides an objective mechanism for determining which candidate garment template accurately corresponds to a segmented garment, yielding predictable results in automating overlaid template selection, improving the accuracy and robustness of garment-template identification and alignment before texture and avatar rendering.
Majithia in view of Chen does not disclose processing, using a generative machine learning model, the received image depicting the set of fashion items together with the set of 3D avatar fashion item assets to generate target textures.
In the same art of generating avatar fashion items, Kolen discloses processing, using a generative machine learning model, the received image depicting the set of fashion items together with the set of 3D avatar fashion item assets to generate target textures (Par. 0004-0005; provide at least a portion of the input media to a first machine learning model configured to extract visual information regarding one or more humans depicted in image or video data; alter the 3D mesh data of the human base model based on visual information extracted by the first machine learning model to generate custom 3D model data corresponding to the real person…extract, from the input media, visual information regarding a first item of clothing worn by the real person as depicted in the input media; and generate a virtual clothing item corresponding to the first item of clothing worn by the real person, wherein the virtual clothing item includes a texture generated based on the visual information…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a generative machine learning model for avatar fashion items, as taught by Kolen, into Majithia and Chen's combined virtual garment digitization system. Doing so enables automation of design variations and content generation, and using well-known generative machine-learning models, already widely applied in image processing to synthesize textures and images, to automate texture creation in a virtual apparel system so that large catalogs can be handled without manual authoring yields predictable results in improving realism and personalization of rendered garments.
Regarding claim 3, Majithia in view of Chen in further view of Kolen discloses the method of claim 1, and further discloses processing the image using a segmentation model to detect and segment each fashion item in the set of fashion items depicted in the image (Majithia Fig. 2; Landmark Detection Module which predicts a set of 2D landmarks and associated garment segmentation map. Pg. 3, Right Column, Section 3 and Fig. 3; clothing semantic segmentation maps predicted by JPPNet are used to remove pixels…labeled as background or fashion articles occluding the clothing…).
Majithia, Chen, and Kolen are combined for the reasons set forth above with respect to claim 1.
Regarding claim 10, Majithia in view of Chen in further view of Kolen discloses the method of claim 1, and further discloses wherein the set of fashion items comprises a portion of an upper body fashion item (T-shirts), further comprising:
determining that the image partially depicts a front portion of the upper body fashion item (Majithia Fig. 2; Input --> T-shirt),
identifying an upper body 3D avatar fashion item asset as part of the set of 3D avatar fashion item assets (Majithia Fig. 2; Template Mesh of T-Shirt),
and generating the target textures to include an entire front portion of the upper body 3D avatar fashion item asset corresponding to the partially depicted front portion of the upper body fashion item (Majithia Fig. 2; Texture Mapping Module --> Draping & Rendering) and a back portion of the upper body 3D avatar fashion item asset corresponding to an artificial rendering of a back portion of the upper body fashion item (Majithia Pg. 5, Left Column, Section 3.2; take into account the front panel and copy it to the back or take a uniform gradient patch from the front panel and perform texture copying to the back panel by replicating that patch on the entire back panel…if the back view image is readily available (as part of the catalog), we can generate the back panel by using landmarks predicted on the back view of the T-shirt).
Majithia, Chen, and Kolen are combined for the reasons set forth above with respect to claim 1.
Regarding claim 13, Majithia in view of Chen in further view of Kolen discloses the method of claim 1, and further discloses wherein the avatar is generated to have a full body outfit that covers a top portion of the avatar above a waist of the avatar and a lower portion of the avatar below the waist (Majithia Fig. 1; Textured template meshes of T-shirts & Trousers draped on a human avatar. Fig. 2; target 3D human avatar…results on T-shirts and Trousers…both garments draped together).
Majithia, Chen, and Kolen are combined for the reasons set forth above with respect to claim 1.
Regarding claim 14, Majithia in view of Chen in further view of Kolen discloses the method of claim 1, but Majithia and Chen does not appear to explicitly disclose wherein the image is captured by a camera of a user system.
In the same art of generating avatar fashion items, Kolen discloses wherein the image is captured by a camera of a user system (Par. 0004; wherein the input media comprises at least a video recording of the real person captured by a camera).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a camera of a user system, as taught by Kolen, into Majithia and Chen’s combined virtual garment digitization system. Doing so allows live capture directly by a camera of the user’s device, thereby enabling real-time feedback without relying only on pre-captured catalog imagery, and providing compatibility with AR/VR applications.
Regarding claim 18, Majithia discloses
receiving an image depicting a set of fashion items (Pg. 3, Right Column, Section 3; generic e-commerce catalog images having body pose variations…known types of garments (e.g. T-shirts) and perform mapping of high quality texture from an input catalog image…Fig. 2; Input (T-shirt, Trouser). Fig. 5; (a)T-shirts and (b) Trousers),
identifying a set of three-dimensional (3D) avatar fashion item assets corresponding to the set of fashion items depicted in the image (Pg. 3, Right Column, Section 3; …fixed topology parametric template mesh models for known types of garments (e.g., T-shirts) UV map panels corresponding to the parametric mesh model of the garment…Fig. 2; 3D garment digitization. Pg. 5, Left Column, Section 4.1; We have taken template 3D garment meshes),
replacing textures of the set of 3D avatar fashion item assets with target textures generated using the set of fashion items depicted in the image to generate a set of target avatar fashion item assets (Pg. 3, Right Column, Section 3; perform mapping of high-quality texture from an input catalog image to UV map panels…Pg. 4-5, Section 3.2; module takes predicted landmarks a input and maps the relevant regions of the 2D image onto the UV map of the template garment mesh for texture transfer…TPS enables us to transfer high-frequency texture details and provides a pixel-level accurate mask required for texture inpainting…recover consistent UV maps, we employ automated texture inpainting network MADFNet… Fig. 5; sample images…highlighting diverse textures and poses for (a) T-shirts and (b) Trousers),
and generating an avatar using the set of target avatar fashion item assets (Fig. 2; Draping & Rendering Module drapes the template mesh…on the target 3D human avatar. Pg. 5, Left Column, Section 3.3.; Given the fixed template for the T-shirt and trousers alongside a human avatar, each garment is aligned with the avatar…We individually apply the deformations to each template and perform a collision resolution to visualize both garments together on a human avatar).
Majithia does not disclose the identifying of the set of 3D avatar fashion item assets comprising, overlaying a segmentation of a given fashion item on a candidate 3D avatar fashion item asset; determining a percentage of overlap between the segmentation and the candidate 3D avatar fashion item asset; and selecting the candidate 3D avatar fashion item asset in response to determining that the segmentation overlaps the candidate 3D avatar fashion item asset without extending more than a threshold distance away from pixels of the candidate 3D avatar fashion item asset;
In the same art of image-based 3D garment reconstruction and garment-template fitting, Chen discloses the identifying of the set of 3D avatar fashion item assets comprising, overlaying a segmentation of a given fashion item on a candidate 3D avatar fashion item asset (Fig. 4 and Par. 0169-0176; create a 3D garment model from all the texture cut-outs of the garment in different camera views…parametric template model fitting and shape registration framework…Given an initial 3D garment template model…and image silhouettes…we generate a 3D garment model…that matches those image silhouettes in all of the V views);
determining a percentage of overlap between the segmentation and the candidate 3D avatar fashion item asset (Par. 0183-0188; The normalized overlapping area in all views, formulated as…Eq (4)…where G.sub.v is the projection of the garment template in view v, and T.sub.v is the silhouette of the garment texture image in view v. The score in each view will be a number between 0 and 1…);
and selecting the candidate 3D avatar fashion item asset in response to determining that the segmentation overlaps the candidate 3D avatar fashion item asset (Fig. 4, Fig. 11, and Par. 0182-0191; Once we have defined the garment template for a garment type, in the first stage of the template fitting, we change the shape morph parameters m and the pose parameters p of the 3D morphable garment template to achieve an approximate agreement in shape with the garment texture images in all V views…By adjusting morph and pose parameters of the garment template, we obtain a garment model which approximately matches the image silhouettes in all camera views…) without extending more than a threshold distance away from pixels of the candidate 3D avatar fashion item asset (Par. 0188; A large discrepancy in the bounding box of the silhouette will cause penalty. Par. 0197-0199; Data term F.sub.data. This term regulates the distance between the projected silhouette of the 3D garment template and the silhouette of the garment texture image I.sub.ν in each corresponding view ν. It enforces the silhouette consistency in all V camera views as the following equation shows…Eq. (6)…DT.sub.S.sub.ν.sup.2(.Math.) denotes the Euclidean distance transform of the silhouette S.sub.ν of the garment texture image in view ν, which returns the closest Euclidean distance from the point to the silhouette for an arbitrary image position x…Eq. (7)…Examiner's note: determines the degree of overlap between the projected garment template and the garment-image silhouette, penalizes large silhouette discrepancies, and further regulates the closest Euclidean distance between the projected garment-template silhouette vertices and the observed garment silhouette.);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Chen’s silhouette-based 3D garment template matching technique into Majithia’s garment digitization system. Doing so provides an objective mechanism for determining which candidate garment template accurately corresponds to a segmented garment, yielding predictable results in automating overlaid template selection, improving the accuracy and robustness of garment-template identification and alignment before texture and avatar rendering.
Majithia in view of Chen does not appear to explicitly disclose at least one processor; and at least one memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising.
Kolen discloses a system comprising at least one processor; and at least one memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising. (Fig. 8 and Par. 0113; graphics processor 24…RAM)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the combined method of Majithia and Chen on the system of Kolen comprising at least one processor; and at least one memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations. The motivation lies in the advantage that computer systems with processors and storage media are a standard means of executing image and video processing methods, and would have been an obvious design choice, allowing the method to be automated, executed, and practically deployed in electronic devices.
Majithia in view of Chen does not disclose processing, using a generative machine learning model, the received image depicting the set of fashion items together with the set of 3D avatar fashion item assets to generate target textures.
In the same art of generating avatar fashion items, Kolen discloses processing, using a generative machine learning model, the received image depicting the set of fashion items together with the set of 3D avatar fashion item assets to generate target textures (Par. 0004-0005; provide at least a portion of the input media to a first machine learning model configured to extract visual information regarding one or more humans depicted in image or video data; alter the 3D mesh data of the human base model based on visual information extracted by the first machine learning model to generate custom 3D model data corresponding to the real person…extract, from the input media, visual information regarding a first item of clothing worn by the real person as depicted in the input media; and generate a virtual clothing item corresponding to the first item of clothing worn by the real person, wherein the virtual clothing item includes a texture generated based on the visual information…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a generative machine learning model for avatar fashion items, as taught by Kolen, into Majithia and Chen's combined virtual garment digitization system. Doing so enables automation of design variations and content generation, and using well-known generative machine-learning models, already widely applied in image processing to synthesize textures and images, to automate texture creation in a virtual apparel system so that large catalogs can be handled without manual authoring yields predictable results in improving realism and personalization of rendered garments.
Regarding claim 20, Majithia discloses
receiving an image depicting a set of fashion items (Pg. 3, Right Column, Section 3; generic e-commerce catalog images having body pose variations…known types of garments (e.g. T-shirts) and perform mapping of high-quality texture from an input catalog image…Fig. 2; Input (T-shirt, Trouser). Fig. 5; (a)T-shirts and (b) Trousers),
identifying a set of three-dimensional (3D) avatar fashion item assets corresponding to the set of fashion items depicted in the image (Pg. 3, Right Column, Section 3; …fixed topology parametric template mesh models for known types of garments (e.g., T-shirts) UV map panels corresponding to the parametric mesh model of the garment…Fig. 2; 3D garment digitization. Pg. 5, Left Column, Section 4.1; We have taken template 3D garment meshes),
replacing textures of the set of 3D avatar fashion item assets with target textures generated using the set of fashion items depicted in the image to generate a set of target avatar fashion item assets (Pg. 3, Right Column, Section 3; perform mapping of high-quality texture from an input catalog image to UV map panels…Pg. 4-5, Section 3.2; module takes predicted landmarks a input and maps the relevant regions of the 2D image onto the UV map of the template garment mesh for texture transfer…TPS enables us to transfer high-frequency texture details and provides a pixel-level accurate mask required for texture inpainting…recover consistent UV maps, we employ automated texture inpainting network MADFNet… Fig. 5; sample images…highlighting diverse textures and poses for (a) T-shirts and (b) Trousers),
and generating an avatar using the set of target avatar fashion item assets (Fig. 2; Draping & Rendering Module drapes the template mesh…on the target 3D human avatar. Pg. 5, Left Column, Section 3.3.; Given the fixed template for the T-shirt and trousers alongside a human avatar, each garment is aligned with the avatar…We individually apply the deformations to each template and perform a collision resolution to visualize both garments together on a human avatar).
Majithia does not disclose the identifying of the set of 3D avatar fashion item assets comprising, overlaying a segmentation of a given fashion item on a candidate 3D avatar fashion item asset; determining a percentage of overlap between the segmentation and the candidate 3D avatar fashion item asset; and selecting the candidate 3D avatar fashion item asset in response to determining that the segmentation overlaps the candidate 3D avatar fashion item asset without extending more than a threshold distance away from pixels of the candidate 3D avatar fashion item asset;
In the same art of image-based 3D garment reconstruction and garment-template fitting, Chen discloses the identifying of the set of 3D avatar fashion item assets comprising, overlaying a segmentation of a given fashion item on a candidate 3D avatar fashion item asset (Fig. 4 and Par. 0169-0176; create a 3D garment model from all the texture cut-outs of the garment in different camera views…parametric template model fitting and shape registration framework…Given an initial 3D garment template model…and image silhouettes…we generate a 3D garment model…that matches those image silhouettes in all of the V views);
determining a percentage of overlap between the segmentation and the candidate 3D avatar fashion item asset (Par. 0183-0188; The normalized overlapping area in all views, formulated as…Eq (4)…where G.sub.v is the projection of the garment template in view v, and T.sub.v is the silhouette of the garment texture image in view v. The score in each view will be a number between 0 and 1…);
and selecting the candidate 3D avatar fashion item asset in response to determining that the segmentation overlaps the candidate 3D avatar fashion item asset (Fig. 4, Fig. 11, and Par. 0182-0191; Once we have defined the garment template for a garment type, in the first stage of the template fitting, we change the shape morph parameters m and the pose parameters p of the 3D morphable garment template to achieve an approximate agreement in shape with the garment texture images in all V views…By adjusting morph and pose parameters of the garment template, we obtain a garment model which approximately matches the image silhouettes in all camera views…) without extending more than a threshold distance away from pixels of the candidate 3D avatar fashion item asset (Par. 0188; A large discrepancy in the bounding box of the silhouette will cause penalty. Par. 0197-0199; Data term F.sub.data. This term regulates the distance between the projected silhouette of the 3D garment template and the silhouette of the garment texture image I.sub.ν in each corresponding view ν. It enforces the silhouette consistency in all V camera views as the following equation shows…Eq. (6)…DT.sub.S.sub.ν.sup.2(.Math.) denotes the Euclidean distance transform of the silhouette S.sub.ν of the garment texture image in view ν, which returns the closest Euclidean distance from the point to the silhouette for an arbitrary image position x…Eq. (7)…Examiner's note: determines the degree of overlap between the projected garment template and the garment-image silhouette, penalizes large silhouette discrepancies, and further regulates the closest Euclidean distance between the projected garment-template silhouette vertices and the observed garment silhouette.);
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Chen’s silhouette-based 3D garment template matching technique into Majithia’s garment digitization system. Doing so provides an objective mechanism for determining which candidate garment template accurately corresponds to a segmented garment, yielding predictable results in automating overlaid template selection, improving the accuracy and robustness of garment-template identification and alignment before texture and avatar rendering.
Majithia in view of Chen does not appear to explicitly disclose a non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising
Kolen discloses a non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising (Fig. 8 and Par. 0114; storage element 40…removable storage media 44)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the combined method of Majithia and Chen on the system of Kolen comprising at least one processor; and at least one memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations. The motivation lies in the advantage that computer systems with processors and storage media are a standard means of executing image and video processing methods, and would have been an obvious design choice, allowing the method to be automated, executed, and practically deployed in electronic devices.
Majithia in view of Chen does not disclose processing, using a generative machine learning model, the received image depicting the set of fashion items together with the set of 3D avatar fashion item assets to generate target textures.
In the same art of generating avatar fashion items, Kolen discloses processing, using a generative machine learning model, the received image depicting the set of fashion items together with the set of 3D avatar fashion item assets to generate target textures (Par. 0004-0005; provide at least a portion of the input media to a first machine learning model configured to extract visual information regarding one or more humans depicted in image or video data; alter the 3D mesh data of the human base model based on visual information extracted by the first machine learning model to generate custom 3D model data corresponding to the real person…extract, from the input media, visual information regarding a first item of clothing worn by the real person as depicted in the input media; and generate a virtual clothing item corresponding to the first item of clothing worn by the real person, wherein the virtual clothing item includes a texture generated based on the visual information…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate a generative machine learning model for avatar fashion items, as taught by Kolen, into Majithia and Chen's combined virtual garment digitization system. Doing so enables automation of design variations and content generation, and using well-known generative machine-learning models, already widely applied in image processing to synthesize textures and images, to automate texture creation in a virtual apparel system so that large catalogs can be handled without manual authoring yields predictable results in improving realism and personalization of rendered garments.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Majithia, in view of Chen, in view of Kolen, and in further view of Lin et al. (US 20190043269 A1), hereinafter referred to as “Lin”.
Regarding claim 2, Majithia in view of Chen in further view of Kolen discloses the method of claim 1, and further discloses wherein the set of 3D avatar fashion item assets comprises 3D mesh primitive avatar fashion item assets (Majithia Pg. 3, Right Column, Section 3; parametric template mesh models for known types of garments).
Majithia does not appear to explicitly disclose and wherein identifying the set of 3D avatar fashion item assets comprises: determining that one of a front portion or a back portion of a particular 3D avatar fashion item asset matches a segmentation of a fashion item in the set of fashion items; and in response, retrieving both the front portion and the back portion of the particular 3D avatar fashion item asset.
In the same art of 3D garment template matching, Lin discloses wherein identifying the set of 3D avatar fashion item assets comprises: determining that one of a front portion or a back portion of a particular 3D avatar fashion item asset matches a segmentation of a fashion item in the set of fashion items (Par. 0035; To estimate the clothing model, we first compute a semantic parse of the garments in the image to identify and localize depicted clothing items. This semantic segmentation is computed automatically using a data-driven method for clothing recognition…We then use the semantic parsing to extract garment sizing information, such as waist girth, skirt length and so on, which are then used to map the depicted garments onto the existing garment templates and adjust the sewing patterns based on the extracted parameters…Par. 0083; The sizing and style of the garment described by the parameter G obtained from the parsed garment are matched by minimizing the silhouette which is a 2D polygon differences between our recovered garment S ( C , G and the reference garment silhouette Starget…); and in response, retrieving both the front portion and the back portion of the particular 3D avatar fashion item asset (Par. 0047; extracts a visual representation of the clothing the person is wearing, and then visually matches the outfit to a large database of clothing items to compute a clothing parse of the query image…Par. 0052-0056; the last three parameters indicate the total length, back upper, and front upper length. For each basic garment category, we can manually define this set of parameters G. By manipulating the values of the parameters G, garments of different styles and sizes can be modeled…Par. 0063-0064; combines global pre-trained parse models with local models learned from nearest neighbors and transferred parse masks to estimate the types of garments and their locations on the person…Par. 0073;we select garment parameters based on the sizing and detailed information…we adjust the garment matieral and sizing parameters…to obtain the garment that best matches the one shown in the image…Par. 0083; The sizing and style of the garment described by the parameter G obtained from the parsed garment are matched by minimizing the silhouette which is a 2D polygon differences between our recovered garment S ( C , G and the reference garment silhouette Starget…).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Lin’s garment-template database and single-view garment matching technique into the combined garment digitization system of Majithia, Chen, and Kolen. The combined system already identifies segmented garments and corresponding 3D garment assets, while Lin’s technique would predictably allow a garment identified form the visible portion of a single image (e.g., a front portion or texture) to obtain a corresponding garment template, allowing a diverse full-garment reconstruction and improve user experience in virtual try-on sessions.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Majithia in view of Chen, in view of Kolen, and in further view of Alzu’bi et al. "An interactive attribute-preserving fashion recommendation with 3D image-based virtual try-on." International Journal of Multimedia Information Retrieval 12, no. 2 (2023): 24, hereinafter referred to as "Alzu’bi".
Regarding claim 4, Majithia in view of Chen in further view of Kolen discloses the method of claim 3, but does not appear to explicitly disclose searching a database of 3D avatar fashion item assets for 3D avatar fashion item assets that match a type of each fashion item that has been detected and segmented to output the set of 3D avatar fashion item assets.
In the same art of 3D fashion content generation, Alzu’bi discloses searching a database of 3D avatar fashion item assets for 3D avatar fashion item assets that match a type of each fashion item (Pg. 9, Section 3.6.3; Image similarity search aims to find images in a database that are in the same vector space as the query image. Fig. 7; A sample of the fashion retrieval results) that has been detected and segmented to output the set of 3D avatar fashion item assets (Fig. 3; A user-submitted image and its corresponding segments).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Majithia, Chen, and Kolen’s combined virtual garment digitization system to include Alzu’bi’s database search system. Doing so allows automatic retrieval of content/item assets to match the detected and segmented items, thereby reducing manual selection of avatar assets, enabling the system to scale to large online catalog inventories, and improving the efficiency and accuracy of an avatar-based virtual experience.
Claim(s) 6 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Majithia in view of Chen, in view of Kolen, and in further view of Day (US 20250278876).
Regarding claim 6, Majithia in view of Chen in further view of Kolen discloses the method of claim 1, but does not disclose generating a prompt with instructions for the generative machine learning model to process the image depicting the set of fashion items and a mesh associated with the set of 3D avatar fashion item assets and to generate a set of style textures of a front portion and back portion of each fashion item in the set of 3D avatar fashion item assets to match textures of the set of fashion items.
In the same art of visualization of apparel items using a generative model, Day discloses generating a prompt with instructions for the generative machine learning model to process the image depicting the set of fashion items and a mesh associated with the set of 3D avatar fashion item assets (Par. 0065; the controlled libraries 506 may include a material image library 510 that may include images and/or latent representations of particular materials, textures, and/or fashion patterns for rendering, a text description library 520 comprising one or more text descriptions 134… The text description library 520 may store carefully engineered prompts (e.g., developed via “prompt engineering”) to be used within the text description 134. Par. 0089-0090; For example, the text description 134 for a generative image of a new shirt might be: “brick-red shirt, satin, with blue hemmed short sleeves made of cotton,”. This generative image of this new shirt would, for example, would have a high likelihood of including a specific-colored red shirt having a satin sheen with blue hemmed short sleeves made of cotton…The generative model selection routine 306 may be configured to select and/or receive a selection specifying a generative image model 408 to be utilized, and/or selecting a generative image model 408 to be utilized. The selection may also occur automatically depending on the type of draft file 102, the type of text description 134 provided (e.g., detection of the word “photorealistic”, and/or other keywords)) and to generate a set of style textures of a front portion and back portion of each fashion item in the set of 3D avatar fashion item assets to match textures of the set of fashion items (Par. 0204; FIG. 18A also visually illustrates the boundary designation 1802 for the jacket, specifically comprised of the boundary designation 1802A (a right-side of the jacket viewed from the front of the subject), the boundary designation 1802B (a left side of the jacket viewed from the front of the subject), and the boundary designation 1802C (the back of the jacket viewed from the back of the subject).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate prompt generation style textures for front and back portions of fashion items, as taught by Day, into Majithia, Chen, and Kolen’s combined virtual garment digitization system. Input prompts for image generation is a common technique in modern generative-image systems, allowing users or the application to specify desired styles and constraints, and obtain corresponding variations, yielding predictable results in enhancing user experience and reducing trial-and-error manual design.
Additionally, generating style textures of a front portion and back portion of each fashion item enables better exploration of apparel from all angles, providing comprehensive fit assessment (draping, length, style), mimicking a realistic fitting room experience, and prevent blank or mismatched regions when the avatar is rotated in 3D or in an AR environment.
Regarding claim 15, Majithia in view of Chen in further view of Kolen discloses the method of claim 1, but does not disclose generating a prompt with instructions to generate an artificial image depicting one or more artificial fashion items; and processing the prompt, by a generative machine learning model, to generate the artificial image depicting the one or more artificial fashion items, wherein the artificial image is received as the image depicting the set of fashion items.
In the same art of visualization of apparel items using a generative model, Day discloses generating a prompt with instructions to generate an artificial image depicting one or more artificial fashion items; and processing the prompt, by a generative machine learning model, to generate the artificial image depicting the one or more artificial fashion items, wherein the artificial image is received as the image depicting the set of fashion items (Par. 0065; the controlled libraries 506 may include a material image library 510 that may include images and/or latent representations of particular materials, textures, and/or fashion patterns for rendering, a text description library 520 comprising one or more text descriptions 134… The text description library 520 may store carefully engineered prompts (e.g., developed via “prompt engineering”) to be used within the text description 134. Par. 0089-0090; For example, the text description 134 for a generative image of a new shirt might be: “brick-red shirt, satin, with blue hemmed short sleeves made of cotton,”. This generative image of this new shirt would, for example, would have a high likelihood of including a specific-colored red shirt having a satin sheen with blue hemmed short sleeves made of cotton…The generative model selection routine 306 may be configured to select and/or receive a selection specifying a generative image model 408 to be utilized, and/or selecting a generative image model 408 to be utilized. The selection may also occur automatically depending on the type of draft file 102, the type of text description 134 provided (e.g., detection of the word “photorealistic”, and/or other keywords))
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate prompt generation, as taught by Day, into the Majithia, Chen, and Kolen’s combined virtual garment digitization system. Input prompts for image generation is a common technique in modern generative-image systems, allowing users or the application to specify desired styles and constraints, and obtain corresponding variations, yielding predictable results in enhancing user experience and reducing trial-and-error manual design.
Claim(s) 7-9, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Majithia in view of Chen, in view of Kolen, in view of Day (US 20250278876), and in further view of Milman et al. (US 20190340419), hereinafter referred to as “Milman”.
Regarding claim 7, Majithia in view Chen, in view of Kolen and in further view of Day discloses the method of claim 6, and further discloses processing, by a machine learning model, the set of style textures of the front portion and back portion of each fashion item in the set of 3D avatar fashion item assets (Day Fig. 13-14, Fig. 24-25 and Par. 0204; FIG. 18A also visually illustrates the boundary designation 1802 for the jacket, specifically comprised of the boundary designation 1802A (a right-side of the jacket viewed from the front of the subject), the boundary designation 1802B (a left side of the jacket viewed from the front of the subject), and the boundary designation 1802C (the back of the jacket viewed from the back of the subject)
Majithia, Chen, Kolen, and Day are combined for the reasons set forth above with respect to claim 6.
Majithia in view of Kolen and in further view of Day does not disclose to estimate the target textures comprising avatar style textures.
Milman further discloses processing, by a machine learning model, the set of style textures of(Par. 0032; The training image data 122 represents sets of digital images that the avatar generation framework 120 uses to generate and train machine-learning models…training image data 122 includes digital photographs of persons. The digital photographs of this first subset are further paired with parameters (e.g., manually created parameterizations) of digital cartoon images that match the digital photographs).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate avatar style textures, as taught by Milman, into Majithia, Chen, and Kolen’s combined virtual garment digitization system. Doing so allows rendering of stylized versions of the same garments, providing users with a variation of styles and textures. Non-photorealistic visual styles are commonly used in many virtual games and social-media avatars, so providing an avatar style texture yields predictable results in enhancing user experience, especially for users/businesses/applications who prefer a more cartoon or avatar style-based content rendering.
Regarding claim 8, Majithia in view of Chen, in view of Kolen, in view of Day and in further view of Milman discloses the method of claim 7, and further discloses a training fashion item (Majithia Fig. 2).
Majithia, Chen, Kolen, and Day does not disclose wherein the machine learning model is trained by performing training operations comprising: accessing training data comprising a first training image depicting a training fashion item in an individual style texture and a second ground truth image depicting the training fashion item in an avatar style texture; analyzing, using the machine learning model, the first training image to estimate an avatar style texture for the training fashion item; computing a loss based on a deviation between the estimated avatar style texture for the training fashion item and the ground truth image depicting the training fashion item in the avatar style texture; and updating one or more parameters of the machine learning model based on the computed loss.
Milman further discloses wherein the machine learning model is trained by performing training operations comprising:
accessing training data comprising a first training image (Par. 0032; The training image data 122 represents sets of digital images that the avatar generation framework 120 uses to generate and train machine-learning models…training image data 122 includes digital photographs of persons. The digital photographs of this first subset are further paired with parameters (e.g., manually created parameterizations) of digital cartoon images that match the digital photographs)
analyzing, using the machine learning model, the first training image to estimate an avatar style texture (Par. 0050; The avatar generation framework 120 then provides these “real person” feature vectors as input to the parameter network 302, which outputs parameterized “cartoon” feature vectors…),
computing a loss based on a deviation between the estimated avatar style texture(Par. 0054; the image comparison module 308 compares the avatar image 314 and the coarse avatar image 312 for loss—differentiable loss…),
and updating one or more parameters of the machine learning model based on the computed loss (Par. 0083; Weights of the parameter network are adjusted based on the loss… the avatar generation framework 120 adjusts weights associated with hidden layers or latent features of the parameter network 302 based on the loss determined according to the comparison at block 614.).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate Milman’s supervised training framework into Majithia, Chen Kolen, and Day’s combined virtual garment digitization system. Majithia, Chen, Kolen, and Day’s system already generate clothing-based avatar visualization from garment inputs, and Milman teaches a known way to train a model with paired realistic image and avatar-style targets, therefore using a known photo to avatar training technique to train the model with paired realistic image and avatar style targets would have been a predictable way to convert realistic apparel appearance into a stylized avatar appearance. The motivation lies in the advantage of applying a well-known loss-based training framework to the existing apparel-generation pipeline to improve stylization quality and consistency for each garment texture or fashion item.
Regarding claim 9, Majithia in view of Chen, in view of Kolen, in view of Day and in further view of Milman discloses the method of claim 8. Majithia in view of Chen, in view of Kolen and in further view of Day does not disclose wherein the avatar style textures appear flatter than the set of style textures, wherein the avatar style textures have less light than the set of style textures, wherein the avatar style textures have fewer wrinkles than the set of style textures, and wherein the avatar style textures have fewer shadows than the set of style textures.
Milman further discloses wherein the avatar style textures appear flatter than the set of style textures, wherein the avatar style textures have less light than the set of style textures, wherein the avatar style textures have fewer wrinkles than the set of style textures, and wherein the avatar style textures have fewer shadows than the set of style textures (Fig. 4 and Par. 0032; cartoon avatar features…non-photorealistic (relatively cartoony) styles…).
Majithia, Chen, Kolen, Day, and Milman are combined for the reason set forth above with respect to claim 7 and 8.
Regarding claim 21, claim 21 has similar limitations as of claim 9, except it is a CRM claim (Kolen Fig. 8), therefore it is rejected under the same rationale as claim 9.
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Majithia in view of Chen, in view of Kolen, and in further view of Jang et al. (WO 2020055154 A1), hereinafter referred to as “Jang”.
Note: For the attached foreign reference, Jang et al. (WO 2020055154 A1), the English translation document was edited by the examiner to include paragraph numbers and the following cited paragraphs numbers correspond to edited translation document numbering.
Regarding claim 11, Majithia in view of Chen in further view of Kolen discloses the method of claim 10, but does not disclose determining that the set of fashion items excludes a lower body fashion item; in response to determining that the set of fashion items excludes the lower body fashion item, selecting a lower body 3D avatar fashion item asset that matches visual attributes of the upper body fashion item depicted in the image.
In the same art of 3D fashion content generation, Jang discloses determining that the set of fashion items excludes a lower body fashion item; in response to determining that the set of fashion items excludes the lower body fashion item (Fig. 16(a) and Par. 0260; a specific user may send a request command for requesting recommendation of clothing while at least partially wearing the clothing. For example, while a particular user wears a beige viscose tennis skirt (A37)…a request for requesting a recommendation of a blouse matching the tennis skirt (A37) is requested), selecting a lower body 3D avatar fashion item asset that matches visual attributes of the upper body fashion item depicted in the image (Par. 0127-0129; The matching rate determination unit 253 is a matching between various products such as the degree of matching the top and bottom of a particular style, the degree of matching of clothes of a specific color, the degree of matching of accessories…the top matches the bottom, specifically, the shirt or blouse among the tops…the degree of matching may be different depending on the type, color, and material of the top and bottom).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Majithia, Chen, and Kolen’s combined virtual garment digitization system to include Jang’s attribute matching technique. Doing so allows automatic completion of coordinated outfits for an avatar, thereby reducing user effort in choosing matching garments, and providing a more efficient and visually coherent virtual try-on experience.
Regarding claim 12, Majithia in view of Chen, in view of Kolen, and in further view of Jang discloses the method of claim 11, and further discloses generating the target textures to include a front portion of the lower body 3D avatar fashion item asset based on the visual attributes of the upper body fashion item and a back portion of the lower body 3D avatar fashion item asset based on the visual attributes of the upper body fashion item (Majithia Fig. 1; Textured template meshes of T-shirts & Trousers draped on a human avatar. Pg. 5, Left Column, Section 3.2; take into account the front panel and copy it to the back or take a uniform gradient patch from the front panel and perform texture copying to the back panel by replicating that patch on the entire back panel…if the back view image is readily available (as part of the catalog), we can generate the back panel by using landmarks predicted on the back view of the T-shirt).
Majithia, Chen, Kolen, and Jang are combined for the reason set forth above with respect to claim 11.
Claim(s) 16-17, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Majithia in view of Chen, in view of Kolen, and in further view of O’Brien et al. (US 20190130649), hereinafter referred to as “O’Brien”.
Regarding claim 16, Majithia in view of Chen and in further Kolen discloses the method of claim 1, but does not disclose overlaying the avatar on a video depicting a real-world environment
In the same art of avatar content generation, O’Brien discloses overlaying the avatar on a video depicting a real-world environment (Fig. 14A-14G and Par. 0030; garment and body models…overlaid into the real world environment of each user)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate content media overlays, as taught by O’Brien, into Majithia, Chen, and Kolen’s virtual garment digitization system with avatar animation. Combining a virtual try-on avatar with known AR techniques that overlay graphics on live camera video allows the garments on an avatar to be viewed in the user’s real environment in real-time, yielding predictable results in increased immersive experience.
Regarding claim 17, Majithia in view of Chen, in view of Kolen, and in further view of O’Brien discloses the method of claim 16, and further discloses animating the avatar that has been generated using the set of target avatar fashion item assets (Majithia Pg. 5, Left Column, Section 3.3; We apply a motion sequence to animate the human avatar and deform the garment corresponding to a target pose…).
Majithia, Chen, Kolen, and O’Brien are combined for the reason set forth above with respect to claim 16.
Regarding claim 19, claim 19 has similar limitations as of claim 16, except it is a system claim, therefore it is rejected under the same rationale as claim 16.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JENNY NGAN TRAN whose telephone number is (571) 272-6888. The examiner can normally be reached Mon-Thurs 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner' s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JENNY N TRAN/Examiner, Art Unit 2615
/ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615
Prosecution Insights