DETAILED ACTION
This action is in response to communications: Amendment filed July 2, 2026.
Claims 1-20 are pending in this case. Claims 1-5, 7, 8, 11-14, 16, 19, and 20 have been newly amended. No claims have been newly added or cancelled. This action is made FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed July 2, 2026 have been fully considered but they are not persuasive. Applicant amends independent claims 1, 8 and 16 to similarly recite, “… obtain media assets comprising the subject and an accessory associated with the subject from a digital asset library; generate a texture corresponding to the accessory based on the media assets; and generate, by the local device, a virtual representation of the subject and the accessory using the sensor data and the texture.” Applicant argues on pages 7 and 8 of the Amendment filed that the prior art of record previously cited, e.g. KARMI in view of Alkalay et al., fails to teach or suggest, singly or combined the limitations of the claims as now amended.
In reply, in light of the amendments of independent claims 1, 8, and 16, the claims are now rejected in view of newly found reference Kolen et al. (US 2020/0306640). Please see the rejection and notes regarding the claims below.
Applicant further argues on page 8 of the Amendment filed regarding dependent claims 7 and 20 that these claims are allowable for the reasons noted for respective independent claims 1 and 16.
In reply, in light of the amendments of independent claims 1, 8, and 16, the claims are now rejected in view of newly found reference Kolen et al. (US 2020/0306640). Please see the rejection and notes regarding the claims below.
Applicant further argues on pages 8 and 9 of the Amendment filed regarding dependent claims 9 and 17 that these claims are allowable for the reasons noted for respective independent claims 8 and 16.
In reply, in light of the amendments of independent claims 1, 8, and 16, the claims are now rejected in view of newly found reference Kolen et al. (US 2020/0306640). Please see the rejection and notes regarding the claims below.
Allowable Subject Matter
The indicated allowability of claims 2-6, 11-15, and 19 are withdrawn in view of the newly discovered reference(s) to Kolen et al. (US 2020/0306640)(and in light of the amendments of the claims). Rejections based on the newly cited reference(s) follow.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6 and 8-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kolen et al. (US 2020/0306640).
As to claim 1, Kolen et al. disclose a non-transitory computer readable medium (Figure 8, ROM 46, RAM 48, or storage 40 of computing device 10 implemented in at least the player computing system 102 and/or interactive computing system 120 of Figure 1, [0111]) comprising computer readable code (e.g. program code) executable by one or more processors (processing unit 20 and/or graphics processor 24)([0118], [0119] notes program code stored in ROM 46, RAM 48, or storage 40 and executable, e.g. by processing unit 20 and/or graphics processor 24, [0112],[0113])(Figure 2, method 200, further including Figures 4A, method 400, Figure 4B, method 420, and Figure 5, method 500) to: obtain, at a local device (e.g. camera of a mobile phone, standalone digital camera, security or in-home camera and/or other device), sensor data of a subject (e.g. step 202, [0067] notes obtaining input media, e.g. video and/or images, of a real person to be imported into a game, where [0038] notes the obtained media may be captured by a camera, such as from a mobile phone’s camera, a standalone digital camera, a security or in-home camera, and/or other device, [0068] further notes the camera may include a 3D sensor, e.g. a Kinect sensor, thus the obtained media may include 3D sensor information); obtain media assets comprising the subject and an accessory associated with the subject from a digital asset library (e.g. third party media source 103)(step 202, [0067] notes obtaining input media, e.g. video and/or images, of a real person to be imported into a game, where, additionally, the input media may be obtained by accessing a third party media source, e.g. a social media account or social network account of the player, a cloud storage service used by the player to store digital media, home security camera or other video monitoring services, and/or other sources of media, the obtained media may include clothing and accessories that the person is wearing (further described below)); generate a texture corresponding to the accessory based on the media assets (step 208, [0071] notes generating one or more custom virtual clothing items and/or virtual accessories that can be worn by a virtual character, e.g. the custom clothing or accessories may be generated based on an automated analysis of the input data (e.g. obtained media described above) using one or more machine learning models, where Figure 4B, [0086] notes method 420 further elaborates on generating custom textures and/or 3D model data for one or more clothing or accessory items to be worn by a virtual character, e.g. step 430, [0087] notes identifying items of clothing and/or accessories worn by the person in the input media using appearance learning system 136, step 432, [0088] notes retrieving stored 3D mesh and/or texture data for a similar generic clothing item or accessory for each of the identified items using appearance learning system 136, and step 434, [0089] notes customizing the retrieved texture data and/or 3D mesh of the clothing or accessory item(s) based on the corresponding clothing or accessory item(s) identified in the input media); and generate, by the local device, a virtual representation of the subject and the accessory using the sensor data and the texture (e.g. step 206, [0070] notes generating one or more custom behavior models and one or more custom appearance models, where Figure 4A, [0077] notes method 400 further elaborates on generating custom textures and 3D model data for the virtual character that appears visually similar to the real person depicted in the obtained media, see [0078]-[0085], and step 208, [0071] notes generating one or more custom virtual clothing items and/or virtual accessories that can be worn by a virtual character, where Figure 4B, [0086] notes method 420 further elaborates on generating custom textures and/or 3D model data for one or more clothing or accessory items to be worn by a virtual character, where step 438, [0091] notes causing in-game presentation of the custom 3D virtual character (e.g. generated by method 400 as described) wearing one or more of the custom clothing or accessory items (e.g. generated by method 420 as described)).
As noted above, Kolen et al. disclose its player computing system 102 and/or interactive computing system 120 may include one or more of the components of computing device 10, where [0111] notes the computing device may include a game device, a smart phone, a tablet, a personal computer, a laptop, a smart television, a car console display, a server, and the like. Kolen et al. further discloses the sensor data of the subject may be obtained by a camera that may include camera of a mobile phone, standalone digital camera, security or in-home camera, where a mobile phone is well known to encompass a smart phone. Kolen et al. further describes the method as outlined may be performed by the components of the interactive computing system (e.g. computing device) to generate the custom 3D virtual character. Therefore, it may be considered the local device, encompassing the computing device, e.g. as a smart phone (e.g. the mobile phone that obtained the sensor data), may further generate the virtual representation, e.g. custom 3D virtual character as described, yielding predictable results, without changing the scope of the invention.
As to claim 2, Kolen et al. disclose the computer readable code to generate the texture corresponding to the accessory based on the media assets further comprising computer readable code to: extract, from the media assets having the subject, image data comprising a subject accessory (e.g. step 208, further described in method 420 of Figure 4B, e.g. step 430, [0087] notes identifying (e.g. extracting) items of clothing and/or accessories worn by the person in the input media using appearance learning system 136); apply the image data comprising the subject accessory to a network trained to generate the texture corresponding to the subject accessory to obtain a subject accessory component (e.g. step 208, further described in method 420 of Figure 4B, e.g. step 430, [0087] notes appearance learning system 136 identifies items of clothing and/or accessories worn, which may include application of one or more machine learning models (such as convolutional neural networks) or classifiers configured to identify predefined object in image data, step 432, [0088] notes the appearance learning system 136 may then retrieve stored 2D mesh and/or texture data for a similar generic clothing item or accessory for each of the items identified in step 430, and step 434, [0089] notes the appearance learning system 136 may customize the retrieved texture data and/or 3D mesh of the clothing or accessory item(s) based on the corresponding clothing or accessory item(s) identified in step 430 (e.g. subject accessory component), and notes step 434 may further be implemented in a similar manner as visual style extraction of Figure 4A, e.g. where step 404, [0079] notes the appearance learning system 136 may retrieve, access, or load into memory a learned model for visual style extraction, the learned model may include a generative adversarial network (GAN), step 406, [0080, [0081] notes the appearance learning system 136 may provide input images and/or video data of the input media to the retrieved visual style extraction model, e.g. by applying an image-to-image translation using a cGAN, the cGAN may take a generic visual model and generate a full texture and/or other visual style elements that closely resemble the real person depicted in one or more input images and/or video); and generate an accessory library for the subject comprising the subject accessory component (e.g. step 212, [0051] notes data store 145 may be configured to store data such as generic character models as well as custom character models (both appearance and behavior) generated by the appearance learning system 136 and/or behavior learning system 134, [0073] further notes depending on the embodiment, the custom virtual character may be stored in one or more of data store 145, 124, and/or 126).
As to claim 3, Kolen et al. disclose the subject accessory component comprises a subject accessory texture (e.g. as noted in claim 1, the customized texture data and/or 3D mesh of the clothing or accessory item(s) may be considered the “subject accessor component”), and wherein the computer readable code to generate the virtual representation of the subject comprises computer readable code to: apply the subject accessory texture to at least a portion of the virtual representation of the subject (e.g. as noted in claim 1, the custom 3D virtual character presented wearing one or more of the custom clothing or accessory items generated).
As to claim 4, Kolen et al. disclose the virtual representation is provided as one or more alternative virtual representations of the subject ([0067] notes the obtained input media depicting the real person may alternatively or additionally include obtaining input media from a third party media source 103, thus may be considered the alternative may generate a different 3D virtual character than using both sensor data, e.g. from a camera, and data from third party media sources as described in claim 1).
As to claim 5, Kolen et al. disclose the computer readable code to generate the virtual representation of the subject comprises computer readable code to: generate an initial virtual representation using the sensor data (Figure 4A, [0077]-[0085] notes method 400 for generating custom textures and 3D model data for a virtual character that appears visually similar to the real person); and augment the initial virtual representation with the subject accessory component (Figure 4B, [0086]-[0091] notes method 420 for generating custom textures and/or 3D model data for one or more clothing or accessory items to be worn by the virtual character, e.g. generated as a result of method 400, then step 438, [0091] notes causing presentation of the custom 3D virtual character wearing one or more of the custom clothing or accessory items generated, thus may be considered to “augment” the custom virtual character generated as a result of method 400 with the custom clothing or accessory items generated as a result of method 420).
As to claim 6, Kolen et al. disclose the subject accessory component comprise a three-dimensional hair style component ([0084] notes modular add-on 3D data or templates, such as piercings, hairstyles, facial hair and/or others may be used in adjusting the 3D model data to better match the input media).
As to claim 8, Kolen et al. disclose a method comprising the steps as performed by the one or more processors as outlined in claim 1. Please see the rejection and rationale of claim 1.
As to claim 9, Kolen et al. disclose obtaining sensor data of the subject (step 202, [0067] notes obtaining input media, e.g. video and/or images, of a real person to be imported into a game, e.g. from a camera) comprises: determining geometric characteristics of the subject and texture characteristics of the subject based on the sensor data (step 206, [0070] notes generating one or more custom behavior models and one or more custom appearance models, where Figure 4A, [0077] notes method 400 further elaborates on generating custom textures and 3D model data for the virtual character that appears visually similar to the real person depicted in the obtained media, e.g. step 404, [0079] notes retrieving, accessing, or loading into memory a learned model for visual style extraction, step 406, [0080, [0081] notes providing input images and/or video data of the input media as input to the retrieved visual style extraction model, e.g. by applying an image-to image translation using a cGAN, which may take a generic visual model and/or texture, and generate a full texture and/or other visual style elements that closely resemble the real person depicted in one or more input images and/or video, [0084] notes further adjusting model parameters of the 3D model data to better match the real person depicted in one or more input images and/or video, e.g. piercings, hairstyles, facial hair and/or other characteristics).
As to claim 10, Kolen et al. disclose identifying media assets comprising the subject (step 202, [0067] notes obtaining input media, e.g. video and/or images, of a real person to be imported into a game, e.g. from a third party media source 103) comprises: requesting one or more media items from the digital asset library associated with the subject ([0063] notes the third party media sources 103 may include a social media service or social network service, a cloud storage service, home security camera or other video monitoring services, and/or other sources of personal recorded or streaming media associated with the player, the player may have a user account with each such third party media source or service, the credentials of which (such as user name and password) the player may select to provide to a game application 104 and/or the interactive computing system 120 in order for certain images or videos accessible via the given service or source to be accessible by the game application 104 and/or the interactive computing system 120, where communications between a given third party media source 103 and the game application 104 or the interactive computing system 120 may be via an application programming interface (API) provided by the given third party media source, thus may be considered the game application 104 and/or interactive computing system 120 may have to request access to a player’s account to obtain images and/or video, e.g. permission using the player’s user name and password).
Claims 11-15 are similar in scope to claims 2-6, respectively, and are therefore rejected under similar rationale.
As to claim 16, Kolen et al. disclose a system (Figure 8, computing device 10, which may be implemented in at least the interactive computing system 120 of Figure 1, [0111]) comprising: one or more processors (processing unit 20 and/or graphics processor 24); and one or more computer readable media (ROM 46, RAM 48, or storage 40) comprising computer readable code (e.g. program code) executable by the one or more processors (processing unit 20 and/or graphics processor 24)to perform the steps as performed by the one or more processors ([0118], [0119] notes program code stored in ROM 46, RAM 48, or storage 40 and executable, e.g. by processing unit 20 and/or graphics processor 24, [0112],[0113])(Figure 2, method 200, further including Figures 4A, method 400, Figure 4B, method 420, and Figure 5, method 500) as outlined in claim 1. Please see the rejection and rationale of claim 1.
Claims 17-19 are similar in scope to claims 9-11, respectively, and are rejected under similar rationale.
Claim(s) 7 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over as applied to claims 1 and 16 above, and further in view of Bakunov et al. (US 2024/0296535).
As to claim 7, Kolen et al. disclose the media assets comprising the subject are obtained from the digital asset library ([0067] notes the obtained input media depicting the real person may alternatively or additionally include obtaining input media from a third party media source 103), but do not disclose, but Bakunov et al. disclose in accordance with a determination that the sensor data fails to satisfy one or more quality parameters ([0041], [0042] notes images may be captured by a camera, where [0161] notes performing an automatic quality evaluation to facilitate ranking or comparison of different generators, e.g. the automatic quality evaluation that can be applied to images generated by a single generator, e.g. to compare images generated by the same generative model in order to select one or more images that are preferred in a particular situation (or simply to gain a better understanding as differences in quality between images generated by the same model)).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify Kolen et al.’s method of alternatively obtaining input media, e.g. images and/or video, from a digital asset library, e.g. third party media source, rather than using input media from a camera with Bakunov et al.’s method of comparing images and selecting a better quality image such that the characteristics of the subject may be better visualized to further generate virtual representation more accurately depicting the subject as described.
Claim 20 is similar in scope to claim 7, and is therefore rejected under similar rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Aluru et al. (US 2019/0266778) disclose a system and method of personalizing a virtual look, fit, and animation apparel, accessories, and cosmetics on a virtual representation of a user;
Reville et al. (US 2010/0056273) disclose a system and method of adding items to a virtual avatar associated with a user of a video game, the items may include accessories worn; and
Xu et al. (US 2009/0033667) disclose a system and method of providing an avatar with a rendering of an accessory captured from an image.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACINTA M CRAWFORD whose telephone number is (571)270-1539. The examiner can normally be reached 8:30a.m. to 4:30p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACINTA M CRAWFORD/Primary Examiner, Art Unit 2617
