DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on November 27, 2024 and January 14, 2025 were filed after the mailing date of the application on September 6, 2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Drawings
The drawings were received on September 6, 2024. These drawings are accepted.
Claim Objections
Claims 2, 11, and 19 are objected to because of the following informalities:
Claims 2, 11, and 19 similarly recite, “…generate an accessory library for subject comprising the subject accessory component…” but should be “…generate an accessory library for the subject comprising the subject accessory component…”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 8, 10, 16, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over KARMI (US 2023/0410378) in view of Alkalay et al. (US 2024/0144565).
As to claim 1, KARMI discloses a non-transitory computer readable medium comprising computer readable code executable by one or more processors (Figure 1, [0070] notes image capture and processing system 100, where software and/or firmware can include one or more instructions stored on a computer-readable storage medium and executable by one or more processors of the electronic device implementing the image capture and processing system 100) to: obtain, at a local device (Figure 2A, user device 210 of user 215), sensor data of a subject (sensor data 235 via sensor(s) 230)([0073] notes user device 210 may include one or more sensors 230, e.g. one or more cameras, that may capture image data that can include one or more images, videos, portions thereof, or combinations thereof, [0074] notes sensors 230 may capture data measuring and/or tracking information about aspects of the user’s 215 body and/or behaviors by the user 215, [0075] notes sensors 230 may track information about the user and/or the environment, including pose (e.g. position and/or orientation), body of the user 215, and/or behaviors of the user 215, [0076] notes sensors 230 capture sensor data 235); obtain media assets comprising the subject from a digital asset library ([0077] and [0078] notes server(s) 205 includes face detector 240, which receives sensor data 235 detects a face of the person depicted in an image of the sensor data 235, [0079] and [0080] notes server(s) 205 includes a profile identifier 245, which receives at least some of the sensor data 235 and retrieves a profile of the user whose face was detected by the face detector 240 from a data store 290 and/or of the user based on the received sensor data 235, where [0081] notes data store 290 may store profiles for users and other users detected in the sensor data 235, where a profile may include one or more images of the user (e.g. user’s face), one or more identifiers (e.g. name, email address, phone number, mailing address, number, code, etc.), one or more avatars for the user, one or more settings and/or preferences identifying conditions for when different avatars are to be used or not (e.g. conditions to be detected using the condition detector 255)); generate…a virtual representation of the subject using the sensor data ([0085] thru [0088] notes server(s) 205 includes an avatar processor 260 which generates display avatar data 265, which can include a representation of the display avatar that is to be displayed to the user, e.g. the display avatar data 265 generated by modifying an image from the sensor data 235 to use the selected displayed avatar for the user in places of at least a portion of the user in the image, [0089] notes avatar processor 260 outputs the display avatar data 265, e.g. by sending the display avatar data 265 to the user device).
KARMI differs from the invention defined in claim 1 in that KARMI does not disclose “…generate a visual artifact for the subject based on the media assets; and generate, by the local device, a virtual representation of the subject using the sensor data and the visual artifact…”
Alkalay et al. further disclose a non-transitory computer readable medium comprising computer readable code executable by one or more processors (Figure 1, where [0058] notes a computer program product, e.g. a computer program embodied in a machine-readable storage device, for execution by, or to control the operation of, a data apparatus, e.g. a programmable processor, a computer, and/or multiple computers, where the computer program can be written in any form of computer or programming language, including source code, compiled code, interpreted code and/or machine code, [0060] notes processors suitable for the execution of the computer program)(Figures 2 and 3) to: obtain, at a local device (e.g. at one or more remote computing devices 102, e.g. as a mobile device, a desktop computer, and a smart wearable device, where it is considered remote computing device is “local” to the user, but “remote” to the server computing device 106), sensor data of a subject (step 1, [0039] notes input capture module 108 receives a video feed of a user (via a webcam) from the user’s remote computing device 102 and other user-specific input data from the remote computing device, e.g. wearable data, user device data, demographic data, etc., where step 202, [0034] further notes input capture module 108 captures input data from one or more remote computing devices 102, the plurality of remote computing devices are associated with a single user, e.g. a mobile device, a desktop computer, and a smart wearable device); obtain media assets comprising the subject from a digital asset library (step 2, [0039] notes input capture module 108 retrieves other user-specific input data from, e.g. data storage area 112, such as user profile data, historical user interaction data, historical visual artifact data, and the like); generate a visual artifact for the subject based on the media assets (step 3, [0040] notes input capture module 108 provides the collected input data (including the video camera feed) to visual artifact creation module 110 for analysis to generate input data, where step 204, [0037] further notes visual artifact creation module 110 analyzes incoming video feed and determines one or more characteristics of the user (e.g. physical features, mood) and/or the user’s background (e.g. objects, pets, lighting, weather) that may be used as input data and may further analyze audio feed and determine characteristics of the user (e.g. emotion, speaking style, accent, foreign language) that may be used as input data, and step 4, [0040] further notes visual artifact creation module 110 initiates one or more API calls, with the input data as payload, to AI visual artifact generator 114 for creation of corresponding visual artifact(s)); and generate, by the local device (e.g. one or more remote computing devices 102), a virtual representation of the subject using the sensor data and the visual artifact (step 5, [0040] notes visual artifact generator 114 transmits the created visual artifact(s) back to visual artifact creation module 110, and step 6, [0040] further notes visual artifact creation module 110 transmits the created visual artifact(s) to the remote computing device 102, where the artifact(s) are integrated into the communication session, e.g. an image artifact is inserted as a virtual background into the user’s virtual meeting video feed, where Figure 3, step 6 illustrates the visual artifact as the virtual background integrated with the user’s video feed (e.g. sensor data) at the remote computing device 102, thus considered generated at the remote computing device 102, where [0047] further notes visual artifact generation techniques may further include creating AI-driven visual representation/avatar of remote participant, e.g. the user, that incorporates features of the input data, thus the visual artifact including the avatar of the user).
It would have been obvious to one of ordinary skill in the art at the time of the invention to modify KARMI’s system and method of generating an avatar using sensor data, e.g. captured images and/or videos of a user, with Alkalay et al.’s method of generating visual artifacts to provide more advanced customizations of the user using user-specific data, thus enhancing user experience (see at least [0041] of Alkalay et al.).
As to claim 8, KARMI modified with Alkalay et al. disclose a method comprising the steps as performed by the one or more processors as outlined in claim 1. Please see the rejection and rationale of claim 1.
As to claim 10, KARMI modified with Alkalay et al. disclose identifying media assets comprising the subject comprises: requesting one or more media items from the digital asset library associated with the subject (KARMI, [0079] notes querying the data store 290 to retrieve the profile of the user as noted in claim 1, where “querying” is considered “requesting;” modified with Alkalay, step 2, [0039] notes, optionally, input capture module 108 retrieves other user-specific input data from, e.g. data storage area 112, such as user profile data, historical user interaction data, historical visual artifact data, and the like, thus may be considered to “request” when necessary).
As to claim 16, KARMI modified with Alkalay et al. disclose a system (KARMI, Figure 1, further implementing system of Figure 2A; modified with Alkalay, Figure 1, further implementing system of Figure 3) comprising: one or more processors (KARMI, modified with Alkalay, one or processors as noted in claim 1); and one or more computer readable media comprising computer readable code executable by the one or more processors (KARMI, modified with Alkalay, computer readable media as noted in claim 1) to perform the steps as performed by the one or more processors as outlined in claim 1. Please see the rejection and rationale of claim 1.
Claim 18 is similar in scope to claim 10, and is therefore rejected under similar rationale.
Claim(s) 7 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over KARMI (US 2023/0410378) in view of Alkalay et al. (US 2024/0144565) as applied to claims 1 and 16 above, and further in view of Voss et al. (US 2012/0075482)
As to claim 7, KARMI modified with Alkalay et al. disclose the media assets comprising the subject are obtained from a digital asset library (KARMI, profile retrieved from data store 290; modified with Alkalay, user-specific input data from data storage area 112), but do not disclose, but Voss et al. disclose the subject are obtained from a digital asset library in accordance with a determination that the sensor data fails to satisfy one or more quality parameters (Figure 3 performed in accordance with the apparatus of Figure 2, step 310, [0028] notes receiving a source image captured by a device, e.g. a digital camera, cell phone camera, smartphone camera, video camera, tagged with reference information ([0015], [0016]), step 320, [0029] notes based on the reference information, a target image is obtained from a database ([0017] thru [0021]), step 330, [0030] thru [0032] notes the selected target image is blended with the source image to generate a blended image ([0023], [0024]), step 340, [0033] notes the final blended image is provided to the device that captured the original source image, where [0006] thru [0009] notes the method described, e.g. image registration and blending, performed if the quality of the capture device and/or the image is low to improve the overall quality of the image captured by the image capture device, thus considered a target image having a high quality is obtained from a database (obtain from a digital asset library) if the image capture device and/or image (e.g. a determination that sensor data) has low quality (fails to satisfy one or more quality parameters)).
It would have been obvious to one of ordinary skill in the art at the time of the invention to further modify KARMI modified with Alkalay et al.’s method of obtaining the subject from a digital asset library with Voss et al.’s method of image registration and blending based on whether the image capture device and/or image is of low quality to improve the overall quality of the image captured, thus enhancing the system (see [0006] thru [0009] of Voss et al.).
Claim 20 is similar in scope to claim 7, and is therefore rejected under similar rationale.
Claim(s) 9 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over KARMI (US 2023/0410378) in view of Alkalay et al. (US 2024/0144565) as applied to claims 8 and 16 above, and further in view of Weise et al. (US 10,452,896).
As to claim 9, KARMI modified with Alkalay et al. disclose obtaining sensor data of the subject (KARMI, sensor data from one or more cameras; modified with Alkalay, capturing input data via input capture module 108) comprises: determining characteristics of the subject based on the sensor data (KARMI, via face detector 240, where [0077] and [0078] notes face detector receives sensor data 235 and detects, extracts, recognizes, and/or tracks features of portion(s) of the user (e.g. face, facial features, head, body), objects, and/or portions of the environment in order to detect the face of the user), but do not expressly disclose, but Weise et al. disclose determining geometric characteristics of the subject (e.g. sizes and shapes of eyes, nose, mouth, ears, outline of head, etc.) and texture characteristics of the subject (e.g. attributes, such as beard, hair color, hair styles, etc.) based on the sensor data (e.g. based on input image(s) from an image source 160, such as a camera, column 2, lines 30-33)(Figure 3, feature extractor 300 including landmark detector 310, measurement unit 320, image parser 330, and a plurality of attribute detectors 340, where column 4, lines 36-52 notes landmark detector 310 may identify predetermined facial characteristics from content of input image(s) 360, such as the eyes, nose, mouth, ears, outline of head, etc., column 4, lines 53-61 notes measurement unit 320 may measure characteristics of the faces identified in image content, e.g. measure sizes and shapes of the eyes, nose, mouth, eyebrows and head outlines that are identified by the landmark detector 310, column 4, lines 62 thru column 5, lines 11 notes parser 330 may extract patches of image content from the source image 360 corresponding to facial regions on which the respective attribute detectors may work, and column 5, lines 12 thru column 6, lines 16 notes attribute detector 340 may include a variety of detectors that identify characteristics of the input images(s) 360 that may be useful to identify to create an avatar).
It would have been obvious to one of ordinary skill in the art at the time of the invention to further modify KARMI modified with Alkalay et al.’s method of obtaining sensor data and determining characteristics based on the sensor data with Weise et al.’s method of determining additional specific characteristics based on the sensor data to create avatars that better resemble a user’s characteristics and features (see Background of Weise et al.).
Claim 17 is similar in scope to claim 9, and is therefore rejected under similar rationale.
Allowable Subject Matter
Claims 2-6, 11-15, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Regarding dependent claims 2, 11, and 19, the prior art of record fails to teach or suggest, singly or combined, the limitations of the claims as recited. Dependent claims 3-6 are indicated allowable for depending upon indicated allowable claim 2. Dependent claims 12-15 are indicated allowable for depending upon indicated allowable claim 11.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACINTA M CRAWFORD whose telephone number is (571)270-1539. The examiner can normally be reached 8:30a.m. to 4:30p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon can be reached at (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACINTA M CRAWFORD/Primary Examiner, Art Unit 2617