DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 2, 6, 9, 13, 14, 15, 16, 20, and 21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 4, 6, 7, 10, and 12 of U.S. Patent No. US 12,217,346 B2 (Application No. 18/308,321). Although the claims at issue are not identical, they are not patentably distinct from each other because they both claim the same subject matters and limitations as explained below.
Claim 2 is determined to be obvious in light of claim 1 of US 12,217,346 B2 (Application No. 18/308,321) based on reasons below for having similar limitations.
Instant application claim 2 (19/001,850)
US 12,217,346 B2 (App. 18/308,321) claim 1
A method to operate a three-dimensional (3D) environment, the method comprising: receiving a first input signal from a first user; detecting, based on the first input signal, one or more of a user action and a user characteristic of the first user; generating an animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user and the detected one or more of the user action and the user characteristic; and presenting the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user, and wherein the presenting includes causing the animation of the first avatar representative of the first user in the 3D environment to be displayed to the second user.
A method to operate a three-dimensional (3D) environment, the method comprising: receiving a first input signal from a first user, and using the first input signal for animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user; presenting the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user, wherein a first appearance of the first avatar during the 3D session is based on a setting chosen by the first user for presentation of the first avatar in the 3D environment to the second user, and wherein the first output mode selected by the second user overrides the setting chosen by the first user for presentation of the first avatar, so as to present the first avatar to the second user with a second appearance during the 3D session that is different from the first appearance; in response to a change in a physical location of the first user, changing an input mode of the first user from the first input mode to a second input mode; in response to the change in the physical location of the first user, changing the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; and in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user.
Although the claims at issue are not identical, they are not patentably distinct from each other. For example, claim 1 of US 12,217,346 B2 discloses “receiving a first input signal from a first user, and using the first input signal for animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user” and “presenting the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user”, while instant claim 2 recites the same “first input signal”, “first input mode”, “first avatar”, “3D environment”, “first user”, “second user”, “3D session”, and “respective first output modes selected by the first user and by the second user”, and the “causing the animation of the first avatar . . . to be displayed to the second user” is inherent in claim 1’s presentation of the first avatar with the second appearance to the second user. Instant claim 2’s additional step of “detecting, based on the first input signal, one or more of a user action and a user characteristic of the first user” is necessarily encompassed by claim 1’s “using the first input signal for animation” step, since deriving the animation from the input signal requires detection of the user action or characteristic conveyed by that signal. Since claim 1 of US 12,217,346 B2 contains every limitation of instant claim 2 and adds further limitations, claim 1 anticipates (and at the very least renders obvious) instant claim 2.
Claim 6 is determined to be obvious in light of claims 1 and 6 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 6
US 12,217,346 B2 claim 6 (depends on claim 1)
The method of claim 2, wherein the first input signal is an audio transmission from a first user device of the first user, wherein detecting the user characteristic comprises detecting a tone of the audio transmission, and wherein generating the animation of the first avatar comprises generating the animation of the first avatar to be indicative of the tone of the audio transmission.
The method of claim 1, wherein the animation of the first avatar is further based in part on emotion of the first user as determined by a sensor on a device of the first user.
Although the claims at issue are not identical, they are not patentably distinct from each other. Instant claim 6 narrows claim 2 by reciting that the first input signal is an audio transmission and that the detected “user characteristic” is the “tone of the audio transmission” used to drive the avatar animation. Claim 6 of US 12,217,346 B2 recites that the animation of the first avatar is “further based in part on emotion of the first user as determined by a sensor on a device of the first user.” A microphone capturing the user’s voice is a sensor on the user’s device, and tone of voice is a recognized indicator of emotion; one of ordinary skill would have understood detecting tone of an audio transmission to be an obvious species of the broader “emotion . . . determined by a sensor” taught by claim 6 of the reference patent. Therefore claim 6 of US 12,217,346 B2, taken with parent claim 1, renders instant claim 6 obvious.
Claim 9 is determined to be obvious in light of claim 7 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 9
US 12,217,346 B2 claim 7
An apparatus, comprising: a processor; and a non-transitory computer-readable medium coupled to the processor and having instructions stored thereon that, responsive to execution by the processor, cause the processor to perform or control performance of operations for a three-dimensional (3D) environment, wherein the operations comprise: receiving a first input signal from a first user; detecting, based on the first input signal, one or more of a user action and a user characteristic of the first user; generating an animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user and the detected one or more of the user action and the user characteristic; and presenting the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user, and wherein the presenting includes causing the animation of the first avatar representative of the first user in the 3D environment to be displayed to the second user.
An apparatus, comprising: a processor; and a non-transitory computer-readable medium coupled to the processor and having instructions stored thereon that, responsive to execution by the processor, cause the processor to perform or control performance of operations for a three-dimensional (3D) environment, wherein the operations comprise: receive a first input signal from a first user, and use the first input signal for animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user; present the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user . . . ; in response to a change in a physical location of the first user, change an input mode of the first user from the first input mode to a second input mode; in response to the change in the physical location of the first user, change the first output mode . . . ; and in response to a change in a physical location of the second user, changing the first output mode selected by the second user . . . based on capabilities of a device of the second user.
Although the claims at issue are not identical, they are not patentably distinct from each other. Both claims recite an apparatus comprising a processor and a non-transitory computer-readable medium coupled to the processor with instructions causing operations for a 3D environment, and both recite the same core operations of receiving a first input signal from a first user, using the first input signal for animation of a first avatar (based at least in part on a first input mode) representative of the first user in the 3D environment, and presenting the 3D environment to the first and second user based on respective first output modes selected by the first user and by the second user. Instant claim 9’s explicit recitation of “detecting . . . one or more of a user action and a user characteristic of the first user” is inherently encompassed by the reference claim’s “use the first input signal for animation of a first avatar . . . based at least in part on a first input mode,” because deriving an animation from the input signal necessarily involves detection of the user action or characteristic conveyed by that signal. Since claim 7 of US 12,217,346 B2 contains every limitation of instant claim 9 and adds further limitations, claim 7 anticipates (and at the very least renders obvious) instant claim 9.
Claim 13 is determined to be obvious in light of claims 7 and 12 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 13
US 12,217,346 B2 claim 12 (depends on claim 7)
The apparatus of claim 9, wherein the first input signal is an audio transmission from a first user device of the first user, wherein detecting the user characteristic comprises detecting a tone of the audio transmission, and wherein generating the animation of the first avatar comprises generating the animation of the first avatar to be indicative of the tone of the audio transmission.
The apparatus of claim 7, wherein the animation of the first avatar is further based in part on emotion of the first user as determined by a sensor on a device of the first user.
Although the claims at issue are not identical, they are not patentably distinct from each other. (US 12,217,346 B2, see comments on claim 6.) Instant claim 13 mirrors instant claim 6 in the apparatus category; claim 12 of the reference patent mirrors reference claim 6 in the apparatus category. Tone of an audio transmission captured by a microphone on the user’s device is an obvious species of “emotion of the first user as determined by a sensor on a device of the first user” recited in claim 12 of the reference patent. Therefore claim 12 of US 12,217,346 B2, taken with parent claim 7, renders instant claim 13 obvious.
Claim 14 is determined to be obvious in light of claim 7 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 14
US 12,217,346 B2 claim 7
The apparatus of claim 9, wherein the operations further comprise: in response to a change in a physical location of the first user, changing an input mode of the first user from the first input mode to a second input mode; in response to the change in the physical location of the first user, changing the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; and in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user.
An apparatus . . . wherein the operations comprise: . . . in response to a change in a physical location of the first user, change an input mode of the first user from the first input mode to a second input mode; in response to the change in the physical location of the first user, change the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; and in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user.
Although the claims at issue are not identical, they are not patentably distinct from each other. The three “in response to a change in a physical location” limitations of instant claim 14 are recited verbatim (in the same operative substance, with only the conventional difference between bare-infinitive operational phrasing (“change”) and gerund phrasing (“changing”)) within claim 7 of US 12,217,346 B2 itself. Reference claim 7 already requires all three location-driven mode-change operations as part of the independent claim; instant claim 14 simply restates these operations as a dependent claim from instant claim 9. Therefore claim 7 of US 12,217,346 B2 anticipates instant claim 14.
Claim 15 is determined to be obvious in light of claim 10 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 15
US 12,217,346 B2 claim 10 (depends on claim 7)
The apparatus of claim 14, wherein: the first input mode of the first user is audio input, and the second input mode, which has been changed from the first input mode in response to the change in the physical location of the first user, is text input.
The apparatus of claim 7, wherein: the first input mode of the first user is audio input, and the second input mode, which has been changed from the first input mode in response to the change in the physical location of the first user, is text input.
Although the claims at issue are not identical, they are not patentably distinct from each other. The two limitations of instant claim 15 — “the first input mode of the first user is audio input” and “the second input mode, which has been changed from the first input mode in response to the change in the physical location of the first user, is text input” — are recited verbatim in claim 10 of US 12,217,346 B2. Reference claim 10 depends from reference claim 7, which already establishes the same first input mode/second input mode location-driven change framework that instant claim 14 (parent of instant claim 15) recites. Therefore claim 10 of US 12,217,346 B2 anticipates instant claim 15.
Claim 16 is determined to be obvious in light of claim 7 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 16
US 12,217,346 B2 claim 7
A non-transitory computer-readable medium having instructions stored thereon that, responsive to execution by a processor, cause the processor to perform or control performance of operations for a three-dimensional (3D) environment, wherein the operations comprise: receiving a first input signal from a first user; detecting, based on the first input signal, one or more of a user action and a user characteristic of the first user; generating an animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user and the detected one or more of the user action and the user characteristic; and presenting the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user, and wherein the presenting includes causing the animation of the first avatar representative of the first user in the 3D environment to be displayed to the second user.
An apparatus, comprising: a processor; and a non-transitory computer-readable medium coupled to the processor and having instructions stored thereon that, responsive to execution by the processor, cause the processor to perform or control performance of operations for a three-dimensional (3D) environment, wherein the operations comprise: receive a first input signal from a first user, and use the first input signal for animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user; present the 3D environment to the first user and to a second user during a 3D session . . . based on respective first output modes selected by the first user and by the second user . . . ; [and the location-driven input-mode/output-mode change operations].
Although the claims at issue are not identical, they are not patentably distinct from each other. The non-transitory computer-readable medium with stored instructions recited in instant claim 16 is the same non-transitory computer-readable medium recited within claim 7 of the reference patent (“a non-transitory computer-readable medium coupled to the processor and having instructions stored thereon that, responsive to execution by the processor, cause the processor to perform or control performance of operations for a three-dimensional (3D) environment”). The operations recited in instant claim 16 are identical in substance to those discussed above for instant claim 9. (US 12,217,346 B2, see comments on claim 9.) Therefore claim 7 of US 12,217,346 B2 anticipates (and at the very least renders obvious) instant claim 16.
Claim 20 is determined to be obvious in light of claims 7 and 12 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 20
US 12,217,346 B2 claim 12 (depends on claim 7)
The non-transitory computer-readable medium of claim 16, wherein the first input signal is an audio transmission from a first user device of the first user, wherein detecting the user characteristic comprises detecting a tone of the audio transmission, and wherein generating the animation of the first avatar comprises generating the animation of the first avatar to be indicative of the tone of the audio transmission.
The apparatus of claim 7, wherein the animation of the first avatar is further based in part on emotion of the first user as determined by a sensor on a device of the first user.
Although the claims at issue are not identical, they are not patentably distinct from each other. (US 12,217,346 B2, see comments on claim 6 and claim 13.) Instant claim 20 mirrors instant claim 6 in the computer-readable-medium category. Tone of an audio transmission captured by a microphone on the user’s device is an obvious species of “emotion of the first user as determined by a sensor on a device of the first user” recited in claim 12 of the reference patent. Therefore claim 12 of US 12,217,346 B2, taken with parent claim 7, renders instant claim 20 obvious.
Claim 21 is determined to be obvious in light of claim 7 of US 12,217,346 B2 based on reasons below for having similar limitations.
Instant application claim 21
US 12,217,346 B2 claim 7
The non-transitory computer-readable medium of claim 16, wherein the operations further comprise: in response to a change in a physical location of the first user, changing an input mode of the first user from the first input mode to a second input mode; in response to the change in the physical location of the first user, changing the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; and in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user.
An apparatus . . . wherein the operations comprise: . . . in response to a change in a physical location of the first user, change an input mode of the first user from the first input mode to a second input mode; in response to the change in the physical location of the first user, change the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; and in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user.
Although the claims at issue are not identical, they are not patentably distinct from each other. (US 12,217,346 B2, see comments on claim 14.) The three location-driven mode-change operations recited in instant claim 21 are recited within claim 7 of US 12,217,346 B2 in the same operative substance. Instant claim 21 simply re-recites these operations as a dependent of instant claim 16 (the computer-readable-medium independent claim), whereas reference claim 7 recites them as part of the apparatus independent claim. Therefore claim 7 of US 12,217,346 B2 anticipates instant claim 21.
Note regarding claims 3–5, 7–8, 10–12, and 17–19: These dependent claims recite further narrowing features (muting action and avatar mute state; gag/zipper on the avatar’s face; language detection and audio translation between languages; camera-off detection with default avatar state; noise filtering of user action) that are not recited in, nor obvious over, any claim of US 12,217,346 B2. Accordingly, no nonstatutory double patenting rejection is made over the reference patent for these dependent claims; they are addressed under the separate prior-art rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 3, 5, 6, 9, 10, 12, 13, 16, 17, 19, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lembersky et al. (US 20190095775 A1, hereinafter Lembersky) in view of Chavez (US 9105013 B2).
Regarding Claim 2, Lembersky teaches: a method to operate a three-dimensional (3D) environment, the method comprising: (Lembersky, Paragraph [0006], "illustratively as a three-dimensional (3D) holographic character model"; [0080], "the display may comprise a television (TV), a monitor... or virtual reality (VR) headset"). receiving a first input signal from a first user (Lembersky, Paragraph [0007], "an AI character system receives, in real-time, one or both of an audio user input and a visual user input of a user interacting with the AI character system."); detecting, based on the first input signal, one or more of a user action and a user characteristic of the first user (Lembersky, Paragraph [0022], "Also, through a facial recognition API 110 and/or skeletal tracking, the techniques herein can determine the mood <read on user characteristic> of the user."; [0084], "Once the positions and movements <read on user action> of the user are mapped, the user's facial expression may then be determined"); generating an animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user and the detected one or more of the user action and the user characteristic; (Lembersky, Paragraph [0022], "...triggers visual display “blend shapes” 120 to morph a face of the AI character or avatar <read on first avatar representative of the first user>... into a proper facial expression to convey the appropriate emotional response and mouth movement (lip synching) for the response."; [0038],"...selectable groups that can be selected based on the determined mood of a user (indicated by audio and/or visual input <read on first input mode> of the user)."); presenting the 3D environment to the first user and to a second user during a 3D session, (Lembersky, Paragraph [0100], "multi-user transactions may also be performed, such as where two authenticated users are required for a single transaction... the two users can agree to meet at the AI character location (e.g., a kiosk)"); [[ wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user, ]] and wherein the presenting includes causing the animation of the first avatar representative of the first user in the 3D environment to be displayed to the second user. (Lembersky, Paragraph [0100], "The second user can then be told by the AI character, with confidence, that the second user's account has received the transfer... the transaction is then financially complete... and both users are satisfied.").
But Lembersky does not explicitly disclose but Chavez teaches wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user (Chavez, Column 2, Line 5-10, "One or both of the first and second avatars has multiple appearances and/or behaviors depending on a communication device and/or resource or entity to be serviced viewing the avatar."; Column 16, Line 52-60, "In yet another example, a virtual reality environment appears differently to different participants in the virtual reality environment. For instance, a resource can receive a first virtual reality environment, and a customer... a second virtual reality environment. The first and second virtual reality environments have differing appearances <read on respective first output modes selected>, such as differently appearing avatars, different constituent objects, different backgrounds, and the like.").
Chavez and Lembersky are analogous since both of them are dealing with providing interactive 3D virtual environments populated by avatars representing users. Lembersky provided a way of managing natural verbal and visual interactions based on real-time detection of user characteristics (emotions). Chavez provided a way of tailoring the visual presentation of that environment and those avatars specifically to the device or identity of the person viewing them. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the individualized participant rendering system taught by Chavez into modified invention off Lembersky such that the shared interaction between the two users at the kiosk or in the 3D environment is rendered using output modes (appearances/formats) specifically selected by or for each user. The motivation is to "permit more effective service... and/or provide a more appealing virtual reality environment for each customer" discussed by Chavez in Column 3, Line 15-20.
Regarding Claim 3, the combination of Lembersky and Chavez teaches the invention in claim 2.
The combination further teaches further comprising detecting that the user action comprises a muting action, and wherein generating the animation of the first avatar comprises displaying the first avatar that is indicative of a mute state for the first avatar, (Chavez, Column 11, Line 56-57, "the user starts/stops <read on muting action> talking to another user”; Column 13, Line 24-28, “the work item or members of a set of work items may have an appearance e <read on displaying the first avatar that is indicative of a mute state> change. Such as in color, size, or other physical characteristic or behavior of the work items avatar, to signify the change”).
Chavez and Lembersky are analogous since both of them are dealing with providing interactive 3D virtual environments populated by avatars representing users. Lembersky provided a way of managing natural verbal and visual interactions based on real-time detection of user characteristics. Chavez provided a way of changing avatar appearances and behaviors in response to specific interaction trigger events.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the appearance-change triggering system taught by Chavez into modified invention off Lembersky such that the AI avatar's appearance is updated to a specific visual state when the system detects the user has stopped talking (muting). The motivation is to "signify the change to a servicing resource" or another participant.
Regarding Claim 5, the combination of Lembersky and Chavez teaches the invention in claim 2.
The combination further teaches wherein the first input signal is an audio transmission from a first user device of the first user, (Lembersky, Paragraph [0007], "an AI character system receives, in real-time, one or both of an audio user input and a visual user input"; [0101], "the audio files may capture speech of the user").
and wherein detecting the user characteristic comprises detecting a language of the audio transmission (Lembersky, Paragraph [0072], "The client also chooses an available language package which would translate and understand how to properly use the poses to form words."; [0005], "Natural language processing, in particular, gives machines the ability to read and understand human language, such as for machine translation..."), and wherein generating the animation of the first avatar comprises translating the audio transmission from first language to a second language, (Lembersky, Paragraph [0005], " Natural language processing, in particular, gives machines the ability to read and understand human language, such as for machine translation and question answering"; [0006], "various embodiments are described that... process the text and a response to the text, convert the response back to speech and associated lip-syncing motion <read on generating the animation>... through an AI character."; [0072], "language package which would translate and understand how to properly use the morph targets <read on generating the animation> to form words" [0103], " playing audio for the avatar's speech, etc., where the responses are based on the audio user input and/or the visual user input ") and further comprising providing audio playback of the translated audio transmission to the second user (Lembersky, Paragraph [0100], "multi-user transactions may also be performed... two users can agree to meet at the AI character location (e.g., a kiosk)... a first user can authenticate the transfer... The second user <read on second user> can then be told by the AI character <read on providing audio playback>... that the second user's account has received the transfer").
Regarding Claim 6, the combination of Lembersky and Chavez teaches the invention in claim 2.
The combination further teaches further wherein the first input signal is an audio transmission from a first user device of the first user, (Lembersky, Paragraph [0101], "the audio files may capture speech of the user"), wherein detecting the user characteristic comprises detecting a tone of the audio transmission (Lembersky, Paragraph [0101], "...associate and/or determine an emotion of the user based on the aforementioned audio user input (e.g., the words themselves and/or how they are spoken <read on tone>)"), and wherein generating the animation of the first avatar comprises generating the animation of the first avatar to be indicative of the tone of the audio transmission. (Lembersky, Paragraph [0049], "AI and machine learning processes can then be implemented to morph the face bones into the proper emotional response <read on generating the animation... indicative of the tone>... articulation algorithms may be used to adjust the articulation of the character <read on generating the animation> to match the context and the sentiment."; [0102], "the device can be configured to modify features of an avatar that is to be presented to the user based on the user(s) themselves, such as how to respond, with what emotion to display, with what words to say, with what tone to speak").
Regarding Claim 9, it recites limitations similar in scope to the limitations of claim 1, but in a apparatus. As shown in the rejection, the combination of Lembersky and Chavez disclose the limitations of claims 2. Additionally, Lembersky discloses an apparatus that maps to Paragraph [0081] (Lembersky, Fig. 1, Paragraph [0081], “the simplified device 400 may comprise one or more network interfaces 410 ( e . g ., wired , wireless , etc . ), a user interface 415 , at least one processor 420 , and a memory 440 interconnected by a system bus 450”); Thus, Claim 9 is met by Lembersky according to the mapping presented in the rejection of claims 2, given the method corresponds to the apparatus.
Regarding Claim 10, it recites limitations similar in scope to the limitations of Claim 3 and therefore is rejected under the same rationale.
Regarding Claim 12, it recites limitations similar in scope to the limitations of Claim 5 and therefore is rejected under the same rationale.
Regarding Claim 13, it recites limitations similar in scope to the limitations of Claim 6 and therefore is rejected under the same rationale.
Regarding Claim 16, it recites limitations similar in scope to the limitations of claim 2 and the combination of Lembersky and Chavez teaches all the limitations as of Claim 2. And Lembersky discloses these features can be implemented on a computer-readable storage medium (Lembersky, Paragraph [0083], “It will be apparent to those skilled in the art that other processor and memory types , including various computer-readable media , may be used to store and execute program instructions pertaining to the techniques described herein”).
Regarding Claim 17, it recites limitations similar in scope to the limitations of Claim 3 and therefore is rejected under the same rationale.
Regarding Claim 19, it recites limitations similar in scope to the limitations of Claim 5 and therefore is rejected under the same rationale.
Regarding Claim 20, it recites limitations similar in scope to the limitations of Claim 6 and therefore is rejected under the same rationale.
Claim(s) 4, 11, 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lembersky et al. (US 20190095775 A1, hereinafter Lembersky) in view of Chavez (US 9105013 B2) as applied to Claim 2, 9, 16 above respectively and further in view of Blattner et al. (US 20040179039 A1, hereinafter Blattner).
Regarding Claim 4, the combination of Lembersky and Chavez teaches the invention in claim 2.
The combination does not explicitly disclose but Blattner teaches displaying the first avatar that is indicative of the mute state comprises displaying a gag or a zipper on a face of the first avatar (Blattner, Paragraph, [0169], “or example, each animation identifier may identify one or more morph targets to describe display changes to transform the mesh of an avatar”; [0050], "In one implementation, to show that an avatar is muted, the avatar may appear to be wearing a gag.").
Blattner and Lembersky are analogous since both of them are dealing with Providing interactive real-time avatar-based communications where the avatar's appearance and animations are updated to reflect the status of the user,. Lembersky provides the 3D multi-modal conferencing framework. Blattner provides a specific visual implementation for communicating a detected "mute" state to other participants.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the specific "gag" visual indicator taught by Blattner into modified invention of Lembersky such that when the system detects the user's "muting action", the avatar's face is rendered with a gag. The motivation is to "show that an avatar is muted" and to convey "out-of-band" context information to the recipient regarding the sender's current communication status which is not part of a text message.
Regarding Claim 11, it recites limitations similar in scope to the limitations of Claim 4 and therefore is rejected under the same rationale.
Regarding Claim 18, it recites limitations similar in scope to the limitations of Claim 4 and therefore is rejected under the same rationale.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lembersky et al. (US 20190095775 A1, hereinafter Lembersky) in view of Chavez (US 9105013 B2) as applied to Claim 2, 9, 16 above respectively and further in view of Armando et al. (WO2020247590A1, hereinafter Armando).
Regarding Claim 8, the combination of Lembersky and Chavez teaches the invention in claim 2.
The combination does not explicitly disclose but Armando teaches determining that at least a portion of the user action is noise, wherein generating the animation is based on the user action excluding the portion of the user action that is noise (Armando, Paragraph [0033], "Validated information about a user's past can be used to correct noisy signals being collected in the present."; [0046], "Multi-modal error correction can assist to reduce some of the errors introduced when the sensor output is converted into data signals. Additionally, if data signals have higher accuracy, it may be easier to interpret them").
Armando and Lembersky are analogous since both of them are dealing with processing multi-modal user inputs (audio/video) to drive the actions and responses of virtual characters. Lembersky focuses on detecting user characteristics like mood and emotion to drive animations. Armando provides a specific methodology for multi-modal error correction to improve response accuracy by identifying and correcting "noisy signals." Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the noise filtering and signal correction techniques taught by Armando into modified invention off Lembersky such that environmental or sensor-induced noise is excluded from the data used to animate the avatar. The motivation is to ensure "a more accurate response to an input... [to] increase the user experience in interacting with the virtual character" as discussed by Armando in Paragraph [0004].
Claim(s) 7, 14, 15, 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lembersky et al. (US 20190095775 A1, hereinafter Lembersky) in view of Chavez (US 9105013 B2) as applied to Claim 2, 9, 16 above respectively and further in view of McCulloch (US 20160134840 A1).
Regarding Claim 7, the combination of Lembersky and Chavez teaches the invention in claim 2.
The combination does not explicitly disclose but McCulloch teaches detecting the user action that comprises an action of the first user switching off a camera associated with the first user (McCulloch, Paragraph [0416], "whenever the user walks out of camera view <read on user action... switching off a camera>, the systems automatically displays standby mode."; it is noted when a user "switching off a camera" is functionally equivalent to "walking out of camera view," as both actions result in the intentional termination of the visual input stream used for real-time tracking”), and wherein generating the animation of the first avatar comprises causing the first avatar in the animation to be in a default state that is indicative of the camera being an unavailable input mode for the first user. (McCulloch, Paragraph [0416], "the systems automatically displays standby mode <read on a default state>."; [0413], "standby mode will display the avatar as calm, looking ahead, displaying motions of breathing and blinking <read on indicative of the camera being an unavailable input mode>.")
McCulloch and Lembersky are analogous since both of them are dealing with Providing interactive 3D virtual environments populated by avatars representing users. Lembersky focuses on real-time animation based on detected emotions. McCulloch provides techniques for managing "standby and pause modes" when input quality is insufficient or when the user is no longer tracked by the camera. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the standby state logic taught by McCulloch into modified invention off Lembersky such that the system automatically transitions the avatar to a "calm" default animation when it detects the user has disabled the camera. The motivation is to ensure the avatar maintains a believable appearance (breathing/blinking) even when live tracking is unavailable, rather than simply freezing or disappearing.
Regarding Claim 14, the combination of Lembersky and Chavez teaches the invention in claim 2.
The combination further teaches in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user. (Chavez, Column 2, Line 5-10, "One or both of the first and second avatars has multiple appearances and/or behaviors depending on a communication device and/or resource or entity to be serviced viewing the avatar .")
does not explicitly disclose but McCulloch teaches in response to a change in a physical location of the first user, changing an input mode of the first user from the first input mode to a second input mode (McCulloch, Paragraph [0045], "This group of inventions permit a 3D avatar to be animated in real-time using live or recorded audio input, instead of video <read on changing an input mode>... when the user is in transit "). in response to the change in the physical location of the first user, changing the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; (McCulloch, Paragraph [0142], "an individual working remotely has inconveniences... extra effort to find a quiet, peaceful spot... background noise considerations."; [0144], "Broadband enabled forms of transportation are becoming more prevalent—from the subway, to planes to automobiles <read on changed physical location>... with improved access, users are starting to see out solutions <read on changing to a second output mode usable in the location>").
in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user. (McCulloch, Paragraph [0045], "This is a valuable option... when the user is in transit <read on change in physical location>").
McCulloch and Lembersky are analogous since both of them are dealing with providing real-time avatar-mediated communications. Lembersky provides the core animation and multi-user kiosk-style environment. McCulloch provides techniques for ensuring telepresence continuity when bandwidth or environments change, specifically teaching the transition to audio-driven animation when a user is "in transit." Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the environment-adaptive mode switching taught by McCulloch into modified invention of Lembersky such that the input and output modes for participants automatically adjust as they move between locations (e.g., from home to being "in transit" in a car). The motivation is to address "inconveniences that have not been appropriately addressed" regarding remote work and transit to "provide completely realistic video calls under conditions which can include unpredictably low bandwidth or transient bandwidth" discussed by McCulloch in Paragraph [0002].
Regarding Claim 15, the combination of Lembersky, Chavez and McCulloch teaches the invention in claim 2.
The combination further teaches the first input mode of the first user is audio input, (McCulloch, Paragraph [0419], "TABLE 1 Animation and communication I/O Scenarios... Scenario: Audio Driven... Input: Audio <read on first input mode... is audio input>") and the second input mode, which has been changed from the first input mode in response to the change in the physical location of the first user, is text input. (McCulloch, Paragraph [0419], "TABLE 1 Animation and communication I/O Scenarios... Scenario: Text Driven... Input: Text <read on second input mode... is text input>"; [0007], "an individual working remotely has inconveniences... extra effort to find a quiet, peaceful spot <read on change in physical location>... background noise considerations.")
McCulloch and Lembersky are analogous since both of them are dealing with providing real-time avatar-mediated communications. Lembersky provides the core environment. McCulloch provides Scenario Tables for switching between Video, Audio, and Text-driven inputs to ensure continuity. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the "Text Driven" input mode of McCulloch into modified invention of Lembersky such that the system changes the user's input mode from audio to text when their location changes to an environment where audio is impractical (e.g., a "quiet" spot). The motivation is to address "inconveniences" and "background noise considerations" associated with remote work environments as discussed by McCulloch in Paragraph [0142].
Regarding Claim 21, it recites limitations similar in scope to the limitations of Claim 14 and therefore is rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20140019878 A1 System and method for reflecting player emotional state in an in-game character
US 6215498 B1 Virtual command post
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights