DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page 7, filed 6/8/2026 with respect to 35 USC §112a rejection of claims 1, 11 have been fully considered and are persuasive. Applicant has amended claims to removed previously claimed “separating”. The 35 USC35 USC §112a rejection of claims 1, 11 has been withdrawn.
Applicant’s arguments, see page 7-10, filed 6/8/2026 with respect to 35 USC §103 rejection of claims 1-2,46,8-12,14-16,18-20 have been fully considered but were not persuasive. Applicant has amended claims and argues previously cited references do not teach amended claim language. Applicant argues previously cited Du is directed at changing gaze direction of a person from image data and animate an avatar to change a gaze direction including rotating mesh representation for the avatar. Applicant argues Grant discloses improving eye movement in virtual world including managing eye movement of an avatar to redirect viewing angle including when gaze is moved away significantly system may further adjust orientation of the avatar to keep a degree of consistency between orientation of avatar and direction of the gaze of the avatar. Applicant argues amended claim limitations recite determining a total angular magnitude of the avatar gaze movement and determining a set of component movements that combined to equal total angular magnitude. Applicant argues at best Du describes rotating a mesh of a head to change direction of a gaze and Grant’s eye and heave movement are independently trigged based on thresholds. Applicant argues Grant’s posture selection is silent regarding torso movement being a movement component that combines with eye and head movements to equal a total angular magnitude of an avatar gaze movement. Applicant argues Grant’s posture model is separate from gaze direction. In response, examiner first notes claimed “total angular magnitude” is not defined in specifications, closest found paragraphs are paragraphs [0381-0384] which discloses sum s=(b+t+h+e) is equal to the movement of the gaze of the avatar. Paragraphs [0381-0384] disclose movement of gaze of the avatar but specifications do not define total angular magnitude. Based on these paragraphs and total angular magnitude is defined in amend claims as comprising angular eye, head and torso movement, as cited and argued in previous office actions Grant discloses angular eye, head and torso movement therefore Grant teaches claim limitation of determining a set of component movements that combine to equal the total angular magnitude of the avatar gaze movement. Applicant argument that Grant’s posture selection is silent to regarding to torso movement is not persuasive as Grant clearly discloses posture includes relative positions of various movable body parts including torso. Grant also discloses posture model is computed by tracking relative positions or movements of portions of the user that corresponding to the relative positions or movements of corresponding portions of the avatar. Grant’s posture is combination of movements from tracking relative positions. Therefore Grant disclose all argued claim limitations.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 1,11 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claims recite “determining a set of component movements that combine to equal the total angular magnitude of the avatar gaze movement” and “wherein a combination of the angular eye movement, the angular head movement, and the angular torso movement equals the total angular magnitude of the avatar gaze movement” however specifications do not disclose total angular magnitude Closest found paragraph is paragraph [0322].
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-6,8-12,14-16,18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Du et al (US 20230051409 A1) and Grant (US 20180268589 A1).
Regarding claim 1, Du discloses a method for changing a direction of gaze of an avatar within a virtual environment of a video conference ([0033] virtual conferencing that includes rendering a conference display with animated avatars), the method comprises:
sensing that a certain participant of a video conference changes his direction of gaze from a first virtual item to a second virtual item ([0057] gaze direction (gaze point) information can be determined for a user associated with the avatar to be animated);
wherein the first virtual item and the second virtual item appear in a version of a virtual video conference environment that is displayed on a display of the certain participant ([0063] when a user (Alice) is looking at another person, e.g., another participant's avatar, in the VC display 330, as can be determined from provided eye-tracking gaze point information, Alice's avatar can then be animated to look at that person in the VC display 330. For example, as shown in FIG. 3, gaze information provided for Alice can indicate she is looking, within a threshold distance in the VC display 330, at Dave. That gaze information can then be used (e.g., transmitted to other VC participants) and an avatar for Alice can be animated to look at Dave in the VC display);
wherein the sensing comprises image processing of consecutive images acquired by a camara that captures the participant ([0075] a sensing system (such as including sensors such as those described herein), a control system, at least one processor, and/or an outward facing image sensor, or camera, the smartglasses may include a gaze tracking device including, for example, one or more sensors, to detect and track eye gaze direction and movement. e.g., which information can be provided to virtual conference participants ),
wherein different participants of the video conference receive different versions of the virtual video conference environment ([0065] Each VC device can send current gaze point location, as determined by the eye tracking algorithm, and/or an audio signal including speech of an associated VC participant to the VC server for broadcast to other VC participant(s), The VC devices can then use the provided information (e.g., images, photo, depth map, 3D mesh, gaze point and/or audio signal) to render gaze-aware 3D photorealistic avatars):
determining a total angular magnitude an avatar gaze movement corresponding to the change in direction of the gaze by the certain participant based on a change in the direction of the gaze of the avatar from the first virtual item to second virtual item ([0005] he indication of the current gaze direction can include changes in the current gaze direction, [0057] captured gaze information and/or speech information can, also at block 230, be sent (continuously sent to reflect changes in gaze and audio stream) to other VC participants.)
wherein, once applied, the set of component movements cause the avatar of the certain participant to shift his gaze from the first virtual item to the second virtual item ([0064] animation of gaze redirection for the VC participant Alice);
wherein a combination of the set of component movements equals the virtual movement ([0094] Positioning of the computing device/virtual object by the user when incorporated into the AR space can allow the user to position the computing device so as to view the virtual object in certain manners in the AR space)
Grant discloses determining a set of component movements that combine to equal the total angular magnitude of the avatar gaze movement, the set of component movements comprising ([0084] the posture model (241) can be personalized with parameters (161) identified by the user of the avatar (141) as part of the preferences (147). The personalization parameters (161) adjust one or more parameters of the posture model (241) to provide personal characters of the avatar (141))
an angular eye movement ([0048] the eye movements change the rendering of the eyes of the avatar (141) relative to the face of the avatar, the eye movements also change the viewing angle and/or focus of the avatar)
an angular head movement ([0034] the adjustment can be made via an animation of the head movements of the avatar relative to the body of the avatar to simulate the turning of the head of the avatar). and
an angular torso movement ([0074] The posture may include the relative positions of various movable body parts of the avatar (e.g., limbs, torso) with respect to each other and/or their relative movements with respect to each other.)
wherein a combination of the angular eye movement, the angular head movement, and the angular torso movement equals the total angular magnitude of the avatar gaze movement ([0109] Using the predictive posture model, the posture animation output (245) can be computed without input from a client device tracking relative positions or movements of portions of the user that corresponding to the relative positions or movements of corresponding portions of the avatar having the posture.).
Du and Grant are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify animated avatars of Du to include determining a set of component movements that combine to equal the total angular magnitude of the avatar gaze movement, the set of component movements comprising an angular eye movement , an angular head movement and an angular torso movement, wherein a combination of the angular eye movement, the angular head movement, and the angular torso movement equals the total angular magnitude of the avatar gaze movement as described by Grant.
The motivation for doing so would have been to present eye movement and eye contact in virtual reality using a computer-based eye movement model (Grant, [0025]).
Therefore, it would have been obvious to combine Du and Grant to obtain the invention as specified in claim 1.
Regarding claim 2, Du discloses further comprising generating, by an alpha channel machine learning process, alpha channel information related to the certain participant, and updating the avatar using the alpha channel information ([0067] the selected images are referenced as Img2 and Img3 444, which can be blended using alpha blending with a blending ratio, [0032] machine learning techniques can be used to generate a set of synthesized images, e.g., based on a photograph of a person.)
Regarding claim 4, Du discloses wherein the set of component movements are determined by a machine learning process and wherein the machine learning process is trained by a training process that comprises receiving videos of the certain participant while performing actual movement ([0061] The synthesized images 326 and 328 can be generated using respective ML models (neural networks) that are trained using an image animation model) and
wherein the actual movements comprises at least one out of (a) an actual movement of eyes of the certain participant ([0061] an ML model can be trained, using a first order motion model, to generate synthesized mouth shape images and gaze direction image), and (b) one or more additional actual movements that differ from the gaze movement of eyes of the certain participant.
Regarding claim 5, Du discloses wherein the set of component movements are determined by a machine learning process and wherein the machine learning process is trained by a training process that comprises receiving videos of multiple persons while performing actual movement ([0061] The synthesized images 326 and 328 can be generated using respective ML models (neural networks) that are trained using an image animation model) and
wherein the actual movement comprise (a) an actual movement of eyes of the multiple persons, and (b) one or more additional gaze movement that differ from the actual movement of eyes of the multiple persons ([0040] an ML model for generating gaze direction images can be trained using images of people gazing in different directions, while another ML model for generating different mouth shapes can be trained using videos of people speaking, where synthesized mouth shapes can be based on pitch and amplitude of associated speech).
Regarding claim 6, Du discloses wherein there is a difference between at least one actual movement of the certain participant that led to the change to the direction of gaze and between the set of component movements ([0069] Rotation of a 3D mesh of an associated animated avatar can also follow the current gaze point by rotating the 3D mesh.)
Regarding claim 8, Du discloses wherein the first virtual item is a first participant of the video conference ([0058] Alice during a virtual conference)
Regarding claim 9, Du discloses wherein the second virtual item is a second participant of the video conference ([0063] , gaze information provided for Alice can indicate she is looking, within a threshold distance in the VC display 330, at Dave).
Regarding claim 10, Du discloses wherein the first virtual item differs from any participant of the video conference ([0058] Alice during a virtual conference with Charlie, Bob and Dave)
Regarding claim 11, Du discloses a non-transitory computer readable medium for changing a direction of gaze of an avatar within a virtual environment of a video conference ([0078] the storage device 706 may be or contain a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices), the non-transitory computer readable medium stores instructions for:
sensing that a certain participant of a video conference changes his direction of gaze from a first virtual item to a second virtual item ([0057] gaze direction (gaze point) information can be determined for a user associated with the avatar to be animated)
wherein the first virtual item and the second virtual item appear in a version of a virtual video conference environment that is displayed on a display of the certain participant ([0063] when a user (Alice) is looking at another person, e.g., another participant's avatar, in the VC display 330, as can be determined from provided eye-tracking gaze point information, Alice's avatar can then be animated to look at that person in the VC display 330. For example, as shown in FIG. 3, gaze information provided for Alice can indicate she is looking, within a threshold distance in the VC display 330, at Dave. That gaze information can then be used (e.g., transmitted to other VC participants) and an avatar for Alice can be animated to look at Dave in the VC display);
the sensing comprises image processing of consecutive images acquired by a camara that captures the participant ([0075] a sensing system (such as including sensors such as those described herein), a control system, at least one processor, and/or an outward facing image sensor, or camera, the smartglasses may include a gaze tracking device including, for example, one or more sensors, to detect and track eye gaze direction and movement. e.g., which information can be provided to virtual conference participants ), different participants of the video conference receive different versions of the virtual video conference environment ([0065] Each VC device can send current gaze point location, as determined by the eye tracking algorithm, and/or an audio signal including speech of an associated VC participant to the VC server for broadcast to other VC participant(s), The VC devices can then use the provided information (e.g., images, photo, depth map, 3D mesh, gaze point and/or audio signal) to render gaze-aware 3D photorealistic avatars)
determining a total angular magnitude an avatar gaze movement corresponding to the change in direction of the gaze by the certain participant based on a change in the direction of the gaze of the avatar from the first virtual item to second virtual item ([0005] he indication of the current gaze direction can include changes in the current gaze direction, [0057] captured gaze information and/or speech information can, also at block 230, be sent (continuously sent to reflect changes in gaze and audio stream) to other VC participants.)
wherein, once applied, the set of component movements cause the avatar of the certain participant to shift his gaze from the first virtual item to the second virtual item ([0064] animation of gaze redirection for the VC participant Alice);
wherein a combination of the set of component movements equals the virtual movement ([0094] Positioning of the computing device/virtual object by the user when incorporated into the AR space can allow the user to position the computing device so as to view the virtual object in certain manners in the AR space)
Grant discloses determining a set of component movements that combine to equal the total angular magnitude of the avatar gaze movement, the set of component movements comprising ([0084] the posture model (241) can be personalized with parameters (161) identified by the user of the avatar (141) as part of the preferences (147). The personalization parameters (161) adjust one or more parameters of the posture model (241) to provide personal characters of the avatar (141))
an angular eye movement ([0048] the eye movements change the rendering of the eyes of the avatar (141) relative to the face of the avatar, the eye movements also change the viewing angle and/or focus of the avatar)
an angular head movement ([0034] the adjustment can be made via an animation of the head movements of the avatar relative to the body of the avatar to simulate the turning of the head of the avatar). and
an angular torso movement ([0074] The posture may include the relative positions of various movable body parts of the avatar (e.g., limbs, torso) with respect to each other and/or their relative movements with respect to each other.)
wherein a combination of the angular eye movement, the angular head movement, and the angular torso movement equals the total angular magnitude of the avatar gaze movement ([0109] [0109] Using the predictive posture model, the posture animation output (245) can be computed without input from a client device tracking relative positions or movements of portions of the user that corresponding to the relative positions or movements of corresponding portions of the avatar having the posture.).
Du and Grant are combinable because they are from the same field of invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify animated avatars of Du to include determining a set of component movements that combine to equal the total angular magnitude of the avatar gaze movement, the set of component movements comprising an angular eye movement , an angular head movement and an angular torso movement, wherein a combination of the angular eye movement, the angular head movement, and the angular torso movement equals the total angular magnitude of the avatar gaze movement as described by Grant.
The motivation for doing so would have been to present eye movement and eye contact in virtual reality using a computer-based eye movement model (Grant, [0025]).
Therefore, it would have been obvious to combine Du and Grant to obtain the invention as specified in claim 11.
Regarding claim 12, Du discloses stores instructions for generating, by an alpha channel machine learning process, alpha channel information related to the certain participant, and updating the avatar using the alpha channel information ([0067] the selected images are referenced as Img2 and Img3 444, which can be blended using alpha blending with a blending ratio, [0032] machine learning techniques can be used to generate a set of synthesized images, e.g., based on a photograph of a person.)
Regarding claim 14, Du discloses wherein the set of component movements are determined by a machine learning process and wherein the machine learning process is trained by a training process that comprises receiving videos of the certain participant while performing actual movement ([0061] The synthesized images 326 and 328 can be generated using respective ML models (neural networks) that are trained using an image animation model) and
wherein the actual movement comprise at least one out of (a) an actual movement of eyes of the certain participant ([0061] an ML model can be trained, using a first order motion model, to generate synthesized mouth shape images and gaze direction image), or (b) one or more additional actual movements that differ from the gaze movement of the certain participant.
Regarding claim 15, Du discloses wherein the set of component movements are determined by a machine learning process and wherein the machine learning process is trained by a training process that comprises receiving videos of multiple persons while performing at least one actual movement ([0061] The synthesized images 326 and 328 can be generated using respective ML models (neural networks) that are trained using an image animation model) and
wherein the at least one actual movement comprise (a) an actual movement of eyes of the multiple persons, and (b) one or more additional gaze movement that differ from the actual movement of eyes of the multiple persons ([0040] an ML model for generating gaze direction images can be trained using images of people gazing in different directions, while another ML model for generating different mouth shapes can be trained using videos of people speaking, where synthesized mouth shapes can be based on pitch and amplitude of associated speech).
Regarding claim 16, Du discloses wherein there is a difference between at least one actual movement of the certain participant that led to the change to the direction of gaze of the certain participant and the set of component movements ([0069] Rotation of a 3D mesh of an associated animated avatar can also follow the current gaze point by rotating the 3D mesh.)
Regarding claim 17, Du discloses wherein the determining of the virtual movement of the avatar is made regardless of the at least one actual movement of the certain participant ([0075] a microphone of the smartglasses 696 can be used to capture speech of a virtual conference participant, and an audio signal corresponding with that speech can be provided to other participants for avatar animation using the approaches described herein).
Regarding claim 18, Du discloses wherein the first virtual item is a first participant of the video conference ([0058] Alice during a virtual conference)
Regarding claim 19, Du discloses wherein the second virtual item is a second participant of the video conference ([0063] , gaze information provided for Alice can indicate she is looking, within a threshold distance in the VC display 330, at Dave).
Regarding claim 20, Du discloses wherein the first virtual item differs from any participant of the video conference ([0058] Alice during a virtual conference with Charlie, Bob and Dave)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIVANG I PATEL whose telephone number is (571)272-8964. The examiner can normally be reached on M-F 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached on 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIVANG I PATEL/Primary Examiner, Art Unit 2615
Prosecution Insights