DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is responsive to the amendment received 12/18/2025.
In the response to the Non-Final Office Action 09/18/2025, the applicant indicates that claims 1, 4-17, and 19-26 are pending in current application.
No claims have been amended in current amendment. In summary, claims 1, 4-17, and 19-26 are pending in current application.
Response to Arguments
Applicant's arguments filed 12/18/2025 have been fully considered but they are not persuasive.
Regarding to claim objection of claim 23, the amendment has cured the basis of claim objections. Therefore, the claim objection of claim 23 is hereby withdrawn.
Regarding to 35 U.S.C 112 (d) rejection of claim 19, the applicant did not argue the 35 U.S.C 112 (d) rejection of claim 19. The applicant did not amend claim 19 in current remark. Therefore, the examiner maintains the 35 U.S.C 112 (d) rejection of claim 19.
Regarding to claim 1, the applicant argues that Clanton, Zhang do not teach “wherein during a period of control, a user input device that controls a position and orientation of the first avatar before the period of the control is automatically reconfigured to control a movement of a gaze of the first avatar during the period of the control”. The applicant further argues that None of the remaining references remedy the deficiencies of Clanton and Zhang. The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
The remaining references Inomata and Biswas discloses “wherein during a period of control, a user input device that controls a position and orientation of the first avatar before the period of the control is automatically reconfigured to control a movement of a gaze of the first avatar during the period of the control”.
Inomata and Biswas in combination with Zhang and Clanton discloses “wherein during a period of control, a user input device that controls a position and orientation of the first avatar before the period of the control is automatically reconfigured”. For example, Inomata in paragraph [0089] teaches when the user 5 is a user and moves the HMD 120 worn on his or her head, the virtual camera 14 is also moved in synchronization with the movement. Inomata in paragraph [0092] teaches the processor 210 automatically moves the virtual camera 14 in the virtual space 11 in synchronization with the movement in the real space of the user 5 wearing the HMD 120. Inomata in paragraph [0165] teaches the virtual object control module 1425 controls the movements of the avatar in the virtual space 11. Inomata in paragraph [0183] and Fig. 16 teaches changing the position or orientation of the avatar using detected information from the user; Inomata further teaches the avatar objects 6A to 6C include, as parts capable of moving in association with a motion of a user, a head, a face direction, and eyes.
PNG
media_image1.png
530
668
media_image1.png
Greyscale
. Intomata in paragraph [0186] teaches the HMD sets 110A and 110C are capable of detecting all of the above-mentioned direction data, eye tracking data, face tracking data, and hand tracking data. Inomata in paragraph [0187] teaches the processor 210 changes the head direction of the avatar objects 6A and 6C based on the direction data of the users 5A and 5C; Inomata further teaches the processor 210 changes the line of sight of the avatar objects 6A and 6C or cause the avatar objects 6A and 6C to blink based on the eye tracking data of the users 5A and 5C. Biswas in paragraph [0189] teaches of having a privacy setting for the user to control whether or not they want to share their eye tracking data. For example, Biswas recites: in paragraph [0189] “In some examples a user may define privacy policies which may determine whether or not they wish to share the created gaze tracking information”. The claimed “reconfiguring” occurs when the user in Biswas decides to switch from not sharing to now sharing their eye tracking data on their user input device to control a movement of a gaze of the first avatar during the period of the control.
Inomata and Biswas in combination with Zhang and Clanton further discloses “to control a movement of a gaze of the first avatar during the period of the control”. Inomata in paragraph [0165] teaches the virtual object control module 1425 controls the motion, e.g., movements and state changes, of the target object and the avatar in the virtual space 11. Inomata in paragraph [0183] and Fig. 16 teaches changing the position or orientation of the avatar using detected information from the user; Inomata further teaches the avatar objects 6A to 6C include, as parts capable of moving in association with a motion of a user, a head, a face direction, and eyes;
PNG
media_image1.png
530
668
media_image1.png
Greyscale
; Inomata further more teaches the head is a part that moves in association with a motion of the HMD 120 detected by the HMD sensor 410 or the like; and the eyes are a part that moves in association with the motion and change in line of sight of the eyes of a user detected by the second camera 160 and the eye gaze sensor 140 or the like. Inomata in paragraph [0186] teaches the HMD sets 110A and 110C are capable of detecting all of the above-mentioned direction data, eye tracking data, face tracking data, and hand tracking data. Inomata in paragraph [0187] teaches the processor 210 changes the head direction of the avatar objects 6A and 6C based on the direction data of the users 5A and 5C; Inomata further teaches the processor 210 changes the line of sight of the avatar objects 6A and 6C; Inomata further more teaches the processor 210 causes the avatar objects 6A and 6C to blink based on the eye tracking data of the users 5A and 5C. The claimed feature is taught when the user in Biswas in paragraph [0189] decides to now share eye-tracking information so that this eye tracking information may be displayed as avatar eye controls as taught in Inomata in paragraph [0183].
Applicant’s arguments presented do not appear to directly address the prior art references of Intomata and Biswas. In particular, Applicant’s last response argues both the prior art references of Clanton and Zhang. However, the remarks do not appear to specifically address these cited portions from Intomata and Biswas which are used in combination with both Clanton and Zhang. Clanton and Zhang alone are not relied upon for teaching the claimed feature being argued. Thus, the filed remarks are not persuasive because the portions of Intomata and Biswas being relied upon for teaching the claimed: “wherein during a period of control, a user input device that controls a position and orientation of the first avatar before the period of the control is automatically reconfigured to control a movement of a gaze of the first avatar during the period of the control” have not been directly addressed. The 35 USC 103 rejection in both this office action and the previous office action rely upon Intomata and Biswas for teaching these claimed features.
Applicant’s arguments presented do not appear to directly address the prior art references of Colburn (NPL Doc, “The Role of Eye Gaze in Avatar Mediated Conversational Interfaces”). Colburn teaches the amended claim limitation “wherein the controlling the at least one eye movement comprises moving the gaze of the first avatar to a point of interest for a period of time and turning the gaze back to an initial point” in claim 1.
Claims 5-14 and 19-23 are not allowable due to the similar reasons as discussed above.
Allowable Subject Matter
Claims 15-17 and 26 are allowed.
Claims 24-25 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 19 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 19 depends on claim 14. Both claim 14 and claim 19 include the claim limitation “wherein the controlling the at least one eye movement comprises moving the gaze of the first avatar to a point of interest for a period of time and turning the gaze back to an initial point”. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Clanton et al. (Patent No.: US 7,386,799 B1) in view of Zhang (Pub No. US 2015/0310263 A1) in further view of Inomata et al. (Pub No. US 2018/0300926), Biswas (Pub No. 2016/0187972), and Colburn (NPL Doc, “The Role of Eye Gaze in Avatar Mediated Conversational Interfaces”).
As per claim 1, Clanton teaches the claimed:
1. A method, comprising:
controlling at least one eye movement of a first avatar in a virtual reality world (Clanton in col 28, lines 16-25 lists various scenarios of controlling eye movement of a first avatar, e.g. this portion recites: “(1) “Avatar A momentarily looks at avatar B if the user of avatar B types the name of avatar A in B's Chat Balloon”, (2) “The avatars in a Chat Prop glance at a new avatar joining the Chat Prop and then return to where they were previously looking”, and (3) “When a user types text rapidly (above a tunable words-per-minute threshold), other avatars are more likely to gaze at that user's avatar.”
Clanton in figures 4 and 5A show an example of a given avatar (a first avatar) in a virtual reality world),
wherein the first avatar has at least one eye (Clanton shows this feature in figures 4 and 5A-C where one of the avatars in the conversation (a first avatar) has 2 eyes);
presenting the at least one eye movement of the first avatar to a second avatar in the virtual reality world (Clanton teaches this feature in figures 4 and 5A-C where avatar eye movements are viewable to other avatars within a virtual world. Clanton refers to eye movement in particular in col 26 in lines 25-33),
wherein the second avatar has visibility into the at least one eye of the first avatar (Clanton shows this feature in figures 4 and 5A-C);
and wherein the second avatar is presented with the at least one eye movement of the first avatar relative to a face of the first avatar (Clanton in figure 29 teaches this feature where the second avatar at point 293 is presented with at least one eye movement of the first avatar (avatar on the right) relative to a face of the first avatar. For example, eye movement of the first avatar moves from its vector 291 on the left side in figure 29 from the middle avatar on the left to the bottom avatar on the left at point 293 in the right image).
Clanton alone does not explicitly teach the remaining claim limitations.
However, Zhang in combination with Clanton teaches the claimed:
controlling at least one eye movement of a first avatar in a virtual reality world based on a predetermined eye movement model trained using a machine learning technique (Please see Zhang in figure 1 where user one controls the eye movement of the first avatar 204(1). Also please see [0015]-[0016] “[0015] In this scenario, wearable smart device 102(1) can capture images of user one. These images can be utilized to control a representation of user one. In this case, the representation of user one is manifest as an avatar 204(1) of user one that is presented on display device 202(2) that is proximate to user two … [0016] The avatars 204 can be manifest in any form, such as a cartoon character or a computer generated character that captures the eye movements, facial expressions, and/or mouth movements”.
A predetermined eye movement model trained using a machine learning technique is taught in Zhang in paragraph [0023] “… The training stage can build an avatar model for the user at 606. As mentioned above, the avatar may be photo-realistic or some other form. The user's avatar can be defined via an avatar model 608. The avatar model can include a number of facial parameters or animation parameters, such as corresponding to eye gaze, mouth movements, etc. Further, the process can train for correlation between sensory inputs and the avatar animation parameters at 610 to create a mapping 612.” The eye movement model is predetermined in Zhang because it is determined how it will be trained at step 602 in figure 6 ahead of time before it is used in conversation at step 604.
The claimed feature is taught when this predetermined eye movement model of Zhang is incorporated to control the eye movement animations of the first avatar in the virtual reality world in Clanton).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the machine learning to control eye movement as taught by Zhang with the system of Clanton in order to provide a more realistic and complex eye movement model to control the eye movements of the avatar. The machine learning technique of Zhang may offer improved eye movement controls because they are learned from real world examples recording by the user.
Inomata and Biswas in combination with Zhang and Clanton teaches the claimed:
wherein during a period of control, a user input device that controls a position and orientation of the first avatar before the period of the control is automatically reconfigured (Intomata in [0186] states that they can use a hardware device (user input device) with or without eye tracking capabilities and Inomata in [0183] teaches of changing the position or orientation of the avatar using detected information from the user. For example, Inomata in [0183] recites: “[0183] “The avatar objects 6A to 6C include, as parts capable of moving in association with a motion of a user, a head (face direction”. Biswas in [0189] teaches of having a privacy setting for the user to control whether or not they want to share their eye tracking data. For example, Biswas recites: [0189] “In some examples a user may define privacy policies which may determine whether or not they wish to share the created gaze tracking information” The claimed “reconfiguring” occurs when the user is Biswas decides to switch from not sharing to now sharing their eye tracking data on their user input device to control a movement of a gaze of the first avatar during the period of the control) to control a movement of a gaze of the first avatar during the period of the control (Inomata can control movement of a gaze of the first avatar during the period of control using eye tracking data assuming that Biswas was now reconfigured to share that eye tracking data. Please see Inomata in [0183] “The avatar objects 6A to 6C include, as parts capable of moving in association with a motion of a user, a head (face direction), eyes (e.g., line of sight and blinking), a face (facial expression), and hands. The head is a part that moves in association with a motion of the HMD 120 detected by the HMD sensor 410 or the like. The eyes are a part that moves in association with the motion and change in line of sight of the eyes of a user detected by the second camera 160 and the eye gaze sensor 140 or the like”. The claimed feature is taught when the user in Biswas in [0189] decides to now share eye-tracking information so that this eye tracking information may be displayed as avatar eye controls as taught in Inomata in [0183]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the user input device to control the position and orientation of the first avatar before the period of the control and to control a movement of a gaze of the first avatar during the period of the control as taught by Inomata with the system of Clanton as modified by Zhang. This allows for a wider variety of types of hardware capabilities that may be used to control the avatar in their system (Inomata in [0186]). For example, eye gaze controls are then available to user input devices that have eye tracking capabilities.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have a privacy setting to allow the user input device to be reconfigured either to share or not share the user’s eye tracking data as taught by Biswas with the system of Clanton as modified by Zhang and Inomata. This allows the user to have more control and say over which forms of their personal data is shared over a computer network with other users.
Clanton alone does not explicitly teach wherein the controlling the at least one eye movement comprises moving the gaze of the first avatar to a point of interest for a period of time and turning the gaze back to an initial point.
However, in same field of endeavor, Colburn in combination with Clanton, Zhang, Inomata, and Biswas teaches wherein the controlling the at least one eye movement comprises moving the gaze of the first avatar to a point of interest for a period of time and turning the gaze back to an initial point ((Please see the state transition table shown in figure 4 of Colburn. For example, when the avatar is in the state “Looking at Speaker” the avatar may stay in this state for a period of time, e.g. 4.5 seconds. After this period of time, the avatar returns to the initial point (e.g. where the initial point is either in state “Looking Away” or the state “Looking At Non-speaker” in Colburn).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to move a gaze of the first avatar to a point of interest for a period of time and turning the gaze back to an initial point as taught by Colburn with the system of Clanton as modified by Zhang, Inomata, and Biswas. This type of eye gaze movement is advantageous to add because this is a typically and common eye movement that would occur in a regular conversation between two or more participants. Thus, it adds a natural and common eye movement to the avatar conversation animations.
As per claim 13, Clanton alone does not explicitly teach the claimed limitations.
However, Clanton in combination with Zhang teaches the claimed:
13. The method of claim 1, wherein the method further comprises: training the eye movement model using eye movements captured using eye tracking devices of users using the virtual reality world (Please see Zhang in figure 6 and in paragraph [0023]. For example, please see Zhang in paragraph [0023] where they refer to: “… The training stage can build an avatar model for the user at 606 … The avatar model can include a number of facial parameters or animation parameters, such as corresponding to eye gaze, mouth movements, etc. Further, the process can train for correlation between sensory inputs and the avatar animation parameters at 610 to create a mapping 612. This part of the process flow can be accomplished by simultaneously recording the user with the wearable smart device (e.g., with sensors on the wearable smart device) and an another imaging device such as a red, blue, green plus depth (RGBD) camera.”
Also please see Zhang in [0028] “During training stage 602, the RGBD camera 702 can capture full face images 704. Similarly, the wearable smart device's low-angle camera 104(3) and eye-tracking camera 106(3) capture images 706 and 708, respectively of the user simultaneously to the RGBD camera 702”. Zhang in figure 2 indicates that more than 1 user may use this system to train the eye movement model used by the avatars as well).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to train the eye movement model using eye movements captured using eye tracking devices of users as taught by Zhang with the system of Clanton. The use of this camera allows the eye movement model to gather real world eye movement data of the users wearing head-mounted displays. Thus, this provides a convenience yet realistic data source of data in which to use to train the eye movement model.
As per claim 14, this claim is similar in scope to limitations recited in claim 1, and thus is rejected under the same rationale. The system of Clanton would have to have some type of non-transitory computer storage medium present in order to function and run on a computer as described by the reference (e.g. please see figure 1 and col 3, lines 40-47 of Clanton where they show computer-based device being used to help implement their system).
Claims 5-8 and 19-23 are rejected under 35 U.S.C. 103 as being unpatentable over Clanton in view of Zhang in further view of Inomata, Biswas, and Colburn (NPL Doc, “The Role of Eye Gaze in Avatar Mediated Conversational Interfaces”).
As per claim 5, Clanton teaches the claimed:
5. The method of claim 4, wherein the controlling is performed is computed without using user inputs during the controlling (Clanton teaches that some of the eye movements may be automatically implemented without using user input, e.g. please see Clanton in col 26, lines 50-54 “Automated Gaze--Gaze which happens without direct user-control and is based on social dynamics, events in the world, and conversational dynamics. These behaviors may or may not be distributed across the network, depending on the situation.”)
As per claim 6, Clanton teaches the claimed:
6. The method of claim 5, wherein the point of interest includes an eye of the second avatar (Clanton teaches this feature in figure 21 where the point of interest may be looking directly at a second avatar (and thus looking includes the eyes of the second avatar as well).
As per claim 7, Clanton alone does not explicitly teach the claimed limitations.
However, Colburn in combination with Clanton, Zhang, Inomata, and Biswas teaches the claimed:
7. The method of claim 6, wherein an event corresponds to a real-time communication to or from the second avatar (Please see Colburn in figure 9 and the paragraph below figure 9 where these portions indicate that the event (e.g. an avatar beginning to speak) corresponds to real-time communication between 6 participants in a virtual conference where each participant is equipped with their own microphone. Also see Colburn in the abstract on the front page where it states “In this paper we present behavior models of eye gaze patterns in the context of real-time verbal communication.”
Also see Colburn in caption of figure 2 where it states “The highest level of our eye gaze model is depicted in Figure 1. Figure 1 displays two states depending on whom is talking. Transitions between states are triggered when there is a change in which participant is talking.”
In this passage, the claimed “event” corresponds to “when there is a change in which participant is talking”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for an event to correspond to a real-time communication to or from the second avatar as taught by Colburn with the system of Clanton as modified by Zhang, Inomata, and Biswas. This allows the eye movement animation to be more interactive and responsive during a conversation between multiple, remotely located users.
As per claim 8, Clanton alone does not explicitly teach the claimed limitations.
However, Colburn in combination with Clanton, Zhang, Inomata, and Biswas teaches the claimed:
8. The method of claim 7, wherein the controlling is based on a context determined from the event (Please see Colburn in the flowcharts in figures 1 and 3 show that the animation is computed to move the eyes to the point of interest. The point of interest is based on the context, thus the eye animation is also based on the context as well. In Colburn, the context corresponds to whether there are two people present or whether there are more than 2 people present. The state diagram in figure 1 corresponds to a context where there are two people present and the state diagram in figure 3 corresponds to a context where there are more than two people present. The animation of the eye movement is affected by the context. For example, figure 3 shows that the state diagram has an additional state to animate eye movement towards a non-speaker. This type of animation eye movement towards a non-speaker is not present in figure 1 for conversations where only two people are present).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention for the controlled animation to be computed based on a context determined from the event as taught by Colburn with the system of Clanton as modified by Zhang, Inomata, and Biswas. This allows the animation to be more specifically tailored to reflect the particular situation being displayed by the avatars during the conversation between one or more participants.
As per claim 19, Clanton alone does not explicitly teach the claimed limitations.
However, Colburn in combination with Clanton, Zhang, Inomata, and Biswas teaches the claimed:
19. The non-transitory computer storage medium of claim 14, wherein the controlling the at least one eye movement comprises moving the gaze of the first avatar to a point of interest for a period of time and turning the gaze back to an initial point (Please see the state transition table shown in figure 4 of Colburn. For example, when the avatar is in the state “Looking at Speaker” the avatar may stay in this state for a period of time, e.g. 4.5 seconds. After this period of time, the avatar returns to the initial point (e.g. where the initial point is either in state “Looking Away” or the state “Looking At Non-speaker” in Colburn).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to move a gaze of the first avatar to a point of interest for a period of time and turning the gaze back to an initial point as taught by Colburn with the system of Clanton as modified by Zhang, Inomata, and Biswas. This type of eye gaze movement is advantageous to add because this is a typically and common eye movement that would occur in a regular conversation between two or more participants. Thus, it adds a natural and common eye movement to the avatar conversation animations.
As per claims 20-21, these claims are similar in scope to limitations recited in claims 5-6, respectively, and thus are rejected under the same rationale.
As per claims 22 and 23, Clanton alone does not explicitly teach the claimed limitations.
However, Clanton in combination with Zhang teaches the claimed:
22. The non-transitory computer storage medium of claim 21, wherein the predetermined eye movement model is trained based on inputs received during eye movements of at least one avatar the virtual reality world. And 23. The non-transitory computer storage medium of claim 22, wherein the at least one avatar include the first avatar, or the second avatar (Zhang teaches this feature in figure 6 and in paragraph [0023]. In particular, the Zhang at step 602 teaches of training the predetermined eye movement model based on inputs received during eye movements of the first avatar).
As per claims 22 and 23, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to train the eye movement model based on inputs received during eye movements of the first avatar as taught by Zhang with the system of Clanton. The motivation of claim 1 is incorporated herein.
Claims 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Clanton in view of Zhang in further view of Inomata, Biswas, Colburn, and Shuster et al. (Pub No. US 2008/0309671).
As per claim 9, Clanton alone does not explicitly teach the claimed limitations.
However, Shuster in combination with Clanton, Zhang, Inomata, Biswas, and Colburn teaches the claimed:
9. The method of claim 8, wherein the controlling is based on personalization parameters of the first avatar for which the controlling is performed (Shuster teaches of computing the eye movement animation by choosing a desired target from one or more possible areas of interest in the scene using a set of prioritized targets, e.g. see in Shuster in [0010] “The avatar's gaze is animated to simulate a natural gaze pattern instead of a fixed gaze” and Shuster in the abstract “Control of eye movement may be performed autonomously using software to select and prioritize targets in a visual field. Sequence and duration of apparent gaze may then be controlled using automatically determined priorities”
Shuster teaches that the prioritized targets from which the eye animation is computed may be based on personalization parameters of the avatar, e.g. see Shuster in [0037] where it states “A database of user preference data may be provided in the personalized space … the user preference data may comprise preference values for user-defined features, characteristics, or objects. For example, a user whose avatar has a hobby of collecting books may define a high preference value for books that are in the avatar's field of view.” Also see Shuster in [0044] and [0046] for more details as well.
Shuster teaches that the eye animation is computed based upon the personalized parameters. For example, one avatar may be personalized to highly prioritize visual targets that contain collectable books while another avatar may show little to no interest in these for their respective eye movements. The avatar that highly prioritizes collectable books will tend to look at them more often with their eye animations when the avatar’s preferences are personalized in this way).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention compute the eye gaze animation based on personalization parameters of the avatar as taught by Shuster with the system of Clanton as modified by Zhang, Inomata, Biswas, and Colburn. Shuster teaches one advantage to this feature at the end of [0050] where it states that this personalized eye movement allows other avatars in the scene observing the avatar to better visually gauge their personalized interest, e.g. by looking to see where the avatar’s eyes often gaze to in the virtual scene.
As per claim 10, Clanton alone does not explicitly teach the claimed limitations.
However, Clanton in combination with Zhang teaches the claimed:
10. The method of claim 9, wherein the predetermined eye movement model is trained based on inputs received during eye movements of the first avatar or the second avatar (Zhang teaches this feature in figure 6 and in paragraph [0023]. In particular, the Zhang at step 602 teaches of training the predetermined eye movement model based on inputs received during eye movements of the first avatar).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to train the eye movement model based on inputs received during eye movements of the first avatar as taught by Zhang with the system of Clanton. The motivation of claim 1 is incorporated herein.
As per claim 11, Clanton alone does not explicitly teach the claimed limitations.
However, Clanton in combination with Zhang teaches the claimed:
11. The method of claim 9, wherein the method further comprises: determining, using the machine learning technique, personalization parameters of the eye movement model based on inputs received during eye movements of the first avatar or the second avatar (Please see Zhang in figure 6 at step 602 and Zhang in paragraph [0023] “… The training stage can build an avatar model for the user at 606. As mentioned above, the avatar may be photo-realistic or some other form. The user's avatar can be defined via an avatar model 608. The avatar model can include a number of facial parameters or animation parameters, such as corresponding to eye gaze, mouth movements, etc. Further, the process can train for correlation between sensory inputs and the avatar animation parameters at 610 to create a mapping 612.”
In particular, Zhang is determining personalization parameters of the eye movement model based of the user on inputs received during eye movements of the first avatar).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the machine-learning technique as taught by Zhang with the system of Clanton. The motivation of claim 1 is incorporated herein.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Clanton in view of Zhang in further view of Inomata, Biswas, Colburn, and Reidsma et al. (NPL Doc, “Virtual meeting rooms: from observation to simulation”).
As per claim 12, Clanton alone does not explicitly teach the claimed limitations.
However, Clanton, Zhang, Inomata, Biswas, and Colburn in combination with Reidsma teaches the claimed:
12. The method of claim 1, wherein the method further comprises: training the eye movement model using eye movements captured in video images of people engaging in social activities (Please see Zhang in figure 6 and in paragraph [0023] teaches of training the eye movement model using captured video images. However, Zhang does not mention that the video images are “of people engaging in social activities” per se. Reidsma teaches these claimed features, e.g. see figure 1 where it shows on the left bottom portion of using “Recordings (video, audio)” in order to find “Observable events”. This information is used for “Deriving/building models of interaction” on the far-left side in figure 1. Then, the “models of interaction” are used for 3D VR animation of avatars on the right side in figure 1. For example, these are used for “Rules for generation of communication” in the 3D VR animation (right side in figure 1 of Reidsma).
Reidsma teaches of machine learning of people engaging in social activities in particular, e.g. see in the middle of page 135 where item 2 refers to “Information can be directly obtained from recordings of behaviors in real meetings (e.g., tracking of head or body movements, voice), from annotations or from machine learning models that induce higher level features from recordings.” and on page 137, bottom paragraph where it refers to “The rules for generation of communication are derived from domain knowledge (models and theories of human interaction) collected through the analysis of large amounts of data from real world examples.”
Also see Reidsma on the bottom of page 142 to the top of page 143 where it states “The VMR may add value to the already existing technological means people have to meet and communicate. The various modalities such as speech, gaze, distance gestures and facial expressions can be controlled, which allows VMRs to be used to improve remote meeting participation, to visualize multimedia data and as an instrument for research into social interaction in meetings”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to train the eye movement model via machine learning using eye movements captured in video images of social activities as taught by Reidsma with the system of Clanton as modified by Zhang, Inomata, Biswas, and Colburn. Reidsma teaches the advantage to this type of learning on the top of page 139 where they state that this machine learning using eye movements captured in video images of people engaging in social meeting activities allows the system to find the most essential features in order to maximize the quality of impressions and feelings of presence in corresponding virtual meeting interactions.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights