DETAILED ACTION
This Office Action is in response to the communication dated 29 October 2025 concerning Application No. 18/216,236 filed on 29 June 2023.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
Claims 1-20 are pending and currently under consideration for patentability; claims 1, 4, 5, 7, 10, 14, and 17 have been amended.
Response to Arguments
Applicant’s arguments dated 29 October 2025 have been fully considered, but they are not persuasive or moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant has amended the independent claims to recite receiving external task data, correlating emotional states and social cues with the external task data, and displaying modified representations of the emotional states and social cues. Applicant argues that previously cited references do not disclose or suggest such limitations.
The Examiner has addressed the amended limitations in the updated text of the rejection below.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-3, 5-9, 11-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Neven et al. (US 2012/0290401 A1) in view of Peters et al. (US 2021/0185276 A1).
Regarding claims 1, 7, and 14, Neven describes a computer apparatus ([0028]) comprising
at least one camera 144
a display 148
a data communication device 140
at least one processor 146 in data communication with a memory 150 storing processor executable code ([0055])
wherein the processor executable code configures the at least one processor to
receive a video stream from the at least one camera ([0021], [0029])
receive external task data comprising an instrument reading or alert condition ([0052], the user interface may be applied to “other interfaces, such automobile dashboards, flight controls, etc” which the Examiner respectfully submits is analogous to an “instrument reading” or “alert condition” as recited)
determine one or more emotional states and social cues based on the video stream ([0047], surprise and interest being social cues)
correlate the determined one or more emotional states and social cues with the external task data in temporal association with the instrument rating or alert condition ([0047]: “pupil dilation can be correlated with emotional states, (e.g., surprise, interest, etc.)… as with the gaze direction information, the pupil dilation information can be generated in real-time and keyed to the real-time scene images”)
generate a modified representation of the emotional states and social cues that reflects the correlation with the external task data ([0047]: “an emotional state indication may be included with entries in gazing log 175 indexed to the identified item viewed while the associated pupil dilation information was gathered”)
Regarding claims 1, 7, and 14, Neven does not explicitly disclose wherein the processor is configured to
receive a predefined baseline user measurement of at least gaze and pupil dynamics and personal facial expression idiosyncrasies
determine the emotional states and social cues compared to a predefined baseline
receive one or more contemporaneous team member emotional states and social cues via the data communication device
send the one or more emotional states and social cues to each team member
display the one or more contemporaneous team member emotional states and social cues associated with a corresponding team member
Regarding claim 14 in particular, Neven does not explicitly disclose or suggest a team monitoring system comprising a plurality of team member monitoring computers ([0334]), each comprising the elements listed above.
However, Peters also describes a computer apparatus comprising a camera, display, and processor configured to determine emotional states and social cues based on eye tracking information ([0017], [0334] - [0335]), including wherein the processor is configured to
– receive a predefined baseline user measurement of at least gaze and pupil dynamics ([0009]), and personal facial expression idiosyncrasies ([0213])
– determine the emotional states and social cues compared to a predefined baseline ([0334] - [0335])
– receive one or more contemporaneous team member emotional states and social cues via the data communication device ([0334] - [0335])
– send the one or more emotional states and social cues to each team member ([0304] - [0305], [0342] - [0343])
– display the one or more contemporaneous team member emotional states and social cues associated with a corresponding team member ([0304] - [0306])
Regarding claim 14 in particular, Peters describes a team monitoring system comprising a plurality of team member monitoring computers ([0334]), each comprising the elements listed above.
As Peters is also directed towards an eye tracking apparatus and is in a similar field of endeavor, it would have been obvious to a person having ordinary skill in the art at the time the invention was filed to incorporate baseline user measurements and contemporaneous team member information similar to those described by Peters when using the apparatus described by Neven, as doing so advantageously allows the resulting apparatus to track the responses for an entire team that may be performing a particular task.
Regarding claims 2, 8, and 15, Peters describes one or more physiological data recording devices in data communication with the at least one processor ([0200], [0244]), wherein the processor executable code further configures the at least one processor to receive physiological data from the one or more physiological data recording devices ([0244]) and correlate the physiological data with the video stream ([0244] - [0245]), and identifying the one or more emotional states and social cues includes reference to the physiological data ([0334] - [0335]).
Regarding claims 3, 9, and 16, Peters describes wherein the processor executable code further configures the at least one processor to receive a task or user specific profile of gaze, scan pattern, facial expression, and physiological data ([0201], [0213]), and identifying the one or more emotional states and social cues includes reference to the task or user specific profile ([0201], [0213]).
Regarding claims 5, 11, and 20, Peters describes wherein the processor executable code further configures the at least one processor to determine a modification to a video stream to enhance at least one of the one or more emotional states and social cues in a corresponding video stream ([0141], [0161, [0177]), and apply the modification to the corresponding video stream ([0141], [0161, [0177]).
Regarding claims 6 and 19, Peters describes wherein the processor executable code further configures the at least one processor as a machine learning neural network ([0033]).
Regarding claims 12 and 18, Peters describes receiving at least one video stream from a team member ([0334] - [0335]), displaying the at least one video stream from the team member on a display ([0334] - [0335]), and determining the user emotional states and social cues with reference to the at least one video stream from the team member ([0334] - [0335]).
Regarding claim 13, Peters describes recording the one or more emotional states and social cues, and one or more contemporaneous team member emotional states and social cues over time (0091], [0355]), and determining a team composition based on one or more emotional states and social cues, and one or more contemporaneous team member emotional states and social cues (0355]).
Claims 4, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Neven in view of Peters, further in view of Chen et al. (US 2023/0161404 A1).
Regarding claims 4, 10, and 17, Neven in view of Peters suggests the computer apparatus of claim 1, the method of claim 7, and the monitoring system of claim 14. Peters further describes wherein the processor executable code further configures the at least one processor to determine a relative gaze direction with respect a speaking team member ([0154]), but Neven and Peters do not explicitly disclose modifying one or more video streams to render a team member’s gaze toward the speaking team member. However, Chen also describes an eye tracking system and method, including determining a relative gaze direction of a user during a video conference and modifying one or more video streams to render a team member’s gaze towards a speaking team member ([0029] - [0030], [0032]). As Chen is also directed towards tracking a user’s gaze and is in a similar field of endeavor, it would have been obvious to a person having ordinary skill in the art at the time the invention was filed to modify the video stream based on a user’s attention, in a manner similar to that described by Chen, when using the system and method described by Neven and Peters, as doing so advantageously allows the resulting system to provide a more visually appealing interface for the speaking team member, as the other team members will appear to be focused and attentive to the speaker.
Statement on Communication via Internet
Communications via Internet e-mail are at the discretion of the applicant. Without a written authorization by applicant in place, the USPTO will not respond via Internet e-mail to any Internet correspondence which contains information subject to the confidentiality requirement as set forth in 35 U.S.C. 122. Where a written authorization is given by the applicant, communications via Internet e-mail, other than those under 35 U.S.C. 132 or which otherwise require a signature, may be used. USPTO employees are NOT permitted to initiate communications with applicants via Internet e-mail unless there is a written authorization of record in the patent application by the applicant. The following is a sample authorization form which may be used by applicant:
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with the undersigned and practitioners in accordance with 37 CFR 1.33 and 37 CFR 1.34 concerning any subject matter of this application by video conferencing, instant messaging, or electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Please refer to MPEP 502.03 for guidance on Communications via Internet.
Conclusion
Applicant’s amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to Ankit D. Tejani, whose telephone number is 571-272-5140. The Examiner may normally be reached on Monday through Friday, 8:30AM through 5:00PM EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, Carl Layno, can be reached by telephone at 571-272-4949. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (in USA or Canada) or 571-272-1000.
/Ankit D Tejani/
Primary Examiner, Art Unit 3792
Prosecution Insights