DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Introduction
This office action is in response to communications filed 05/28/2025. Claims 1-11 are pending and likewise have been examined.
Information Disclosure Statement
The information disclosure statement filed 05/29/2025 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered. The Non-Patent Literature(NPL) #10, Smith “cyclical learning rates for training neural networks” was not provided. Additionally, another NPL by Smith, “GENERAL CYCLICAL TRAINING OF NEURAL NETWORKS”, which is not the same NPL, was provided. The provided Smith reference was not cited in any IDS, which is itself another issue. All provided NPL must be cited in an IDS in order to be considered. Likewise, Jensen “An algorithm for predicting the intelligibility of speech masked by modulated noise maskers”, was not provided by applicant. Instead, a different NPL by Jensen, “Speech Intelligibility Prediction Based on Mutual Information”, was provided. Similarly to Smith above, both the missing reference must be provided, along with the provided reference being cited, in order for the Examiner to consider the references.
The IDS filed 05/29/2025 also includes an incorrect date and name. US Patent No# 12107809, cited by applicant does not have a publishing date of 2025-01-14, or an inventor/assignee named Carrigan. The reference must be cited properly to be considered by Examiner.
The information disclosure statement (IDS) submitted on 08/03/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
In the specification, the number “530” is used for “active speaker 530” in Para 533, as well as “processing layer 530” directly before Para 553. The same number should not be used for two different things.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites “a method for attention modeling in multi-participant environments, comprising:”, “real-time processing of sensory inputs from multiple co-located and remote participants;”, “and generating participant attention states indicating to whom each participant is paying attention”.
The limitations “a method for attention modeling in multi-participant environments, comprising:”, “real-time processing of sensory inputs from multiple co-located and remote participants;”, “and generating participant attention states indicating to whom each participant is paying attention”, cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 2 recites the additional limitations “further comprising modality-specific processing for different sensor inputs”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 3 recites the additional limitations “further comprising cross-modal integration between different sensor inputs”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 4 recites the additional limitations “further comprising processing spatial and temporal relationships between participants”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 5 recites the additional limitations “further comprising multi-level attention processing”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 6 recites the additional limitations “further comprising hierarchical processing with local, regional and global attention spans.”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 7 recites the additional limitations “further comprising adaptive weighting of different input modalities using learned importance scores”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 8 recites the additional limitations “further comprising dynamic processing window adjustment based on conversation dynamics”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 9 recites the additional limitations “further comprising optimization for binary attention state prediction”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 10 recites the additional limitations “further comprising maintaining temporal consistency in attention predictions”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim 11 recites the additional limitations “further comprising employing sparsity constraints to focus on relevant attention patterns”. These limitations cover a mental process as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The claim does not contain any additional limitations.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2 and 4-5 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Haro (US 20230146178 A1).
Regarding Claim 1:
Haro teaches a method for attention modeling in multi-participant environments, comprising: real-time processing of sensory inputs from multiple co-located and remote participants(Para [0054], Ln 1-8, FIG. 4 illustrates an example three-dimensional (3D) chat room/videoconference, including three participants, in accordance with some embodiments. The gaze estimator vector can continuously or regularly be shifting based on the focus of a given participant on various locations of the screen. Para [0004], Ln 1-5, Such technologies include the exchange of video and audio signals to facilitate real-time collaboration of people in different locations. Para [0044], Ln 1-20, a point or region of focus of attention of a first videoconference user is estimated, and a determination is made as to whether the point or region of focus of attention of the first videoconference user overlaps a field of view that includes another (e.g., a second) videoconference user for/during a given videoconference session. In response to determining that the point or region of focus of attention of the first videoconference user overlaps the field of the second videoconference user, one or more audio settings or parameters of a compute device of the first videoconference user and related to the second videoconference user may be adjusted. Para [0060], Ln 1-15, attention-based audio adjustment method 800 includes identifying, at 802, at a processor and at a first time, a first estimated gaze direction of a first participant from a plurality of participants within a virtual environment. The method 800 also includes receiving, at 804, at the processor and from a compute device of the first participant, first audio data including a first set of at least one audio parameter);
and generating participant attention states indicating to whom each participant is paying attention(Para [0044], Ln 1-20, a point or region of focus of attention of a first videoconference user is estimated, and a determination is made as to whether the point or region of focus of attention of the first videoconference user overlaps a field of view that includes another (e.g., a second) videoconference user for/during a given videoconference session. In response to determining that the point or region of focus of attention of the first videoconference user overlaps the field of the second videoconference user, one or more audio settings or parameters of a compute device of the first videoconference user and related to the second videoconference user may be adjusted).
Regarding Claim 2:
Haro teaches the method of claim 1, further comprising modality-specific processing for different sensor inputs(Para [0104], Ln 1-7, an appearance-based gaze estimator is configured to detect a face, resize it, identify facial landmarks thereof, and process the resulting image.(Para [0005], Ln 1-29, in response to detecting that the second estimated gaze direction of the first participant differs from the first estimated gaze direction of the first participant, automatically generating, via the processor, second audio data including a second set of at least one audio parameter different from the first set of at least one audio parameter, based on the first audio data and the second estimated gaze direction. The second audio data can include a modification relative to the first audio data, and can be associated with one of (1) a virtual representation of a second participant from the plurality of participants).
Regarding Claim 4:
Haro teaches The method of claim 1 further comprising processing spatial and temporal relationships between participants(Para [0054], Ln 1-8, FIG. 4 illustrates an example three-dimensional (3D) chat room/videoconference, including three participants, in accordance with some embodiments. The gaze estimator vector can continuously or regularly be shifting based on the focus of a given participant on various locations of the screen. Para [0148], Ln 1-23, The sequence of video frames or packets can be used to interpret the user's attention (e.g., what/who the user is paying attention to and/or looking at). The back end computing system has the capacity, due to computing routines and algorithms, to predict the next region of interest of a certain user at the client side. The next region of focus of the user can be predicted with an accuracy that depends on the region(s) on which the user was focusing several consecutive time instants earlier).
Regarding Claim 5:
The method of claim 1 further comprising multi-level attention processing(Para [0044], Ln 1-20, a point or region of focus of attention of a first videoconference user is estimated, and a determination is made as to whether the point or region of focus of attention of the first videoconference user overlaps a field of view that includes another (e.g., a second) videoconference user for/during a given videoconference session. See Fig 3).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 7, 9, 11is/are rejected under 35 U.S.C. 103 as being unpatentable over Haro as applied to claim 1 above, and further in view of Ryan et al. “Egocentric Auditory Attention Localization in Conversations”, hereinafter Ryan.
Regarding Claim 3:
Haro teaches the method of claim 1, but does not teach further comprising cross-modal integration between different sensor inputs.
In the same field of attention modeling, Ryan teaches further comprising cross-modal integration between different sensor inputs(Pg 14665, Fig 2. Input Video frames Input Audio features, attention heatmaps).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Haro with the attention modeling of Ryan, as it can help improve performance(Pg 14668, 4.3. Comparison results, Para 1, Ln 1-5).
Regarding Claim 7:
Haro teaches the method of claim 1, but does not teach further comprising adaptive weighting of different input modalities using learned importance scores.
In the same field of Attention Modeling, Ryan teaches further comprising adaptive weighting of different input modalities using learned importance scores(Pg 14665, Gig 2, video and audio features are combined in neural network. Pg 14667, 3.3. Implementation & Model Training, Para 1, Ln 1-6, Weights of the model are trained).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Haro with the attention modeling of Ryan, as it can help improve performance(Pg 14668, 4.3. Comparison results, Para 1, Ln 1-5).
Regarding Claim 9:
Haro teaches the method of claim 1, but does not teach further comprising optimization for binary attention state prediction.
In the same field of Attention Modeling, Ryan teaches further comprising optimization for binary attention state prediction(Pg 14666, Col 1, Para 1, Ln 1-8, We use video from a 180-degree fish-eye camera with resolution 200 × 212 and audio from a 6-channel microphone array. In this paper, we demonstrate how to use {I,a} to infer H, where we con struct H_label by setting all pixels within the head bounding box of an attended speaker in a frame as 1. Pg 14665, Fig 2, Attention Heatmaps. Pg 14667, 3.3. Implementation & Model Training, Para 1, Ln 1-6, Weights of the model are trained/optimized).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Haro with the attention modeling of Ryan, as it can help improve performance(Pg 14668, 4.3. Comparison results, Para 1, Ln 1-5).
Regarding Claim 11:
Haro teaches the method of claim 1, but does not teach further comprising employing sparsity constraints to focus on relevant attention patterns.
In the same field of Attention Modeling, Ryan teaches further comprising employing sparsity constraints to focus on relevant attention patterns(Pg 14666, 3.2. Model, Para 2, Ln 1-20, We construct XV by cropping the people’s heads in the raw grayscale video frames and filling the background with black. This representation leverages 1) our focus on conversational attention, and 2) the effectiveness of face detection methods).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Haro with the attention modeling of Ryan, as it can help improve performance(Pg 14668, 4.3. Comparison results, Para 1, Ln 1-5).
Claim(s) 6, 8 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Haro as applied to claim 1 above, and further in view of Duan et al. (US 20260178161 A1).
Regarding Claim 6:
Haro teaches the method of claim 1, but does not teach further comprising hierarchical processing with local, regional and global attention spans.
In the same field of Attention Modeling Duan teaches further comprising hierarchical processing with local, regional and global attention spans(Abstract, Ln 1-14, a metaverse platform determines an attention direction associated with an avatar in a virtual space having two or more groups of additional avatars rendered therein. Each of two or more conversations may be associated with a respective group of the two or more groups. The metaverse platform may determine, based on the attention direction, an interest prediction indicative of a predicted group of interest of the two or more groups. Para [0020], Ln 1-17, The interest prediction may be a predicted interest, of the avatar or the user thereof, in a group of other avatars (referred to herein as a “predicted group”), a conversation associated with a group of other avatars (referred to herein as a “predicted conversation”), and/or an avatar of a group of other avatars (referred to herein as a “predicted avatar”). In some implementations, if an attention direction of the avatar is directed to a particular group for a predefined period of time. See Fig 2B, attention is processed for each avatar, to each group, among all groups being tracked).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Haro with the Attention Modeling of Duan, as it improves user experience(Para [0003], Ln 8-20).
Regarding Claim 8:
Haro teaches the method of claim 1, but does not teach further comprising dynamic processing window adjustment based on conversation dynamics.
In the same field of Attention Modeling Duan teaches further comprising dynamic processing window adjustment based on conversation dynamics(Para [0020], Ln 1-7, The interest prediction may be a predicted interest, of the avatar or the user thereof, in a group of other avatars (referred to herein as a “predicted group”), a conversation associated with a group of other avatars (referred to herein as a “predicted conversation”), and/or an avatar of a group of other avatars (referred to herein as a “predicted avatar”). In some implementations, if an attention direction of the avatar is directed to a particular group for a predefined period of time, or if the avatar is moving towards the group, the metaverse platform may determine the group to be a predicted group. Para [0021], Ln 1-10, metaverse platform may provide an experience enhancement based on the interest prediction. The experience enhancement may include providing a summary of a predicted conversation, a keyword of a predicted conversation, and/or any other information about a predicted group, a predicted conversation, or a predicted avatar).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Haro with the Attention Modeling of Duan, as it improves user experience(Para [0003], Ln 8-20).
Regarding Claim 10:
Haro teaches the method of claim 1, but does not teach further comprising maintaining temporal consistency in attention predictions.
In the same field of Attention Modeling Duan teaches further comprising maintaining temporal consistency in attention predictions(Para [0020], Ln 1-17, The interest prediction may be a predicted interest, of the avatar or the user thereof, in a group of other avatars (referred to herein as a “predicted group”), a conversation associated with a group of other avatars (referred to herein as a “predicted conversation”), and/or an avatar of a group of other avatars (referred to herein as a “predicted avatar”). In some implementations, if an attention direction of the avatar is directed to a particular group for a predefined period of time).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Haro with the Attention Modeling of Duan, as it improves user experience(Para [0003], Ln 8-20).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Caplow-Munro et al. (US 20250191582 A1)
Device directed speech and attention determination
Du et al (US 12061738 B2)
Attention determination using wearable device and feedback to user based on attention determination.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER G MARLOW whose telephone number is (571)272-4536. The examiner can normally be reached Monday - Thursday 10:00 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richmond Dorvil can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER G MARLOW/Assistant Examiner, Art Unit 2658
/RICHEMOND DORVIL/Supervisory Patent Examiner, Art Unit 2658