DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Introduction
This office action is in response to communications filed 06/13/2025. Claims 12-19 are pending and likewise have been examined.
Information Disclosure Statement
The information disclosure statement filed 06/17/2025 fails to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered. The Non-Patent Literature(NPL) #10, Smith “cyclical learning rates for training neural networks” was not provided. Additionally, another NPL by Smith, “GENERAL CYCLICAL TRAINING OF NEURAL NETWORKS”, which is not the same NPL, was provided. The provided Smith reference was not cited in any IDS, which is itself another issue. All provided NPL must be cited in an IDS in order to be considered. Likewise, Jensen “An algorithm for predicting the intelligibility of speech masked by modulated noise maskers”, was not provided by applicant. Instead, a different NPL by Jensen, “Speech Intelligibility Prediction Based on Mutual Information”, was provided. Similarly to Smith above, both the missing reference must be provided, along with the provided reference being cited, in order for the Examiner to consider the references.
The IDS filed 06/217/2025 also includes an incorrect date and name. US Patent No# 12107809, cited by applicant does not have a publishing date of 2025-01-14, or an inventor/assignee named Carrigan. The reference must be cited properly to be considered by Examiner.
The information disclosure statement (IDS) submitted on 08/03/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
In the specification, the number “530” is used for “active speaker 530” in Para 519, as well as “processing layer 530” directly before Para 539. The same number should not be used for two different things.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 12-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent Claim 12 recites the limitations “A method for dynamic conversation graph generation, comprising: receiving as input multimodal sensor data;”, “and building a real-time representation of dynamic social interactions having varying numbers of participants and sub-group formations over time, in the form of a graph with dynamic edge weights”.
The limitations “A method for dynamic conversation graph generation, comprising: receiving as input multimodal sensor data;”, “and building a real-time representation of dynamic social interactions having varying numbers of participants and sub-group formations over time, in the form of a graph with dynamic edge weights”, are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Dependent Claim 13 recites the additional limitations “further comprising computing edge weights using sigmoid-normalized attention scores”. These limitations are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Dependent Claim 14 recites the additional limitations “further comprising computing edge weights using temporal smoothing with an exponential moving average”. These limitations are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Dependent Claim 15 recites the additional limitations “further comprising preserving a graph state using recurrent connections.”. These limitations are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Dependent Claim 16 recites the additional limitations “further comprising thresholding hysteresis for state changes.”. These limitations are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Dependent Claim 17 recites the additional limitations “further comprising applying mel- frequency cepstral correlation on audio input”. These limitations are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Dependent Claim 18 recites the additional limitations “further comprising applying inverse square distance weighting on spatial input.”. These limitations are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Dependent Claim 19 recites the additional limitations “further comprising applying normalized intersection duration to gaze input”. These limitations are directed towards a mental process, as they could be done mentally or by hand on pen and paper.
This judicial exception is not integrated into a practical application. The Claim does not contain any additional limitations.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The Claim does not contain any additional limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 12, 15-16 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chatterjee et al. (US 20230020834 A1), and further in view of Duan et al. (US 20260178161 A1).
Regarding Claim 12:
Chatterjee teaches a method for dynamic…..graph generation, comprising: receiving as input multimodal sensor data(Abstract, Ln 1-15, The scene-aware audio video representation corresponds to a graph of nodes connected by edges. A node in the graph is indicative of the video features of an object in the scene. An edge in the graph connecting two nodes indicates an interaction of the corresponding two objects in the scene);
and building a real-time representation of dynamic social interactions having varying numbers of participants and sub-group formations over time, in the form of a graph with dynamic edge weights(Abstract, Ln 1-15, The scene-aware audio video representation corresponds to a graph of nodes connected by edges. A node in the graph is indicative of the video features of an object in the scene. An edge in the graph connecting two nodes indicates an interaction of the corresponding two objects in the scene. Para [0063], Ln 1-14, may be trained to provide an attention value to each edge, i.e., an interaction of corresponding two objects in the graph 304. Para [0010], Ln 1-11, an object may be classified as a person and not as a group of people unless a group of people belongs to the set of possible objects. Para [0011], Ln 1-16, the scene includes a complex interaction of two objects including one or a combination of spatial interaction and time interaction. For instance, the captured scene may include dynamic content with different….in a spatial domain as well as the evolution of corresponding objects in a time domain. Further, the scene may also include different interactions or relationships existing among the objects. In a real-time example scenario, the scene may correspond to a busy street with objects, such as people in different actions. Para [0083], Ln 1-7, system 400 may process the captured video and assist in navigation of the vehicle 604. For instance, the system 400 may detect and identify a vehicle, such as an ambulance that may be behind the vehicle(new objects may enter over time)).
Chatterjee does not specifically teach conversation graph generation.
In the same field of interaction modeling, Duan teaches conversation graph generation(Para [0039], Ln 1-11, The profile engine 216 may utilize various data structures and algorithms to efficiently organize and retrieve profile information. For example, it may employ graph databases to represent complex relationships between users, interests, and behaviors. Abstract, Ln 1-14, platform determines an attention direction associated with an avatar in a virtual space having two or more groups of additional avatars rendered therein. Each of two or more conversations may be associated with a respective group of the two or more groups. The metaverse platform may determine, based on the attention direction, an interest prediction indicative of a predicted group of interest of the two or more groups).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify Chatterjee with the conversation modeling of Duan, as it can improve user experience(Para [0005], Ln 8-18).
Regarding Claim 15:
The combination of Chatterjee and Duan teaches the method of claim 12, and Chatterjee teaches further comprising preserving a graph state using recurrent connections(Para [0059], Ln 1-9, The graph 304 is provided as input to a Graph Attention Network (GAN) and Recurrent Neural Network (RNN) 306. In an example embodiment, the GAN and RNN 306 may process the graph 304 that includes i) updating nodes of the graph 304, ii) capturing pair-wise interactions between the nodes of the graph).
Regarding Claim 16:
The combination of Chatterjee and Duan teaches the method of claim 12, but does not teach further comprising thresholding hysteresis for state changes.
In the same field of interaction modeling, Duan teaches further comprising thresholding hysteresis for state changes(Para [0020], Ln 1-17, based on the attention direction, an interest prediction. The interest prediction may be a predicted interest, of the avatar or the user thereof, in a group of other avatars (referred to herein as a “predicted group”), a conversation associated with a group of other avatars (referred to herein as a “predicted conversation”), and/or an avatar of a group of other avatars (referred to herein as a “predicted avatar”). In some implementations, if an attention direction of the avatar is directed to a particular group for a predefined period of time…..identifying a predicted group, a predicted conversation, and/or a predicted avatar, the metaverse platform may determine a level of interest of the avatar).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Chatterjee and Duan with the conversation modeling of Duan, as it can improve user experience(Para [0005], Ln 8-18).
Regarding Claim 19:
The combination of Chatterjee and Duan teaches the method of claim 12, but does not teach further comprising applying normalized intersection duration to gaze input.
In the same field of interaction modeling, Duan teaches further comprising applying normalized intersection duration to gaze input(Para [0020], Ln 1-17, if an attention direction of the avatar is directed to a particular group for a predefined period of time, or if the avatar is moving towards the group, the metaverse platform may determine the group to be a predicted group. Para [0019], Ln 1-18, An attention direction may be determined based on a gaze direction of an avatar, a direction of movement of the avatar, and/or any other attribute associated with an avatar that may be indicative of a direction of the attention of the avatar or the user thereof).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Chatterjee and Duan with the conversation modeling of Duan, as it can improve user experience(Para [0005], Ln 8-18).
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Chatterjee and Duan as applied to claim 12 above, and further in view of Li et al “ChatMDG: A discourse parsing graph fusion based approach for multi-party dialogue generation”, hereinafter Li.
Regarding Claim 13:
The combination of Chatterjee and Duan teaches the method of claim 12, but does not teach further comprising computing edge weights using sigmoid-normalized attention scores.
In the same field of Interaction Modeling, Li teaches further comprising computing edge weights using sigmoid-normalized attention scores(Pg 4, Col 2, Para 3(after equation 5), Ln 1-10, node and edge features respectively, 𝛼𝑣 𝑖𝑗 and 𝛼𝑟 𝑖𝑗 denote the attention weights assigned to the current node in relation to its neighboring nodes and edges. Pg 4, Col 2, Para 4, Ln 1-4, edge is acquired as 𝑟𝐿 𝑖𝑗 , with the multi-party dialogue structure conceptualized as a graph. Subsequently, a sigmoid function is employed to ascertain the relevance of each relational edge).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Chatterjee and Duan, with the edge calculations of Li, as it can improve the ability to ascertain relevance of the relations(Pg 4, Col 2, Para 4, Ln 1-4).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Chatterjee and Duan as applied to claim 12 above, and further in view of Duesterwald et al. (US 20250037718 A1).
Regarding Claim 14:
The combination of Chatterjee and Duan teaches the method of claim 12, but does not teach further comprising computing edge weights using temporal smoothing with an exponential moving average.
In the same field of Interaction graphs, Duesterwald teaches further comprising computing edge weights using temporal smoothing with an exponential moving average(Para [0056], Ln 1-12, The weights of the edges of ACG 300 can be updated accordingly. To emphasize the fact that more recent changes are more likely significant, composite discoverer 102 updates the weights assigned to the edges using an edge weight decay function. Para [0057], Ln 1-9, with the time parameter T advancing periodically depending on the number of updates and the underlying processes. Using an exponential weighted moving average (EWMA), composite discoverer 102 computes a decayed weight average as a recursion of time t).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Chatterjee and Duan, with the exponential weighted moving average edges of Duesterwald, as it improves practicality of implementation relative to conventional decay functions(Para [0057], Ln 1-9).
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Chatterjee and Duan as applied to claim 12 above, and further in view of Lumezanu et al. (US 20210065735 A1).
Regarding Claim 17:
The combination of Chatterjee and Duan teaches the method of claim 12, but does not teach further comprising applying mel-frequency cepstral correlation on audio input.
In the same field of Audio processing, Lumezanu teaches further comprising applying mel-frequency cepstral correlation on audio input(Para [0091], Ln 1-8, feed the MFCC features of each of the different acoustic windows into LSTM units such that a hidden state of each of the LSTM units is passed through an attention layer to identify correlations between hidden states. Para [0089], Ln 1-3, input acoustic sequence. Para [0090], Ln 1-7, obtain the MFCC features).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Chatterjee and Duan with the audio processing system of Lumezanu, as it can automate the correction of errors in devices, improving user convenience(Para [0098], Ln 1-16).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Chatterjee and Duan as applied to claim 12 above, and further in view of Chen et al. (US 20200365163 A1).
Regarding Claim 18:
The combination of Chatterjee and Duan teaches the method of claim 12, but does not teach further comprising applying inverse square distance weighting on spatial input.
In the same field of audio processing, Chen teaches further comprising applying inverse square distance weighting on spatial input(Para [0021], Ln 1-20, the weight calculation circuit is configured to calculate a distance between the virtual location of the user and each of the microphones, based on the reported position of the user. The weight factor may be calculated to be inversely proportional to the square of the distance. The weight factor may further be based on a measured or estimated directional sensitivity 500 of the human ear associated with the given head angle, as shown for example in FIG. 5).
It would have been obvious for one skilled in the art, at the effective time of filling, to modify the combination of Chatterjee and Duan, with the distance weighting of Chen, as it can improve encoding efficiency(Para [0022], Ln 1-8).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Caplow-Munro et al. (US 20250191582 A1)
Device directed speech and attention determination
Du et al. (US 12061738 B2)
Attention determination using wearable device and feedback to user based on attention determination.
Tumuluri (US 20220164548 A1)
Constructing attention movement graph based on multi-participant conversations.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER G MARLOW whose telephone number is (571)272-4536. The examiner can normally be reached Monday - Thursday 10:00 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richmond Dorvil can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER G MARLOW/ Assistant Examiner, Art Unit 2658
/RICHEMOND DORVIL/ Supervisory Patent Examiner, Art Unit 2658