Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
1. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
DETAILED ACTION
Claim Rejections - 35 USC § 103
2. Claims 1-3, 5, 8-10, 12, 15, 17, 19 are rejected under 35 U.S.C. 103 as being unpatentable over submitted prior art Chotai et al. (EP 2022249), Centelles Martin et al. (2025/0023934) in view of Carteri et al. (2020/0145363) and Mu et al. (2025/0038918).
As to claim 1, Chotai teaches a method comprising:
receiving, at a push-to-talk server (Fig. 1, PTT server 12) while audio is being played back at a push-to-talk client device, one or more audio messages ([0017] – PTT server 12 that manages or controls a logical floor 13 wherein one of a plurality of participants 16 is permitted to speak at a time via corresponding endpoint devices 16; when a participant wishes to speak during the session he transmits, via his corresponding end point device 16, a talk burst request message to the server 12);
storing the one or more audio messages in a buffer at the push-to-talk server in an order determine based at least in part on an initiation time of the one or more audio messages ([0026] – the buffered talk burst floor control temple which basically allows an open floor. And that is, anyone is free to speak at any time. All requests are time-stamped, buffered and played out in the order that they were generated);
determining an importance for at least a portion of the one or more audio messages in the buffer ([0019] – “Barge-In” floor control algorithm; the idea is to grant a specially designated person(s) permission to barge-in and capture the floor from someone else anytime they want or need to speak to the group. Typically, 911 operators, command center operators/dispatchers, and the like, are persons that might appropriately be conferred with barge-in privileges. Once the barge-in floor control template is applied – either as a general policy or as an overlapping policy on top of another floor control algorithm – any participant that has been granted barge-in privileges is free to take over the floor from anyone who already has the floor and who may be in the middle of speaking);
reordering the one or more audio messages based on the importance ([0018] - ; [0019] - “Barge-In” floor control algorithm; the idea is to grant a specially designated person(s) permission to barge-in and capture the floor from someone else anytime they want or need to speak to the group. Typically, 911 operators, command center operators/dispatchers, and the like, are persons that might appropriately be conferred with barge-in privileges. Once the barge-in floor control template is applied – either as a general policy or as an overlapping policy on top of another floor control algorithm – any participant that has been granted barge-in privileges is free to take over the floor from anyone who already has the floor and who may be in the middle of speaking; hence it is technically reordering audio messages based on barge-in control algorithm with 911 operators, dispatchers…with barge-in privileges); and
transmitting the one or more audio messages for playback at the push-to-talk client device based on the order of the one or more audio messages ([0019], [0026] – all requests are time-stamped, buffered and played out in the order that they were generated).
Chotai does not explicitly discuss using an artificial intelligence engine and based on nonverbal features; determining the importance score comprises considering importance levels of previous messages of a user of the client device, and the artificial intelligence engine is selected from a plurality of models based on a computational capability of the push-to-talk server.
Centelles Martin teaches The processing device 86 receives data from the monitoring module 17 wherein the communication device 12 is configured with push-to-talk (PTT) technology, the user may engage the voice command button 26 to communicate audio to other communication devices 12 of the plurality of communication devices 12 ([0023]); keyword information for each user of a channel 16a-16c and levels of usage or non-usage at a user-specific level and communicates that information to an artificial intelligence (AI) engine 90 on the server 62. The AI engine 90 may process the information captured by the monitoring module 17 in one or more of the machine learning models 84 trained to adjust the turn-based communications based on the information received ([0053]) and The server includes an artificial intelligence engine configured to generate at least one machine learning model trained to determine the modification ([0083]); audio processing unit configured to recognize the voice by encoding phonetic information and nonverbal vocalizations (e.g., laughter, cries, screams, grunts) ([0041, 0051]); and the monitoring module 17 to detect attributes that indicate a level of importance, urgency, or value associated with the communication and the processing routines may apply one or more learning algorithms (e.g., a machine learning algorithm, neural network, etc.) to determine a modification for the at least one channel 16a-16c. The modification may be any modification to the turn-based communications and/or the at least one channel 16a-16c to prioritize important communications over less important communications. For example, the modification may be an adjustment to membership of the at least one channel 16a-16c, an adjustment to allotted talking time for participants on the at least one channel 16a-16c, and/or an adjustment to the priority of one of the at least one channel 16a-16c over another of the at least one channel 16a-16c. In this way, the control circuitry 18 may determine the modification based on the at least one communication attribute and adjust a messaging feature or delivery priority of the at least one channel 16a-16c in response to the modification. The messaging feature or delivery priority includes at least one of a timer for push-to-talk (PTT) messaging, a membership of the user identity 14a-14d to the at least one channel 16a-16c, and a priority level for the at least one channel 16a-16c ([0018-0019]).
Carteri teaches notification performed for the message and the further delivered message if the respective score fulfills the delivery condition. That is, each prior message may have an importance score update to following the delivery of a further delivered message, thus showing which message(s) within the thread specifically need to be responded to ([0098]).
Mu teaches selecting one AI model matching one of a moving speed of the UE, channel quality, computational capability of the UE, and storage capability of the UE from the plurality of AI models ([0066]). It would have been obvious to select AI model based on computational capability of server for the purpose of offloading computational task from a user device to a more powerful server in order to optimize performance, efficiency and resource availability in resource constrained environments.
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Centelles Martin, Carteri, and Mu into the teachings of Chotai for the purpose of applying one or more learning algorithm such as neural network to determine a modification to prioritize important communications over less important communications; determining which prior messages within the thread with an importance score in order to determine which message is specifically need to be responded to and selecting AI models matching one of the moving speed of the UE and computational capability of the UE.
As to claims 2 and 9, Centelles Martin teaches the method of claim 1 and the non-transitory computer readable medium of claim 8, wherein the nonverbal features comprise at least one of a voice tonality, volume, or a pitch ([0041, 0051] - audio processing unit configured to recognize the voice by encoding phonetic information and nonverbal vocalizations (e.g., laughter, cries, screams, grunts)).
As to claims 3, 10, and 17, Centelles Martin teaches the method of claim 1, the non-transitory computer readable medium of claim 8 and the system of claim 15, wherein determining the importance based on verbal features comprising natural language words ([0018] - the monitoring module 17 may monitor communications and detect attributes that indicate a level of importance, urgency, or value of the communication based on other attributes that may be associated with messages communicated among the communication devices 12).
As to claims 5, 12, and 19, Chotai teaches the method of claim 1, the non-transitory computer readable medium of claim 8 and the system of claim 15, further comprising: stopping playback of the audio at the push-to-talk client device before completion of the audio; and immediately starting playback of an audio message from the buffer, based on the importance of the audio message being in a highly importance range [0018] – in a “Priority -based” floor control different subscribers or participants are assigned to different weights or priority values. Participants having higher weights assigned to the name, i.e., a higher priority, therefore have a better chance to capturing the floor in an arbitration contest with another participant having a lower priority weight. For example, in an emergency response or natural disaster situation, a Police Chief for Fire Chief granted the highest priority such that he will gain access to the floor every time he wants to communicate instructions to his subordinates and hence stopping playback audio before completion of the other audio; [0019] – grant a specially designated persons permission to barge-in and capture the floor from someone anytime they want or need to speak to the group; 911 operators, command center operators/dispatchers, and the like, are persons appropriately be conferred with barge-in privileges. Once the barge-in floor control template is applied any participant that has been granted barge-in privileges is free to take over floor from anyone who already has the floor and who may be in the middle of speaking).
Claims 8 and 15 are rejected for the same reasons discussed above with respect to claim 1. Furthermore, Chotai teaches a non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations, a memory subsystem storing instructions and processing circuitry to execute the instructions ([0038]). Centelles Martin teaches control circuitry 18 may be communicatively coupled with the monitoring module 17 and configured to execute one or more processing routines to evaluate the attributes of the communications and associate the communications with a corresponding value, importance, or priority ([0019]), memory 50 and processor 48.
3. Claims 4, 11, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chotai, Centelles Martin, Carteri, and Mu in view of Jabara et al. (20213/0231088).
As to claim 4, Chotai, Centelles Martin, Carteri, and Mu do not explicitly discuss the method of claim 1, further comprising: removing at least one audio message from the buffer based on the importance score being in an unimportant range.
Jabara teaches when the threshold is exceeded, the controller 182 begins to delete the oldest messages first. In another alternative embodiment, messages may be deleted on the basis of message type. For example, business messages may have a lower priority and be deleted first. In contrast, emergency messages may not be deleted until a specific instruction is received to delete the emergency message or until a Message Read Receipt is received ([0134]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Jabara into the teachings of Chotai, Centelles Martin, Carteri, and Mu for the purpose of reserving at least a portion of that capacity for emergency messages, status messages, and the like when the data storage may have a certain capacity.
As to claim 11, Jabara teaches the non-transitory computer readable medium of claim 8, the operations further comprising when the threshold is exceeded, the controller 182 begins to delete the oldest messages first. In another alternative embodiment, messages may be deleted on the basis of message type. For example, business messages may have a lower priority and be deleted first. In contrast, emergency messages may not be deleted until a specific instruction is received to delete the emergency message or until a Message Read Receipt is received ([0134]).
As to claim 18, Jabara teaches the system of claim 15, the processing circuitry further configured to execute the instructions to: remove at least one audio message from the buffer based on the importance score of the at least one audio message ([0134] - when the threshold is exceeded, the controller 182 begins to delete the oldest messages first. In another alternative embodiment, messages may be deleted on the basis of message type. For example, business messages may have a lower priority and be deleted first. In contrast, emergency messages may not be deleted until a specific instruction is received to delete the emergency message or until a Message Read Receipt is received).
6. Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chotai, Centelles Martin, Carteri, and Mu in view of Du et al. (2024/0265605).
As to claim 6, Chotai, Centelles Martin, Carteri, and Mu do not explicitly discuss the method of claim 1, wherein the artificial intelligence engine comprises a convolutional neural network for processing the nonverbal features and a transformer based engine for processing verbal features.
Du teaches Encoder 312 configured to categorize verbal communication information 302, non-verbal communication information 304. Encoder 312 may be a neural network (e.g., deep-learning, a two-dimensional (2D) convolutional neural network (CNN), LSTM, Transformer, etc.) trained (e.g., pretrained) to generate the embeddings including being trained (e.g., pretrained) to identify the verbal and non-verbal communication, categorize the identified verbal communication information 302, non-verbal communication information 304 ([0065]); and Transformer model 316 may work with any combination of verbal communication information 302, non-verbal communication information 304 ([0073]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Du into the teachings of Chotai, Centelles Martin, Carteri, and Mu for the purpose of training the neural network (of the encoder 312) by using multiple sources of information, transformer model 316 may make more accurate predictions and understand the relationships between different types of information.
As to claim 13, Chotai and Centelles Martin do not explicitly discuss the non-transitory computer readable medium of claim 8, wherein the artificial intelligence engine comprises a first sub-engine for processing the nonverbal features and a second sub-engine for processing verbal features.
Du teaches Encoder 312 configured to categorize verbal communication information 302, non-verbal communication information 304. Encoder 312 may be a neural network (e.g., deep-learning, a two-dimensional (2D) convolutional neural network (CNN), LSTM, Transformer, etc.) trained (e.g., pretrained) to generate the embeddings including being trained (e.g., pretrained) to identify the verbal and non-verbal communication, categorize the identified verbal communication information 302, non-verbal communication information 304 ([0065]); and Transformer model 316 may work with any combination of verbal communication information 302, non-verbal communication information 304. Encoder 312 may combine multiple inputs into embeddings 313 to form a complete understanding of the input. Multi-head attention 315 weighs the importance of each type of information and make predictions based on the combined information. By using multiple sources of information, transformer model 316 may make more accurate predictions and understand the relationships between different types of information ([0073]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Du into the teachings of Chotai and Centelles Martin for the purpose of training the neural network (of the encoder 312) by using multiple sources of information, transformer model 316 may make more accurate predictions and understand the relationships between different types of information.
As to claim 20, Chotai and Centelles Martin do not explicitly discuss the system of claim 15, wherein the artificial intelligence engine comprises a first artificial neural network for processing the nonverbal features and a second artificial neural network for processing verbal features.
Du teaches Encoder 312 configured to categorize verbal communication information 302, non-verbal communication information 304. Encoder 312 may be a neural network (e.g., deep-learning, a two-dimensional (2D) convolutional neural network (CNN), LSTM, Transformer, etc.) trained (e.g., pretrained) to generate the embeddings including being trained (e.g., pretrained) to identify the verbal and non-verbal communication, categorize the identified verbal communication information 302, non-verbal communication information 304 ([0065]); and Transformer model 316 may work with any combination of verbal communication information 302, non-verbal communication information 304. Encoder 312 may combine multiple inputs into embeddings 313 to form a complete understanding of the input. Multi-head attention 315 weighs the importance of each type of information and make predictions based on the combined information. By using multiple sources of information, transformer model 316 may make more accurate predictions and understand the relationships between different types of information ([0073]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Du into the teachings of Chotai and Centelles Martin for the purpose of training the neural network (of the encoder 312) by using multiple sources of information, transformer model 316 may make more accurate predictions and understand the relationships between different types of information.
7. Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Chotai, Centelles Martin, Carteri, and Mu in view of Yuksel et al. (2022/0318619).
As to claim 7, Chotai teaches determining an importance for at least a portion of the one or more audio messages in the buffer ([0019] – “Barge-In” floor control algorithm; the idea is to grant a specially designated person(s) permission to barge-in and capture the floor from someone else anytime they want or need to speak to the group. Typically, 911 operators, command center operators/dispatchers, and the like, are persons that might appropriately be conferred with barge-in privileges. Once the barge-in floor control template is applied – either as a general policy or as an overlapping policy on top of another floor control algorithm – any participant that has been granted barge-in privileges is free to take over the floor from anyone who already has the floor and who may be in the middle of speaking). Chotai, Centelles Martin, Carteri, and Mu do not explicitly discuss combining outputs of a first portion of the plurality of artificial neural networks by a second portion of the artificial neural networks.
Yuksel teaches types of models may include artificial neural networks ([0019]). Generating the AI-based solution the processing logic may: identify a second machine learning model in a first database within the marketplace platform, wherein the second machine learning model is a first portion of the AI-based solution; identify a third machine learning model in a second database external to the marketplace platform, wherein the third machine learning model is a second portion of the AI-based solution; and generate the AI-based solution by combining the second machine learning model and the third machine learning model ([0078]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Yuksel into the teachings of Chotai, Centelles Martin, Carteri, and Mu for the purpose of generating the AI-based solution by combining machine learning models and/or artificial neural networks.
As to claim 14, Chotai teaches determining an importance for at least a portion of the one or more audio messages in the buffer ([0019] – “Barge-In” floor control algorithm; the idea is to grant a specially designated person(s) permission to barge-in and capture the floor from someone else anytime they want or need to speak to the group. Typically, 911 operators, command center operators/dispatchers, and the like, are persons that might appropriately be conferred with barge-in privileges. Once the barge-in floor control template is applied – either as a general policy or as an overlapping policy on top of another floor control algorithm – any participant that has been granted barge-in privileges is free to take over the floor from anyone who already has the floor and who may be in the middle of speaking). Chotai and Centelles Martin do not explicitly discuss combining outputs of a first portion of the plurality of artificial intelligence sub-engines by a second portion of the artificial intelligence sub-engines.
Yuksel teaches types of models may include artificial neural networks ([0019]). Generating the AI-based solution the processing logic may: identify a second machine learning model in a first database within the marketplace platform, wherein the second machine learning model is a first portion of the AI-based solution; identify a third machine learning model in a second database external to the marketplace platform, wherein the third machine learning model is a second portion of the AI-based solution; and generate the AI-based solution by combining the second machine learning model and the third machine learning model ([0078]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Yuksel into the teachings of Chotai and Centelles Martin for the purpose of generating the AI-based solution by combining machine learning models and/or artificial neural networks.
8. Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Chotai, Centelles Martin, Carteri, and Mu in view of Hu et al. (2021/0369163).
As to claim 16, Chotai, Centelles Martin, Carteri, and Mu do not explicitly discuss system of claim 15, wherein the nonverbal features comprises at least one of a rate of speech or a pause duration.
Hu teaches audio data input 102 can include speech from multiple speakers (e.g., clinicians, patients). The audio data input also include verbal and non-verbal information… the audio data input 102 can include non-verbal information, such as varying speech rates and energy levels, silences, and pauses ([0024]).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Hu into the teachings of Chotai, Centelles Martin, Carteri, and Mu for the purpose of deriving non-verbal cues from the audio characteristics of the recording (e.g., volume, pitches) and determining speech rate of the speaker if the speech rate is higher than the average speech rate of the speaker during the conversation.
9. Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Chotai, Centelles Martin, Carteri, and Mu in view of Wu et al. (CN 116909855 A).
As to claim 21, Chotai, Centelles Martin, Carteri, and Mu do not explicitly discuss system of claim 1, wherein determining the importance score comprises considering a frequency or a number of previous messages from the user of the client device.
Wu teaches calculating an importance score of the message based on the number of occurrences of the message comprises: counting the times of generating the current message in the threshold time period; and calculating the importance score of the message according to the pre-set single message score and the times of the message after receiving a message, inquiring the occurrence times n of the same message in the near period, and calculating the important degree score of the recent occurrence times of the message according to the set single score PT, namely the single score multiplies the occurrence times (Under specific implementation examples, 20th – 22th paragraphs).
It would have been obvious before the effective filing date of the claimed invention to incorporate the teachings of Wu into the teachings of Chotai, Centelles Martin, Carteri, and Mu for the purpose of calculating the important degree score of the recent or prior occurrence times of the message according to the set single score.
Response to Arguments
10. Applicant’s arguments with respect to claims 1-21 have been considered but are moot because the new ground of rejection(s).
Conclusion
11. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUYNH H NGUYEN whose telephone number is (571)272-7489. The examiner can normally be reached Monday-Thursday 7:30AM-5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached on 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUYNH H NGUYEN/Primary Examiner, Art Unit 2693
Prosecution Insights