Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to application filed 02/25/2026.
Claims 1-20 are pending in this application.
Response to Arguments
Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 6 are rejected under 35 U.S.C. 103 as being unpatentable over Nagpal et al. (US 11,540,092 B1) in view of Nishide et al. (US 2013/0335517 A1)
Regarding claim 1, Nagpal discloses a video conferencing system, (column 2, 36-37: The conference monitoring system may generate and/or identify parameters associated with a given conference) comprising:
a lead session manager comprising a processing system including a processor deployed at a network edge (column 22, 11-12: conference monitoring system 100 may be located at the network edge);
a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations (column 23, 40-43: Memory 1330 may include any type of dynamic storage device that may store information and instructions for execution by processor 1320), comprising:
monitoring content of respective data streams of a video conferencing session at the network edge (column 3, 40-45: conference monitoring system 100 (e.g. located at network edge) may track, derive, and/or extract these and other parameters as each participant joins a conference and/or based on network, performance, participant activity, and/or other metrics that conference monitoring system 100 tracks during a conference);
obtaining a performance assessment at the network edge based on the monitoring and responsive to initiation of a communication session (column 8, 56-61: Conference monitoring system 100 may detect (at 306) when the particular conference commences, and may generate (at 308) a second set of parameters to track the experience of each user during the particular conference. The particular conference may commence once two or more users have joined the particular conference. Column 9, 7-20: the second set of parameters may measure (e.g. performance assessment) packet loss, jitter, bandwidth, and/or other packet delivery metrics for each user, processor usage on the user devices, fan speed on the user devices, and/or other metrics obtained from the user devices, each user's conference feed, application, and/or service, and/or the network connection established by each user device);
determining, at the network edge and according to the performance assessment, capabilities of respective devices of participants of a video conferencing session provisioned via a communication network, the capabilities comprising audio-visual capabilities and connection capabilities of the respective devices related to communicating the respective data streams of the video conferencing session via the communication network (column 8, 67-column 9, 7: conference monitoring system 100 may track metrics related to how often a user's feed is interrupted, delayed, and/or changed, which conference settings are manually adjusted by users (e.g., volume, brightness, etc.), performance metrics of the user devices (e.g., processor load, memory load and/or capacity, etc.), network performance (e.g., throughput, latency, jitter, etc.), and/or other experience-related metrics);
determining, at the network edge, respective audio-visual parameters of the video conferencing session applicable to the respective devices as a function of the capabilities of the respective devices (column 2, 51-58: the conference monitoring system may implement various actions to optimize an ongoing conference experience, future conferences (e.g., that involve one or more conference participants from the newly classified conference and/or different conference participants), or conferences with the same or similar parameters. The various actions may include adjusting audio, video, and/or other settings of the conference), the respective audio-visual parameters of the video conferencing session including measures of audio quality, audio quality parameters, measures of video quality, video quality parameters, and user preferences (column 3, 8-17: the metrics may include a quantity of times each participant was disconnected and reconnected, a quantity of instances of packet loss or stuttering experienced by each participant, the resolution or bitrate for each participant connection, a quantity of times that audio, video, and/or other settings were toggled by each participant (e.g. user preferences), network and/or performance metrics (e.g., jitter, round-trip delay time, throughput, etc.), and/or other values related to individual participants during the conference); and
determining a context of the video conferencing session (column 18, 7-21: conference monitoring system 100 may generate customized UIs to mix-and-match different parameters, scores, participants, and/or conference, and to present a visualization that provide customized insight into the impact of specific monitored parameters, common issues affecting sets of conferences, experiences of specific participants across different conferences, and/or groups of conferences of a certain size (e.g., a minimum number of participants), at specific sites or locations, at different times of day, etc.); and controlling, at the network edge, performance of the video conferencing session at the respective devices in accordance with the respective audio-visual parameters based on the context and at least one of the user preferences (column 14, 24-28: conference monitoring system 100 may monitor a conference, may detect a particular parameter for a particular participant that has an anomalous value during a first time, a settings change made by the particular participant. Column 15, 38-47: conference monitoring system 100 (e.g. at network edge) may automatically implement the corrective actions in response to the model classifying one or more conferences or conference participants as having a negative experience. The set of corrective actions may include reducing the bitrate at which the particular participant's device accesses the conference, deactivating the video stream for the conference on the particular participant device, and/or adjusting other settings of the application or device used by the particular participant to access the conference. Column 18, 7-21: conference monitoring system 100 may generate custom UIs to track the experience of specific persons-of-interest (e.g., the Chief Executive Officer, sales agents, technical support staff, teachers, doctors, etc.), and to present visualizations that summarize the experience of those users across different conferences (e.g. context)).
However, Nagpal does not disclose controlling comprises, based on the context and at least one of the user preferences, either:(i) enhancing an audio quality parameter of the audio quality parameters relative to a video quality parameter of the video quality parameters based on the context and/or the at least one of the user preferences, wherein the context and/or the at least one of the user preferences indicates the audio quality parameter has a higher priority relative to the video quality parameter or (ii) enhancing the video quality parameter of the video quality parameters relative to the audio quality parameter of the audio quality parameters based on the context and/or the at least one of the user preferences, wherein the context and/or the at least one of the user preferences indicates the video quality parameter has a higher priority relative to the audio quality parameter.
In an analogous art, Nishide discloses controlling comprises, either:(i) enhancing an audio quality parameter of the audio quality parameters relative to a video quality parameter of the video quality parameters based on the context and/or the at least one of the user preferences, wherein the context and/or the at least one of the user preferences indicates the audio quality parameter has a higher priority relative to the video quality parameter or (ii) enhancing the video quality parameter of the video quality parameters relative to the audio quality parameter of the audio quality parameters based on the context and/or the at least one of the user preferences, wherein the context and/or the at least one of the user preferences indicates the video quality parameter has a higher priority relative to the audio quality parameter ([0055]: The priority information held by the priority defining unit 24, the bandwidth, and a time interval of bandwidth control conducted by the bandwidth control unit 22 may be defined or set in advance, but also can be customized by the user as needed. [0104]: The priority of the video information may be lowered to improve quality of the voice information and the screen information and it may be moved back to an original priority when it becomes possible to secure at least certain quality for the video information to provide video information of at least the certain quality).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal to comprise “controlling comprises, either:(i) enhancing an audio quality parameter of the audio quality parameters relative to a video quality parameter of the video quality parameters based on the context and/or the at least one of the user preferences, wherein the context and/or the at least one of the user preferences indicates the audio quality parameter has a higher priority relative to the video quality parameter or (ii) enhancing the video quality parameter of the video quality parameters relative to the audio quality parameter of the audio quality parameters based on the context and/or the at least one of the user preferences, wherein the context and/or the at least one of the user preferences indicates the video quality parameter has a higher priority relative to the audio quality parameter” taught by Nishide.
One of ordinary skilled in the art would have been motivated because it would have enabled to allocate the bandwidth starting from information with a high priority (Nishide, [0063]).
Regarding claim 2, Nagpal-Nishide discloses the video conferencing system of claim 1, wherein the determining, at the network edge, respective audio-visual parameters comprises determining the respective audio-visual parameters to balance at least one of the measures of audio quality and at least one of the measures of video quality (Nagpal, column 15, 14-18: conference monitoring system 100 may attribute the negative experience to insufficient bandwidth, and may deactivate the video feed and/or lower the audio quality of the audio feed to improve the user experience), and wherein the capabilities of respective devices include one or more of: a type of audio-visual equipment (Nagpal, column 8,44-55: conference monitoring system 100 may obtain the network address, software version, device type (e.g., make and model), bandwidth, round-trip-time, and/or other metrics for the first user based on data packets the user devices send in order to join the conference and/or other identifying information provided by the user devices in response to queries from conference monitoring system 100), a number of equipment components, audio-visual capabilities of equipment components, a quality of audio and video capable of being captured and streamed, maximum and minimum capture and rendering resolution capabilities of image/video data (Nagpal, column 8, 67-column 9, 7: conference monitoring system 100 may track metrics related to how often a user's feed is interrupted, delayed, and/or changed, which conference settings are manually adjusted by users (e.g., volume, brightness, etc.), performance metrics of the user devices), display dimensions, input and output power, frequency response, signal-to-noise ratio, total harmonic distortion, sensitivity, and impedance.
Regarding claim 3, Nagpal-Nishide discloses the video conferencing system of claim 2, wherein the operations further comprise: monitoring the respective measures of audio and video quality at the respective devices during the video conferencing session; and adapting the respective audio-visual parameters based on changes to the respective measures of audio and video quality (Nagpal, column 14, 24-34: For example, conference monitoring system 100 may monitor a conference, may detect a particular parameter for a particular participant that has an anomalous value during a first time, a settings change made by the particular participant, and the particular parameter returning to a normal value after the settings change by particular participant. In this example, conference monitoring system 100 may dynamically determine to automatically apply the same settings change in response to the particular parameter for other conference participants having the same anomalous values).
Regarding claim 4, Nagpal-Nishide discloses the video conferencing system of claim 1, wherein the operations further comprise: detecting a speech characteristic that indicates a need to adjust a rendering of an audio component of at least one of the respective data streams, wherein the speech characteristic comprises at least one of a speed, a loudness, a pause, a network-induced delay, a stuttering, or a background noise (Nagpal, [0082]: A microphone may pick up a loud noise which has started occurring in the background which muffles someone speaking as a presenter. In such situations, different optimal settings can be configured on the fly to account for the change in situation); based on the speech characteristic, performing an audio adjustment comprising at least one of adjusting an audio playback rate, adjusting a volume level, adding a filler audio, suppressing a portion of the audio component, or filtering the background noise; and the controlling further comprises controlling respective communication parameters applicable to communicating the respective data streams in accordance with the respective audio- visual parameters and audio adjustment. (Nagpal, column 15, 38-47: conference monitoring system 100 may automatically implement the corrective actions in response to the model classifying one or more conferences or conference participants as having a negative experience. The set of corrective actions may include reducing the bitrate at which the particular participant's device accesses the conference, deactivating the video stream for the conference on the particular participant device, and/or adjusting other settings of the application or device used by the particular participant to access the conference. Column 2, 56-60: The various actions may include adjusting audio, video, and/or other settings of the conference (e.g., bitrate, resolution, activating or deactivating video, adjusting volume of individual conference participants, etc.).
Regarding claim 6, Nagpal-Nishide discloses the video conferencing system of claim 4, wherein the operations further comprise: monitoring, at the network edge, the respective measures of audio and video quality at the respective devices during the video conferencing session (Nagpal, column 14, 24-34: For example, conference monitoring system 100 may monitor a conference, may detect a particular parameter for a particular participant that has an anomalous value during a first time, a settings change made by the particular participant); and adapting the respective communication parameters based on changes to the respective measures of audio and video quality determined based on the monitoring (Nagpal, column 15, 38-47: conference monitoring system 100 may automatically implement the corrective actions…The set of corrective actions may include reducing the bitrate at which the particular participant's device accesses the conference, deactivating the video stream for the conference on the particular participant device, and/or adjusting other settings of the application or device used by the particular participant to access the conference).
Claim 5, 7, 8 are rejected under 35 U.S.C. 103 as being unpatentable over Nagpal in view of Nishide, as applied to claim 4, in view of Gizis et al. (US 2023/0379377 A1 – Priority date May 19, 2022) .
Regarding claim 5, Nagpal-Nishide discloses the video conferencing system of claim 4.
However, Nagpal-Nishide does not disclose wherein the respective communication parameters are selected from a group of communication parameters comprising a downlink speed parameter, an uplink speed parameter, a latency parameter, and a reliability parameter.
In an analogous art, Gizis discloses wherein the respective communication parameters are selected from a group of communication parameters comprising a downlink speed parameter, an uplink speed parameter, a latency parameter, and a reliability parameter ([0050]: the initial data rates to be used during the conference may be based on the lowest data rate of any of the client devices to ensure no device experiences latency or loss of data. [0054]: Also, historical data that is saved by a primary VPN or database may be retrieved to identify the latency and other data signal degradation characteristics of any of the VPNs to ensure a reliable VPN was selected. [0069]: device 312 so the data provided to the other device 314 may remain optimal since the device 314 may be identified as having a lower data rate than device 312 so the lower bit rate video/audio, etc., may provide optimization for the lower data rate device 314 and avoid real time communication errors).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “wherein the respective communication parameters are selected from a group of communication parameters comprising a downlink speed parameter, an uplink speed parameter, a latency parameter, and a reliability parameter” taught by Gizis.
One of ordinary skilled in the art would have been motivated because it would have enabled data management operations, such as optimizing data flow for one or more client devices participating in the conference call via a conference application operating on a client device and/or a corresponding server (Gizis, [0022]).
Regarding claim 7, Nagpal-Nishide discloses the video conferencing system of claim 6.
However, Nagpal-Nishide does not disclose wherein the adapting further comprises re-allocating network resources of the communication network used to communicate the respective data streams in accordance with the respective communication parameters.
In an analogous art, Gizis discloses wherein the adapting further comprises re-allocating network resources of the communication network used to communicate the respective data streams in accordance with the respective communication parameters ( [0072]: the VPN is managing a conference for multiple client devices and performing resource balancing for all client devices' optimal communication experience. The process may include receiving, at a virtual private network (VPN) server, a plurality of conference confirmations from a plurality of client devices, and determining initial optimal conference settings. [0073]: The process may further include modifying the one or more of the first set of conference parameters by assigning one or more of a higher video data rate and a higher audio data rate to one of the plurality of client devices and maintaining the first set of conference parameters for another one of the plurality of conference client devices. The process may further include identifying a conference action trigger performed by one or more of the client devices by determining one or more of the client devices initiated: one or more of a video and an audio sharing function, a change in the data rate used during the conference, and one or more of a jitter or latency threshold being exceeded).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “wherein the adapting further comprises re-allocating network resources of the communication network used to communicate the respective data streams in accordance with the respective communication parameters” taught by Gizis.
One of ordinary skilled in the art would have been motivated because it would have enabled data management operations, such as optimizing data flow for one or more client devices participating in the conference call via a conference application operating on a client device and/or a corresponding server (Gizis, [0022]).
Regarding claim 8, Nagpal-Nishide discloses the video conferencing system of claim 1, the operations further comprise: detecting a video content rendering characteristic that indicates a need to adjust a rendering of a video component of at least one of the respective data streams, wherein the video content rendering characteristic comprises at least one of a video rendering rate, a network-induced delay, a cropping boundary, a placement, a size, a coloration, a hue, a brightness, a contrast, a lighting, or a combination thereof (Nagpal, column 8, 66-column 9, 7: Instead, conference monitoring system 100 may track metrics related to how often a user's feed is interrupted, delayed, and/or changed, which conference settings are manually adjusted by users (e.g., volume, brightness, etc.), performance metrics of the user devices (e.g., processor load, memory load and/or capacity, etc.), network performance (e.g., throughput, latency, jitter, etc.), and/or other experience-related metrics);
based on the video content rendering characteristic, performing a video adjustment comprising at least one of adjusting a playback rate, adding a filler video, cropping, centering, enlarging, changing coloration, changing hue, changing brightness, changing contrast, changing lighting, or a combination thereof (Nagpal, column 7, 45-51: Conference monitoring system 100 may isolate a first cause for the issues experienced by the particular participant based on the detected anomalies, and may implement (at 216) a first set of actions that may include reducing the resolution or bitrate at which the particular participant connects to the first unlabeled conference, if still ongoing, or for future conferences that the particular participant participates in).
However, Nagpal-Nishide does not disclose controlling respective communication parameters applicable to communicating the respective data streams in accordance with the respective audio-visual parameters and the video adjustment.
In an analogous art, Gizis discloses controlling respective communication parameters applicable to communicating the respective data streams in accordance with the respective audio-visual parameters and the video adjustment ([0050]: the initial data rates to be used during the conference may be based on the lowest data rate of any of the client devices to ensure no device experiences latency or loss of data. [0054]: Also, historical data that is saved by a primary VPN or database may be retrieved to identify the latency and other data signal degradation characteristics of any of the VPNs to ensure a reliable VPN was selected. [0069]: device 312 so the data provided to the other device 314 may remain optimal since the device 314 may be identified as having a lower data rate than device 312 so the lower bit rate video/audio, etc., may provide optimization for the lower data rate device 314 and avoid real time communication errors).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “controlling respective communication parameters applicable to communicating the respective data streams in accordance with the respective audio-visual parameters and the video adjustment” taught by Gizis.
One of ordinary skilled in the art would have been motivated because it would have enabled data management operations, such as optimizing data flow for one or more client devices participating in the conference call via a conference application operating on a client device and/or a corresponding server (Gizis, [0022]).
Claims 9-10, 13 are rejected under 35 U.S.C. 103 as being unpatentable over Nagpal in view of Nishide, as applied to claim 8, in further view of Sun et al. (US 2015/0156598 A1).
Regarding claim 9, Nagpal-Nishide discloses the video conferencing system of claim 1.
However Nagpal-Nishide does not disclose wherein operations further comprise, in response to detecting that a participant of the video conferencing session is speaking, but an audio setting of the participant's device is set to muted, automatically changing the audio setting to unmuted.
In an analogous art, Sun discloses wherein operations further comprise, in response to detecting that a participant of the video conferencing session is speaking, but an audio setting of the participant's device is set to muted, automatically changing the audio setting to unmuted ([0070]: the control circuit can determine whether the interference or high-level noise events continue for a further period of time or increase in volume, and if so, then the mute operation can be entered by the control circuit without manual operation by the speaker. A corresponding mute notification can be generated to indicate that this automated process by the control circuit has been executed).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “wherein operations further comprise, in response to detecting that a participant of the video conferencing session is speaking, but an audio setting of the participant's device is set to muted, automatically changing the audio setting to unmuted” taught by Sun.
One of ordinary skilled in the art would have been motivated because it would have enabled to execute a mute or unmute state change corresponding to the mute or unmute indication after a predetermined amount of time has lapsed during which no mute or unmute operation from a user is received (Sun, [0011]).
Regarding claim 10, Nagpal-Nishide-Sun discloses the video conferencing system of claim 9, wherein the detecting that the participant of the video conferencing session is speaking comprises determining at least one of whether a mouth of the participant is moving, whether sound content transmitted by a respective device of the participant is consistent with norms of the video conferencing session, whether the sound content is consistent with background noise associated with the video conferencing session, or a combination thereof (Sun, [0029]: if a participant does not speak for a while and forgets to mute the microphone, but the speaker's environment is noisy (caused by interferences such as keyboard typing, mouse clicking, furniture moving, colleague speaking, etc.), then a system is able to detect the interferences). The same rationale applies as in claim 9.
Regarding claim 13, Nagpal-Nishide-Sun discloses the video conferencing system of claim 9, wherein the operations further comprise notifying at least one participant's device that the audio setting has been changed to unmuted (Sun, [0070]: the control circuit can determine whether the interference or high-level noise events continue for a further period of time or increase in volume, and if so, then the mute operation can be entered by the control circuit without manual operation by the speaker. A corresponding mute notification can be generated to indicate that this automated process by the control circuit has been executed). The same rationale applies as in claim 9.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Nagpal in view of Nishide, as applied to claim 10, in view of Olivieri et al. (US 2023/0117301 A1).
Regarding claim 11, Nagpal-Nishide discloses the video conferencing system of claim 1.
However, Nagpal-Nishide does not disclose wherein determining the respective audio-visual parameters further comprises selecting background image content for rendering at the devices based on the context of the video conferencing session, and wherein the controlling further comprises: controlling the rendering of the background image content at the devices during the video conferencing session.
In an analogous art, Olivieri discloses wherein determining the respective audio-visual parameters further comprises selecting background image content for rendering at the devices based on the context of the video conferencing session, and wherein the controlling further comprises: controlling the rendering of the background image content at the devices during the video conferencing session ([0035]: a conferencing system terminal device may apply a virtual background generated as a function of the contextual information to portions of the images depicting objects other than the subject to create contextually background modified images of the subject. A communication device of the conferencing system terminal device may then transmit the contextually background modified images of the subject across a network to another conferencing system terminal device engaged in the videoconference).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “wherein the determining the respective audio-visual parameters further comprises selecting background image content for rendering at the devices based on the context of the video conferencing session, and wherein the controlling further comprises: controlling the rendering of the background image content at the devices during the video conferencing session” taught by Olivieri.
One of ordinary skilled in the art would have been motivated because it would have enabled to automatically applying virtual background as a function of contextual information of a conferencing system terminal device engaged in the videoconference (Olivieri, [0017]).
Claims 12, 14-15 is rejected under 35 U.S.C. 103 as being unpatentable over Nagpal in view of Nishide, as applied to claim 1, in view of Swierk et al. (US 11,350,059 B1).
Regarding claim 12, Nagpal-Nishide discloses the video conferencing system of claim 1.
However, Nagpal-Nishide does not disclose wherein the determining the respective audio-visual parameters further comprises determining an audio filter or a video filter to apply to a data stream of the respective data streams based on a context of the video conferencing session and a preference associated with a participant identity of a participant in the video conferencing session and associated with the data stream, and wherein the controlling comprises applying the audio filter of the video filter to the data stream.
In an analogous art, Swierk discloses wherein the determining the respective audio-visual parameters further comprises determining an audio filter or a video filter to apply to a data stream of the respective data streams based on a context of the video conferencing session and a preference associated with a participant identity of a participant in the video conferencing session and associated with the data stream, and wherein the controlling comprises applying the audio filter of the video filter to the data stream (column 4, 13-19: the intelligent appearance monitoring management system may allow for options to select optimized appearance filtering adjustments, such as selection of only a notification of a user appearance anomaly, or what types of adjustments to a user's image may be invoked and associated with one or more types of user appearance anomalies detected. Column 19, 17-23: determining whether a videoconference call is a business meeting or a casual event, along with assessment of invited participants, to determine in some embodiments a type of optimized appearance filter adjustment that may be output by the intelligent appearance monitoring management system executing a trained neural network. Column 30, 47-50: trained intelligent appearance monitoring management system neural network issuing an optimized appearance filtering adjustment that provides for an alteration of a user's image in the videoframes).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “wherein the determining the respective audio-visual parameters further comprises determining an audio filter or a video filter to apply to a data stream of the respective data streams based on a context of the video conferencing session and a preference associated with a participant identity of a participant in the video conferencing session and associated with the data stream, and wherein the controlling comprises applying the audio filter of the video filter to the data stream” taught by Swierk.
One of ordinary skilled in the art would have been motivated because it would have enabled generating optimized appearance filtering adjustments to intelligently monitor or adjust a user's appearance during capture (Swierk, column, 15 37-39).
Regarding claim 14, Nagpal-Nishide discloses the video conferencing system of claim 1.
However, Nagpal-Nishide does not disclose wherein the operations further comprise: analyzing audio-visual data of the respective data streams during the video conferencing session; detecting events associated with the video conferencing session based on the analyzing; and adapting the respective audio-visual parameters based on the events.
In an analogous art, Swierk discloses wherein the operations further comprise: analyzing audio-visual data of the respective data streams during the video conferencing session; detecting events associated with the video conferencing session based on the analyzing; and adapting the respective audio-visual parameters based on the events (column 19, 17-23: determining whether a videoconference call is a business meeting or a casual event, along with assessment of invited participants, to determine in some embodiments a type of optimized appearance filter adjustment that may be output by the intelligent appearance monitoring management system executing a trained neural network).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “wherein the operations further comprise: analyzing audio-visual data of the respective data streams during the video conferencing session; detecting events associated with the video conferencing session based on the analyzing; and adapting the respective audio-visual parameters based on the events” taught by Swierk.
One of ordinary skilled in the art would have been motivated because it would have enabled generating optimized appearance filtering adjustments to intelligently monitor or adjust a user's appearance during capture (Swierk, column, 15 37-39).
Regarding claim 15, Nagpal-Nishide discloses the video conferencing system of claim 1.
However, Nagpal-Nishide does not disclose wherein the operations further comprise: monitoring eye movement of the participants of the video conferencing session; and controlling capture of image data of the respective data streams as a function of the eye movement.
In an analogous art, Swierk discloses wherein the operations further comprise: monitoring eye movement of the participants of the video conferencing session; and controlling capture of image data of the respective data streams as a function of the eye movement (column 4, 13-19: he intelligent appearance monitoring management system may allow for options to select optimized appearance filtering adjustments, such as selection of only a notification of a user appearance anomaly, or what types of adjustments to a user's image may be invoked and associated with one or more types of user appearance anomalies detected. Column 55, 7-16: a boundary detection AV processing instruction module may process the captured videoframes for identification of the user's image within the videoframe, location of the hair of a user in the image, location of the face or eyes of the user, or to reduce the area of the videoframe needing to be analyzed for user appearance anomalies. It is contemplated that other AV processing instruction modules may execute code instructions for videoframe processing, such as color sensor detection and analysis across the videoframe to assist in detection of user appearance anomalies).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Nishide to comprise “wherein the operations further comprise: monitoring eye movement of the participants of the video conferencing session; and controlling capture of image data of the respective data streams as a function of the eye movement” taught by Swierk.
One of ordinary skilled in the art would have been motivated because it would have enabled generating optimized appearance filtering adjustments to intelligently monitor or adjust a user's appearance during capture (Swierk, column, 15 37-39).
Claim 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Nagpal et al. (US 11,540,092 B1) in view of Viswanathan Iyer al. (herein after Viswanathan, US 2022/0236782 A1) .
Regarding claim 16, Nagpal discloses method, comprising:
obtaining at the network edge, by a processing system, a performance assessment upon initiation of a communication session (column 8, 56-61: Conference monitoring system 100 may detect (at 306) when the particular conference commences, and may generate (at 308) a second set of parameters to track the experience of each user during the particular conference. The particular conference may commence once two or more users have joined the particular conference. Column 9, 7-20: the second set of parameters may measure (e.g. performance assessment) packet loss, jitter, bandwidth, and/or other packet delivery metrics for each user, processor usage on the user devices, fan speed on the user devices, and/or other metrics obtained from the user devices, each user's conference feed, application, and/or service, and/or the network connection established by each user device);
determining at the network edge, by the processing system and according to the performance assessment, capabilities of the respective devices of participants of the video conferencing session provisioned via a communication network, the capabilities comprising audio-visual capabilities and connection capabilities of the respective devices related to communicating respective data streams of the video conferencing session via the communication network (column 8, 67-column 9, 7: conference monitoring system 100 may track metrics related to how often a user's feed is interrupted, delayed, and/or changed, which conference settings are manually adjusted by users (e.g., volume, brightness, etc.), performance metrics of the user devices (e.g., processor load, memory load and/or capacity, etc.), network performance (e.g., throughput, latency, jitter, etc.), and/or other experience-related metrics);
determining at the network edge, by the processing system, respective audio-visual settings of the video conferencing session applicable to the respective devices as a function of the capabilities of the respective devices (column 2, 51-58: the conference monitoring system may implement various actions to optimize an ongoing conference experience, future conferences (e.g., that involve one or more conference participants from the newly classified conference and/or different conference participants), or conferences with the same or similar parameters. The various actions may include adjusting audio, video, and/or other settings of the conference); and
controlling at the network edge, by the processing system, performance of the video conferencing session at the respective devices in accordance with the respective audio-visual settings (column 14, 24-28: conference monitoring system 100 may monitor a conference, may detect a particular parameter for a particular participant that has an anomalous value during a first time, a settings change made by the particular participant. Column 15, 38-47: conference monitoring system 100 (e.g. at network edge) may automatically implement the corrective actions in response to the model classifying one or more conferences or conference participants as having a negative experience. The set of corrective actions may include reducing the bitrate at which the particular participant's device accesses the conference, deactivating the video stream for the conference on the particular participant device, and/or adjusting other settings of the application or device used by the particular participant to access the conference).
However, Nagpal does not disclose wherein the controlling comprises; modeling, by the processing system, performance of respective data streams corresponding to respective devices of participants of a video conferencing session at the network edge to predict a delay impacting at least one of a measure of audio quality, a measure of video quality at a respective device of the respective devices, or a combination thereof; and in response to predicting the delay, mitigating the delay based on adjusting one or more communication parameters, re-allocating network resources, or a combination thereof.
In an analogous art, Viswanathan discloses wherein the controlling comprises; modeling, by the processing system, performance of respective data streams corresponding to respective devices of participants of a video conferencing session at the network edge to predict a delay impacting at least one of a measure of audio quality, a measure of video quality at a respective device of the respective devices, or a combination thereof ([0126]: media capture instructions, A/V processing instructions, processor utilization settings gathered at block 712, and one or more meeting metrics describing performance of the MMCA (e.g., latency, jitter, dropped packets, etc.) The error function for latency may be equivalent to the difference between this output predicted latency value and the known latency value measured during the videoconference user videoconference session. As another example, the error function for jitter may be equivalent to the difference between this output predicted jitter value and the known jitter value measured during the videoconference user videoconference session; and in response to predicting the delay, mitigating the delay based on adjusting one or more communication parameters, re-allocating network resources, or a combination thereof ([0145]: These optimized instructions and adjustments in an embodiment may be predicted to cause measured latency, jitter, or dropped packets (or other meeting metrics) to remain at or below the preset threshold value input into the trained neural network or provide for optima MMCA operation for features selected or implemented for post-processing of audio and visual data for the videoconference session).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal to comprise “wherein the controlling comprises; modeling, by the processing system, performance of respective data streams corresponding to respective devices of participants of a video conferencing session at the network edge to predict a delay impacting at least one of a measure of audio quality, a measure of video quality at a respective device of the respective devices, or a combination thereof; and in response to predicting the delay, mitigating the delay based on adjusting one or more communication parameters, re-allocating network resources, or a combination thereof” taught by Viswanathan.
One of ordinary skilled in the art would have been motivated because it would have enabled applying optimized A/V processing instruction adjustments and optimized MMCA processor utilization instructions within a videoconference user videoconference session (Viswanathan, [0051]).
Regarding claim 17, Nagpal-Viswanathan discloses the method of claim 16, wherein the modeling comprises a time/average prediction model (Viswanathan, [0116]: meeting metrics gathered by the intelligent collaboration multi-application and power management system during a training session may describe latency, or a measurement of time elapsing between a first information handling system transmitting the processed, encoded media sample and a second information handling system receiving the processed, encoded media sample). The same rationale applies as in claim 16.
Claim 18 are rejected under 35 U.S.C. 103 as being unpatentable over Nagpal in view of Viswanathan, as applied to claim 16, in view of Sridhar et al. (US 2021/0014725 A1).
Regarding claim 18, Nagpal-Viswanathan discloses the method of claim 16.
However, Nagpal-Viswanathan does not disclose wherein the modeling predicts the delay between 10 and 60 seconds ahead of time.
In an analogous art, Sridhar discloses wherein the modeling predicts the delay between 10 and 60 seconds ahead of time ([0038]: The training process may map the input features to delay measurements associated with later times (e.g., 10s or 100s of milliseconds later, for example) in relation to the times associated with the input features such that the model is trained to predict future delays (e.g., 10s or 100s of milliseconds in the future, for example) based on the input features).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Nagpal-Viswanathan to comprise wherein the modeling predicts the delay between 10 and 60 seconds ahead of time” taught by Sridhar.
One of ordinary skilled in the art would have been motivated because it would have enabled to predict delays and encoding rate associated with the conversational video traffic and allocate resources to the conversational video traffic based on the likely encoding rates and other network conditions. (Sridhar, [0006]).
Claims 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Sun et al. (US 2015/0156598 A1) in view of Nagpal et al. (US 11,540,092 B1) .
Regarding claim 19, Sun discloses a non-transitory machine-readable medium, comprising executable instructions that, when executed by a processing system facilitate performance of operations, comprising: detecting, based on the monitoring, that a participant of the video conferencing session is acting in a manner consistent with a respective device of the participant being muted based whether sound content transmitted by the respective device is consistent with norms of the video conferencing session, whether the sound content is consistent with background noise associated with the video conferencing session, or a combination thereof; determining that the respective device is unmuted ([0029]: if a participant does not speak for a while and forgets to mute the microphone, but the speaker's environment is noisy (caused by interferences such as keyboard typing, mouse clicking, furniture moving, colleague speaking, etc.), then a system is able to detect the interferences;
in response to the detecting that the participant of the video conferencing session is acting in a manner consistent with the respective device of the participant being muted and to the determining that the respective device is unmuted, muting the respective device automatically ([0070]: the control circuit can determine whether the interference or high-level noise events continue for a further period of time or increase in volume, and if so, then the mute operation can be entered by the control circuit without manual operation by the speaker); and
notifying at least one participant's device that the respective device has been muted ([0070]: A corresponding mute notification can be generated to indicate that this automated process by the control circuit has been executed).
However, Sun does not disclose a processing system including a processor of a network edge node deployed at a network edge, monitoring content of respective data streams corresponding to respective devices of participants of a video conferencing session at the network edge.
In an analogous art, Nagpal discloses a processing system including a processor of a network edge node deployed at a network edge (column 22, 11-12: conference monitoring system 100 may be located at the network edge), monitoring content of respective data streams corresponding to respective devices of participants of a video conferencing session at the network edge (column 3, 40-45: conference monitoring system 100 (e.g. located at network edge) may track, derive, and/or extract these and other parameters as each participant joins a conference and/or based on network, performance, participant activity, and/or other metrics that conference monitoring system 100 tracks during a conference).
Therefore, it would have been obvious before the effective filed date of the claimed invention to a person having ordinary skill in the art to modify Sun to comprise “a processing system including a processor of a network edge node deployed at a network edge, monitoring content of respective data streams corresponding to respective devices of participants of a video conferencing session at the network edge” taught by Nagpal.
One of ordinary skilled in the art would have been motivated because it would have enabled to take appropriate remedial measures to enhance or improve the experience of conference participants (column 4, 26-28, [0006]).
Regarding claim 20, Sun-Nagpal discloses the non-transitory machine-readable medium of claim 19, wherein the operations further comprise: detecting, based on the monitoring, that the participant is acting in a manner consistent with the respective device of the participant being unmuted based on the at least one of whether sound content transmitted by the respective device is consistent with norms of the video conferencing session or whether the sound content is consistent with background noise associated with the video conferencing session; determining that the respective device is muted (Sun, [0029]: when in a call, if the participant starts speaking while the microphone is muted, the conferencing system is able to detect this active speaker event); in response to detecting that the participant of the video conferencing session is acting in a manner consistent with the respective device of the participant being unmuted and to the determining that the respective device is muted, unmuting the respective device automatically; and notifying at least one participant's device that the respective device has been unmuted (Sun, [0066]: the control circuit can determine whether the speaker events continue for a further period of time or increase in volume, and if so, then the unmute operation can be entered by the control circuit without manual operation by the speaker. A corresponding unmute notification can be generated to indicate that this automated process by the control circuit has been executed). The same rationale applies as in claim 19.
Additional References
The prior art made of record and not relied upon is considered pertinent to applicants disclosure.
Roper, US 2023/0126108 A1: Dynamic Context-Sensitive Virtual Backgrounds for Video Conferences
Jeapes et al., US 2022/0360896 A1: Modular Conferencing System .
Springer., US 2022/0360635 A1: Intelligent Configuration of Personal Endpoint Devices.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUAN C TURRIATE GASTULO whose telephone number is (571)272-6707. The examiner can normally be reached Monday - Friday 8 am-4 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian J Gillis can be reached at 571-272-7952. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.C.T/Examiner, Art Unit 2446
/MICHAEL A KELLER/Primary Patent Examiner, Art Unit 2446
Prosecution Insights