DETAILED ACTION
This action is in response to remarks filed 05/18/2026:
Claims 1 – 20 are pending and have been examined
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1 - 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Response to Amendment
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 7 and 10 - 19 are rejected under 35 U.S.C. 103 as being unpatentable over Faulkner et al. (W.O. Publication WO 2018169735 A1, hereinafter "Faulkner (W.O. Pub.)") in view of Faulkner et al. (U.S. Patent No. 10,917,608, hereinafter “Faulkner (U.S. Patent)”), Higuchi (J.P. Pub. No. 2010128665) and Kanevsky et al. (U.S. Pub. No. 2014/0093059, hereinafter “Kanevsky”).
Regarding Claim 1, Faulkner (W.O. Pub.) teaches
A method (see Faulkner (W.O. Pub.) Paragraph [0127] and Figure 6, method) comprising:
providing, for presentation on a first client device of a plurality of client devices of a plurality of participants of a video conference, a user interface (UI) comprising a plurality of regions to display a plurality of visual items each corresponding to one of a plurality of video streams from the plurality of client devices (see Faulkner (W.O. Pub.) Figure 3C, in which a video conference interface displays regions to display video output of participants of the video conference);
identifying one of a plurality of events associated with the plurality of participants of the video conference (see Faulkner (W.O. Pub.) Paragraph [0068], activity level priority ranks participants based on their likely contribution to the teleconference);
causing each visual item of the first subset to be invisible in the UI, and causing at least one of remaining visual items to be rearranged in the UI (see Faulkner (W.O. Pub.) Paragraph [0069], placement of graphic elements (or display of video conference participants) may also reflect the activity level priority of the participants, and Paragraph [0072], system may remove graphic element from the primary stream display area, and Figure 4A, in which on the bottom of the video conference interface, there is a region (418) stating "+2", in which there are 2 other participants in this Figure that are hidden, and Paragraph [0073], primary stream display is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed).
Faulkner (W.O. Pub.) does not expressively teach
wherein the plurality of events comprise a change in a volume level associated with at least one of the plurality of client devices of the plurality of participants, and detection of an audio mute signal associated with at least one of the plurality of client devices of the plurality of participants;
determining, based on the one of the plurality of events comprising the change in the volume level associated with the at least one of the plurality of client devices and the detection of the audio mute signal associated with the at least one of the plurality of client devices, a first subset of the plurality of visual items that satisfy at least one of screen invisibility criteria that define whether respective visual items are to be removed from the UI, wherein the screen invisibility criteria comprise a low audio volume criterion that requires a volume level of a corresponding client device be below a threshold audio volume for a predefined period of time as determined by a timer associated with the corresponding client device, and an audio mute criterion that requires an audio of a corresponding client device remain muted for a predefined period of time as determined by a timer associated with the corresponding client device;
However Faulkner (U.S. Patent) teaches
wherein the plurality of events comprise a change in a volume level associated with at least one of the plurality of client devices of the plurality of participants (see Faulkner (U.S. Patent) Column 8, lines 39 – 42, the order in which renderings are arranged can be based on other factors, such as a level of activity, e.g., a volume or rate in which a person is speaking);
determining, based on the one of the plurality of events comprising the change in the volume level associated with the at least one of the plurality of client devices (see Faulkner (U.S. Patent) Column 8, lines 39 – 42, the order in which renderings are arranged can be based on other factors, such as a level of activity, e.g., a volume or rate in which a person is speaking), a first subset of the plurality of visual items that satisfy at least one of screen invisibility criteria that defines whether respective visual items are to be removed from the UI (see Faulkner (U.S. Patent) Figures 1, 2A – 2E, 3A – 3C, 5, and 6, in which item 140 reflects overflow users that are ranked the lowest according to the defined criteria, which may be level of activity such as volume in which a person is speaking. Respective video stream/representation of user is placed in item 140 due to being ranked the lowest, and thus is invisible and removed from the UI), wherein the screen invisibility criteria comprise a low audio volume criterion that requires a volume level of a corresponding client device be below a threshold audio volume for a predefined period of time (see Faulkner (U.S. Patent) Figures 1, 2A – 2E, 3A – 3C, 5, and 6, in which item 140 reflects overflow users that are ranked the lowest according to the defined criteria, which may be level of activity such as volume in which a person is speaking. Respective video stream/representation of user is placed in item 140 due to being ranked the lowest, and thus is invisible and removed from the UI, Column 12, lines 6 – 8, People that are identified in a video component of a stream can be filtered from the count of the number of depicted people based on an activity they are engaged in, people that are talking may also be counted while other people who are not talking for a period of time are not counted, Column 8, lines 40 – 46, the order in which renderings are arranged can be based on other factors, such as a level of activity, e.g., a volume or rate in which a person is speaking. In such an embodiment, an activity level may be generated for each stream, and the rendering of each stream may be ordered within a user interface arrangement 102 based on the activity level, therefore the less active participants, with low volume for a period of time, are ordered as lowest priority, falling into the “+3” area in Figures 1, 2A-2E, 3A-3C, 5 and 6 thus making the less active participants “invisible”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a video conference comprising a layout of video streams and, using events associated with participants, removing video streams from the user interface (as taught by Faulkner (W.O. Pub.)), with ranking participants in a video conference based on voice aspects such as volume and duration to determine what determine which visual items representing users should be removed from the UI (as taught by Faulkner (U.S. Patent)), the motivation being to provide a dynamic and interactive video conference that allows speakers to be the focus of a meeting, to prevent distraction or confusion when there is a large amount of people participating (see Faulkner (U.S. Patent), Abstract).
Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent) does not expressively teach
detection of an audio mute signal associated with at least one of the plurality of client devices of the plurality of participants;
the detection of the audio mute signal associated with the at least one of the plurality of client devices,
as determined by a timer associated with the corresponding client device, and an audio mute criterion that requires an audio of a corresponding client device remain muted for a predefined period of time as determined by a timer associated with the corresponding client device;
However, Higuchi teaches
as determined by a timer associated with the corresponding client device (see Higuchi Paragraph [0018] timer 131, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131, and Paragraph [0011], As shown in FIG. 2, the mobile communication terminal 100 includes an antenna 102, a signal processing unit 103, a clock 104, a speaker 106, a microphone 107, an encoder / decoder 108, an LCD (Liquid Crystal Display) 109, a silence determination unit 111, and voice recognition);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a video conference comprising a layout of video streams and, using events associated with participants, removing video streams from the user interface (as taught by Faulkner (W.O. Pub.)), with ranking participants in a video conference based on voice aspects such as volume and duration to determine what determine which visual items representing users should be removed from the UI (as taught by Faulkner (U.S. Patent)), the motivation being to provide a dynamic and interactive video conference that allows speakers to be the focus of a meeting, to prevent distraction or confusion when there is a large amount of people participating (see Faulkner (U.S. Patent), Abstract).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a video conference comprising a layout of video streams and, using events associated with participants such as volume of speaking for durations of time, removing video streams from the user interface (as taught by Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent)), with implementing a timer when measuring if volume is less than the volume threshold for a duration of time (as taught by Higuchi), the motivation being to establish thresholds that can distinguish intentional communications and an end to a person speaking from random noise or brief pauses, which thus will lead to organized segmentation of speaking (see Higuchi Paragraphs [0002], [0003], [0038], and [0039]). This would thus prevent the interface “flickering” or accidently changing the layout of video displays.
Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent) and Higuchi does not expressively teach
detection of an audio mute signal associated with at least one of the plurality of client devices of the plurality of participants;
the detection of the audio mute signal associated with the at least one of the plurality of client devices,
an audio mute criterion that requires an audio of a corresponding client device remain muted for a predefined period of time as determined by a timer associated with the corresponding client device;
However, Kanevsky teaches
detection of an audio mute signal associated with at least one of the plurality of client devices of the plurality of participants (see Kanevsky Paragraph [0046], After a probability value has been calculate for each participant and/or channel node, a value most indicative of a typing noise, for example, the highest value, may be selected and temporarily muted (Step S32). Where the highest value corresponds to a channel node, that channel node may be temporarily muted and where the highest value corresponds to a participant than the channel node on which the participant is provided from may be temporarily muted. Thereafter, analysis on the combined audio stream may be performed to determine whether the temporary muting of the audio stream of the channel node having the highest probability value was successful in removing the typing noise (Step S34). This determination may be made, for example, by comparing a probability value for the combined audio stream indicative of a likelihood of the presence of a typing noise with the stream with a similar value calculated during the temporary muting. If there is little to no statistically significant difference and/or the likelihood of the presence of the typing value exceeds a predetermined threshold, then it may be determined that there has been no resolution (No, Step S34). In that case, the channel with the next-highest probability value may be temporarily muted (Step S33) and the determination repeated (Step S34) until it is determined that there is a resolution (Yes, Step S34), in which audio mute signal associated with a user and their respective device is detected);
the detection of the audio mute signal associated with the at least one of the plurality of client devices (see Kanevsky Paragraph [0046], After a probability value has been calculate for each participant and/or channel node, a value most indicative of a typing noise, for example, the highest value, may be selected and temporarily muted (Step S32). Where the highest value corresponds to a channel node, that channel node may be temporarily muted and where the highest value corresponds to a participant than the channel node on which the participant is provided from may be temporarily muted. Thereafter, analysis on the combined audio stream may be performed to determine whether the temporary muting of the audio stream of the channel node having the highest probability value was successful in removing the typing noise (Step S34). This determination may be made, for example, by comparing a probability value for the combined audio stream indicative of a likelihood of the presence of a typing noise with the stream with a similar value calculated during the temporary muting. If there is little to no statistically significant difference and/or the likelihood of the presence of the typing value exceeds a predetermined threshold, then it may be determined that there has been no resolution (No, Step S34). In that case, the channel with the next-highest probability value may be temporarily muted (Step S33) and the determination repeated (Step S34) until it is determined that there is a resolution (Yes, Step S34), in which audio mute signal associated with a user and their respective device is detected),
and an audio mute criterion that requires an audio of a corresponding client device remain muted for a predefined period of time as determined by a timer associated with the corresponding client device (see Kanevsky Paragraph [0049], The combined audio stream and/or the audio stream for the channel of the warned participant may be monitored continuously or periodically for some length time to verify that the typing noise has been resolved (Step S37). If the typing noise is not resolved (No, Step S38), then the offending channel node may be muted for some predetermined length of time);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a video conference comprising a layout of video streams and, using events associated with participants, removing video streams from the user interface (as taught by Faulkner (W.O. Pub.)), with ranking participants in a video conference based on voice aspects such as volume and duration to determine what determine which visual items representing users should be removed from the UI (as taught by Faulkner (U.S. Patent)), the motivation being to provide a dynamic and interactive video conference that allows speakers to be the focus of a meeting, to prevent distraction or confusion when there is a large amount of people participating (see Faulkner (U.S. Patent), Abstract).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a video conference comprising a layout of video streams and, using events associated with participants such as volume of speaking for durations of time, removing video streams from the user interface (as taught by Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent)), with implementing a timer when measuring if volume is less than the volume threshold for a duration of time (as taught by Higuchi), the motivation being to establish thresholds that can distinguish intentional communications and an end to a person speaking from random noise or brief pauses, which thus will lead to organized segmentation of speaking (see Higuchi Paragraphs [0002], [0003], [0038], and [0039]). This would thus prevent the interface “flickering” or accidently changing the layout of video displays.
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a video conference comprising a layout of video streams and, using events associated with participants such as volume of speaking for durations of time as determined by a timer, and removing video streams from the user interface (as taught by Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent) and Higuchi), with detecting when a client device is muted and verifies that it remains muted for a predefined period of time (as taught by Kanevsky), the motivation being to establish thresholds that can distinguish intentional communications and thus reduces distractions (see Kanevsky Paragraphs [0005], [0006], [0046] and [0049]).
Regarding Claim 2, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 1, wherein causing the at least one of the remaining visual items to be rearranged in the UI (see Faulkner (W.O. Pub.) Paragraph [0073], primary stream display is rearranged to render one or more streams of teleconference data in the first display area) comprises:
increasing a size of a remaining region displaying the at least one of the remaining visual items to occupy at least a part of a region that displayed a visual item of the first subset that was caused to be invisible in the UI (see Faulkner (W.O. Pub.) Paragraph [0073], primary stream display is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed, graphic element may be enlarged so that the graphic element is at least partially rendered in the first display area, Paragraph [0111], position of graphic elements may indicate the rank or priority based on activity level priority, and Figures 4A and 4B, in which participants with highest activity levels have the largest sized display of their video output, in comparison to the participants seen in the bottom area, or hidden/invisible to the UI (item 418)).
Regarding Claim 3, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 1, wherein the change in the volume level associated with the at least one of the plurality of client devices comprises a change to a low audio volume level associated with a first client device of the plurality of client devices (see Faulkner (U.S. Patent) Column 8, lines 39 – 42, the order in which renderings are arranged can be based on other factors, such as a level of activity, e.g., a volume or rate in which a person is speaking), and wherein determining the first subset of the plurality of visual items that satisfy the at least one of the screen invisibility criteria (see Faulkner (U.S. Patent) Figures 1, 2A – 2E, 3A – 3C, 5, and 6, in which item 140 reflects overflow users that are ranked the lowest according to the defined criteria, which may be level of activity such as volume in which a person is speaking. Respective video stream/representation of user is placed in item 140 due to being ranked the lowest, and thus is invisible and removed from the UI) comprises:
determining that an audio volume level associated with the first client device satisfies the low audio volume criterion for the predefined period of time (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, Paragraph [0038], As a result of the comparison, if volume ≦ volume threshold (YES in step S112), the silence determination unit 111 subsequently compares the time measured by the timer 131 with the time threshold “30 seconds” 122 (step S113). If the measured time is smaller than the time threshold (NO in step S113), the silence determining unit 111 returns to step S112 and continues measuring the time, and Paragraph [0039], As a result of the comparison, if measured time ≧ time threshold (YES in step S113), the silence determination unit 111 outputs a silence notification indicating that a silence state has occurred to the control unit 113 (step S114). When acquiring the silent notification, the control unit 113 acquires and presents information that assists the conversation (step S116). This process will be described in detail later), wherein the determining comprises:
setting a timer associated with the first client device when detecting that the audio volume level associated with the first client device is below the threshold audio volume (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, and Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122); and
determining, after the predefined period of time, that the audio volume level associated with the first client device is still below the threshold audio volume (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, Paragraph [0038], As a result of the comparison, if volume ≦ volume threshold (YES in step S112), the silence determination unit 111 subsequently compares the time measured by the timer 131 with the time threshold “30 seconds” 122 (step S113). If the measured time is smaller than the time threshold (NO in step S113), the silence determining unit 111 returns to step S112 and continues measuring the time, and Paragraph [0039], As a result of the comparison, if measured time ≧ time threshold (YES in step S113), the silence determination unit 111 outputs a silence notification indicating that a silence state has occurred to the control unit 113 (step S114). When acquiring the silent notification, the control unit 113 acquires and presents information that assists the conversation (step S116). This process will be described in detail later); and
adding a visual item corresponding to the video stream from the first client device to the first subset (see Faulkner (W.O. Pub.) Paragraph [0084], the primary stream display area is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed. For example, the graphic element may be enlarged (see FIG. 3D) so that the graphic element is at least partially rendered in the first display area. In another example, the graphic element remains in the second display area, and one or more streams of teleconference data is rendered in the first display area 306 in place of the removed graphic element 304a).
Regarding Claim 4, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 1, wherein causing the at least one of the remaining visual items to be rearranged in the UI (see Faulkner (W.O. Pub.) Paragraph [0073], primary stream display is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed, graphic element may be enlarged so that the graphic element is at least partially rendered in the first display area) comprises:
determining a second subset of the plurality of visual items that do not satisfy the screen invisibility criteria (see Faulkner (W.O. Pub.) Paragraph [0068], activity level priority for identifying active versus passive participants may be determined at the server by analyzing the media stream of each participant. The teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants); and
modifying a position or a size of one or more of the second subset of the plurality of visual items (see Faulkner (W.O. Pub.) Paragraph [0073], primary stream display is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed, Paragraph [0111], position of graphic elements may indicate the rank or priority based on activity level priority, and Figures 4A and 4B, in which participants with highest activity levels have the largest sized display of their video output, in comparison to the participants seen in the bottom area, or hidden).
Regarding Claim 5, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 4, wherein the change in the volume level associated with the at least one of the plurality of client devices comprises a change to a high audio volume level associated with the plurality of client devices (see Faulkner (U.S. Patent) Column 8, lines 39 – 42, the order in which renderings are arranged can be based on other factors, such as a level of activity, e.g., a volume or rate in which a person is speaking), and wherein determining the second subset of the plurality of visual items that do not satisfy the screen invisibility criteria (see Faulkner (U.S. Patent) Figures 1, 2A – 2E, 3A – 3C, 5, and 6, in which item 140 reflects overflow users that are ranked the lowest according to the defined criteria, which may be level of activity such as volume in which a person is speaking. Respective video stream/representation of user is placed in item 140 due to being ranked the lowest, and thus is invisible and removed from the UI) comprises:
comparing an audio volume level associated with a first client device of the plurality of client devices to an audio volume level associated with each other client device of the plurality of client devices (see Faulkner (W.O. Pub.) Paragraph [0079], activity level priority ranks participants based on their likely contribution to the teleconference, the teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants);
determining, based on the comparing, that the audio volume level associated with the first client device is a highest audio volume level (see Faulkner (W.O. Pub.) Paragraph [0090], placement of the graphic elements may also reflect the activity level priority of the participants to which the graphic elements correspond. For example, an overlay view may be defined as having a convention in which the top right of the primary stream display area displays the graphic element corresponding to the most dominant participant. In some sessions, the dominant participant may be a presenter) as compared to each other client device of the plurality of client devices (see Faulkner (W.O. Pub.) Paragraph [0079], activity level priority ranks participants based on their likely contribution to the teleconference, the teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants) for a threshold period of time, wherein the determining is based on a timer associated with the first client device to measure a period of time (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, and Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122) during which the audio volume level associated with the first client device is higher than an audio volume level of each other client device of the plurality of client devices (see Faulkner (W.O. Pub.) Paragraph [0079], activity level priority ranks participants based on their likely contribution to the teleconference, the teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants) for the threshold period of time (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, and Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122); and
adding a visual item corresponding to the video stream from the first client device to the second subset of the plurality of visual items (see Faulkner (W.O. Pub.) Paragraph [0084], the primary stream display area is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed. For example, the graphic element may be enlarged (see FIG. 3D) so that the graphic element is at least partially rendered in the first display area. In another example, the graphic element remains in the second display area, and one or more streams of teleconference data is rendered in the first display area 306 in place of the removed graphic element 304a).
Regarding Claim 6, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 1, wherein causing each visual item of the first subset to be invisible in the UI further (see Faulkner (W.O. Pub.) Figure 4A, in which on the bottom of the video conference interface, there is a region (418) stating "+2", in which there are 2 other participants in this Figure that are hidden) comprises:
causing each respective region that displayed a visual item of the first subset to become hidden on the UI (see Faulkner (W.O. Pub.) Figure 4A, in which on the bottom of the video conference interface, there is a region (418) stating "+2", in which there are 2 other participants in this Figure that are hidden and not shown in the primary stream display area of video conference display in a persistent view in which "active participants" are shown in the primary stream display).
Regarding Claim 7, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 1, wherein causing each visual item of the first subset to be invisible in the UI further (see Faulkner (W.O. Pub.) Figure 4A, in which on the bottom of the video conference interface, there is a region (418) stating "+2", in which there are 2 other participants in this Figure that are hidden) comprises:
causing each respective region that displayed a visual item of the first subset to be removed from the UI (see Faulkner (W.O. Pub.) Paragraph [0073], primary stream display is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed, graphic element may be enlarged so that the graphic element is at least partially rendered in the first display area and Figure 4A, in which on the bottom of the video conference interface, there is a region (418) stating "+2", in which there are 2 other participants in this Figure that are hidden/removed from the UI).
Regarding Claim 10, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 1, wherein the plurality of events further comprise a user selection of a UI element to remove a visual item from the UI (see Faulkner (W.O. Pub.) Paragraph [0071], an event may occur that causes removal of the video component associated with the graphic element. The event may be a user caused event, such as a user toggled control associated with a user interface of the client computing device, an event caused by a client computing device, or an event caused by the device), and wherein the method comprises:
identifying a client device associated with the visual item selected to be removed (see Faulkner (W.O. Pub.) Paragraph [0071], an event may occur that causes removal of the video component associated with the graphic element. The event may be a user caused event, such as a user toggled control associated with a user interface of the client computing device, an event caused by a client computing device, or an event caused by the device); and
adding a visual item corresponding to the video stream from the identified client device to the first subset (see Faulkner (W.O. Pub.) Paragraph [0084], the primary stream display area is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed. For example, the graphic element may be enlarged (see FIG. 3D) so that the graphic element is at least partially rendered in the first display area. In another example, the graphic element remains in the second display area, and one or more streams of teleconference data is rendered in the first display area 306 in place of the removed graphic element 304a).
Regarding Claim 11, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 1, wherein the plurality of events further comprise a user selection of a UI element to adjust a maximum number of visual items displayed in the UI (see Faulkner (W.O. Pub.) Paragraph [0073], primary stream display is rearranged to render one or more streams of teleconference data in the first display area), and wherein the method further comprises:
responsive to determining that an audio volume level of an audio activity of a first client device of the plurality of client devices exceeds a threshold (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, Paragraph [0038], As a result of the comparison, if volume ≦ volume threshold (YES in step S112), the silence determination unit 111 subsequently compares the time measured by the timer 131 with the time threshold “30 seconds” 122 (step S113). If the measured time is smaller than the time threshold (NO in step S113), the silence determining unit 111 returns to step S112 and continues measuring the time, and Paragraph [0039], As a result of the comparison, if measured time ≧ time threshold (YES in step S113), the silence determination unit 111 outputs a silence notification indicating that a silence state has occurred to the control unit 113 (step S114). When acquiring the silent notification, the control unit 113 acquires and presents information that assists the conversation (step S116). This process will be described in detail later), setting a timer associated with the audio activity of the first client device (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, Paragraph [0038], As a result of the comparison, if volume ≦ volume threshold (YES in step S112), the silence determination unit 111 subsequently compares the time measured by the timer 131 with the time threshold “30 seconds” 122 (step S113). If the measured time is smaller than the time threshold (NO in step S113), the silence determining unit 111 returns to step S112 and continues measuring the time, and Paragraph [0039], As a result of the comparison, if measured time ≧ time threshold (YES in step S113), the silence determination unit 111 outputs a silence notification indicating that a silence state has occurred to the control unit 113 (step S114). When acquiring the silent notification, the control unit 113 acquires and presents information that assists the conversation (step S116). This process will be described in detail later);
for each other client device of the plurality of client devices, responsive to determining that an audio volume level of an audio activity of each other client device exceeds the threshold (see Faulkner (W.O. Pub.) Paragraph [0068], activity level priority for identifying active versus passive participants may be determined at the server by analyzing the media stream of each participant. The teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants (in comparison to the other participants)), setting a timer associated with the audio activity of each respective client device of each other client device (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, Paragraph [0038], As a result of the comparison, if volume ≦ volume threshold (YES in step S112), the silence determination unit 111 subsequently compares the time measured by the timer 131 with the time threshold “30 seconds” 122 (step S113). If the measured time is smaller than the time threshold (NO in step S113), the silence determining unit 111 returns to step S112 and continues measuring the time, and Paragraph [0039], As a result of the comparison, if measured time ≧ time threshold (YES in step S113), the silence determination unit 111 outputs a silence notification indicating that a silence state has occurred to the control unit 113 (step S114). When acquiring the silent notification, the control unit 113 acquires and presents information that assists the conversation (step S116). This process will be described in detail later);
comparing a current value of the timer associated with the audio activity of the first client device to a current value of the timer associated with the audio activity of each respective client device of each other client device (see Faulkner (W.O. Pub.) Paragraph [0068], teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants (in comparison to the other participants));
determining, based on the comparing, that the current value of the timer associated with the audio activity of the first client device is a lower value as compared to the current value of the timer associated with the audio activity of each respective client device of each other client device (see Faulkner (W.O. Pub.) Paragraph [0068], teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants (in comparison to the other participants), Figure 4A, in which the participants that are the least active/have the lowest audio activity are hidden in the "+2" region, and the participants with the highest audio activity are displayed in the primary and larger region of the video conference interface); and
adding a visual item corresponding to the video stream from the first client device to the first subset (see Faulkner (W.O. Pub.) Paragraph [0084], the primary stream display area is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed. For example, the graphic element may be enlarged (see FIG. 3D) so that the graphic element is at least partially rendered in the first display area. In another example, the graphic element remains in the second display area, and one or more streams of teleconference data is rendered in the first display area 306 in place of the removed graphic element 304a).
Regarding Claim 12, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
The method of claim 4, wherein the plurality of events further comprise a user selection of a UI element to pin a visual item in the UI (see Faulkner (W.O. Pub.) Paragraph [0060], a given user interface element in the secondary stream display area can be selected by a user to "pin" a particular display of content or a person to the user interface), and wherein the method comprises:
identifying a client device associated with the visual item selected to be pinned (see Faulkner (W.O. Pub.) Paragraph [0060], a given user interface element in the secondary stream display area can be selected by a user to "pin" a particular display of content or a person to the user interface); and
adding a visual item corresponding to the video stream from the identified client device to the second subset (see Faulkner (W.O. Pub.) Paragraph [0060], the user interface element for the content or user for which the "pin" element is selected does not fade over time, but rather remains displayed in the secondary stream display area overlaid on the primary stream display area).
Regarding Claims 13 – 16, they are rejected similarly as Claims 1 – 4, respectively. The system can be found in Faulkner (W.O. Pub.) (Paragraph [0026] and Figure 1, system).
Regarding Claims 17 – 19, they are rejected similarly as Claims 1 – 3, respectively. The non-transitory computer readable storage medium can be found in Faulkner (W.O. Pub.) (Paragraph [0044] and Figure 1, non-transitory computer readable storage medium).
Claims 8, 9 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Faulkner et al. (W.O. Publication WO 2018169735 A1, hereinafter "Faulkner (W.O. Pub.)") in view of Faulkner et al. (U.S. Patent No. 10,917,608, hereinafter “Faulkner (U.S. Patent)”), Higuchi (J.P. Pub. No. 2010128665), Kanevsky et al. (U.S. Pub. No. 2014/0093059, hereinafter “Kanevsky”) Johnson (U.S. Pub. No. 2015/0244981), and Kasilya Sudarsan et al. (U.S. Pub. No. 2018/0047414, hereinafter “Kasilya Sudarsan”).
Regarding Claim 8, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky teaches
and wherein determining the first subset of the plurality of visual items that satisfy the at least one of the screen invisibility criteria (see Faulkner (W.O. Pub.) Paragraph [0068], activity level priority for identifying active versus passive participants may be determined at the server by analyzing the media stream of each participant. The teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants) comprises:
receiving the audio mute signal from a first client device of the plurality of client devices (see Kanevsky Paragraph [0046], After a probability value has been calculate for each participant and/or channel node, a value most indicative of a typing noise, for example, the highest value, may be selected and temporarily muted (Step S32). Where the highest value corresponds to a channel node, that channel node may be temporarily muted and where the highest value corresponds to a participant than the channel node on which the participant is provided from may be temporarily muted. Thereafter, analysis on the combined audio stream may be performed to determine whether the temporary muting of the audio stream of the channel node having the highest probability value was successful in removing the typing noise (Step S34). This determination may be made, for example, by comparing a probability value for the combined audio stream indicative of a likelihood of the presence of a typing noise with the stream with a similar value calculated during the temporary muting. If there is little to no statistically significant difference and/or the likelihood of the presence of the typing value exceeds a predetermined threshold, then it may be determined that there has been no resolution (No, Step S34). In that case, the channel with the next-highest probability value may be temporarily muted (Step S33) and the determination repeated (Step S34) until it is determined that there is a resolution (Yes, Step S34), in which audio mute signal associated with a user and their respective device is detected);
in response to receiving the audio mute signal (see Kanevsky Paragraph [0046], After a probability value has been calculate for each participant and/or channel node, a value most indicative of a typing noise, for example, the highest value, may be selected and temporarily muted (Step S32). Where the highest value corresponds to a channel node, that channel node may be temporarily muted and where the highest value corresponds to a participant than the channel node on which the participant is provided from may be temporarily muted. Thereafter, analysis on the combined audio stream may be performed to determine whether the temporary muting of the audio stream of the channel node having the highest probability value was successful in removing the typing noise (Step S34). This determination may be made, for example, by comparing a probability value for the combined audio stream indicative of a likelihood of the presence of a typing noise with the stream with a similar value calculated during the temporary muting. If there is little to no statistically significant difference and/or the likelihood of the presence of the typing value exceeds a predetermined threshold, then it may be determined that there has been no resolution (No, Step S34). In that case, the channel with the next-highest probability value may be temporarily muted (Step S33) and the determination repeated (Step S34) until it is determined that there is a resolution (Yes, Step S34), in which audio mute signal associated with a user and their respective device is detected),
in response to determining that the timer reaches a threshold value (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, Paragraph [0038], As a result of the comparison, if volume ≦ volume threshold (YES in step S112), the silence determination unit 111 subsequently compares the time measured by the timer 131 with the time threshold “30 seconds” 122 (step S113). If the measured time is smaller than the time threshold (NO in step S113), the silence determining unit 111 returns to step S112 and continues measuring the time, and Paragraph [0039], As a result of the comparison, if measured time ≧ time threshold (YES in step S113), the silence determination unit 111 outputs a silence notification indicating that a silence state has occurred to the control unit 113 (step S114). When acquiring the silent notification, the control unit 113 acquires and presents information that assists the conversation (step S116). This process will be described in detail later), adding a visual item corresponding to the video stream from the first client device to the first subset (see Faulkner (W.O. Pub.) Paragraph [0084], the primary stream display area is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed. For example, the graphic element may be enlarged (see FIG. 3D) so that the graphic element is at least partially rendered in the first display area. In another example, the graphic element remains in the second display area, and one or more streams of teleconference data is rendered in the first display area 306 in place of the removed graphic element 304a); and
resetting the timer associated with the first client device (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, and Paragraph [0037], the silence determination unit 111 resets the timer 131).
Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky does not expressively teach
The method of claim 1, wherein the change in the volume level associated with the at least one of the plurality of client devices comprises detection of an audio mute signal,
setting a timer associated with the first client device to initiate a countdown from an initial value;
However Johnson teaches
The method of claim 1, wherein the change in the volume level associated with the at least one of the plurality of client devices (see Johnson Paragraph [0035] In some implementations, a view rank value may be based in part on one or more voice aspects such as volume and duration. For example, system 102 may detect an increase in the volume of the voice of a user who is not the current presenter. If the volume at a particular endpoint increases above a predefined threshold and remains above the predefined threshold longer than a predefined duration, system 102 may change the view rank value for a given endpoint based on an average volume being over a predetermined duration (e.g., average volume over the last 3 seconds, 5 seconds, 10 seconds, etc.) at that endpoint) comprises detection of an audio mute signal (see Johnson Paragraph [0042], in some implementations, a view rank value may be based in part on whether particular video functions have been turned on. For example, in some implementations, a view rank value may be based in part on whether a video mute feature has been turned on),
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of determining that a participant’s video stream should be added to a subset of hidden visual items after receiving an audio mute signal, determining that a timer reaches a threshold value, and resetting the timer (as taught in Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky), with a change in a volume level associated with a client devices comprises detection of an audio mute signal (as taught in Johnson), the motivation being that if a participant is muted they will not have an audio output, thus meaning they will not have any audio activity while muted (see Johnson, Paragraph [0042]).
Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi, Kanevsky and Johnson does not expressively teach
setting a timer associated with the first client device to initiate a countdown from an initial value;
However, Kasilya Sudarsan teaches
setting a timer associated with the first client device to initiate a countdown from an initial value (see Kasilya Sudarsan Paragraph [0060], In one aspect, the device 406 may activate the microphone 414 when a volume level associated with an audio signal output through the speaker 412 meets or exceeds the first threshold. The device 406 may then begin countdown of a first timer. The microphone may remain activated for the duration of the timer. At expiration of the first timer, the device 406 may again determine a volume level associated with an audio signal output through the speaker 412 and either deactivate or continue activation of the microphone 414 based on the comparison of the volume level to the threshold);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of determining that a participant’s video stream should be added to a subset of hidden visual items after receiving an audio mute signal, determining that a timer reaches a threshold value, and resetting the timer (as taught in Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi and Kanevsky), with a change in a volume level associated with a client devices comprises detection of an audio mute signal (as taught in Johnson), the motivation being that if a participant is muted they will not have an audio output, thus meaning they will not have any audio activity while muted (see Johnson, Paragraph [0042]).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of determining that a participant’s video stream should be added to a subset of hidden visual items based on a volume level associated with a client device that includes detection of an audio mute signal, after a timer reaches a threshold value and is reset (as taught in Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi, Kanevsky and Johnson), with setting a timer associated with a client device to initiate a countdown (as taught in Kasilya Sudarsan), the motivation being to establish thresholds that can distinguish intentional communications and an end to a person speaking from random noise or brief pauses, which thus will lead to organized segmentation of speaking (see Kasilya Sudarsan, Paragraph [0058]).
Regarding Claim 9, Faulkner (W.O. Pub.) in view of Faulkner (U.S. Patent), Higuchi, Kanevsky, Johnson and Kasilya Sudarsan teaches
The method of claim 1, wherein determining the first subset of the plurality of visual items that satisfy the at least one of the screen invisibility criteria (see Faulkner (W.O. Pub.) Paragraph [0068], activity level priority for identifying active versus passive participants may be determined at the server by analyzing the media stream of each participant. The teleconference system may include a function that compares the activity of participants and dynamically promotes those who speak more frequently or those that move and/or speak more frequently to be the active participants) comprises:
receiving the audio mute signal from a host of the video conference (see Johnson Paragraph [0042], system may decrease the view rank value based on another user, which may be a host, turning on the mute feature for another endpoint),
in response to receiving the audio mute signal (see Johnson Paragraph [0042], in some implementations, a view rank value may be based in part on whether a video mute feature has been turned on):
identifying a first client device of the plurality of client devices associated with the audio mute signal (see Johnson Paragraph [0042], system may decrease the view rank value of a given endpoint, in which the given endpoint is a specified user that another participant, which may be a host, specifically muted); and
setting a timer associated with the first client device to initiate a countdown from an initial value (see Kasilya Sudarsan Paragraph [0060], In one aspect, the device 406 may activate the microphone 414 when a volume level associated with an audio signal output through the speaker 412 meets or exceeds the first threshold. The device 406 may then begin countdown of a first timer. The microphone may remain activated for the duration of the timer. At expiration of the first timer, the device 406 may again determine a volume level associated with an audio signal output through the speaker 412 and either deactivate or continue activation of the microphone 414 based on the comparison of the volume level to the threshold);
in response to determining that the timer reaches a threshold value (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, Paragraph [0038], As a result of the comparison, if volume ≦ volume threshold (YES in step S112), the silence determination unit 111 subsequently compares the time measured by the timer 131 with the time threshold “30 seconds” 122 (step S113). If the measured time is smaller than the time threshold (NO in step S113), the silence determining unit 111 returns to step S112 and continues measuring the time, and Paragraph [0039], As a result of the comparison, if measured time ≧ time threshold (YES in step S113), the silence determination unit 111 outputs a silence notification indicating that a silence state has occurred to the control unit 113 (step S114). When acquiring the silent notification, the control unit 113 acquires and presents information that assists the conversation (step S116). This process will be described in detail later), adding a visual item corresponding to the video stream from the first client device to the first subset (see Faulkner (W.O. Pub.) Paragraph [0084], the primary stream display area is rearranged to render one or more streams of teleconference data in the first display area once the graphic element is removed. For example, the graphic element may be enlarged (see FIG. 3D) so that the graphic element is at least partially rendered in the first display area. In another example, the graphic element remains in the second display area, and one or more streams of teleconference data is rendered in the first display area 306 in place of the removed graphic element 304a); and
resetting the timer associated with the first client device (see Higuchi Paragraph [0018], Silence determination unit 111 The silence determination unit 111 acquires sound from the encoder / decoder 108 under the control of the control unit 113, and the silence time during which the volume of the acquired sound is continuously equal to or lower than a volume threshold (described later). It is a circuit that measures. The silence determination unit 111 stores a volume threshold “−70 db” 121 and a time threshold “30 seconds” 122 in an internal memory. As shown in FIG. 3, the silence determination unit 111 includes a comparator 130 and a timer 131, Paragraph [0019], The sound volume threshold 121 is generally the minimum sound volume when a person has a conversation, and is “−70 db” as an example here. The time threshold value 122 is a time length estimated that the conversation is stagnant, and is “30 seconds” as an example here. The volume threshold 121 and the time threshold 122 may be set when the mobile communication terminal 100 is manufactured, or may be set by a user operation after shipment, Paragraph [0020], The silence determination unit 111 has a function of converting the acquired sound into a digital signal and measuring the volume (unit db). It is possible to measure up to 0db. The silence determination unit 111 compares the volume measured by the comparator 130 with the volume threshold value 121, and measures the silence period when the measured volume is equal to or less than the volume threshold 121 by the timer 131. When the measured silent time exceeds the time threshold value 122, a silent notification indicating that the conversation is delayed is output to the control unit 113. Note that the silence notification may include a time when the measured silence time exceeds the time threshold 122, and Paragraph [0037], the silence determination unit 111 resets the timer 131).
Regarding Claim 20, it is rejected similarly as Claim 8. The non-transitory computer readable storage medium can be found in Faulkner (W.O. Pub.) (Paragraph [0044] and Figure 1, non-transitory computer readable storage medium).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARISSA A JONES whose telephone number is (703)756-1677. The examiner can normally be reached Telework M-F 6:30 AM - 4:00 PM CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached on 5712727503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARISSA A JONES/Examiner, Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691
Prosecution Insights