DETAILED ACTION
Notice of Pre-AIA or AIA Status
Claims 1, 6-9, 11-17, and 19-26 are pending in this application. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 11 and 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant’s arguments, see pages 13-15, filed 12/29/2025, with respect to claims 11-12 have been fully considered and are persuasive. The rejection of claims 11-12 has been withdrawn.
With respect to applicant’s arguments on pages 9-11, with respect to former claim 4, applicant’s argue the examiner’s prior rationale improperly relied solely upon “omission” of the claimed limitation, the examiner respectfully disagrees.
The examiner clarifies that the prior rejection was not intended to assert that mere silence in the references, standing alone, renders the limitation obvious. Rather, the cited omission was directed to the disclosed conditional prioritization framework of Faulkner and Harpur.
Specifically, Faulkner and Harpur collectively disclose that participant video feeds are prioritized based on detected emotional expressions and participant activity satisfying the criteria used for participant ranking, sentiment analysis, and conference layout selection. Thus, the cited references disclose conditional prioritization logic in which prioritization occurs responsive to qualifying emotional expressions associated with conferencing activity.
Accordingly, when the detected emotional expression does not satisfy the relevance criteria used for prioritization within the conference context, the participant video feed is not prioritized. The examiner’s prior reference to “omission” was intended to reflect that the references do not prioritize participant video feeds when the detected emotional expression fails to satisfy the disclosed prioritization criteria, rather than merely asserting that silence alone renders the claim obvious.
Therefore, the rejection is based on the disclosed selective prioritization operation of the cited references, including the corresponding absence of prioritization when the emotional expression is not treated as qualifying for conference-related prioritization.
With respect to applicant’s arguments on pages 11-13, applicant’s argue that independent claim 16 (amended to include limitations of former claim 18), is not disclosed by Faulkner in view of Harpur as it would have been obvious to one of ordinary skill in the art to arrive at former claims 6-9 and 18 as a matter of obvious design choice. In response to applicant’s arguments, the examiner has further considered the scope and content of the prior art and has identified additional reference evidencing that the disputed features was known in the art and represented recognized implementation within video conferencing and participant-analysis systems.
The addition of refences is responsive to applicant’s traversal and is provided to further support the previously stated obviousness rationale with additional evidentiary support. The examiner is permitted to cite supporting evidence during prosecution in response to applicant’s arguments where such references further demonstrate that the claimed features would have been obvious to one of ordinary skill in the art at the time of the invention.
Applicant’s arguments with respect to claim(s) 13 have been considered but are moot because the new ground of rejection does not rely on the combination of references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The examiner notes that previous claim 13 is a dependent off claim 1 alone. Claim 1 now recites claim limitations from former dependent claims 2-4 in which changes the scope of the invention of current claim 13.
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, Faulkner discloses dynamically generating candidate conference layouts and prioritizing participant video feeds based on detected participant activities and expressions ([0074]-[0076]). Harpur discloses performing sentiment analysis on participant video feeds and generating sentiment parameter-value datasets associated with participant facial features and emotional analytics in order to prioritize or deprioritize video feeds according to participant influence and conferencing (col. 4, lines 1-40, col. 13, lines 17-23). One of ordinary skill in the art would have been motivated to incorporate Harpur’s sentiment-analysis techniques into Faulkner’s conference layout-selection system in order to improve accuracy, contextual relevance, and effectiveness of participant prioritization during conferences.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Faulkner (US 2019/0377586, hereinafter Faulkner). and in view of Harpur (US 10,250,851).
Re claim 1, Faulkner discloses an apparatus, comprising:
a processor system ([0164] processors that execute the performed functions); and
storage accessible to the processor system and comprising instructions executable by the processor system to ([0164], computer-executable instructions stored on one or more computer readable media executed by one or more processors):
facilitate a video conference ([0070], [0074], selecting video feed interacting with multiple individuals to configure video feeds to generate candidate layouts); and
during facilitation of the video conference, prioritize presentation of first video of a first video conference participant over second video of a second video conference participant based on one or more criteria other than the first video conference participant speaking and/or other than a client device of the first video conference participant transmitting audio data indicating the first video conference participant as speaking ([0074]-[0076], candidate layouts are generated based on the detected activity (speaking, stands up, raises a hand, clapping, laughing, frowning, etc.). the system can rank the candidate layouts and display the candidate layouts based on the ranking. One of the parameters can applied based on importance of individual participants);
wherein the one or more criteria comprise the first video conference participant expressing an emotion selected from the group consisting of: laughter, happiness, sadness, crying, cheering ([0074], candidate layouts are generated based on the detected activity (speaking, stands up, raises a hand, clapping, laughing, frowning, etc.).
wherein the instructions are executable to: determine, in a first instance, that the emotion the first video conference participant is expressing is related to the video conference ([0074]-[0076], candidate layouts are generated based on the detected activity (speaking, stands up, raises a hand, clapping, laughing, frowning (emotion), etc.). the system can rank the candidate layouts and display the candidate layouts based on the ranking. One of the parameters can applied based on importance of individual participants.);
based on the determination in the first instance, prioritize presentation of the first video ([0074]-[0076], candidate layouts are generated based on the detected activity (speaking, stands up, raises a hand, clapping, laughing, frowning, etc.). the system can rank the candidate layouts and display the candidate layouts based on the ranking. One of the parameters can applied based on importance of individual participants). While Faulkner discloses in [0074]-[0076] candidate layouts are generated based on the detected activity (speaking, stands up, raises a hand, clapping, laughing, frowning, etc.). The system can rank the candidate layouts and display the candidate layouts based on the ranking with the activity of laughing, frowning (emotions), Faulkner does not clearly disclose wherein the instructions are executable to: determine, in a second instance, that the emotion the first video conference participant is expressing is not related to the video conference; and
based on the determination in the second instance, not prioritizing presentation of the first video.
Hapur discloses in col. 4, lines 1-40 the manager run sentiment analysis based on emotion video analytics. It determines sentiment parameter-value datasets associated with the video feed representing facial features of a participant. Hapur also discloses in col. 13, lines 17-23, the updated video conference view presented to the certain user can prioritize the presenting of video feed views of users identified as positive influencing users and can deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant.
It would have been obvious to one of ordinary skill in the art at the time of the invention to combine the participant activity-based conference layout techniques of Faulkner with the sentiment-analysis and participant influence analytics of Harpur in order to improve participant prioritization and conference relevance during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings of Faulkner’s conferencing layout-selection system with Harpur’s sentiment-analysis techniques in order to improve the accuracy, contextual relevance, and effectiveness of participant prioritization during conferences and predictably enabled the conferencing system to more intelligently prioritize participant video feeds based no only on detected participant activity, by also on emotional expressions and participant influence information derived from sentiment analysis, thereby improving conferencing presentation and reducing less relevant participant emphasis.
Re Claim 14, Faulkner discloses wherein the apparatus comprises one or more of: a server facilitating the video conference at least in part by routing audio video communications between client devices of respective video conference participants, a first client device facilitating the video conference by transmitting local audio video to the server and/or to other client devices of other respective video conference participants ([0086], communication session of a user participating in the conference session with the audio data that captures the speech of the user).
Re Claim 15, one of ordinary level of skill in the art would have been compelled to make the proposed modification to Faulkner for the same reasons identified in the rejection of claim 1. In addition, Harpur discloses wherein prioritizing presentation of the first video over the second video comprises one or more of: presenting the first video higher up on a graphical user interface (GUI) than the second video, highlighting the first video on the GUI, the highlighting comprising one or more of: presenting a first color along a border of the first video, presenting a graphic overlay in relation to the first video (Fig. 6F, col. 13, lines 25-40, Positively influencing user J is presented in an enlarged video feed while identified negatively influencing user R is depicted in a relatively smaller video area 604).
Claim(s) 6, 8-9, 16-17, 19-21, and 23-25 are rejected under 35 U.S.C. 103 as being unpatentable over Faulkner and in view of Harpur and in view of Bastide (US 10,069,878).
Re claim 16, Faulkner discloses a method, comprising:
facilitate a video conference ([0070], [0074], selecting video feed interacting with multiple individuals to configure video feeds to generate candidate layouts); and
during facilitation of the video conference, prioritize presentation of first video of a first video conference participant over second video of a second video conference participant based on one or more criteria other than the first video conference participant speaking, the one or more criteria comprising the first video conference participant visually and/or non-verbally expression a particular emotion ([0074]-[0076], candidate layouts are generated based on the detected activity (speaking, stands up, raises a hand, clapping, laughing, frowning, etc.). the system can rank the candidate layouts and display the candidate layouts based on the ranking. One of the parameters can applied based on importance of individual participants).
Faulkner does not disclose, however Hapur discloses deprioritizing third video of a third video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant).
It would have been obvious to one of ordinary skill in the art at the time of the invention to combine the participant activity-based conference layout techniques of Faulkner with the sentiment-analysis and participant influence analytics of Harpur in order to improve participant prioritization/deprioritization and conference relevance during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings of Faulkner’s conferencing layout-selection system with Harpur’s sentiment-analysis techniques in order to improve the accuracy, contextual relevance, and effectiveness of participant prioritization/deprioritization during conferences and predictably enabled the conferencing system to more intelligently prioritize participant video feeds based no only on detected participant activity, by also on emotional expressions and participant influence information derived from sentiment analysis, thereby improving conferencing presentation and reducing less relevant participant emphasis.
Faulkner and Harpur does not disclose, however Bastide discloses video conference participant based on another person besides the third video conference participant coming into view in the third video (col. 2, lines 36-46, In audiovisual conference calls, distracting elements include other people in the background).
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Bastide’s distraction-marker analysis in order to improve participant prioritization and deprioritization during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Re Claim 17, one of ordinary level of skill in the art would have been compelled to make the proposed modification to Faulkner for the same reasons identified in the rejection of claim 16. In addition, Harpur discloses wherein prioritizing presentation of the first video over the second video comprises one or more of: presenting the first video higher up on a graphical user interface (GUI) than the second video, highlighting the first video on the GUI, the highlighting comprising one or more of: presenting a first color along a border of the first video, presenting a graphic overlay in relation to the first video (Fig. 6F, col. 13, lines 25-40, Positively influencing user J is presented in an enlarged video feed while identified negatively influencing user R is depicted in a relatively smaller video area 604).
Re Claim 19, one of ordinary level of skill in the art would have been compelled to make the proposed modification to Faulkner for the same reasons identified in the rejection of claim 16. In addition, Harpur discloses wherein the third video conference participant is different from the second video conference participant (Fig. 6F, col. 13, lines 25-40, Multiple conference participants H, N, R, T, J, A).
Re claim 20, Faulkner discloses an apparatus, comprising:
at least one computer readable storage medium (CRSM) that is not a transitory signal, the at least one CRSM comprising instructions executable by a processor system to ([0048]-[0049], non-transitory computer storage media):
facilitate a video conference ([0070], [0074], selecting video feed interacting with multiple individuals to configure video feeds to generate candidate layouts);
during facilitation of the video conference, prioritize presentation of first video of a first video conference based on one or more criteria other than the first video conference participant currently speaking ([0074]-[0076], candidate layouts are generated based on the detected activity (speaking, stands up, raises a hand, clapping, laughing, frowning, etc.). the system can rank the candidate layouts and display the candidate layouts based on the ranking. One of the parameters can applied based on importance of individual participants).
Faulkner does not disclose, however Hapur discloses deprioritizing second video of a second video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant).
It would have been obvious to one of ordinary skill in the art at the time of the invention to combine the participant activity-based conference layout techniques of Faulkner with the sentiment-analysis and participant influence analytics of Harpur in order to improve participant prioritization/deprioritization and conference relevance during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings of Faulkner’s conferencing layout-selection system with Harpur’s sentiment-analysis techniques in order to improve the accuracy, contextual relevance, and effectiveness of participant prioritization/deprioritization during conferences and predictably enabled the conferencing system to more intelligently prioritize participant video feeds based no only on detected participant activity, by also on emotional expressions and participant influence information derived from sentiment analysis, thereby improving conferencing presentation and reducing less relevant participant emphasis.
Faulkner and Harpur does not disclose, however Bastide discloses video conference participant based on one or more of: the second video conference participant being classified as distracted (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant), the second video conference participant not being currently shown in the second video, the second video conference participant being identified as looking in a direction away from a camera that generates the second video (col. 6, line 63- col. 7, line 5, the meeting distraction management program monitors the conference call for likely distraction markers which include people not looking at the camera), another person besides the second video conference participant coming into view in the second video (col. 2, lines 36-46, In audiovisual conference calls, distracting elements include other people in the background), an animal coming into view in the second video, the second video conference participant being different from the animal (col. 2, lines 36-46, In audiovisual conference calls, distracting elements include other animals in the background).
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Bastide’s distraction-marker analysis in order to improve participant prioritization and deprioritization during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Re Claims 6 and 21, Hapur discloses deprioritize third video of a third video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant). Faulkner and Harpur does not disclose, however Bastide discloses the video conference participant classified as distracted (col. 6, line 63- col. 7, line 5, the meeting distraction management program monitors the conference call for likely distraction markers.
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Bastide’s distraction-marker analysis in order to improve participant prioritization and deprioritization during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Re Claims 8 and 23, Hapur discloses deprioritize third video of a third video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant). Faulkner and Harpur does not disclose, however Bastide discloses video conference participant being identified as looking in a direction away from a camera that generates the third video (col. 6, line 63- col. 7, line 5, the meeting distraction management program monitors the conference call for likely distraction markers which include people not looking at the camera).
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Bastide’s distraction-marker analysis in order to improve participant prioritization and deprioritization during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Re Claims 9 and 24, Hapur discloses deprioritize third video of a third video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant). Faulkner and Harpur does not disclose, however Bastide discloses video conference participant based on another person besides the third video conference participant coming into view in the third video (col. 2, lines 36-46, In audiovisual conference calls, distracting elements include other people in the background).
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Bastide’s distraction-marker analysis in order to improve participant prioritization and deprioritization during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Re Claim 25, Hapur discloses deprioritize third video of a third video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant). Faulkner and Harpur does not disclose, however Bastide discloses video conference participant based on the animal coming into view in the second video, the second video conference participant being different from the animal (col. 2, lines 36-46, In audiovisual conference calls, distracting elements include other animals in the background).
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Bastide’s distraction-marker analysis in order to improve participant prioritization and deprioritization during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Claim(s) 7 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Faulkner and in view of Harpur and in view of Magi (US 2021/0218845).
Re Claims 7 and 22, Hapur discloses deprioritize third video of a third video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant). Faulkner and Harpur does not disclose conference participant not being currently shown in the third video.
In the same field of endeavor Magi discloses in [0060], [0239], Analyzes there is a distracting event whether the user is standing up or walking away from their laptop.
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Magi’s audiovisual analyzer in order to improve participant prioritization and deprioritization during video conferencing. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Faulkner and in view of Harpur and in view of Lum (US 2023/0036861).
Re Claim 13, Faulkner and Hapur does not disclose, however Lum discloses wherein the one or more criteria comprise the first video and third video of a third video conference participant both showing a same object and/or showing respective objects of a same object type ([0063], participant video feeds may automatically be added to the automatic spotlight queue based on object detection. A relevance score for each participant video is determined based on the correlation of an object in a participant video with the object in the host video feed.)
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Faulkner’s sentiment-analysis and Harpur’s participant influence analytics with Lum’s automatic spotlight queue with spotlight correlation. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Faulkner and in view of Harpur and in view of Bastide and in view of Magi.
Re Claim 26, Hapur discloses deprioritize third video of a third video conference participant (col. 13, lines 25-40, deprioritize video feed area depicting users identified as negatively influencing users who negatively influence the certain participant). Bastide discloses in col. 2, lines 36-46, In audiovisual conference calls, distracting elements include other animals in the background. However, Faulkner, Harpur, and Bastide does not disclose wherein the animal is a dog.
In the same field of endeavor Magi discloses in [0039] a distraction based on the sound of a dog barking.
It would have been obvious for one of ordinary skill in the art before the date the current invention was effectively filed to have modified the teachings of Bastide’s distraction which includes an animal in the background of the video with Magi’s dog as the animal. One of ordinary skill in the art would have been motivated to incorporate the teachings with one another in order to improve the contextual relevance, accuracy, and effectiveness of participant video presentation during conferencing.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HO T SHIU whose telephone number is (571)270-3810. The examiner can normally be reached Mon-Fri (9:00am - 5:00pm).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nicholas Taylor can be reached at 571-272-3089. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HO T SHIU/Examiner, Art Unit 2443
HO T. SHIU
Examiner
Art Unit 2443
/CHRISTOPHER B ROBINSON/Primary Examiner, Art Unit 2443
Prosecution Insights