Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This non-final office action is in response to the Application filed on 09/20/2023.
Claim(s) 1-20 are pending for examination. Claim(s) 1, 14, 18 is/are independent claim(s).
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required:
Claim 12 recites:
… detecting that the online communication session is ending; and automatically saving the visualization information to a persistent datastore accessible to the participants of the online communication session after the online communication session has ended.
The specification recites:
[0044] The trigger monitoring unit 206 can listen for layout change triggers … When the topic of conversation in the online communication session changes, the current visualization can be saved to the meeting information datastore 196, and a new visualization instantiated for the new topic of conversation.
The Specification does not mention the claimed phrase. Thus, there is no support or antecedent basis for the recited term that allows the meaning of the term to be ascertained, as required in 37 CFR 1.75(d)(1).
The applicant may amend the specification to include the terms and provide antecedent basis without introducing new matter into the specification, or the terms may be removed from the claims.
NOTE: Because the original claims are part of the original specification and the term was in the original claims, the specification may be amended to include the term without introducing new matter into the specification.
Claim Objections
Claims 1, 2 objected to because of the following informalities:
Claim 1 recites “the meeting category”, but the claim fails to recite “a meeting category” before.
Claim 2 recites “a meeting category”, but claim 1 already recites “the meeting category”, therefor it is unclear if this is intended to refer to the same “meeting category” in claim 1 or a different one.
Claim 2 recites “a transcript”, but claim 1 already recites “a transcript”, therefor it is unclear if this is intended to refer to the same “transcript” in claim 1 or a different one.
Claim 16 recites “the user prompts”, however this lacks antecedent basis in the claims.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8, 13, 14-16, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu; Chenguang et al. US Pub. No. 2021/0375289 (Zhu) in view of Bryan; Kathleen Alexandra et al. US Pub. No. 2024/0380800 (Bryan).
Claim 1:
Zhu teaches:
A data processing system comprising:
a processor [¶ 0040-43, 557] (processor, memory, instructions); and
a memory storing executable instructions that, when executed, cause the processor alone or in combination with other processors to perform operations of:
detecting an occurrence of a trigger condition during an online communication session among a plurality of client devices associated with participants of the online communication session [¶ 0070, 77, 80-81] (trigger in online meeting), the occurrence of the trigger condition indicating that a visualization of content associated with the online communication session should be generated and presented on the client devices associated with the participants of the online communication session [¶ 0070, 541, 546-547] (AI assistant, templates for display);
selecting a candidate visualization layout from a plurality of visualization layouts in a visualization layout datastore based on the meeting category associated with the online communication session [¶ 0052, 90-96, 515] (template is selected based on a meeting type that is determined from analyzing the transcription and which is automatically populated with content from the transcription, a template could be considered a “visualization layout”); …
Zhu does not appear to explicitly disclose “providing the prompt as an input to a language model”.
However, the disclosure of Bryan teaches:
… constructing a prompt for a language model based on the selected candidate visualization layout and a transcript of the online communication session [¶ 0076] (speech-to-text operation to convert the user's speech into text that is to be displayed in the virtual whiteboard UI element 326);
providing the prompt as an input to a language model [¶ 0029, 116, 123] (input query to generative machine learning model, a query could be a “prompt”);
obtaining visualization information as an output from the language model, wherein the visualization information is formatted according to the selected candidate layout and includes information based on the transcript, the additional information associated with the online communication session, or both [¶ 95-96, 121-124] (content of one or more virtual whiteboard UI elements can be used as input (e.g., query) to a trained generative machine learning model, and the trained generative machine learning model can generate new content (e.g., summary) of the original content from the one or more virtual whiteboard UI elements);
causing the plurality of client devices associated with the participants of the online communication session to present an interactive virtual whiteboard populated with the visualization information on a user interface of the plurality of client devices [¶ 0096-97] (whiteboard with multiple users) [¶ 0025-30, 38-39, 49-95] (Figs. 3A-3F, user using the whiteboard); and
synchronizing the changes to the visualization information among the plurality of client devices during the online communication session [¶ 0084] (livestream of whiteboard updated with participant’s contributing content).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of meeting transcription in Zhu and the method of whiteboard collaboration in Bryan, with a reasonable expectation of success.
The motivation for doing so would have been the use of known technique to improve similar devices (methods, or products) in the same way; (See KSR Int’l Co. v. Teleflex Inc., 550 US 398, 82 USPQ2d 1385, 1396 (U.S. 2007) and MPEP § 2143(D)).
The know technique of using a generative LLM in Bryan could be applied to the collaboration meeting in Zhu. Zhu and Bryan are similar devices because each relate to online meetings and collaboration. One of ordinary skill in the art would have recognized that applying the known technique would improve the similar devices and resulted in an improved system, with a reasonable expectation of success, to “increase of overall efficiency and functionality of the video conference platform” [Bryan ¶ 0002, 30].
Claim 2:
Zhu teaches:
The data processing system of claim 1, wherein the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
analyzing information associated with the online communication session to determine a meeting category associated with the online communication session, wherein analyzing the information associated with the online communication session includes generating a transcript of an audio portion of the online communication session using a transcription model, and analyzing the transcript and additional information associated with the online communication session as an input to a categorization model that outputs the meeting category associated with the online communication session [¶ 0052, 90-96, 515] (template is selected based on a meeting type that is determined from analyzing the transcription and which is automatically populated with content from the transcription, a template could be considered a “visualization layout”).
Claim 3:
Zhu teaches:
The data processing system of claim 1, wherein selecting a candidate visualization layout further comprises:
selecting a set of candidate visualization layouts from a plurality of visualization layouts in a visualization layout datastore based on the meeting category associated with the online communication session;
causing a client device of a participant to the online communication session to present the set of candidate visualization layouts on a user interface of the client device;
receiving an indication from the client device indicating a selected candidate layout from among the plurality of candidate visualization layouts [¶ 0052, 90-96, 515] (template is selected based on a meeting type that is determined from analyzing the transcription and which is automatically populated with content from the transcription, a template could be considered a “visualization layout”)..
Claim 4:
Bryan teaches:
The data processing system of claim 1, wherein the user interface provides controls that enable users of the plurality of client devices to make changes to the visualization information [¶ 0008, 108] (detects changes, object movement).
Claim 5:
Bryan teaches:
The data processing system of claim 1, wherein detecting the occurrence of the trigger condition further comprises:
detecting an indication received from a control of the user interface of a communications application on the client device of a participant of the online communication session indicating that the participant has requested that the visualization be created for the online communication session [¶ 0123] (user query analyzer 503 may receive a user input, e.g., user query, and generate one or more intermediate queries to generative model 520 to determine what type of user data GM 520 might need to successfully respond to user input).
Claim 6:
Zhu teaches:
The data processing system of claim 1, wherein detecting the occurrence of the trigger condition further comprises:
analyzing natural language prompts using the language model to detect a use of one or more trigger words, phrases, or questions that indicate that the visualization should be created for the online communication session, the natural language prompts being submitted through a user interface of a communication application on the client device of a participant of the online communication session; and
receiving an output from the language model indicating the occurrence of the trigger condition responsive to detecting the use of the one or more trigger words, phrases, or questions in the user prompts [¶ 0070, 77, 80-81] (trigger in online meeting, keyword trigger).
Claim 8:
Zhu teaches:
The data processing system of claim 1, wherein the additional information associated with the online communication session includes messages from a meeting chat associated with the online communication session, whether screen sharing of content is currently occurring, attendee information for the participants to the online communication session, or a combination thereof [¶ 0094] (chat logs).
Bryan also teaches: [¶ 0034] (video chat).
Claim 13:
Bryan teaches:
The data processing system of claim 1, wherein the language model comprises a Large Language Model (LLM) [¶ 0095-96] (generative machine learning models are LLMs).
Claims 14-16, 18-20:
Claim(s) 14, 18 is/are substantially similar to claim 1 and is/are rejected using the same art and the same rationale.
Claim 14 is a “method” claim, claim 1 is a “system” claim, but the steps or elements of each claim are essentially the same.
Claim 14 also recites the “analyzing …” step of claim 2, but lacks the prompt of claim 1. Claim 14 uses a “transcript” and generates a “visualization”
Claim(s) 15 is/are substantially similar to claim 5 and is/are rejected using the same art and the same rationale.
Claim(s) 16 is/are substantially similar to claim 6 and is/are rejected using the same art and the same rationale.
Claim(s) 17 is/are substantially similar to claim 7 and is/are rejected using the same art and the same rationale.
Claim(s) 18-20 is/are substantially similar to claims 1-2 and is/are rejected using the same art and the same rationale.
Claim 18 lacks the “detecting an occurrence of a trigger condition …” of claim 1 as well as the layout “based on the meeting category”. Claim 18 does not use the term “trigger condition” but uses the phrase “receiving an indication to generate a visualization”.
Claim(s) 19 is/are substantially similar to claim 1 and is/are rejected using the same art and the same rationale. Claim 19 adds more of the elements of claim 1 to make is similar to claim 1.
Claim(s) 20 is/are substantially similar to and a combination of claims 1 and 3 and is/are rejected using the same art and the same rationale.
Claim(s) 7, 9, 11, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu; Chenguang et al. US Pub. No. 2021/0375289 (Zhu) in view of Bryan; Kathleen Alexandra et al. US Pub. No. 2024/0380800 (Bryan) in view of Reynolds; Spencer Thomas et al. US Pat. No. 12,141,903 (Reynolds).
Claim 7:
Zhu teaches: [¶ 0015-17] (transcription).
Bryan teaches:
The data processing system of claim 1, wherein detecting the occurrence of the trigger condition further comprises:
obtaining the transcript and the additional information associated with the online communication session [¶ 0076] (speech-to-text operation to convert the user's speech into text that is to be displayed in the virtual whiteboard UI element 326); …
Zhu and Bryan do not appear to explicitly disclose “obtain a prediction”.
However, the disclosure of Reynolds teaches:
… analyzing the transcript and additional information associated with the online communication session using the language model to obtain a prediction whether the transcript and the additional information indicate that a visualization recommendation should be made to a participant of the online communication session [Col. 12, Ln. 1-68] (real time transcript, visual display template presented to each of the plurality of participants within the video conference user interface may be adjusted according to the relevance score being generated, the topic being discussed) [Col. 15, Ln. 20-35] (adjust the visual display template for each of the participants based on the relevance score generated); and
detecting the occurrence of the trigger condition responsive to the language model outputting the prediction that the visualization recommendation should be made [Col. 14, Ln. 27-68] (video conference optimization module 150 may adjust the visual display template for the scheduled web conference in real time as the participants transition through the topics of the agenda) [Col. 12, Ln. 1-68] (real time transcript, visual display template presented to each of the plurality of participants within the video conference user interface may be adjusted according to the relevance score being generated, the topic being discussed).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of meeting transcription in Zhu and the method of whiteboard collaboration in Bryan and the method of video conference optimization in Reynolds, with a reasonable expectation of success.
The motivation for doing so would have been the use of known technique to improve similar devices (methods, or products) in the same way; (See KSR Int’l Co. v. Teleflex Inc., 550 US 398, 82 USPQ2d 1385, 1396 (U.S. 2007) and MPEP § 2143(D)).
The know technique of adjust the visual display template in Reynolds could be applied to the collaboration meeting in Zhu and the generative LLM in Bryan. Zhu, Bryan and are similar devices because each relate to online meetings and collaboration. One of ordinary skill in the art would have recognized that applying the known technique would improve the similar devices and resulted in an improved system, with a reasonable expectation of success, “for an improved video conference experience” [Reynolds: Col. 3, Ln. 18-30].
Claim 9:
Reynolds teaches:
The data processing system of claim 1, wherein selecting the set of candidate visualization layouts from the plurality of visualization layouts further comprises:
ranking candidate visualization layouts selected for the meeting category based on a frequency with which the candidate visualization layouts are selected for the meeting category[Col. 15, Ln. 20-35] (adjust the visual display template for each of the participants based on the relevance score generated); and
selecting highest ranking candidate visualization layouts to include in the set of candidate visualization layouts [Col. 15, Ln. 20-35] (adjust the visual display template for each of the participants based on the relevance score generated) [Col. 14, Ln. 27-68] (video conference optimization module 150 may adjust the visual display template for the scheduled web conference in real time as the participants transition through the topics of the agenda) [Col. 12, Ln. 1-68] (real time transcript, visual display template presented to each of the plurality of participants within the video conference user interface may be adjusted according to the relevance score being generated, the topic being discussed).
Claim 11:
Reynolds teaches:
The data processing system of claim 1, wherein the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
detecting the occurrence of a second trigger condition during the online communication session, the occurrence of the trigger condition indicating that a change in topic has occurred by analyzing the transcript and messages from a meeting chat associated with the online communication session using the language model [Col. 15, Ln. 20-35] (adjust the visual display template for each of the participants based on the relevance score generated);
responsive to detecting the occurrence of the second trigger condition, selecting a set of second candidate visualization layouts from the plurality of visualization layouts in the visualization layout datastore based on the meeting category associated with the online communication session and a current topic; and
causing the client device of the participant to the online communication session to present the set of second candidate visualization layouts on a user interface of the client device [Col. 14, Ln. 27-68] (video conference optimization module 150 may adjust the visual display template for the scheduled web conference in real time as the participants transition through the topics of the agenda) [Col. 12, Ln. 1-68] (real time transcript, visual display template presented to each of the plurality of participants within the video conference user interface may be adjusted according to the relevance score being generated, the topic being discussed).
Claim 17:
Claim(s) 17 is/are substantially similar to claim 7 and is/are rejected using the same art and the same rationale.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu; Chenguang et al. US Pub. No. 2021/0375289 (Zhu) in view of Bryan; Kathleen Alexandra et al. US Pub. No. 2024/0380800 (Bryan) in view of Faulkner; Jason Thomas et al. US Pub. No. 2020/0371677 (Faulkner).
Claim 10:
Zhu and Bryan teach all the elements as shown above.
Zhu and Bryan do not appear to explicitly disclose “candidate visualization layout pane”.
However, the disclosure of Faulkner teaches:
The data processing system of claim 1, wherein causing a client device of a participant to the online communication session to present the candidate visualization layouts further comprises:
causing the client device of the participant to display a candidate visualization layout pane that displays the set of candidate visualization layouts, the candidate visualization pane including a preview of each visualization layout of the set of candidate visualization layouts [¶ 0015, 89, 92, 101-103] (view template, type of content to be displayed, preview; previews of available view templates).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of meeting transcription in Zhu and the method of whiteboard collaboration in Bryan and the method of template previews in Faulkner, with a reasonable expectation of success.
The motivation for doing so would have been the use of known technique to improve similar devices (methods, or products) in the same way; (See KSR Int’l Co. v. Teleflex Inc., 550 US 398, 82 USPQ2d 1385, 1396 (U.S. 2007) and MPEP § 2143(D)).
The know technique of previews in Faulkner could be applied to the collaboration meeting in Zhu and the generative LLM in Bryan. Faulkner, Zhu, Bryan are similar devices because each relate to online meetings and collaboration. One of ordinary skill in the art would have recognized that applying the known technique would improve the similar devices and resulted in an improved system, with a reasonable expectation of success, to “improve user engagement” [Faulkner: abstract].
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu; Chenguang et al. US Pub. No. 2021/0375289 (Zhu) in view of Bryan; Kathleen Alexandra et al. US Pub. No. 2024/0380800 (Bryan) in view of Crumley; Ryan et al. US Pub. No. 2023/0208895 (Crumley).
Claim 12:
Zhu and Bryan teach all the elements as shown above.
Zhu and Bryan do not appear to explicitly disclose “automatically saving”.
However, the disclosure of Crumley teaches:
The data processing system of claim 1, wherein the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
detecting that the online communication session is ending; and
automatically saving the visualization information to a persistent datastore accessible to the participants of the online communication session after the online communication session has ended [¶ 0046] (shared dynamic collaborative presentation progression interface can be automatically stored/saved and synced between all users in real-time, next topic button).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the method of meeting transcription in Zhu and the method of whiteboard collaboration in Bryan and the method of saving in Crumley, with a reasonable expectation of success.
The motivation for doing so would have been the use of known technique to improve similar devices (methods, or products) in the same way; (See KSR Int’l Co. v. Teleflex Inc., 550 US 398, 82 USPQ2d 1385, 1396 (U.S. 2007) and MPEP § 2143(D)).
The know technique of automatic saving in Crumley a could be applied to the collaboration meeting in Zhu and the generative LLM in Bryan. Zhu, Bryan and are similar devices because each relate to online meetings and collaboration. One of ordinary skill in the art would have recognized that applying the known technique would improve the similar devices and resulted in an improved system, with a reasonable expectation of success, improved efficiency [Crumley: ¶ 0035-36].
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Please See PTO-892: Notice of References Cited.
Evidence of the level skill of an ordinary person in the art for Claim 1:
Digital assistant for meetings
Mahmoud; Mohamed Gamal Mohamed et al. US 20210021558 virtual assistant, meeting, visualization.
Video Layout/template based on context/type of meeting
Hartung; Frank et al. US 20140218464 layout for the UI may be selected which is adequate with respect to an underlying conversation scenario of the communication session; type of the communication session is determined automatically on the basis of an analysis of audio data of the communication session.
Lee; Miyoung et al. US 20230164294
Adapt whiteboard to meeting
MARTIN; DAVID et al. US 20110087973 A1
Mehmeri; Victor Dantas US 20250028579
Bailey; Alexander US 20240205174
Evidence of the level skill of an ordinary person in the art for Claim 7:
Beaver; Ian et al. US Pub. No. 2022/0028138 (Beaver) [¶ 0030, 46, 51, 64] (visualization 198 may be updated based on the topic(s) that are identified during this continuous (or periodic, or random) monitoring).
Nakajima; Kaisuke et al. US 20240354493
Citations to Prior Art
A reference to specific paragraphs, columns, pages, or figures in a cited prior art reference is not limited to preferred embodiments or any specific examples. It is well settled that a prior art reference, in its entirety, must be considered for all that it expressly teaches and fairly suggests to one having ordinary skill in the art. Stated differently, a prior art disclosure reading on a limitation of Applicant's claim cannot be ignored on the ground that other embodiments disclosed were instead cited. Therefore, the Examiner's citation to a specific portion of a single prior art reference is not intended to exclusively dictate, but rather, to demonstrate an exemplary disclosure commensurate with the specific limitations being addressed. In re Heck, 699 F.2d 1331, 1332-33,216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968". In re: Upsher-Smith Labs. v. Pamlab, LLC, 412 F.3d 1319, 1323,75 USPQ2d 1213,1215 (Fed. Cir. 2005); In re Fritch, 972 F.2d 1260, 1264,23 USPQ2d 1780, 1782 (Fed. Cir. 1992); Merck & Co. v. Biocraft Labs., Inc., 874 F.2d 804, 807,10 USPQ2d 1843, 1846 (Fed. Cir. 1989); In re Fracalossi, 681 F.2d 792,794 n.1, 215 USPQ 569, 570 n.1 (CCPA 1982); In re Lamberti, 545 F.2d 747, 750, 192 USPQ 278, 280 (CCPA 1976); In re Bozek, 416 F.2d 1385,1390,163 USPQ 545, 549 (CCPA 1969).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN J SMITH whose telephone number is (571)270-3825. The examiner can normally be reached Monday - Friday 11:00 - 7:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ADAM QUELER can be reached at (571) 272-4140. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Benjamin Smith/Primary Examiner, Art Unit 2172 Direct Phone: 571-270-3825
Direct Fax: 571-270-4825
Email: benjamin.smith@uspto.gov