DETAILED ACTION
This communication is in response to the Application filed on 05/30/2024. Claims 1-20 are pending and have been examined. Claims 1, 10 and 15 are independent. This Application was published as U.S. Pub. No. 2024/0403551A1
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/30/2024 was filed. Accordingly, the information disclosure statement is being considered by the examiner.
Priority
Applicant’s claims for benefit of a provisional application 63/505,403 submitted on 05/31/2023 is acknowledged.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Daredia et al., (US Pub No. 2025/0070994, hereinafter, Daredia) in view of Leaman et al., (US Pub No. 2025/0217781, hereinafter, Leaman).
Regarding Claim 1,
Daredia discloses a computer-implemented method, comprising (Daredia, Abstract, "…a digital transcription system, a digital transcription model..."):
accessing, by a computer system associated with a content management platform, a first content item from a content repository maintained by the content management platform (Daredia, Fig.15, paras [175-179], "…the server device 701 includes a content management system 102 that hosts the digital transcription system 704..."; par [311], "…The meeting context manager 1510 can store and retrieve meeting context data 1534 from a database maintained by the storage manager 1532...");
retrieving, by the computer system, use data associated with the first content item that characterizes user activity with respect to the first content item (Daredia, Figs.1-2, par [058], "…the content management system 102 also receives input data for one or more of the client devices 106a-106n...in response to a keyboard input...or sensor input..."; par [077], "…The content management system 102 also correlates identified portions of audio data, meeting materials, or other meeting data with device input data (e.g., timestamp) from one or more client devices associated with the meeting...");
upon receiving a request to generate a second content item based on the first content item, sending, to the LLM, a prompt that includes the description for the first content item and that instructs the LLM to use the description to generate the second content item (Daredia, paras [074-080], "…the content management system 102 can analyze text or metadata of content items generated and synchronized with the content management system 102 to determine text content relative to audio data for the meeting...the content management system 102 can generate a summary document for the user including the identified content, or otherwise describing the identified content, of the relevant portion...").
Daredia discloses the metadata and digital lexicon of the content item, but failed to explicitly disclose the limitation regarding the generation of the description of the first content item as metadata.
However, Leaman, in the analogous field of endeavor, discloses sending at least a portion of the first content item to a large language model (LLM) to cause the LLM to generate a description for the first content item (Leaman, Fig. 67, par [073], "…the multi-service business platform 510...the conversation intelligence (CI) system 528. The CI system 528 may include a preprocessing layer 4002...a transcription layer 4004...a postprocessing layer 4006...various data in the storage system 550..."; paras [207-211], "…artificial intelligence may include one or more models used for one or more problem domains...a transformer model include bidirectional encoder representations from transformers (BERT) and generative pretrained transformer (GPT)...Problem domains includes natural language processing (NLP)…(i.e., artificial intelligence systems (BERT, GPT) with NLP is construed as LLM)");
wherein sending at least the portion of the first content item to the LLM comprises sending the use data associated with the first content item to cause the LLM to generate the description for the first content item based in part on the use data storing, by the computer system, the description for the first content item as metadata associated with the first content item in the content repository (Leaman, par [120-126], "…The CI system 528 may take this text and feed it through a set of models to get metadata such as sentiment, topics, keywords ( e.g., keyword prediction), and determination/prediction of other information...This metadata may then be used by the reporting service ( e.g., reporting system 526 or reporting 534 of the services 530) to generate reports ( e.g., cross-object custom reports)...").
Therefore, it would have been obvious to one of ordinary skill in the art, before effective filing date of the claimed invention, to have modified a digital transcription and content management system pre-processing, transcribing, and post-processing functionalities of natural language processing transformer (BERT, GPT) based conversation intelligence system of Daredia with of Leaman with a reasonable expectation of success to enable keyword/topic extraction, feature extraction, event generation, trigger action, and search indexing. (Leaman, Abstract, paras [209-211]).
Regarding Claim 2,
Daredin in view of Leaman discloses the computer-implemented method of claim 1, further comprising:
Daredin further discloses selecting the portion of the first content item to send to the LLM based on the use data (Daredin, paras [010, 066-073], "…The disclosed system analyzes the data, user inputs, and other information to determine portions of the gathered media data that are relevant to a specific user or to all meeting participants ( e.g., based on timestamps of the user inputs and corresponding times in the media data)…").
Regarding Claim 3,
Daredin in view of Leaman discloses the computer-implemented method of claim 2, wherein the use data includes dwell time by a plurality of users in different sections of the first content item (Daredin, par [195], "…the portable client device 908 can provide, to the digital transcription system 704, metadata (e.g., timestamps) regarding the timing of each note with respect the meeting..."; Leaman, par [076], "…the conversation record ( or an associated record, such as an event record) may store, for each extracted keyword, metadata relating to the keyword, such as how many times the keyword was mentioned, time stamps/ locations in the transcript when the keyword was mentioned, who mentioned the keyword, and/or the like..." ), and
Daredin further discloses wherein selecting the portion of the first content item based on the use data comprises: determining, based on the use data, a first section of the first content item for which an average of the dwell time of the plurality of users in the first section is higher than an average dwell time of the plurality of users in a second section of the first content item (par [121], "…the content management system 102 can determine whether a threshold percentage of users is taking notes at any given time for identifying relevant portions of a meeting...which may help verify the importance of the portion of a meeting..."); and selecting the first section of the first content item as the portion to send to the LLM (par [077], "…the content management system 102 can mark the portion of the audio data, document, or other content item corresponding to the device input as relevant based on a timestamp of the device input and a corresponding time stamp of the relevant portion of the audio data or other materials...").
Regarding Claim 4,
Daredin in view of Leaman discloses the computer-implemented method of claim 1.
Daredin further discloses wherein sending at least the portion of the first content item to the LLM comprises:
applying a reduction method to the first content item to divide the first content item into multiple sections (Daredin, Figs. 4, paras [118-127], "…the content management system 102 can identify relevant portions generally or per individual...");
sending each of the multiple sections to the LLM to instruct the LLM to generate a respective summary of each section (par [123], "…The content management system 102 can then identify the text corresponding to the relevant portions and paste the text or summarize/rephrase the text into the meeting summary 430 as highlights 434. The highlights 434 can include bullet points or other easily digestible representations of the discussion points from the meeting…"); and
refining the summaries of each of the multiple sections to generate the description for the first content item (par [123], "…determining the text to include in the highlights 434 includes reviewing the text corresponding to the relevant portions to determine how much of the text is included in the relevant portion...").
Regarding Claim 5,
Daredin in view of Leaman discloses the computer-implemented method of claim 1. Daredin further discloses wherein sending the LLM the prompt instructing the LLM to generate the second content item comprises: instructing the LLM to generate an electronic message based on the first content item, wherein the electronic message includes a summary of the first content item that is generated by the LLM based on the description of the first content item (Daredia, paras [074-080], "…the content management system 102 can analyze text or metadata of content items generated and synchronized with the content management system 102 to determine text content relative to audio data for the meeting...the content management system 102 can generate a summary document for the user including the identified content, or otherwise describing the identified content, of the relevant portion...").
Regarding Claim 6,
Daredin in view of Leaman discloses the computer-implemented method of claim 1. Daredin further discloses wherein the first content item is a first type of content item (Daredin, par [051], "…"content item" refers to digital content that includes text or audiovisual data..."), and wherein sending the LLM the prompt instructing the LLM to generate the second content item comprises: instructing the LLM to generate, as the second content item, a second type of content item to summarize the first content item, based on the description of the first content item (Daredin, par [056], "…"meeting insights" refers to content generated by the content management system 102 based on an analysis of data related to a meeting...Meeting insights can include a meeting summary, highlights from the meeting (e.g., portions of the meeting marked by users as important)..."; par [057], "…The content management system 102 can use natural language processing or other audio analysis to analyze audio data (i.e. first content item)..."; Fig.4E, par [077], "…The content management system 102 can then extract text, images, audio, or video from a portion of meeting data marked as relevant to use in generating meeting insights (e.g., summary/highlight in Fig.4E as text)").
Regarding Claim 7,
Daredin in view of Leaman discloses the computer-implemented method of claim 1. Daredin further discloses wherein the first content item is accessed based on detecting the first content item is a new content item in the content repository (par [186], "…the digital transcription system 704 accesses digital documents stored on a content management system associated with the user...the digital transcription system 704 can also access a collaboration graph to determine where to obtain additional data relevant to the meeting..."; par [233], "…the digital transcription system 704 can update the digital lexicon 1022 offline as new meeting context data is provided to the content management system...").
Regarding Claim 8,
Daredin in view of Leaman discloses the computer-implemented method of claim 1, further comprising: Daredin further discloses retrieving, by the computer system, use data associated with the first content item that characterizes user activity with respect to the first content item (Daredin, paras [077, 093, 125], "…device input data from one or more client devices associated with the meeting...");
Daredin further discloses wherein the first content item is accessed based on detecting the use data satisfies a criterion for generating a new description for the first content item (par [121], "…the content management system 102 can determine whether a threshold percentage of users is taking notes at any given time for identifying relevant portions of a meeting...which may help verify the importance of the portion of a meeting...").
Regarding Claim 9,
Daredin in view of Leaman discloses the computer-implemented method of claim 1. Leaman further discloses wherein accessing the first content item comprises accessing a plurality of content items that includes the first content item (Leaman, par [118], "…The CI system 528 may receive a series of video feeds that have audio associated with them. Each feed may be associated with a different speaker…"), and wherein generating the description for the first content item comprises generating a description for the plurality of content items (Leaman, par [118], "…The CI system 528 may receive from the transcription service a transcript that may have time stamps on when specific utterances were made..."; par [120], "…The CI system 528 may take this text and feed it through a set of models to get metadata such as sentiment, topics, keywords ( e.g., keyword prediction), and determination/prediction of other information...").
Claim 10 is a non-transitory computer-readable storage medium claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale.
Additionally, Daredin discloses a non-transitory computer-readable storage medium storing executable computer program instructions, the computer program instructions when executed by one or more processors of a system causing the system to (Daredin, Figs.6, par [141], "…flowchart of a series of acts 600 of generating meeting insights from media data and device input data...a non-transitory computer readable medium can comprise instructions, that when executed by one or more processors, cause a computing device to perform the acts of FIG. 6..."):
Leaman discloses receive a search query directed to the content repository (Leaman, Figs.73-74, par [107], "…graphical user interface (GUis) 7300, 7400 for displaying global searching capabilities (e.g., may use the search service 6734). The CI system 528 may allow for searching across every conversation ..."; par [074], "…the conversation object 5000 defines a set of properties of a recorded conversation ( e.g., a video call, an audio call, a chat transcript, or the like). In this way, any conversations that are processed by the CI system 528 may be represented in a client's datastores (e.g., knowledge base instance of the instances knowledge graph 640 and/or events data store(s)..."); and
identify the first content item as a search result based at least in part on match between the search query and the description for the first content item (Leaman, Fig.16, par [014], "…the search indexing may include the postprocessing layer indexing the transcript such that the conversation of the transcript may be searchable via a search service (6734 in Fig.63)"; par [107], "…this searching may be a unified search that may combine keywords with the search feature ( e.g., unified search) providing interesting and related features...").
...
Rationale for combination is similar to that provided for Claim 1.
Claim 11 is a non-transitory computer-readable storage medium claim with limitations similar to the limitations of Claim 2 and is rejected under similar rationale.
Claim 12 is a non-transitory computer-readable storage medium claim with limitations similar to the limitations of Claim 3 and is rejected under similar rationale.
Claim 13 is a non-transitory computer-readable storage medium claim with limitations similar to the limitations of Claims 1 and 10 and is rejected under similar rationale.
Claim 14 is a non-transitory computer-readable storage medium claim with limitations similar to the limitations of Claim 10 and is rejected under similar rationale.
Additionally, Daredin further discloses determine audience information that indicates an audience for the first content item (Daredin, par [158], "…the digital transcription system can utilize meeting context data corresponding to a meeting participant (e.g., a user). Meeting context data for a user can include user digital documents maintained by a content management system. For example, meeting context data can include user features, such as a user's name, profile, job title, job position, workgroups, assigned projects, etc..."); and
generate the description that includes the audience information (Fig.11A, par [243], "…the digital transcription generator 1100 within the digital transcription model 706 generates the digital transcript 1004 based on the audio data 1002 of the meeting and the meeting context data 1010 of a meeting participant...");
Claim 15 is a system claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale. Additionally,
Daredin discloses one or more processors; and one or more non-transitory computer-readable storage media storing executable computer program instructions, the computer program instructions when executed by the one or more processors cause the data processing system to (Daredin, Figs.6, par [141], "…flowchart of a series of acts 600 of generating meeting insights from media data and device input data...a non-transitory computer readable medium can comprise instructions, that when executed by one or more processors, cause a computing device to perform the acts of FIG. 6..."):
Daredin further discloses access a first content item from a content repository maintained by a content management platform ( Fig. 15, par [311-312], "…storage manager 1532...meeting context manager 1510…the document manager1512 facilitates the retrieval of digital documents..."); wherein the content management platform maintains a plurality of lists that each include two or more content items related to one another (Fig. 14, par [152], "…digital collaboration graphs (i.e., list) reflecting connections between participants, interests, and organizational structures..."; par [300], "…the digital transcription system 704 can utilize a collaboration graph to locate, gather, analyze, filter, and/or weigh meeting context data of one or more users... the digital transcription system 704 generates, maintains, modifies, stores, and/or implements one or more collaboration graphs in one or more data stores...");
Leaman further disclose add the first content item to a first list of the plurality of lists based on a determined similarity between the description for the first content item and other content items in the first list (Leaman, par [115-117], "…The CI system 528 may connect the transcript to searching and indexing capabilities...CI system 528 may provide lexical searches... may provide fuzzy matching from conversations, then the CI system 528 may also determine relevant scores and provide the relevant scores so that more important parts of the conversation may be pulled...").
Claim 16 is a system claim with limitations similar to the limitations of Claim 8 and is rejected under similar rationale.
Claim 17 is a system claim with limitations similar to the limitations of Claim 2 and is rejected under similar rationale.
Claim 18 is a system claim with limitations similar to the limitations of Claim 3 and is rejected under similar rationale.
Claim 19 is a system claim with limitations similar to the limitations of Claim 7 and 14 (with reference to "first list") and is rejected under similar rationale.
Claim 20 is a system claim with limitations similar to the limitations of Claim 9 and 14 (with reference to "first list") and is rejected under similar rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Mishra (US Pub No. 2023/0410801) discloses methods or systems to automatically generate scripts, decision branches, training data for generative artificial intelligence from the merged transcript using Large language models (LLM) (Mishra, paras [002-003]).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANGWOEN LEE whose telephone number is (703)756-5597. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BHAVESH MEHTA can be reached at (571)272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANGWOEN LEE/Examiner, Art Unit 2656
/BHAVESH M MEHTA/Supervisory Patent Examiner, Art Unit 2656