DETAILED ACTION
This action is in response to communication filed on 30 April 2024. Claims 1-19 are pending in the application and have been considered below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 6-7, 9-13, 15-17 and 19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by CARBUNE et al. (US20220076668A1).
As to claim 1, CARBUNE teaches a computing system for automatically generating structured content (See figs. 1A-5, par. 0004, wherein implementations set forth herein relate to an automated assistant that can operate as a modality for completing various document-related actions for content-rich documents. A content-rich document can refer to any set of data incorporated into a single document; as taught by CARBUNE);
the computing system comprising: one or more processors (See fig. 5, item 514; as taught by CARBUNE);
and one or more non-transitory computer-readable media that collectively store instructions (See fig. 5, item 524; as taught by CARBUNE)
that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising: providing a user interface to a user computing system (See figs. 2A-2D, par. 0006, wherein the user can provide this spoken utterance to an interface of their watch, which can provide access to the automated assistant but may not include a native document editing application for editing the spreadsheet. In response to receiving the spoken utterance, the automated assistant can process audio data corresponding to the spoken utterance and determine one or more actions to perform; as taught by CARBUNE);
receiving a prompt from the user computing system via the user interface (See fig. 2A, par. 0040, wherein when the user 202 acknowledges the output 204 from the automated assistant, the user 202 can provide a second spoken utterance 214 such as, “Could you consolidate those notes into a spreadsheet and read it to me?” In response to receiving the second spoken utterance 214, the automated assistant can generate one or more functions to be executed by a document application in order to cause the document application to consolidate the identified documents into a single document; as taught by CARBUNE),
the prompt comprising existing content within an integrated development environment (see figs. 1A-2D, par. 0004, wherein implementations set forth herein relate to an automated assistant that can operate as a modality for completing various document-related actions for content-rich documents. A content-rich document can refer to any set of data incorporated into a single document. The set of data can include, but is not limited to, multiple different sections, topics, subtopics, styles, cells in a spreadsheet, slides in a presentation, graphics, and/or any combination of features that can be incorporated into a document; as taught by CARBUNE),
providing the prompt to a generative model, the generative model being a machine-learned model trained to process language input prompts to generate an output (see par. 0007, wherein processing of the audio data can involve utilizing one or more trained machine learning models, heuristic processes, semantic analyses, and/or any other processes that can be employed when processing a spoken utterance from a user. As a result of the processing, the automated assistant can initialize performance of one or more actions specified by the user via the spoken utterance. In some implementations, the automated assistant can use an application programming interface (API) in order to cause a particular document application to perform the one or more actions. For instance, in response to the aforementioned spoken utterance, the automated assistant that is accessible via the watch of the user can generate one or more functions to be executed in response to the spoken utterance from the user; as taught by CARBUNE);
receiving a generative output generated by the generative model in response to the prompt (see par. 0007, wherein in response to the aforementioned spoken utterance, the automated assistant that is accessible via the watch of the user can generate one or more functions to be executed in response to the spoken utterance from the user; as taught by CARBUNE),
the generative output including generative content divided into one or more generative content cells (see par. 0017, wherein the automated assistant can determine that the spoken utterance from the other user included some amount of dictation and select portions of textual data that was transcribed from the spoken utterance to be incorporated into certain cells. Alternatively, or additionally, a format of each cell in the new column can be modified to correspond to “degree” values, in order that the new column will reflect units (e.g., degrees in Celsius) of the data to be added to the new column, and as specified in the spoken utterance. The automated assistant can input each numerical value according to the spoken utterance, at least based on text to speech processing and natural language understanding of the entire spoken utterance; as taught by CARBUNE);
and providing the generative output via the user interface (see fig. 1C, par. 0038, wherein as a result, and in response to the spoken utterance 142, the automated assistant can provide an output 148 such as, “Howard added text to the new paragraph.” In some implementations, the automated assistant can generate semantic annotations for a document as the document is being edited. In this way, subsequent automated assistant inputs related to the document can be more readily fulfilled, while mitigating latency that can occur during document identification. For example, when the first user 102 provides another spoken utterance 150 such as, “Read the conclusion,” the automated assistant can identify a semantic annotation characterizing a portion of the report document as having conclusory language—despite the report document not having the word “conclusion” in the content of the report document. Thereafter, the automated assistant can provide an audible output 152 via the client computing device 146 and/or a visual output at a television computing device 144; as taught by CARBUNE).
As to claim 2, CARBUNE teaches the limitations of claim 1. CARBUNE further teaches wherein the existing content is within a first series of content cells of a grid within the integrated development environment, and wherein the one or more generative content cells comprises a second series of content cells, the generative content within each of the second series of content cells being associated with the existing content within a respective one of the first series of content cells (See fig. 2B, par. 0041, wherein in order for the automated assistant to perform further operations with respect to the consolidated document, the automated assistant can cause semantic annotations to be stored in association with the consolidated document. For example, and as provided in FIG. 2B, the automated assistant can generate a request for semantic annotations to be associated with the consolidated document (e.g., spreadsheet 232). The request can be executed at the vehicle computing device and/or a remote computing device 222, such as a remote server device. In some implementations, one or more techniques for generating a semantic annotation for a particular subsection of a document can be employed. For instance, one or more trained machine learning models and/or one or more heuristic approaches can be utilized in order to generate semantic annotations for the spreadsheet 232. Data that is processed in order to generate a particular semantic annotation can include: content of the spreadsheet 232, interaction data characterizing interactions between the user 202 and the automated assistant, documents from which the spreadsheet 232 was based, and/or any other source of data that can be associated with the spreadsheet 232 and/or the automated assistant. For example, each of the respective semantic annotations (224, 226, 228, and 230) can be generated based on content of the spreadsheet 232 and/or various design documents that were used to create each corresponding row of the spreadsheet 232 (Design_1 238, Design_2 240, Design_3 242, and Design 244); as taught by CARBUNE).
As to claim 6, CARBUNE teaches the limitations of claim 1. CARBUNE further teaches wherein the prompt further comprises a natural language processing instruction (see par. 0051, wherein the automated assistant application includes, and/or has access to, on-device speech recognition, on-device natural language understanding, and on-device fulfillment. For example, on-device speech recognition can be performed using an on-device speech recognition module that processes audio data (detected by the microphone(s)) using an end-to-end speech recognition machine learning model stored locally at the computing device 302. The on-device speech recognition generates recognized text for a spoken utterance (if any) present in the audio data. Also, for example, on-device natural language understanding (NLU) can be performed using an on-device NLU module that processes recognized text, generated using the on-device speech recognition, and optionally contextual data, to generate NLU data; see also pars. 0027; as taught by CARBUNE).
As to claim 7, CARBUNE teaches the limitations of claim 6. CARBUNE further teaches wherein the natural language processing instruction requests generating the generative content by at least one of classifying, extracting, summarizing, or standardizing the existing content or suggesting additional content based at least in part on the existing content (see par. 0036, wherein based on the edit made by the second user 126, the automated assistant can generate one or more additional semantic annotations that characterize one or more subsections of an entirety of the report document. For instance, an additional semantic annotation can characterize the edits made in FIG. 1B as “new paragraph by Howard; results.” Thereafter, content of this semantic annotation can be used when selecting whether the report document and/or the subsection of the report document is the subject of another spoken utterance from the first user 102 or the second user 126. As an example, in FIG. 1C, the first user 102 can provide a spoken utterance 142 to an automated assistant that is accessible via a client computing device 146 that is a standalone speaker device. The spoken utterance 142 can be, for example, “Assistant, were anymore edits made by Howard?” The spoken utterance 142 can be provided by the first user 102 at a point in time that is subsequent to a period of time that includes the interactions described with respect to FIG. 1A and FIG. 1B; see also pars. 0007, 0010 and 0078; as taught by CARBUNE).
As to claim 9, CARBUNE teaches the limitations of claim 1. CARBUNE further teaches wherein providing the generative output comprises providing the generative output in a generative area of the integrated development environment, the generative area separating the generative output from being in-line within the integrated development environment (see fig. 1B, par. 0033, wherein in some implementations, when the first user 102 invokes an automated assistant to perform an action that is associated with a document that the second user 126 can access, the automated assistant can cause a graphical notification 138 to be rendered at the GUI 136. The graphical notification 138 can include a rendering of a portion 134 of the report created by the first user 102, and the particular portion 134 that is rendered can include a comment 132 that is directed to the second user 126; as taught by CARBUNE).
As to claim 10, CARBUNE teaches the limitations of claim 9. CARBUNE further teaches the operations further comprising: receiving an insertion request from the user computing system via the user interface subsequent to providing the generative output; and inserting the generative output in-line within the integrated development environment in response to receiving the insertion request (see par. 0014, wherein when the automated assistant has identified the particular source document and column of data that the other user is referring to in the spoken utterance, the automated assistant can execute the “insert( )” function using the column of data. For example, the automated assistant can generate a command such as “insert (column(“August_TL-9000”,11), column(“Research_Document”, 17), wherein “11” refers to the column of the sensor data spreadsheet that includes “this month's” data, and wherein “17” refers to the “new” column previously added by the user; as taught by CARBUNE).
As to claim 11, CARBUNE teaches the limitations of claim 10. CARBUNE further teaches wherein inserting the generative output in-line within the integrated development environment comprises inserting the generative output in-line within the integrated development environment according to formatting rules of the integrated development environment (see par. 0014, wherein the automated assistant can determine that the spoken utterance from the other user included some amount of dictation and select portions of textual data that was transcribed from the spoken utterance to be incorporated into certain cells. Alternatively, or additionally, a format of each cell in the new column can be modified to correspond to “degree” values, in order that the new column will reflect units (e.g., degrees in Celsius) of the data to be added to the new column, and as specified in the spoken utterance. The automated assistant can input each numerical value according to the spoken utterance, at least based on text to speech processing and natural language understanding of the entire spoken utterance; as taught by CARBUNE).
Claims 12-13 and 15 amount to the method performed by the computer system of claims 1-2 and 6 respectively. Accordingly, claims 12-13 and 15 are rejected for substantially the same reasons as presented above for claims 1-2 and 6 and based on the references’ disclosure of the necessary supporting hardware and software.
Claims 16-17 and 19 amount to the method performed by the computer system of claims 1-2 and 6 respectively. Accordingly, claims 16-18 and 19 are rejected for substantially the same reasons as presented above for claims 1-2 and 6 and based on the references’ disclosure of the necessary supporting hardware and software.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3-4, 14 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over CARBUNE et al. (US20220076668A1) in view of SAYRE et al. (US20220043971A1).
As to claim 3, CARBUNE teaches the limitations of claim 2. CARBUNE does not expressly teach wherein the generative content within each of the second series of content cells includes a respective smart chip defined based at least in part on the existing content within the respective one of the first series of content cells.
In similar field of endeavor, SAYRE teaches wherein the generative content within each of the second series of content cells includes a respective smart chip defined based at least in part on the existing content within the respective one of the first series of content cells (See figs. 2-14, par. 0133, wherein the page visually presents, as dynamically generated, a label and an input element, such as a text entry box, a checkbox, a slider, a button, a dropdown menu, a sliding scale, or another other visual element. The label comprises the first statement, such as an alphabetic string, sourced from the second statement cell, as modified if at all in the second worksheet, based on dynamic page generation, such as via the first key, as modified if at all in the second worksheet. The input element is configured for an input, such as via a radio or option button or a dropdown menu, according to the first data type identifier sourced from the second data type cell in the second worksheet, as modified if at all in the second worksheet, based on dynamic page generation; see also pars. 0102, 0107, 0144 and 0151; as taught by SAYRE).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the CARBUNE apparatus to include the teachings of SAYRE wherein the generative content within each of the second series of content cells includes a respective smart chip defined based at least in part on the existing content within the respective one of the first series of content cells. Such a person would have been motivated to make this combination as there is a need for an accurate and efficient computer-implemented system and method for dynamically generating and rendering an online data intake questionnaire, which is configured to be modified by a person with a minimal knowledge of programming (SAYRE, par. 0006).
As to claim 4, CARBUNE and SAYRE teach the limitations of claim 3. CARBUNE further teaches wherein providing the generative output comprises replacing the first series of content cells with the second series of content cells (see figs. 2B-2D, par. 0042, wherein as provided in FIG. 2C, the user 202 can provide another spoken utterance 252 such as, “Assistant, anytime wattage is mentioned, add a comment.” In response, the automated assistant can generate one or more functions to be executed by a document application in order to fulfill the request from the user 202. When the document application executes the one or more functions, the document application and/or the automated assistant can identify instances of the term “wattage” in the spreadsheet 232 and correlate a respective comment with each instance of the term wattage. Upon completion, the document application can optionally invoke an API call to the automated assistant in order to cause the automated assistant to provide an indication 254 (e.g., “Sure.”) that the requested action(s) has been fulfilled. Alternatively, or additionally, the indication 254 can be generated with a summary of recent edits such as: a number of edits performed across the document, a summary of changes that were made, a graphical indication of a latest version of the document, and/or any other information that can characterize changes to a document; as taught by CARBUNE).
Claim 14 amounts to the method performed by the computer system of claim 3. Accordingly, claims 14 is rejected for substantially the same reasons as presented above for claim 3 and based on the references’ disclosure of the necessary supporting hardware and software.
Claim 18 amounts to the method performed by the computer system of claim 3. Accordingly, claims 18 is rejected for substantially the same reasons as presented above for claim 3 and based on the references’ disclosure of the necessary supporting hardware and software.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over CARBUNE et al. (US20220076668A1) in view of ZHANG et al. (US20200160113A1).
As to claim 5, CARBUNE teaches the limitations of claim 2. CARBUNE does not expressly teach wherein the existing content within the first series of content cells comprises only unpaired inputs.
In similar field of endeavor, ZHANG teaches further teaches wherein the existing content within the first series of content cells comprises only unpaired inputs (See fig. 3, par. 0017, wherein the source training images in the source training dataset and the target training images in target training dataset are unpaired; as taught by ZHANG).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the CARBUNE apparatus to include the teachings of ZHANG wherein the existing content within the first series of content cells comprises only unpaired inputs. Such a person would have been motivated to make this combination as it is advantageous for the user when the system does not have access to or, more generally, does not make use of any data that associates data in the source data set with data in the target data set (see also ZHANG, par. 0017).
Allowable Subject Matter
Claim 8 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Publication Number
Filing Date
Title
US20220261817A1
2022-02-17
Collaborative user support portal
US20240104306A1
2022-09-22
Collaboration content generation and selection for presentation
US20220092026A1
2020-11-03
Work spaces including links to content items in their native storage location
US20180097877A1
2016-09-30
Linking content items and collaboration content items
US20070244906A1
2007-04-13
Collaborative Content Generation System And Method
US11537784B2
2021-06-25
Collaborative in-line content item annotations
US10223136B2
2017-04-14
Generating content objects using an integrated development environment
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KOOROSH NEHCHIRI whose telephone number is (408)918-7643. The examiner can normally be reached M-F, 11-7 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William L. Bashore can be reached on 571-272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KOOROSH NEHCHIRI/Examiner, Art Unit 2174
/WILLIAM L BASHORE/ Supervisory Patent Examiner, Art Unit 2174
Prosecution Insights