DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Response to Amendments and Arguments
Regarding an outstanding rejection under 35 U.S.C. §101, applicant amended independent claims by adding new limitations. Applicant provided arguments (Remarks, pages 7-11).
By reviewing the amended claims and applicant’s arguments, the examiner agreed that when considering all limitations as a whole, the claimed inventions are no longer directed to a judicial exception. The arguments are persuasive. The rejection under §101 has been withdrawn.
Regarding an outstanding rejection under 35 U.S.C. §102, applicant amended independent claims by adding several new limitations. Applicant argued (Remarks pages 11-13) that a previously cited Tumbde reference (US PG Pub. 2022/0350954) fails to teach the newly added limitations.
In response, the examiner pointed out that an argued “a programmatic model (software)” is interpreted as a model implemented by computer program / software. For example, Tumbde describes using natural language processing model to identify spoken words (Tumbde, [0023]).
The examiner has performed an update search and discovered a reference to Joly et al. (US Pat. 12/243,511). Joly discloses using a neural network implemented emphasis model to emphasize certain words based on determined contexts (Joly, Col. 8, lines 31-55, Fig. 2). Joly meets the newly added limitations:
wherein the generating the emphasis representation in the programmatic model comprises (Joly, Col. 3, lines 1 40, a neural network implemented emphasis model, Fig. 1, #150):
assigning a weight to the identified element in the programmatic model (Joly, Col. 7, lines 35-66);
tagging an identifier to the identified element (Joly, Col. 3, lines 59-64; Col. 9, lines 56-67); and
prioritizing the identified element in the programmatic model based on the assigned weight to generate the emphasis representation (Joly, Col. 8, lines 31-55; Fig. 2);
Below is Fig. 2 replicated from the newly cited Joly reference.
PNG
media_image1.png
850
766
media_image1.png
Greyscale
In the following rejection, the examiner combines the previously cited primary reference to Tumbde with a newly discovered Joly reference to reject the amended independent claims. The arguments regarding the anticipation rejection under §102 have been considered but are moot because the arguments do not apply to the following new rejection necessitated by the amendment.
Claim Rejections - 35 USC § 103
Claims 1-20 are rejected under 35 U.S.C. §103 as being unpatentable over Tumbde et al. (US PG Pub. 2022/0350954, referred to as Tumbde) in view of Joly et al. (US Pat. 12,243,511, referred to as Joly).
Tumbde is a published patent application by the same assignee (IBM Inc.) of the instant application. Tumbde discloses monitoring conversations between meeting participants during a video conference (Tumbde, Abstract, [0003], [0017], using Webex meeting software, [0024-0025]). Tumbde further discloses highlighting displayed words / sentences on presenting slides (a claimed “applying an emphasis effect to the element”) when contents or contexts being discussed match displayed elements (Tumbde, [0028-0029], [0033-0040], Fig. 3C).
Joly discloses a neural network implemented system for emphasizing certain words based on context information. Joly discloses parsing a sentence by tagging words to indicate as {Verb}, {Object} (Joly, Col. 20, lines 46-48, Col. 30, lines, 1-10). Joly further discloses calculating weights / scores (Joly, Col. 7, lines 35-40; Col. 36, lines 5-15; Col. 37, lines 24-35).
Regarding claims 1, 11 and 20, Tumbde discloses a method, a computer program product and a system (Tumbde, [0018], Fig. 4, Fig. 5, a computer implemented system / method for highlighting displayed elements that is being discussed during a video conference), comprising:
analyzing, by a processor set and via natural language processing, a conversational input between a plurality of users to identify a description of an element (Tumbde, [0023-0027], Fig. 2, #204-#212, monitoring and analyzing discussions between meeting participants during a video conference using natural language processing, NLP, techniques; identifying keywords, context of discussions);
applying, by the processor set and via a machine learning model, a matching procedure between the conversational input and a virtual environment to identify the element matching the description of the element (Tumbde, [0014], [0032-0033], comparing and matching displayed elements in slides with discussed topics; Fig. 2, #210);
generating, by the processor set, an emphasis representation in a programmatic model of the virtual environment based on the description of the element (Tumbde, [0021-0023], [0034-0039], Fig. 3C, generating highlighting displayed elements that matches discussed topics / keywords using API calls); and
applying, by the processor set, an emphasis effect to the element within the virtual environment corresponding to the emphasis representation (Tumbde, [0034-0036], Fig. 3C, highlighting elements on a displaying slide, the highlighted elements match discussed topics / contexts).
Tumbde discloses monitoring a discussion between meeting participants during a virtual meeting and highlighting certain words on a displayed slide. These words are related to the discussion (See Fig. 3A - 3C). Tumbde does not explicitly disclose the newly added limitations. Joly discloses emphasizing certain words based on a context in a previous question. Joly discloses the newly added limitations:
wherein the generating the emphasis representation in the programmatic model comprises (Joly, Col. 3, lines 1 40, a neural network implemented emphasis model, Fig. 1, #150):
assigning a weight to the identified element in the programmatic model (Joly, Col. 7, lines 35-66);
tagging an identifier to the identified element (Joly, Col. 3, lines 59-64; Col. 9, lines 56-67); and
prioritizing the identified element in the programmatic model based on the assigned weight to generate the emphasis representation (Joly, Col. 8, lines 31-55; Fig. 2);
Both Tumbde and Joly are related to processing natural language text and emphasizing certain words. It would have been obvious to a person having ordinary skill in the art at the time the invention was filed to modify Tumbde’s teaching with Joly’s teaching to emphasis certain words based on context and implementing the emphasis using a neural network. One having ordinary skill in the art would have been motivated to make such a modification so that it is easy to see relevant sections in the presentation (Tumbde, [0002-0003]). In addition, all the claimed elements were known in the prior art and one skilled in the art could have combined the elements as claimed by known methods, and in the combination each element merely would have performed the same function as it did separately. “A combination of familiar elements according to known methods is likely to be obvious when it does no more than yield predictable results.” KSR, 550 U.S. ___, 82 USPQ2d at 1395 (2007). One of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claims 2 and 12, Tumbde in view of Joly further discloses the applying the emphasis effect comprises rendering the emphasis effect corresponding to the emphasis representation of the element within the virtual environment (Tumbde, [0036], applying highlighting, bold face or underline to certain portions in a slide that matches discussed topics / contexts during the video conference, Fig. 3C).
Regarding claims 3 and 13, Tumbde in view of Joly further discloses the applying the matching procedure comprises:
identifying, by the processor set and via the natural language processing, the description of the element within the conversational input (Tumbde, [0025-0028], using natural language processing techniques to identify topics / context in a meeting discussions); and
identifying, by the processor set, the element within the virtual environment based on the description of the element within the conversational input (Tumbde, [0023], [0030], [0035], Fig. 3C, identifying displayed elements in presenting slides that match meeting discussions during a video conference).
Regarding claims 4 and 14, Tumbde in view of Joly further discloses the identifying the description of the element within the conversational input comprises inferring the description of the element within the conversational input by utilizing semantic similarity analysis (Tumbde, [0028], [0031-0032], comparing semantic similarity between meeting discussions and displayed slides).
Regarding claims 5 and 15, Tumbde in view of Joly further discloses adjusting the emphasis effect applied to the element based on an amount of time that has elapsed since the element was identified (Tumbde, [0030], [0035], displaying one slide at a time on a screen, highlighting applying previous / current slides based on context).
Regarding claims 6 and 16, Tumbde in view of Joly further discloses:
determining a level of confidence that the emphasis representation includes the element (Tumbde, [0032-0033], [0043], determining confidence of relevant content and similarity scores); and
displaying a confidence score on a display which corresponds with the level of confidence (Tumbde, [0044], Fig. 3C, #308, displaying a thurm up showing a valid content being highlighted).
Regarding claims 7 and 17, Tumbde in view of Joly further discloses the level of confidence is distinct from the emphasis effect (Tumbde, Fig. 3C, #306, emphasis effect is underline and bold face, #308, confidence level is showing as a thumb-up symbol).
Regarding claims 8 and 18, Tumbde in view of Joly further discloses the conversational input comprises a voice input, and wherein the virtual environment is an immersive virtual environment comprising a virtual world environment (Tumbde, [0015], [0017], a virtual meeting environment).
Regarding claims 9 and 19, Tumbde in view of Joly further discloses the conversational input comprises multi-modal conversational input (Tumbde, [0015], audio, video and text inputs; [0023-0024], [0038], during a video meeting, voice input using microphone, facial and body language inputs using camera).
Regarding claim 10, Tumbde in view of Joly further discloses the machine learning model comprises generative artificial intelligence for performing the matching procedure between the conversational input and the virtual environment to identify the element matching the description of the element (Tumbde, [0025], [0028-0029], [0031], [0033], using machine learning models to compare and identify matching between discussion topics / contexts and displayed elements in presenting slides).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jialong He, whose telephone number is (571) 270-5359. The examiner can normally be reached on Monday – Friday, 8:00AM – 4:30PM, EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Pierre Desir can be reached on (571) 272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIALONG HE/Primary Examiner, Art Unit 2659
Prosecution Insights