Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Claims 1, 10-11, and 16 have been amended.
Claims 1-20 are presently pending.
Claims 4-9, 13-15, and 18-20 remain withdrawn.
Applicant’s arguments with respect to claims 1, 11, and 16 have been considered but are moot in view of the new ground(s) of rejection.
Although a new ground of rejection has been used to address additional limitations that have been added to Claims 1, 11, and 16 a response is considered necessary for several of applicant's arguments since references Archibong and Cha will continue to be used to meet several claimed limitations.
Regarding Applicant’s arguments with respect to the Archibong and Cha references (see Remarks, pgs. 9-10), the Examiner disagrees.
Applicant first notes that the social TV dongle of Archibong is separate from the TV and other display devices of Archibong (see Remarks, pg. 9, first two sentences). The Examiner notes that there are no particular arguments added to this comment by Applicant, and further notes that there are no limitations in the claims that preclude intermediary devices. Furthermore, Examiner draws attention to Archibong [0341-342] specifically teach that the various computing elements and processes disclosed may be embodied as unitary or distributed computing systems.
Applicant subsequently draws attention to Archibong [0223], arguing that ‘the ability to later view relevant chat messages about content that is currently being viewed is not reasonably equivalent to the limitations recited in Claim 1’ (see Remarks, pg. 9, ¶1). While the Examiner does not dispute that an embodiment for later viewing is described in [0223], Applicant ignores that Archibong [Figs. 21-24] and [0227-230] (NOTE: there was a typographical error in the Non-Final Rejection mailed out on 17 October 2025 – hereinafter the Non-Final – pg. 6, ¶1-2, citing [0027-230] which should have read as [0227-230] which coincides with the description of [Figs. 23-24] which was cited and relied upon) explicitly disclose an embodiment where the system may utilize a second screen of a mobile device 840 to permit a user to participate in a chat while viewing particular content on a separate TV display 830 as shown in [Fig. 23]. Archibong [0212-215] and [0223] are particularly cited to note that various call to actions and chat features may be time-stamped and synced to particular moments of a television program, and where various call-to-action options may change according to metadata associated with particular moments of the television program.
Applicant further argues that the Cha reference ‘merely indicates that chat room applications may recommend different emoticons according to meta data information’…and ‘not reasonably equivalent to execution screen of the message application which changes in response to data related to the content’ (see Remarks, pg. 9, ¶1). The Examiner notes that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. Furthermore, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Per Applicant’s own admission, Cha’s reference may provide different emoticons (i.e., selectable options) in a chat interface according to available metadata. As noted above, Archibong’s system may provide time-stamped metadata for a particular content that may dynamically provide a variety of selectable options (i.e., the call-to-actions) on a mobile device while viewing the content on a separate display device. Additionally, Archibong discloses that a chat interface may be provided to allow users to actively chat on their mobile devices while viewing particular content on a separate display device. It would be understood by one of ordinary skill in the art from these teachings that the chat interface of Archibong could be modified by Cha’s teachings to provide different emoticons according to metadata associated with the chat, which in the case of Archibong, could be metadata associated with the content for which the chat is associated. It would also follow that the emoticons of Cha could reasonably pull from the metadata of Archibong used for dynamic call-to-action elements (as emoticons may be considered a form of call-to-action). DiFrancesco is newly introduced to teach the newly added limitations that metadata for selecting relevant emojis associated with some content (such as the chat interface emojis of Archibong and Cha) may be at least in part determined from facial expression analysis of the underlying content. As such, the combination of Archibong, Cha, and DiFrancesco disclose, teach, and suggest all of the limitations of the claimed invention.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 10-12, and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Archibong et al. (US 2014/0068692 A1) (of record, hereinafter Archibong), in view of Cha et al. (US 2022/0404952 A1) (of record, hereinafter Cha), and further in view of DiFrancesco (US 2021/0160581 A1) (hereinafter DiFrancesco).
Regarding Claim 1, Archibong discloses an electronic device, [Figs. 37-39] comprising:
communication circuitry; [Figs. 37-39; communication component 3730/communication interface 3910]
a display; [Figs. 37-39; 0324, 0347: computing device may include a touchscreen]
at least one processor; [Figs. 37-39: processor 3710/3902] and
memory storing instructions that, when executed by the at least one processor, [Figs. 37-39: memory 3720/3904 and software 3810] cause the electronic device to:
establish a connection between an external display device and the electronic device through the communication circuitry; [Figs. 4, 8, 15-17, 21-24, 27-28; 0107, 0110: mobile device 840 may be connected to some TV 830 (external display device) via TV dongle 810]
execute a message application that allows participation of the electronic device and at least one external electronic device; [Figs. 23-24: 0217-221: user watching TV 830 may have chat application launched on mobile device 840, where chat room may be associated with content on TV]
obtain data related to content displayed on the external display device based on the connection between the external display device and the electronic device; [Figs. 21-24, 27-28: 0212-0215, 0223, 0227-230, 0267: where call-to-actions/other buttons may be provided on mobile device 840 based on what is being shown on TV 830, where such information may be tied to metadata and timestamps associated with the what is shown on TV 830] and
display, on the display, an execution screen of an application, which changes in response to the data related to the content. [Figs. 21-24, 27-28: 0212-0215, 0223, 0227-230, 0267: where call-to-actions/other buttons may be provided on mobile device 840 based on what is being shown on TV 830, where such information may be tied to metadata and timestamps associated with the what is shown on TV 830]
Archibong fails to explicitly disclose display, on the display, an execution screen of the message application, which changes two or more emojis in response to the data related to the content. (Emphasis on the particular element of the limitations not explicitly disclosed by Archibong – namely wherein aspects of the message application itself change responsively to the data related to the content).
Cha, in analogous art, teaches display, on the display, an execution screen of the message application, which changes two or more emojis in response to the data related to the content. [Fig. 7; 0064-67, 0130-132: chat room applications running on a user terminal (such as the chat application of Archibong above) may recommend different emoticons to users according to meta data information, such as classification of topic and context of the chat room (such as the TV-related information of Archibong above), where the example of Fig. 7 shows that a plurality of emoji may be recommended/displayed]
It would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to modify the apparatus of Archibong with the teachings of Cha to display an execution screen of the message application responsive to data related to the content in order to recommend suitable emoticons according to the user’s context in a simple and fast manner. [Cha – 0002-6]
Archibong and Cha fail to explicitly disclose display, on the display, an execution screen of the message application, which changes two or more emojis on the display in response to the data related to the content displayed on the external display device based at least in part on a facial expression analysis of one or more characters appearing in the content displayed on the external display device. (Emphasis on the particular elements of the limitations not explicitly disclosed by Archibong and Cha, namely that the emojis are decided at least in part on the facial expressions of characters in the content).
DiFrancesco, in analogous art, teaches display, on the display, an execution screen of the message application, which changes two or more emojis on the display in response to the data related to the content displayed on the external display device based at least in part on a facial expression analysis of one or more characters appearing in the content displayed on the external display device. [Figs. 1, 4; 0021-24, 0027-28, 0033, 0044, 0049-52: video data (such as the content displayed on the external display of Archibong and Cha) may be analyzed such that video features may be analyzed to detect and identify facial expressions of people in the video to determine emotion descriptors to select emotional descriptor icon (emoji) with temporal timing association with the video position, wherein multiple emoji may be identified for any given temporal position]
It would have been obvious to one of ordinary skill in the art prior to the filing date of the invention to modify the apparatus of Archibong and Cha with the teachings of DiFrancesco to change emojis based at least in part of facial expressions of characters in the content in order to effectively and accurately select relevant emoji from a video that convey the same meaning and nuances as intended by the originator of the content. [DiFrancesco – 0008]
Regarding Claim 2, Archibong, Cha, and DiFrancesco disclose all of the limitations of Claim 1, which are analyzed as previously discussed with respect to that claim.
Furthermore, Archibong and Cha disclose wherein the execution screen of the message application is configured to change a first component of the execution screen to a second component in response to the data related to the content. [Archibong – Figs. 21-24, 27-28: 0212-0215, 0223, 0027-230, 0267: where call-to-actions/other buttons may be provided on mobile device 840 based on what is being shown on TV 830, where such information may be tied to metadata and timestamps associated with the what is shown on TV 830; Cha – 0125-0132: where recommended emoticons may be arranged according to some priority according to the chat room classification/context]
Regarding Claim 3, Archibong, Cha, and DiFrancesco disclose all of the limitations of Claim 2, which are analyzed as previously discussed with respect to that claim.
Furthermore, Cha discloses wherein the first component and the second component include at least one of a theme, an emoji, an emoticon, or a background image. [Cha – 0125-0132: where recommended emoticons may be arranged according to some priority according to the chat room classification/context]
Regarding Claim 10, Archibong, Cha, and DiFrancesco disclose all of the limitations of Claim 1, which are analyzed as previously discussed with respect to that claim.
Furthermore, Archibong and Cha disclose wherein the instructions cause the electronic device to: receive a sorted group of the two or more emojis based on the facial expression analysis of one or more characters appearing in the content displayed on the external display device; arrange the two or more emojis of the sorted group according to priorities of the two or more emojis based on a result of recognizing content displayed on the external display device; an display at least a portion of the sorted group of the two or more emojis on the execution screen, wherein an updated sorted group of the two or more emojis is received based on a change to the content displayed on the external display device. [Archibong – Figs. 21-24, 27-28: 0212-0215, 0223, 0027-230, 0267: where call-to-actions/other buttons may be provided on mobile device 840 based on what is being shown on TV 830, where such information may be tied to metadata and timestamps associated with the what is shown on TV 830; Cha – Fig. 7; 0125-0132: where recommended emoticons may be arranged according to some priority according to the chat room classification/context; DiFrancesco – Figs. 1, 4; 0021-24, 0027-28, 0033, 0044, 0049-52: one or more emoji may be selected and temporally associated with scenes of a video content at least in part based on facial expression analysis of characters in the scene]
Regarding Claim 11, Claim 11 recites a method that performs the functions of the apparatus of Claim 1. As such, Claim 11 is analyzed and rejected similarly as Claim 1, mutatis mutandis.
Regarding Claim 12, Archibong, Cha, and DiFrancesco disclose all of the limitations of Claim 11, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 12 recites nearly identical limitations as Claim 3 (which also includes the limitations of Claim 2) and is rejected similarly as that claim.
Regarding Claim 16, Claim 16 recites a non-transitory computer program product that performs the functions of the apparatus of Claim 1. As such, Claim 16 is analyzed and rejected similarly as Claim 1, mutatis mutandis.
Regarding Claim 17, Archibong, Cha, and DiFrancesco disclose all of the limitations of Claim 16, which are analyzed as previously discussed with respect to that claim.
Furthermore, Claim 17 recites nearly identical limitations as Claim 3 (which also includes the limitations of Claim 2) and is rejected similarly as that claim.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM J KIM whose telephone number is (571)272-2767. The examiner can normally be reached 9:30am - 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hadi Armouche can be reached at (571) 270-3618. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM J KIM/Primary Examiner, Art Unit 2409