DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 51 and 61 have been amended. Claims 51-54, 56-64 and 66-70 are pending.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 51-53, 56-63, 66-70 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tesch et al. (US 20140161356) in view of HAN et al. (US 20190379942) and in further view of GU et al. (US 20170052946), CHEN et al. (US 20180255009) and Yang et al. (US 2018/0348890).
Regarding claim 51, Tesch teaches a method comprising:
receiving a query comprising a plurality ([0055], [0074] “user … typing a phrase or text … with emoticons and/or acronyms”) of query emoji ([0059] “a smiley face identified in a text message”, [0067], [0079]-[0080], [0082] “emoticons and/or acronyms can be received … as a query term or the like”, [0102], [0138]);
accessing metadata for a video, the metadata comprising a set of emojis generated based at least in part on reactions to the video received from a plurality of user devices in communication with a social platform ([0071] “components described herein can be presented on a social network”, [0074] “emoticons and/or acronyms associated with the set of media content portions”, [0108] “metadata associated with the media content and/or characteristics or features of the videos/images/audio content being analyzed”, [0101]), the metadata comprising a set of emojis based on reactions to the video ([0057] “matching of audio content of the media content with words that are represented by the acronym or the matching of an action, an expression, or audio content with an image or an emotion represented by the emoticon”; [0058] “media splicing … receives the identified emoticons and/or acronyms from the image analysis”; [0059] “portions of media content or media content portions include segments of video clips and/or images that express the emoticon and/or acronym”, [0083], [0178]),
wherein each emoji of the set of emojis has a corresponding classification for the video based in part on instances of the respective emoji in the reactions to the video ([0152] “media content corresponding to the phrases, words, and/or images that meet a set of classification criteria, such as for popular videos”, [0153], [0155]-[0156], [0171] “criteria can include a vague extraction, an estimated extraction or, in other words, an imprecise extraction”);
selecting, from the set of emojis of the metadata, a metadata emoji based at least in part on determining the corresponding classification for the video meets or exceeds a first score threshold;
for each query emoji in the plurality of query emojis determining a second matching score between the respective query emoji and the metadata emoji ([0068] “generate different associations among an acronym and/or an emoticon with an image of media content”, [0080], [0083], [0090], [0092])(see NOTE),
wherein the metadata emoji is different from the respective query emoji ([0060] as in for a “smiley face emoticon (:)) and/or LOL acronym … returns media content portions having a smiley face made by a vampire, werewolf, jack-o-lantern, ghost, or any other hallowed like theme with images, videos segments, or sounds having the Halloween theme and that also correspond to the emoticon a smiley face”, “a smiley face or LOL received … return a vampire smiling or laughing out loud from scenes of the movie "Salem's Lot" … many different classifications”, [0061]) (see NOTE I)
determining that the second matching score of the query emoji and the metadata emoji meets or exceeds a second score threshold ([0073], [0079] “a match of the image … with the identified word/phrase/image of the emoticon and/or acronym can determine what portions are extracted from the media content”, [0080], [0126], wherein fuzzy logic allows for degrees of truth between 0 and 1, with the threshold acting as a point on this scale, also see [0131] “matches or matching criteria of the predetermined criteria can be weighted”);
based at least in part on
(i) the classification for the metadata emoji corresponding to the video
(ii) determining that the second matching score of at least one of the query emojis in the plurality of query emojis and the metadata emoji meets or exceeds the second score threshold ([0073], [0079], [0080], [0126], wherein fuzzy logic allows for degrees of truth between 0 and 1, with the threshold acting as a point on this scale, also see [0131] “matches or matching criteria of the predetermined criteria can be weighted”),
determining that at least one of the query emoji matches the video ([0073], [0079] “a match of the image … with the identified word/phrase/image of the emoticon and/or acronym can determine what portions are extracted from the media content”, [0080], [0126], wherein fuzzy logic allows for degrees of truth between 0 and 1, with the threshold acting as a point on this scale, also see [0131] “matches or matching criteria of the predetermined criteria can be weighted”, C1 “extract a set of media content portions from media content that correspond to the emoticon or the acronym”); and
based at least in part on determining that the at least one of query emoji matches the video, providing for display a representation of the video as a query result ([0050] “generate any number of portions of a movie, film or other video, audio content, photos or the like as candidate to place within the multimedia message for the portion of the multimedia message that corresponds to or is expressed by the emoticon received”, [0059]-[0060], [0074]-[0075] “user, for example, could prefer a scene from a movie (e.g., Rocky) to represent an emoticon and/or acronym, rather than a segment of a home video”, [0098], [0126], [0148], [0158] “generate a sequence of media content portions that correspond to words, phrases or images of the inputs”) with representations of the plurality of query emojis ([0073] “the labeled emoticon or acronym associated with the media content portion can be presented with the media content”, [0074] “a portion of video may be labeled … with emoticons and/or acronym”), wherein the representations of media with the plurality of query emojis are visually ordered based at least in part on the second matching score of each of the corresponding query emojis in the plurality of query emojis NOTE II).
◊ Tesch does not explicitly teach, however, HAN discloses – wherein each emoji of the set of emojis has a corresponding first matching score ([0033], [0038]) for the video based in part on instances of the respective emoji in the reactions to the video (Table 1, [0038]-[0039], [0041]) and
based at least in part on (i) the first matching score for the metadata emoji corresponding to the video meeting or exceeding the first score threshold ([0041], [0055], [0075], [0079], [0087] where “icon occurrence density is greatest are selected” is a threshold).”
NOTE I - HAN further discloses the limitation - “wherein the metadata emoji is different from the query emoji” (see [0128] where “emojis “V5” and “thumbs up”, the text features and the emoji category corresponding to “compliment”, [0148] “correspondence relationships between emotion labels and various emojis on each theme may be established”).
◊ Tesch does not explicitly teach, however, Gu discloses – “wherein the metadata emoji is different from the query emoji, and wherein the metadata emoji and the query emoji are untranslated” ([0075], [0078] wherein the evaluation is based on picture content and not based on text, thus, it is reasonable and obvious to conclude that the emoji is not translated during matching, [0080] “pictures may also be matched”, [0111], [0114], [0188]-[0189]).
NOTE I - Gu further discloses the limitation - “wherein the metadata emoji is different from the query emoji”([0033], [0038], [0128] where “emojis “V5” and “thumbs up”, the text features and the emoji category corresponding to “compliment”, [0148] “correspondence relationships between emotion labels and various emojis on each theme may be established”).
NOTE Tesch teaches –
“text based message having words or phrases that are matched with the words or phrases correlated to or identified with the media content portions” [0092], wherein - “Each word or phrase… can be any tag, label or metadata that identifies the media content portion” [0089], wherein such “tag, label or metadata” include emoticons –“the words or phrases connected with each portion from the set of home videos … for labeling with an emoticon and/or acronym”; “a portion of video may be labeled according to the word or phrase … and also with emoticons and/or acronyms” [0074].
Based on the above, matching words or phrases (which can be label or metadata, such as emoticons and/or acronyms) in user query with the words or phrases (which can be label or metadata, such as emoticons and/or acronyms) in the media content portions, is obviously analogous to matching emoticon and/or acronym with emoticon and/or acronyms in the video content. Tesch further teaches that such matching is based on “a rule based logic, fuzzy logic, probabilistic, statistical reasoning, classifiers, neural networks” [0126], which implicitly provides a thresholds. Therefore, Tesch obviously and implicitly teaches the limitations of - “determining a second matching score between the query emoji and the metadata emoji; determining that the second matching score of the query emoji and the metadata emoji meets or exceeds a second score threshold.”
However, to merely obviate such reasoning Gu teaches - “determining a second matching score between the query emoji and the metadata emoji; determining that the second matching score of the query emoji and the metadata emoji meets or exceeds a second score threshold” ([0080], [0084], [0091], [0155]).
Gu also teaches “emojis generated based at least in part on reactions to the video received from a plurality of user devices in communication with a social platform” [0029], [0063].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Tesch to include second matching score of the query emoji and the metadata emoji meets or exceeds a second score threshold as disclosed by GU. Doing so quickly and accurately meet current input needs of users and input efficiency (CU [0012]).
◊ Once again, Tesch teaches – that matching is based on various factors, such as (1) fuzzy matching between emojis in a query and emojis in the media content and (2) such as popularity [0152]-[0153], which at least indicates a frequency of selection of such media content.
As stated above, Tesch does not explicitly teach a first threshold (i.e. matching based on “quantity and/or frequency of an emoji associated with a content item”, as disclosed in the applicant’s specification). While Tesch as modified by Han teaches prong (i), Chen additionally or alternatively discloses selecting content based on both prongs–
based at least in part on (i) the first matching score for the metadata emoji corresponding to the video meeting or exceeding the first score threshold ([0123]) and (ii) determining that the second matching score of the query emoji and the metadata emoji meets or exceeds the second score threshold ([0053] “emojis that are frequently used”, [0088] “occurrence density for each type of emoji is greatest can be selected”, [0095], [0100], [0105], [0125]).
Chen discloses - wherein the first matching score is a statistical value ([0033] “with respect to a certain video within a certain period (e.g., a month) can be statistically analyzed”, [0038]) based in part on at least one of a quantity or a frequency of instances of the metadata emoji in the reactions to the video ([0053], [0055]-[0056], [0075]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Tesch to include prong (i) and (ii) as disclosed by CHEN. Doing so increasing an accuracy of the emoji recommendation (CHEN [0125]) and improve interaction between different users and a click rate of multimedia resources (CHEN [0069]).
Tesch does not explicitly teach, however Yang discloses wherein the representations of the plurality of query emojis are visually ordered based at least in part on the second matching score of each of the corresponding query emojis in the plurality of query emojis and the metadata emoji ([0028] “rank the emojis of the collection of emojis based on the weighted scores”, [0059] “rank the emojis of the collection of emojis based on the weighted scores”, [0066] “plurality of factor values can be calculated for each emoji returned by the query”, [0075] “presentation is based on a ranking of the one or more emojis”, F4a:414, 418, 420, 422).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Tesch as modified to include visually ordering based on matching score as disclosed by Yang. Doing so help users express ideas nonverbally by easily choosing a desired emoji from the available emojis (Yang [0002]).
Claim 61 recites substantially the same limitations as claim 1, and is rejected for substantially the same reasons.
Regarding claims 52 and 62, Tesch as modified teaches the method and the system, wherein the video comprises a plurality of scenes having corresponding genres, the method further comprising: determining one or more scenes of the plurality of scenes having a genre associated with the query emoji (Tesch [0050], [0178], [0196], HAN [0048], [0084], GU [0052]).
Regarding claims 53 and 63, Tesch as modified teaches the method and the system, wherein the set of emojis for the video comprises one or more emojis for each scene of the plurality of scenes (HAN [0048], [0084], [0088]-[0089], Tesch [0059]-[0060] “portions of media content or media content portions include segments of video clips and/or images that express the emoticon and/or acronym … generate any number of portions of a … video … as candidate to place within the multimedia message for the portion of the multimedia message that corresponds to or is expressed by the emoticon received”, [0066] “words or phrases are associated with the image identified and then the media content is searched and spliced for video segments, audio segments, and/or image content portions that represent the words or phrases”, [0083] “determined portions of video corresponding to the emoticon and/or acronym”, [0089], [0148], [0155], [0193]).
Regarding claims 56 and 66, Tesch as modified teaches the method and the system, wherein the video is posted at one or more social platforms, the method further comprising identifying the video based on the one or more social platforms (Tesch [0071], [0101], CHEN [0026], GU [0063], HAN [0032]).
Regarding claims 57 and 67, Tesch as modified teaches the method and the system, wherein the metadata emoji and the respective query emoji are untranslated images (GU [0078] wherein the evaluation is based on picture content and not based on text, thus, it is reasonable and obvious to conclude that the emoji is not translated during matching [0111], [0114], [0188]-[0189]).
Regarding claims 58 and 68, Tesch as modified teaches the method and the system, wherein the representation of the video comprises a video clip from the video (Tesch [0060], [0075] “user, for example, could prefer a scene from a movie (e.g., Rocky) to represent an emoticon and/or acronym, rather than a segment of a home video”, [0098], [0126], [0158] “generate a sequence of media content portions that correspond to words, phrases or images of the inputs”).
Regarding claims 59 and 69, Tesch as modified teaches the method and the system, wherein the representation of the video comprises an icon corresponding to the respective emoji (Tesch [0053], [0062], HAN [0032], [0039]).
Regarding claims 60 and 70, Tesch as modified teaches the method and the system, wherein the representation of the video is displayed at a remote device (Tesch [0118], [0120], [0222], HAN [0140]).
Claim(s) 54, 64 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tesch as modified and in further view of ALEY et al. (US 2016/0292148).
Regarding claims 54 and 64, Tesch as modified teaches the method and the system, wherein the query comprises a text portion, the method further comprising determining, based on the text portion, a textual match
Tesch does not explicitly teach, however ALEY discloses match score ([0130]-[0131], [0082]-[0084], [0086], [0105]). It would have been obvious to one of ordinary skill in the art at the time of invention to modify the teachings of Tesch to include match score as disclosed by ALEY. Doing so provides a significant performance improvement (ALEY [0095]).
Claim(s) 51 and 61 can be alternatively rejected under 35 U.S.C. 103 as being unpatentable over previously cited Huang et al. (US 2017/0083524) in view of HAN et al. (US 20190379942) and in further view of Yang et al. (US 2018/0348890).
Response to Arguments
Applicant's arguments filed 09/02/2026, in regard to the presently amended claims, are addressed in the updated rejections to the claims above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to POLINA G PEACH whose telephone number is (571)270-7646. The examiner can normally be reached Monday-Friday, 9:30 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aleksandr Kerzhner can be reached at 571-270-1760. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/POLINA G PEACH/ Primary Examiner, Art Unit 2165 September 15, 2026
