DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Application
Claims 1-7, 9-15, and 17-22 were pending and were rejected in the previous office action. Claims 1, 9 and 17 were amended. Claims 1-7, 9-15, and 17-22 remain pending and are examined in this office action.
Priority
This application claims priority to U.S. Provisional Patent Application No. 63/508,790, filed on June 16, 2023.
Response to Arguments
35 USC § 103:
Applicant’s arguments with respect to the previous § 103 rejections of claims 1-7, 9-15, and 17-22 (pgs. 11-14, remarks filed 3/19/2026) have been considered but are moot, as they do not apply to the current grounds of rejection in the new § 103 rejections of claims 1-7, 9-15, and 17-22 below, in response to applicant’s amendments.
Please see current § 103 rejections of claims 1-7, 9-15, and 17-22 below.
In addition, regarding applicant’s argument specific to the previous § 103 rejection of claim 21, that Zhu does not rely on a mapping data structure for lookup (pgs. 13-14, remarks), the examiner respectfully disagrees.
As seen in the office action below:
Cheng teaches wherein receiving the second template comprises: accessing a mapping data structure that associates different types of first media content items with corresponding templates (Cheng: ¶ 0020-0022 showing mapping activities to an event, wherein as per ¶ 0060 activities corresponding an event type and associated with a stored template); determining a content type associated with the first media content item (Cheng: ¶ 0060 showing the service identifies an event type from analyzing the input video files and key activities typically associated with that event type); performing a lookup operation using the mapping data structure to identify and retrieve a template corresponding to the determined content type (Cheng: ¶ 0058-0060, with ¶ 0058 showing “mapping 140 defines logical links or connections 420, 422, 424, 426, 428 between the various types of detected features 410, concepts 412, relations 414, events 416, and salient activities 418. The system 140 can use the mapping 140 and particularly the links 420, 422, 424, 426, 428, in performing the semantic reasoning to determine events and salient activities based on the features 410, concepts 412, and relations 414”, and ¶ 0060 “the event type corresponds to a stored template identifying the key activities typically associated with that event type. For example, for a birthday party, associated activities would include blowing out candles, singing the Happy Birthday song, opening gifts, posing for pictures, etc. Using feature detection algorithms for complex activity recognition, such as those described above and in the aforementioned priority patent applications, the service automatically identifies segments (sequences of frames) within the uploaded file that depict the various key activities or moments associated with the relevant event type”).
Cheng/Barr/Evans do not explicitly teach that the mapping data structure is used to associate types of media content items with template identifiers, performing the lookup operation to identify a template identifier, or retrieving the second template based on the identified template identifier.
However, Zhu teaches analyzing and determining types of content located in first media content/video to determine a content type identifier, and determining a material set identifier corresponding to the content type identifier to retrieve relevant video templates/”video editing material sets” that are determined according to a material set identifiers, (Zhu: Fig. 6 steps s604-s607 and ¶ 0090-0096, ¶ 0007, ¶ 0034-0037, ¶ 0043, ¶ 0050-0052, ¶ 0070-0075; and Fig. 7 showing template recommendation including a number of templates; specifically see ¶ 0034-0037, ¶ 0043 showing accessing and determining a content type identifier associated with each content element in a video, i.e. accessing a mapping data structure used to lookup a material set identifier, and determining a material set identifier corresponding to the content type identifier, i.e. performing a lookup of the matching material set for that content type identifier; ¶ 0050-0052, ¶0070-0075 showing material set for video editing corresponds to video editing templates). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included the analyzing and mapping data included in video content in order to determine and retrieve a video editing material set/template according to material set identifier(s) as taught by Zhu in the video highlight generation system of Cheng/Barr/Evans with a reasonable expectation of success of arriving at the claimed invention, with the motivation that to “improve the quality of video editing and diversify video editing experience, editing templates may be variously generated and determined according to contents of a video” (Zhu: ¶ 0005) and solve the problem that “In a conventional video editing technique, an editing template may include fixed material or editing schemes, and therefore, images or videos edited using the same editing template may result in the same or substantially the same images or videos in style and may be homogenous. Thus, the conventional video editing technique does not provide diversified video editing techniques for a user, thereby degrading the user experience” (Zhu: ¶ 0025). It would have also been obvious to one of ordinary skill in the art before the effective filing date of the invention to do so, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
In other words, Zhu identifies obtains a to-be-edited object and determines from the accessed content a content element and associated content type identifier (i.e. a mapping data structure) and directly uses the content type identifier to determine the corresponding material set identifier corresponding to a video editing material set corresponding to the content type identifier according to the material set identifier, thereby curing the deficiencies of Cheng/Barr/Evans. The claims do not specify a particular technical data configuration corresponding to the “data mapping structure.” An identifier (“content type identifier”) is necessarily a data structure, and as it is used for determining a corresponding material set identifier, it reads on a “mapping data structure.” Therefore, the examiner argues that the combination of Cheng/Barr/Evans/Zhu teach claim 21, as Zhu’s accessing of a content type identifier in order to look up a corresponding material set identifier does read on “accessing a mapping data structure” and using the mapping data structure (using the content type identifier) to determine a corresponding template identifier. Zhu’s use of a decision tree logic does not teach away or change the fact that Zhu directly determines a material set identifier (template identifier) using the content type identifier.
The examiner has updated the citations to Zhu as appropriate to further clarify the examiner’s position in the current § 103 rejection of claim 21 below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Such claim limitation(s) is/are:
“means for receiving a first media content item…” in claim 17
“means for receiving, by a second media service, a second template…” in claim 17
“means for processing, by the second media service, the second template…” in claim 17
“means for causing presentation of the second media content…” in claim 17
The specification indicates these various “means” for receiving, processing, and receiving in ¶ 0118 of the filed specification as “Example 17,” and further describes the same functions that are claimed in claim 17 as being performed by at least one processor or a processor in ¶ 0102 and ¶ 0110. The at least one processor is further detailed in ¶ 0092/Figure 8. Therefore, the corresponding structure to the various “means for receiving,” “means for processing,” and “means for presenting” is interpreted as being at least one processor.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-7, 9-15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over US 20140328570 A1 to Cheng et al. (Cheng) in view of US 20150046842 A1 to Barr et al. (Barr), and further in view of US 20210150222 A1 to Evans et al. (Evans).
Claim 1: Cheng teaches:
A system (Cheng: Fig. 1 and ¶ 0014 computing system 100) comprising: at least one processor; at least one memory storage device storing instructions thereon that, when executed by the at least one processor, cause the system to perform operations (Cheng: ¶ 0014 computing system running machine readable instructions, with ¶ 0065-0067 showing computing system 100 implemented using one or more processors and memory, also see ¶ 0084) comprising:
With respect to the limitations:
receiving a first media content item comprising a plurality of individual media content items that were previously and independently published as separate posts by one or more users via a network-based interaction service,
the individual media content items arranged for sequential playback,
the first media content item having been generated by a first media service selecting and arranging the individual media content items in accordance with first criteria specified in a first template,
the first criteria specifying arrangement of the individual media content items in chronological order based on when the individual media content items were captured
Cheng teaches receiving first media content comprising a plurality of individual media content items from an external source/service, e.g. YouTube or Instagram, wherein the media content items may be previously tagged as retrieved from the source (Cheng: ¶ 0015-0016 showing multimedia input 102, i.e. first media content item, which as per ¶ 0017 “multimedia input” may refer to, among other things, a collection of digital images, a video, a collection of videos, or a collection of images and videos (where a “collection” includes two or more images and/or videos). References herein to a “video” may refer to, among other things, a relatively short video clip, an entire full-length video production, or different segments within a video or video clip (where a segment includes a sequence of two or more frames of the video); also see ¶ 0032 “The video collection 150 refers generally to one or more bodies of retrievable multimedia digital content that may be stored in computer memory at the computing system 100 and/or other computing systems or devices. The video collection 150 may include images and/or videos stored remotely at Internet sites such as YOUTUBE and INSTAGRAM, and/or images/videos that are stored in one or more local collections, such as storage media of a personal computer or mobile device (e.g., a “camera roll” of a mobile device camera application)…To the extent that any of the content in the collection 150 is already tagged with descriptions, any of the learning modules 152 can learn and apply those existing descriptions to the knowledge base 132 and/or the templates 142, 144”). Cheng further teaches that segments from the multimedia input may be identified using a first template (Cheng: ¶ 0025, ¶ 0028, ¶ 0056-0057 showing receiving template selection which is used to select the salient event segments for inclusion from the multimedia input; also see ¶ 0050-0051 showing salient event criteria and presentation templates used for selecting the segments).
Cheng does not explicitly teach that the individual media content items of the first media content item were individual media content items that were previously and independently published as separate posts by one or more users via a network-based interaction service arranged for sequential playback, that they are generated by a first media service selecting/arranging the individual media content items in accordance with first criteria specified in a first template, or that the first criteria specify arrangement of the individual media content items in chronological order based on when the individual media content items were captured, as claimed.
However, Barr teaches receiving a first media content item comprising a plurality of individual media content items (Barr: ¶ 0081-0085 showing obtaining a generated social media compilation made up of a plurality of individual social media content items) that were previously and independently published as separate posts by one or more users via a network-based interaction service (Barr: ¶ 0080, ¶ 0083, ¶ 0057 showing the items of social media content aggregated from social media service provider, includes any one or a combination of: (1) photographic social media entries, (2) graphical social media entries, (3) videographic social media entries), the individual media content items arranged for sequential playback (Barr: ¶ 0064 showing “the compilation generation module 304 arranges the items of social media content in chronological order”; also see ¶ 0084), the first media content item having been generated by a first media service selecting and arranging the individual media content items (Barr: ¶ 0080 “ the items of social media content are aggregated by using one or more application programming interfaces of one or more social media service providers”; also see ¶ 0057) in accordance with first criteria specified in a first template (Barr: ¶ 0072-0081, Fig. 4 step 401 showing the items of social media content are aggregated into compilation according to a plurality of received configuration information; e.g. see ¶ 0078 “the compilation configuration information inputted by the first account holder at block 401 includes sequence information for ordering the items of social media content, and the items of social media content are aggregated at block 402 in a desired sequence according to the sequence information”), the first criteria specifying arrangement of the individual media content items in chronological order based on when the individual media content items were captured (Barr: ¶ 0078 “the compilation configuration information inputted by the first account holder at block 401 includes sequence information for ordering the items of social media content, and the items of social media content are aggregated at block 402 in a desired sequence according to the sequence information” which as per ¶ 0084; see ¶ 0123-0126 showing that the date/time associated with items of social media content for sequencing corresponds to when the items were recorded, i.e. captured). Barr further teaches that after aggregation of the multiple social media content items into a compilation, the compilation is then passed to an editing module for editing (Barr: ¶ 0067-0068, ¶ 0085-0087, Fig. 3 editing module 306).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included receiving the social media compilation (i.e. first media content item) containing a plurality of chronologically arranged video clips/segments generated via a first service module of a social media application of Barr in the video highlight generation system of Cheng with a reasonable expectation of success of arriving at the claimed invention, with the motivation that “Given the growing use of numerous social media accounts and/or platforms in memorializing and reminiscing about memorable experiences, it would be beneficial to have a convenient, centralized means of providing a custom video and/or audio compilation of social media content that is aggregated from numerous social media accounts and/or platforms, and to enable the compilation to be easily shared and enjoyed by many users” (Barr: ¶ 0006).
Cheng, as modified above (such that the sequenced compilation video content comprising a template of initial social media content items is provided/generated via a first social media application/service as per Barr above), further teaches:
receiving, by a second media service (Cheng: Fig. 1 and ¶ 0014 showing “ multimedia content understanding and assistance computing system 100”, which provides as per ¶ 0025, ¶ 0060 a video creation service), a second template for generating an abbreviated version of the first media content item (Cheng: ¶ 0025-0026, ¶ 0028-0029, ¶ 0031 showing a presentation template is selected and received by the output generator module, wherein the templates “provide the specifications that the output generator module 114 uses to select salient event segments 112 for inclusion in the visual presentation 120, arrange the salient event segments 112, and create the visual presentation 120”; also see ¶ 0015, ¶ 0025-0026, ¶ 0060-0061 clarifying that the templates are used to create a highlight video/reel; and ¶ 0013-0015 indicating that the point is to extract the most important moments or scenes from a lengthier video),
the second template specifying second criteria including at least i) a number of media content items to be selected from the plurality of individual media content items (Cheng: ¶ 0028 “a presentation template 142 specifies, for a particular event type, the type of content to include in the visual presentation 120, the number of salient event segments 112), and ii) content selection criteria for selecting the number of individual media content items from the plurality of individual media content items (Cheng: ¶ 0028 “The presentation templates 142 provide the specifications that the output generator module 114 uses to select salient event segments 112 for inclusion in the visual presentation 120, arrange the salient event segments 112, and create the visual presentation 120. For example, a presentation template 142 specifies, for a particular event type, the type of content to include in the visual presentation 120, the number of salient event segments 112, the order in which to arrange the segments 112, (e.g., chronological or by subject matter), the pace and transitions between the segments 112…”; also see ¶ 0021, ¶ 0030, ¶ 0050-0051, ¶ 0056-0057 showing identifying the salient event segments using salient event criteria);
With respect to the following limitations, Cheng teaches that the content selection criteria specifies characteristics for identifying salient (i.e. important) or engaging segments (Cheng: ¶ 0045, ¶ 0059, ¶ 0028-0029), but Cheng/Barr do not explicitly teach the following. However, Evans teaches:
wherein the content selection criteria specifies characteristics for identifying engaging media content items based on user interactions received when the media content items were-presented independently via a network-based interaction service, the user interactions comprising at least one of: likes, comments, shares, or views (Evans: ¶ 0023 showing using criteria for identifying “noteworthy portions” of a media content, i.e. segments of the media content, based on user engagement levels, via an online social network, including a number of shares, a number of “likes,” “love,” “haha,” “wow,” “sad,” or “angry” user-engagement actions, etc.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included the identifying of noteworthy segments of a media content item based on social user-engagement interactions of Evans in the highlight generation system of Cheng/Barr with a reasonable expectation of success of arriving at the claimed invention, with the motivation that “it is imperative to foster growth in user demand for such content by providing functionalities for sharing, promoting, and discovering such content. Highlight reels have long been popular in sports, and they have traditionally served the purpose of merely compiling noteworthy moments in a sporting event. This disclosure proposes using highlights as a solution to the task of growing user demand for video content and for improving user experience when viewing video content. The disclosure contemplates predicting portions of a video that are noteworthy and presenting these portions as highlights that may be quickly shared by users, for example, on an online social network. This sharing may allow other users to discover the video or pique their interest in viewing related videos. It may also increase user engagement with the highlight or the video from which the highlight was extracted” (Evans: ¶ 0005).
Cheng, as modified above, further teaches:
processing, by the second media service, the second template by: selecting the number of individual media content items from the plurality of individual media content items in accordance with the content selection criteria (Cheng: ¶ 0028, ¶ 0051, ¶ 0057 showing selecting the plurality of segments (“salient event segments”) based on the template and salient event criteria); and
generating a second media content item from the number of individual media content items (Cheng: ¶ 0057 “At block 332, the system 100 identifies the salient event segments 112 in the multimedia input file(s)…the computing system 100 generates the visual presentation 120 (e.g., a video clip or “montage”), and/or the NL description 122, using, e.g., the templates 142, 144 as described above (block 338)”; also ¶ 0015 “The computing system 100 can compile the salient event segments into a visual presentation 120 (e.g., a “highlight reel” video clip) that can be stored and/or shared over a computer network”; and ¶ 0025 “The visual presentation generator module 116…incorporates the extracted segments 112 into a visual presentation 120, such as a video clip (e.g., a “highlight reel”) or multimedia presentation, using a presentation template 142”),
the second media content item being the abbreviated version of the first media content item (Cheng: ¶ 0015, ¶ 0060-0061 showing it is a highlight video/reel; and ¶ 0013-0015 indicating that the point is to extract the most important moments or scenes from a lengthier video); and
causing presentation of the second media content item via the network-based interaction service (Cheng: ¶ 0034 showing “The sharing module 128 is responsive to user interaction with the computing system 100 that indicates that the user would like to “share” the newly created or updated presentation 120 with other people, e.g., over a computer network, e-mail, a messaging service, or other electronic communication mechanism. The illustrative sharing module 128 can be pre-configured or user-configured to automatically enable sharing in response to the completion of a presentation 120, or to only share the presentation 120 in response to affirmative user approval of the presentation 120. In either case, the sharing module 128 is configured to automatically share (e.g., upload to an Internet-based photo or video sharing site or service) the presentation 120 in response to a single user interaction (e.g., “one click” sharing)” and ¶ 0060 “The user can review the clip and instruct the service to save the clip, download it, and/or post/share it on a desired social network or other site”)
Claim 2: Cheng/Barr/Evans teach claim 1. Cheng, as modified above, further teaches:
wherein selecting the number of individual media content items in accordance with the content selection criteria further comprises: selecting individual media content items based on a metric indicating an engagement level associated with each media content item (Cheng: ¶ 0059 “The system 100 can extract the segments 520, 522, 524, 526 from the video 510 and incorporate the segments 520, 522 524, 526 into a video clip that includes only the most interesting or salient portions of the video 510”);
selecting individual media content items based on a measure of virality associated with each media content item;
selecting individual media content items based on user interactions associated with each media content item (Cheng: ¶ 0033 showing the user may also modify the arrangement by selecting and/or arranging the media content items for inclusion as desired; also see ¶ 0057, ¶ 0061 showing user interaction with the segments to edit the presentation; ¶ 0050 “the interactive storyboard module 124 may determine salient event criteria based on user inputs received by the editing module 126”);
or, a combination thereof
Claim 3: Cheng/Barr/Evans teach claim 1. Cheng, as modified above, further teaches:
wherein selecting the number of individual media content items in accordance with the content selection criteria further comprises: selecting individual media content items to provide diversity in content types;
selecting individual media content items to provide variety in subject matter depicted;
selecting individual media content items based on user preferences (Cheng: ¶ 0037 “The illustrative computing system 100 also includes a user preference learning module 148. The user preference learning module 148 is embodied as software, firmware, hardware, or a combination thereof. The user preference learning module 148 monitors implicit and/or explicit user interactions with the presentation 120 (user feedback 146) and executes, e.g., machine learning algorithms to learn user-specific specifications and/or preferences as to, for example, the types of activities that the user considers to be “salient” with respect to particular events, the user's specifications or preferences as to the ordering of salient events in various types of different presentations 120, and/or other aspects of the creation of the presentation 120 and/or the NL description 122. The user preference learning module 148 updates the templates 142, 144 and/or portions of the knowledge base 132 (e.g., the salient event criteria 138) based on its analysis of the user feedback 146”; also see ¶ 0033 showing user edits used to select/arrange the generated video/presentation of salient event segments, which also reads on user preferences); or
a combination thereof
Claim 4: Cheng/Barr/Evans teach claim 1. Cheng, as modified above, further teaches:
wherein the operations further comprise: receiving a selection to modify the second media content item (Cheng: ¶ 0033, ¶ 0057 and ¶ 0061 showing user modification of the generated presentation 120);
in response to receiving the selection, providing an interface for modifying the second media content item (Cheng: ¶ 0033, ¶ 0057, and ¶ 0061 interactive storyboard module displaying the presentation and allowing user to make modifications, i.e. a user interface);
receiving one or more modifications to the second media content item via the interface (Cheng: ¶ 0033, ¶ 0057, ¶ 0061 user makes modifications to review edit the presentation before sharing);
generating a modified second media content item based on the one or more modifications (Cheng: ¶ 0033 “When the user's interaction with the presentation 120 is complete, the interactive storyboard module 124 stores the updated version of the presentation 120”); and
presenting the modified second media content item via the user interface (Cheng: ¶ 0033, ¶ 0057, ¶ 0060-0061 the user may view the modified clip while making the modifications and review before sharing)
Claim 5: Cheng/Barr/Evans teach claim 1. Cheng, as modified above, further teaches:
wherein selecting the number of individual media content items comprises: inputting a video content item selected based on the content selection criteria into a machine learning model (Cheng: ¶ 0053 the multimedia input is received, wherein as per ¶ 0029-¶ 0030, ¶ 0037, ¶ 0058 machine learning models are used to analyze the input video to identify salient events segments);
analyzing, by the machine learning model, the video content item to identify a segment of the video content item representing a highlight portion (Cheng: ¶ 0029-¶ 0030, ¶ 0037, ¶ 0058 as above, showing analyzing the video to detect salient segments within the video); and
extracting the identified highlight portion of the video content item as one of the number of individual media content items for inclusion in the second media content item (Cheng: ¶ 0023, ¶ 0028, ¶ 0055-0057 extracting the salient event segments used to generate the presentation 120)
Claim 6: Cheng/Barr/Evans teach claim 1. Cheng, as modified above, further teaches:
wherein the machine learning model is trained to identify the highlight portion of a video content item by at least one of: analyzing user engagement metrics associated with a plurality of video content items to determine segments with high engagement as highlights for training data (Cheng: ¶ 0037 “The user preference learning module 148 is embodied as software, firmware, hardware, or a combination thereof. The user preference learning module 148 monitors implicit and/or explicit user interactions with the presentation 120 (user feedback 146) and executes, e.g., machine learning algorithms to learn user-specific specifications and/or preferences as to, for example, the types of activities that the user considers to be “salient” with respect to particular events, the user's specifications or preferences as to the ordering of salient events in various types of different presentations 120, and/or other aspects of the creation of the presentation 120 and/or the NL description 122. The user preference learning module 148 updates the templates 142, 144 and/or portions of the knowledge base 132 (e.g., the salient event criteria 138) based on its analysis of the user feedback 146”);
receiving input identifying highlight segments in a plurality of video content items to generate training data (Cheng: ¶ 0050 using previous salient events and videos as training data, and receiving user input that is used to generate training data, and see “The salient event criteria 138 may also include salient event criteria that is specified by or derived from user inputs 256. For example, the interactive storyboard module 124 may determine salient event criteria based on user inputs received by the editing module 126. As another example, the user preference learning module 148 may derive new or updated salient event criteria based on its analysis of the user feedback 146”); and
analyzing audiovisual features such as motion, speech, scene changes, facial expressions, or lighting in a plurality of video content items to determine highlight segments for training data (Cheng: ¶ 0030 “the salient event learning module 158 may identify a new event or activity for inclusion in the mapping 140, or identify new salient event criteria 138, based on the frequency of occurrence of certain features and/or concepts in the collection 150. The salient event learning module 158 can also identify multi-modal salient event markers including “non visual” characteristics of input videos such as object motion, changes in motion patterns, changes in camera position or camera motion, amount or direction of camera motion, camera angle, audio features (e.g., cheering sounds or speech)”)
Claim 7: Cheng/Barr/Evans teach claim 1. Cheng, as modified above, further teaches:
wherein generating the second media content item comprises: analyzing, by the processor, an audio track specified by the template to detect audio cues within the audio track (Cheng: ¶ 0043 “The illustrative audio feature detection module 214 analyzes the audio track of an input 102 using mathematical sound processing algorithms and uses the audio feature model 238 (e.g., an acoustic model) to detect and classify audio features 222. For example, the audio feature detection module 214 may detect an acoustic characteristic of the audio track of a certain segment of an input video 102, and, with the audio feature model 238, classify the acoustic characteristic as indicating a “cheering” sound or “applause.”);
determining time locations of the detected audio cues within the audio track (Cheng: ¶ 0043-0044 showing identifying spoken works at a particular frame/point within the audio track);
analyzing each of the selected number of media content items to determine appropriate start and end times for presentation based on the time locations of the detected audio cues (Cheng: ¶ 0064 showing defining beginning and end frames of segments, wherein as per above the automated identification of salient event segments are identified at least in part based on the audio analysis in ¶ 0043-0044; also see ¶ 0046-0051);
arranging the selected number of media content items according to the determined start and end times for each media content item to synchronize transitions between the selected number of media content items with the detected audio cues in the specified audio track (Cheng: ¶ 0028 showing “a presentation template 142 specifies, for a particular event type, the type of content to include in the visual presentation 120, the number of salient event segments 112, the order in which to arrange the segments 112, (e.g., chronological or by subject matter), the pace and transitions between the segments 112, the accompanying audio or text, and/or other aspects of the visual presentation 120”); and
combining the arranged media content items with the specified audio track to generate the second media content item having synchronized transitions (Cheng: ¶ 0028 “The presentation templates 142 provide the specifications that the output generator module 114 uses to select salient event segments 112 for inclusion in the visual presentation 120, arrange the salient event segments 112, and create the visual presentation 120. For example, a presentation template 142 specifies, for a particular event type, the type of content to include in the visual presentation 120, the number of salient event segments 112, the order in which to arrange the segments 112, (e.g., chronological or by subject matter), the pace and transitions between the segments 112, the accompanying audio or text, and/or other aspects of the visual presentation 120”; see ¶ 0057 and ¶ 0062 creating the presentation including transitions, from the arrange segments extracted from the multimedia input, which as per above included an audio track)
Note: Under the broadest reasonable interpretation, the limitation arranging the selected number of media content items according to the determined start and end times for each media content item to synchronize transitions between the selected number of media content items with the detected audio cues in the specified audio track, does not require that the arranging to synchronize transitions is based on the detected audio cues specifically, and instead just requires that the media content items are media content items with the detected audio cues.
Claim 9: See the rejection of claim 1 above teaching analogous limitations. Cheng further teaches: A method (Cheng: Fig. 3, ¶ 0053-0057, ¶ 0060-0061 processes for generating highlight videos).
Claim 10: See the rejection of claim 2 above teaching analogous limitations.
Claim 11: See the rejection of claim 3 above teaching analogous limitations.
Claim 12: See the rejection of claim 4 above teaching analogous limitations.
Claim 13: See the rejection of claim 5 above teaching analogous limitations.
Claim 14: See the rejection of claim 6 above teaching analogous limitations.
Claim 15: See the rejection of claim 7 above teaching analogous limitations.
Claim 17: See the rejection of claim 1 above. Cheng further teaches means for receiving, means for processing, and means for displaying (Cheng: Fig. 6, ¶ 0065-0066 processor, memory, communications systems, and display device(s)). Note as per the § 112(f) interpretation of claim 17, the structure for “means for receiving,” “means for processing,” and “means for causing presentation” appear to correspond to a processor, which is similar to claim 1.
Claim 18: See the rejection of claim 2 above teaching analogous limitations.
Claim 19: See the rejection of claim 3 above teaching analogous limitations.
Claim 20: See the rejection of claim 4 above teaching analogous limitations.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over US 20140328570 A1 to Cheng et al. (Cheng) in view of US 20150046842 A1 to Barr et al. (Barr), further in view of US 20210150222 A1 to Evans et al. (Evans), and further in view of US 20210264952 A1 to Zhu et al. (Zhu).
Claim 21: Cheng/Barr/Evans teach claim 1. With respect to the limitations:
wherein receiving the second template comprises: accessing a mapping data structure that associates different types of first media content items with corresponding template identifiers;
determining a content type associated with the first media content item;
performing a lookup operation using the mapping data structure to identify a template identifier corresponding to the determined content type; and
retrieving the second template based on the identified template identifier
Cheng teaches wherein receiving the second template comprises: accessing a mapping data structure that associates different types of first media content items with corresponding templates (Cheng: ¶ 0020-0022 showing mapping activities to an event, wherein as per ¶ 0060 activities corresponding an event type and associated with a stored template); determining a content type associated with the first media content item (Cheng: ¶ 0060 showing the service identifies an event type from analyzing the input video files and key activities typically associated with that event type); performing a lookup operation using the mapping data structure to identify and retrieve a template corresponding to the determined content type (Cheng: ¶ 0058-0060, with ¶ 0058 showing “mapping 140 defines logical links or connections 420, 422, 424, 426, 428 between the various types of detected features 410, concepts 412, relations 414, events 416, and salient activities 418. The system 140 can use the mapping 140 and particularly the links 420, 422, 424, 426, 428, in performing the semantic reasoning to determine events and salient activities based on the features 410, concepts 412, and relations 414”, and ¶ 0060 “the event type corresponds to a stored template identifying the key activities typically associated with that event type. For example, for a birthday party, associated activities would include blowing out candles, singing the Happy Birthday song, opening gifts, posing for pictures, etc. Using feature detection algorithms for complex activity recognition, such as those described above and in the aforementioned priority patent applications, the service automatically identifies segments (sequences of frames) within the uploaded file that depict the various key activities or moments associated with the relevant event type”).
Cheng/Barr/Evans do not explicitly teach that the mapping data structure is used to associate types of media content items with template identifiers, performing the lookup operation to identify a template identifier, or retrieving the second template based on the identified template identifier.
However, Zhu teaches analyzing and determining types of content located in first media content/video to determine a content type identifier, and determining a material set identifier corresponding to the content type identifier to retrieve relevant video templates/”video editing material sets” that are retrieved according to a material set identifiers, (Zhu: Fig. 6 steps s604-s607 and ¶ 0090-0096, ¶ 0007, ¶ 0034-0037, ¶ 0043, ¶ 0050-0052, ¶ 0070-0075; and Fig. 7 showing template recommendation including a number of templates; specifically see ¶ 0034-0037, ¶ 0043 showing accessing and determining a content type identifier associated with each content element in a video, i.e. accessing a mapping data structure used to lookup a material set identifier, and determining a material set identifier corresponding to the content type identifier, i.e. performing a lookup of the matching material set for that content type identifier; ¶ 0050-0052, ¶0070-0075 showing material set for video editing corresponds to video editing templates). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included the analyzing and mapping data included in video content in order to determine and retrieve a video editing material set/template according to material set identifier(s) as taught by Zhu in the video highlight generation system of Cheng/Barr/Evans with a reasonable expectation of success of arriving at the claimed invention, with the motivation that to “improve the quality of video editing and diversify video editing experience, editing templates may be variously generated and determined according to contents of a video” (Zhu: ¶ 0005) and solve the problem that “In a conventional video editing technique, an editing template may include fixed material or editing schemes, and therefore, images or videos edited using the same editing template may result in the same or substantially the same images or videos in style and may be homogenous. Thus, the conventional video editing technique does not provide diversified video editing techniques for a user, thereby degrading the user experience” (Zhu: ¶ 0025). It would have also been obvious to one of ordinary skill in the art before the effective filing date of the invention to do so, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over US 20140328570 A1 to Cheng et al. (Cheng) in view of US 20150046842 A1 to Barr et al. (Barr), further in view of US 20210150222 A1 to Evans et al. (Evans), and further in view of US 20220060788 A1 to Harling et al. (Harling).
Claim 22: Cheng/Barr/Evans teach claim 1. With respect to the following limitations, Cheng/Barr/Evans do not explicitly teach the following, however Harling teaches:
wherein selecting the number of individual media content items in accordance with the content selection criteria comprises: analyzing an original sequential arrangement of the individual media content items within the first media content item (Harling: ¶ 0043-0044 showing “The segments used to create the new mp4 file are in chronological order from a stored loop of segments”);
identifying positional relationships between the individual media content items based on the original sequential arrangement; and selecting the individual media content items while preserving the positional relationships from the original sequential arrangement in the second media content item (Harling: ¶ 0043-0044 showing a series of video segments from a video loop, are selected and stitched together into a new mp4 file (referred to as a “moment”) that comprises several of the segments from the video loop and preserves the chronological order of the original video loop; also see ¶ 0044 specifying “Preferably, the number of segments that form a loop is ten times greater than the number of segments within a moment. Skilled artisans will appreciate that the number of segments that form a loop can range from one segment more than the number of segments that form a moment to twenty times to thirty times or more than the number of segments that form a moment,” and therefore it is clear that “The segments used to create the new mp4 file are in chronological order from a stored loop of segments” as per ¶ 0043 are selected to be in the moment are selected from a total number of segments in the loop, but remain in chronological order)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have included selecting and arranging a subset of video segments while preserving chronological order as taught by Harling in the video highlight generation system of Cheng/Barr/Evans with a reasonable expectation of success of arriving at the claimed invention, with the motivation to make it “easier to save highlights or moments of events for later viewing and sharing” (Harling: ¶ 0008). It would have also been obvious to one of ordinary skill in the art before the effective filing date of the invention to do so, since the claimed invention is merely a combination of old elements (including the preservation of a chronological order in a second highlight video file from a first video file of Harling, in the creation of the second video file as per Cheng/Barr/Evans in the rejection of claim 1 above), and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hunter Molnar whose telephone number is (571)272-8271. The examiner can normally be reached Monday - Friday, 7:30 - 4:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeffrey Zimmerman can be reached at (571)272-4602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUNTER MOLNAR/Examiner, Art Unit 3628
Prosecution Insights