DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments, filed 06/14/2024, have been entered and made of record. Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because of the new ground of rejection sets forth below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-9, 11-15 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Goldberg (US 2014/0169767) in view of Frigon (US PG PUB 2014/0267749) and further in view of Bloch (US Pat. No. 9, 190, 110).
Regarding claim 1, Goldberg discloses a system comprising:
one or more data storage devices storing a primary video and candidate secondary videos (see paragraph 0062 events are captured as audio/video by recording device and digitized recording are stored in an asset database; see figure 11 which shows events); and
a processing device configured for executing a rendering engine for inserting secondary video content from one or more of the secondary videos into primary video content of the primary video, wherein inserting secondary video content comprises: identifying a target portion of the primary video based on a selection via a navigable transcript of the primary video content, wherein the target portion of the primary video comprises primary video frames and primary audio portions; and detecting a user selection of one or more secondary video frames of the one or more candidate secondary videos (see paragraph 0037 selecting a word in the transcript locates the corresponding recorded content; see paragraph 0188 the user can search video clips based on any combination of text, project name, labels, recommended clips and other factors including options regarding what parts of the event(s) should be included; see Table 1, move an audio/video clip from one event to another event; see paragraphs 0038, 0062 and 0144 and the reason discussed in the previous office action).
Claim 1 differs from Goldberg in that the claim further requires replacing at least one of the primary video frames in the target portion of the primary video at which the primary audio portion is played with the selected one or more secondary video frames without changing the primary audio portion of the target portion of the primary video when performing a playback operation.
In the same field of endeavor Frigon discloses replacing at least one of the primary video frames in the target portion of the primary video at which the primary audio portion is played with the selected one or more secondary video frames without changing the primary audio portion of the target portion of the primary video when performing a playback operation (see paragraph 0039 publish different video streams; see paragraph 0040 video feeds ae transitioned from one to another; see paragraph 0042 the user is allowed to select a sound as desired and the editing software is capable of separating sound from video streams; see also claim 14 of the prior art).
Therefore, in light of the teaching in Frigon it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Goldberg by replacing primary with secondary video without changing the audio portions in order to allow the user to listen to the sound of her or his choice or mute a sound.
Claim 1 further differs from the combination of Goldberg and Frigon in that the claim further requires replacing the primary video automatically.
In the same field of endeavor Bloch discloses a system and method for assembling segments of recorded music or video from among various versions or variations into a new version or composition (see abstract). Borch further discloses automatically playing a default or random audio or video segment from the second alternative audio or video segments at the location of intended insertion instead of the selected second alternative audio or video segment, wherein the first playback of the video presentation results in a first one of the predefined versions of the video presentation; and iv) during a second, later playback of the video presentation: automatically playing the selected second alternative video segment at the location of intended insertion without requiring interaction from the user, wherein the second playback of the video presentation results in a second one of the predefined versions of the video presentation (see claim 1 of the prior art).
Therefore, in light of the teaching in Bloch it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above combination by automatically adding different version of video segment in order to reduce operator’s burden and perform seamless reproduction.
Regarding claim 2, Goldberg discloses identifying a first time stamp, a second time stamp, and a third time stamp that are associated with playback of the primary video content, wherein the second time stamp corresponds to a portion of the primary video content identified via a text-based navigation section having selectable portions of a transcript and wherein the first time stamp corresponds to a start of the target portion of the primary video; and performing a playback operation that comprises: retrieving the primary video frames from a primary video file that includes the primary video content and that lacks any content from the secondary video content wherein the primary video frames comprise first frames and second frames; rendering the first frames from the primary video content for display between the first time stamp of the primary video content and the second time stamp of the primary video content, rendering the selected one or more secondary frames retrieved from the secondary video content for display starting at the second time stamp and continuing for a duration between the second time stamp and the third time stamp, and rendering the second frames from the primary video content for display starting from the third time stamp of the primary video content (see paragraph 0149 desired positions of recordings are associated with timestamps; timestamps associated with text; the word whose associated timestamp is the closest match to the current time timestamp; see Table 4, use the project tree structure to copy and/or move project pieces and/or project sections between projects; see Table 1, paragraph 0178; see also claims 1-2, 5, paragraph 0229; and the reason discussed above).
Regarding claim 3, Goldberg discloses a means for generating a query for candidate secondary videos to be inserted into primary video content based on selections via a navigable transcript of the primary video content (see paragraph 0216 footage needed from another video; see Table 4, move project pieces and/or projects sections between projects; see also paragraph 0188 and Table 1).
Regarding claim 5, Goldberg discloses receiving a target term via a search field; generating a candidate video query having a query parameter that includes or is derived from the received target term; retrieving candidate secondary video content by submitting the candidate video query to one or more data sources having the candidate secondary video content; and displaying selectable visual representations of the retrieved candidate secondary video content (see abstract, figure 11, paragraphs 0161, 0164, 0186, 0188 and 0238).
Regarding claim 6, Goldberg discloses subsequent to insertion of the selected one or more secondary video into the primary video content, audio content associated with a portion of the primary video content that is (1) identified via a text-based navigation section having selectable portions of a transcript a portion of the primary video content and (ii) playable with the selected one or more secondary video frames is included with the inserted secondary video content (see figure 12 and paragraphs 0148, 0201 and 0223).
Regarding claim 7, the limitation of claim 7 can be found in claim 1 above. Therefore claim 7 is analyzed and rejected for the same reasons as discussed in claim 1 above. See also claim 15 of the applied prior art.
Claims 8-9 and 11-12 are rejected for the same reasons as discussed in claims 2-3 and 5-6 respectively above.
Regarding claim 13, the limitation of claim 13 can be found in claim 1 above. Therefore claim 13 is analyzed and rejected for the same reasons as discussed in claim 1 above.
Claims 15 and 17-18 are rejected for the same reasons as discussed in claims 3 and 5-6 respectively above.
Regarding claim 19, Goldberg discloses selecting a portion of the transcript corresponding to a text-selection input received at the text-based navigation section (see abstract quickly find and access portions of transcripts and the corresponding video segments within a single, multi-video project and across multiple projects; see Table 2 portion of text selected; see paragraph 0198);
identifying a portion of the primary video corresponding to the selected portion of the transcript (see abstract portions of transcripts and the corresponding video segments within a single, multi-video project and across multiple projects; see paragraph 0207 Panel E allows the viewer to view video that corresponds to the selected transcript(s));
selecting, from the candidate secondary videos, a secondary video corresponding to a video-selection input received at the secondary video menu section (see paragraph 0190 select the representative thumbnail displayed in panel B); and
inserting the selected secondary video into the primary video at the identified portion of the primary video (see paragraph 0216 footage needed from another video; see Table 4, move project pieces and/or projects sections between projects).
Regarding claim 20, Goldberg discloses inserting the selected secondary video into the primary video at the identified portion of the primary video comprises: identifying a first time stamp, a second time stamp, and a third time stamp that are associated with playback of the primary video, wherein the second time stamp corresponds to the identified portion of the primary video; and performing a playback operation in the video editing interface, wherein the playback operation comprises: rendering first frames from the primary video for display between the first time stamp of the primary video and the second time stamp of the primary video, determining that the selected secondary video has been selected for insertion into the primary video rendering frames retrieved from the secondary video for display starting at the second time stamp and continuing for a duration between the second time stamp and the third time stamp, and rendering second frames from the primary video for display starting from the third time stamp of the primary video (see Table 4 use the project library tree structure to copy and/or move project pieces and/or projects sections between projects; see paragraph 0149 timestamp associated with desired positions; see paragraph 0229; claim 2 rejection above and also claims 1-2 and 5 of the applied prior art).
Claims 4, 10 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Goldberg (US 2014/0169767) in view of Frigon (US PG PUB 2014/0267749) and further in view of Bloch (US Pat. No. 9, 190, 110) and Omote et al. (US PG PUB 2019/0341025 hereinafter referred as Omote).
Regarding claim 4, although the combination of Goldberg, Frigon and Bloch discloses the limitation of claim 1 and accessing a set of words included in a transcript of the primary video content (see Goldberg’s paragraphs 0058, 0090 and 0188), the combination fails to specifically disclose computing a set of target term probabilities for the set of words, wherein, for each word, a respective target term probability is computed by performing operations comprising: generating a frequency feature vector representing a frequency of the word within the transcript, a sentiment feature vector representing sentiments associated with the word within the transcript, and a part-of-speech feature vector representing syntaxes of the word within the transcript, combining the frequency feature vector, the sentiment feature vector, and the part-of-speech feature vector into a target feature vector for the word, and computing the respective target term probability by applying a recommendation machine-learning model to the target feature vector, wherein the recommendation machine- learning model is trained to associate training target feature vectors with training words tagged as secondary video content search terms in training transcripts; and selecting, from the set of words, target terms having respective target term probabilities that exceed a threshold probability, wherein one or more of the target terms are included in the query for candidate secondary videos.
In the same field of endeavor Omote discloses a method where frequency information are extracted from the audio; and classifying fundamental frequency information, textual information and video feature information using multimodal neural network (see abstract). See claim 10 of the applied prior art where the claim recites the fundamental frequency information is classified for each word or viseme. In paragraph 0066 Omote further discloses ‘linguistic feature analysis according to aspects of the present disclosure uses text input generated from either automatic speech recognition or directly from a text input such as an image caption and generates feature vectors for the text. the resulting feature vector may be language dependent, as in the case of word embedding and part of speech, or language independent, as in the case of sentiment score and word count or duration. In some embodiments these word embeddings may be generated by such systems a SentiWordNet in combination with other text analysis systems known in the art. These multiple textual features are combined to form a feature vector that is input to the multimodal neural network for emotion classification”. Furthermore, in paragraph 0051 Omote discloses the fundamental frequency data may be modified for multimodal processing using average of fundamental frequency (F0) estimations and a voicing probability. By way of example and not by way of limitation F0 may be estimated every 10 ms and averaging. Every 25 consecutive estimates that contain a real F0 may be averaged. Each F0 estimate is checked to determine whether contains a voice. Each F0 estimate value is checked to determine if the estimate is greater than 40 Hz. If the F0 estimate is greater than 40 Hz then the frame is considered voiced and as such the audio contains a real F0 and is included in the average. If the audio signal in the sample is lower than 40 Hz, that F0 sample is not included in the average and the frame is considered unvoiced. The voicing probability is estimated as followed: (Number voiced frames)/(Number voiced+Number of unvoiced frames over a signal segment). The F0 averages and the voicing probabilities are estimated every 250 ms or every 25 frames. The speech or signal segment is 250 ms and it includes 25 frames. According to some embodiments the system estimated 4 F0 average values and 4 voicing probabilities every second. The four average values and 4 voicing probabilities may then be used are as feature vectors for multimodal classification of user characteristics. It should be note that the system may generate any number of average values and voice probabilities for use with the multimodal neural network. See paragraph 0085 where Omote discloses the overall structure and probabilities of the NNs is stored as data in the mass store. See claims 9 and 32 of the applied prior art where the art recites classifying the text representation of the audio comprises extracting a part of speech vector and sentiment lexical feature vector; see claim 12 which recites generating sentence level embeddings and identifying attention features before generating a single fusion vector and classifying the fundamental frequency information and the other feature information using a multimodal neural network.
Therefore, in light of the teaching in Omote it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the above combination by computing a set of target term probabilities and generating a frequency feature vector as claimed in order to combine multiple textual features and to form a feature vector that is input to the multimodal neural network for emotion classification.
Claims 10 and 16 are rejected for the same reasons as discussed in claim 4 above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HELEN SHIBRU whose telephone number is (571)272-7329. The examiner can normally be reached M-TR 8:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI TRAN can be reached on 571 272 7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HELEN SHIBRU/ Primary Examiner, Art Unit 2484 June 15, 2024
Prosecution Insights