DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Remarks
This is a reply to the application filed on 04/01/2025, in which, claims 1-7, 9, and 12-23 remain pending in the present application with claims 1, 9, and 18 being independent claims.
When making claim amendments, the applicant is encouraged to consider the references in their entireties, including those portions that have not been cited by the examiner and their equivalents as they may most broadly and appropriately apply to any particular anticipated claim amendments.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on April 01, 2025 and November 21, 2025 are in compliance with the provisions of 37 CFR 1.97 and are being considered by the Examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 9, and 12-23 are rejected under 35 U.S.C. 103 as being unpatentable over Tian et al. (US 20200320306 A1, hereinafter referred to as “Tian”) in view of Krishnamurthy et al. (US 20200134316 A1, hereinafter referred to as “Krishnamurthy”).
Regarding claim 1, Tian discloses a method for video generation, comprising:
obtaining an original video (see Tian, paragraph [0062]: “The news may also be just text, image or video. In practice, the obtained news may also be marked with a corresponding news event”), and segmenting the original video based on video content in the original video to obtain at least two video segments (see Tian, paragraph [0063]: “the executing body may semantically segment videos in the respective news included in the target news cluster, to obtain at least one video segment, and determine the obtained video segments as a target video set”); and
generating a target video according to the video segments and the corresponding commentary speeches (see Tian, paragraph [0156]: “The target video set and the target image set are combined to obtain the candidate material resource set corresponding to the commentary”).
Regarding claim 1, Tian discloses all the claimed limitations with the exception of generating commentary speeches corresponding to the video segments, the commentary speeches being used to describe picture content of the video segments.
Krishnamurthy from the same or similar fields of endeavor discloses generating commentary speeches corresponding to the video segments, the commentary speeches being used to describe picture content of the video segments (see Krishnamurthy, paragraph [0060]: “The classification of the action generated by the third Action Description NN 303 may be provided to the control module 101, e.g., in the form of text describing the action occurring in the window” and paragraph [0065]: “The Scene Annotation module 120 may generate text describing these scene elements and combine the text with image data to form a caption for the scene. For example and without limitation for the scene shown in FIG. 4, the Scene Annotation module 120 may produce a caption like “It is a sunny day by the sea, a sail boat floats in the distance. A road is in front of a wall.” Several neural networks may be used to generate the text”).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Krishnamurthy with the teachings as in Tian. The motivation for doing so would ensure the system to have the ability to use system and method disclosed in Krishnamurthy to generate the classification of the action in the form of text describing the action occurring in the window and to generate text describing these scene elements and combine the text with image data to form a caption for the scene thus generating commentary speeches corresponding to the video segments wherein the commentary speeches being used to describe picture content of the video segments in order to generate output image frame data containing closed captioning and audio data that includes text to speech (TTS) descriptions of corresponding images so that the commentary speech can be used to help visually impaired individuals clearly understand the content of the video through the descriptive audio.
Regarding claim 2, the combination teachings of Tian and Krishnamurthy as discussed above also disclose the method of claim 1, wherein generating the commentary speeches corresponding to the video segments comprises:
performing a content recognition on the video segments to obtain content information representing picture content of the video segments (see Tian, paragraph [0175]: “When it is determined in step 20421 that the candidate material resource is an image, the image candidate material resource is semantically segmented by various implementations to obtain at least one semantic labeling result corresponding to the image candidate material resource, where a semantic labeling result may be a word or phrase”);
generating corresponding commentary texts according to the content information (see Tian, paragraph [0175]: “For example, if the image candidate material resource describes that a lion is chasing an antelope on a grassland, the at least one semantic labeling result obtained by semantically segmenting the image may be {“lion”, “grassland”, “antelope”}”); and
generating the commentary speeches according to the commentary texts (see Tian, paragraph [0213]: “for each paragraph in the commentary, generating a speech corresponding to the paragraph by using a speech synthesis technology”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 3, the combination teachings of Tian and Krishnamurthy as discussed above also disclose the method of claim 2, wherein generating the commentary speeches according to the commentary texts comprises:
determining target speech rates of the commentary speeches according to the content information of the video segments (see Tian, paragraph [0203]: “The corresponding playing duration of the paragraph is determined by the number of words in the paragraph and a default speech rate”); and
generating the commentary speeches based on the target speech rates (see Tian, paragraph [0202]: “for each paragraph in the commentary, candidate material resources sequentially arranged are selected from the candidate material resource set determined in step 403, to constitute the candidate material resource sequence corresponding to the paragraph, and the playing duration of the selected candidate material resource sequence corresponding to the paragraph is equal to the corresponding playing duration of the paragraph”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 4, the combination teachings of Tian and Krishnamurthy as discussed above also disclose the method of claim 2, wherein the method further comprises:
obtaining video speeches of the video segments, and extracting first semantic features of the video speeches (see Tian, paragraph [0184]: “Each sample image is semantically segmented in step 20423′, and for each sample image, the specific operation of determining the semantic vector corresponding to the sample image based on the semantic vectors corresponding to semantic labeling results”);
obtaining second semantic features of the content information, and determining a feature similarity of the second semantic features and the first semantic features (see Tian, paragraph [0188]: “determining a similarity between a semantic vector corresponding to a paragraph in the commentary and a semantic vector corresponding to a candidate material resource in the candidate material resource set as the degree of matching between the paragraph and the candidate material resource”); and
wherein generating the corresponding commentary texts according to the content information, comprises:
in response to the feature similarities being less than a similarity threshold, generating the commentary texts according to the second semantic features (see Tian, paragraph [0189]: “The similarity between a semantic vector corresponding to a paragraph in the commentary and a semantic vector corresponding to a candidate material resource in the candidate material resource set may be calculated using various methods for calculating a similarity between vectors”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 5, the combination teachings of Tian and Krishnamurthy as discussed above also disclose the method of claim 2, wherein the content information comprises at least first information and second information, wherein the first information represents target characters in the video segments, and the second information represents target actions corresponding to the target characters (see Krishnamurthy, paragraph [0035]: “Action description requires multiple sequential video frames as input in order to determine the temporal relationship between the frames to determine the action classification. Scene Annotation, by contrast, is more concerned with relatively static elements of an image and can use a single screen shot as input”);
wherein generating corresponding commentary text according to the content information comprises:
generating at least one group of sub-texts according to the first information and the second information, the sub-texts being used to represent target characters and corresponding target actions in the video segments at corresponding playback moments (see Krishnamurthy, paragraph [0037]: “The Action Description module 110 and the Scene Annotation component module 120 may both generate information in the form of text information … Once generated, this text information may be converted into speech audio by the video tagging module 107 ... The video tagging module 107 may combine the speech output or audio with other audio output generated by the host system 102 for presentation to the user”); and
generating the commentary texts according to the sub-texts and the corresponding playback moments (see Krishnamurthy, paragraph [0063]: “the Action description module 110 may generate live commentary on the action in a simulated sporting event shown in the video stream from the host system”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 6, the combination teachings of Tian and Krishnamurthy as discussed above also disclose the method of claim 1, wherein the method further comprises:
obtaining video speeches in the video segments (see Krishnamurthy, paragraph [0028]: “the control module 101 may use the output data to generate output image frame data containing closed captioning and style/color transformations or audio data that includes text to speech (TTS) descriptions of corresponding images”);
determining second playback time periods corresponding to the commentary speeches according to first playback time periods of the video speeches (see Krishnamurthy, paragraph [0060]: “window level features data may range from 1 feature to 60 or 120 or more features depending on the framerate sent by the host system. The classification of the action generated by the third Action Description NN 303 may be provided to the control module 101, e.g., in the form of text describing the action occurring in the window. Alternatively, the Classification data may be provided to a text to speech synthesis module 304 to produce speech data that can be combined with other audio occurring during the window, or shortly thereafter”); and
wherein generating the target video according to the video segments and the corresponding commentary speeches comprises:
inserting the commentary speeches into target positions of the original video according to the second playback time periods to generate the target video (see Tian, paragraph [0130]: “combining the target video set and the target image set to obtain the candidate material resource set corresponding to the commentary”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 7, the combination teachings of Tian and Krishnamurthy as discussed above also disclose the method of claim 1, wherein segmenting the original video based on the video content in the original video to obtain at least two video segments comprises:
obtaining a target video frame and a corresponding video subtitle in the original video (see Krishnamurthy, paragraph [0039]: “an audio tagging module 109 that combines the text information, e.g., as subtitles or captions with video frame information so that the text information appears on corresponding video images presented by the video output device”);
determining at least two video events according to the target video frame and the corresponding video subtitle (see Tian, paragraph [0271]: “determine a commentary for the target news cluster”); and
segmenting the original video based on the video events to obtain video segments corresponding to the video events (see Tian, paragraph [0271]: “determine, based on the target video set and a target image set, a candidate material resource set corresponding to the commentary, where the target image set is composed of respective images included in the target news cluster; and for each paragraph in the commentary”).
The motivation for combining the references has been discussed in claim 1 above.
Claim 9 is rejected for the same reasons as discussed in claim 1 above. In addition, the combination teachings of Tian and Krishnamurthy as discussed above also disclose an electronic device, comprising:
a processor (see Tian, paragraph [0031]: “one or more processors”), and
a memory communicatively connected to the processor (see Tian, paragraph [0031]: “a computer readable medium, storing a computer program thereon, wherein the computer program, when executed by one or more processors”);
wherein the memory stores computer-executable instructions (see Tian, paragraph [0272]: “memory, storing one or more programs; and one or more processors operatively connected to the interface and the memory”);
the processor executes the computer-executable instructions stored in the memory to implement acts (see Tian, paragraph [0269]: “each of the blocks in the flow charts or block diagrams may represent a module, a program segment, or a code portion, said module, program segment, or code portion comprising one or more executable instructions for implementing specified logic functions”).
Claim 12 is rejected for the same reasons as discussed in claim 2 above.
Claim 13 is rejected for the same reasons as discussed in claim 3 above.
Claim 14 is rejected for the same reasons as discussed in claim 4 above.
Claim 15 is rejected for the same reasons as discussed in claim 5 above.
Claim 16 is rejected for the same reasons as discussed in claim 6 above.
Claim 17 is rejected for the same reasons as discussed in claim 7 above.
Claim 18 is rejected for the same reasons as discussed in claim 1 above. In addition, the combination teachings of Tian and Krishnamurthy as discussed above also disclose a non-transitory computer-readable storage medium storing computer-executable instructions, the computer-executable instructions (see Tian, paragraph [0031]: “a computer readable medium, storing a computer program thereon, wherein the computer program, when executed by one or more processors”).
Claim 19 is rejected for the same reasons as discussed in claim 2 above.
Claim 20 is rejected for the same reasons as discussed in claim 3 above.
Claim 21 is rejected for the same reasons as discussed in claim 4 above.
Claim 22 is rejected for the same reasons as discussed in claim 5 above.
Claim 23 is rejected for the same reasons as discussed in claim 6 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIENRU YANG whose telephone number is (571)272-4212. The examiner can normally be reached Monday-Friday 10AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI TRAN can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NIENRU YANG
Examiner
Art Unit 2484
/NIENRU YANG/Examiner, Art Unit 2484
/THAI Q TRAN/Supervisory Patent Examiner, Art Unit 2484
Prosecution Insights