DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Remarks
This is a reply to the application filed on 05/24/2025, in which, claims 1-15 remain pending in the present application with claims 1 and 11 being independent claims.
When making claim amendments, the applicant is encouraged to consider the references in their entireties, including those portions that have not been cited by the examiner and their equivalents as they may most broadly and appropriately apply to any particular anticipated claim amendments.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on May 24, 2025 and September 23, 2025 are in compliance with the provisions of 37 CFR 1.97 and are being considered by the Examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5 and 9-13 are rejected under 35 U.S.C. 103 as being unpatentable over Syed et al. (US 20170228600 A1, hereinafter referred to as “Syed”) in view of Lee et al. (US 20200110941 A1, hereinafter referred to as “Lee”).
Regarding claim 1, Syed discloses an image editing assistance method (see Syed, paragraph [0023]: “provide content-specific video analysis within video indexing and editing tools that facilitate video editing, creation and/or sharing”), the method comprising:
preprocessing a broadcast video of an event (see Syed, paragraph [0040]: “A video may represent media streamed from prerecorded files or may be distributed as part of a live broadcast feed”) for which a game time period for each round is specified (see Syed, paragraph [0032]: “The term “clip” generally refers to a continuous portion or segment of a video having a start time and an end time. A clip may have one or more annotations associated therewith. In some embodiments, users may modify the start and/or end time of the clip. In one embodiment, clips may be shared with other users and a clip can be part of a highlight video”);
analyzing the plurality of video clips using an event detection model (see Syed, paragraph [0054]: “watchability scoring and highlight generation module 218 analyzes changes in the game score over time and the nearness of a player to completing one or more game objectives to assign a watchability score to the video as a whole and/or to individual portions thereof. A higher watchability score may represent a higher likelihood that viewers will find the video or portions thereof interesting. Watchability scoring and highlight generation module 218 may also generate highlight videos based on watchability scores”) to generate editing guide information indicating at least one valid section within the game progress section, the valid section corresponding to at least one of a plurality of event types (see Syed, paragraph [0045]: “The extraction and use of this data is thought to result in, among other benefits, one or more of the following: … (iii) automatic differentiation between ‘interesting’ games from ‘boring’ ones through generation of watchability scores … (v) assistance to users in connection with editing videos by automatically removing or identifying dead content, labeling game sessions and highlighting game achievements; and (vi) the ability to automatically generate personalized video game highlights and walkthroughs”).
Regarding claim 1, Syed discloses all the claimed limitations with the exception of to identify a game progress section from which a game non-progress sections has been removed from the broadcast video; and extracting a plurality of video clips from the game progress section.
Lee from the same or similar fields of endeavor discloses to identify a game progress section from which a game non-progress sections has been removed from the broadcast video (see Lee, paragraph [0091]: “the apparatus for highlight extraction may create a highlight video by removing the frames corresponding to the noise from the frames from the starting frame of the target pitch count—1 to the last frame of the target pitch count”); and
extracting a plurality of video clips from the game progress section (see Lee, paragraph [0095]: “the apparatus for highlight extraction may tag the video with game information related to the video. In this case, the apparatus for highlight extraction may tag the video with game information identified using a scoreboard”).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Lee with the teachings as in Syed. The motivation for doing so would ensure the system to have the ability to use the method and apparatus for extracting highlight of sporting event disclosed in Lee to tag a video with game information identified using a scoreboard and to create a highlight video by removing the frames corresponding to the noise from the frames thus identifying a game progress section from which a game non-progress sections has been removed from the broadcast video and extracting a plurality of video clips from the game progress section in order to extract and tag video segment from a broadcast video of a sports game so that it will help user to identify the section needed to generate highlight video.
Regarding claim 2, the combination teachings of Syed and Lee as discussed above also disclose the method of claim 1, further comprising:
acquiring the game progress section (see Lee, FIG.2 and paragraph [0050]: “The apparatus 100 for highlight extraction may tag the video with game information related to the video. Tagging the video with the game information will be described in detail with reference to FIG. 2”);
sampling at least one reference frame from the broadcast video (see Lee, paragraph [0051]: “The apparatus 100 for highlight extraction may extract at least one piece of log information that corresponds to a keyword and determine at least one frame that corresponds to the log information extracted from the tagged video. In this case, the apparatus 100 for highlight extraction may create a highlight video by combining at least one frame”);
generating reference time information indicating an estimate value of at least one of a start time and an end time of at least one round in the broadcast video based on the reference frame (see Lee, paragraph [0047]: “identify log information that sequentially records events occurring in the sporting event. For example, the log information may be information, such as text broadcasting, which sequentially records events occurring in a baseball game and may be information recorded according to an order in which the events occur” and paragraph [0087]: “when log information is searched based on keywords “Sung-Bum Na's 30th home-run,” the log information may include a target pitch count and a pitcher corresponding to Sung-Bum Na's 30th home-run. The apparatus for highlight extraction may extract frames from the starting frame of the target pitch count—1 to the last frame of the target pitch count and create a highlight video corresponding to Sung-Bum Na's 30th home-run by combining the extracted frames”); and
removing the game non-progress section from the broadcast video based on the reference time information (see Lee, paragraph [0065]: “when the scoreboard is not displayed in the current frame, the apparatus for highlight extraction cannot extract information from the scoreboard and accordingly may tag the current frame with information indicating that there is no game information”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 3, the combination teachings of Syed and Lee as discussed above also disclose the method of claim 2, wherein the generating of the reference time information comprises:
extracting a broadcast scoreboard from the reference frame (see Lee, paragraph [0058]: “in the case of a frame displaying a scoreboard, the apparatus for highlight extraction may tag game information (e.g., inning, score, ball count, runner status, pitch count, and the like) identified through the scoreboard to the corresponding frame”);
determining, from the broadcast scoreboard, an elapsed time period from the start time of at least one round (see Lee, paragraphs [0085]-[0086]: “the apparatus for highlight extraction may determine whether the frame corresponds to the log information by matching with the searched log information using game information identified on the basis of a scoreboard ... the apparatus for highlight extraction may determine at least one frame for creating a highlight video on the basis of the target pitch count corresponding to the searched log information. At least one frame for creating the highlight video may include any frames from the starting frame of the target pitch count—1 to the last frame of the target pitch count”); and
estimating, based on the elapsed time period, at least one of a start time and an end time of at least one round (see Syed, paragraph [0032]: “The term “clip” generally refers to a continuous portion or segment of a video having a start time and an end time. A clip may have one or more annotations associated therewith”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 4, the combination teachings of Syed and Lee as discussed above also disclose the method of claim 1, wherein the event detection model is trained by a learning data set including a plurality of highlight videos extracted from a plurality of different videos of the same event and labeled with any one of the plurality of event types (see Lee, paragraph [0076]: “An apparatus for highlight extraction may recognize a type of an output image of a video including a sporting event by applying deep learning with ResNet v2 architecture as an example of deep learning. Specifically, the apparatus for highlight extraction may classify types of output images of a video according to specific criteria and recognize which criterion a displayed image corresponds to. In this case, the specific criteria may differ from one sporting event to another”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 5, the combination teachings of Syed and Lee as discussed above also disclose the method of claim 1, wherein between two adjacent video clips of the plurality of video clips, an end time of a preceding video clip is subsequent to a start time of a following video clip (see Syed, paragraph [0085]: “the generated game progression information may be used to generate a Table of Contents (ToC) 810, which indicates where the actual game session starts in the video, when levels changed or major battles happened in the video game and when the game ended. The viewer can directly click on a TOC entry, e.g., “Game Start,” and jump to the corresponding location in the video”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 9, the combination teachings of Syed and Lee as discussed above also disclose the method of claim 1, further comprising:
outputting an image editing interface presented with the editing guide information (see Syed, paragraph [0054]: “interacts with the video editing user interface module 260, allowing the user of the system to view and evaluate the game information extracted by system server 210 and provide input in terms of selection of clips”), wherein the image editing interface comprises an indicator indicating the location or range of the valid section in the broadcast video (see Syed, paragraph [0062]: “Once the game being shown in the video is recognized, at block 420, the temporal extent of the video game depicted in the video is determined by detecting the temporal boundaries of the game in the video. In one embodiment, the temporal boundaries may be computed by training in-game and out of game visual classifiers on the video features”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 10, the combination teachings of Syed and Lee as discussed above also disclose the method of claim 1, further comprising:
processing, in response to receiving an automatic editing request specified with a desired time period from a user, the at least one valid section to generate a recommended highlight video having the same time length as the desired time period (see Syed, paragraph [0074]: “input provided by the user (using Module 260, for example) may be used to partially or fully guide the clip selection for generating the video summary or highlights. As discussed above, the user input may relate to video locations (in terms of time) to be included in the highlights, or conditions relating to game information (e.g., scores, achievement, game level, etc.), which if met should be included in the highlights”).
The motivation for combining the references has been discussed in claim 1 above.
Claim 11 is rejected for the same reasons as discussed in claim 1 above. In addition, the combination teachings of Syed and Lee as discussed above also disclose an image editing assistance apparatus, the apparatus comprising:
a memory that stores a computer program in which instructions for executing an image editing assistance method are recorded (see Syed, paragraph [0026]: “the present invention may be provided as a computer program product, which may include a machine-readable medium having stored thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process”); and
a processor operably coupled to the memory (see Syed, paragraph [0095]: “Mass storage device 925 includes a computer readable and writeable nonvolatile, or non-transitory, data storage medium in which instructions are stored that define a program or other object that is executed by processor 905”),
wherein when the computer program is executed by the processor (see Syed, paragraph [0095]: “Mass storage device 925 also may include information that is recorded, on or in, the medium, and that is processed by processor 905 during execution of the program”).
Claim 12 is rejected for the same reasons as discussed in claim 2 above.
Claim 13 is rejected for the same reasons as discussed in claim 3 above.
Claims 6-8 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Syed and Lee as applied to claim 1, and further in view of Zeng et al. (US 20210224550 A1, hereinafter referred to as “Zeng”).
Regarding claim 6, the combination teachings of Syed and Lee as discussed above also disclose the method of claim 1, wherein the generating of the editing guide information comprises:
converting the plurality of video clips into a plurality of feature vectors corresponding thereto on a one-to-one basis, using a first deep learning model of the event detection model (see Syed, paragraph [0062]: “the temporal boundaries may be computed by training in-game and out of game visual classifiers on the video features. Those skilled in the art will appreciate a variety of artificial intelligence or machine learning methods may be used for this purpose, including, but not limited to neural networks, deep networks, random forest classifiers and the like”); and
following operations using a second deep learning model of the event detection model (see Lee, paragraph [0075]: “FIG. 8 is a diagram for describing scene recognition based on deep learning according to one embodiment”):
mapping each of the plurality of feature vectors to any one of a plurality of clusters, each cluster at least partially representing at least one of the plurality of event types (see Lee, FIG. 8 and paragraph [0077]: “highlight extraction may recognize a scene of a frame of interest on the basis of the most matching criterion among the specific criteria by applying deep learning with ResNet v2 architecture to the frame of interest. In this case, the apparatus for highlight extraction may use a recognizer score for each frame and recognize the corresponding frame on the basis of the highest score”).
The motivation for combining Syed and Lee has been discussed in claim 1 above.
Regarding claim 6, the combination teachings of Syed and Lee as discussed above disclose all the claimed limitations with the exceptions of grouping the plurality of feature vectors in chronological order to generate a plurality of vector groups; and identifying the valid section within the game progress section from a correspondence relationship between the plurality of vector groups and at least one of the plurality of event types.
Zeng from the same or similar fields of endeavor discloses grouping the plurality of feature vectors in chronological order to generate a plurality of vector groups (see Zeng, paragraph [0236]: “For the gradual change points combined into the group, the first gradual change point and the last gradual change point in the chronological order in the group are set to cps and cpc. If the number of the gradual change points in the group is at least 2, the gradually changed shot boundary formed by the group gradual change points is Bk=(cps, cpc−1, 2). If there is only one gradual change point in the group, the gradually changed shot boundary would not be formed”); and
identifying the valid section within the game progress section from a correspondence relationship between the plurality of vector groups and at least one of the plurality of event types (see Zeng, paragraph [0236]: “the deep learning based classification model outputs the embedded feature vector of the image, and then, the semantic similarity between the images is calculated based on the embedded feature vector. Through this method, the influence of a non-scene change (e.g., an illumination change, or an intensive movement of a person or an object in a scene) on the semantic similarity between the images is eliminated”).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Zeng with the teachings as in Syed and Lee. The motivation for doing so would ensure the system to have the ability to use the method and apparatus for segmenting video disclosed in Zeng to group gradual change points in the chronological order to detect scene boundary; to output embedded feature vector of the image by using deep learning based classification model and to eliminate no scene changes thus grouping the plurality of feature vectors in chronological order to generate a plurality of vector groups and identifying the valid section in order to treat two similar adjacent video sections as a single section so that it will help user to identify the valid section during editing.
Regarding claim 7, the combination teachings of Syed, Lee, and Zeng as discussed above also disclose the method of claim 6, further comprising:
receiving setting information on at least one of a plurality of filtering items used to extract a highlight video from the broadcast video (see Lee, paragraph [0077]: “The apparatus for highlight extraction may recognize a scene of a frame of interest on the basis of the most matching criterion among the specific criteria by applying deep learning with ResNet v2 architecture to the frame of interest. In this case, the apparatus for highlight extraction may use a recognizer score for each frame and recognize the corresponding frame on the basis of the highest score”),
wherein the second deep learning model is operated according to the setting information (see Lee, paragraph [0076]: “An apparatus for highlight extraction may recognize a type of an output image of a video including a sporting event by applying deep learning with ResNet v2 architecture as an example of deep learning. Specifically, the apparatus for highlight extraction may classify types of output images of a video according to specific criteria and recognize which criterion a displayed image corresponds to”).
The motivation for combining the references has been discussed in claims 1 and 6 above.
Regarding claim 8, the combination teachings of Syed, Lee, and Zeng as discussed above also disclose the method of claim 7, wherein the plurality of filtering items comprises an event type (see Lee, paragraph [0076]: “the specific criteria may differ from one sporting event to another”), an event similarity (see Lee, paragraph [0077]: “recognize a scene of a frame of interest on the basis of the most matching criterion among the specific criteria”), and an event importance (see Lee, paragraph [0077]: “the apparatus for highlight extraction may use a recognizer score for each frame and recognize the corresponding frame on the basis of the highest score”).
The motivation for combining the references has been discussed in claims 1 and 6 above.
Claim 14 is rejected for the same reasons as discussed in claim 6 above.
Claim 15 is rejected for the same reasons as discussed in claim 7 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIENRU YANG whose telephone number is (571)272-4212. The examiner can normally be reached Monday-Friday 10AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI TRAN can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NIENRU YANG
Examiner
Art Unit 2484
/NIENRU YANG/Examiner, Art Unit 2484
/THAI Q TRAN/Supervisory Patent Examiner, Art Unit 2484
Prosecution Insights